# Metamorphic Testing for Quality Assurance of Protein Function Prediction Tools

## TL;DR

This paper applies metamorphic testing to evaluate the quality of protein function prediction tools, revealing that many such tools fail critical tests due to the oracle problem, thus raising concerns about their reliability.

## Contribution

It introduces a novel application of metamorphic testing to AFP tools, addressing the oracle problem and providing a systematic evaluation method.

## Key findings

- Several AFP tools fail all test cases
- Metamorphic relations effectively identify prediction failures
- Highlights need for improved AFP tool validation

## Abstract

Proteins are the workhorses of life and gaining insight on their functions is of paramount importance for applications such as drug design. However, the experimental validation of functions of proteins is highly-resource consuming. Therefore, recently, automated protein function prediction (AFP) using machine learning has gained significant interest. Many of these AFP tools are based on supervised learning models trained using existing gold-standard functional annotations, which are known to be incomplete. The main challenge associated with conducting systematic testing on AFP software is the lack of a test oracle, which determines passing or failing of a test case; unfortunately, due to the incompleteness of gold-standard data, the exact expected outcomes are not well defined for the AFP task. Thus, AFP tools face the \emph{oracle problem}. In this work, we use metamorphic testing (MT) to test nine state-of-the-art AFP tools by defining a set of metamorphic relations (MRs) that apply input transformations to protein sequences. According to our results, we observe that several AFP tools fail all the test cases causing concerns over the quality of their predictions.