Measuring the State of the Art of Automated Pathway Curation Using Graph Algorithms - A Case Study of the mTOR Pathway

TL;DR

This study assesses the gap between human-curated pathway maps and NLP-based automated systems using graph analysis on the mTOR pathway, highlighting areas for future improvement.

Contribution

It introduces graph analysis methods to quantify differences between human curation and NLP systems in pathway mapping, providing insights for advancing automation.

Findings

Current NLP systems perform well in certain pathway regions

Significant gaps remain in complex pathway areas

Identifies specific challenges for future NLP improvements

Abstract

This paper evaluates the difference between human pathway curation and current NLP systems. We propose graph analysis methods for quantifying the gap between human curated pathway maps and the output of state-of-the-art automatic NLP systems. Evaluation is performed on the popular mTOR pathway. Based on analyzing where current systems perform well and where they fail, we identify possible avenues for progress.