# Evaluation of a machine learning system for genomic antimicrobial susceptibility determination on a clinically representative test set

## TL;DR

A new machine learning system accurately predicts antibiotic resistance from bacterial genomes, outperforming traditional methods.

## Contribution

The paper introduces Keynome gAST, a machine learning system for genomic AST with high accuracy across diverse species-drug combinations.

## Key findings

- Keynome gAST achieved 96.9% categorical agreement with phenotypic AST results across 97 species-drug combinations.
- Keynome gAST outperformed ResFinder with 96.0% binary accuracy versus 83.5%.
- ResFinder's limitations were due to low sensitivity and specificity in identifying resistance markers.

## Abstract

Next-generation sequencing is anticipated to transform infectious disease treatment by enabling faster diagnosis, yet a clinical-grade method for genomic AST determination is still lacking. We assessed the accuracy of Keynome gAST, a k-mer based machine learning system for genomic AST, on 956 clinical bacterial isolates. We compared 7,801 predictions of susceptible (S), intermediate (I), and resistant (R) across 97 species-drug combinations against phenotypic AST results. Across the full data set, the Keynome gAST Qualified panel achieved a categorical agreement of 96.9% with very major and major error rates of 1.4% and 0.8%, respectively. At the level of individual species-drug combinations with sufficient data to assess, performance was comparable to the aggregate, with median categorical agreement of 97.4%. We also compared the performance of Keynome gAST to ResFinder, a simple resistance marker approach, in distinguishing susceptible vs non-susceptible samples. On the full data set, Keynome gAST performance was significantly better than that of ResFinder (binary accuracy of 96.0% vs 83.5%). When analyzed at the species-drug level, ResFinder’s poor performance was seen to be driven by either low sensitivity due to phenotypically resistant samples lacking resistance markers (18.0% of such samples) or low specificity due to the presence of resistance markers in phenotypically susceptible samples (15.1% of such samples). In total, these results demonstrate the dual advantages of machine learning and whole-genome profiling that Keynome gAST has over simple resistance marker approaches, enabling high accuracy genomic AST across a range of clinically relevant species-drug combinations.

Microbial genome sequencing presents an exciting opportunity for rapid diagnosis of infectious diseases, but interpreting the resulting data for clinical use remains a challenge. We report a new machine learning method that predicts a bacterial strain's antibiotic resistance profile based solely on its genomic sequence. This method could lead to new, faster diagnostic tools that quickly identify the most effective antibiotic therapy.