Accountable Error Characterization
Amita Misra, Zhe Liu, Jalal Mahmud

TL;DR
This paper introduces AEC, a method for understanding and characterizing errors in black-box machine learning models using human-understandable features, improving error analysis and data sampling.
Contribution
The paper presents AEC, a novel error characterization approach that identifies error sources and informs data sampling using linguistic features in black-box models.
Findings
AEC effectively categorizes errors into understandable groups.
AEC improves error sample selection over uncertainty-based methods.
Case study on sentiment analysis demonstrates practical utility.
Abstract
Customers of machine learning systems demand accountability from the companies employing these algorithms for various prediction tasks. Accountability requires understanding of system limit and condition of erroneous predictions, as customers are often interested in understanding the incorrect predictions, and model developers are absorbed in finding methods that can be used to get incremental improvements to an existing system. Therefore, we propose an accountable error characterization method, AEC, to understand when and where errors occur within the existing black-box models. AEC, as constructed with human-understandable linguistic features, allows the model developers to automatically identify the main sources of errors for a given classification system. It can also be used to sample for the set of most informative input points for a next round of training. We perform error…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
