From Leaderboard to Deployment: Code Quality Challenges in AV Perception Repositories
Mateus Karvat, Bram Adams, Sidney Givigi

TL;DR
This study systematically analyzes 178 AV perception repositories, revealing that most do not meet production standards and highlighting the need for improved code quality and safety practices for real-world deployment.
Contribution
First large-scale empirical analysis of code quality in AV perception repositories, identifying gaps between research benchmarks and production readiness.
Findings
Only 7.3% of repositories meet basic production criteria.
Security issues are concentrated in top five vulnerabilities.
CI/CD adoption correlates with better maintainability.
Abstract
Autonomous vehicle (AV) perception models are typically evaluated solely on benchmark performance metrics, with limited attention to code quality, production readiness and long-term maintainability. This creates a significant gap between research excellence and real-world deployment in safety-critical systems subject to international safety standards. To address this gap, we present the first large-scale empirical study of software quality in AV perception repositories, systematically analyzing 178 unique models from the KITTI and NuScenes 3D Object Detection leaderboards. Using static analysis tools (Pylint, Bandit, and Radon), we evaluated code errors, security vulnerabilities, maintainability, and development practices. Our findings revealed that only 7.3% of the studied repositories meet basic production-readiness criteria, defined as having zero critical errors and no high-severity…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Safety Systems Engineering in Autonomy · Ethics and Social Impacts of AI
