Latenrgy: Model Agnostic Latency and Energy Consumption Prediction for Binary Classifiers

TL;DR

This paper introduces a model-agnostic theoretical framework to predict latency and energy consumption in binary classifiers, addressing scalability and responsible AI challenges by integrating classifier, dataset, and guardrail factors.

Contribution

It develops the first generalized predictive equations for latency and energy in binary classifiers, unifying multiple factors into a comprehensive analytical model.

Findings

Provides foundational equations for latency and energy prediction

Enables benchmarking and optimization of inference performance

Supports balancing efficiency with ethical AI principles

Abstract

Machine learning systems increasingly drive innovation across scientific fields and industry, yet challenges in compute overhead, specifically during inference, limit their scalability and sustainability. Responsible AI guardrails, essential for ensuring fairness, transparency, and privacy, further exacerbate these computational demands. This study addresses critical gaps in the literature, chiefly the lack of generalized predictive techniques for latency and energy consumption, limited cross-comparisons of classifiers, and unquantified impacts of RAI guardrails on inference performance. Using Theory Construction Methodology, this work constructed a model-agnostic theoretical framework for predicting latency and energy consumption in binary classification models during inference. The framework synthesizes classifier characteristics, dataset properties, and RAI guardrails into a unified analytical instrument. Two predictive equations are derived that capture the interplay between these factors while offering generalizability across diverse classifiers. The proposed framework provides foundational insights for designing efficient, responsible ML systems. It enables researchers to benchmark and optimize inference performance and assists practitioners in deploying scalable solutions. Finally, this work establishes a theoretical foundation for balancing computational efficiency with ethical AI principles, paving the way for future empirical validation and broader applications.