AutoNeural: Co-Designing Vision-Language Models for NPU Inference

TL;DR

AutoNeural introduces an NPU-optimized vision-language model architecture that significantly improves inference efficiency and robustness for edge AI applications by co-designing hardware-friendly components and reducing latency.

Contribution

The paper presents a novel NPU-native VLM architecture with a MobileNetV5-style backbone and SSM-based language model, optimized for integer-only inference and real-time edge deployment.

Findings

Reduces vision encoder quantization error by up to 7x

Achieves 14x lower end-to-end latency

Enables real-time automotive cockpit AI on Qualcomm SoC

Abstract

While Neural Processing Units (NPUs) offer high theoretical efficiency for edge AI, state-of-the-art Vision--Language Models (VLMs) tailored for GPUs often falter on these substrates. We attribute this hardware-model mismatch to two primary factors: the quantization brittleness of Vision Transformers (ViTs) and the I/O-bound nature of autoregressive attention mechanisms, which fail to utilize the high arithmetic throughput of NPUs. To bridge this gap, we propose AutoNeural, an NPU-native VLM architecture co-designed for integer-only inference. We replace the standard ViT encoder with a MobileNetV5-style backbone utilizing depthwise separable convolutions, which ensures bounded activation distributions for stable INT4/8/16 quantization. Complementing this, our language backbone integrates State-Space Model (SSM) principles with Transformer layers, employing efficient gated convolutions to achieve linear-time complexity. This hybrid design eliminates the heavy memory I/O overhead of Key-Value caching during generation. Our approach delivers substantial efficiency gains, reducing quantization error of vision encoder by up to 7x and end-to-end latency by 14x compared to conventional baselines. The AutoNeural also delivers 3x decoding speed and 4x longer context window than the baseline. We validate these improvements via a real-world automotive case study on the Qualcomm SA8295P SoC, demonstrating real-time performance for cockpit applications. Our results highlight that rethinking model topology specifically for NPU constraints is a prerequisite for robust multi-modal edge intelligence.