Adaptable Embeddings Network (AEN)
Stan Loosmore, Alexander Titus

TL;DR
The paper introduces Adaptable Embeddings Networks (AEN), a lightweight dual-encoder model using Kernel Density Estimation that enables runtime adaptation for text classification, suitable for edge devices and real-time applications.
Contribution
AEN is a novel dual-encoder architecture that allows runtime adaptation without retraining, using KDE for efficient and flexible text classification in low-resource environments.
Findings
AEN achieves comparable or superior results to larger autoregressive models.
AEN is highly suitable for edge computing and real-time monitoring.
The model's architecture enables effective preprocessing and caching of embeddings.
Abstract
Modern day Language Models see extensive use in text classification, yet this comes at significant computational cost. Compute-effective classification models are needed for low-resource environments, most notably on edge devices. We introduce Adaptable Embeddings Networks (AEN), a novel dual-encoder architecture using Kernel Density Estimation (KDE). This architecture allows for runtime adaptation of classification criteria without retraining and is non-autoregressive. Through thorough synthetic data experimentation, we demonstrate our model outputs comparable and in certain cases superior results to that of autoregressive models an order of magnitude larger than AEN's size. The architecture's ability to preprocess and cache condition embeddings makes it ideal for edge computing applications and real-time monitoring systems.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Data Classification · Advanced Neural Network Applications · Data Stream Mining Techniques
