NLP Service APIs and Models for Efficient Registration of New Clients

TL;DR

This paper proposes a lightweight, unsupervised method for clients to adapt centralized NLP services using corpus sketches, enabling immediate improved accuracy without traditional fine-tuning.

Contribution

It introduces a novel architecture where clients register via corpus sketches and servers use auxiliary networks to enhance NLP task performance.

Findings

Effective sentiment, NER, and language modeling results

Immediate accuracy improvements for new clients

Lightweight adaptation without labeled data

Abstract

State-of-the-art NLP inference uses enormous neural architectures and models trained for GPU-months, well beyond the reach of most consumers of NLP. This has led to one-size-fits-all public API-based NLP service models by major AI companies, serving large numbers of clients. Neither (hardware deficient) clients nor (heavily subscribed) servers can afford traditional fine tuning. Many clients own little or no labeled data. We initiate a study of adaptation of centralized NLP services to clients, and present one practical and lightweight approach. Each client uses an unsupervised, corpus-based sketch to register to the service. The server uses an auxiliary network to map the sketch to an abstract vector representation, which then informs the main labeling network. When a new client registers with its sketch, it gets immediate accuracy benefits. We demonstrate the success of the proposed architecture using sentiment labeling, NER, and predictive language modeling