Lucy: edgerunning agentic web search on mobile with machine generated task vectors

TL;DR

Lucy introduces a novel approach where small language models dynamically construct and refine task vectors during reasoning, enabling efficient web search and achieving competitive accuracy with larger models.

Contribution

The paper presents a new paradigm of dynamic task vector construction in small language models, trained via RLVR, to enhance reasoning and web search capabilities.

Findings

Lucy achieves 78.3% accuracy on SimpleQA.

Small models can rival larger models with structured reasoning.

Dynamic reasoning improves performance in knowledge-intensive tasks.

Abstract

Small language models (SLMs) are inherently limited in knowledge-intensive tasks due to their constrained capacity. While test-time computation offers a path to enhanced performance, most approaches treat reasoning as a fixed or heuristic process. In this work, we propose a new paradigm: viewing the model's internal reasoning, delimited by <think> and </think> tags, as a dynamic task vector machine. Rather than treating the content inside these tags as a mere trace of thought, we interpret the generation process itself as a mechanism through which the model \textbf{constructs and refines its own task vectors} on the fly. We developed a method to optimize this dynamic task vector machine through RLVR and successfully trained an agentic web-search model. We present Lucy, a 1.7B-parameter SLM that leverages this dynamic reasoning mechanism with MCP integration to achieve 78.3% accuracy on the SimpleQA benchmark, performing on par with much larger models such as DeepSeek-V3. This demonstrates that small models can rival large ones when equipped with structured, self-constructed task reasoning.