Sketch: A Toolkit for Streamlining LLM Operations

TL;DR

Sketch is a comprehensive toolkit that simplifies the deployment and management of large language models by providing structured task schemas, user-friendly interfaces, and open-source resources for output control.

Contribution

It introduces a modular toolkit with schemas, interactive processes, datasets, and an open-source model to enhance LLM usability across diverse NLP tasks.

Findings

Facilitates structured output control for LLMs

Provides an open-source dataset and tools for dataset construction

Includes an open-source model based on LLaMA3-8B-Instruct

Abstract

Large language models (LLMs) represented by GPT family have achieved remarkable success. The characteristics of LLMs lie in their ability to accommodate a wide range of tasks through a generative approach. However, the flexibility of their output format poses challenges in controlling and harnessing the model's outputs, thereby constraining the application of LLMs in various domains. In this work, we present Sketch, an innovative toolkit designed to streamline LLM operations across diverse fields. Sketch comprises the following components: (1) a suite of task description schemas and prompt templates encompassing various NLP tasks; (2) a user-friendly, interactive process for building structured output LLM services tailored to various NLP tasks; (3) an open-source dataset for output format control, along with tools for dataset construction; and (4) an open-source model based on LLaMA3-8B-Instruct that adeptly comprehends and adheres to output formatting instructions. We anticipate this initiative to bring considerable convenience to LLM users, achieving the goal of ''plug-and-play'' for various applications. The components of Sketch will be progressively open-sourced at https://github.com/cofe-ai/Sketch.