Prompto: An open source library for asynchronous querying of LLM endpoints
Ryan Sze-Yin Chan, Federico Nanni, Angus R. Williams, Edwin Brown,, Liam Burke-Moore, Ed Chapman, Kate Onslow, Tvesha Sippy, Jonathan Bright,, Evelina Gabasova

TL;DR
Prompto is an open source Python library that enables asynchronous, efficient querying of multiple LLM endpoints simultaneously, simplifying research, comparison, and experimentation with large language models.
Contribution
It introduces a novel library that streamlines multi-model LLM interaction, improving efficiency and reproducibility in research workflows.
Findings
Enables concurrent querying of multiple LLMs
Maximizes efficiency by respecting individual rate limits
Facilitates faster experimentation and data collection
Abstract
Recent surge in Large Language Model (LLM) availability has opened exciting avenues for research. However, efficiently interacting with these models presents a significant hurdle since LLMs often reside on proprietary or self-hosted API endpoints, each requiring custom code for interaction. Conducting comparative studies between different models can therefore be time-consuming and necessitate significant engineering effort, hindering research efficiency and reproducibility. To address these challenges, we present prompto, an open source Python library which facilitates asynchronous querying of LLM endpoints enabling researchers to interact with multiple LLMs concurrently, while maximising efficiency and utilising individual rate limits. Our library empowers researchers and developers to interact with LLMs more effectively and allowing faster experimentation, data generation and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPower Systems and Technologies · Natural Language Processing Techniques
MethodsLib
