What You Prompt is What You Get: Increasing Transparency of Prompting Using Prompt Cards
Amandine M. Caut, Beimnet Zenebe, Amy Rouillard, David J. T. Sumpter

TL;DR
This paper introduces prompt cards as a structured documentation tool to enhance transparency, reproducibility, and evaluation of prompt engineering practices in large language models.
Contribution
It proposes the prompt card approach inspired by model cards, demonstrating its use in a specific task to improve prompt methodology and transparency.
Findings
Prompt cards improve reproducibility of prompt engineering.
They enhance transparency and systematic documentation.
Prompt cards serve as an effective alternative to benchmarking.
Abstract
The rapid advancement and impressive capabilities of large language models (LLMs) have given rise to the field of prompt engineering, the practice of crafting inputs to guide LLMs toward high-quality, task-relevant outputs. A critical challenge facing the field is the lack of standardised prompt documentation and evaluation practices. Prompts can be long, complex and difficult to evaluate on subjective tasks. To address this challenge, we propose the use of prompt cards, structured summaries of prompt engineering practices inspired by the concept of model cards. Through prompt cards, the specific goals, considerations and steps taken during prompt engineering can be systematically documented and assessed. We present the prompt card approach and illustrate it on a specific task called wordalisation, in which structured numerical data is transformed into text. We argue that a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDesign Education and Practice · Model-Driven Software Engineering Techniques · Machine Learning in Materials Science
