NeurIPS 2020 NLC2CMD Competition: Translating Natural Language to Bash Commands
Mayank Agarwal, Tathagata Chakraborti, Quchen Fu, David Gros, Xi, Victoria Lin, Jaron Maene, Kartik Talamadupula, Zhongwei Teng, Jules White
TL;DR
The paper reports on the NeurIPS 2020 NLC2CMD competition where models were developed to translate natural language descriptions into Bash commands, highlighting the challenge and progress in NLP for command line automation.
Contribution
It introduces a benchmark task for translating natural language to Bash commands and provides insights from various participant solutions and lessons learned.
Findings
Models achieved varying accuracy levels in translating language to Bash.
The competition highlighted the challenges of understanding and generating precise command syntax.
Lessons learned suggest directions for improving NLP models for command line tasks.
Abstract
The NLC2CMD Competition hosted at NeurIPS 2020 aimed to bring the power of natural language processing to the command line. Participants were tasked with building models that can transform descriptions of command line tasks in English to their Bash syntax. This is a report on the competition with details of the task, metrics, data, attempted solutions, and lessons learned.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Text Readability and Simplification