PARAGEN : A Parallel Generation Toolkit
Jiangtao Feng, Yi Zhou, Jun Zhang, Xian Qian, Liwei Wu, Zhexi Zhang,, Yanming Liu, Mingxuan Wang, Lei Li, Hao Zhou
TL;DR
PARAGEN is a versatile PyTorch toolkit designed for parallel generation in NLP, offering customizable plugins and features to facilitate rapid experimentation and industrial deployment.
Contribution
It introduces a flexible, plugin-based framework for parallel NLP generation, enabling quick experimentation with different models and strategies.
Findings
Supports various research and industry applications
Provides extensive customization options
Enhances industrial usability with features like automatic model selection
Abstract
PARAGEN is a PyTorch-based NLP toolkit for further development on parallel generation. PARAGEN provides thirteen types of customizable plugins, helping users to experiment quickly with novel ideas across model architectures, optimization, and learning strategies. We implement various features, such as unlimited data loading and automatic model selection, to enhance its industrial usage. ParaGen is now deployed to support various research and industry applications at ByteDance. PARAGEN is available at https://github.com/bytedance/ParaGen.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsWeb Data Mining and Analysis · Machine Learning and Data Classification · Natural Language Processing Techniques