Fish Audio S2 Technical Report
Shijia Liao, Yuxuan Wang, Songting Liu, Yifan Cheng, Ruoyi Zhang, Tianyu Li, Shidong Li, Yisheng Zheng, Xingwei Liu, Qingzheng Wang, Zhizhuo Zhou, Jiahua Liu, Xin Chen, Dawei Han

TL;DR
Fish Audio S2 is an open-source text-to-speech system that supports multi-speaker, multi-turn, and instruction-following generation, with a scalable training pipeline and a production-ready inference engine.
Contribution
The paper introduces Fish Audio S2, a novel open-source TTS system with instruction-following capabilities and a comprehensive training pipeline, including multi-stage data processing.
Findings
Achieves an RTF of 0.195 for streaming inference
Supports multi-speaker and multi-turn speech generation
Provides open-source model weights and inference tools
Abstract
We introduce Fish Audio S2, an open-sourced text-to-speech system featuring multi-speaker, multi-turn generation, and, most importantly, instruction-following control via natural-language descriptions. To scale training, we develop a multi-stage training recipe together with a staged data pipeline covering video captioning and speech captioning, voice-quality assessment, and reward modeling. To push the frontier of open-source TTS, we release our model weights, fine-tuning code, and an SGLang-based inference engine. The inference engine is production-ready for streaming, achieving an RTF of 0.195 and a time-to-first-audio below 100 ms.Our code and weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/s2-pro). We highly encourage readers to visit https://fish.audio to try custom voices.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
- 🤗fishaudio/s2-promodel· 22k dl· ♡ 79422k dl♡ 794
- 🤗drbaph/s2-pro-fp8model· 1.6k dl· ♡ 161.6k dl♡ 16
- 🤗mlx-community/fish-audio-s2-pro-8bitmodel· 437 dl· ♡ 5437 dl♡ 5
- 🤗ieduardj/s2-promodel· 22 dl22 dl
- 🤗mlx-community/fish-audio-s2-pro-bf16model· 1.6k dl· ♡ 131.6k dl♡ 13
- 🤗Salama1429/s2-promodel· 23 dl23 dl
- 🤗sunnyyy/s2-promodel· 20 dl20 dl
- 🤗np-deploys/fishaudio-s2-promodel· 15 dl15 dl
- 🤗nexyllofficial/s2-pro-backupmodel· 7 dl7 dl
- 🤗on1onmangoes/s2-promodel· 10 dl10 dl
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpeech and Audio Processing · Speech and dialogue systems · Speech Recognition and Synthesis
