Joint Beam Search Integrating CTC, Attention, and Transducer Decoders
Yui Sudo, Muhammad Shakeel, Yosuke Fukumoto, Brian Yan, Jiatong Shi,, Yifan Peng, Shinji Watanabe

TL;DR
This paper introduces a joint 4D modeling approach for end-to-end speech recognition that combines four decoders sharing the same encoder, along with novel joint beam search algorithms, resulting in improved accuracy and robustness.
Contribution
It proposes a unified 4D modeling framework with a two-stage training strategy and three new joint beam search algorithms for enhanced speech recognition performance.
Findings
The 4D model outperforms single-decoder models in accuracy.
Joint beam search algorithms improve decoding performance.
The approach enhances model robustness and efficiency.
Abstract
End-to-end automatic speech recognition (E2E-ASR) can be classified by its decoder architectures, such as connectionist temporal classification (CTC), recurrent neural network transducer (RNN-T), attention-based encoder-decoder, and Mask-CTC models. Each decoder architecture has advantages and disadvantages, leading practitioners to switch between these different models depending on application requirements. Instead of building separate models, we propose a joint modeling scheme where four decoders (CTC, RNN-T, attention, and Mask-CTC) share the same encoder -- we refer to this as 4D modeling. The 4D model is trained jointly, which will bring model regularization and maximize the model robustness thanks to their complementary properties. To efficiently train the 4D model, we introduce a two-stage training strategy that stabilizes the joint training. In addition, we propose three novel…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvancements in Photolithography Techniques · Advanced Radiotherapy Techniques
