Mask-combine Decoding and Classification Approach for Punctuation Prediction with real-time Inference Constraints

TL;DR

This paper introduces a unified decoding framework for punctuation prediction that improves accuracy in real-time applications by combining multiple strategies and comparing tagging versus classification methods without retraining.

Contribution

The paper presents a novel multi-prediction decoding strategy for punctuation prediction, enabling post-training optimization and a first comparison of tagging and classification approaches in real-time.

Findings

Significant accuracy improvements with the new decoding strategy.

Classification approaches outperform tagging when limited right-side context is available.

Post-training optimization enhances inference without retraining.

Abstract

In this work, we unify several existing decoding strategies for punctuation prediction in one framework and introduce a novel strategy which utilises multiple predictions at each word across different windows. We show that significant improvements can be achieved by optimising these strategies after training a model, only leading to a potential increase in inference time, with no requirement for retraining. We further use our decoding strategy framework for the first comparison of tagging and classification approaches for punctuation prediction in a real-time setting. Our results show that a classification approach for punctuation prediction can be beneficial when little or no right-side context is available.