MOSS Transcribe Diarize Technical Report

TL;DR

This paper introduces MOSS Transcribe Diarize, an end-to-end multimodal large language model that accurately transcribes speech with speaker attribution and timestamps, overcoming previous limitations in context and memory.

Contribution

It presents a novel end-to-end model with a large context window trained on extensive data, achieving superior performance in speaker-attributed, time-stamped transcription tasks.

Findings

Outperforms state-of-the-art commercial systems on multiple benchmarks.

Handles inputs up to 90 minutes with a 128k context window.

Robust generalization across diverse datasets.

Abstract

Speaker-Attributed, Time-Stamped Transcription (SATS) aims to transcribe what is said and to precisely determine the timing of each speaker, which is particularly valuable for meeting transcription. Existing SATS systems rarely adopt an end-to-end formulation and are further constrained by limited context windows, weak long-range speaker memory, and the inability to output timestamps. To address these limitations, we present MOSS Transcribe Diarize, a unified multimodal large language model that jointly performs Speaker-Attributed, Time-Stamped Transcription in an end-to-end paradigm. Trained on extensive real wild data and equipped with a 128k context window for up to 90-minute inputs, MOSS Transcribe Diarize scales well and generalizes robustly. Across comprehensive evaluations, it outperforms state-of-the-art commercial systems on multiple public and in-house benchmarks.