Semi-automatic Simultaneous Interpreting Quality Evaluation

TL;DR

This paper proposes a semi-automatic, meaning-based evaluation method for simultaneous interpreting quality, using frame semantics and semantic matching to correlate with human judgment.

Contribution

It introduces a novel semantic-scoring measurement based on frame and Frame Element matching for automatic interpreting quality assessment.

Findings

Semantic-scoring metrics significantly correlate with human judgment.

Frame and FE matching effectively evaluate interpreting quality.

Proposed method offers an objective supplement to human evaluation.

Abstract

Increasing interpreting needs a more objective and automatic measurement. We hold a basic idea that 'translating means translating meaning' in that we can assessment interpretation quality by comparing the meaning of the interpreting output with the source input. That is, a translation unit of a 'chunk' named Frame which comes from frame semantics and its components named Frame Elements (FEs) which comes from Frame Net are proposed to explore their matching rate between target and source texts. A case study in this paper verifies the usability of semi-automatic graded semantic-scoring measurement for human simultaneous interpreting and shows how to use frame and FE matches to score. Experiments results show that the semantic-scoring metrics have a significantly correlation coefficient with human judgment.