NatLan: Native Language Prompting Facilitates Knowledge Elicitation Through Language Trigger Provision and Domain Trigger Retention

TL;DR

NatLan introduces a novel multilingual prompting approach that enhances knowledge retention and accuracy in non-dominant languages by leveraging a collaborative strategy and role-enhanced translation, outperforming existing methods.

Contribution

The paper proposes NatLan, a new multilingual prompting method that improves domain trigger retention and accuracy in non-dominant languages using multi-model collaboration and role-enhanced translation.

Findings

Achieves up to 31.28% accuracy improvement across five language QA benchmarks.

Maintains up to 87% domain trigger retention compared to state-of-the-art methods.

Demonstrates effectiveness of multi-MLLM collaboration in multilingual knowledge elicitation.

Abstract

Multilingual large language models (MLLMs) do not perform as well when answering questions in non-dominant languages as they do in their dominant languages. Although existing translate-then-answer methods alleviate this issue, the mechanisms behind their effectiveness remain unclear. In this study, we analogize the dominant language of MLLMs to the native language of humans and use two human cognitive features: the Language Trigger (LT) and the Domain Trigger (DT), to interpret the mechanisms behind translate-then-answer methods. This reveals that while sufficient LTs are provided by these methods, there remains a deficiency in DT retention. To mitigate this issue, we propose Native Language Prompting (NatLan), employing a Multi-MLLM collaboration strategy and introducing an additional role-enhanced domain-specific MLLM with stronger multilingual understanding capabilities as the translator. Across five language QA benchmarks, NatLan achieves up to a 31.28% improvement in accuracy and, compared to existing state-of-the-art methods, provides comparable or greater retention of DTs in up to 87% of cases. Our code is available at https://github.com/AnonyNLP/NatLan.