CAIRNS: Balancing Readability and Scientific Accuracy in Climate Adaptation Question Answering

TL;DR

CAIRNS is a framework that improves the readability and reliability of climate adaptation question-answering by structuring evidence sources and evaluating consistency, enabling experts to obtain credible preliminary answers without fine-tuning.

Contribution

It introduces a novel framework that enhances answer readability and citation reliability in climate adaptation QA without requiring model fine-tuning or reinforcement learning.

Findings

CAIRNS outperforms baseline models on most metrics.

The framework achieves robust evaluation through a consistency-weighted hybrid evaluator.

Ablation study confirms the effectiveness of components.

Abstract

Climate adaptation strategies are proposed in response to climate change. They are practised in agriculture to sustain food production. These strategies can be found in unstructured data (for example, scientific literature from the Elsevier website) or structured (heterogeneous climate data via government APIs). We present Climate Adaptation question-answering with Improved Readability and Noted Sources (CAIRNS), a framework that enables experts -- farmer advisors -- to obtain credible preliminary answers from complex evidence sources from the web. It enhances readability and citation reliability through a structured ScholarGuide prompt and achieves robust evaluation via a consistency-weighted hybrid evaluator that leverages inter-model agreement with experts. Together, these components enable readable, verifiable, and domain-grounded question-answering without fine-tuning or reinforcement learning. Using a previously reported dataset of expert-curated question-answers, we show that CAIRNS outperforms the baselines on most of the metrics. Our thorough ablation study confirms the results on all metrics. To validate our LLM-based evaluation, we also report an analysis of correlations against human judgment.