DQA: Diagnostic Question Answering for IT Support

TL;DR

DQA is a diagnostic question-answering framework designed for enterprise IT support that maintains diagnostic state and improves troubleshooting efficiency over standard retrieval-augmented systems.

Contribution

It introduces a novel approach combining conversational rewriting, retrieval aggregation, and state-conditioned generation for systematic troubleshooting.

Findings

DQA achieves a 78.7% success rate in enterprise IT scenarios.

DQA reduces average troubleshooting turns from 8.4 to 3.9.

DQA outperforms baseline RAG systems significantly.

Abstract

Enterprise IT support interactions are fundamentally diagnostic: effective resolution requires iterative evidence gathering from ambiguous user reports to identify an underlying root cause. While retrieval-augmented generation (RAG) provides grounding through historical cases, standard multi-turn RAG systems lack explicit diagnostic state and therefore struggle to accumulate evidence and resolve competing hypotheses across turns. We introduce DQA, a diagnostic question-answering framework that maintains persistent diagnostic state and aggregates retrieved cases at the level of root causes rather than individual documents. DQA combines conversational query rewriting, retrieval aggregation, and state-conditioned response generation to support systematic troubleshooting under enterprise latency and context constraints. We evaluate DQA on 150 anonymized enterprise IT support scenarios using a replay-based protocol. Averaged over three independent runs, DQA achieves a 78.7% success rate under a trajectory-level success criterion, compared to 41.3% for a multi-turn RAG baseline, while reducing average turns from 8.4 to 3.9.