DeepWriter: A Fact-Grounded Multimodal Writing Assistant Based On Offline Knowledge Base

TL;DR

DeepWriter is a multimodal writing assistant that uses an offline knowledge base, task decomposition, and reflection to generate factually accurate, professional documents in specialized domains, overcoming limitations of existing retrieval methods.

Contribution

It introduces a novel pipeline combining hierarchical knowledge, multimodal retrieval, and reflection for factually grounded long-form writing in specialized fields.

Findings

Outperforms baselines in financial report generation

Produces more accurate and coherent documents

Effectively integrates textual and visual information

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in various applications. However, their use as writing assistants in specialized domains like finance, medicine, and law is often hampered by a lack of deep domain-specific knowledge and a tendency to hallucinate. Existing solutions, such as Retrieval-Augmented Generation (RAG), can suffer from inconsistency across multiple retrieval steps, while online search-based methods often degrade quality due to unreliable web content. To address these challenges, we introduce DeepWriter, a customizable, multimodal, long-form writing assistant that operates on a curated, offline knowledge base. DeepWriter leverages a novel pipeline that involves task decomposition, outline generation, multimodal retrieval, and section-by-section composition with reflection. By deeply mining information from a structured corpus and incorporating both textual and visual elements, DeepWriter generates coherent, factually grounded, and professional-grade documents. We also propose a hierarchical knowledge representation to enhance retrieval efficiency and accuracy. Our experiments on financial report generation demonstrate that DeepWriter produces high-quality, verifiable articles that surpasses existing baselines in factual accuracy and generated content quality.