KathDB: Explainable Multimodal Database Management System with Human-AI Collaboration

TL;DR

KathDB is a novel multimodal database system that integrates relational data with foundation models, enabling explainable, human-AI collaborative querying across text, images, and videos.

Contribution

It introduces KathDB, a system combining relational semantics with foundation models and human-AI interaction for explainable multimodal data management.

Findings

Supports explainable queries over multimodal data.

Enables iterative human-AI collaboration during query processing.

Bridges the gap between traditional DBMS and machine learning-based systems.

Abstract

Traditional DBMSs execute user- or application-provided SQL queries over relational data with strong semantic guarantees and advanced query optimization, but writing complex SQL is hard and focuses only on structured tables. Contemporary multimodal systems (which operate over relations but also text, images, and even videos) either expose low-level controls that force users to use (and possibly create) machine learning UDFs manually within SQL or offload execution entirely to black-box LLMs, sacrificing usability or explainability. We propose KathDB, a new system that combines relational semantics with the reasoning power of foundation models over multimodal data. Furthermore, KathDB includes human-AI interaction channels during query parsing, execution, and result explanation, such that users can iteratively obtain explainable answers across data modalities.