Unveiling Ontological Commitment in Multi-Modal Foundation Models

TL;DR

This paper presents a method to extract and validate ontological hierarchies from multimodal foundation models, enhancing interpretability and verification of their learned concepts for qualitative reasoning.

Contribution

It introduces a novel approach to derive ontological structures from DNNs using hierarchical clustering and ontology search, bridging the gap between learned representations and qualitative reasoning models.

Findings

Meaningful class hierarchies can be extracted from foundation models.

The method enables validation of DNN representations against existing ontologies.

Initial results show promising alignment with qualitative reasoning concepts.

Abstract

Ontological commitment, i.e., used concepts, relations, and assumptions, are a corner stone of qualitative reasoning (QR) models. The state-of-the-art for processing raw inputs, though, are deep neural networks (DNNs), nowadays often based off from multimodal foundation models. These automatically learn rich representations of concepts and respective reasoning. Unfortunately, the learned qualitative knowledge is opaque, preventing easy inspection, validation, or adaptation against available QR models. So far, it is possible to associate pre-defined concepts with latent representations of DNNs, but extractable relations are mostly limited to semantic similarity. As a next step towards QR for validation and verification of DNNs: Concretely, we propose a method that extracts the learned superclass hierarchy from a multimodal DNN for a given set of leaf concepts. Under the hood we (1) obtain leaf concept embeddings using the DNN's textual input modality; (2) apply hierarchical clustering to them, using that DNNs encode semantic similarities via vector distances; and (3) label the such-obtained parent concepts using search in available ontologies from QR. An initial evaluation study shows that meaningful ontological class hierarchies can be extracted from state-of-the-art foundation models. Furthermore, we demonstrate how to validate and verify a DNN's learned representations against given ontologies. Lastly, we discuss potential future applications in the context of QR.