# Hyper-MDR: An open-world multimodal reasoning framework based on dynamic hypergraph and meta-strategy optimization

## TL;DR

Hyper-MDR is a new framework for open-world object detection that improves multimodal reasoning and adapts to new categories and changing environments.

## Contribution

Hyper-MDR introduces a dynamic hypergraph and meta-strategy optimization for open-world multimodal perception.

## Key findings

- Hypergraph enhancement models high-order dependencies in vision–language–semantic triplets.
- The hierarchical hypergraph convolutional network enables knowledge propagation between known and unknown categories.
- The meta-policy gradient-based controller adapts feature fusion and attention topology dynamically.

## Abstract

Open-world object detection (OWOD) has become a crucial paradigm for advancing intelligent perception systems, as it requires not only accurate recognition of known categories but also autonomous discovery and continuous learning of emerging unknown categories in dynamic environments. However, existing methods often suffer from shallow cross-modal interaction and rigid reasoning mechanisms, making them unable to cope with the continuous emergence of new categories and the dynamic changes in modality reliability in open-world environments. First, to achieve hypergraph enhancement, image regions, text, and semantic prototypes are treated as nodes, while a gating network dynamically generates hyperedges under semantic and spatial constraints, thereby modeling the high-order dependencies of vision–language–semantic triplets and enabling deep multimodal fusion at the topological level. Second, a hierarchical hypergraph convolutional network is designed to facilitate knowledge propagation between known and unknown categories. Finally, a meta-policy gradient-based adaptive controller is proposed, which dynamically adjusts feature fusion weights, propagation depth, and attention topology based on the detection state and historical trajectories. Experimental results on the OWOD dataset show that our proposed method achieves an accuracy of 76.8%, providing a new paradigm for open-world multimodal perception that integrates semantic depth and adaptability.