OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization

TL;DR

OmniFysics introduces a unified omni-modal network that combines signal processing and physical knowledge to enhance environmental perception and physical understanding in AI systems.

Contribution

The paper presents OmniFysics, a novel compact network integrating multi-modal signals with explicit physical knowledge and an adaptive physical data engine for improved physical reasoning.

Findings

Achieves competitive performance on standard multimodal benchmarks.

Significantly improves physics-oriented evaluation metrics.

Demonstrates effective autonomous optimization through staged alignment and instruction tuning.

Abstract

The autonomous evolution of networked AI systems relies heavily on robust environmental perception. However, physical understanding remains brittle in current models because key physical signals are visually ambiguous and sparsely represented in web-scale data. To bridge the gap between data-centric learning and knowledge-based physical rules, we present OmniFysics, a compact omni-modal network that unifies signal processing and understanding across images, audio, video, and text. To enable autonomous optimization and inject explicit physical knowledge, we construct a dynamic physical data engine. Within this engine, FysicsAny acts as an adaptive mechanism that produces physics-grounded supervision by mapping salient objects to verified physical attributes via hierarchical retrieval and physics-law-constrained signal verification. Concurrently, FysicsOmniCap distills web videos utilizing advanced audio-visual cross-modal signal processing, generating high-fidelity data pairs that emphasize dynamic physical cues. We optimize the OmniFysics network through staged multimodal alignment and evolutive instruction tuning, integrating latent-space flow matching for generation and an adaptive intent router for efficient execution. Experiments demonstrate that this evolutive optimization paradigm not only achieves competitive performance on standard multimodal benchmarks but also significantly advances physics-oriented evaluations.