TL;DR
RoboMD introduces a deep RL-based framework that predicts robot vulnerabilities in a semantic embedding space, enabling scalable, safe, and effective vulnerability detection and policy improvement without physical testing.
Contribution
The paper presents a novel virtual vulnerability prediction method using deep RL in a semantic embedding space, outperforming existing baselines and enhancing manipulation policies with less data.
Findings
Uncovered up to 23% more vulnerabilities than state-of-the-art methods.
Enabled safe vulnerability analysis without physical trials.
Improved manipulation performance with minimal fine-tuning data.
Abstract
Robot manipulation policies, while central to the promise of physical AI, are highly vulnerable in the presence of external variations in the real world. Diagnosing these vulnerabilities is hindered by two key challenges: (i) the relevant variations to test against are often unknown, and (ii) direct testing in the real world is costly and unsafe. We introduce a framework that tackles both issues by learning a separate deep reinforcement learning (deep RL) policy for vulnerability prediction through virtual runs on a continuous vision-language embedding trained with limited success-failure data. By treating this embedding space, which is rich in semantic and visual variations, as a potential field, the policy learns to move toward vulnerable regions while being repelled from success regions. This vulnerability prediction policy, trained on virtual rollouts, enables scalable and safe…
Peer Reviews
Decision·ICLR 2026 Poster
1. I really like the idea of reframing failure discovery as an active exploration problem rather than a passive evaluation or stress test. It’s an elegant conceptual step that connects representation learning, uncertainty estimation, and RL in a unified way. 2. Extensive experiments across simulated and physical domains, ablation on embedding design (BCE vs. BCE + contrastive), and real robot validation strengthen the empirical credibility. 3. I appreciate that the paper doesn’t just stay empiri
1. The method heavily relies on the assumption that the learned multimodal embedding forms a smooth manifold where semantic similarity aligns with failure likelihood. While visualizations suggest separability, it’s unclear whether embedding-space distances correspond to physically meaningful variations. Without empirical measures of “semantic continuity,” this assumption could fail for unseen scenarios. 2. The baselines include standard RL and VLM models, but exclude other uncertainty-based fai
The idea of creating a failure predictor is a good one, as can be used in many dynamic settings. This can identify ‘distance’ from key vulnerabilities, and this can include lighting variation and object color and size (cases that are notorious for their difficulty to handle generally). The method can be used to isolate and avoid some key failure modes. With sufficient prior examples, generalization over virtual roll outs is carried out. The theory shows how a shaped MDP will lead to efficie
The primary weakness is lack of adequate baselines. There are various approaches for control that provide safety, such as barrier methods. It is not surprising that a conventional RL would not do as well as one supplemented with an additional failure mode predictor. The semantic potential field amounts to a kind of semantic barrier. Also not clear is what happens to a vanilla RL if it is given some explicit bad examples with negative reward. The method keys on the training selection and var
### Originality - Reformulation of diagnosis as RL search in a semantically meaningful potential field is elegant and novel. - Using multimodal embeddings to structure the search space is a strong innovation. ### Quality - Broad evaluation: 4 manipulation tasks × multiple policy types × real vs. simulated settings. - Benchmarks vs. GPT-4o, Gemini, Qwen2-VL show strong generalization and robustness. ### Clarity - Excellent visuals and modular structuring. - Detailed Appendices with ablation studi
This work is of great practical significance, but I am allowed to express some concerns. ### Assumption of Embedding Quality: 1. The success of the approach hinges critically on the quality of the semantic embedding space. If poorly structured, RL search may yield meaningless trajectories. 2. Though addressed via BCE + contrastive loss, there is no quantitative measure of embedding smoothness across multiple datasets. ### Limited Real-World Testing: 1. While real-world results (UR5e) are include
Videos
Taxonomy
TopicsReinforcement Learning in Robotics · Online Learning and Analytics
