Graph-based Heuristic Search for Module Selection Procedure in Neural Module Network
Yuxuan Wu, Hideki Nakayama

TL;DR
This paper introduces a graph-based heuristic search method for training Neural Module Networks in visual question answering, eliminating the need for ground-truth programs and improving training efficiency.
Contribution
It proposes a novel heuristic search algorithm on Program Graphs to optimize program discovery in NMN training, surpassing reinforcement learning approaches.
Findings
Effective training without ground-truth programs
Superior efficiency over reinforcement learning methods
Achieved high accuracy on FigureQA and CLEVR datasets
Abstract
Neural Module Network (NMN) is a machine learning model for solving the visual question answering tasks. NMN uses programs to encode modules' structures, and its modularized architecture enables it to solve logical problems more reasonably. However, because of the non-differentiable procedure of module selection, NMN is hard to be trained end-to-end. To overcome this problem, existing work either included ground-truth program into training data or applied reinforcement learning to explore the program. However, both of these methods still have weaknesses. In consideration of this, we proposed a new learning framework for NMN. Graph-based Heuristic Search is the algorithm we proposed to discover the optimal program through a heuristic search on the data structure named Program Graph. Our experiments on FigureQA and CLEVR dataset show that our methods can realize the training of NMN…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Domain Adaptation and Few-Shot Learning · Reinforcement Learning in Robotics
