SRA: Semantic Relation-Aware Flowchart Question Answering
Xinyu Li, Bowei Zou, Yuchong Chen, Yifan Fan, Yu Hong

TL;DR
This paper introduces SRA, a novel approach for FlowchartQA that enhances reasoning by incorporating semantic relations between nodes using LLMs, leading to improved accuracy on the FlowVQA benchmark.
Contribution
It proposes a semantic relation-aware interlanguage for flowchart question answering, enabling deeper reasoning and better handling of complex semantic relationships.
Findings
Significant performance improvements on FlowVQA dataset
Effective integration of semantic relations enhances reasoning depth
Versatile application across different interlanguage formats
Abstract
Flowchart Question Answering (FlowchartQA) is a multi-modal task that automatically answers questions conditioned on graphic flowcharts. Current studies convert flowcharts into interlanguages (e.g., Graphviz) for Question Answering (QA), which effectively bridge modal gaps between questions and flowcharts. More importantly, they reveal the link relations between nodes in the flowchart, facilitating a shallow relation reasoning during tracing answers. However, the existing interlanguages still lose sight of intricate semantic/logic relationships such as Conditional and Causal relations. This hinders the deep reasoning for complex questions. To address the issue, we propose a novel Semantic Relation-Aware (SRA) FlowchartQA approach. It leverages Large Language Model (LLM) to detect the discourse semantic relations between nodes, by which a link-based interlanguage is upgraded to the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Multimodal Machine Learning Applications · Natural Language Processing Techniques
