Domain-Aware Tensor Network Structure Search
Giorgos Iacovides, Wuyang Zhou, Chao Li, Qibin Zhao, Danilo Mandic
TL;DR
This paper introduces tnLLM, a novel framework that leverages large language models and domain knowledge to efficiently identify optimal tensor network structures, reducing computational costs and enhancing interpretability.
Contribution
The proposed tnLLM framework uniquely incorporates domain information and LLM reasoning to improve tensor network structure search efficiency and transparency.
Findings
Achieves comparable results with fewer function evaluations.
Accelerates convergence of existing methods using domain-informed initializations.
Provides domain-aware explanations for tensor network structures.
Abstract
Tensor networks (TNs) provide efficient representations of high-dimensional data, yet identification of the optimal TN structures, the so called tensor network structure search (TN-SS) problem, remains a challenge. Current state-of-the-art (SOTA) algorithms solve TN-SS as a purely numerical optimization problem and require extensive function evaluations, which is prohibitive for real-world applications. In addition, existing methods ignore the valuable domain information inherent in real-world tensor data and lack transparency in their identified TN structures. To this end, we propose a novel TN-SS framework, termed the tnLLM, which incorporates domain information about the data and harnesses the reasoning capabilities of large language models (LLMs) to directly predict suitable TN structures. The proposed framework involves a domain-aware prompting pipeline which instructs the LLM to…
Peer Reviews
Decision·Submitted to ICLR 2026
- **S1**. The idea of leveraging LLMs to incorporate domain information into a fundamentally combinatorial search problem (TN-SS) is compelling, which brings a fresh angle to tensor network design. - **S2**. The experimental results demonstrate substantial reductions in the number of required evaluations (e.g., up to ~78× fewer than a baseline method) while maintaining comparable performance on the target objective. - **S3**. The ability to provide human-interpretable explanations for the result
- **W1**. The core technical contribution mainly relies on prompt engineering with LLMs, which limits the methodological novelty and may not be considered a strong theoretical contribution. - **W2**. The framework lacks theoretical guarantees regarding when the LLM will propose reliable structures and avoid hallucinations, making its behavior difficult to predict. - **W3**. The experimental evaluation is restricted to relatively small-scale and limited domains (images, videos, and time-series),
* By injecting tensor mode interdependence (e.g., shared physical indices in quantum many-body systems) into prompts, the LLM implicitly learns to suppress topologically invalid contractions, reducing the rate of malformed outputs. * The experimental results on three types of tensor data across different domains demonstrate the effectiveness of the proposed method.
* While a scalarized objective (Eq.(1) in the paper) is used, the paper omits raw compression ratio and relative reconstruction error across the Pareto frontier — the de facto standard in TN compression. Reporting only $\mathcal{L}$ obscures trade-offs and hinders comparison with heuristics. * As shown in Table 2, performance degrade with weak LLMs and may suffer from hallucination issue of LLMs. No robustness analysis is provided.
- The idea to let an llm guide the discrete optimization is interesting. While this is not the first work in this general direction, the proposed method for TN-SS is novel. While some may consider the mindest "throwing an llm at the problem" to be uninspiring, I believe it is important to thoroughly test und understand the limits of llms nonetheless. Replacing complex heuristics with concise prompts and llm reasoning has potential - The experiments show a quick convergence to good solutions with
In its current form the paper has several weaknesses with regard to the related work and the experiments, which overall do not meet the standards for a publication at a top conference like ICLR. However, I believe most of them can be addressed and I will raise my score if this is done properly. I will go over them according to my priorities. Major Weaknesses: 1. The state of the art seems less clear than claimed in the paper. In particular, SVDinsTN claims to be much faster than TNLS and TnALE,
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTensor decomposition and applications · Advanced Graph Neural Networks · Topic Modeling