Towards High-Order Mean Flow Generative Models: Feasibility, Expressivity, and Provably Efficient Criteria
Yang Cao, Yubin Chen, Zhao Song, Jiahao Zhang
TL;DR
This paper introduces Second-Order MeanFlow, extending flow matching models with acceleration fields, and provides theoretical guarantees on its feasibility, expressivity, and efficient implementation.
Contribution
The work establishes the feasibility, expressivity, and scalable implementation criteria for Second-Order MeanFlow, a novel high-order flow matching model incorporating acceleration fields.
Findings
Proves average acceleration satisfies a generalized consistency condition.
Shows Second-Order MeanFlow can be implemented by circuits in the $ ext{TC}^0$ class.
Demonstrates attention operations can be approximated efficiently within $n^{2+o(1)}$ time.
Abstract
Generative modelling has seen significant advances through simulation-free paradigms such as Flow Matching, and in particular, the MeanFlow framework, which replaces instantaneous velocity fields with average velocities to enable efficient single-step sampling. In this work, we introduce a theoretical study on Second-Order MeanFlow, a novel extension that incorporates average acceleration fields into the MeanFlow objective. We first establish the feasibility of our approach by proving that the average acceleration satisfies a generalized consistency condition analogous to first-order MeanFlow, thereby supporting stable, one-step sampling and tractable loss functions. We then characterize its expressivity via circuit complexity analysis, showing that under mild assumptions, the Second-Order MeanFlow sampling process can be implemented by uniform threshold circuits within the…
Peer Reviews
Decision·Submitted to ICLR 2026
1. Clear formalization. The manuscript is meticulous in its definitions, maintaining consistency with prior Flow-Matching literature. 2. The assumptions are explicitly stated.
1. The main technical ideas are not very novel. Most of the argument builds upon earlier work on the circuit complexity of Transformer or VAR models, such as [1] and [2], and the extension to MeanFlow seems relatively straightforward. 2. There are no empirical or simulated results to illustrate whether the proposed approximations actually improve runtime. 3. The paper relies on many assumptions which may not hold in reality or have little practical guidance. The TC^0 result relies on (i) O(1)
- The paper is original in that it bridges the fields of generative modeling and circuit complexity. While several works on transformers leverage circuit complexity techniques, these haven’t been applied in the context of generative modeling. - In particular, the finding that MeanFlows and high order MeanFlows are in TC0, which is the same class to which transformers without CoT belong, is very interesting, and may prompt researchers to think of CoT-like approaches for flows.
- The main weakness of the paper is that it does not include experiments testing the performance of the second-order MeanFlow. This may not be perceived as a weakness in a theoretical computer science venue, but ICLR is a general machine learning conference and the empirical performance of proposed algorithm must be shown. In that sense, this work is incomplete, as we do not know whether second-order MeanFlow is only a theoretical construction, or whether it has applications in practice. - Smal
1. The paper provides formal definitions, clear assumptions, and detailed proofs for all major results. Each theorem (on feasibility, expressivity, and efficiency) is logically consistent and well supported by prior work in flow matching and circuit complexity. 2. Extending MeanFlow to second-order dynamics is a meaningful and nontrivial contribution. By modeling average accelerations, SOMF potentially captures richer temporal dependencies in generative flows, offering a principled direction to
1. The paper remains entirely theoretical, and while the authors acknowledge this, the absence of even small-scale experiments or numerical simulations limits the understanding of how SOMF behaves in practice. Demonstrating a toy example or a simple empirical comparison would significantly enhance credibility. 2. The motivation for introducing acceleration terms is described formally but not intuitively. The paper could benefit from geometric or dynamical explanations illustrating how incorpora
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Low-power high-performance VLSI design · Parallel Computing and Optimization Techniques