Neural Eulerian Scene Flow Fields
Kyle Vedder, Neehar Peri, Ishan Khatri, Siyi Li, Eric Eaton, Mehmet Kocamaz, Yue Wang, Zhiding Yu, Deva Ramanan, Joachim Pehserl
TL;DR
EulerFlow introduces a neural prior-based method to estimate continuous space-time ODEs for scene flow, achieving state-of-the-art results across diverse real-world datasets without domain-specific tuning.
Contribution
The paper presents EulerFlow, a novel neural scene flow method that models motion as a continuous ODE, enabling high-quality, self-supervised scene flow estimation across various domains.
Findings
Outperforms all prior methods on the Argoverse 2 Scene Flow Challenge.
Achieves high-quality flow on small, fast-moving objects like birds and tennis balls.
Demonstrates emergent 3D point tracking over long time horizons.
Abstract
We reframe scene flow as the task of estimating a continuous space-time ODE that describes motion for an entire observation sequence, represented with a neural prior. Our method, EulerFlow, optimizes this neural prior estimate against several multi-observation reconstruction objectives, enabling high quality scene flow estimation via pure self-supervision on real-world data. EulerFlow works out-of-the-box without tuning across multiple domains, including large-scale autonomous driving scenes and dynamic tabletop settings. Remarkably, EulerFlow produces high quality flow estimates on small, fast moving objects like birds and tennis balls, and exhibits emergent 3D point tracking behavior by solving its estimated ODE over long-time horizons. On the Argoverse 2 2024 Scene Flow Challenge, EulerFlow outperforms all prior art, surpassing the next-best unsupervised method by more than 2.5x, and…
Peer Reviews
Decision·ICLR 2025 Poster
- The paper proposes an interesting idea to represent the scene flow as a velocity field using a neural network, making it very easy to combine the temporal information (time) and the spatial information (position of points). - The authors have done extensive analysis of the proposed method, and have shown different ablation studies to validate the effectiveness of the method. - The proposed method also shows the potential to deal with small objects and emergent flows in robotics scenarios, whic
- When using the time interval between [-1, 1] for the time encoding, will the proposed method not be able to handle time step outside the range? Given that the representation is a continuous neural network, how does it extrapolate to a longer sequence with the current representation? - When comparing with a method like NSFP, I wondered if the authors could show the results of pure Euler integration of the method and highlight the benefits of wrapping a PDE with a neural network. - The authors m
- I'm not up-to-date to the latest scene flow models, but from the results in the paper it surpass the prior art by a large margin, which is very significant - Introducing the concept of modeling scene flow as a PDE is innovative and offers a new direction for research in motion estimation. - The method is rigorously developed, with comprehensive experiments and ablation studies that validate the approach. - The paper is well-written, with clear explanations and effective use of figures to illus
- As stated in the paper, the speed of the proposed method is a big concern, preventing it from deploying on real world application. - Some hyperparameters, such as the depth of the neural prior, seem to require dataset-specific tuning (e.g., increasing depth to 18 for the Argoverse 2 challenge), which may affect the method's out-of-the-box applicability. - It would be great if the author could show more failure cases to help readers better understand its limitations.
1. The proposed scene flow representation is simple and technical sound. Compared to prior work NSFP, extending it to multi-frame and learns a bi-directional consistency scene flow is a very intuitive step forward. 2. The performance of this method (both qualitative and quantitative) is impressive. The method can learn very consistent scene flow in trajectory despite not explicitly considering common issues such as occlusion artifacts. As the paper demonstrated, it can tackle well on small obje
1. The paper title and introduction is very general and does not provide a precise position of this paper's main contribution. "Scene flow a partial differential equation" has been historically formulated long time ago in many prior paper, e.g. [1] as one examplar reference, and it has been proposed as a continuous representation in one early seminal work [2]. Many related work studied this optimization problem using images input and solved it using differential optimization approaches before. I
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel Reduction and Neural Networks