Discrete Markov Bridge
Hengli Li, Yuxuan Wang, Song-Chun Zhu, Ying Nian Wu, Zilong Zheng

TL;DR
This paper introduces Discrete Markov Bridge, a new framework for discrete representation learning that enhances expressiveness and performance through matrix and score learning, with theoretical guarantees and strong empirical results.
Contribution
The paper presents Discrete Markov Bridge, a novel approach with theoretical analysis and empirical validation, improving discrete data modeling beyond fixed transition matrix methods.
Findings
Achieves an ELBO of 1.38 on Text8, outperforming baselines.
Demonstrates competitive results on CIFAR-10.
Provides theoretical guarantees and convergence analysis.
Abstract
Discrete diffusion has recently emerged as a promising paradigm in discrete data modeling. However, existing methods typically rely on a fixed rate transition matrix during training, which not only limits the expressiveness of latent representations, a fundamental strength of variational methods, but also constrains the overall design space. To address these limitations, we propose Discrete Markov Bridge, a novel framework specifically designed for discrete representation learning. Our approach is built upon two key components: Matrix Learning and Score Learning. We conduct a rigorous theoretical analysis, establishing formal performance guarantees for Matrix Learning and proving the convergence of the overall framework. Furthermore, we analyze the space complexity of our method, addressing practical constraints identified in prior studies. Extensive empirical evaluations validate the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSmart Grid Security and Resilience
MethodsDiffusion
