Learning to Solve Multiresolution Matrix Factorization by Manifold Optimization and Evolutionary Metaheuristics

TL;DR

This paper introduces a learnable multiresolution matrix factorization method optimized via manifold techniques and evolutionary algorithms, leading to superior graph wavelet bases and improved performance in graph learning tasks.

Contribution

It presents a novel learnable MMF framework combining manifold optimization and metaheuristics, outperforming existing methods in graph wavelet basis learning and graph neural network applications.

Findings

Learned wavelet bases outperform prior MMF algorithms.

Wavelet neural networks achieve competitive results in graph classification.

Proposed method effectively models complex multiscale graph structures.

Abstract

Multiresolution Matrix Factorization (MMF) is unusual amongst fast matrix factorization algorithms in that it does not make a low rank assumption. This makes MMF especially well suited to modeling certain types of graphs with complex multiscale or hierarchical strucutre. While MMF promises to yields a useful wavelet basis, finding the factorization itself is hard, and existing greedy methods tend to be brittle. In this paper, we propose a ``learnable'' version of MMF that carfully optimizes the factorization using metaheuristics, specifically evolutionary algorithms and directed evolution, along with Stiefel manifold optimization through backpropagating errors. We show that the resulting wavelet basis far outperforms prior MMF algorithms and gives comparable performance on standard learning tasks on graphs. Furthermore, we construct the wavelet neural networks (WNNs) learning graphs on the spectral domain with the wavelet basis produced by our MMF learning algorithm. Our wavelet networks are competitive against other state-of-the-art methods in molecular graphs classification and node classification on citation graphs. We release our implementation at https://github.com/HySonLab/LearnMMF