Subspace Kernel Learning on Tensor Sequences
Lei Wang, Xi Ding, Yongsheng Gao, Piotr Koniusz
TL;DR
This paper introduces UKTL, a scalable, uncertainty-aware kernel framework for tensor sequence data that improves robustness, interpretability, and achieves state-of-the-art results in action recognition benchmarks.
Contribution
It proposes a novel uncertainty-driven kernel tensor learning method with scalable Nyström linearization and adaptive mode weighting for structured tensor sequence analysis.
Findings
Achieves state-of-the-art performance on NTU-60, NTU-120, Kinetics-Skeleton datasets.
Demonstrates improved robustness and interpretability in tensor sequence comparisons.
Provides a fully end-to-end trainable kernel learning framework for multi-way tensor data.
Abstract
Learning from structured multi-way data, represented as higher-order tensors, requires capturing complex interactions across tensor modes while remaining computationally efficient. We introduce Uncertainty-driven Kernel Tensor Learning (UKTL), a novel kernel framework for -mode tensors that compares mode-wise subspaces derived from tensor unfoldings, enabling expressive and robust similarity measure. To handle large-scale tensor data, we propose a scalable Nystr\"{o}m kernel linearization with dynamically learned pivot tensors obtained via soft -means clustering. A key innovation of UKTL is its uncertainty-aware subspace weighting, which adaptively down-weights unreliable mode components based on estimated confidence, improving robustness and interpretability in comparisons between input and pivot tensors. Our framework is fully end-to-end trainable and naturally incorporates both…
Peer Reviews
Decision·ICLR 2026 Poster
- The principled formulation bridging tensor subspaces and kernels is well-formulated. The idea of defining kernels over mode-wise Grassmann subspaces is conceptually elegant and mathematically well-grounded, combining multilinear structure preservation with nonlinear kernel flexibility. - The introduction of the Multi-mode SigmaNet to learn mode-wise uncertainty is a thoughtful innovation that improves robustness and interpretability, addressing a long-standing issue in tensor learning where a
- The paper combines multiple ideas—Grassmann kernels, Nyström approximation, HoT encoders, and uncertainty modeling—making it dense and difficult to parse. The derivations are mathematically sound but the overall design is somehow overcomplex. What's more, as each component (tensor subspace kernel, Nyström approximation, uncertainty weighting) has precedent, the paper’s contribution lies in their combination and engineering coherence rather than a fundamental breakthrough. - The related wor
- The paper presents a novel approach for learning from structured, high-order data with original compnents such as uncertainty-driven subspace weighting and sum–product Grassmann kernels. - The theoretical formulation is sound and is a valuable contribution bridging kernel theory,and uncertainty modeling. - The paper is clearly written and well-organized
1. Evaluation is limited to skeleton data. This is a key limitation which hinders the generality of the method. Although these datasets are standard and challenging, they represent a very narrow class of structured motion data. Demonstrating the approach on a different tensor domains (uch as video features, fMRI signals, or audio visual data) would better support claims of generality and broad applicability. Even a small-scale study in a non-skeletal domain would reinforce the method’s versatili
The problem of tensor learning is very interesting.
The boilerplate goes until page 4 using space that seems to be missing for details later. I find the description of the uncertainty vectors and the other material in Sections 3 and 4 not easy to follow. Many details are missing and the description is too brief to be able to follow the discussion.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTensor decomposition and applications · Domain Adaptation and Few-Shot Learning · Generative Adversarial Networks and Image Synthesis