Meta-Optimization for Higher Model Generalizability in Single-Image Depth Prediction

TL;DR

This paper introduces a meta-learning approach for indoor single-image depth prediction that enhances model generalizability across unseen datasets, achieving better zero-shot cross-dataset performance without extra data.

Contribution

It proposes a novel fine-grained meta-optimization treating each RGB-D pair as a task, improving depth prediction generalizability without additional data or explicit task boundaries.

Findings

Meta-learning improves prior by up to 29.4%.

Meta-initialized models outperform baselines in cross-dataset tests.

Zero-shot protocols demonstrate higher robustness and accuracy.

Abstract

Model generalizability to unseen datasets, concerned with in-the-wild robustness, is less studied for indoor single-image depth prediction. We leverage gradient-based meta-learning for higher generalizability on zero-shot cross-dataset inference. Unlike the most-studied image classification in meta-learning, depth is pixel-level continuous range values, and mappings from each image to depth vary widely across environments. Thus no explicit task boundaries exist. We instead propose fine-grained task that treats each RGB-D pair as a task in our meta-optimization. We first show meta-learning on limited data induces much better prior (max +29.4\%). Using meta-learned weights as initialization for following supervised learning, without involving extra data or information, it consistently outperforms baselines without the method. Compared to most indoor-depth methods that only train/ test on a single dataset, we propose zero-shot cross-dataset protocols, closely evaluate robustness, and show consistently higher generalizability and accuracy by our meta-initialization. The work at the intersection of depth and meta-learning potentially drives both research streams to step closer to practical use.