Should I take a walk? Estimating Energy Expenditure from Video Data
Kunyu Peng, Alina Roitberg, Kailun Yang, Jiaming Zhang, Rainer, Stiefelhagen

TL;DR
This paper introduces Vid2Burn, a benchmark dataset for estimating human caloric expenditure from video data, highlighting the challenges of generalizing across different activity types and proposing a new research direction.
Contribution
The paper presents Vid2Burn, a comprehensive benchmark dataset for energy expenditure estimation from videos, and evaluates state-of-the-art methods, emphasizing the need for models that generalize beyond activity categories.
Findings
State-of-the-art models struggle with unseen activity types
Energy expenditure estimation from video is a challenging task
Cross-category generalization remains a key challenge
Abstract
We explore the problem of automatically inferring the amount of kilocalories used by human during physical activity from his/her video observation. To study this underresearched task, we introduce Vid2Burn -- an omni-source benchmark for estimating caloric expenditure from video data featuring both, high- and low-intensity activities for which we derive energy expenditure annotations based on models established in medical literature. In practice, a training set would only cover a certain amount of activity types, and it is important to validate, if the model indeed captures the essence of energy expenditure, (e.g., how many and which muscles are involved and how intense they work) instead of memorizing fixed values of specific activity categories seen during training. Ideally, the models should look beyond such category-specific biases and regress the caloric cost in videos depicting…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsContext-Aware Activity Recognition Systems · Physical Activity and Health · Nutritional Studies and Diet
