Online Refinement of a Scene Recognition Model for Mobile Robots by Observing Human's Interaction with Environments

TL;DR

This paper presents a real-time method for refining scene recognition models on mobile robots by observing human interactions, improving accuracy in recognizing traversable plants without heavy computation.

Contribution

It introduces a novel online refinement framework using few-shot segmentation and robust weight imprinting based on human interaction data, avoiding extensive fine-tuning.

Findings

Outperforms ordinary weight imprinting in accuracy

Achieves results comparable to fine-tuning with less computational cost

Effective in real-world robot navigation scenarios

Abstract

This paper describes a method of online refinement of a scene recognition model for robot navigation considering traversable plants, flexible plant parts which a robot can push aside while moving. In scene recognition systems that consider traversable plants growing out to the paths, misclassification may lead the robot to getting stuck due to the traversable plants recognized as obstacles. Yet, misclassification is inevitable in any estimation methods. In this work, we propose a framework that allows for refining a semantic segmentation model on the fly during the robot's operation. We introduce a few-shot segmentation based on weight imprinting for online model refinement without fine-tuning. Training data are collected via observation of a human's interaction with the plant parts. We propose novel robust weight imprinting to mitigate the effect of noise included in the masks generated by the interaction. The proposed method was evaluated through experiments using real-world data and shown to outperform an ordinary weight imprinting and provide competitive results to fine-tuning with model distillation while requiring less computational cost.