Abnormal activity capture from passenger flow of elevator based on unsupervised learning and fine-grained multi-label recognition

TL;DR

This paper introduces a system that uses sensors and computer vision to monitor elevator passenger flow, applying unsupervised learning and multi-label recognition to detect abnormal or illegal activities in residential buildings.

Contribution

It proposes GraftNet for fine-grained multi-label recognition of human attributes and a hierarchical unsupervised anomaly detection method for identifying suspicious activities.

Findings

Effective recognition of human attributes like gender, age, appearance, and occupation.

Successful detection of abnormal activities such as drug dealing and overcrowding.

Direct reporting system for property managers to verify incidents.

Abstract

We present a work-flow which aims at capturing residents' abnormal activities through the passenger flow of elevator in multi-storey residence buildings. Camera and sensors (hall sensor, photoelectric sensor, gyro, accelerometer, barometer, and thermometer) with internet connection are mounted in elevator to collect image and data. Computer vision algorithms such as instance segmentation, multi-label recognition, embedding and clustering are applied to generalize passenger flow of elevator, i.e. how many people and what kinds of people get in and out of the elevator on each floor. More specifically in our implementation we propose GraftNet, a solution for fine-grained multi-label recognition task, to recognize human attributes, e.g. gender, age, appearance, and occupation. Then anomaly detection of unsupervised learning is hierarchically applied on the passenger flow data to capture abnormal or even illegal activities of the residents which probably bring safety hazard, e.g. drug dealing, pyramid sale gathering, prostitution, and over crowded residence. Experiment shows effects are there, and the captured records will be directly reported to our customer(property managers) for further confirmation.