# From Photo Streams to Evolving Situations

## TL;DR

This paper presents a semi-supervised, graph-based deep learning approach for recognizing evolving situations from photo streams, effectively handling unknown situations and noisy data.

## Contribution

It introduces a novel framework combining visual concepts, space-time info, and noise-robust norms for situation recognition from diverse photo streams.

## Key findings

- Effective recognition of evolving situations demonstrated on large-scale Flickr dataset.
- Enhanced accuracy through noise-robust norms that mitigate outlier effects.
- Capability to identify both predefined and new, unknown situations.

## Abstract

Photos are becoming spontaneous, objective, and universal sources of information. This paper develops evolving situation recognition using photo streams coming from disparate sources combined with the advances of deep learning. Using visual concepts in photos together with space and time information, we formulate the situation detection into a semi-supervised learning framework and propose new graph-based models to solve the problem. To extend the method for unknown situations, we introduce a soft label method which enables the traditional semi-supervised learning framework to accurately predict predefined labels as well as effectively form new clusters. To overcome the noisy data which degrades graph quality, leading to poor recognition results, we take advantage of two kinds of noise-robust norms which can eliminate the adverse effects of outliers in visual concepts and improve the accuracy of situation recognition. Finally, we demonstrate the idea and the effectiveness of the proposed model on Yahoo Flickr Creative Commons 100 Million.