Sharkzor: Interactive Deep Learning for Image Triage, Sort and Summary

TL;DR

Sharkzor is an interactive web application that combines user input and deep learning to efficiently triage, organize, and summarize large collections of images, enhancing data management tasks.

Contribution

The paper introduces Sharkzor, a novel system integrating user interactions with deep learning for improved image sorting and summarization.

Findings

Effective user interface supports triage, organization, and automation tasks.

Deep learning automates image grouping based on user interactions.

System accelerates understanding of large image datasets.

Abstract

Sharkzor is a web application for machine-learning assisted image sort and summary. Deep learning algorithms are leveraged to infer, augment, and automate the user's mental model. Initially, images uploaded by the user are spread out on a canvas. The user then interacts with the images to impute their mental model into the application's algorithmic underpinnings. Methods of interaction within Sharkzor's user interface and user experience support three primary user tasks; triage, organize and automate. The user triages the large pile of overlapping images by moving images of interest into proximity. The user then organizes said images into meaningful groups. After interacting with the images and groups, deep learning helps to automate the user's interactions. The loop of interaction, automation, and response by the user allows the system to quickly make sense of large amounts of data.