# Nail Polish Try-On: Realtime Semantic Segmentation of Small Objects for Native and Browser Smartphone AR Applications

## TL;DR

This paper introduces a real-time semantic segmentation system optimized for small objects like fingernails, enabling efficient nail polish AR try-on in native and browser smartphone applications with high accuracy and speed.

## Contribution

It presents a flexible neural network design for small object segmentation that balances performance and runtime, along with a postprocessing algorithm for nail polish AR applications.

## Key findings

- Achieves 94.5 mIoU at 29.8ms runtime on iPad Pro
- Enables client-side real-time nail polish AR in native and web apps
- Provides a novel postprocessing method for nail polish rendering

## Abstract

We provide a system for semantic segmentation of small objects that enables nail polish try-on AR applications to run client-side in realtime in native and web mobile applications. By adjusting input resolution and neural network depth, our model design enables a smooth trade-off of performance and runtime, with the highest performance setting achieving~\num{94.5} mIoU at 29.8ms runtime in native applications on an iPad Pro. We also provide a postprocessing and rendering algorithm for nail polish try-on, which integrates with our semantic segmentation and fingernail base-tip direction predictions.