# Programmable Spectrometry -- Per-pixel Classification of Materials using Learned Spectral Filters

## TL;DR

This paper introduces a programmable spectral camera that efficiently captures material-specific spectral information for per-pixel classification, improving speed and signal quality over traditional hyperspectral imaging methods.

## Contribution

The paper presents a novel programmable camera capable of optical spectral filtering tailored for material classification, combining hardware innovation with machine learning techniques.

## Key findings

- Enhanced acquisition speed by capturing only relevant spectral measurements.
- Improved signal-to-noise ratio by avoiding narrowband filters.
- Successful validation on simulated and real datasets.

## Abstract

Many materials have distinct spectral profiles. This facilitates estimation of the material composition of a scene at each pixel by first acquiring its hyperspectral image, and subsequently filtering it using a bank of spectral profiles. This process is inherently wasteful since only a set of linear projections of the acquired measurements contribute to the classification task. We propose a novel programmable camera that is capable of producing images of a scene with an arbitrary spectral filter. We use this camera to optically implement the spectral filtering of the scene's hyperspectral image with the bank of spectral profiles needed to perform per-pixel material classification. This provides gains both in terms of acquisition speed --- since only the relevant measurements are acquired --- and in signal-to-noise ratio --- since we invariably avoid narrowband filters that are light inefficient. Given training data, we use a range of classical and modern techniques including SVMs and neural networks to identify the bank of spectral profiles that facilitate material classification. We verify the method in simulations on standard datasets as well as real data using a lab prototype of the camera.