# Point-SPV: end-to-end enhancement of object recognition in simulated prosthetic vision using synthetic viewing points

## TL;DR

This paper introduces Point-SPV, a deep learning model that improves object recognition in simulated prosthetic vision by simulating gaze points and optimizing visual outputs for the task.

## Contribution

Point-SPV is the first end-to-end model for prosthetic vision that uses synthetic gaze points to enhance object recognition performance.

## Key findings

- Point-SPV outperformed edge detection in object recognition accuracy and speed.
- The model enabled more efficient visual exploration in simulated prosthetic vision.
- Task-oriented representation improved performance in gaze-contingent object discrimination.

## Abstract

Prosthetic vision systems aim to restore functional sight for visually impaired individuals by replicating visual perception by inducing phosphenes through electrical stimulation in the visual cortex, yet there remain challenges in visual representation strategies such as including gaze information and task-dependent optimization. In this paper, we introduce Point-SPV, an end-to-end deep learning model designed to enhance object recognition in simulated prosthetic vision. Point-SPV takes an initial step toward gaze-based optimization by simulating viewing points, representing potential gaze locations, and training the model on patches surrounding these points. Our approach prioritizes task-oriented representation, aligning visual outputs with object recognition needs. A behavioral gaze-contingent object discrimination experiment demonstrated that Point-SPV outperformed a conventional edge detection method, by facilitating observers to gain a higher recognition accuracy, faster reaction times, and a more efficient visual exploration. Our work highlights how task-specific optimization may enhance representations in prosthetic vision, offering a foundation for future exploration and application.