High performance artificial visual system with plasmon-enhanced 2D material neural network

TL;DR

This paper introduces a novel artificial visual system that integrates sensing, pre-processing, and recognition in a single hardware neural network using plasmonic 2D materials, significantly improving speed and energy efficiency.

Contribution

It presents a new hardware-based AVS design with integrated sensing and processing using plasmonic 2D materials, reducing latency and power consumption.

Findings

Achieved large dynamic range of 180 dB

Demonstrated high-speed processing at 500 ns

Ultralow energy consumption per spike (2.4e-17 J)

Abstract

Artificial visual systems (AVS) have gained tremendous momentum because of its huge potential in areas such as autonomous vehicles and robotics as part of artificial intelligence (AI) in recent years. However, current machine visual systems composed of complex circuits based on complementary metal oxide semiconductor (CMOS) platform usually contains photosensor array, format conversion, memory and processing module. The large amount of redundant data shuttling between each unit, resulting in large latency and high power consumption, which greatly limits the performance of the AVS. Here, we demonstrate an AVS based on a new design concept, which consists of hardware devices connected in an artificial neural network (ANN) that can simultaneously sense, pre-process and recognize optical images without latency. The Ag nanograting and the two-dimensional (2D) heterostructure integrated plasmonic phototransistor array (PPTA) constitute the hardware ANN, and its synaptic weight is determined by the adjustable regularized photoresponsivity matrix. The eye-inspired pre-processing function of the device under photoelectric synergy ensures the considerable improvement of the efficiency and accuracy of subsequent image recognition. The comprehensive performance of the proof-of-concept device demonstrates great potential for machine vision applications in terms of large dynamic range (180 dB), high speed (500 ns) and ultralow energy consumption per spike (2.4e(-17) J).