I-Keyboard: Fully Imaginary Keyboard on Touch Devices Empowered by Deep Neural Decoder

TL;DR

This paper introduces I-Keyboard, an invisible touch keyboard using deep neural decoding that enables eyes-free, full-finger typing on mobile screens without calibration, enhancing usability and performance.

Contribution

It presents the first exploration of an eyes-free, fully imaginary keyboard with deep neural decoding, eliminating the need for calibration or predefined regions.

Findings

18.95% increase in typing speed

4.06% increase in accuracy

Achieved 45.57 WPM and 95.84% accuracy

Abstract

Text-entry aims to provide an effective and efficient pathway for humans to deliver their messages to computers. With the advent of mobile computing, the recent focus of text-entry research has moved from physical keyboards to soft keyboards. Current soft keyboards, however, increase the typo rate due to lack of tactile feedback and degrade the usability of mobile devices due to their large portion on screens. To tackle these limitations, we propose a fully imaginary keyboard (I-Keyboard) with a deep neural decoder (DND). The invisibility of I-Keyboard maximizes the usability of mobile devices and DND empowered by a deep neural architecture allows users to start typing from any position on the touch screens at any angle. To the best of our knowledge, the eyes-free ten-finger typing scenario of I-Keyboard which does not necessitate both a calibration step and a predefined region for typing is first explored in this work. For the purpose of training DND, we collected the largest user data in the process of developing I-Keyboard. We verified the performance of the proposed I-Keyboard and DND by conducting a series of comprehensive simulations and experiments under various conditions. I-Keyboard showed 18.95% and 4.06% increases in typing speed (45.57 WPM) and accuracy (95.84%), respectively over the baseline.