Type Anywhere You Want: An Introduction to Invisible Mobile Keyboard

TL;DR

This paper introduces an Invisible Mobile Keyboard (IMK) that allows users to type freely on any area, overcoming soft keyboard limitations, supported by extensive data collection, a high-accuracy decoder, and positive user feedback.

Contribution

The paper presents the first large-scale dataset and a semantic typo correction decoder for IMK, enabling unconstrained typing with high accuracy and user satisfaction.

Findings

Achieved 96.0% decoding accuracy.

Users typed faster and reported increased satisfaction.

Publicly released dataset and source code for research use.

Abstract

Contemporary soft keyboards possess limitations: the lack of physical feedback results in an increase of typos, and the interface of soft keyboards degrades the utility of the screen. To overcome these limitations, we propose an Invisible Mobile Keyboard (IMK), which lets users freely type on the desired area without any constraints. To facilitate a data-driven IMK decoding task, we have collected the most extensive text-entry dataset (approximately 2M pairs of typing positions and the corresponding characters). Additionally, we propose our baseline decoder along with a semantic typo correction mechanism based on self-attention, which decodes such unconstrained inputs with high accuracy (96.0%). Moreover, the user study reveals that the users could type faster and feel convenience and satisfaction to IMK with our decoder. Lastly, we make the source code and the dataset public to contribute to the research community.