Dr. Tongue: Sign-Oriented Multi-label Detection for Remote Tongue Diagnosis

TL;DR

This paper introduces Dr. Tongue, a multi-label detection framework for remote tongue diagnosis that combines adaptive feature extraction and a sign-oriented network, validated on a new dataset for telemedicine.

Contribution

The paper presents a novel sign-oriented multi-label detection framework and a dedicated tongue image dataset tailored for remote medical diagnosis.

Findings

Improved accuracy in tongue attribute detection

Effective standardization of tongue images for telehealth

Openly available dataset for research use

Abstract

Tongue diagnosis is a vital tool in Western and Traditional Chinese Medicine, providing key insights into a patient's health by analyzing tongue attributes. The COVID-19 pandemic has heightened the need for accurate remote medical assessments, emphasizing the importance of precise tongue attribute recognition via telehealth. To address this, we propose a Sign-Oriented multi-label Attributes Detection framework. Our approach begins with an adaptive tongue feature extraction module that standardizes tongue images and mitigates environmental factors. This is followed by a Sign-oriented Network (SignNet) that identifies specific tongue attributes, emulating the diagnostic process of experienced practitioners and enabling comprehensive health evaluations. To validate our methodology, we developed an extensive tongue image dataset specifically designed for telemedicine. Unlike existing datasets, ours is tailored for remote diagnosis, with a comprehensive set of attribute labels. This dataset will be openly available, providing a valuable resource for research. Initial tests have shown improved accuracy in detecting various tongue attributes, highlighting our framework's potential as an essential tool for remote medical assessments.