Automatic construction of Chinese herbal prescription from tongue image via CNNs and auxiliary latent therapy topics

TL;DR

This paper presents a neural network framework that automatically constructs Chinese herbal prescriptions from tongue images, incorporating auxiliary therapy topics to improve diversity and accuracy, with promising results on real-world data.

Contribution

It introduces a novel CNN-based model with an auxiliary therapy topic loss mechanism for automatic herbal prescription generation from tongue images.

Findings

Generated prescriptions closely match real samples.

The method is effective across various photographic environments.

It demonstrates feasibility for clinical and mobile healthcare applications.

Abstract

The tongue image provides important physical information of humans. It is of great importance for diagnoses and treatments in clinical medicine. Herbal prescriptions are simple, noninvasive and have low side effects. Thus, they are widely applied in China. Studies on the automatic construction technology of herbal prescriptions based on tongue images have great significance for deep learning to explore the relevance of tongue images for herbal prescriptions, it can be applied to healthcare services in mobile medical systems. In order to adapt to the tongue image in a variety of photographic environments and construct herbal prescriptions, a neural network framework for prescription construction is designed. It includes single/double convolution channels and fully connected layers. Furthermore, it proposes the auxiliary therapy topic loss mechanism to model the therapy of Chinese doctors and alleviate the interference of sparse output labels on the diversity of results. The experiment use the real world tongue images and the corresponding prescriptions and the results can generate prescriptions that are close to the real samples, which verifies the feasibility of the proposed method for the automatic construction of herbal prescriptions from tongue images. Also, it provides a reference for automatic herbal prescription construction from more physical information.