Menta: A Small Language Model for On-Device Mental Health Prediction

TL;DR

Menta is a lightweight, fine-tuned small language model optimized for on-device mental health prediction from social media data, achieving high accuracy while being computationally efficient and privacy-preserving.

Contribution

This paper introduces Menta, the first optimized small language model specifically designed for multi-task mental health prediction from social media, with a novel training framework and on-device deployment.

Findings

Menta outperforms nine SLM baselines with 15.2% average improvement.

Menta surpasses 13B-parameter LLMs in depression and stress classification.

Menta runs on an iPhone 15 Pro Max using only 3GB RAM.

Abstract

Mental health conditions affect hundreds of millions globally, yet early detection remains limited. While large language models (LLMs) have shown promise in mental health applications, their size and computational demands hinder practical deployment. Small language models (SLMs) offer a lightweight alternative, but their use for social media--based mental health prediction remains largely underexplored. In this study, we introduce Menta, the first optimized SLM fine-tuned specifically for multi-task mental health prediction from social media data. Menta is jointly trained across six classification tasks using a LoRA-based framework, a cross-dataset strategy, and a balanced accuracy--oriented loss. Evaluated against nine state-of-the-art SLM baselines, Menta achieves an average improvement of 15.2\% across tasks covering depression, stress, and suicidality compared with the best-performing non--fine-tuned SLMs. It also achieves higher accuracy on depression and stress classification tasks compared to 13B-parameter LLMs, while being approximately 3.25x smaller. Moreover, we demonstrate real-time, on-device deployment of Menta on an iPhone 15 Pro Max, requiring only approximately 3GB RAM. Supported by a comprehensive benchmark against existing SLMs and LLMs, Menta highlights the potential for scalable, privacy-preserving mental health monitoring. Code is available at: https://hong-labs.github.io/menta-project/