# Molecular-level protein semantic learning via structure-aware coarse-grained language modeling

## TL;DR

This paper introduces a new protein language model that uses structural patterns to better understand protein function, especially for large proteins.

## Contribution

A novel structure-aware coarse-grained protein language that improves molecular-level semantic learning.

## Key findings

- The proposed method achieves stable performance across multiple downstream tasks, especially for long proteins.
- The coarse-grained language preserves critical structural and functional semantics.
- It outperforms existing fine-grained and classical language modeling approaches in several tasks.

## Abstract

Protein language models (PLMs) have emerged as pivotal tools for protein representation, enabling significant advances in structure-function prediction and computational biology. However, current PLMs predominantly rely on fine-grained amino acid sequences as input, treating individual residues as tokens. While this approach facilitates semantic learning at the residue level, it struggles to capture molecular-level semantics, particularly for large proteins, where sequence truncation and inefficient local pattern extraction hinder holistic understanding. The spatial structure of a protein determines its function. Despite the critical role of protein function analysis, coarse-grained protein language frameworks that bridge sequence and structural semantics remain underdeveloped.

To fill this gap, we introduce a novel structure-aware coarse-grained protein language that discretizes proteins into local structural patterns derived from their secondary structures. By constructing a vocabulary of these patterns as “words,” we represent proteins as compact, structure-aware “sentences” significantly shorter than raw amino acid sequences. We benchmark the proposed coarse-grained language against three state-of-the-art fine-grained protein languages and a classical language modeling method in natural language processing, using two architectures: a lightweight Doc2Vec model and a Transformer-based BERT model, and evaluating performance across diverse downstream tasks, including function prediction, enzyme classification, and interaction identification. The proposed method achieves stable performance across three tasks, especially for long proteins. These results demonstrate that the proposed coarse-grained protein language preserves critical structural and functional semantics and improves molecular-level analysis, offering a promising direction for decoding higher-order biological insights.

The data and source code of the proposed method are available at GitHub (https://github.com/bug-0x3f/coarse-grained-protein-language) and Zenodo (DOI: 10.5281/zenodo.17674298).