# Large Language Model (LLM)-Predicted and LLM-Assisted Calculation of the Spinal Instability Neoplastic Score (SINS) Improves Clinician Accuracy and Efficiency

## TL;DR

A privacy-preserving AI model improves the speed and accuracy of calculating a score used to decide if spinal tumor patients need surgery.

## Contribution

A privacy-preserving LLM is shown to enhance clinician accuracy and efficiency in calculating the SINS score for spinal tumors.

## Key findings

- LLM-assisted clinicians achieved excellent agreement with a reference standard (ICC = 0.993).
- LLM alone calculated scores in about 5 seconds, much faster than clinicians without AI assistance.
- Clinicians using LLM assistance were notably faster than those without it.

## Abstract

Spinal tumors can cause instability in the spine, and doctors use the Spinal Instability Neoplastic Score (SINS) to decide whether surgery is needed. However, calculating this score can be time-consuming and may vary between doctors. This study evaluates whether a privacy-preserving large language model (LLM) can improve the accuracy and speed of SINS scoring. The model was tested in three ways: on its own, assisting a clinician, and compared to clinicians working without any AI support. The authors aim to show that LLMs can reduce variation, improve efficiency, and support more consistent decision-making for patients with spinal metastases, potentially leading to faster treatment and better care.

Background: The Spinal Instability Neoplastic Score (SINS) guides treatment for patients with spinal tumors, but issues arise with complexity, interobserver variability, and time demands. Large language models (LLMs) may help overcome these limitations. Objectives: This study evaluates the accuracy and efficiency of a privacy-preserving LLM (PP-LLM) for SINS calculation, with and without clinician involvement, to assess its feasibility as a clinical decision-support tool. Methods: This retrospective observational study was granted a Domain-Specific Review Board waiver owing to minimal risk. Patients from 2020 to 2022 were included. A PP-LLM was employed to maintain secure handling of patient data. A consensus SINS reference standard was established by musculoskeletal radiologists and an orthopedic surgeon. Eight orthopedic and oncology trainees were divided into two groups to calculate SINS, with and without PP-LLM assistance. LLM-predicted scores were also generated independently of any human input. Results: The main outcomes were agreement with the reference standard (measured by intraclass correlation coefficients [ICCs]) and time required for SINS calculation. The LLM-assisted method achieved excellent agreement (ICC = 0.993, 95%CI = 0.991–0.994), closely followed by the LLM-predicted approach (ICC = 0.990, 95%CI = 0.984–0.993). Clinicians working without LLM support showed a significantly lower ICC compared to both LLM methods (0.968, 95%CI = 0.960–0.975) (both p < 0.001). The LLM alone produced scores in approximately 5 s, while the median scoring time for LLM-assisted clinicians was 60.0 s (IQR = 46.0–80.0), notably shorter than the 83.0 s (IQR = 58.0–124.0) required without LLM assistance. Conclusions: An LLM-based approach, whether used autonomously or in conjunction with clinical expertise, enhances both accuracy and efficiency in SINS calculation. Adopting this technology may streamline oncologic workflows and facilitate more timely interventions for patients with spinal metastases.