# Treatment Fidelity in a Feasibility Trial of the Aphasia Intervention, Virtual Elaborated Semantic Feature Analysis

## TL;DR

This study evaluated how consistently a new aphasia therapy called VESFA was delivered in a virtual environment, finding high treatment fidelity and useful insights for future trials.

## Contribution

The study provides a detailed application of treatment fidelity strategies in a novel aphasia intervention delivered in a virtual environment.

## Key findings

- The VESFA intervention was delivered with high fidelity, with 81% average adherence across all sessions.
- Individual sessions had lower fidelity (78%) compared to group sessions (84%).
- Participants used practiced words in real-world conversations, showing effective treatment enactment.

## Abstract

The reliability and validity of an intervention can be improved by checking treatment fidelity (TF). TF methods identify core components of an intervention, check their presence (or absence) and identify threats to fidelity. The Virtual Elaborated Semantic Feature Analysis (VESFA) intervention comprised individual sessions of word‐finding treatment and group sessions of conversation practice. All sessions were delivered in the virtual world of EVA Park. This paper describes the TF in the VESFA trial that explored (1) if the treatment was delivered as planned, (2) which components influenced treatment adherence scores and (3) the reliability of the fidelity checklists.

Strategies to improve fidelity were employed in the study design, the delivery of treatment, treatment receipt and treatment enactment. Two fidelity checklists were developed with input from advisors with aphasia to establish the core components of the intervention (individual and group). During the trial, treatment sessions were video‐recorded. A sample of 20% of sessions was randomly selected for adherence rating. Seven research students were trained to rate the videos using the fidelity checklists. Inter‐ and intra‐rater reliability was established.

Study design strategies ensured 94% of sessions ran as planned and 75% of participants (12/16) received over 90% (>36/40h) of the intended dose. The average TF across all sessions rated was 81%, demonstrating a high degree of fidelity in the delivery of the VESFA intervention. The fidelity of the individual sessions was lower (78%) than the group elements (84%). The components that most threatened treatment adherence were (1) providing a rationale for the activities and (2) specific feedback for performance. Nevertheless, participants consistently practised target words both in individual sessions and in conversations in the group sessions, demonstrating treatment receipt. Ninety‐four percent of participants (14/15) reported the words and phrases practiced in EVA Park were used in real‐world conversations, indicating treatment enactment. The fidelity checklists were reliable: Inter‐rater reliability was moderate (average Kappa of 0.76) and intra‐rater reliability was strong (average Kappa of 0.89).

A range of TF strategies were embedded within the trial protocol leading to high adherence to the core components of the VESFA intervention. Findings add to the evidence that aphasia therapies can be administered faithfully within the virtual environment of EVA Park.

The feasibility trial was not registered.

What is already known on this subject
Monitoring treatment fidelity improves both internal and external validity. Reports of treatment fidelity from aphasia trials are increasing, but the guidance is not yet applied uniformly.

Monitoring treatment fidelity improves both internal and external validity. Reports of treatment fidelity from aphasia trials are increasing, but the guidance is not yet applied uniformly.

What this study add to the existing knowledge
This study demonstrates how treatment fidelity guidance has been applied across a range of fidelity areas to monitor and support a feasibility trial of a novel aphasia intervention. It is a rare reporting of strategies to monitor treatment enactment.

This study demonstrates how treatment fidelity guidance has been applied across a range of fidelity areas to monitor and support a feasibility trial of a novel aphasia intervention. It is a rare reporting of strategies to monitor treatment enactment.

What are the potential or actual clinical implications of this study?
This study adds to the evidence base for the VESFA intervention, demonstrating that the intervention can be delivered faithfully to the manual. It builds on the evidence base for treatment fidelity monitoring in aphasia, broadening the strategies to improve the validity of interventions.

This study adds to the evidence base for the VESFA intervention, demonstrating that the intervention can be delivered faithfully to the manual. It builds on the evidence base for treatment fidelity monitoring in aphasia, broadening the strategies to improve the validity of interventions.