Development and Evaluation of the Reliability and Validity of Video-Based Assessment Checklists of Nursing Skills via Chest-Mounted Cameras for Home-Visiting Nurses

TL;DR

This study develops and evaluates video-based checklists to objectively assess home-visiting nurses' skills, offering a reliable and valid alternative to subjective evaluations.

Contribution

The study introduces a novel video-based assessment method with high reliability and validity for evaluating nursing skills in home care settings.

Findings

Checklists showed substantial to almost perfect inter- and intrarater reliability (AC1: 0.63–1.00).

Sensitivity and specificity of the checklists exceeded 0.90, indicating strong concurrent validity.

The video-based assessment is feasible and cost-effective for quality assurance in home care.

Abstract

Ensuring that home-visiting nurses (HVNs) possess adequate skills for providing appropriate care is crucial in the context of increasing service demand with advanced care needs. However, current quality assurance relies on mentors' subjective assessments during accompanying visits, which are burdensome and rater-dependent. This study aimed to develop and evaluate the inter- and intrarater reliability and concurrent validity of objective video-based assessment (VBA) checklists for HVNs' nursing skills, focusing on peripheral intravenous catheter placement, pressure injury care, defecation care, and skin tear care to reduce the burden of skill assessment at home and ensure valid skill assessments.

The checklists were developed through a literature review, focus group interviews, and a group meeting using the nominal group technique to ensure content validity. The inter- and intrarater reliabilities and concurrent validity of the checklists were evaluated using videos of nurses providing care to simulated patients. Five nursing researchers with experience in mentoring nurses assessed the videos twice using checklists with a washout period. The inter- and intrarater reliabilities were analyzed using Gwet's AC1 statistics. Concurrent validity was evaluated with sensitivity and specificity of assessment using checklists against raters' subjective assessments, with expertise in nursing skills as the gold standard.

The checklists demonstrated substantial to almost perfect inter- and intrarater reliabilities (AC1: 0.63–1.00) for each item. The overall assessments also showed more than substantial inter- and intrarater reliabilities (AC1: 0.72–1.00). The sensitivity and specificity of the checklists were both above 0.90, indicating high concurrent validity.

The checklists were reliable and valid for objectively assessing HVNs' skills. The VBA suggests feasible and cost-effective methods for quality assurance, potentially improving the quality of home care. Further studies with patients in diverse clinical settings are required to enhance the generalizability of the results.

Figures