Content validity of measures in early numeracy in children up to eight years: A COSMIN systematic review

TL;DR

This review evaluates the quality of early numeracy tests for children up to eight years old and finds insufficient evidence to recommend any for use.

Contribution

The study is the first to systematically assess the content validity of early numeracy measures using the COSMIN framework.

Findings

Six early numeracy measures were evaluated, but none were recommended due to insufficient high-quality evidence on content validity.

No high-quality evidence indicated poor content validity, leaving the measures potentially usable.

Limited access to measures may have reduced the completeness of the review.

Abstract

Early numeracy skills are considered essential predictors for later mathematical and educational achievement. Therefore, there is a need for early numeracy measures with psychometrically sound properties. This systematic review aimed to determine the content validity of all current early numeracy measures in accordance with the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) framework and methodological guidelines, and was conducted and reported by following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement and checklist. Systematic literature searches were conducted in January 2024 in five electronic databases: CINAHL, Embase, Eric, PsycINFO, and PubMed. Eligible measures assessed numeracy, targeted children up to eight years of age, were published in English in 1995 or later, and had psychometric data on measure dimensionality. Eligible psychometric reports that were published in English described instrument development and/or content validity of included measures. The measures’ methodological quality was assessed using the COSMIN Risk of Bias checklist, after which all three aspects of content validity (i.e., relevance, comprehensiveness and comprehensibility) were evaluated. Six early numeracy measures and eleven psychometric reports were included. None of the measures could be recommended for use in clinical practice, education, or research due to a lack of high-quality evidence on content validity. However, no high-quality evidence was found to indicate insufficient content validity, therefore, all measures still have the potential to be used. Limited access to measures in the domain of early numeracy, despite having contacted both publishers and instrument developers, may have negatively impacted the completeness of the current overview of content validity of early numeracy measures. In line with the COSMIN guidelines, after the initial evaluation of content validity, future studies should evaluate the remaining psychometric properties of the included measures to identify the most robust measures in terms of validity, reliability, and responsiveness.

Figures