Development and psychometric properties of the Clinical Anxiety Scale for People with Intellectual Disabilities (ClASP-ID)

TL;DR

Researchers developed a new anxiety assessment tool for people with intellectual disabilities, which works well across different groups and communication abilities.

Contribution

The paper introduces a novel parent-report questionnaire, the Clinical Anxiety Scale for People with Intellectual Disabilities (ClASP-ID), with strong psychometric properties for assessing anxiety in individuals with moderate to severe intellectual disabilities.

Findings

The ClASP-ID has a four-factor structure with the anxiety factor explaining the most variance.

The anxiety subscale showed strong internal consistency, reliability, and validity across diverse subgroups.

Individuals with both autism and anxiety had higher anxiety scores compared to those without anxiety.

Abstract

There is a critical need for the development of dependable and valid anxiety assessment tools suitable for people with moderate to severe intellectual disabilities, particularly those who speak few or no words. Distinguishing anxiety from distress caused by physical discomfort (pain) or characteristics associated with autism, prevalent in this population, necessitates specialised assessment tools. This study (a) developed a parent-report anxiety questionnaire tailored for individuals with severe to moderate intellectual disabilities, potentially with a co-diagnosis of autism, and (b) evaluated the psychometric attributes of this novel measure.

A comprehensive approach involving literature reviews, inspection of existing tools, and interviews with clinicians and parents guided the creation of the Clinical Anxiety Scale for People with Intellectual Disabilities. The tool was completed by parents or caregivers (N = 311) reporting on individuals aged 4 or older with intellectual disabilities.

Exploratory factor analysis indicated a four-factor structure encompassing anxiety, pain, low energy/withdrawal, and consolability. The anxiety factor explained the most variance in scores (26.3%). The anxiety, pain, low energy/withdrawal subscales demonstrated robust internal consistency (α = 0.81-0.92), and convergent, divergent, and discriminant validity. Robustness of these subscales was further evidenced by test-retest reliability (ICC = 0.79-0.88) and inter-rater reliability (ICC = 0.64-0.71). Subgroup analyses consistently demonstrated strong psychometric properties among individuals diagnosed with non-syndromic autism (N = 98), children (N = 135), adults (N = 175), and across diverse communication abilities within the sample. Moreover, individuals diagnosed with both autism and anxiety exhibited significantly higher scores on the anxiety subscale compared to those without an anxiety diagnosis, while showing no difference in autism characteristic scores.

The findings indicate that the Clinical Anxiety Scale for People with Intellectual Disabilities is a promising measure for use across diverse diagnostic groups, varying communication abilities, and with people with moderate to severe intellectual disabilities.

The online version contains supplementary material available at 10.1186/s11689-024-09554-9.

Full text

Background

Anxiety in intellectual disabilities

Anxiety is more prevalent among individuals with intellectual disabilities, ranging from 14 to 42%, compared to the 2–24% rate in the general population [1–4]. This elevated anxiety occurrence is influenced by the aetiology of intellectual disabilities, with certain rare genetic syndromes demonstrating higher anxiety rates than mixed aetiology groups [5–7]. Despite being individually rare, these genetic syndromes collectively represent a significant proportion of people with severe intellectual disabilities, which may heighten anxiety risk within this population [8–10]. Moreover, many individuals with severe intellectual disabilities have co-occurring autism1 (lifetime prevalence: 42%), which is a further risk factor for anxiety [13–15].

Anxiety significantly impacts the well-being of individuals with intellectual disabilities, as well as the general population. It leads to adverse outcomes such as depression, disrupted sleep patterns, compromised social functioning, and diminished quality of life [16–19]. However, despite concerns regarding anxiety risk within people with moderate to severe intellectual disabilities, and the consequential effects of anxiety, there remains a significant scarcity of assessment measures tailored for individuals in this population [20]. This dearth of measures severely limits research aimed at quantifying, understanding, and addressing anxiety in this population.

Challenges with anxiety measurement in intellectual disabilities

Measuring anxiety in individuals with moderate to severe intellectual disabilities, especially those who speak few or no words, presents several challenges. First, anxiety assessment is complicated by the overlap between behavioural indicators of anxiety, distress due to physical health issues (i.e., pain), or the presence of alternative diagnoses (e.g., autism or depression). For instance, negative vocalisations like screaming and crying, sleep disturbances, and behaviours that challenge are commonly associated with pain, anxiety, and low mood [21–25]. Furthermore, psychomotor agitation, such as restlessness and irritability have been associated with both anxiety and low mood in people with intellectual disabilities [22, 26]. Differentiating anxiety from other possible causes of presenting behaviour is crucial, as mental health difficulties require a different approach to intervention compared to distress arising from physical health difficulties. Alternatively, a multi-modal intervention may be required where there is an interaction between factors leading to distress. The importance of disentangling anxiety from indicators of pain, and raising awareness among clinicians and researchers of the need to consider differential diagnosis, is paramount given the high rates of painful physical health conditions experienced by people with moderate to severe intellectual disabilities [27–29].

A further challenge is that qualitative differences in anxiety presentation in people with intellectual disabilities have been reported compared to the general population. For example, repetitive behaviours have been associated with anxiety in people with intellectual disabilities and autistic people [30–32]. Including the full range of potential behavioural markers of anxiety during the development of measures may be crucial for detecting anxiety in people who speak few or no words who cannot communicate their anxiety verbally. Despite this, atypical behavioural indicators of anxiety can be overlooked or misattributed to intellectual disabilities, a person’s genetic syndrome, autism, or other co-occurring conditions, a phenomenon known as diagnostic overshadowing [21].

There are no anxiety measurement tools that were developed with the aim of thoroughly addressing these challenges. In the absence of available measures, researchers and clinicians often rely on anxiety measures developed for the general population. However, many of these measures depend on a person being verbal and able to describe worries or label their internal states. Parents and caregivers reporting on anxiety in their children with intellectual disabilities may be less likely to endorse items regarding their child’s internal thoughts and feelings, leading to the under-reporting of anxiety symptomatology [33]. Hence, these measures are not appropriate for people with severe intellectual disabilities.

Some questionnaire measures designed to assess anxiety in people with intellectual disabilities do exist, such as the Anxiety Depression and Mood Scale (ADAMS; 34) and the Diagnostic Assessment for the Severely Handicapped (DASH-II; 67). The ADAMS has robust psychometric properties, however, its suitability for people with severe intellectual disabilities is uncertain as it was developed for those with mild to moderate intellectual disabilities [20, 34]. The DASH-II, while a reliable and valid measure developed for people with severe intellectual disabilities, was partly informed by the diagnostic criteria from the Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev., DSM-III-R) [35], which may not be appropriate when applied to people with intellectual disabilities [22, 36]. Furthermore, while the DASH-II was a pioneering measure developed in the 1990s, the recent developments in the understanding of the anxiety profile observed in autistic people with intellectual disabilities necessitates further measure development [37]. Finally, there is a need for a measure that uses contemporary terminology and that prompts clinicians and researchers to consider differential diagnosis (e.g., pain) when behavioural indicators of distress are observed [38].

Some promising new measures have been developed to examine anxiety experienced by autistic people, such as the Anxiety Scale for Children with Autism Spectrum Disorder (ASC-ASD) [39] and the Anxiety Disorders Interview Schedule-Autism Addendum (ADIS/ASA) [40]. Both measures perform well but were developed specifically for people without intellectual disabilities or with a mild intellectual disability. Therefore, there remains a dearth of measures designed to assess anxiety that are appropriate for people with moderate to severe intellectual disabilities, particularly those who speak few or no words and those with co-occurring autism [20].

Flynn et al. [20] advocate for the integration of bottom-up approaches in the development of measures tailored for individuals with severe intellectual disabilities. They suggest reducing reliance on conventional diagnostic criteria and existing measures developed for people without intellectual disabilities. Such an approach is justified given the anxiety-related behaviours shown by people with severe intellectual disabilities may elude capture by current assessment tools [6]. Thus, adopting a bottom-up approach, which draws insights from detailed behavioural descriptions of anxiety, is imperative for the development of new measures specifically designed for individuals with moderate to severe intellectual disabilities.

Aims

The present study had three aims:

1. To draw on a range of approaches, including bottom-up methodology, to develop an informant-report anxiety screening measure that is suitable for people with moderate to severe intellectual disabilities, including those who speak few or no words and individuals with a diagnosis of autism.
2. To develop a measure to overcome limitations of previous measures by having the potential to inform differential diagnosis between anxiety and other forms of internal distress, namely pain and low mood.
3. To investigate the psychometric properties of the newly developed measure, including the factor structure, internal validity, test-retest reliability, inter-rater reliability, convergent and divergent validity.

Method

The study has received a favourable ethical decision from the NHS Research Ethics Committee Wales REC 3 (ref: 18/WA/0139); Research Registry (UIN: researchregistry5086).

Development of items for the clinical anxiety scale for people with intellectual disabilities (ClASP-ID)

Literature review

Due to the complexity of differentiating anxiety in people with intellectual disabilities from other forms of internal distress (pain and low mood) a broad and inclusive approach was taken to item generation. Systematic literature reviews on the behavioural indicators of anxiety and low mood in intellectual disabilities were conducted [22, 41]. Behaviours were extracted from these reviews and entered onto a database for the purpose of item development. To generate additional behavioural items to those identified by the systematic reviews, thirty-six existing measures pertaining to measure anxiety, low mood and/or pain within typically developing children (e.g., Spence Children’s Anxiety Scale, Face, Legs, Activity, Cry, Consolability scale), autistic individuals (e.g. Autism Spectrum Disorder -Comorbidity for Adults) and individuals with mild to moderate intellectual disabilities (e.g. ADAMS, Glasgow Anxiety Scale), were examined, along with existing measures of mental health and pain for individuals with severe to profound intellectual disabilities and individuals who speak few or no words (e.g. Mood Interest and Pleasure Questionnaire; see additional file 1 for full list). These 36 measures were identified from the systematic reviews of behavioural indicators of anxiety and low mood [22, 41–43] and existing reviews of mental health tools and pain indicators in people with intellectual disabilities and autism [20, 44].

Across these measures 1028 items were reviewed. To aid item generation and reduction, these items were grouped into categories based on topographical similarity. All items and item categories were reviewed by three members of the research team (JT, GE and JW).

Interviews

To further supplement the item database, and due to the complexity of identifying anxiety in the intellectual disability population, parent interviews were conducted to gather information on individual anxiety signatures displayed by children and adults with intellectual disabilities; published in [38]. Parent strategies to manage anxiety were discussed as part of these interviews to provide insight into mutually reinforcing behaviour that may indicate anxiety (e.g., avoidance of feared situations; published in [25]. The interview schedule was previously developed as part of a research project aiming to explore anxiety in intellectual disabilities utilising clinical formulation frameworks [45]. In total, 30 parent/carer interviews were conducted with parents of people who speak few or no words. This included 21 autistic children and adults with no known genetic condition, in addition to 9 people with intellectual disabilities of mixed aetiology (e.g. fragile X syndrome, Cornelia de Lange syndrome) to pick up wider anxiety phenomenology (3/9 also had a diagnosis of autism). For the purposes of tool development and item generation, the interviews were coded using an existing coding proforma to identify behaviours and triggers for anxiety [45]. There were few differences between reported anxiety signatures and triggers between children and adults [38]. See additional file 2 for a summary of coded interview data.

Nine healthcare workers from NHS learning disability clinics (clinical psychologist n = 2; community learning disability nurse n = 2; clinical nurse specialist n = 2; paediatric neuropsychiatrist n = 1; psychiatrist n = 1; paediatrician n = 1) were also interviewed about the presentation of anxiety in people with intellectual disabilities, differential diagnosis (e.g., pain, low mood, autistic characteristics) and the implications for developing assessment measures (for full details see 38).

Beta version of the measure

The information generated from these steps was synthesised, and items were drafted to represent the identified behavioural codes. This resulted in a 69-item beta version of the scale that captured the breadth of behavioural expression identified in the interviews and literature reviews (see additional file 3). The beta version also included one additional open-ended question to ask parents about other changes in mood or behaviour not captured in the questionnaire. Due to the processes followed above, it was anticipated that the 69-items would assess behaviours associated with anxiety, low mood, and pain in people with intellectual disabilities. Due to previous studies indicating overlapping behavioural indicators across anxiety, low mood and pain, no a-priori assumptions were made about which behaviour items would load onto which specific scales, as this was to be examined via the exploratory factor analysis.

The 69 items of the ClASP-ID items were measured on a 7-point Likert scale from ‘almost never’ to ‘all the time’ or ‘almost never’ to ‘more than once a day’. Research has demonstrated several benefits of the 7-point scale including it providing sufficient granularity, leading to more nuanced and accurate data, reducing midpoint bias, and providing balanced response options [46–48]. In line with methodological guidance, each point on the scale was operationalised, to reduce need for individual interpretation of the response options [47, 49]. Respondents are asked to report on the frequency of behaviours over the last one month.

Following clinician feedback of the importance of being able to assess a person’s change from baseline, particularly on items which overlapped with autistic characteristics (e.g., repetitive behaviour; [38], 14 items included follow-up questions which asked whether behaviour is currently occurring more, at the same rate, or less than is typical of the person with intellectual disability. These questions were designed to provide clinical context and so were not included in the scoring of the questionnaire and therefore the psychometric validation.

Parent and clinician review of the 70-item scale

As a final step, the beta version of the measure was circulated to four parents of children with intellectual disabilities. All parents were part of the Cerebra Centre advisory panel. Due to the length of the draft measure and to reduce participant burden, the feedback process was conducted using a questionnaire feedback proforma (see additional file 4). Parents were asked to read the questionnaire instructions and provide feedback on elements of the instructions that were unclear. Parents then read each item of the questionnaire and were asked to note next to the item if it was confusing or unclear and suggest any alternatives for how a question could be phrased. Following feedback from the families, a panel of researchers (JW, JT & GE) discussed the comments, and final amendments were made to the questionnaire.

Psychometric properties of the ClASP-ID

Recruitment

In total, 314 parents or caregivers of people with intellectual disabilities were recruited for the refinement and psychometric evaluation of the measure. When asked how they heard about the study, parents reported it was via invite from participating NHS trusts (n = 32), mailing lists of people who had previously taken part in research (n = 203), snowball sampling (n = 6) or social media (n = 18); some parents reported multiple recruitment routes and some declined to answer this question. Parents or caregivers were asked to complete study questionnaires about a person aged 4 or older with a diagnosis of an intellectual disability. There was no upper age limit, and no exclusion criteria based on diagnosis of co-occurring conditions such as rare genetic syndromes, neurodevelopmental conditions, physical or mental health difficulties. The minimal exclusion criteria facilitated a representative sample, given the high prevalence of co-occurring conditions in people with intellectual disabilities.

Participants

Of the 314 parents and caregivers who were recruited, three were excluded from the present analysis. This was because inclusion criteria necessitated a diagnosis of an intellectual disability, either by parent report or by scores on the Wessex Questionnaire [50]. The Wessex Questionnaire acts as a proxy measure for degree of disability (see measures section for more information). These three participants were excluded as they did not report that the person they care for had an intellectual disability and the scores provided on the Wessex Questionnaire were not consistent with the presence of an intellectual disability diagnosis. A further ten parents and caregivers did not report that the person they care for had a diagnosis of an intellectual disability, but scores on the Wessex questionnaire indicated that the person had communication or self-help needs consistent with the presence of intellectual disabilities. These ten participants were included in the analysis.

In total, 311 parents and caregivers were included in the analysis. They reported on a group of people with intellectual disabilities who ranged in age from 4 to 83 (M = 21.6, SD = 12.0) and were 61.4% male. Owing to the broad inclusion criteria, a range of rare genetic syndromes associated with intellectual disabilities were reported including Angelman syndrome (n = 45, 14.5%), fragile X syndrome (n = 26, 8.4%), and Cornelia de Lange syndrome (n = 22, 7.1%). Neurodevelopmental conditions were also commonly reported, including autism (n = 156, 50.2%) and ADHD (n = 44, 14.1%). Parents and caregivers reported a diagnosis of an anxiety disorder in approximately one fifth of the people with intellectual disabilities (n = 68, 21.9%). Parents and caregivers most frequently reported that the person they care for spoke odd words only (n = 127, 40.8%), was partly able in terms of self-help ability (n = 119, 38.3%) and fully mobile (n = 187, 60.1%). For full demographic information, see Table 1.

Table 1. Demographic characteristics of the sampleN (%)Person with ID gender Male191 (61.4) Female120 (38.6)Age of person with ID Under 18135 (43.4) 18–2475 (24.1) 25–3449 (15.8) 35–4438 (12.2) 45–5411 (3.5) 55+3 (1.0)Intellectual Disability Diagnosis Yes301 (96.8) No10 (3.2)Syndrome Diagnosis Angelman Syndrome45 (14.5) Cornelia de Lange Syndrome22 (7.1) Fragile X Syndrome26 (8.4) Prader-Willi Syndrome16 (5.2) Down Syndrome15 (4.8) Cri du Chat Syndrome11 (3.5) Potocki-Lupski Syndrome11 (3.5) Tuberous Sclerosis10 (3.2) 9q34 Deletion8 (2.6) Phelan McDermid Syndrome6 (1.9) Pitt Hopkins Syndrome5 (1.6) Other44 (14.2)Psychiatric and/or Neurodevelopmental Diagnosis Autism156 (50.2) Anxiety68 (21.9) ADHD44 (14.1) Obsessive-Compulsive Disorders27 (8.7) Depression20 (6.4)Verbal Ability^a^ Non-verbal81 (26.0) Odd words only127 (40.8) Fully verbal89 (28.6) Can talk but doesn’t14 (4.5)Self-Help^a^ Not able112 (36.0) Partly able119 (38.3) Able80 (25.7)Mobility^a^ Non-ambulant41 (13.2) Partly mobile82 (26.4) Fully mobile187 (60.1)^a^ Information obtained using the Wessex Questionnaire [50]

Procedure

Information about the study was sent to participants via post, email or telephone to invite them to take part. The information sheets and study consent forms were available to participants on the survey engine Qualtrics. Once consent forms were completed, participants could proceed to complete study questionnaires. Parents and caregivers could also request paper copies of the information sheets, consent forms and study questionnaires if they preferred. Parents and caregivers completed an online battery of questionnaires on anxiety, pain, health difficulties, mood, autistic characteristics, intolerance of uncertainty, restricted and repetitive behaviours, behaviours that challenge and sensory processing differences. The battery of questionnaires was conducted as part of a wider project whereby the validation of the ClASP-ID was one aim of this project.

At the end of the questionnaire battery, parents and caregivers were asked if they were happy to be contacted in 2–4 weeks to complete portions of the battery for a second time to conduct an analysis of test-retest reliability. This data was available for 78 participants (see additional file 5 for demographics) and there were no significant differences in age, gender, ability level, and proportion of participants with an autism, anxiety or depression diagnosis between the test-retest reliability sample and the total sample. Participants were also asked if there was anyone else who knew the person they care for well who would also be willing to complete a small portion of the questionnaires to obtain a measure of inter-rater reliability. A second rater was available for 20 completions of the ClASP-ID (see additional file 5 for demographics). There were no significant differences in age, gender, ability level, and proportion of participants with an autism, anxiety or depression diagnosis between the inter-rater reliability sample (n = 20, 6% of sample) and the total sample.

Participants who did not complete the full questionnaire battery or had missing data were contacted once by the research team to complete their dataset. Where missing data remained, available-case analysis was used to preserve as much data as possible.

Measures

In the interest of brevity, only measures included in the present analysis, for the purpose of refinement and evaluation of the psychometric properties of the ClASP-ID, are described below.

Measure of interest- clinical anxiety scale for people with intellectual disabilities (ClASP-ID)

Parent and caregivers were asked to complete the beta version of the questionnaire (described above) which asks about behavioural indicators pertaining to anxiety, low mood and pain in people with intellectual disabilities. Items are scored on a seven-point scale based on frequency of behaviours.

Wessex questionnaire [50]

This is a 16-item informant report questionnaire designed to assess ability level including communication, self-help skills and mobility, used in this study as a proxy measure of intellectual disability. Presence of difficulties with communication, self-help skills such as washing and feeding, and mobility are consistent with diagnosis of intellectual disability [51]. Items are scored on a three-point scale, except for an item on speech which includes a fourth option to encompass those who can speak but do not. Higher scores on items indicate higher ability levels. Inter-rater reliability of the speech, self-help and mobility subscales are good at 82%, 78% and 92% respectively. This measure has been used extensively in large-scale questionnaire studies as a proxy measure of ability level, where in-depth assessments are not feasible [52–54].

Anxiety, depression and mood scale (ADAMS; 34)

The ADAMS is a 28-item informant report screening questionnaire for psychiatric disorders in intellectual disabilities. Items are scored on a four-point scale based on interference with daily life. Items are summed to create a total score and five subscales including general anxiety, social avoidance, depressed mood, manic/hyperactive, and obsessive/compulsive. The ADAMS has good internal validity (0.75–0.83), excellent test-retest reliability (0.81) and fair inter-rater reliability (0.48) [34]. The general anxiety subscale and the depressed mood subscales from the ADAMS were used in the ClASP-ID validation study given they capture the constructs of interest (anxiety and low mood), whereas the ClASP-ID was not developed to measure manic/hyperactive and obsessive/compulsive behaviours.

Health questionnaire- current [55]

The health questionnaire (current) is a 16-item informant questionnaire used to assess the presence and severity of health difficulties in the past month. Items are scored on a four-point scale and are summed to create an overall indicator of physical health in the past month. Inter-rater reliability of the measure is good (ICC = 0.71). The health questionnaire was used as a proxy measure of physical discomfort due to concerns that existing pain measures included items that could also be measuring anxiety, autism, or low mood.

Social communication questionnaire- current (SCQ; 68)

The SCQ (current) is a 40-item informant report questionnaire designed to assess behaviours associated with autism, including reciprocal social interaction, communication, and restricted and repetitive behaviours. Items are scored yes or no, and higher scores indicate higher autism characteristics. The SCQ is a sensitive screener for autism and has good internal consistency [56, 57].

Analytical approach

All the primary 69 items on the ClASP-ID were included in the analysis.

To refine the measure and assess the factor structure, an exploratory factor analysis (EFA) was conducted using principal axis factoring, with a promax rotation applied to account for the anticipated correlations between items and emerging factors. The sample size of 311 was above the minimum of 100 which is often cited as a requirement to conduct an EFA, however, it is slightly below the recommended five times the number of items recommended to conduct an EFA [58–61]. Overall, an EFA was deemed appropriate as statistical tests indicated the sample was appropriate; Bartlett’s test of sphericity was significant, and sampling adequacy was above the recommended 0.5 (KMO = 0.91; [61]). The EFA was repeated several times to obtain a simple structure, whereby every item loaded significantly to only one factor [62].

Internal consistency of the measure was established using Cronbach’s alpha, and test-retest and inter-rater reliability were established using intraclass correlations at scale level. Convergent and divergent validity was established by conducting Spearman’s rho correlations between the ClASP-ID subscales and the ADAMS and health questionnaire. Known groups validity was established by conducting t-tests to establish if those with a reported anxiety disorder diagnosis would score significantly higher on the ClASP-ID anxiety scale than those without an anxiety disorder diagnosis. This analysis was repeated in an autistic group only, to ensure the measure performed well in autistic populations. All data was analysed using SPSS 28.0.

Results

Exploratory factor analysis

An exploratory factor analysis (EFA, N = 304) was conducted using principal axis factoring, and a promax rotation was applied to account for the anticipated correlations between items and emerging factors [61]. All items were included in the EFA, regardless of their psychometric properties with the view to refine by factor loadings and psychometric properties later. This approach is common in the literature, for example in the development of the Anxiety Scale for Children with Autism Spectrum Disorder (ASC-ASD; 40).

The initial EFA extracted 17 factors. The first factor explained 26.3% of the variance, whilst the second, third and fourth factors explained a further 4.7%, 3.5% and 2.9% respectively. The remaining 13 factors explained fewer than 2% of the variance each and had few items loading to them. A four-factor solution was retained based on:

1. The percentage of variance explained by each factor, as described above.
2. Visual examination of the scree plot.
3. Clinical judgement of how items loaded to each factor, which was reviewed by a clinical psychologist and senior researcher with experience of mental health assessment.

The EFA was run for a second time, this time forcing a four-factor solution. At this stage, the ClASP-ID was refined, and several items were removed. This included: items which did not load to the final four factors (23 items), items with dual loadings above 0.4 (1 item), items with poor test-retest reliability of below 0.4 by Spearman’s correlation (1 item), items where the Cronbach’s alpha of the scale improved when removed (1 item), and items which could not distinguish between a those with and without a diagnosis of an anxiety disorder, a high and low pain group, and those with and without a diagnosis of depression by t-test (8 items). For a list of items which were removed from the ClASP-ID, see additional file 3.

The EFA was run for a third and final time with only the 35 remaining items, again forcing a four-factor solution. Two items no longer loaded to any of the four factors and were subsequently removed. Therefore, the final scale contained 33 items across four factors. Please see additional file 3 for a full list of items removed from the ClASP-ID. With the input of two clinical psychologists, the first, and largest, factor which contained 14 items was labelled anxiety, the second factor contained 9 items and was labelled pain, the third factor contained 6 items and was labelled low energy/withdrawal (a potential proxy measure of depression/low mood), and the final factor contained four items and was labelled consolability. Please see Table 2 for the final factor structure of the ClASP-ID.

Table 2. Factor structure of the ClASP-IDItemFactor 1AnxietyFactor 2PainFactor 3Low Energy/ WithdrawalFactor 4 Consolability68. We are unable to do ‘typical’ day to day activities because of the emotional distress that would cause him/her (e.g. holidays, visiting friends, going for meals, general days out)0.77923. Does he/she ever look very worried or anxious?0.77569. We are unable to do activities we used to do with the person I care for because of the emotional distress he/she would experience0.7687. Does he/she appear on edge OR on the look out for danger?0.75232. Over the past month, have you noticed his/her face look tense?0.68725. Does he/she have an angry look on his/her face?0.68012. Does he/she appear restless or agitated?0.67113. Does he/she ever run away or hide from certain objects or situations?0.63120. Does he/she avoid (or try to avoid) certain objects or places?0.6286. Does he/she ever make negative or frustrated vocalisations? (e.g. whining, grumbling, growling, shouting, screaming)0.59014. Does he/she ever cover him/herself with a blanket or try to place a barrier between him/herself and others or a situation?0.55011. Does he/she pace around the room?0.52326. Does he/she startle easily, or easily alarmed?0.52017. Does he/she ever freeze suddenly (stick to the spot) in response to specific situations?0.45735. Over the past month, has his/her movements ever become jerky?0.68628. Over the past month, have you noticed increased or different leg movements? (e.g. restlessness, tense, tremors, kicking, drawing legs up, jerking)0.64710. Does he/she ever seem protective of a particular part of his/her body? (e.g. holding it, guarding it, flinching? )0.56621. Does he/she ever take sharp intakes of breath or gasp?0.54336. Over the past month, has his/her lips ever become tight, pout or quiver?0.53716. Does he/she ever have watery eyes that is different from crying?0.49419. Does he/she ever grind his/her teeth?0.47631. Over the past month, have you noticed that he/she shakes or trembles?0.42434. Over the past month, has he/she been hitting, holding or touching a part of their body?0.41052. Does he/she lack energy?0.79553. Does he/she get tired for no apparent reason?0.7374. Has he/she seemed withdrawn with ‘vacant’?0.61860. Is he/she spending more time asleep than usual? (e.g. not waking in the morning, sleeping during the day)0.56645.Has he/she lost interest in activities that he/she used to enjoy?0.43461. Is he/she quiet and spending time alone?0.43066. When the person I care for is distressed, I am able to calm or comfort him/her0.66867. When in certain preferred environments (e.g. home, their bedroom) the person I care for generally appears calm and relaxed0.62765. Removing the person I care for from a situation, or removing an item/object generally calms them down0.47664. Preparing him/her before things happen helps to reduce his/her distress0.464

Reliability

Internal consistency

Internal consistency (N = 309) of the anxiety subscale was excellent (α = 0.92) and the pain and low energy/withdrawal subscales were good (α = 0.81, and α = 0.81). Cronbach’s alpha for the consolability subscale was lower (α = 0.63). Cronbach’s alpha was also calculated for several sub-groups to ensure performance was maintained across the sample. This included: children only (n = 135), adults only (n = 175), autistic groups (n = 156 all autism, n = 98 non-syndromic autism), and those with the highest level of support needs (n = 211 minimally verbal, n = 81 non-verbal, n = 123 non- or partially ambulant, n = 105 minimally verbal and not able in terms of self-help skills). For all these groups, internal consistency of the anxiety, pain and low energy/withdrawal subscales remained excellent or good. The Cronbach’s alpha of the consolability subscales remained consistent in most groups but improved in others (α = 0.58-0.64). Please see additional file 6 for a full breakdown of internal consistency in these sub-groups.

Test-retest reliability

Test-retest reliability at 2–4 weeks (N = 78) was good for the anxiety, pain and low energy/withdrawal subscales (ICC = 0.88, 0.80 and 0.79 respectively), and moderate for the consolability subscale (ICC = 0.67). Where sample sizes allowed, test-retest reliability was calculated for the same sub-groups in the sample as defined above. Test-retest reliability of the anxiety, pain and low energy/withdrawal subscales remained good or improved to excellent in most groups. The most notable change was for the children only group, where test-retest reliability of the pain subscale was moderate (ICC = 0.63) and the consolability subscale was good (ICC = 0.78) Please see additional file 6 for a full breakdown of test-retest reliability in these sub-groups.

Inter-rater reliability

Inter-rater reliability (N = 20) was completed by a second parent (n = 12, 60%), care or support worker (n = 4, 20%) or other family members (n = 2, 10%). Intraclass correlations showed that inter-rater reliability was moderate for the anxiety, pain and low energy/withdrawal subscales (ICC = 0.640, 0.690 and 0.712 respectively) and poor for the consolability subscale (ICC = 0.162).

Validity

Convergent and divergent validity

To assess convergent and divergent validity of the measure, anxiety, pain and low energy/withdrawal subscale scores were correlated against scores on the generalised anxiety and depression subscales of the ADAMS and health questionnaire severity scores for painful items only. There was evidence for good convergent and divergent validity of the anxiety subscale, which correlated significantly with the ADAMS general anxiety subscale (r = .771, p < .001), but lower with the ADAMS depressed mood subscale (r = .469, p < .001), and health questionnaire severity scores (r = .230, p < .001).

Evidence for convergent and divergent validity of the pain subscale was mixed. Whilst the pain subscale correlated significantly against health questionnaire severity scores (r = .363, p < .001), it correlated more strongly with the ADAMS general anxiety and depressed mood subscales ([r = .522, p < .001], r = .413, p < .001]). However, of the ClASP-ID subscales, the pain subscale had the highest correlation with the health questionnaire severity scores.

There was evidence for good convergent and divergent validity of the low energy/withdrawal scale, which correlated most strongly with the ADAMS depressed mood subscale (r = .702, p < .001), and less strongly with the ADAMS general anxiety subscale and health questionnaire severity scores ([r = .476, p < .001], [r = .352, p < .001]). Please see Table 3 for a summary of these results.

Table 3. Convergent and divergent validity of the ClASP-ID subscales, Spearman’s rho correlationsADAMS General AnxietyADAMS Depressed MoodHealth Questionnaire Severity (painful only)ClASP-ID Anxiety0.7710.4690.230ClASP-ID Pain0.5520.4130.363ClASP-ID low energy/withdrawal0.4760.7020.352Note: Due to missing data on some measures, N varied. Minimum N = 290

Known-groups validity

The discriminant validity of the ClASP-ID was explored by conducting t-tests to compare scores of clinical and non-clinical groups. Those whose parents or caregivers reported a clinical diagnosis of an anxiety disorder (n = 64) scored significantly higher on the anxiety subscale of the ClASP-ID than those without a reported clinical diagnosis of anxiety (n = 244) (t(306) = 6.02, p < .001). The effect size was large (d = 0.845). Individuals whose parents and caregivers reported a clinical diagnosis of depression (n = 20) scored significantly higher on the low energy/withdrawal scale of the ClASP-ID than those who did not report a diagnosis of depression (n = 289), (t(307) = 4.31, p < .001). The effect size was large (d = 0.846).

Known-groups validity: autistic group

To ensure the ClASP-ID was appropriate for autistic people specifically, further analysis was conducted. Autistic people with an anxiety disorder (ASD + anx, n = 49) scored significantly higher on the ClASP-ID anxiety subscale than those without an anxiety disorder diagnosis (ASD-anx, n = 107) by t-test (t(154) = 2.96, p = .004). There were no significant differences in SCQ scores between the ASD + anx group and the ASD-anx group (p_s_ > 0.05). Autistic people with a depression diagnosis (ASD + dep, n = 13) scored significantly higher on the ClASP-ID low energy/withdrawal subscale than those without a depression diagnosis (ASD-dep, n = 143) by t-test (t(153) = 4.69, p < .001). There were no significant differences in scores on the SCQ between the ASD + dep group and the ASD-dep group by (p_s_ > 0.05).

Discussion

This study investigated the psychometric properties of the ClASP-ID in a large, heterogenous sample of people with intellectual disabilities. The ClASP-ID is one of few measures designed to screen for anxiety in people with severe intellectual disabilities, and it is suitable for people who speak few to no words and those with a diagnosis of autism. In this initial sample, the EFA revealed four factors that were labelled anxiety, pain, low energy/withdrawal and consolability. Low energy/withdrawal is a subscale that may be capturing depression symptomatology, but it was not labelled as a low/mood or depression scale given that mood related items did not load to the scale. Including items to screen for pain and low energy/withdrawal within the ClASP-ID prompts clinicians to consider differential diagnosis early within the assessment process, which is important due to the high level of overlap between behavioural indicators of anxiety, pain, and low mood in this group and the different intervention strategies required for each [22, 23, 26].

When exploring the results of the factor analysis and labelling latent variables, we considered the 14 items loading onto the largest factor to represent anxious behaviours (e.g. startle/appear on edge). Whilst we have labelled this factor ‘anxiety’, without assessment of cognition underlying the behaviours or whether behaviours occur in anticipation of a threat, rather than in direct response to a threat, it is possible the items are also indicative of a similar (albeit related) construct (e.g., fear). However, the ClASP-ID is not intended as a diagnostic measure of anxiety. Instead, it is our hope that the ClASP-ID can be used as a screening tool to prompt clinicians to consider anxiety, potential differential diagnoses and identify concerning behaviours requiring more in-depth diagnostic assessment, which may include observations to confirm antecedents of behaviours endorsed in the scale.

The psychometric properties of the ClASP-ID are promising, with the anxiety, pain, and low energy/withdrawal subscales showing excellent or good internal consistency, good test-retest reliability, and moderate inter-rater reliability. The consolability subscale, however, had lower internal consistency and inter-rater reliability, and moderate test-retest reliability, so caution is recommended until further testing of the subscale is conducted. There was evidence for good convergent and divergent validity of the ClASP-ID anxiety and low energy/withdrawal subscales. There also was encouraging evidence for the validity of the pain scale, although, due to mixed findings, further studies are important to validate this scale. Finally, there were significant differences between clinical groups and non-clinical groups at scale level. The effect size for this was large. Owing to statistical power, factor structure could not be examined in specific groups in the study. However, psychometric properties remained stable across a range of groups, including children, adults, autism, and those with the highest support needs.

The lower performance of the consolability subscale may reflect difficulties with assessing this construct. More variation could be expected in measures of consolability due to individual differences both within individuals and between raters. This scale asks questions that examine whether preparation before an event reduces distress and whether a person can be comforted. Preparation before an event may help an individual, but once they become distressed it may become difficult to provide comfort; or preparation before an event may not be sufficient to reduce distress, which only occurs if a person is removed from a situation. Hence variation across individuals and contexts may contribute to poor internal consistency scores. Further, consolability is expected to differ between two raters. For example, a primary caregiver may be able to calm or comfort an individual when distressed, whereas a second rater might have more difficulty.

When compared to the other psychometric properties, inter-rater reliability of the ClASP-ID subscales was lower. This is consistent with other measures reporting on people with intellectual disabilities, owing to the heterogeneity in behaviours shown across different contexts in this population and raters basing their assessment on behaviours across differing settings (for example, ADAMS; 34). Our second raters were primarily a second parent (n = 12, 60%), but also included care or support workers (n = 4, 20%) and extended family members (n = 2, 10%).

There are several possible explanations for the mixed pattern of results observed regarding the convergent and divergent validity of the pain subscale. Whilst the pain subscale correlated more strongly than any other ClASP-ID subscale with health questionnaire severity scores, it correlated most strongly with the ADAMS general anxiety subscale, followed by the ADAMS depressed mood subscale. These mixed results may be caused by the overlap in behavioural presentation between anxiety and pain in people who speak few or no words, making the context these behaviours occur in more important [24, 25]. It is also important to note that the ADAMS general anxiety subscale contains items such as “trembles when frightening situations are not present” which may be assessing an indicator of pain, hence strengthening the association between the ClASP-ID pain subscale and the ADAMS general anxiety subscale. Similarly, a small number of items such as “tearful” on the ADAMS depressed mood subscale may be capturing indicators of pain.

Further, anxiety and pain often co-occur in people with complex needs. For example, there is a link between anxiety and gastrointestinal (GI) difficulties in autistic people, which may interact with one another, causing a cycle of heightened anxiety and heightened GI difficulties [24, 63–65]. Despite this, the pain subscale yielded promising results and therefore warrants further evaluation using more direct assessments. For example, completing this scale pre and post intervention for physical health difficulties would indicate whether the scale captures change due to a reduction in pain, and if this was demonstrated this would provide insightful validation data.

One of the significant strengths of the ClASP-ID is that it was developed using a combination of approaches, including bottom-up interview methodology, literature reviews and examination of existing measures with the involvement of mental health clinicians. All items can be scored based on observations of a person’s behaviour and are carefully operationalised; hence completion of the ClASP-ID does not require family members to access the internal thoughts or feelings of the person they support. The initial item pool was comprehensive and included items that were representative of traditional anxiety symptomology and atypical anxiety indicators (e.g. repetitive behaviours) that have been highlighted as important anxiety markers in autistic individuals [30–32].

It is of interest that in the final version of the measure, repetitive behaviours were no longer included as they did not load on to the anxiety factor. One explanation for this may be that while repetitive behaviours are strongly associated with anxiety and are of clinical importance for detecting anxiety, increases in these behaviours may reflect a response or coping mechanism to reduce anxiety rather than being a core symptom of anxiety. In the same way that behaviours that challenge are often a response to anxiety but are not a behavioural equivalent of anxiety, this may also be true for repetitive behaviours [22]. Furthermore, repetitive behaviour may be an important indicator of anxiety for a subset of individuals but not all, hence reducing the likelihood of these behaviours loading on to the anxiety factor across a heterogeneous group of individuals with severe intellectual disabilities.

The present study had some limitations that will need to be addressed in further studies. First, during interview development it was not feasible to conduct cognitive interviews to establish that parents and caregivers interpreted the questions correctly due to the number of items included in the initial version of the measure and the available timeframe for completing the study. Further, development did not include autistic people with intellectual disabilities themselves. A future study could evaluate the face-validity and acceptability of the measure for autistic people and people with intellectual disabilities, however, it is also important to recognise that the behavioural presentation of anxiety of people with mild intellectual disabilities may differ considerably from those with severe intellectual disabilities. So, whilst such a study would provide insight into the views of this population, the generalisability of these views to all individuals with intellectual disabilities could be a limitation of this approach.

Secondly, small sample sizes for the inter-rater reliability calculations (n = 20, 6% of sample) meant we were not able to use this to refine the ClASP-ID. Some items on the ClASP-ID have poor inter-rater reliability, and at present it is unclear if this is a consequence of a small sample size and high variability in relationship of the second rater to the person with intellectual disability, or poorly performing items. Further work should seek to investigate the item-level inter-rater reliability of the ClASP-ID in a larger sample. In addition, replicating this work in a large independent mixed aetiology group would allow the factor structure to be examined in larger samples of more specific groups (such as children only, adults only, or autistic groups only) where there was not adequate statistical power to do so in the present study. Finally, this larger sample would also allow for confirmatory factor analysis of this initial factor structure to be conducted.

This study was also reliant on parent reports of anxiety disorder diagnosis to define clinical groups and assess known-groups validity. Issues with anxiety diagnosis in people with severe to profound intellectual disabilities, such as diagnostic overshadowing and a lack of suitability of standard diagnostic criteria mean it is likely that the non-anxious group contains a substantial proportion of individuals with clinical levels of anxiety that is undiagnosed [20, 21, 66]. Known-groups validity of the measure may be strengthened once clinical groups are defined by direct assessment or physiological measures of anxiety as opposed to proxy report. Further work is ongoing to assess the validity of the ClASP-ID in samples where direct and clinical assessments of anxiety are being conducted.

Further work is needed to replicate the findings of this study across a range of intellectual abilities, using more in-depth measures of ability level which were not feasible to conduct in the present study due to the large sample size. As the ClASP-ID relies on behavioural markers of anxiety, we expect the measure may perform well in people with mild intellectual disabilities, but this has not yet been fully investigated owing to the smaller proportion of people with mild to moderate intellectual disabilities in this sample. In-depth assessments of ability level would provide additional confidence about the groups the ClASP-ID is most effective for and provide additional evidence of its utility for detecting anxiety in severe intellectual disabilities.

Whilst there is good evidence that the ClASP-ID is likely to be a measure that screens for anxiety across groups, it will be beneficial to validate the ClASP-ID in rare genetic syndromes associated with intellectual disabilities. Whilst rare genetic syndromes were included in the present study, recruitment was not focused on specific syndrome groups. Some syndromes may require additional validation given the complexity of anxiety, low mood, and pain presentations [32]. Similarly, whilst both adults and children with intellectual disabilities were included in this study, there were only a small proportion of adults in the study who lived away from their family (for example, in supported living). Data from adults not living with their family were included in the present study, although numbers were too low to conduct any psychometric analysis. Family members of adults with intellectual disability living away from home may have a different perspective on the person they care for, and future work should seek to validate the measure specifically in adults with intellectual disability not living with their parents.

Conclusions

In summary, this study has demonstrated that the ClASP-ID is a promising tool for the measurement of anxiety in children and adults with severe intellectual disabilities. In particular, the tool addresses a significant gap in the literature because it is suitable for people who speak few to no words, who have a co-diagnosis of autism, and across a range of ages. By prompting clinicians and researchers to consider differential diagnosis, the tool facilitates precision in measurement and assessment. The tool has significant potential to enhance research in the neglected field of mental health in people with severe intellectual disabilities. Furthermore, detecting anxiety earlier in people with intellectual disabilities increases the likelihood of improving the long-term wellbeing and outcomes of individuals and their caregivers.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Additional file 1- List of measures consulted during ClASP-ID development.

Additional file 2- Interview study data.

Additional file 3- Beta version of the ClASP-ID and list of items removed to form the final ClASP-ID.

Additional file 4- Parent questionnaire feedback proforma.

Additional file 5- Demographic information for the inter-rater reliability and test-retest reliability samples.

Additional file 6- Further psychometric properties of the ClASP-ID, including internal consistency and test-retest reliability of specific sample subgroups.