SEARCH WITHIN CONTENT
Citation Information : Scandinavian Journal of Child and Adolescent Psychiatry and Psychology. Volume 9, Pages 137-146, DOI: https://doi.org/10.21307/sjcapp-2021-015
License : (CC-BY-NC-ND 4.0)
Received Date : 19-February-2021 / Accepted: 22-June-2021 / Published Online: 02-July-2021
Within the field of psychiatry, the definition of a mental disorder implies significant distress or disability in social, occupational and/or other activities (1). However, many adolescents have multiple mild symptoms, or fewer severe symptoms, and do not meet the criteria for a mental disorder. Therefore, it is crucial to also consider the level of distress and disability in adolescents with psychiatric symptoms. Impairment can be measured in terms of the level of function (2-5). To assess this level, it is important to determine the capacity to perform a task in different settings (2-4).
The International Classification of Impairments, Disabilities, and Handicaps was developed in the 1970s. Using such a classification, the consequence of disabilities may be better assessed within a given physical and social environment. Some diagnostic instruments lack clarity in how to assess domains of functioning. For example, Angold et al. (6) indicated that a condition may be considered to be subthreshold according to the intensity of symptoms but can still cause low functioning (6-9). There is also a need to improve the measures of adolescent functioning and to develop psychometrically sound instruments. In psychiatric patients, disability is associated with the severity of specific symptoms, the number of different symptoms and the number of comorbid diagnoses (10, 11). The reduction of symptoms corresponds to an improvement in occupational and social functioning (12). It has been suggested that each diagnosis has its own pattern of disability (13, 14). For instance, major depressive disorder (MDD) is associated with low physical, cognitive and social functioning (15-17), social anxiety disorder (SAD) is associated with absence from school, isolation and social impairment (7), and attention-deficit/hyperactivity disorder (ADHD) decreases the ability to sustain attention and negatively influences school achievement, peer relationships and adjustment in the family (7, 18, 19). However, multiple factors beyond a diagnosis are related to the level of disability. These include the child’s intellectual development, the characteristics of the school and peers, and several lifestyle factors (20-22).
Previous research on functioning in adolescents has considered it in different settings: at school, with peers and at home in the family (20). One instrument used to assess functioning in adults is the Sheehan Disability Scale (SDS) (8). Moreover, Hörberg et al. have indicated that the correlation between the self-reported SDS score and the expert-rated Global Assessment of Functioning (GAF) score in a sample of adult patients was (r = – .606) (23).The SDS measures function in terms of impairment at work/school and in social and family life in adults with mental illness, and it is sensitive to change during treatment (14, 23-25). Whiteside (9) developed the Child Sheehan Disability Scale for self-report (CSDS) and parent report (CSDS-P) for use among children and adolescents (5–19 years) with anxiety. The CSDS estimates impairment in educational, social and family domains and has similar psychometric properties to those of the adult SDS.
To our knowledge, Whiteside (9) has published the only study to have applied the CSDS in child populations (one clinical and one community sample). Whiteside examined the inter-item correlation separately for the child and parent scales, and, because of the redundancy, the child’s ability to function in the family item was replaced by the parent’s ability to function in the family item in the parent scale. Thereafter, the principal component analysis of the CSDS-P found that a one-factor solution explained the variance in the parent scale.
This psychometric study explored the reliability and validity of the Swedish versions of the CSDS and CSDS-P in an adolescent psychiatric sample. Reliability was assessed by exploring internal consistency and factor structure. The sensitivity and specificity, and appropriate cut-offs of CSDS and CSDS-P to identify patients with at least one diagnosis versus those without a diagnosis, were tested. A functional impairment is a prerequisite for just about every diagnosis in psychiatry. Therefore, any diagnosis versus no diagnosis was chosen as the outcome. Sensitivity (to detect a diagnosis) was set to at least 80%, while specificity (to detect those without a diagnosis) was set to at least 50%.
To investigate concurrent validity, the Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (K-SADS-PL) and the Strengths and Difficulties Questionnaire (SDQ) were used for comparisons (8, 9, 26-29). Spearman’s rho correlations were used to calculate the concurrent validity of the total score of CSDS/CSDS-P and K-SADS. Moreover, the correlations between the total scores for CSDS and CSDS-P were analyzed, as well as the correlations between the items for “school” and “friends” from the two scales.
The participants were recruited from an ongoing cohort study of adolescents seeking help for psychiatric problems. Recruitment was performed between September 2011 and June 2013. All new psychiatric outpatients aged 12–17 years (N = 202) and their parents from two clinics in the County of Västmanland, Sweden, were invited to participate. Exclusion criteria were non-Swedish-speaking adolescent/parents or need of in-patient care at the time of the K-SADS-PL interview. A total of 125 (61.9%) patients participated, n = 53 (42.4%) male.
Of these, 107 adolescents provided full information on the K-SADS and CSDS, and 104 provided full information on the SDQ, while their parents completed the CSDS-P (n = 107) and SDQ (n = 104) using the Electronic Psychiatric Intake Questionnaire (EPIQ) (30) (Figure 1).
The remaining 18 patients were excluded from the study because of a lack of information. Internal drop-out analysis showed no significant difference in age or sex between those who completed the study (n = 107, 42% male, mean age 15.7 [SD 1.5] years) and those who did not (n = 18, 44.4% male, mean age 15.7 [SD 1.5] years).
The study was conducted in accordance with the Declaration of Helsinki (31) and approved by the Regional Ethics Committee of Uppsala (Dnr 2008/214).
At the first visit to the clinic, verbal information about the study was given, and informed consent was collected from participants. The adolescents answered the electronic psychiatric intake questionnaire (EPIQ), and parents answered the EPIQ-Parent version. Both versions include the SDQ. At the second visit, within one week, the adolescents and parents were interviewed with the K-SADS-PL. Thereafter, the adolescents and parents reported on the CSDS and the CSDS-P, respectively.
The interviewers were trained to perform the K-SADS interviews. The inter-rater reliability (IRR) was good to excellent, both before and during the period of data collection. The IRR rate was calculated for each diagnosis and each interviewer, and mean IRRs were calculated for the group and all diagnoses. Overall kappa for the group was .84 (range .54–1.00). Mean kappa was .89 for depression disorders, .79 for ADHD, and varied from .64 to 1.00 for anxiety disorders, as reported previously (32).
The CSDS and CSDS-P have been translated into Swedish. The original CSDS by Whiteside assesses general functioning in relation to “fears and worries” (9). In the Swedish version, this was changed to general functioning in relation to “troubles and feelings”. Three independent experts did the translation and back-translation for proper language adaptation and finally, the authors of this study approved the final translation.
For layout, see the Swedish CSDS and Swedish CSDS-P in Swedish and English in the Appendix.
The Swedish Child Sheehan Disability Scale (Swedish CSDS). The Swedish CSDS measures general functioning, in relation to “troubles and feelings”. It includes three items about functioning at school, with friends and at home. The questions are “How much do troubles and feelings cause problems for you when you are in school and when you do homework?”, “How much do troubles and feelings cause problems for you when you are with your friends?” and “How much do troubles and feelings cause problems for you when you are at home?”. The score of each item ranges from 0 (“not at all”) to 10 (“very, very much”), resulting in a total scale range of 0–30.
To define patients with low functioning, Leon et al. suggested a cut-off score of 5 for the SDS in adult primary care populations (25). In addition, Sheehan et al. proposed cut-off scores of ≤1 on each item of the SDS and <5 on the total score as remission criteria (14). By contrast, Whiteside studied the CSDS and the CSDS-P in two samples with a primary anxiety disorder and analysed the mean values of the most common anxiety diagnoses and the effect of the treatment without reference to a remission cut-off (9). Based on the SDS studies, a cut-off of 5 was chosen to define functional impairment on the CSDS (14, 25)
The Swedish Child Sheehan Disability Scale-Parent Report (Swedish CSDS-P). The Swedish CSDS-P (parent version) is analogous to the Swedish CSDS (child version) and has three items that assess the level of interference with the parent’s functioning at work and in social and family life: “The symptoms have disrupted your work”, “The symptoms have disrupted your social life” and “The symptoms have disrupted your family life/home responsibilities”. The CSDS-P also contains an additional two items concerning the impact on the child: “The symptoms have disrupted your child’s schooling” and “The symptoms have disrupted your child’s social life”. The score of each item ranges from 0 (“not at all”) to 10 (“very, very much”), resulting in a total scale range of 0–50. In Whiteside’s study (9), the CSDS-P total mean (M) and standard deviation (SD) were M = 22.7 (SD = 11.9) for children with anxiety diagnoses and M = 2.89 (SD = 5.5) for the community controls.
Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (K-SADS-PL). The K-SADS-PL is a semi-structured diagnostic interview that integrates information from children, parents and the clinical judgement of the interviewer (26, 27). The K-SADS-PL consists of a screening interview and eight supplements for 23 diagnostic domains (26). It has excellent IRR and test–retest reliability supporting its ability to generate valid child psychiatric diagnoses (26, 27, 33, 34). In the K-SADS-PL, comorbidity is presented as the number of specific diagnoses. The K-SADS diagnoses are grouped as follows: any depression (Major Depression, Major Depression with Psychotic Features*), any anxiety (Separation Anxiety, Specific Phobia, Social Anxiety Disorder, Panic Disorder, Agoraphobia, Generalized Anxiety, Obsessive-Compulsive Disorder, Post-traumatic Stress Disorder and Acute Stress Disorder) and any ADHD (ADHD combined, predominantly inattentive, predominantly hyperactive/impulsive and unspecified). At the time of the assessment, the only patient with bipolar diagnosis presented a manic episode. Moreover, a general K-SADS-PL symptom summation index was calculated by adding up the obtained scores during the screening interview and supplements, in a way analogous to that used by Sonnby et al. and Olofsdotter et al. in their previous studies (32, 35). The different ranges for symptom summation indexes were as follows: depression (0–114), manic episode (0–52), psychotic disorder (0–103), panic disorder (0–56), agoraphobia (0–6), separation anxiety disorder (0–27), social anxiety disorder (0–9), specific phobia (0–56), generalized anxiety disorder (0–35), obsessive-compulsive disorder (OCD; 0–98), enuresis (0–9), encopresis (0–9), anorexia nervosa (0–12), bulimia nervosa (0–30), ADHD (0–57), oppositional defiant disorder (0–24), conduct disorder (0–45), tic disorders (0–56), autism (0–43), tobacco use disorder (0–14), alcohol use disorders (0–514), drug use disorder (0–53) and post-traumatic stress disorder (0–66). Scores range from 0 to 2 or 0 to 3 for each symptom depending on the item, and the total range is 0–1488.
Furthermore, the total K-SADS function summation index was calculated by adding the obtained scores of summations of function questions of each specific diagnosis. Ranges were as follows: depression (0–6), manic episode (0–10), psychotic disorder (0–8), panic disorder (0–6), separation anxiety disorder (0–6), social anxiety disorder (0–6), specific phobia (0–6), generalized anxiety disorder (0–6), OCD (0–16), ADHD (0–6), oppositional defiant disorder (0–6), conduct disorder (0–6), tic disorders (0–6), autism (0–6) and post-traumatic stress disorder (0–8). Each function question has a score range from 0 to 2, and the total K-SADS function summation index is 0–108.
Strengths and Difficulties Questionnaire (SDQ)–self-report and parent versions. The SDQ is a brief behavioural screening questionnaire (29). There are equivalent versions for adolescents and parents/teachers (28, 29). Goodman developed the SDQ based on the brief Rutter questionnaires and the Child Behaviour Checklist (28, 29, 36, 37). The Swedish translation of the SDQ parent version has been evaluated with the parents of children aged 6–10 years and deemed to be a reliable and valid measure (29, 38). Adolescent self-reports in Nordic populations show similar distributions of SDQ scores, and in a study of adolescent self-reports, the emotional subscale of the SDQ showed usefulness in differentiating depressed from non-depressed cases (39, 40).
The SDQ consists of 25 items grouped on five subscales, each with five items, with each item using a three-point Likert scale (0 = not true, 1 = somewhat true and 2 = certainly true) (28). The subscales produce scores for emotional symptoms, conduct problems, hyperactivity-inattention, peer problems and prosocial behavior. Some items are reverse scored, and prosocial behavior is considered protective.
The chi-square test was used to compare sex and the Mann–Whitney U test was used to compare age between internal drop-outs and participants with complete interviews and forms.
Following Leon et al. (25), the SDS cut-off of 5 was used to define patients with low functioning. Because of the inherent definition of functional impairment related to the determination of any diagnosis, the sensitivity and specificity of the CSDS/CSDS-P for at least one diagnosis were taken into account (42). When at least one diagnosis was confirmed indicating low functioning, the optimal cut-offs for CSDS or CSDS-P were considered if the sensitivities were above 80% and the specificities were not below 50% (43)
Spearman’s rho was used to calculate all correlations, including those between a) the CSDS/CSDS-P and K-SADS-PL symptom summation index, b) the CSDS/CSDS-P and the total K-SADS function summation index, c) the CSDS/CSDS-P and the K-SADS function summation index of each diagnosis, d) the CSDS/CSDS-P and SDQ adolescents/SDQ parents, respectively, and between the CSDS/CSDS-P and the number of diagnoses.
In addition, this was used to calculate the correlations between the total scores of the CSDS and the CSDS-P, between the first items of both scales (“school” and “child’s school”) and between the second items of both scales (“friends” and “child’s friends”).
Spearman’s rho correlations were classified as very high (.90–1), high (.70–.90), moderate (.50–.70), low (.30–.50) and negligible (0–.30) (44). IBM SPSS Statistics for Windows (Armonk, NY, USA), version 24.0, was used for the statistical analyses.
There was no significant age difference between the internal drop-out group and the study group (U = 947, p = .910), and no significant sex difference (χ2 = .036, p = .850). There was no significant age difference between males and females (U = 1202, p = .223). The most frequent diagnoses in the study sample were ADHD (55.1%), MDD (42.1%) and SAD (31%) (Table 1). Diagnoses were grouped into any depression (one patient with bipolar diagnosis, who presented a manic episode at the time of the interview, was not included in depression group), any anxiety, any ADHD or any other diagnoses (Table 1). Seven patients (6.5%) did not fulfil criteria for any diagnosis.
Five (4.6%) of patients were diagnosed with bipolar or psychotic disorders; bipolar depressive episode n=1, schizoaffective disorder manic episode n=1, and unspecified psychotic disorder n=3.
Factor structure and internal consistency.
The CSDS showed one component with an eigenvalue of 2.19, explaining 72.9% of the variance. Items loaded on one component, with a maximum for “school” of .868, for “friends”.855 and a minimum for “home” of .837. For the CSDS-P, the corresponding eigenvalue was 3.09, explaining 61.8% of the variance. Items loaded on one component with a maximum “for parent’s life at home” of .874, for “parent’s work” .861, for “parent’s social life”.784, for “child’s friends”.767 and a minimum for “child’s school” of .619. Internal consistency was Cronbach’s α of .81 for the adolescent scale and .84 for the parent scale.
The CSDS at cut-off 5 and the CSDS-P at cut-off 17 are suggested for the purpose of screening, considering a sensitivity above 80% and a specificity with a minimum of 50%. The CSDS at cut-off 5 showed a sensitivity of 85% (85 patients with low function out of 100 patients with diagnosis) and specificity of 71% (five patients without low function out of seven patients without diagnosis) and the CSDS-P at cut-off 17 showed a sensitivity of 81% (81 patients with low function out of 100 patients with diagnosis) and specificity of 57% (four patients without low function out of seven patients without diagnosis).
CSDS/CSDS-P in relation to the K-SADS symptom summation index. (Not shown in tables). The correlation of the CSDS with the K-SADS symptom summation index was low (rs = .332, p < .001). Furthermore, among the correlations between the CSDS and the symptom summation index for each diagnosis, only the correlation between the adolescent self-report of the CSDS and the summation index for MDD was moderate (rs = .500, p < .001). Other correlations of the CSDS in relation to the summation index for panic disorder (rs = .322, p < .001), the summation index for separation anxiety (rs = .317, p < .317) and the summation index for anorexia (rs = .311, p = .001) were low, and the rest were negligible.
The correlation between the CSDS-P and the K-SADS symptom summation index and the correlations of the CSDS-P with most diagnoses were negligible (rs ≤ .300).
Correlations of the CSDS/CSDS-P and the total score K-SADS function summation index. There were negligible correlations for both scales in relation to the total K-SADS function summation index (rs = .268, p = .006 and rs = .275, p =.005). With respect to correlations between CSDS and the K-SADS function summation index of each diagnosis, the correlations that stood out were manic episode (rs = .752, p = .031), psychotic episode (rs = .741, p = .035), panic disorder (rs = .495 p = .043) and social anxiety (rs = .327, p = .028). Others were negligible or insignificant.
CSDS/CSDS-P in relation to the SDQ adolescent or parent version. Table 2 shows that the correlation between the CSDS total score and the SDQ adolescent subscale score for emotional problems and the correlation of the CSDS-P and the total difficulties score of the SDQ among parents were low. Prosocial sub-scale scores were not included in the correlation because these were not considered a direct measure of the level of difficulties.
Relationship between the total score on CSDS/CSDS-P and the number of diagnoses. The correlations between the CSDS total score and the number of diagnoses were low (r = .338, p < .0001); for parents, the correlations between the CSDS-P total score and the number of diagnoses were negligible (r = .256, p < .008).
Correlations of the CSDS and CSDS-P.
The correlation for the CSDS total score and CSDS-P total score was rs = .375, p <.001. Additionally, the correlations between the CSDS “school” item and the CSDS-P “Child’s school” item and between the CSDS “friends” item, and CSDS-P “Child’s friends” were rs = .354, p <.01 and rs = .375, p < .01 respectively.
In this clinical sample of adolescent psychiatric outpatients aged 12–17 years, the Swedish CSDS and CSDS-P showed high internal consistency, and the factor analyses replicated the previously shown one-factor structure for both scales. The concurrent validity was weak. The factor analysis in the present study found one component for both scales in line with the English, Spanish and Swedish versions of the adult SDS (23, 24, 45). In this study, the ratings for each item and the total score of the CSDS were similar to the results of the validation of the adult SDS Swedish version performed in a young adult psychiatric sample (23).
As noted, Leon et al. (25) suggested a cut-off value of 5 to identify cases with SDS in adults in primary care; Sheehan and Sheehan proposed a cut-off below 5 as a potential remission criterion (14). In this study, CSDS at a cut-off 5 overlapped with more than 80% of the cases with at least one diagnosis. Specificity seemed acceptable, but due to a low number of patients without diagnosis, this finding is uncertain. We recommend the use of a cut-off for CSDS of 5 and for CSDS-P of 17 to identify patients with low functioning related to a psychiatric diagnosis, considering the results of previous studies mentioned above (14, 24), the cut-off and mean values previously described by Whiteside (9) for children with anxiety, and the results obtained for CSDS/CSDS-P sensitivity and specificity for at least one diagnosis in the present study. However, it is important to emphasise that some cases may have low levels of function without having a diagnosis (6) and that the rate of undiagnosed patients was very low due to the characteristics of the population, which decreases the precision of the measurements and hinders a generalization of applicability to other populations.
There were weak general correlations between the CSDS/CSDS-P and the K-SADS symptom summation index, as well as with the SDQ. The only exception was the K-SADS symptom summation for MDD together with the SDQ adolescent total difficulties and emotional problems, which were low but still significant, as opposed to other correlations with SDQ, which were negligible. The correlation with the SDQ emotional subscale indicates increased adolescent awareness of depressive symptoms and is consistent with expectations (46).
Interestingly, the correlation between the total score on the CSDS-P and that on the SDQ parent version was lower than that for the adolescent report, which might be explained by the tendency of adolescents to under-report depressive symptoms to their parents (47, 48).
Notably, high correlations with adolescent reports of dysfunction in the manic episode and with parental reports of dysfunction in adolescents’ psychotic episodes were found when the correlations of the CSDS/CSDS-P with the K-SADS function summation index were separated for each diagnosis. This suggests the ability of the scales to identify differences in functioning in various disorders with severe symptoms despite apparent low overall correlations. However, one should bear in mind that the prevalence of mood disorders with severe impairment in U.S adolescents is 11.2% according to the epidemiological study performed by Merikangas et al. (49), and that in our study only 1.8% presented severe mood disorders, because the most severe cases did not receive outpatient care and therefore were not included in the study. The generalization of these conclusions is limited by the small number of cases in the sample.
Moreover, the relationship between the total score on the CSDS/CSDS-P and the number of diagnoses was weak. If one observes only the correlation coefficients, which are based on the notion of linearity, and the whole range of the compared scales, one might overlook that the correlation could be negligible at one end of the continuum and very high at the other end. If so, a low or mediocre correlation would result when the total correlations are compared.
However, this phenomenon elucidates the problem with statistical measures that are built to estimate the magnitude of association compared with the actual agreement at a certain cut-off (such as the suggested score of 5)(50).
The results highlight a number of issues. First, adolescent self-reports of low functioning levels were in line with expert raters using a standardised diagnostic instrument. Second, correlation tests that use the full range of variation, which is better suited for detecting covariance, might risk revealing the low usefulness of a scale, which should be able to detect disability that meets the impairment criteria for a psychiatric disorder when a certain cut-off has been reached. Therefore, we propose that the CSDS and CSDS-P scales could provide complementary information for the assessment of mental health problems.
The study was performed in a clinical population with multiple diagnoses and most of the participants had comorbid disorders. This limits its generalizability in diagnosing specific populations. On the other hand, our results are not limited to patients with a specific diagnosis (27). An additional limitation is the absence of a comparative community sample to evaluate the psychometric properties of the scales in different settings. Moreover, the small number of undiagnosed patients as well as the small sample size decreased the accuracy of the measurement of sensitivity and specificity.
A further limitation in the design is that no other specific scales for measuring functioning were included. However, the K-SADS, a “gold standard” among diagnostic interviews, was used to identify diagnoses and to have a diagnosis imply a clinical level of dysfunction. Therefore, this assessment was considered to be a relevant reference for the CSDS. In addition, the interview was conducted first, and then the scale was completed, which may have influenced the performance of the scale.
The study compares the CSDS and CSDS-P against the sum of symptoms and number of diagnoses obtained by using the “gold standard” K-SADS-PL interview and the SDQ, an established scale for measuring strengths and difficulties. The CSDS and CSDS-P are self-report and parent-report scales, respectively. They are easy to use and help to identify dysfunction caused by mental health problems, and thereby identify individuals who need subsequent evaluation.
The results indicate that the Swedish versions of the CSDS and CSDS-P have similar psychometric properties to the Swedish, English and Spanish versions of the adult SDS, and the English versions of the CSDS and CSDS-P. Concurrent validity and correlations with symptoms and functional indexes based on the K-SADS were low, probably due to the impact of differences in the individuals’ functional levels.
However, the symptoms of diagnoses with a severe impact on the function level showed higher correlations with the rated function in the CSDS/CSDS-P.
The CSDS/CSDS-P correctly identified more than 80% of cases with at least one diagnosis at the proposed cut-off value of 5 and 17, respectively. They could therefore be useful in adolescent mental health services in a Swedish population.
The authors report no conflicts of interest and are alone responsible for the content and writing of the paper.
The authors are grateful to all patients and parents who participated, and to David Sheehan and Stephen P. Whiteside for permission to adapt the scale for use in adolescents. The study was supported by the Psychiatric Clinic of Västmanland Region, the County Council of Västmanland (LTV-473811, LTV-466871, LTV-398781, LTV-379991, LTV-375811, LTV369201 and LTV-353451) and the Uppsala and Orebro Regional Research Council (RFR-475881 and RFR-376361).