Validation of a Revised Knowledge Assessment Tool for Children with Inflammatory Bowel Disease (IBD-KID2) Open Access

Inflammatory bowel disease (IBD) has been recognized as one of the most important and serious chronic diseases of childhood [1, 2]. IBD is a term collectively used for the diagnoses of Crohn’s disease (CD) and ulcerative colitis (UC) – both are complex, chronic conditions of the gut that may involve multifaceted treatment regimens to achieve the best outcomes. For children with IBD, the acquisition of knowledge concerning their diagnosis and disease management is integral to their treatment adherence and the development of self-management skills [3, 4]. With increasing disease awareness, treatment adherence improves as patients better understand their disease and the consequences of noncompliance [5].

In order to identify gaps or misconceptions in understanding that may adversely affect disease management, it is necessary to assess the knowledge levels of patients with IBD [6]. This may take the form of assessing knowledge of individual treatment regimens [7], as well as knowledge levels of general IBD information. Among the paediatric IBD population, this general knowledge can be assessed using validated knowledge questionnaires and previous work has shown that levels vary, with overall score results ranging from 43 to 66% [8-13]. One of these questionnaires is the IBD Knowledge Inventory Device (IBD-KID) developed by Haaland et al. [10], which was validated for use with children aged 10 years and over and has been used in a number of studies [9, 10, 12, 14]. A participant response pattern analysis, performed using data from two of these cohorts, identified factors that were affecting the validity, reliability, and feasibility of the tool[15]. This prompted the development of a revised version – IBD-KID2 – which was simplified, shortened, and aimed at children aged 8 years and over.

Following the revisions to IBD-KID, it was important to determine whether IBD-KID2 performed as well as the original in providing an overall assessment of disease-specific knowledge and in highlighting deficient areas. IBD-KID2 was validated using a process similar to the original IBD-KID and other knowledge assessment tools [10, 16, 17], whereby it was administered to four groups of participants whose knowledge levels have been empirically established. Reliability was assessed using test-retest assessments and internal consistency.

IBD-KID2 is a fifteen-item knowledge assessment tool comprised of six multiple-choice questions and nine true/false questions (see online supplementary Appendix, available at www.karger.com/doi/10.1159/000506200). Participant responses are scored as one for each correct answer, to a maximum total of fifteen. Knowledge domains included general IBD, treatment, and lifestyle/nutrition.

Testing took place to establish the construct validity, reliability, and feasibility of IBD-KID2 followed by an assessment of the performance of IBD-KID2 compared to the original version, as determined in the previous item-response analysis. Participants were recruited to four groups: children with IBD, children without IBD, medical staff, and administrative staff. These groups have previously been established as having hierarchical levels of IBD knowledge with medical staff scoring highest, followed by IBD patients, administrative/clerical staff, and the control group of children without IBD scoring lowest [10, 12, 16, 17].

All participants recruited were either patients or staff from Christchurch Hospital, New Zealand. Inclusion criteria for the children with IBD was a confirmed diagnosis of CD or UC and over the age of 8 years. For children without IBD, they had to be over the age of 8 years and were excluded if they had been admitted for a gastrointestinal condition or had a close family member with IBD. All medical staff were suitable for inclusion to the study. Administrative staff were excluded if they or a close family member had IBD.

Participants from the medical and administrative staff groups were recruited in person during their usual working day, and the group of children without IBD were recruited from the hospital inpatient wards. All from these participant groups completed the IBD-KID2 in the presence of the researcher. The parent or researcher helped read the questions if required for the group of children without IBD. The group of children with IBD were recruited from two settings; from hospital (outpatient clinic or inpatient ward) or via phone call (initially with their parents) having been identified from a previous study. Those recruited in person completed IBD-KID2 in the presence of the researcher. Parents of children with IBD recruited via phone completed the consent process using a secure, data-compliant online form provider to provide electronic signed consent. An electronic version of IBD-KID2 was then completed using the same online form provider.

For test-retest assessments, the group of children with IBD were contacted after 2 weeks via an e-mail to their parents with a link to the electronic version of IBD-KID2 to be completed. Reminders were e-mailed after 1 week and then 2 weeks if the repeat assessment was not completed.

The layout and instructions of the electronic version of IBD-KID2 were kept identical to the paper version except for the instruction to choose a radio button for the answer selection. Explicit instructions were included in the IBD-KID2 instructions and contact e-mails that answers were not to be looked up or asked of a family member. In order to assess the influence of a change of environment for the repeat test, the responses given to each item (not just whether it was answered correctly/incorrectly) were examined to determine the number of identical responses between the baseline and repeat assessments.

Ethics approval for the research was granted by the Otago University Human Ethics Committee (Health) (Ref: H16/116). Written signatures were obtained for informed consent from all adult participants and from the parents of the paediatric participants. Written signed assent was obtained from paediatric participants completing the paper version of IBD-KID2. The process of obtaining electronic consent was approved by the ethics committee, and electronic signed parental consent, and verbal assent from the child via phone, was obtained for the four participants with IBD completing the study electronically.

Validity was tested by calculating the mean scores for each group – between-group differences were tested using ANOVA and post hoc pairwise analysis compared the IBD group with the others using Tukey’s test. Within the IBD group, the associations between the IBD-KID2 total scores and the variables age, gender, diagnosis, and disease duration were tested using independent sample t tests for categorical variables and linear regression for continuous variables.

Test-retest reliability in the group of children with IBD was examined by comparing means between the baseline and repeat assessments, and reproducibility was tested using the intraclass correlation coefficient (ICC). Test-retest ICCs are considered to be showing good reproducibility if the value is ≥0.70 [18]. Internal consistency was examined with the Kuder-Richardson 20 (KR-20) formula.

The overall cohort data, and data for the group of children with IBD, were examined for patterns of knowledge deficiencies – considered as those items answered correctly by under 50% of the cohort of children with IBD. The difficulty index for each item was assessed as the percentage of respondents who answered each IBD-KID2 item correctly – the ideal range is between 30 and 70% – with lower than 30% signifying that an item may be too hard, and over 70% too easy [19, 20].

A readability comprehension assessment was performed using the advanced proofing analysis tool in Microsoft Word^TM. This was calculated to allow direct comparison with the original version of IBD-KID. The recommended Flesch reading ease score for paediatric patient material is >70.0 [21].

Statistical analysis was performed using SPSS for Windows [22] and graphs produced using GraphPad Prism version 7.04 for Windows [23].

The mean ages of the paediatric groups were: children with IBD (n = 22): 13.6 years (SD 2.5) (range 8–18), children without IBD (n = 20): 11.9 years (SD 2.2) (range 8–15). The gender distribution for groups was: children with IBD: 50% female, children without IBD: 50% female, medical staff: 67% female, admin staff: 93% female. Of the children with IBD, 20 had a diagnosis of CD and 2 UC. Mean duration since disease diagnosis was 40.6 months (SD 34) (range 0–133 months), with 9 having been diagnosed within 2 years of the initial IBD-KID2 assessment. Of the children in the IBD group, 18 were recruited in the hospital and completed the original paper version of IBD-KID2, and 4 were recruited via phone and completed the electronic version.

The group means (SD) for the IBD-KID2 total score (maximum 15) were: children with IBD: 8.5 (±2.3), children without IBD: 3.7 (±2.2), medical staff: 13.5 (±1.3), administration staff: 6.3 (±2.5) (Fig. 1). The overall difference in mean scores between groups was significant (p < 0.001) and group mean comparisons with children with IBD were: children without IBD: mean difference 4.76 (p < 0.005, CI: 3.0–6.5), medical staff: mean difference –5.01 (p < 0.005, CI: –6.9 to –3.1), and administrative staff: mean difference 2.12 (p = 0.022, CI: 0.2–4).

The significance of the independent variables (diagnosis, gender, age, and duration of disease) on the mean score of the group of children with IBD were examined (Table 1). There was no statistically significant association between any independent variable and mean IBD-KID2 score. The mean scores of IBD group participants using the paper or electronic versions of IBD-KID2 were not significantly different at baseline (p = 0.372, CI: –1.5 to 3.8) or follow-up (p = 0.529, CI –2 to 3.7).

IBD-KID2 assessments were repeated in the children with IBD at a mean of 28 (SD 11) days apart. The mean scores at baseline and repeat were not significantly different (Fig. 2). The ICC between the two time points was high at 0.82 (CI: 0.56–0.92). Eighty-one percent of respondents gave identical answers between time 1 and 2 for ≥10 of 15 items.

KR-20 score was high at 0.85. Item-total statistics showed that this score would not be improved by the removal of any specific items.

Areas of good and poor knowledge for the group of children with IBD are represented by those items scoring over and under 50%, respectively (Fig. 3).

The IBD cohort in this study scored higher in the test overall than all previous studies using the original IBD-KID, and had fewer “don’t know” responses. Of the fifteen items in IBD-KID2, five were unchanged from the original version, five were multiple-choice questions that had been simplified or re-written, four had readability improvements, and one was new. The five items that were the same in both tools scored fewer correct answers in this IBD group than those from the item-response analysis data. Nine of the ten items from the original IBD-KID had improvements made to their readability, structure, or were re-written, and all had more correct scores than in their original format.

The difficulty index of IBD-KID2 items overall was acceptable: 73% of items, easy: 20%, and hard: 7% (Fig. 3). The original IBD-KID had items that were acceptable: 52%, easy: 17%, and hard: 30%. IBD-KID2 respondents scored fewer “don’t know” responses overall.

The original IBD-KID was shown to have a Flesch reading ease score of 66.1 [15], which is deemed plain English, easily understood by 13- to 15-year-olds [24]. The readability of IBD-KID2 was calculated at 77.2, appropriate for children aged 10–11 years [24].

Testing of IBD-KID2 was carried out to determine the discriminatory ability between groups with established knowledge levels – the resulting mean scores reflected the expected hierarchical pattern, with significant differences between groups. Repeated tests showed good intraclass correlation, and the overall reliability of IBD-KID2 was high. The readability and structure improvements made during the process of IBD-KID2 development (from the original IBD-KID) improved the individual items and overall scores. The tool was able to highlight areas of poor knowledge to be addressed.

Focusing on the overall score achieved on an assessment tool can be misleading as absolute knowledge levels cannot identify individual areas of poor understanding – in particular those that may negatively affect disease management [6, 25]. The long-term sequelae of active disease may be mitigated by improvements in knowledge, a factor that has been shown to improve adherence and subsequently disease outcomes [26-28]. Common areas of poor knowledge among adults and children with IBD could indicate that some topics are easier to learn, are taught better, or are remembered more easily [9]. The highlighted areas of poor knowledge in this study were seen across the domains of general information, treatment, and nutrition.

• Knowledge of medications is historically low among both adult and paediatric populations with IBD, with reported levels from a number of studies reporting less than 50% correct answers. Such deficiencies may reflect poor recall of drug names or types [6] or be due to the de-emphasis of side effects by clinicians in an effort to bolster patient adherence [4].

• Despite the use of complementary and alternative medicines being relatively high in children with IBD (50–72% of participants) [29, 30], knowledge levels of children are rarely studied, and parents have shown insufficient levels [31].

• Inadequate nutritional intake or malabsorption in children with IBD can have consequences of micronutrient deficiencies, poor growth, and osteoporosis [32, 33]. A number of studies have shown poor understanding of nutrition in the following areas: the impact of IBD on nutrient absorption, the removal of food groups from the diet, and osteoporosis.

• Linear growth impairment in children with IBD is a frequent occurrence (up to 58%) and has substantial consequences for puberty and skeletal maturation [32]. However, there is a universal poor knowledge of this finding across the literature that extended to all groups in this study, of note being that only 47% of medical staff answered this item correctly.

These knowledge deficiencies need to be addressed in order to promote adherence and subsequently improve disease management and outcomes. Targeted nutritional advice and preventative health promotion may optimize food consumption, thus maximizing the micronutrients available from dietary intake.

No independent variable tested in this study had any significance on total score for the children with IBD. This pattern matches other studies using the original IBD-KID [9, 10, 12] with the exception of disease type in the original validation study [10]. The current study, and one performed in Australia [12], had insufficient numbers in the group with UC to make meaningful comparisons between the two diagnoses. It is noteworthy that disease duration does not have an impact on knowledge scores, as in other disease groups such as asthma and diabetes. All identified studies among adults and children with IBD showed knowledge scores independent of disease duration. It has been hypothesised that those recently diagnosed would have less time to understand and learn aspects of their disease, compared to those with a longer disease duration whose knowledge levels would have accumulated [12]. However, it has been concluded that core information obtained shortly after diagnosis provides a similar extent of knowledge as those with long-standing disease [10, 12]. It may be the case that clinicians assume that previous education has been retained and is still appropriate years later [4], but it is clear that patient education needs to be an ongoing process, with reiteration of important issues and continual reinforcement [16, 34].

For those items that had a difficulty index of hard or easy, it is important to consider that there are different types of information that are helpful for people with chronic disease: essential information that may influence self-management and disease outcomes, information that may allay anxieties, and useful background information [6]. It was strongly considered that the items scored as “easy” in this study were essential information that is integral to IBD management and crucial to include for the purpose of detecting any patients with poor understanding. Similarly, the item scored as “hard” is very important to include and promote as a subject that requires understanding.

Increasing knowledge in patients with IBD may not have the same impact as those diseases where patients have control over their own treatment whereby education may improve disease control [16, 25]. However, all patients with IBD need to have a comprehensive knowledge base as it is reasonable to expect that an increased understanding of IBD may lead to fewer complications [16]. The development of self-management skills are also dependent on increasing disease and treatment knowledge, a process crucial to the successful transition of adolescents with chronic illnesses to adult health care providers [4, 16].

This study was limited by the disproportionate distribution of diagnoses among the group of children with IBD, limiting meaningful comparisons between participants with CD and UC and possibly masking any actual difference that may be present. This disparity of numbers, with CD patients being better represented, was universally found among the studies carried out in the paediatric IBD population, which mirrors local [35, 36] and worldwide trends [37].

Further studies will be performed to address this and other limitations, in that parent and child comparisons were not made, and generalisability to the wider population of children with IBD was not established. The ability of IBD-KID2 to discriminate knowledge levels before and after an intervention will also be determined.

The current format of IBD-KID2 (paper or electronic form) does not allow for instant feedback to medical staff in the clinical setting to provide targeted teaching for areas of poor knowledge. The development of an app version of IBD-KID2 is being considered that will improve the potential for IBD-KID2 to be used as a clinical as well as a research tool as users can produce an instant results display.

Comparisons of these results with studies using the original version has allowed for an examination of overall tool performance, and for universal knowledge deficiencies to be identified. This study had a high number of test-retest respondents (21/22) to allow for further reliability testing. The electronic version of IBD-KID2 had no impact on overall scores and may therefore be used in either format in future studies.

This research has established the validity and reliability of IBD-KID2 and shown that changes to item structure and readability have improved the tool’s performance among the target population and ensured that IBD-KID2 was testing participant’s knowledge, not their comprehension of the items on the questionnaire. The group of children with IBD in this study demonstrated patterns of knowledge similar to those seen in previous studies with children and adults with IBD. IBD-KID2 can be used as an outcome measure to assess the impact of education and self-management initiatives directed at children with IBD. It can also help clinicians to identify topics that require directed attention during their discussions with children in their care and with their parents.

All procedures performed in these studies were in accordance with the ethical standards of the institutional and research ethics committees where the research was performed, and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Written informed consent was obtained from the parents or legal guardians of all individual child participants included in the study, as well as written assent from the children themselves. Written informed consent was obtained from all adult participants in this study.

Professors Richard B. Gearry and Andrew S. Day are on the editorial board of Inflammatory Intestinal Diseases.

Angharad Vernon-Roberts received a doctoral scholarship from the University of Otago.

All named authors made substantial contributions to the conception or design of the work, or the acquisition, analysis, or interpretation of data for the work. All named authors revised the work critically for important intellectual content and gave approval for the final draft to be published. All named authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.