Non-Participation in a Randomized Controlled Trial: The Effect on Clinical and Non-Clinical Variables

Studies reporting clinical and non-clinical parameters of participants and non-participants of clinical trials are scarce. A recent publication showed that non-participants of a study on a caries prevention programme had lower scores of fluorosis and higher caries levels than participants, but in the permanent dentition only [Splieth et al., 2005]. The authors suggested that these findings may be caused by a lower compliance of non-participating children with preventive measures and that they may be in greater need of caries prevention measures than responders of such a study. Furthermore, the authors concluded that when the investigated preventive interventions were transferred to a broader group of children it is very likely that their efficiency will be significantly reduced as compared to the experimental results because the proportion of non-compliant people in the total eligible group would be significantly greater than in the experimental group alone. Davies et al. [2007] concluded that as a result of non-participation only few benefits of a health intervention found in a deprived study population would be discernible at a population level. In their study, the non-participant children had higher dmft levels than the participating children. So non-participation bias may seriously affect the value of clinical trials and it generally is assumed that non-participation bias favours the benefits of preventive interventions and thus may alter the external validity of such interventions. Strategies used for adjustment of non-participation bias have recently been described and it was concluded that they may result in some degree of bias reduction [López et al., 2008]. The simplest way is to use the method of inverse probability weights [Rosenbaum and Rubin, 1984; Greenland, 2008; López et al., 2008]. In this method, the participants are weighted by the inverse of the probability of (non-)participation. The probability of non-participation is based on the distributions of variables in eligible and non-participating children. Likewise, the probability of participation can be determined based on the distribution of variables in eligible and participating children. A considerable amount of parents chose not to let their child participate in this trial. They were, however, willing to subject their child to the clinical baseline measurement and fill out the accompanying questionnaire themselves. This gave the opportunity to assess to what extent eligible, participating and non-participating children differ on both clinical and non-clinical variables. To determine the influence of participation bias in a randomized controlled trial (RCT), the (non-)participation probability ratios are calculated in this sample of participating and non-participating children.

The aim of this study was to identify and quantify clinical and non-clinical differences between eligible, participating and non-participating children of an RCT on caries-preventive strategies in young children and to establish whether the non-participant bias will have a considerable impact on the external validity of this RCT.

From October 2006 till October 2008 parents of all children aged 6.0 years ± 3 months (n = 346) in three dental clinics in three of the larger cities in the Netherlands (’s Hertogenbosch, The Hague and Utrecht) were approached to allow their child to participate in an RCT on the effectiveness of strategies to prevent caries. No special selection criteria were applied in these practices other than the willingness of the staff to participate and there is no reason to assume that the population of patients would differ from other local practices. The practices indicated to have affinity with paediatric dentistry, were located in an area where large families live, have dentists with paid employment (instead of fee for service) and accept all patients with no restrictions. Children were regular visitors of the practice (on average twice a year for check-ups). Data of a national survey show that the frequency of regular dental visits in the Netherlands is high: 99% of all 5-year-old participating children and 91% of the non-participating children in that study were reported to visit the dental practice at least once a year [Poorterman and Schuller, 2006]. So despite the fact that the sample in this study contains only a modest number of 6-year-olds of the total population in these cities, no considerable non-participation bias could be expected because of this.

Of this group 286 (83%) parents consented to participate and agreed to (i) let their child be randomly assigned to an experimental or control group, (ii) let the oral health status of their child be recorded during a period of 6 years by means of clinical check-ups, (iii) let the clinical records and bitewing radiographs taken by their dentist be used for this study, and (iv) fill out 3 times a set of questionnaires scoring demographic and socioeconomic data, knowledge on dental issues and attitudinal and motivational related questions during the length of the trial.

Sixty parents (17%) preferred not to let their child participate in this RCT. Reasons for this refusal were recorded and parents were asked whether they would consent if participation was limited to just one visit. Of these parents, 56 consented to participate on that condition. The remaining 4 persisted in their refusal and were not included.

Of the participants (n = 286) and non-participants (n = 56) oral hygiene was scored applying the simplified Oral Hygiene Index [Greene and Vermillion, 1964]. Scores of this index vary from 0 (surface not covered with plaque) to 3 (incisal/occlusal one third of the surface covered with plaque). No disclosing agent was used but the surfaces were gently explored using a blunt probe to detect dental plaque. Dental health was assessed using dmfs/DMFS scores in the primary and permanent dentition, respectively. Caries was scored at the dentine threshold (d3), after drying with compressed air, using mirror, light and a blunt probe by a calibrated, experienced dentist. Thirty-five of 342 children (10.2%) were re-examined by a second examiner. Inter-examiner agreement was high (ĸ = 0.93) for caries and satisfactory for plaque (ĸ = 0.67).

Parents/caregivers were asked to complete a questionnaire, scoring (i) ethnic background, (ii) education level of both parents/caregivers, (iii) their and their children’s dietary and oral health habits, (iv) knowledge on several dental health topics (oral hygiene, caries, fluoride, snacking, and prevention), (v) health insurance matters and (vi) willingness to invest. Maternal education level was used as a proxy for socioeconomic status (SES) [Desai and Alva, 1998]. Willingness to pay was assessed by asking parents to indicate the maximum price they were willing to pay to keep there child’s teeth healthy in terms of (a) money (euros/month), (b) time (minutes brushing/day) and (c) effort (times visiting dental office/year). The rationale behind these questions is the presumed relation between these factors and caries experience [Sbraraini and Evans, 2008; Ayo-Yusuf et al., 2009].

Differences in distribution of sample characteristics of both parents and children between eligible, participants and non-participants were tested with χ²-tests (non-parametric data) and independent samples t tests (parametric data). To quantify changes in the distribution of variables between eligible and participating or non-participating children the determinant-specific participation and non-participation probability ratios were calculated. The determinant-specific participation or non-participation probability is the ratio of the participation probability in the participating or non-participating children divided by the participation probability in the eligible children. This method has been applied before by López et al. [2008] as a first step in their study on non-participation bias in a case-control study of periodontitis. Determinants with a participation or non-participation probability <1 are less likely to be present in the participant or non-participant group, respectively, compared to the eligible population. For example, when 35.7% of the eligible children had low SES and 32.9% of the participating children, the participation probability ratio yields 32.9/35.7 = 0.92.

Since caries is a disease with a skewed distribution, descriptive characteristics (ds, ms, fs, dmfs, care index) were tested by a non-parametric technique (Mann-Whitney) to analyse possible differences between participants and non-participants. To see whether possible differences between participants and non-participants affect the mean score from the eligible group, a one-sample t test was used (since no non-parametric alternative is available: because the participants are part of the total eligible group) to test whether the participants’ mean score is different from the mean score from the eligible group (test value). The significance level was set at alpha = 0.05. To analyse the possible influence of covariates on the clinical outcomes, an exploratory analysis was performed using ANCOVA.

Reasons given for non-participation were mostly lack of time (46%), inconvenience for the child (18%), lack of interest (17%) or ‘my child is no guinea pig’ (7%). The remaining 12% did not provide a reason. Parental characteristics of eligible, participating and non-participating children are presented in table 1 together with calculated determinant-specific participation and non-participation probability ratios, as mentioned in the ‘Subjects and Methods’ section. Concerning non-participants, an overrepresentation of male accompanying parents, immigrants and parents with low SES was found with non-participation probabilities of 1.35, 1.68 and 1.40, respectively. Furthermore parents in the non-participation group are less likely to eat three regular meals (non-participation probability: 0.88), to brush more than once a day (non-participation probability: 0.93), to use interdental cleaning supplies (non-participation probability: 0.80), but were more likely to use commercial mouthrinses instead (non-participation probability: 1.4). A lower level of knowledge was found in parents of non-participating children as well as a lower self-reported willingness to pay money, time and effort compared to participating children as well as the total eligible group. Comparison between parents of participating and eligible children, however, did not show statistically significant differences (table 1).

Characteristics of eligible, participating children and non-participating children are presented in table 2 together with calculated determinant-specific participation and non-participation probability ratios. The mean reported number of between-meal snacks did not differ between the three groups. Children of the non-participation group were more likely to be held responsible for brushing their own teeth than children in the participation group (non-participation probability: 1.2, participation probability: 0.96). The simplified Oral Hygiene Index was found to be higher in the non-participating children compared to the eligible group (1.25 vs. 0.95; t = –2.91; p = 0.006). The number of decayed surfaces was higher in non-participating children compared to the eligible group (2.16 vs. 1.69), however, not statistically significant, and dmfs scores likewise (4.43 vs. 5.93). Furthermore children of the non-participation group were found to have less extracted surfaces and less filled surfaces than non-participating children (Z = –2.05, p = 0.05; Z = –1.78, p = 0.08, respectively). Besides that, children of the non-participation group had lower scores on their care index (fs/ds + fs; 0.37 vs. 0.55: Z = –2.68, p = 0.05). When the clinical variables of the participating children were compared with all eligible children, differences were not statistically significant.

The non-participating children were more likely to have lower SES. They were also found to have less dmf surfaces. Because SES is frequently negatively correlated to dmfs [Gillchrist et al., 2001; Thomson and Mackay, 2004] this was unexpected. To correct for SES, an exploratory analysis using ANCOVA was performed. Although we are aware of the fact that the data are not fully suitable (given the skewness and kurtosis of the distribution), we consider the results to be informative: it seemed to polarize the initial results: differences between participants’ and non-participants’ dmfs scores increased: participants’ dmfs corrected for SES: 6.73 (±10.07), non-participants’ dmfs corrected for SES: 3.08 (±5.69); F = 6.50 p = 0.01.

Based on the findings of the present study, the assumption that non-participating children are unfavourable in all caries-related variables beforehand cannot be held. Using the calculated participation probability ratios it can easily be identified where the major differences can be expected. Despite less favourable socioeconomic and dietary circumstances, the non-participating children did not appear to have more decayed, missing and filled surfaces than their participating peers or all eligible children. The group did show slightly more untreated carious surfaces but less extracted and filled surfaces, leaving a more positive dmfs than what was to be expected from earlier studies on non-participation in a caries prevention programme [Splieth et al., 2005; Davies et al., 2007]. Our results challenge the conclusions of studies reporting that non-participation will bias the results of a study on preventive programmes in favour of the intervention because, firstly, the non-participating children had lower dmfs and secondly, there were no statistically significant differences between participants and eligible children, even though the groups contained a fair number of children. This suggests that the external validity of the current RCT is not much affected, despite differences between the participants and non-participants, seen from the participation probability point of view. The largest differences were found in SES. Correcting for this covariate surprisingly polarized the initial results. Other factors than SES therefore must contribute to explain differences in dmfs levels in the groups of children. One possible explanation is that non-participating children visit the dentist less regularly. The fact that the non-participating children tend to have more untreated carious surfaces can support this suggestion: it is not unlikely that less dental visits will result in fewer surfaces treated (by restoration or extraction). Another explanation for lower dmfs scores in our non-participant group may be related to more favourable parental health attitudes and health behaviour than in the participant group. Parental attitudes were dealt with in various recent studies [Tickle et al., 2003; Skeie et al., 2006; Vermaire et al., 2010]. The rationale behind this is that parents who are aware of their own ‘power to prevent’ may act differently than their counterparts who consider caries as an inevitable, hereditary disease or parents who are mainly focused on esthetics in their child or those who are too preoccupied with other things.

Of course every possible way to minimize the effect of non-participation in clinical trials is welcomed. We therefore encourage further use of the proposed method of inversed probability weighting [López et al., 2008] to correct for possible overrepresentation of important variables. However, we also expect the impact of these corrections on the external validity to be modest in this RCT, considering the found participation probability ratios.

The first aim of this study was to identify and quantify differences between eligible, participants and non-participants of an RCT on caries-preventive strategies in young children. Differences in socioeconomic, behavioural and clinical variables were found between non-participants and participants, but not between participants and the eligible group. The second aim was to establish whether the non-participants bias would have a considerable impact on the external validity of an RCT. Seen from the participation probability point of view in this study (instead of the non-participation point of view), the impact was considered quite modest. Therefore, it is suggested that the external validity of an RCT on caries-preventive strategies is not necessarily affected by non-participation bias.