Aims: The aim of the current study was to develop and validate a short-form of the internet overuse screening questionnaire (IOS-Qs). Methods: A total of 571 adults were recruited from a representative, stratified, and multistage cluster sample. Among participants, 188 and 383 were used in the development and validation of the IOS-Qs, respectively. Results: Experts’ ratings and Rasch model analyses led to the selection of 8 items from the IOS-Qs; latent-class analysis using these 8 items revealed an estimated prevalence of 8.6% (33 out of 383) of problematic internet over-users. Problematic internet over-users were positively associated with a 1-year prevalence rate of any mental disorder (OR 3.08, p = 0.008), mood disorder (OR 7.11, p = 0.003), and depressive disorder (OR 5.22, p = 0.016). The receiver operating characteristic curves identified an optimal cutoff score of 9.5 for differentiating problematic internet over-users from unproblematic internet users with 94% sensitivity and 94% specificity. Conclusion: The results suggest that the IOS-Qs was valid, and items including social isolation were crucial to the brief distinction of at-risk internet users. Because of its brevity, the questionnaire can be effectively administered as a large-scale survey.

Internet usage has continuously and substantially increased since its advent. The International Telecommunication Union reported that, in 2018, 51.2% of people around the world could use the internet; this percentage has more than tripled since 2005 (15.8%) [1]. Of note, in 2017, 99.5% of individuals in South Korea had access to the internet, which was the highest percentage among OECD members [2]. However, as availability of the internet has increased, so too has internet overuse. According to an internet usage survey conducted by the Ministry of Science, the Information and Communication Technology, and the Korea Internet and Security Agency in 2017, 8.5% of the respondents reported that they use the internet for 35 h or more per week (5 h a day) [3].

Internet overuse is a serious issue, in part because it may be accompanied by one or more problems (e.g., loneliness [4], sleep disorders [5], other psychiatric disorders [6-8]). The lives of those with an addiction to the internet may be very disabled and accompanied by a higher frequency of suicidal-ideation than those not addicted to the internet [9, 10]. In addition, heavy internet usage corresponds to poor academic proficiency [11] and lower academic scores [12]. Since identifying individuals with an addiction to the internet is a crucial step in intervening, we aimed to develop a brief and sensitive screening tool that can be easily administered to a large sample to find at-risk internet over-users.

Various questionnaires related to internet addiction have been developed [13, 14]. Some questionnaires deal with specific types of excessive internet use (e.g., games [15-17], searches [18], social networking services) [19]. To screen for general internet over-users, >20 questionnaires have been developed; however, most have limitations (e.g., improper cut-offs, irrelevant items, lack of diagnostic criteria) [20]. Importantly, Lee et al. [22] noted that: (i) the Internet Addiction Test may not be suitable for those under a certain age (i.e., “youth”) and does not support diagnosis, (ii) the Chinese Internet Addiction Scale [21] is not thoroughly validated, and (iii) the Korean Scale for Internet Addiction includes ambiguous items.

To overcome these limitations, Lee et al. [22] developed the internet overuse screening-questionnaire -(IOS-Q) to screen for at-risk internet over-users (Lee et al. [22]). It is worth noting that the IOS-Q was developed and validated to screen for Korean internet over-users. However, the IOS-Q is composed of 28 items, a potentially excessive number of questions to identify internet over-users, thus making it a lengthy tool that is difficult to broadly administer. Therefore, developing a shorter version of the questionnaire is more applicable in the clinical field due to its shorter administration time. A shorter version would help to screen problematic internet overuse in a brief and concise manner.

In the current study, a short form of the IOS-Qs for adults was developed and validated. Based on the results of an a priori sample size calculation in the G-power software [23], we divided the sample set for the development and the validation of this new tool. The development process was based on expert ratings for the importance and adequacy of each item, and the Rasch model was applied to evaluate the items’ fitness and relative difficulty of the items to select effectual questions. Items were determined through group discussions with experts in psychiatry, psychopathology, and psychology as well as the results from Rasch modeling. In the validation process, psychometric properties were firstly checked via reliability, factor structure, inter-item, and item-total correlations. Concurrent validity was checked via correlations with mental distress [24, 25], sleeping problems [5], low quality in family relationships [26], psychiatric disorders, depressive mood disorders [9, 27], and anxiety disorders [28]. Since there was not a direct question to ask for the quality of participants’ family relationships, a question about perceived importance of family was used instead.

We also performed a latent-class analysis using the items selected in the development process to categorize adults who used the internet problematically and those who did not. The latent-class analysis has been adapted to classify users at risk of problematic Internet overuse and those with no risk [29, 30]. Next, potential psychiatric comorbidities were assessed. Moreover, the optimal cutoff score of the IOS-Qs was suggested from the ROC curve.

Sample

The sample was derived from the “Survey of Mental Disorders in Korea.” The survey was conducted between April 2016 and -November 2016. The target population included all eligible residents 18 years and older who were listed in the 2010 population census of the community registry offices. A total of 5,102 households were selected using a stratified, multi-stage, and cluster sampling method. For the selection of a person per selected household, the “last birthday” method was administered. One person who had the most recent birthday was chosen. Only the day and the month of the birth date, not the birth year, was considered. Of the 5,102 participants randomly selected, 571 were told to answer the -IOS-Qs. Because the “Survey of Mental Disorders in Korea” was mainly for examining the prevalence of mental disorders, questionnaires about internet usage were administered to a relatively small sample (n = 571) due to the time and cost. The flow chart of the studied sample is presented in online supplementary Figure 1 (for all online suppl. material, see www.karger.com/doi/10.1159/000506629).

Among the 571 participants, 188 were randomly selected and assigned to the development of the IOS-Qs, while the remaining 383 were assigned to scale validation. Random assignments were determined using random number generation in Microsoft Excel. The G-power software [23] revealed that 383 participants were required for regression analysis, thus this number of -participants were assigned for the validation process. Demographics for each sample are presented in online supplementary Table 1.

Measures

Internet Overuse Screening Questionnaire

The IOS-Qs was developed and validated to identify at-risk internet users through their self-reported internet usage habits [31]. The questionnaire is composed of 28 items with 4-point Likert scales ranging from 1 (Never), 2 (Sometimes), 3 (Frequently), and 4 (Always) and total scores ranging from 28 to 112. The 28 items selected were identified through expert evaluation and panel discussions.

Mental Distress

Mental distress was measured by an item of “Circle the number (0–10) that best indicates the degree of mental distress you experienced in the past year” with an 11-point Likert scale (0 = not distressed at all, 10 = extremely distressed).

Sleeping Problems

Sleeping problems were measured using the item of measuring sleeping problems in “Depression and dysthymia” of the Diagnostic and Statistical Manual of Mental Disorders criteria (DSM) IV. The question such that “Did you have difficulty sleeping almost every night, for at least 2 weeks, when you were depressed/lost interest/lacked energy in your lifetime? In other words, did you have trouble falling asleep, woke up in the middle of the night, or woke up too early for at least 2 weeks?” was asked to participants with the answer choices of “Yes” or “No”.

Importance of Family

The perceived importance of family was measured by an item of “How important are the items below in your life?: Family” with a 4-point Likert scale (1: not important at all. 4: extremely important).

Diagnostic and Statistical Manual of Mental Disorders-IV

We used the Korean version of the Composite International Diagnostic Interview (K-CIDI) to diagnose psychiatric disorders using the DSM-IV. Cho et al. [32] validated the K-CIDI according to the World Health Organization guidelines [33, 34]. K-CIDI is a fully structured, easily administered diagnostic interview. A 12-month prevalence of DSM-IV psychiatric disorders to evaluate comorbidities of internet overuse was examined.

Experts Ratings

Eight experts in the fields of psychiatry (2 professors, 3 clinical researchers), psychology (1 professor, 1 post-doctoral researcher), and clinical pathology (1 medical laboratory technologist) rated the adequacy and importance of each items via a 5-point Likert scale (1 = not at all, 5 = to a large extent). They were not aware of which items were selected through the Rasch model analysis.

Statistical Analysis

Analyses were conducted using SPSS (version 18.0 for Windows, IL, USA), jMetrik (version 4.1.1 for Mac, created by Patrick Meyer, University of Virginia, www.itemanalysis.com) and the Mplus software (version 6.0 for Windows) [35]. Descriptions for demographics and response frequencies were analyzed using SPSS.

For the development of the IOS-Qs, a Rasch model for polytomous items (i.e., the Rating Scale Model) was applied using the jMetrik [36-38]. The Rating Scale Model is an extension of the 1-parameter Rasch model and involves an assumption that items differ in their level of difficulty but not their slope. Since the RSM assumed parameter invariance, the person and item parameters are independent of each other. In addition, the Rasch model, including RSM, had the benefit of specific objectivity that estimated latent traits and item difficulties are independent from the sample. These characteristics improve interpretability of the results. Item difficulties, fit statistics, and reliabilities were investigated. Infit mean squares and outfit mean squares were checked to determine item fits. The acceptable range of infit mean squares and outfit mean squares ranged from 0.6 to 1.4 [39]. Since the outfit mean square was sensitive to outliers, we focused on the infit mean square [40]. Items for the IOS-Qs were selected based on the group discussion, the item ratings of experts, and the results of statistical analysis (i.e., description of item response, item difficulties, and fit statistics from Rasch modeling).

For the development of the IOS-Qs, we first examined the psychometric properties and concurrent validity of the IOS-Qs. Cronbach’s alpha was calculated. For the Exploratory Factor Analysis, we used principal axis factoring with direct oblimin rotated solution. Appropriate number of factors were determined using the Kaiser criterion (i.e., factors with eigenvalues >1.0) and the screen test. Pearson’s correlation for the continuous variables or -Spearman’s correlation for the nominal variables was calculated to investigate the inter-item correlations, item-total correlations, and concurrent validities.

Distinct classes of internet overuse profiles were identified with the Latent Class Analysis (LCA) [41]. Items selected from the development of the IOS-Qs were used, and models for 2–4 classes were investigated. The goodness-of-fit statistics, Akaike information criteria (AIC), Bayesian information criteria (BIC), sample size adjusted BIC (SSABIC), and bootstrapped likelihood ratio test (BLRT) p values were compared across the 3 models. The lower values in the AIC, BIC, and SSABIC reveal a more parsimonious model. A significant p value in the BLRT indicates that the estimated model shows significant improvement in the model fit when the class number is large (k) rather than small (k-1). In order to determine the class number, we mostly utilized the BLRT p values, BIC [42], and SSABIC [43] value.

The socio-demographic variables were compared between the “unproblematic internet users” group and “problematic internet over-users” group using a t test for the continuous variables or chi-square for the categorical variables. To identify any potential associations with DSM-IV psychiatric disorders, we conducted logistic regressions using SPSS. Adjusted ORs with 95% CIs were calculated based on logistic regressions with the prevalence of psychiatric diagnoses as the main outcome variable and group (i.e., problematic internet over-users, unproblematic internet users) as the main predictor while adjusting for age, gender, education, and marital status. Two-tailed p values below 0.05 were considered statistically significant throughout the analysis.

Development of the Short-Form of the IOS-Qs

Expert Ratings about Adequacy and Importance of the Items

Eight experts in the fields of psychiatry, psychology, and clinical pathology rated the adequacy and importance of each items via a 5-point Likert scale (1 = not at all, 5 = to a large extent). Since the original items of the IOS-Q were developed based on the ratings of adequacy and importance [44], all the items of the IOS-Q showed >3 (neutral) in the 5-point Likert scale except for the importance of items 9 and 11. Thus, these 2 items were not included in the IOS-Qs. In addition, the mean of adequacy and importance of items for the IOS-Qs were higher (4.02 and 4.03, respectively) than those of the nonselected items (3.67 and 3.44 respectively). These item selection procedures involved 2 professors in psychiatry and 1 professor in psychology, participating in a series of group discussions about the contents of the items.

Rasch Modeling and Item Selection

Along with the expert ratings, Rasch modeling was conducted. Response frequencies for the development process (n = 188) of the IOS-Qs are presented in online supplementary Table 2. To develop these IOS-Qs, item difficulties and fit statistics were examined using Rasch analysis (Table 1). Since the scale was developed for screening problematic internet over-users, the items that have high difficulty with good fits were selected. Among the original 28 items, 8 were selected to be validated as the IOS-Qs. Item fits ranged from 0.6 to 1.4, acceptable levels [39]. Item number 25 had the highest difficulty (δ = 2.53), however, not too dissimilar from item 24 (δ = 2.11), a close second; the infit was better in item 24 (infit = 0.97) compared to item 25 (infit = 0.89). In addition, no one answered response 2 (i.e., sometimes) in the validation process (online suppl. Table 2), thus the 25th item was omitted in the IOS-Qs.

Table 1.

Item difficulty and fit statistics for the IOS-Qs (development sample, n = 188)

Item difficulty and fit statistics for the IOS-Qs (development sample, n = 188)
Item difficulty and fit statistics for the IOS-Qs (development sample, n = 188)

Validation of the IOS-Qs

Psychometric Properties of the Short-Form of the IOS-Qs

Using the validation sample (n = 383), psychometric properties of the IOS-Qs were checked. Cronbach’s alpha of the selected 8 items was 0.80. To find the structural validity, Exploratory Factor Analysis was performed. The Kaiser-Meyer-Olkin was 0.790, and the significance of Bartlett’s test of sphericity (χ2 = 1,024.81, p < 0.001) showed that the data were appropriate for the factor analysis. Principle axis factoring with oblimin rotation yielded 2 factor structures, and these latent variables explained 46.20% of the variance. Factor loadings of each item are presented in online supplementary -Table 3. The items reflecting social isolation (family in item 22 and item 23, friends in item 21) and in general (item 24) were highly loaded in factor 1. The coefficient value of item 14 was equivocal, but the item contained social isolation such as school or work, thus it was included in factor 1. Based on the meaning of the items in factor 1, it was named “social isolation.” Factor 2 was named “misuse and dependence.” Two psychiatrists and a psychologist discussed such naming process. Correlations among each item, sub-total scores for the factors, and total score are presented in online supplementary Table 4. Item-total correlations ranged from 0.43 to 0.83. The inter-item correlations were all significant except for the correlation between item 23 and item 18. These 2 items were loaded to different -factors.

Concurrent Validity of the IOS-Qs

The mean of the sum score of IOS-Qs was 8.66 (SD 1.64) and ranged from 8 to 20. Correlations with mental distress, sleeping problems, relationships with family, and psychiatric characteristics were assessed (Table 2). The sum score of IOS-Qs showed positive correlations with mental distress (r = 0.151, p = 0.003), sleeping problems (r = 0.178, p < 0.001), all mental disorders (r = 0.262, p < 0.001), mood disorders (r = 0.223, p < 0.001), depressive disorders (r = 0.201, p = 0.003), and anxiety disorders (r = 0.196, p = 0.035), whereas there was negative correlation with perceived importance of family (r = –0.170, p = 0.001).

Table 2.

Correlation among scores of the short-form of the IOS-Qs and psychological and psychiatric characteristics

Correlation among scores of the short-form of the IOS-Qs and psychological and psychiatric characteristics
Correlation among scores of the short-form of the IOS-Qs and psychological and psychiatric characteristics

Psychiatric characteristics were differentiated depending on factors. Factor 1, named “social isolation,” showed a positive relationship with psychiatric disorders, but factor 2, named “misuse and dependence,” showed positive relationships only with all mental disorders and anxiety disorders.

LCA and the Comorbidity of Psychiatry Disorders Across the Latent Classes

To validate the IOS-Qs developed using the Rasch approach, latent groups were estimated, and the comorbidity of psychiatric disorders depending on group type were assessed. The frequencies of responses in the validation (n = 383) of the IOS-Qs are presented in online supplementary Table 2.

LCA Model Selection

A latent profile analysis with 1–4 classes was performed on the 8 selected items in the validation sample (n = 383). Table 3 reveals the fit indices and test p values for models with 1 through 4 classes. The AIC, BIC, and SSABIC were the smallest in the 2-class solution. Regarding the BLRT, p values were significant in the 2 and more class solution. Solutions with >3 latent classes contained classes representing very small groups of students (i.e., n = 4) and based on the interpretability, the 2-class solution was selected.

Table 3.

Fit indices for the latent-class analysis of the short-form IOS-Qs

Fit indices for the latent-class analysis of the short-form IOS-Qs
Fit indices for the latent-class analysis of the short-form IOS-Qs

Descriptions of the Two Latent Classes: Unproblematic Internet Users and Problematic Internet Over-Users

The largest class, comprising 91.4% (n = 350) of the sample, had low sum scores of the IOS-Qs (sum score = 8.27, SD 0.72), and were thus labeled the unproblematic internet user class. The remaining individuals, comprising 8.6% (n = 33) of the sample, had high sum scores of the IOS-Qs (sum score = 12.76, SD 2.76) and were thus labeled the problematic internet over-user group. An independent t test revealed that the sum score difference between the unproblematic internet user and the problematic internet over-user group was significant (t [32.41] = 9.30, p < 0.001, cohen’s d = 2.23).

The prevalence of problematic internet over-user group by sociodemographic variables is presented in Table 4. The mean age of those in the problematic internet over-user group was younger than those in the unproblematic internet user group (30.06 vs. 37.37 years [t (381) = 2.99, p = 0.003, Cohen’s d = 0.59]). The percentage of females in the problematic internet over-user group was 78.8% (26 out of 33), while the unproblematic internet user group was 45.4% (159 out of 350; χ2 [1] = 13.44, p = 0.003). There was no mean difference in years of education between the 2 groups (t [381] = 0.21, p = 0.835). While the percentage of unmarried individuals in the problematic internet over-user group was higher than expected (63.6% [21 out of 33]), only 50.8% (178 out of 350) were unmarried in the unproblematic internet user group (χ2 [2] = 8.16, p = 0.017).

Table 4.

Prevalence of internet over-users according to sociodemographic variables

Prevalence of internet over-users according to sociodemographic variables
Prevalence of internet over-users according to sociodemographic variables

Comorbid Psychiatric Disorder

The associations between problematic internet overuse and psychiatric disorders are presented in Table 5. Logistic regressions revealed that problematic internet over-users were significantly associated with 1-year prevalence of any DSM-IV disorders (OR 3.08, p = 0.008), mood disorders (OR 7.11, p = 0.003), and depressive disorders (OR 5.22, p = 0.016) compared to unproblematic internet users after adjusting for age, gender, education, and marital status. The significances were the same in the problematic internet over-user group classified using full items (online suppl. Table 3).

Table 5.

Comorbidities of 1 year of mental disorders in problematic internet over-users (n = 33 out of 383)

Comorbidities of 1 year of mental disorders in problematic internet over-users (n = 33 out of 383)
Comorbidities of 1 year of mental disorders in problematic internet over-users (n = 33 out of 383)

Cut-Off Estimation

The ROC curves identified an optimal cutoff score of 9.5 differentiating problematic internet over-users from unproblematic internet users (area under the curve 0.983 p < 0.001) with a sensitivity of 0.94 and a specificity of 0.94.

The current study was designed to develop and validate an IOS-Qs. Results revealed that the IOS-Qs was reliable and valid, thus it would help to enhance efficiency and convenience of screening for problematic internet over-users in research and clinics. Also, patterns of comorbidities in internet over-users were classified using latent-class analysis with the original 28 items and the 8 items, confirming the expected overlap for a valid short-form survey (i.e., the IOS-Qs). Clinicians should pay close attention to adults who answered “sometimes” to >2 items or “frequently” to >1 out of the 8 items on this IOS-Qs.

Eight out of original 28 items were selected based on the item fits and difficulties through Rasch modeling. Since the screening tool was designed to identify problematic internet over-users, items with high difficulties would work more efficiently than the items with low difficulties. It is worth noting that the statistical method (i.e., Rasch modeling) was mainly utilized to select the items, but the content validity of each item should be tested within an expert group. For instance, in the development process of the original IOS-Q, which includes 28 items, 50 members of the addiction society rated the items in terms of adequacy and importance with a 5-point Likert scale [31]. The adequacy and importance of the 8 selected items in the current study were above average in an all item-pool, except the item of “I like using the internet more than spending time with family.” The average for adequacy and importance in all items were 3.94 and 3.90, respectively, while the item (i.e., “I like using the internet more than spending time with family”) showed 3.63 in adequacy and 3.52 in importance in the previous study [31]. Such low adequacy and importance of the item showed the necessity for further examination of content validity.

The content of the items in the IOS-Qs could be a starting point for the early detection of, and intervention for, problematic internet overusers. Results suggest that the neglect of other areas, including social life (e.g., item number 21, 22, 24) and misuse and dependence (e.g., item number 6, 7), are critical symptoms to guide intervention for problematic internet over-users compared to the awareness of self-control (e.g., item number 15, “I don’t think I can reduce my internet usage without help from others”) or internet using behavior (e.g., item number 3, “I stay up all night using internet”). Among the items, misuse and tolerance for the internet were common phenomena for addiction (e.g., alcohol addiction, nicotine addiction); however, the social problem (i.e., item number 21, 22, 24) is a relatively unique characteristic for problematic internet overuse. The factor for “social isolation” is significantly associated with psychological and psychiatric characteristics and these results showed the significance of the social factor in problematic internet overuse. Problematic internet use is related with low social support [45, 46] and dysfunctional social behaviors [47]. Family conflict caused by internet overuse is an indicator of internet addiction [48]. Negative consequences in social life due to internet use may be a critical proxy for identifying problematic internet over-users.

Social factors have been included in many previous instruments for measuring problematic internet use. For instance, perceived social benefits online and perceived social control online are included in the Generalized Problematic Internet Use [49], preference for online social interaction in the modified Generalized Problematic Internet Use [50], and social interactions in the Problematic Internet Use Questionnaire [13]. These assessment instruments are to measure the social characteristics in the internet, but the current study showed the importance of problems in social life. Feeling social connection or social control online may not be harmful, but neglecting real life would be a problem. Therefore, items that assess complications in everyday social life due to the overuse of the internet would increase specificity of the questionnaire. Since we did not confirm the potential causal connection for the neglect of social life and internet overuse, additional studies should be conducted to examine the effects of an educational program(s) that focuses on problems in an individual’s social life in order to prevent or intervene in problematic internet overuse.

To date, there are no clear definitions or diagnostic criterion for internet over-users [51], thus, identifying problematic internet over-users is the most difficult aspect of developing a valid screening questionnaire. To address this problem, we used latent-class analysis to represent distinctive subgroups based on internet use. As suggested by previous studies, this person-centered analysis (i.e., LCA) can be useful to assess the subtypes of problematic internet users [30, 52].

The current study suggests epidemiologic evidence for the prevalence and comorbidities of problematic internet overusers in a nationwide community adult sample. Most previous studies focused on younger populations [11, 53], thus the results of the current study can be helpful in estimating the current status of internet overuse in Korean adults. In 2017, the Ministry of Science, the Information and Communication Technology, and the Korea Internet and Security Agency found that 8.5% of Koreans (i.e., from children older than 3 to elder adults) used the internet for 35 h or more per week (5 h a day) [3]. The percentage was similar with that of the current study, which noted that 33 out of 383 adults (8.6%) were classified as problematic internet over-users. We can assume that the prevalence of problematic internet users in Korean adults was around 8.6%. However, more nationwide studies using this validated questionnaire are needed in order to better estimate the prevalence.

Among the demographic variables, gender and marital status were different between the unproblematic internet users and problematic internet over-users. Gender has been a critical factor associated with internet overuse for internet addiction. In contrast to the result of the current study, previous studies reported that males are represented more with internet addiction than females [54, 55] or no statistical difference [29, 30]. The differences among the results may be caused by the differences in the age range of the samples, the items, or cultural differences. Specifically, most of the previous studies focused on adolescents or early adults (e.g., college students) [20]. Insufficient research on the specific investigation of problematic internet overuse in mid or late adults hinders the formulation of any definitive explanation. However, the frequency graph of problematic internet over-users across gender (online suppl. Fig. 2), in which mid and late adult females have higher frequency than males, supports the explanation that higher prevalence of problematic internet over-users are mid or late adult women rather than men. While mid or late adult women had much higher frequencies than men, women also showed higher frequencies in early age as well. Further studies should be performed to sophisticate the understanding of the prevalence of problematic internet overuse across genders.

In addition, being unmarried in late life is related to maladaptive psychiatric dysfunction such as depression [56]. Since the current study covers participants in late life, marital status may emerge as a significantly different factor. More studies should be performed to explain the reason of the association between problematic internet overuse and marital status.

Internet addiction deserves clinical attention and needs to be better defined to help clarify comorbid psychiatric symptoms. Our results reveal that problematic internet overuse is closely associated with mental disorders, especially depressive disorders. Previous studies consistently suggested that depression is associated with problematic internet overuse behaviors [6, 10, 55, 57]. Although we cannot guarantee the causal relationship with this correlational study, it is hypothetically possible that the depressive symptom is caused by internet overuse, based on the fact that internet communication with people can alleviate depression and loneliness, at least among the socially isolated and moderately depressed populations, who may tend to rely on social technologies to receive social support [58]. In line with the concept that emphasizes the importance of social life in the content of the items, researchers may wish to pay more attention on the social life of adults with problematic internet use.

This study was the first to develop an IOS-Qs through expert discussions and Rasch modeling. It is also the first to evaluate psychometric properties and psychiatric comorbidity of problematic internet overuse in adults older than 18 years of age in a community-based population. Because of the newly designed questionnaire’s brevity, it may be effective for administration in a large-scale survey. Demographic differences and psychiatric comorbidities of problematic internet overuse class indicate internet users with problematic overuse associated with psychosocial and neurobiological factors. Clinicians should consider psychiatric comorbidities when caring for adults overusing the internet.

The authors thank to local investigators, interviewers, and the Korean Ministry of Health and Welfare for their co-operation and assistance.

The review board of Ministry of Health and Welfare, South Korea, approved the study protocol, and all participants gave written informed consent. The research was conducted according to the Helsinki Declaration guidelines.

No competing financial interests exist.

There is no funding to declare.

S.P., J.-P.H., H.J.J., and J.-Y.L. designed study; S.P. and B.C.: analyzed data; S.P. drafted the manuscript; S.C., S.L., J.K., and J.E.P. co-wrote and revised the manuscript; all authors provided critical feedback and helped shape the research, analysis, and manuscript.

1.
Committed to connecting the world [Internet]. 2018. Available from: https://www.itu.int/en/ITU-D/Statistics/Pages/stat/default.aspx.
2.
OECD. OECD Telecommunications and Internet Statistics: ICT Access and Usage by Households and Individuals [Internet]. 2018. Available from: https://data.oecd.org/ict/internet-access.htm.
3.
Agency KI & S. 2017 A Survey on the Conditions of Internet Use. 2017.
4.
Morahan-martin J, Schumacher P. Incidence and correlates of pathological Internet use among college students.
Comput Hum Behav
. 2000;16:13–29.
5.
Choi K, Son H, Park M, Han J, Kim K, Lee B, et al. Internet overuse and excessive daytime sleepiness in adolescents.
Psychiatry Clin Neurosci
. 2009 Aug;63(4):455–62.
6.
Yen JY, Ko CH, Yen CF, Wu HY, Yang MJ. The comorbid psychiatric symptoms of Internet addiction: attention deficit and hyperactivity disorder (ADHD), depression, social phobia, and hostility.
J Adolesc Health
. 2007 Jul;41(1):93–8.
7.
Kuss DJ, Griffiths MD. Online social networking and addiction–a review of the psychological literature.
Int J Environ Res Public Health
. 2011 Sep;8(9):3528–52.
8.
Cao F, Su L. Internet addiction among Chinese adolescents: prevalence and psychological features.
Child Care Health Dev
. 2007 May;33(3):275–81.
9.
Kim K, Ryu E, Chon MY, Yeun EJ, Choi SY, Seo JS, et al. Internet addiction in Korean adolescents and its relation to depression and suicidal ideation: a questionnaire survey.
Int J Nurs Stud
. 2006 Feb;43(2):185–92.
10.
Fu KW, Chan WS, Wong PW, Yip PS. Internet addiction: prevalence, discriminant validity and correlates among adolescents in Hong Kong.
Br J Psychiatry
. 2010 Jun;196(6):486–92.
11.
Stavropoulos V, Alexandraki K, Motti-Stefanidi F. Recognizing internet addiction: prevalence and relationship to academic achievement in adolescents enrolled in urban and rural Greek high schools.
J Adolesc
. 2013 Jun;36(3):565–76.
12.
OECD. How has Internet use changed between 2012 and 2015 ? 2017.
13.
Thatcher A, Goolam S. Development and Psychometric Properties of the Problematic Internet Use Questionnaire.
S Afr J Psychol
. 2005;35(4):793–809.
14.
Bischof-Kastner C, Kuntsche E, Wolstein J. Identifying problematic Internet users: development and validation of the Internet Motive Questionnaire for Adolescents (IMQ-A).
J Med Internet Res
. 2014 Oct;16(10):e230.
15.
Pontes HM, Griffiths MD. Measuring DSM-5 internet gaming disorder : development and validation of a short psychometric scale.
Comput Human Behav
. 2015;45:137–43.
16.
Baysak E, Kaya FD, Dalgar I, Candansayar S. Online Game Addiction in a Sample from Turkey : Development and Validation of the Turkish Version of Game Addiction Scale.
Klinik Psikofarmakol Bülteni
. 2016;26(1):21–31.
17.
Jeromin F, Rief W, Barke A. Validation of the Internet Gaming Disorder Questionnaire in a Sample of Adult German-Speaking Internet Gamers.
Cyberpsychol Behav Soc Netw
. 2016 Jul;19(7):453–9.
18.
Wang Y, Wu L, Zhou H, Xu J, Dong G. Development and Validation of a self-reported Questionnaire for Measuring internet search Dependence.
Front Public Health
. 2016 Dec;4:274.
19.
Dantlgraber M, Wetzel E, Schützenberger P, Stieger S, Reips UD. Simple construct evaluation with latent class analysis: an investigation of Facebook addiction and the development of a short form of the Facebook Addiction Test (F-AT).
Behav Res Methods
. 2016 Sep;48(3):869–79.
20.
Kuss DJ, Griffiths MD, Karila L, Billieux J. Internet addiction: a systematic review of epidemiological research for the last decade.
Curr Pharm Des
. 2014;20(25):4026–52.
21.
Chen S. Development of Chinese Internet Addiction Scale and its psychometric study.
Chin J Psychol
. 2003;45:279–94.
22.
Lee HK, Lee HW, Han JH, Park S, Ju S, Choi K, et al. Development and Validation Study of the Internet Overuse Screening Questionnaire.
Psychiatry Investig
. 2018 Apr;15(4):361–9.
23.
Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences.
Behav Res Methods
. 2007 May;39(2):175–91.
24.
Cotten SR, Goldner M, Hale TM, Drentea P. The importance of type, amount, and timing of internet use for understanding psychological distress.
Soc Sci Q
. 2011;92(1):119–39.
25.
Tsumura H, Kanda H, Sugaya N, Tsuboi S, Fukuda M, Takahashi K. Problematic internet use and its relationship with psychological distress, insomnia, and alcoholism among schoolteachers in Japan.
Cyberpsychol Behav Soc Netw
. 2018;21(12):788–96.
26.
Kabasakal Z. Life satisfaction and family functions as-predictors of problematic Internet use in university students.
Comput Human Behav
. 2015;53:294–304.
27.
te Wildt BT, Putzig I, Zedler M, Ohlmeier MD. [Internet dependency as a symptom of depressive mood disorders].
Psychiatr Prax
. 2007 Sep;34 Suppl 3:S318–22.
28.
Bernardi S, Pallanti S. Internet addiction: a descriptive clinical study focusing on comorbidities and dissociative symptoms.
Compr Psychiatry
. 2009 Nov-Dec;50(6):510–6.
29.
Rumpf HJ, Vermulst AA, Bischof A, Kastirke N, Gürtler D, Bischof G, et al. Occurence of internet addiction in a general population sample: a latent class analysis.
Eur Addict Res
. 2014;20(4):159–66.
30.
Demetrovics Z, Király O, Koronczai B, Griffiths MD, Nagygyörgy K, Elekes Z, et al. Psychometric Properties of the Problematic Internet Use Questionnaire Short-Form (PIUQ-SF-6) in a Nationally Representative Sample of Adolescents.
PLoS One
. 2016 Aug;11(8):e0159409.
31.
Lee HK, Kim JH, Fava M, Mischoulon D, Park JH, Shim EJ, et al. Development and validation study of the Smartphone Overuse Screening Questionnaire.
Psychiatry Res
. 2017 Nov;257(May):352–7.
32.
Cho M, Hahm B, Suh D, Hong J. Development of a Korean version of the composite international diagnostic interview (K-CIDI). J Korean. 2002. [cited 2014 Jan 17]. Available from: http://www.koreamed.org/SearchBasic.php?RID=0055JKNA/2002.41.1.123&DT=1.
33.
World Health Organization. The world health report 2002: reducing risks, promoting healthy life. Geneva; 2002.
34.
World Health Organization. CIDI, Core version 2.1 Trainer’s manual. Geneva; 1997.
35.
Muthén LK, Muthén BO. Mplus user’s guide (6th ed.). Los Angeles; 2007.
36.
Andrich D. Application of a Psychometric Rating Model to Ordered Categories Which Are Scored with Successive Integers.
Appl Psychol Meas
. 1978;2(4):581–94.
37.
Andrich D. A rating formulation for ordered response categories.
Psychometrika
. 1978;43(4):561–73.
38.
Meyer JP. Applied Measurement with jMetrik. New York: Routledge, Taylor & Francis Group; 2014. Available from: .
39.
Right B, Linacre J, Gustafson J, Martin-Lof P. Reasonable mean-square fit values.
Rasch Meas Trans
. 1994;8(3):370.
40.
Linacre J. What do Infit and Outfit, Mean-square and Standardized mean?
Rasch Meas Trans
. 2002;16(2):878.
41.
Hagenaars J, McCutcheon A. Applied Latent Class Analysis. Cambridge, England: Cambridge University Press; 2002. Available from: .
42.
Nylund KL, Asparouhov T, Muthén BO. Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study.
Struct Equ Modeling
. 2007;14(4):535–69.
43.
Henson JM, Reise SP, Kim KH. Detecting mixtures from structural model differences using latent variable mixture modeling: A comparison of relative model fit statistics.
Struct Equ Modeling
. 2007;14(2):202–26.
44.
Lee H, Lee H, Han JH, Park S, Ju S, Choi K, et al. Development and Validation Study of the Internet Overuse Screening Questionnaire.
Korean Neuropsychiatr Assoc
. 2018. pp. 361–9.
45.
Yates TM, Gregor MA, Haviland MG. Child maltreatment, alexithymia, and problematic internet use in young adulthood.
Cyberpsychol Behav Soc Netw
. 2012 Apr;15(4):219–25.
46.
Tsai HF, Cheng SH, Yeh TL, Shih CC, Chen KC, Yang YC, et al. The risk factors of Internet addiction – a survey of university freshmen.
Psychiatry Res
. 2009 May;167(3):294–9.
47.
Whang LS, Lee S, Chang G. Internet over-users’ psychological profiles: a behavior sampling analysis on internet addiction.
Cyberpsychol Behav
. 2003 Apr;6(2):143–50.
48.
Bergmark KH, Bergmark A, Findahl O. Extensive internet involvement–addiction or emerging lifestyle?
Int J Environ Res Public Health
. 2011 Dec;8(12):4488–501.
49.
Caplan SE. Problematic Internet use and psychosocial well-being: development of a theory-based cognitive-behavioral measurement instrument.
Comput Human Behav
. 2002;18(5):553–75.
50.
Caplan SE. Theory and measurement of generalized problematic Internet use: A two-step approach.
Comput Human Behav
. 2010;26(5):1089–97.
51.
Weinstein A, Lejoyeux M. Internet addiction or excessive internet use.
Am J Drug Alcohol Abuse
. 2010 Sep;36(5):277–83.
52.
Koronczai B, Urbán R, Kökönyei G, Paksi B, Papp K, Kun B, et al. Confirmation of the three-factor model of problematic internet use on off-line adolescent and adult samples.
Cyberpsychol Behav Soc Netw
. 2011 Nov;14(11):657–64.
53.
Shaw M, Black DW. Internet addiction: definition, assessment, epidemiology and clinical management.
CNS Drugs
. 2008;22(5):353–65.
54.
Ben Thabet J, Ellouze AS, Ghorbel N, Maalej M, Yaich S, Omri S, et al. Factors associated with internet addiction among Tunisian adolescents.
Encephale
. 2019 Dec;45(6):474–81.
55.
Dalbudak E, Evren C, Aldemir S, Coskun KS, Ugurlu H, Yildirim FG. Relationship of internet addiction severity with depression, anxiety, and alexithymia, temperament and character in university students.
Cyberpsychol Behav Soc Netw
. 2013 Apr;16(4):272–8.
56.
Yan XY, Huang SM, Huang CQ, Wu WH, Qin Y. Marital status and risk for late life depression: a meta-analysis of the published literature.
J Int Med Res
. 2011;39(4):1142–54.
57.
Bozoglan B, Demirer V, Sahin I. Loneliness, self-esteem, and life satisfaction as predictors of Internet addiction: a cross-sectional study among Turkish university students.
Scand J Psychol
. 2013 Aug;54(4):313–9.
58.
Shaw LH, Gant LM. In defense of the internet: the relationship between Internet communication and depression, loneliness, self-esteem, and perceived social support.
Cyberpsychol Behav
. 2002 Apr;5(2):157–71.
Open Access License / Drug Dosage / Disclaimer
This article is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND). Usage and distribution for commercial purposes as well as any distribution of modified material requires written permission. Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug. Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.