The Agreement between Consumer-Driven Self-Assessment of Psoriasis Severity and Physician-Assessed Severity Based on Patient-Taken Photographs Is Weak: A Cross-Sectional Study

Psoriasis is a common, chronic inflammatory skin disease with a strong genetic predisposition. The worldwide prevalence is around 2%, with a lower prevalence in some Asian and African populations and up to 10% in Northern European and Scandinavian populations [1, 2]. The dermatological manifestations of psoriasis can vary. Psoriasis vulgaris/plaque psoriasis (henceforth “psoriasis”) is the most prevalent type, covering about 90% of psoriasis cases. The disease typically presents as sharply demarcated, erythematous, pruritic plaques covered by silvery scales, commonly located on the trunk, the extensor surfaces of the limbs, genitals, nails, and the scalp [2].

Rapid and extensive digital advancements generate and allow access to huge amounts of real-world data (RWD). RWD are data collected or generated from a variety of sources such as electronic medical records, medical claims, billing data, data from product registries, patient-generated data from home, and data gathered from mobile devices, social media, from websites or by life science industry [3]. RWD are data related to patient health status or the delivery of healthcare. The Association of the British Pharmaceutical Industry defined RWD as “data obtained by any non-interventional methodology that describes what is happening in normal clinical practice” [4]. A reputable nonprofit institution, RAND Europe, that helps improve policy and decision-making through research and analysis refers to RWD as an “umbrella” term standing for different types of data related to healthcare that is not collected in context of conventional randomized clinical trials [5]. The use of RWD to inform health-related decisions is defined as real-world evidence (RWE).

With this development, the healthcare professions and patients themselves increasingly value patient autonomy and the responsibility given to the patients. The evidence so far on digital patient and physician agreement of psoriasis severity comes from real patients in dermatological care or from trials [6]. To date, very little is known about consumer-driven evidence, yet with expanding technology and more consumer-driven data, the utility of these data warrant exploration. The objective of this study was to investigate the agreement between patient-assessed psoriasis severity collected from consumer-driven RWD and physician-assessed severity based on smartphone photographs taken by the patient.

NØIE is a Danish skincare company developing and selling personalized dermatologically tested skincare products for target segments such as psoriasis, acne, eczema, and dry skin. NØIE uses data and advanced algorithms to customize skincare based on in-depth phenotyping and direct feedback loops from more than 80,000 patients living with a skin condition. They have a large database with completed questionnaires on patients’ skin condition, self-assessment, lifestyle, and smartphone photographs taken by the patients of affected skin lesions.

Inclusion criteria for the present study were a customer ID in the NØIE database from 2009 to 2022 with a smartphone photograph of psoriasis vulgaris on the body and a corresponding completed questionnaire. Exclusion criteria were customer IDs with only face photographs and photographs without a completed questionnaire.

The patient-taken smartphone photographs were evaluated by a physician with experience in photographic assessment of skin diseases. The assessment was done using a secure browser-based and purpose-built dashboard on a laptop. To assess the severity of psoriasis on photographs, intensity items of the Psoriasis Area and Severity Index (PASI) were used. The intensity items comprised erythema, induration, and scaling assessed on a five-point scale (none [0], mild [1], moderate [2], severe [3], and very severe [4]) [7]. Further, the Physician’s Global Assessment (PGA) was used ranging from clear (0), almost clear (1), mild (2), moderate (3), and severe (4) [8]. Self-assessment done by the patient of erythema, induration, and scaling was done on a scale from 0 to 10 and was converted to a predefined 0–4 scale (0 converted to 0; 1–3 to 1; 4–6 to 2; 7–8 to 3; and 9–10 to 4).

Pearson correlation and intraclass correlation coefficients (ICC) with 95% confidence intervals (CIs) were calculated to evaluate the agreement between assessments made by the physician based on photographs and self-assessments done by the patient. The ICC agreements were based on a two-way random-effects model and average measure [9]. An ICC >0.90, 0.75–0.90, 0.50–0.75, and <0.50 are generally agreed to indicate excellent, good, moderate, and poor agreement, respectively [9]. Statistical analyses were performed using Python version 3.7.12, pandas library version 1.3.5 was used for data handling, numpy library version 1.21.6 was used for Pearson correlation calculations, pingouin library version 0.5.3 was used for ICC calculation, and seaborn library version 0.12.1 for graphing.

In total, 615 customer IDs qualified for inclusion from 2009 to 2022. Of the included customer IDs, 92 were excluded for only uploading a photograph of the face and not the body; further, 157 were excluded as the uploaded photographs did not contain psoriasis. Of the resulting 366 customer IDs, 172 were excluded based on the quality of the photographs (24 photographs had more than one reason resulting in bad photograph quality); 58 were out of focus, 35 were blurred, 10 had bad lightning, and 94 did not have the lesion in the center of the photograph. Further, seven were excluded for other reasons such as photographs of only nails. In total, 187 remaining customer IDs were included, of which 63% were women, and the mean age was 38 years. Around 66% reported to have chronic psoriasis with visual plaques all the time, whereas 4% had flares 1–2 times per year, 9% had flares 3–5 times per year, 7% 6–8 times per year, and 4% had flares 9–12 times per year. Four of the 187 psoriasis patients had photographed the head, 51 had photographed the arms, 26 the trunk, 103 the legs, and 3 had photographed the back (Table 1).

The ICC correlation between self-assessment and physicians’ assessment for all psoriasis patients was 0.23 (95% CI: 0.0–0.92). The ICC between self-assessment and physicians’ assessment based on photographs was higher for chronic patients (0.34, 95% CI: 0.0–0.95) than for non-chronic patients (0.09, 95% CI: −0.01 to 0.82). The agreement was weaker for induration in patients with chronic disease according to Pearson correlation coefficient (0.15, 95% CI: −0.03 to 0.317, p = 0.09) than erythema (0.24, 95% CI: 0.07–0.40, p < 0.01) and scaling (0.23, 95% CI: 0.06–0.39, p < 0.01). Likewise, the ICC correlation was better for men (0.53, 95% CI: −0.02 to 0.98) than for women (0.12, 95% CI: −0.01 to 0.84). Further, men were best to assess induration (Pearson correlation coefficient = 0.34, 95% CI: 0.11–0.43, p < 0.01). The agreement for patients aged 40–50 years ICC 0.32 (95% CI: 0.0–0.95) was better than the agreement for patients in the age group 20–30 years ICC 0.23 (95% CI: −0.01 to 0.93), and 30–40 years ICC 0.22 (95% CI: 0.0–0.91) (Table 2).

In general, self-assessment scores (Fig. 1) were higher than physicians’ assessment scores for all groups, and scaling was closest to the physicians’ assessment, while erythema and induration were most difficult to assess with greater distance between the physicians’ and patients’ assessment. Further, men’s assessments were closest to the physicians’ assessment, and patients with chronic disease were better to assess than patients with flares (Fig. 2). The Pearson correlation coefficient was also calculated without converting the 10-point scale to 0–4 scale with no significant difference in the results meaning that the conversation did not distort the data or results.

The ICC agreement between the average score of self-assessed erythema, induration, and scaling and physician-assessed PGA was 0.22 (95% CI: 0.08–0.35). The Pearson correlation coefficient was 0.23 (95% CI: 0.10–0.37, p < 0.001).

In this study, 615 customer IDs were qualified for inclusion from a Danish skincare company’s library of RWD from 2009 to 2022. Every fourth patient did not upload a photograph of a skin lesion with psoriasis vulgaris, and around 30% of the patients uploaded a photograph with bad quality, leaving 187 for further analysis. The study showed weak agreement between psoriasis severity assessed by the patient and assessed by the physician based on smartphone photographs obtained by the patient, rising concerns on quality of consumer-driven RWD pertaining to psoriasis. The agreement was better for psoriasis patients with chronic disease than flares, and best for men. Patients tend to score higher than the physicians, and scaling was the easiest to score.

Self-assessment done by patients is important not only in clinical trials but also in routine clinical practice, it allows continuously monitoring and detection of flares ahead of time compared to routine clinical scheduled visits or clinical site visits in trials. Self-assessment also gives patient ownership of disease management [10]. It is known that PASI scores can vary substantially between experienced and inexperienced physicians, raising concerns for inter-rater reliability [11]. Hence, a weaker agreement between physicians and patients in the present study was expected. In a study by Feldman et al. [12], they found a significant correlation between the Self-Assessment Psoriasis Area Severity Index (SAPASI) (a self-assessment tool used in clinical trials and research studies for patient-reported psoriasis severity) and PASI for body surface area (r = 0.62–0.75), erythema (r = 0.39), induration (r = 0.24), and scaling (r = 0.38). Another study found that patients generally tend to overrate their disease severity compared to PASI [13]. In the present study, we also found the weakest correlation for induration, as induration may be difficult to assess. Further, the agreement fluctuated between weak and moderate as seen in the study by Feldman et al. [12]. However, there is a difference in self-assessment in clinical trials and routine clinical visits compared to RWD. In trials, patients are carefully educated in using the self-assessment tools and often also tested in using these tools. In clinical practice, patients are educated as well by the physicians as the responsibility of severity assessment and disease management is shifted from the physicians to the patient. The shift is only possible if patients are comfortable in using the tools and taking the responsibilities. It is in the physician’s interest that the patient is comfortable using the tools and uses the tools to assess the skin lesions as accurately as possible. This contrasts RWD data where data often are collected without any patient education. Even though RWD opens new possibilities for providing clinical evidence faster, there have been some concerns regarding the quality of RWD including incomplete data, vulnerability to misclassification or systematic omissions, and information bias [5]. The life science industry is interested in using RWD as the development and approval of drugs, treatments, and therapies are subject to numerous regulatory requirements; hence, it is risky, time-consuming, and costly. Therefore, the industry is currently pioneering the integration of RWD into their experimental and regulatory pipelines. Establishing awareness and insight into the challenges and risks regarding the use of RWD is essential before taking full advantage of the RWD potential [5]. Currently, there is no consensus or guideline determining which qualities RWD must meet to be acceptable for decision-making within regulatory or routine clinical decision support [14]. Even though RWE can provide information on areas like natural history and course of disease, it should be interpreted with high caution. In our study, 25% from the original pool of patients claiming to have psoriasis did not upload a photograph of a skin lesion consistent with psoriasis. Further, around 30% of the patients uploaded a photograph with bad quality so that severity of the lesion could not be assessed. Further, the correlation between the physician and self-assessment done by the patient was also categorized as weak with Pearson correlation coefficient between 0 and 0.3. This is therefore a very important study, highlighting that RWD should be interpreted carefully. RWE can be good in tracking trends and encourage to plan and execute well-designed studies, but it should not be the only reliable source for decisions-making. Machine learning tools or artificial intelligence tools verifying the quality of the photograph or making sure the photographs are of lesional skin area would improve the quality of consumer-driven RWD. Until the tools are ready, this can also be done manually by checking all the photographs frequently and asking the patients to retake the photograph with adequate quality. This will improve the quality and convenience of consumer-driven RWD.

Photographs were used to assess the severity of psoriasis in this study instead of clinical assessment. The use of photographs in assessment of psoriasis severity has been validated in study settings [13] and is considered to be as accurate as clinical assessment. The ICC between clinical face-to-face PASI and PASI assessed by the physician based on smartphone photographs taken by the patients was 0.78 (95% CI: 0.55–0.90) [6]. The rapid development in technology now allows patients to upload photographs in very good quality both in trials and clinical settings. In trials or studies where continuous severity assessment is necessary, submission of lesional photographs should be considered instead of self-assessment as self-assessment in general has weak correlation with physicians’ assessment. Studies exploring if remote assessment based on photographs are more accurate than self-assessment are warranted, particularly studies generating consumer-driven evidence.

Some important limitations need to be mentioned. The study participants were mainly young individuals not representing the standard pool of psoriasis patients. These patients were most probably representing type 1 psoriasis, which has a positive family history, starts before age 40, and is associated with HLA-Cw6, while type 2 psoriasis does not show a family history, presents after age 40, and is not associated with HLA-Cw6 [15]. Further, there was also an overrepresentation of women, which is very common in online studies [16, 17]. The conversion of the self-assessment scale from a 0–10 scale to a 0–4 scale could introduce some inaccuracies. Though we did not find any significant distortion due to this conversion, it should be mentioned as a limitation. Further, the study would also have benefited from severity assessments done by more than one physician.

This study based on consumer-driven RWD from a Danish skincare company with patient-taken smartphone photographs and self-assessment of severity showed a weak agreement between self-assessed psoriasis severity by the patient and photographically assessed severity done by a physician. The agreement was better for psoriasis patients with chronic disease than flares, and men compared to women. These findings are important as they highlight that RWD should be interpreted with caution and also emphasize the need for photographs of lesion in remote settings. Future studies should investigate if better patient education in self-assessment and assessment tools can improve the agreement between physicians’ assessment and self-assessment of psoriasis severity.

An ethical approval was not required from the Danish Regional Committee on Health Research Ethics. The study complied with all applicable General Data Protection Regulation (GDPR) rules.

Z.A. and A.A.-M. have no conflicts of interest with regard to this paper. C.R. and B.B. are employed by NØIE. Unrelated to the present manuscript, Simon Francis Thomsen has been a speaker and/or served on advisory boards for AbbVie, Almirall, Eli Lilly, Janssen, LEO Pharma, Novartis, Pfizer, Sanofi, Symphogen, and UCB and has received research support from AbbVie, Janssen, LEO Pharma, Novartis, Sanofi, and UCB. Unrelated to the present manuscript, Prof. Egeberg has received research funding from Almirall, Pfizer, Eli Lilly, Novartis, Bristol-Myers Squibb, AbbVie, Janssen Pharmaceuticals, Boehringer Ingelheim, the Danish National Psoriasis Foundation, the Simon Spies Foundation, and the Kgl Hofbundtmager Aage Bang Foundation and honoraria as consultant and/or speaker from Amgen, AbbVie, Almirall, Leo Pharma, Zuellig Pharma Ltd., Galápagos NV, Sun Pharmaceuticals, Samsung Bioepis Co., Ltd., Pfizer, Eli Lilly and Company, Novartis, Union Therapeutics, Galderma, Dermavant, UCB, Mylan, Bristol-Myers Squibb, McNeil Consumer Healthcare, Horizon Therapeutics, Boehringer Ingelheim, and Janssen Pharmaceuticals.

This study was funded by NØIE.

Z.A.: conceptualization, methodology, planning analysis, writing – original draft, and project administration; A.A.-M. and A.E.: writing – review and editing; B.B.: conducting analysis and writing – review and editing; C.R.: conceptualization and writing – review and editing; and S.F.T.: writing – review and editing and supervision.

Data are not publicly available due to ethical reasons. Further inquiries can be directed to the corresponding author.