International Dermatology Outcome Measures (IDEOM): Report from the 2020 Annual Meeting Open Access

The International Dermatology Outcome Measures (IDEOM) initiative is a non-profit organization founded in 2013 that aims to develop evidence-based outcome measurements to evaluate the impact of treatments for patients with dermatological disease [1]. The need for such an initiative originated from a lack of evidence-based outcome measurements in dermatology. In order to provide appropriate treatment recommendations and evaluate patient progress, healthcare providers and researchers need high-quality, consensus-based outcome measurements. Furthermore, healthcare payers need widely accepted outcome measurements to determine appropriate payment policies. Thus, IDEOM ensures that all key stakeholders in dermatology (patient, physician, industry, insurer, and government) are involved in the process of developing such outcome measurements.

As part of the strategic plan to advance outcome measurements in dermatology, IDEOM has established workgroups in multiple dermatologic areas. The workgroups contain key stakeholders in psoriasis, psoriatic arthritis, hidradenitis suppurativa (HS), acne, pyoderma gangrenosum (PG), and actinic keratosis (AK). Each workgroup follows the Outcome Measures in Rheumatology (OMERACT) model. The OMERACT model is a methodology developed to identify what domains to measure in a disease (core domain formation), how to measure the core domains (instrument selection and/or development), and the settings in which the measurements should be used [2, 3].

In this report, we summarize the work to date on achieving these aims presented by each IDEOM workgroup at the 2020 IDEOM Virtual Annual Meeting on October 23–24, 2020 (Table 1). In addition, we summarize a presentation on demonstrating efficacy in clinical trials by Dr. Kendall Marcus, the Director of the Food and Drug Administration (FDA) Division of Dermatology and Dental Products.

The Psoriasis Workgroup focused on two subjects at the annual meeting: the development of a treatment satisfaction questionnaire and evaluation of instruments that measure accuracy, quality, and engagement of online content.

Dr. April W. Armstrong presented an update on the development of IDEOM’s treatment satisfaction instrument. Treatment satisfaction was identified by IDEOM as one of six core domains to be measured in psoriasis randomized clinical trials (RCTs), thus creating a need for a widely accepted instrument to measure responses in this domain [4]. After conducting a systematic literature review to evaluate existing treatment satisfaction measures [5], the Psoriasis Workgroup found that no single instrument met the standards for quality measurement properties developed by the Consensus-Based Standards for the selection of Health Measurement Instruments (COSMIN) guidelines [6]. Thus, IDEOM sought to develop a treatment satisfaction instrument specific to dermatological disease that also meets the COSMIN standards for Patient-Reported Outcome Measures (PROMs).

Dr. Armstrong presented the latest draft of IDEOM’s treatment satisfaction instrument. The questionnaire includes seven items reflecting Effectiveness, Convenience, and Overall Satisfaction, and each item will be answered on a 5-point unipolar scale. Following Dr. Armstrong’s overview of IDEOM’s instrument, Alison Kohn presented the next step in its development: a validation study. The validation study will be conducted according to the COSMIN study design checklist for PROMs [6-8]. Measurement properties, such as measurement error, reliability, and construct validity (as measured by convergent and known-groups validity), will be assessed in the validation study.

Additionally, as part of the Psoriasis Workgroup session, Dr. Armstrong led a discussion and voting session on evaluating psoriatic disease health content in the media. As online media has become a popular form of educating patients on psoriatic disease, misinformation has also become prevalent. There currently exists a lack of validated measures to evaluate different aspects of online content intended to inform patients. Thus, a critical need exists for validated, user-friendly instruments to assess accuracy and overall view experience for online educational materials [9]. The Psoriasis Workgroup helped evaluate several instruments that aimed to assess the accuracy [the Dy et al. Accuracy Scale (DAS) and Accuracy in Digital Health Instrument scale (ANDI)], quality (Global Quality Scale; GQS), and overall viewer experience (Armstrong Viewer Assessment scale; AVA) of health content in online media [9-11]. Workgroup attendees were asked to evaluate each instrument based on the Outcome Measures in Rheumatology (OMERACT) Filter 2.1. The OMERACT Filter 2.1 provides a framework for instrument selection, taking into account the truth, discrimination, and feasibility of use of an instrument [2, 3]. Per OMERACT standards, a consensus of >70% indicates that an instrument can be continued to be evaluated. The instruments meeting this criterion were ANDI, AVA, and GQS. The next steps will be to continue evaluating ANDI, AVA, and GQS as instruments to be utilized in assessing health content in online media per OMERACT standards.

Dr. Joseph F. Merola, Dr. Alice Gottlieb, and Dr. Lourdes Perez-Chada first provided an overview of the work conducted to date by the working group. In 2016, “PsA symptoms” was identified as a core domain to be measured in all psoriasis RCTs [12]. Two years later, IDEOM stakeholders agreed that: (1) all psoriasis trial participants should be screened for PsA prior to the measurement of PsA symptoms, (2) measurement of PsA symptoms should occur in those who screen positive or have a prior rheumatologist diagnosis of PsA, and (3) the most appropriate instrument to measure PsA symptoms in a psoriasis trial is the PsA Impact of Disease-9 (PsAID9), with the Routine Assessment Patient Index Data-3 (RAPID3) representing an acceptable alternative [13, 14].

Because the measurement of PsA symptoms in psoriasis trial participants who screen positive for PsA but do not have a confirmed diagnosis faces several challenges [15], including the low specificity of PsA screening tools [15], limited access to rheumatologists in psoriasis trials, and the fact that the PsAID-9 and RAPID3 may not be appropriate measures for psoriasis patients, IDEOM has shifted its focus towards the measurement of musculoskeletal symptoms (e.g., pain, stiffness, fatigue) broadly in the context of psoriasis RCTs. To that end, IDEOM is developing a PROM to quantify and monitor musculoskeletal symptoms and their impact on health-related quality of life in patients with psoriasis for use in RCTs and longitudinal observation studies, the International Dermatology Outcomes Measures Musculoskeletal Symptoms Questionnaire (IDEOM MSK).

During the meeting, the workgroup conducted two breakout sessions in which 41 relevant stakeholders (patient research partners with psoriatic disease, dermatologists, industry representatives, rheumatologists, and patient advocacy group representatives) anonymously voted on how to improve the content validity of the IDEOM MSK through an online platform. Briefly, consensus (≥70% of participants agreed on a response option) was met on: (i) how to modify the stem of questions to improve its comprehensibility, and (ii) what questions should be added or removed to improve its relevance and comprehensiveness. Detailed methods and results of this exercise will be published separately. Next steps involve modifying the instrument based upon the feedback received and re-assessing the content validity of the final instrument by a larger group of patients with psoriasis. In addition, testing of other critical measurement properties including construct validity, reliability, responsiveness, and interpretability will follow.

Dr. Amit Garg presented an update of the work that the Hidradenitis Suppurativa Core Outcomes Set International Collaboration (HISTORIC) has achieved to date. HISTORIC’s objective is to develop a core outcome set (COS) for use across interventional RCTs in HS. Over 100 participants comprise our stakeholder group, within which patients and experts have roughly equal representation. The stakeholders come from 19 different countries spread over 4 continents, making our collaborative truly multinational.

The COS development process began with a review of the literature as well as a nominal process which yielded approximately 60 items deemed relevant for measurement in trials by stakeholders. Since then, and through five rounds of e-Delphi and three international in-person meetings in Vienna, New York City, and Copenhagen, HISTORIC has achieved consensus on a core set of domains. These domains include the following: pain, physical signs, HS-specific quality of life, global assessment, progression of disease course, and symptoms [16].

HISTORIC has made significant progress towards either selecting or developing measures matched to each core domain. The Hidradenitis Suppurativa Quality of Life index is a 17-item patient-reported outcome which represents the first developed and validated disease-specific quality of life measure for HS [17, 18]. The Physical Signs Workgroup has developed a disease activity measure which includes body surface area and signs of inflammation with the goal of mitigating current reliance on lesion counts that are time-consuming and may be less reliable [19, 20]. The Symptoms Workgroup has developed the Hidradenitis Suppurativa Odor and Drainage Scale to measure two of the most troublesome symptoms in HS which have a significant life impact [21]. The patient global assessment is a single-item question which asks: “In the past 7 days, how much has HS influenced your quality of life?” This item has undergone initial psychometric evaluation and will be undergoing further testing [17]. Other measures in development include investigator global assessments as well as the measurement of flare and recurrence after surgery.

The Acne Core Outcomes Research Network (ACORN) divided their presentations into pre-established acne core domains: satisfaction with appearance, long-term acne control, health-related quality of life, extent of scars and dark marks, signs and symptoms, and satisfaction with treatment received [22]. Prof. Alison Layton, Dr. Hayley Smith, and Dr. Heather Whitehouse, from Harrogate, UK, presented a systematic literature review to identify the impacts of acne and the usage of questionnaires to detect them. The objective was to identify the most important disease impacts to patients with acne. The authors concluded that while many instruments were available to identify impacts, there was no universally recognized instrument for use in acne that was standardized with an agreed upon construct. Negative self-perception was a recurrent theme among the small number of qualitative studies [23]. The team is now correlating disease impacts with patient demographic and clinical details to inform future work.

Dr. Whitehouse also presented an ongoing systematic review of appearance and body image scales used to identify appearance-related concerns in acne. This has been developed in collaboration with the Centre for Appearance Research in Bristol.

Dr. Jerry Tan, from Western University, presented a systematic review to identify and appraise PROMs on treatment satisfaction in acne. Only one study described development of an acne treatment satisfaction measure, but development was inadequate and lacking in measurement properties. The conclusion was that there is an unmet need in measurement development for treatment satisfaction in acne [24].

Prof. Jonette Keri and Dr. Hilary Baldwin presented on the extent of scars and dark marks domain in acne. Their next steps are to review data from a systematic review of these topics.

Prof. Diane Thiboutot presented on the signs and symptoms domain in acne. She reiterated that prior consensus was not achieved between patients and providers on which signs and symptoms to measure and who should measure them. Furthermore, consensus was not previously achieved on the inclusion of global assessment as a core outcome domain. The next steps are to expand the workgroup with greater patient representation to address the above steps.

Dr. Vincenzo Bertolli, from the University of Ferrara, presented on progress in the long-term control domain in acne. Future areas of study include defining “long-term control,” measurement of long-term control, and whether outcome measures for acne severity, appearance, and/or health-related quality of life could be used to assess long-term control.

Dr. John Barbieri presented on a health-related quality of life domain in acne. Current goals are to expand his team, and to consider combining manpower among groups to move forward with domains further along the Harmonising Outcome Measures for Eczema (HOME) roadmap.

Dr. Afsaneh Alavi highlighted the results of a recent paper proposing domains for physician-reported outcomes and patient-reported outcomes for PG based on previous outcomes instruments used in clinical trials [25]. This systematic review suggested speed of healing, physician global assessment, and resolution of inflammation as potential useful domains for physician-reported outcomes in PG trials. On the other hand, none of the instruments used to assess patient-reported outcomes was specifically validated for PG. However, the duration of recovery, patient global assessment, pain, and quality of life were identified as potential domains.

Dr. William Huang presented on the development of a PG severity scale. The first two phases of a Delphi consensus exercise will involve the distribution of the Pyoderma Gangrenosum Disease Area and Severity Index (PGASI) scale to dermatologists and PG experts within the group for evaluation. The next steps are to generate additional data, conduct additional Delphi consensus exercises, and validate the PGASI.

Dr. Arash Mostaghimi presented on the eligibility for PG in clinical trials. He emphasized that there is a need for a pragmatic set of inclusion/exclusion criteria that can guide the selection and eligibility of patients for clinical trials in PG. The PG Workgroup internally surveyed out which of the three current sets of diagnostic criteria for PG would be appropriate to be adopted in clinical trials favoring the PARACELSUS diagnostic tool (15/24, 62.5%). Interestingly, a recent study comparing these diagnostic frameworks in a well-established cohort of PG patients also favored the use of the PARACELSUS diagnostic tool to diagnose/identify cases with PG in clinical settings [26]. The next steps are to reach consensus to establish recommendations on inclusion and exclusion criteria among PG experts for upcoming future clinical trials.

Dr. Alex Ortega-Loayza presented on the development of PROMs in PG. He provided an overview of how the COSMIN principles can guide the validation of PG-specific PROMs. The first step will be to generate data to identify appropriate domains for PROMS. Lastly, Dr. Ortega in the PG Workgroup is working on establishing a prospective international registry for PG patients as well as COS initiative in collaboration with the Cochrane Skin Core Outcome Set Initiative (CS-COUSIN) [27].

The AK Workgroup commenced its inaugural meeting with an open discussion led by Dr. Daniel Siegel regarding the need to develop outcome measures that can be validated and approved in the approach to AK management. Participants included physicians with expertise in AKs, patients, and pharmaceutical industry scientists. The substandard quality of life experienced by those with AK, paired with the possibility of progression to squamous cell cancer, which has a high metastatic potential, underlines the necessity of appropriate outcome measures [28, 29]. Before the meeting, stakeholders participated in a survey where they stated areas where there might be an opportunity to improve AK care, which provided a foundation for this meeting. In alignment with IDEOM’s primary initiative of developing outcome measures to better assess the effectiveness of providers and therapeutic options, there was robust discussion among participants about unmet needs in: (1) the measurement of AKs in clinical practice, (2) outcome measures in clinical trials, and (3) care of AKs [1].

On the topic of unmet needs in the measurement of AKs, the group expressed concerns over the lack of adequate technology for measuring clinically apparent lesions, identifying subclinical lesions, and quantifying the extent of mutational burden. The expectation is that with the development of this technology, the risk of future AKs can be predicted, which can guide treatment and management. The need for long-term outcome measures and protocols for combination therapy was also addressed.

Points raised on unmet needs in RCT outcome measures included the necessity of head-to-head comparisons, stricter controls of confounding factors, and accurate lesion counts pre- and post-therapeutic intervention. Other topics included the need for PROMs to assess treatment satisfaction, measurement of cosmetic outcomes of therapy, and more tolerable treatment options with better adverse reactions and cost profile. There was also an emphasis on having a primary endpoint of the percent reduction in AK lesions from baseline, in place of the current FDA-recommended endpoint of 100% clearance.

The discussion on appropriate outcome measures to assess care in AKs centered around evaluating treatment efficacy. It highlighted the need for proper field therapies that treat visible and subclinical lesions. Better therapeutic options for concurrent management of field cancerization and hypertrophic lesions will prove beneficial by providing both preventative and therapeutic care. The need for cost-effective treatments with better tolerance and minimal local skin reaction was also briefly mentioned.

The meeting concluded with a conversation on the next steps, including establishing a platform to promote further dialog surrounding unmet needs in AK. The team’s future goals include implementing a Delphi method, which has shown great utility in other workgroups, to reach group consensus on the most pressing unmet needs [30].

Dr. Kendall Marcus, Director of the Division of Dermatology and Dental Products at the FDA, presented on the use of novel outcome measures to demonstrate efficacy in RCTs. She provided an overview of terminology for endpoint development, as well as how to determine if a novel endpoint is “fit-for-purpose” in a product development program.

Dr. Marcus defined a clinical outcome assessment (COA) and emphasized the difference between COAs and clinical trial endpoints. A COA is an assessment of a clinical outcome made through report by a clinician (ClinRO), patient, non-clinician observer, or a performance-based assessment. A clinical trial endpoint is a quantitative measure of the effect of an intervention, which is analyzed to determine differences between test and control groups in an RCT.

Clinical benefit from treatment was also defined as a positive clinically meaningful effect of an intervention. To establish a clinically meaningful change, the assessment must measure an outcome of significance to patients, and the magnitude of change at the individual patient (within-patient) must be large enough to affect how patients feel or function in daily life.

Dr. Marcus also reviewed key elements that the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) looks for when evaluating ClinROs: treatment target and patient population, measurement concepts, outcomes of interest, endpoint definition and positioning, and scoring and score interpretation. She also demonstrated how to apply these concepts using an example RCT investigating the use of collagenase clostridium histolyticum for improving cellulite severity.

Lastly, Dr. Marcus reviewed the key elements that the FDA looks for when assessing COAs. These include whether or not the instruments are fit-for-purpose, adequate to support labeling claims, and if the primary endpoint detects clinically meaningful change.

In this report, we summarized the work presented by each IDEOM workgroup at the 2020 IDEOM Virtual Annual Meeting (Table 1). During the meeting, each workgroup presented their progress to date, as well as outlining the next steps in furthering outcome measurement research in psoriasis, psoriatic arthritis, HS, acne, PG, and actinic keratosis. Additionally, Dr. Kendall Marcus, Director of the FDA Division of Dermatology and Dental Products presented regarding the demonstration of efficacy in RCTs. The continuing progress of each workgroup will be presented at the 2021 IDEOM Annual Meeting.

Each workgroup continues to work towards IDEOM’s mission of developing evidence-based, consensus-driven, and widely accepted outcome measures. The development of such outcome measures will impact patient care by influencing treatment recommendations and providing important benchmarks. Such outcome measures will also influence clinical trials and payment policies, as clear benchmarks will allow for easier determination of disease severity and treatment efficacy.

The IDEOM dermatologic workgroups reported their progress on developing and validating outcome measures.

Folawiyo Babalola, Lesley Grilli, John Latella, Alison H. Kohn, Kendall Marcus and Lourdes M. Perez-Chada have no conflicts of interest to disclose. Dr. Alavi has served as a consultant for UCB, Janssen, Novartis, Abbvie, InflaRx. In addition, Dr. Alavi is a Section Editor of Dermatology. Dr. Armstrong has served as a research investigator and/or scientific advisor to AbbVie, ASLAN, BI, BMS, EPI, Incyte, Leo, UCB, Janssen, Lilly, Novartis, Ortho Dermatologics, Sun, Dermavant, Dermira, Sanofi, Regeneron, Pfizer, and Modmed. Dr. Garg is an advisor for AbbVie, Anaptys Bio, Boehringer Ingelheim, Bristol Myers Squibb, InflaRx, Incyte, Janssen, Novartis, Pfizer, UCB, and Viela Biosciences, and receives honoraria. Dr. Garg also receives research grants from AbbVie and the National Psoriasis Foundation. In addition, Dr. Garg is a Section Editor of Dermatology. Dr. Gottlieb has received honoraria as an advisory board member and consultant for Anaptyps Bio, Avotres Therapeutics, Beiersdorf, Boehringer Ingelheim, Bristol-Myers Squibb Co., Janssen, LEO Pharma, Eli Lilly, Novartis, Sun Pharmaceutical Industries Inc., UCB, and Xbiotech (only stock options), and has received research/educational grants from Boehringer Ingelheim, Janssen, Novartis, UCB, Xbiotech, and Sun Pharma (all paid to Mount Sinai School of Medicine). Dr. Jemec reports grants and personal fees from Abbvie, personal fees from Coloplast, personal fees from Chemocentryx, personal fees from LEO pharma, grants from LEO Foundation, grants from Afyx, personal fees from Incyte, grants and personal fees from InflaRx, grants from Janssen-Cilag, grants and personal fees from Novartis, grants and personal fees from UCB, grants from CSL Behring, grants from Regeneron, grants from Sanofi, personal fees from Kymera, and personal fees from VielaBio, outside the submitted work. In addition, Dr. Jemec is Editor-in-Chief of Dermatology. Dr. Merola is a consultant and/or investigator for Abbvie, Amgen, Bayer, Dermavant, Eli Lilly, Novartis, Janssen, UCB, Celgene, Sanofi-Regeneron, Biogen, Pfizer, BMS, and LEO Pharma. He may receive consulting fees, honoraria, and grant support. Dr. Ortega has served on the advisory board with honoraria at Janssen Inc. and Bristol Mayer Squibb, and has served as a consultant for Consultant Guidepoint Inc. Dr. Siegel has served as a consultant for Logical Images, MedX Health, Strata Skin Sciences, UCB, SciBASE, Pulse Biosciences, Sol-Gel Technologies, Verrica Pharmaceuticals, and Seapire Skincare. Dr. Siegel has also served on the advisory board of Tetros Group, Sun Pharmaceutical Industries Ltd, SkinVision, Biofrontera AG, Kamedis Ltd, Greenway Therapeutix, Palmm, Modernizing Medicine, and DermTech International. Dr. Strand reports consulting fees from AbbVie, Amgen, Arena, AstraZeneca, BMS, Boehringer Ingelheim, Celltrion, CORRONA, Crescendo/Myriad, Equillium, Genentech/Roche, GSK, Horizon, Inmedix, Janssen, Eli Lilly, Novartis, Pfizer, Regeneron Pharmaceuticals Inc., Samsung, Sandoz, Sanofi, TwoXAR, and UCB, outside the submitted work. Dr. Tan is an advisor, consultant, investigator, and/or speaker and received grants and honoraria from Almirall, Bausch, Botanix, Boots Walgreens, Cipher, Galderma, Novartis, Sol-Gel, Sun, and Vichy.

No funding sources were used for this research endeavor.

Each author made substantial contributions to the conception and design, drafting and revising of this work, final approval of the version to be published, and have agreed to be accountable for all aspects of the work.