Ten-Year Experience of a Global and Freely Accessible e-Learning Website for Pediatric Endocrinology and Diabetes Open Access

e-Learning can be defined as “an approach to teaching and learning, representing all or part of the educational model applied, that is based on the use of electronic media and devices as tools for improving access to training, communication, and interaction and that facilitates the adoption of new ways of understanding and developing learning” [1]. The European Society for Paediatric Endocrinology (ESPE) interactive website, https://www.espe-elearning.org, was first published online in 2012 in an effort to combine education, training, and formative assessment in learning as well as competency-based medical education of pediatric endocrinology and diabetes. It was preceded by a systematic approach to identify relevant aspects of accessibility, dissemination, and analysis of the needs of student and teacher pediatric endocrinologists; to define the key learning objectives, research the educational literature, and explore information and communication technology design specifications [2]. A large module on pediatric diabetes was added with the support of the International Society for Paediatric and Adolescent Diabetes (ISPAD). The e-learning website supports the delivery of educational material in various e-learning settings and allows health care trainees and professionals to further develop their knowledge, skills, attitudes, and competencies, keeping them actively engaged in learning in an ongoing manner [3‒6]. The purpose of the e-learning website is directed at formative assessment, where the goal is to monitor learning and teaching by providing ongoing assessment which is information rich and serves to steer and foster the student’s learning to the maximum of his/her ability [2]. The e-Learning Committees of ESPE and ISPAD are responsible for the development and quality of its content. The website is free of charge and globally accessible.

In this paper, we describe the various applications of the e-learning website that have been developed over the last 10 years. Self-directed study and (virtual) classroom teaching, termed as blended learning and instruction, are complementary as online educational materials can be studied first followed by in-depth and interactive instruction by a tutor either online or in a classroom setting.

We also describe the development of a separate module within the e-learning website that focuses on frontline health care providers, medical doctors, and specialists in limited resource settings. Most recently, master classes in the format of interactive video lectures have also been included. Formal recognition for the educational value of the e-learning project has been obtained by securing formal continuing medical education (CME) accreditation through the European Accreditation Council for Continuing Medical Education (EACCME-UEMS, Brussels).

Finally, given that recent literature has indicated complementary roles for e-learning and e-consultation [7], we describe pilot studies aimed at establishing joint e-learning/e-consultation webinars. We conclude by briefly reviewing some proposals to help develop further e-learning strategies in the future.

The content of the e-learning project, which has been greatly expanded over the years thanks to contributions of many authors from all over the world, is well suited to support self-directed study in order to acquire up-to-date knowledge. Table 1 lists some of the chapters dealing with a wide variety of pediatric endocrine themes, including diabetes, which concisely describe physiology and pathophysiology, along with practical approaches to management and treatment. The chapters are presented in bullet point format. As shown in Figure 1 and Table 1, the content of the website consists of 13 categories comprising 62 chapters and 103 problem-solving cases. The text includes references to textbooks, review articles, and original studies. In addition, real-life clinical cases accompany each chapter so that students can identify practical solutions for diagnosis and management of specific medical conditions in a stepwise and interactive manner. Case studies are focusing on common general pediatric endocrine conditions such as disorders of growth and puberty and are of particular interest to medical students or junior trainees. Registration for the website is free of charge, and the site is readily accessed through a computer, tablet, and with increasing frequency via smartphones as shown in Figure 2.

The e-learning website contains a large module on pediatric diabetes, providing the complete content of the most recent clinical practice consensus guidelines of the International Society for Paediatric and Adolescent Diabetes (ISPAD) [6]. In addition to 24 chapters, 37 problem-solving case studies have been added to complement the various chapters. The module on health care in resource-limited setting contains a chapter and 4 interactive problem-solving cases and focuses particularly on management at primary and secondary health care levels. The ISPAD consensus guidelines are updated and revised every 4 years, and the pediatric diabetes modules will be reviewed and updated to reflect important new management recommendations. ISPAD also offers free 60-min live webinars (including time for questions and answers) on major topics approximately every 3–4 months. These have been selected and prioritized in response to a questionnaire seeking suggestions for topics. Examples of recent webinars include “Advances in medical nutrition therapy in type 1 diabetes – from theory to practice”; “Navigating strategies to encourage physical activity in children and adolescents with type 1 diabetes”; and “Glucose monitoring in different health care settings”; recognizing that resources for diabetes care are highly variable throughout the world, the content is deliberately adapted to a worldwide audience. Importantly, the webinars are all archived and freely available at any time to learners unable to attend the live broadcasts. Future webinars will focus on major updates and revisions to the ISPAD guidelines due to be completed in October 2022. At the present time, all webinars are in English, which limits their utility for potential learners who are not fluent in English. Also, CME accreditation for attendance is currently not available.

From 2017 onward, a Health Care in Limited Resource Setting module has been developed for frontline physicians and health workers and also for trainees or medical specialists in resource-limited countries. The module comprises 16 chapters and 24 interactive problem-solving case studies (vignettes) exemplifying clinical issues and pitfalls, with specific attention to the 3 levels of medical health care in resource-limited settings (Table 1). The entire content has been translated by native French-, Spanish-, Swahili-, and Chinese-speaking colleagues into their respective languages [5]. This module has been well received, particularly in targeted parts of the world as shown in Figure 3.

These master classes have been devised recently in an attempt to present the student with a series of informal interactive video lectures which can be viewed and completed at leisure. After a 12-min introduction, the first master class (25 min) deals with Karlberg’s infantile, childhood, and pubertal growth curves and the factors driving each. The second master class (40 min) describes practical skills – data calculation and plotting, concepts such as centiles, normal distribution, and standard deviations (which students often find difficult), and a discourse on available growth charts. The third and final master class (60 min) deals with common problems of growth and puberty in the context of each of Karlberg’s growth phases. Each class contains exercises and problems for the student to engage with and is intended to be studied in at least two if not three sessions.

This initiative, which represents a departure from traditional teaching, may prove to be a more realistic learning strategy than the classic 45-min lecture model. This is because experience has shown that students have difficulty assimilating and retaining information imparted in a time-restricted setting.

The master classes are now available in English with English subtitles and are currently being translated with Russian subtitles and text. They will be redone in French with French subtitles in early 2023.

The content of the website is also suited for classroom case discussions monitored by a tutor or moderator. The suggested answers to the multiple choice questions in the problem-solving case studies can often lead to lively discussions. In particular, the Health Care in Resource Limited Setting module is useful in helping tertiary health care workers in regional or academic training centers to act as tutors to medical students or other health care workers. Of note, the website offers the option of a virtual classroom setting in which participants, including the instructor or tutor, can join remotely.

As already stated, the various applications discussed are targeted at pre- and postdoctoral medical students, residents, and fellows as well as nurses or nurse practitioners with an interest in pediatric endocrinology and diabetes. The classroom and master class applications are helpful in various postgraduate training settings such as Pediatric Endocrinology Training Center for Africa (PETCA) [8, 9] and ESPE Winter School. For the ESPE Maghreb School, material is available in French, and for the ESPE Caucasus and Central Asia School, material is available in Russian.

The European Accreditation Council for Continuing Medical Education (EACCME), an institution of the European Union of Medical Specialists (UEMS), has signed agreements recognizing EACCME® e-learning credits with most European countries. In addition, agreement of mutual recognition exists with the USA and Canada, while other countries may recognize EACCME® credits on a voluntary basis. Providing accreditation for medical education of the highest quality requires that a set of principles and essential criteria are defined [10, 11]. Sixty minutes of educational activity earns 1 ECMEC®.

A set of 30 e-learning courses taken from the General Content, the Health Care in Limited Resource Setting module, and the Diabetes module of the e-learning website has recently been accredited by EACCME in 2022. This set includes 10 CME credit hours for pediatric endocrinology core topics, pediatric endocrinology in resource-limited setting topics, and Diabetes ISPAD Guidelines core topics, respectively. Each section consists of 10 learning sequences, 1 CME each, containing a part of a chapter and/or one or more problem-solving cases. An example is shown in Figure 4. Learning progress is measured by the learning management system (LMS), tracking whether every page has been visited before allowing the learner to proceed to the next item in the sequence. The last item is a survey, where the learner provides feedback that will be used for statistical purposes and further development. After finishing the survey, the learner receives a personalized downloadable certificate.

While e-learning can be briefly defined as “any educational intervention mediated electronically via the Internet” [12], e-consultation is defined as “synchronous or asynchronous, consultative, provider-to-provider communications within a shared electronic health record” [13] or as “the use of information communication and technology for interaction between health professionals.” e-Learning improves professional knowledge and skills of health care workers, particularly when simple, stable, and easy-to-use technology is applied, and technical problems can be circumvented [14]. Telemedicine networks such as “Médecins Sans Frontières” and Collegium Telemedicus, a store-and-forward consultation platform (www.collegiumtelemedicus.org), enable provision of specialized care through supervision and evaluation provided remotely [15, 16]. It is important to note that explicitly offering e-learning in combination with e-consultation will enhance the usability of e-consultation. As illustrated in Figure 5, e-learning provides background and in-depth understanding of underlying pathophysiology and evidence-based management, which will improve effectiveness, efficiency as well as satisfaction and empowerment within the community [7].

For example, in 2021, a series of combined e-learning and e-consultation webinars were initiated in close collaboration with the Indonesian Pediatric Society (IPS) and ESPE. The format is as follows: the expert tutor starts the session by discussing relevant parts of the e-learning website (chapter and/or case) for 30–40 min during which interactive polling is launched to keep the audience engaged. Participants present 2 or 3 anonymized cases in PowerPoint format. The last slide contains questions or management dilemmas for discussion by the tutor. The total session lasts 90 min. Finally, the tutor prepares a brief written summary including the e-learning points discussed, key messages and provides suggestions for further study using the website content and relevant, up-to-date literature references.

Survey details from respondents of the Indonesian webinar series indicated that after 9 sessions, the attendance number of fellows and junior staff was steady at around 45 participants, and more than 90% indicated that access to the online web series modules was appropriate and objectives of the course were clearly identified and met. Moreover, content was rated most interesting, the level of the e-learning content being in accordance with current knowledge. These sentiments are reflected by 98% of the participants agreeing to use the webinar format to reach out to referring physicians and health care workers in their own city or region.

In this review, we describe the various global applications of an interactive e-learning website that has been developed over the last 10 years. e-Learning is as effective as other educational approaches for the acquisition of knowledge and skills [17]. It serves as an effective adjunct to conventional teaching due to the ease of access online, the ability to quickly revise and update educational materials, and the ability to serve numerous students at a relatively low cost. Research on outcomes of e-learning has predominantly focused on acquisition of knowledge and skills and participant satisfaction, whether at the level of a medical student or of continuing professional development of health care professionals.

However, research on assessment of performance – the combination of skills and behavioral changes that result from e-learning intervention and on patient outcomes – meets with conceptual and practical challenges. Tudor et al. [18] reviewed many research questions identified from an extensive overview of systematic reviews on health professions’ digital education. Questions regarding the evaluation of digital health education include how studies should be designed to ensure the generalizability of their findings across different settings and how to report them and to what extent e-learning is effective in improving the application of knowledge and performance of skills of health care professionals in the clinical setting. To answer these questions, reliable and validated instruments to objectively evaluate behavioral and patient outcomes need to be developed.

Traditional methodologies such as randomized controlled trials are clearly impractical in the postdoctoral clinical setting. Research on e-learning, including face-to-face or virtual classroom teaching, and whether preceded by blended e-learning [19], needs to move from assessing knowledge acquisition and participant experiences toward cultivating an understanding of how e-learning is influencing behavior and consequently improving patient outcomes [20]. In addition, relevant to the application of global e-learning are questions about whether cultural and socioeconomic factors within different countries influence the use of digital education. Urstad et al. [21] concluded that when sharing e-learning resources across countries, an adaptation and translation process that includes a multicultural and multidisciplinary perspective is needed. In view of this concern, authors from all over the world have been invited to contribute to the content of the ESPE e-Learning website, and, in particular, the content of the module “Health Care in Resource Limited Setting” was written by authors from low- and middle-income countries [5].

Although global and free access to up-to-date content under the guidance of members of learned societies such as ESPE and ISPAD is a great asset, the language in which the content is provided may be a major barrier, particularly in continents and countries where English is not the native language. Although in recent years, considerable advances in translation using artificial intelligence have been made, significant limitations with currently available machine translation technologies prevent its general use, so that time-consuming and costly human correction is still required [22, 23].

The social distancing measures introduced to control the COVID-19 pandemic have dramatically changed the delivery of education worldwide. With this sudden shift, digital education, whether individual or in a virtual classroom setting, has become even more important since e-learning can connect and engage individuals across geographic boundaries as well as those who live in remote areas. In a recent review, an up-to-date summary of best practice of virtual classroom e-learning and how to effectively apply blended learning is given [24]. A recommended integrated approach involves case-based learning in which an online case presentation leads the participants via an unfolding case scenario featuring a series of trigger questions and model responses that encourage use of their clinical reasoning skills to reach a definitive diagnosis and management plan. Interpersonal interaction can be provided by facilitator-led group work in virtual break-out rooms or by engaging students using online polling which has the additional advantage of giving the tutor an impression of the level of knowledge of the students and provides an immediate incentive for further discussion.

e-Learning may also be an opportunity to expand or tailor educational activities for learners according to their differing learning needs. Thus, the pediatric endocrinology e-learning website provides guidelines for those interested in general pediatrics, neonatology, clinical genetics, and pediatric gynecology. Moreover, competencies in clinical management and communication from e-learning have been evaluated in two global studies focusing on the “medical expert” and the “communicator” [25, 26]. All the clinical fellows involved appreciated the personalized feedback from the various experts and particularly valued the many different viewpoints expressed and the encouragement provided.

In general, learning by practice is regarded as the most powerful way of learning. However, even in major referral centers, the number of patients with rare diseases, such as disorders of sex development, is small. Thus, exposure to the clinical management of newly referred patients with rare conditions is very limited. There is also a strong need for alternative learning methods to train the necessary competencies for a medical specialist. e-Learning is increasingly being used to train competencies such as “scholar” and medical expert. The use of e-learning to train other competencies such as communicator and collaborator is not yet common practice. The global study evaluating competencies of fellows in pediatric endocrinology, as described, shows that e-learning with personal feedback offers possibilities to teach these competencies. In principle, the LMS allows direct interaction between fellow and tutor/expert. As the emphasis is on formative assessment, the tutor is able to correct any misunderstanding of the assignment or to request further details. Similarly, the combined e-learning and e-consultation webinars have been shown to not only improve knowledge and understanding of pathophysiology but also to promote clinical management by direct interaction between tutor and participant. The role of the tutor in the classroom setting and of the e-moderator in webinar teaching should not be underestimated and warrants training in digital and educational competence, assessment of educational skills as well as specific (academic) accreditation [18, 27‒29]. However, it should be stressed that the ESPE e-Learning website is different from LMS in how it serves both students and tutors. The e-learning website is freely accessible on a global scale but lacks the administration and managerial structure of an institution where specified groups of students are assigned to a tutor who is monitoring learning progress from within the system. However, it is targeting institutions and individuals to use parts of the website content, within or in addition to their internal system.

This review demonstrates that a global e-learning platform can be successfully developed and maintained. However, ensuring that objectives are met and content is maintained requires frequent updates within the e-learning environment – a laborious, time-consuming, and complex task requiring a dedicated team of content experts willing to provide their time voluntarily for the greater good. Moreover, the e-learning platform model depends on a dedicated management team that is well equipped to handle the ever-changing technical online standards, new information technology developments, and the growing workload in project and content management.

During the pandemic, we have seen an exponential growth in digital health information [29]. In the future, clinicians and students with limited time for continuing education may require customized education that is tailored to their specific learning needs. e-Learning platforms such as the ESPE e-Learning website can be more readily updated and customized to the participants’ learning and educational needs to be able to reach geographically dispersed health professionals, especially in low- and middle-income settings.

The future of education delivery may include hybrid learning strategies, which include in-person and e-learning platforms and are likely to become the universal mode of delivering health education [30]. Combined e-learning and e-consultation webinars illustrate how international academic institutions, learned medical specialty societies, and networks are uniquely placed to deliver balanced, disease-oriented, and patient-centered e-learning education and at the same time provide expert consultation, as illustrated in Figure 5 [7]. Moreover, they are well equipped to maintain professional standards and to offer appropriate accreditation [10, 11].

Authors would like to thank International Society for Pediatric and Adolescent Diabetes (ISPAD); https://www.ispad.org/, the International Consortium of Pediatric Endocrinology (ICPE); intpedendo.org, Global Pediatric Endocrinology and Diabetes (GPED); https://www.globalpedendo.org/, the European Reference Network on Rare Endocrine Conditions (Endo-ERN); https://endo-ern.eu/, the Collegium Telemedicus (CT); https://www.collegiumtelemedicus.org/, the European Association of Paediatric and Adolescent Gynaecology; https://www.eurapag.com, the International Federation of Pediatric and Adolescent Gynecology; https://www.figij.org/, Ikatan Dokter Anak Indonesia, Indonesian Pediatric Society; https://www.idai.or.id/.

Sze May Ng is chair of the UK Association of Children’s Diabetes Clinicians, chair of the e-Learning Committee of ESPE, and a member of the e-Learning Committee of ISPAD. Eva Kalaitzoglou is a member of the ESPE e-Learning Committee. Agustini Utari is a moderator of the IPS-ESPE webinar series, and Conny van Wijngaard-de Vugt is a project and content manager of the ESPE e-Learning website. Malcolm Donaldson is a past coordinator of ESPE Winter School and ESPE Maghreb School and a current member of the ESPE Caucasus and Central Asia School. Joseph I. Wolfsdorf is the current chair of the ISPAD e-Learning Committee, a past president of ISPAD, and is an associate editor of the journal. Annemieke Boot is past chair of the ESPE e-Learning Committee. Stenvert Drop is a past member, treasurer, and president of ESPE Council; a former member of the ESPE Caucasus and Central Asia School; and a former chief editor of espe-elearning.org.

Espe-elearning.org online learning website has received financial support from ESPE; ISPAD; Kyowa Kirin, Japan; Sandoz International, Germany; Alexion Pharmaceuticals, USA; and the European Union’s Seventh Framework Program (FP7/2007-2013) under grant agreement No. 2014444. RLC Project has received financial support from ESPE, ISPAD, and Eli Lilly & Co., USA.

Stenvert Drop and Sue May Ng wrote the initial draft. Evangelia Kalaitzoglou, Agustini Utari, Conny van Wijngaard-de Vugt, Malcolm Donaldson, Joseph I. Wolfsdorf, and Annemieke M. Boot provided contributions and approved the final draft.