Introduction: Literacy of health professionals on patient safety aims to prevent and reduce risks and adverse events in healthcare. Objective: The objective of this study was to translate, adapt, and validate for Portugal of the World Health Organization Questionnaire to Assess the Implementation of the Multiprofessional Guide. Methodology: Following the cultural adaptation process, we carried out a psychometric analysis on a sample of 300 health professionals. The scale was tested for apparent and content validity. Psychometric properties were assessed using Cronbach’s alpha coefficient (α) through exploratory and confirmatory factor analysis, convergent validity, and discriminant validity. Results: Exploratory factor analysis led to a tetra factor structure that accounted for 43.0% of the variance and had an overall alpha of 0.759. The hypothesized structure was submitted to confirmatory factor analysis, and the following items were eliminated from factors 1, 2, and 3 and the respective items: 7; 5; and 3, 4, 5, and 6, due to multicollinearity problems. Overall good-of-fit indexes are reliable. Conclusions: The “KIT TOOL-S2 TEXT” scale presents a factor structure with satisfactory validity and reliability results, adequately representing the constructs in question.

Introdução: A literacia dos profissionais de saúde sobre a segurança do doente visa prevenir e reduzir riscos e eventos adversos no cuidado de saúde.Objetivo: Tradução, adaptação e validação para Portugal do Questionário da Organização Mundial de Saúde para Avaliar a Implementação do Guia Multiprofissional.Metodologia: Seguindo o processo de adaptação cultural, realizámos a análise psicométrica numa amostra de 300 profissionais de saúde. A escala foi testada quanto à validade aparente e de conteúdo. As propriedades psicométricas foram avaliadas usando o coeficiente Alfa de Cronbach (α) por meio de análise fatorial exploratória e confirmatória, validade convergente e discriminante.Resultados: A análise fatorial exploratória levou a uma estrutura tetrafatorial que explicou 43,0% da variância e teve um Alfa total de 0,759. A estrutura hipotetizada foi submetida à análise fatorial confirmatória, e os seguintes itens foram eliminados do fator 1, 2 e 3, os respetivos itens: 7; 5 e 3, 4, 5, e 6, devido a problemas de multicolinearidade. Os índices gerais de adequação do modelo são confiáveis.Conclusões: A escala “KIT TOOL-S2 TEXT” apresenta uma estrutura fatorial com resultados satisfatórios de validade e fiabilidade, representando adequadamente os construtos em questão.

Palavras ChaveSegurança do doente, Estudo de validação, Fiabilidade e validade, Guia multiprofissional, KIT TOOL-S2

The field of patient safety has emerged as a critical public health issue, manifesting implications across various clinical domains. This recognition underscores the necessity of a multifaceted approach to patient safety – one that prioritizes interventions, involves all stakeholders, and establishes a global framework for action. This approach is essential for reducing the likelihood of harm to patients and contributes to healthcare excellence grounded in robust evidence. In this context, the promotion of an open and equitable culture, where health professionals are encouraged to report adverse events and errors, becomes imperative [1].

Across the globe, healthcare systems inadvertently inflict harm to patients, both occasionally and unintentionally. Such adverse events are widespread and have significant emotional and physical ramifications for patients, their families, and healthcare professionals [2]. This acknowledgment has propelled patient safety to the forefront of healthcare quality improvement efforts in many countries.

Central to enhancing patient safety is the principle of safety itself, which is fundamental to patient care and a key indicator of management quality. Improvement efforts span across the health system, encompassing a wide range of actions from performance improvement and environmental safety to risk management. This broad framework includes aspects like control of healthcare-associated infections, safe medication practices, equipment safety, safe clinical practices, and ensuring a safe environment, implicating all areas and professionals within healthcare [3].

In the wake of the increasing global emphasis on patient safety, various debates have arisen both nationally and internationally, focusing on implementing best practices in patient care settings. To cultivate a safety culture, addressing patient safety issues in diverse healthcare settings is crucial. Such initiatives foster skill development among health professionals, encouraging proactive measures to minimize adverse events and errors [4].

A key aspect of enhancing patient safety is assessing health professionals’ literacy on this subject, a task that has proven to be complex. The absence of systematic assessment models necessitates the use of reliable instruments for determining the need for intervention in patient safety. In this respect, the World Health Organization (WHO) has proposed an evaluation framework based on investigating health course curricula to identify common and underexplored contents in patient safety, as detailed in their “Patient Safety Curriculum Guide: Multi-professional Edition” [5].

In 2011, the World Health Organization published the Multiprofessional Guide, a guide with a comprehensive program for implementing patient safety training in healthcare settings worldwide to facilitate and support interdisciplinary knowledge, skills, and attitudes on patient safety in healthcare institutions wherever possible. The Multiprofessional Guide has significantly influenced several initiatives such as the Australian Patient Safety Education Framework (APSEF) [6] and the Canadian Framework [7]. Both identified the main areas of activity (knowledge, skills, and behaviors) that contribute to patient safety based on an extensive review of the corpus of knowledge on this topic. These activities were then categorized into learning areas, further subdivided into 22 main themes, and assigned an expected level of competence according to the level of responsibility within the healthcare system [6].

The WHO’s initiative underscores the importance of developing and validating scales, a crucial component in health science research. For data collection instruments to be effective, they must undergo rigorous psychometric validation, demonstrating both cross-sectional and longitudinal measurement properties [8].

This study, therefore, aimed to translate, culturally adapt, and validate for Portugal the “KIT TOOL-S2 TEXT. Questionnaire to Assess the Implementation of the Multiprofessional Guide” scale of the World Health Organization. By doing so, our study offers significant insights and advancements in the domain of patient safety and healthcare quality, specifically for Portuguese health professionals. The adaptation and validation of the WHO’s KIT TOOL-S2 TEXT Multiprofessional Guide questionnaire to the Portuguese context provides a critical tool for evaluating and enhancing knowledge, attitudes, and practices related to patient safety. Our findings contribute to a more comprehensive understanding of the questionnaire’s applicability across diverse settings, with the potential to positively influence patient safety and healthcare quality both within Portugal and internationally.

  • What is the internal structure obtained after exploratory and confirmatory factor analysis of the translated and adapted into European Portuguese version of the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide”?

  • Is the model hypothesized in the exploratory factor analysis verified by the confirmatory factor analysis?

Type of Study and Sample

The methodological study, of psychometric nature, was based on cross-sectional data collection from a non-probabilistic convenience sample of 300 health professionals from a hospital center in the Central region of Portugal from 2019 to 2020.

Instrument

A two-section questionnaire was used to collect the data. The first section contains questions for the sociodemographic and professional characterization of the sample (age, gender, marital status, education, professional category, weekly workload, type of schedule, accumulation of duties, holding management or leadership positions in the service, liking their profession).

The second section contains the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide,” used in this study with the approval of the World Health Organization. The scale contains four sections, that is, Section 1 – Patient Error and Safety, comprising 7 items assessed according to a 5-point scale (1 = low level of knowledge; 3 = moderate level of knowledge; 5 = high level of knowledge); Section 2 – Health System Security, containing 5 items, assessed on a 5-point Likert scale, where 1 corresponds to “Strongly Disagree” and 5 to “Strongly Agree”; Section 3 – Personal Influence on Safety, consisting of 7 items, each rated on a 5-point Likert scale, where 1 corresponds to “Strongly Disagree” and 5 to “Strongly Agree”; Section 4 – Personal Attitudes toward Patient Safety, which comprises 4 items, assessed on a 5-point Likert scale, where 1 corresponds to “Strongly Disagree” and 5 to “Strongly Agree.”

Procedures

Procedures Followed in the Adaptation of the Scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide”

The translation and cross-cultural adaptation process of the scale “KIT TOOL-S2 TEXT. Questionnaire to Assess the Implementation of the Multiprofessional Guide” into European Portuguese was initiated after contact and authorization from the original authors. To ensure the methodological quality, both in the process and in the final result, we followed the methodological recommendations proposed by Beaton, Bombardier, Guillemin, and Ferraz (2000), as well as by the International Test Commission (2013) [9, 10].

In the first stage, two translations were made from the original instrument into European Portuguese, carried out independently by bilingual translators with knowledge of the concepts addressed in the instrument. At this point, functional equivalence was respected to maintain the original characteristics of the text, granting greater reliability and validity of the instrument and keeping the same number of items. In the second stage, the Portuguese versions obtained in the first stage were back translated into English also by independent and bilingual translators with knowledge of the concepts addressed in the instrument. In the third stage, we compared the English versions of the questionnaire (the original and the back-translated versions), made a formal assessment of semantic equivalence, and compared all the expressions. In the fourth stage, an accredited Portuguese language expert made a thorough revision of all the items in the Portuguese versions. In the fifth stage, a clinical expert translated the instrument from the European Portuguese version again into English. Semantic equivalence was later reviewed conceptually by seven experts in the field to check the acceptability of words and phrases. After the necessary changes were made, a pretest was applied to a sample of 30 people to check the feasibility of the instrument. The qualitative evaluation showed that the questions were clear and that there was a high level of agreement among the respondents. We worked to ensure not only linguistic equivalence but mainly conceptual and psychometric equivalence. The conceptual equivalence was obtained by consulting experts, and the psychometric equivalence was obtained through the revalidation of the instrument and the analysis of the psychometric properties. The translation process followed the standard procedures according to international guidelines [10].

Procedures for Data Analysis

As stated by Coutinho, psychometric properties are the most appropriate procedures to guarantee the informative quality of the data [8]. Therefore, these properties of the instrument were assessed based on reliability and validity studies. The reliability was determined by the internal consistency of the items, based on the Pearson’s correlation coefficient of the different items with the overall score and the Cronbach’s alpha coefficient. The quality of the local adjustment was assessed through factor loadings (λ) and individual item reliability (δ). The value references for factor saturation and individual reliability were higher than 0.50 and 0.25, respectively [11]. For the analysis of validity, we used exploratory factor analysis and the validity construct [8] which consists of a technique of exploratory data analysis whose purpose seeks to discover and analyze the structure of a set of interrelated variables to build a measurement scale [11]. Among the most used methods is the “principal components method” sequenced by the orthogonal varimax rotation method. To define the factor, we considered the items with “r” saturations higher than or equal to 0.40, and to determine the factors to be retained, we consider the eigenvalues higher than 1 (one) and the scree plot. We used IBM SPSS software version 25 for Windows for data processing.

To evaluate the quality of adjustment of the measurement model, we used structural equation modeling (SEM) with AMOS software version 25 to validate the KIT TOOL-S2 TEXT Multiprofessional Guide WHO questionnaire for Portuguese healthcare professionals. SEM is a widely used multivariate statistical approach for validating scales and questionnaires, allowing the assessment of relationships between latent and observed variables as well as the adequacy of theoretical models to data [12, 13]. The AMOS software is a powerful and flexible tool for SEM analysis, offering advanced features for modeling, estimation, and diagnosing potential issues in models [14, 15]. By applying SEM and AMOS in this study, we aim to strengthen our analyses and ensure the validity and reliability of the KIT TOOL-S2 TEXT Multiprofessional Guide WHO questionnaire in the context of Portuguese healthcare professionals. Our approach is supported by extensive literature that highlights the importance and applicability of SEM and AMOS in scale and questionnaire validation studies across various fields, including healthcare. The estimation method used was maximum likelihood.

In analyzing the data, the most appropriate statistical procedures were considered, namely: (i) item sensitivity as evaluated by the skewness (Sk) and kurtosis (Ku) coefficients eliminating all items that present absolute values of Sk higher than 3 and of Ku higher than 7; (ii) validity of the construct as assessed by factor validity, convergent validity, and discriminant validity (DV); (iii) quality of the overall fit of the factor model, performed according to the goodness-of-fit indexes. In this study, the ratio between χ2 and degrees of freedom (χ2/df) was acceptable when the value was less than 5; the smaller the root mean square residual (RMR), the standardized root mean square residual (SRMR), and the root mean square error of approximation (RMSEA), the better the fit. A value of less than 0.08 is generally considered a good fit; for the goodness-of-fit index (GFI) and the Comparative Fit Index (CFI) values above 0.90 are recommended to be considered good fits; (iv) quality of local fit, as assessed by factor loadings (saturations) and individual item reliability. The reference values are 0.5 and 0.25, respectively. In initial studies, the former can be lowered to 0.40 or close to it. Model fitting was performed from the modification indexes by changing trajectories with modification indexes greater than 11 proposed by AMOS.

The reliability of the construct was assessed by composite reliability (CR) which indicates the degree to which items are consistent manifestations of the latent factor. It is considered that a CR ≥0.7 indicates appropriate reliability of the construct.

The convergent validity of each factor was assessed by the mean extracted variance (MEV). Convergent validity is considered to exist when the MEV is higher than 0.5 [11].

The DV of the factors is obtained by squaring the correlation between the factors. There is evidence of DV when the value obtained is less than the MEV for each factor. All factors were standardized by setting their variances at 1.00 (one) and correlated with each other.

Ethics Procedures

The study was approved by the Ethics Committee of the locus Hospital Center where it took place. The approval’s reference number is 1027, issued on October 7, 2019. All participants were provided with the necessary information about the study and asked to sign the informed consent form. Their anonymity and data confidentiality were ensured, and their autonomy was respected. They were also informed that their participation was totally free, that they could withdraw from the study at any time, and that by participating, they would not enjoy any gain/benefit nor expose themselves to any loss/harm.

The data regarding the statistical analysis of each item, including means and standard deviations, the correlation coefficients between each item and the overall value, as well as the Cronbach’s alpha coefficient without the item, were comprehensively evaluated. The mean indices for the items were generally well centered, ranging from 2.91 for items 7 and 12 to 4.34 for item 22. However, item 14, “It is easier to find someone to blame instead of focusing on the causes of the error,” had a lower mean index than expected (mean = 2.4 ± 1.05).

The total item correlation coefficients indicated that items 8, 9, 10, and 11 should be eliminated due to values below 0.20, yet these items were retained to perform a confirmatory factorial analysis and since the global alpha coefficient would remain unchanged. Interestingly, item 14 demonstrated a negative correlation, necessitating its inversion.

The obtained Cronbach’s alpha coefficients for the different items varied between weak and reasonable. Despite this variation, the overall alpha was robust, standing at 0.759. This comprehensive assessment allowed for nuanced understanding and subsequent decision-making based on the individual item performance and overall scale reliability.

Since it was assumed that a similar structure to the original scale would be found, all items were submitted to exploratory factorial analysis using the procedures described in the methodology. The combination of the results obtained with the Kaiser-Meyer-Olkin criterion (KMO = 0.760), which indicates the adequacy of the sample, and the results of Bartlett’s test of sphericity (χ2 = 2478.100; p = 0.000), which tests the hypothesis that the variables are not correlated, led to proceeding with the exploratory factorial analysis. Initially, seven factors were extracted which, together, accounted for 64.13% of the variability. However, the sedimentation or slope plot “Scree plot” pointed to the retention of four factors according to the inflection point of the curve.

The factor analysis was thus forced to four factors, which resulted in a percentage of variance of 49.08%. Factor 1, designated Patient Error, and Safety was composed of error items 1, 2, 3, 4, 5, 6, and 7 and accounts for 22.97% of the variability; factor 2 – Health System Safety – is composed of items 1, 2, 3, 4, and explains 10.87%; factor 3 – Personal Influence on Safety – has the items 1, 2, 3, 4, 6, and 7 and justifies 8.21%; and finally factor 4 – Personal Attitudes towards Patient Safety – comprises the items 1, 2, 3, 4, 5, and the item 5 of personal information of the original scale, with a variability of 7.027%.

This tetra factorial structure was submitted to confirmatory factor analysis. After analyzing the sensitivity of the items, it was found that all of them had minimum and maximum values ranging from 1 to 5, except for error 7 (“The role of health organizations (e.g., hospitals, general practitioners) in error reporting”?), and showed absolute values of Sk and Ku reference values from below 3–7, respectively.

Table 1 shows the trajectories, critical rations, and lambda coefficients obtained. The first column of the table lists the estimated standardized regression weights, which indicate the strength and direction of the relationship between the latent variables and their corresponding observed variables or indicators. The values of the standardized regression weights range from 0 to 1, with higher values indicating stronger relationships.

Table 1.

Trajectories, critical ratios, and lambda coefficients

EstimateSECRpλ
Error 2 <--- F1 1.000    0.812 
Error 1 <--- F1 1.022 0.071 14.334 *** 0.774 
Error 3 <--- F1 0.966 0.061 15.846 *** 0.791 
Error 4 <--- F1 1.064 0.078 13.591 *** 0.770 
Error 6 <--- F1 0.994 0.090 11.007 *** 0.671 
Error 5 <--- F1 1.057 0.091 11.567 *** 0.702 
Error 7 <--- F1 0.949 0.219 4.342 *** 0.265 
Personal_actv 2 <--- F2 1.000    0.800 
Personal_actv3 <--- F2 0.742 0.065 11.390 *** 0.720 
Personal_actv 4 <--- F2 0.989 0.091 10.826 *** 0.679 
Personal_actv 1 <--- F2 0.701 0.063 11.090 *** 0.675 
Personal_info 1 <--- F3 1.000    0.730 
Personal_info 7 <--- F3 0.567 0.064 8.856 *** 0.680 
Personal_info 14 <--- F3 0.551 0.089 6.203 *** 0.462 
Info2_inv <--- F3 0.748 0.118 6.340 *** 0.444 
Personal_info 6 <--- F3 0.540 0.113 4.791 *** 0.334 
Personal_info 13 <--- F3 0.504 0.123 4.112 *** 0.273 
s1_inv <--- F4 1.000    0.531 
s3_inv <--- F4 0.800 0.108 7.389 *** 0.493 
Seg_sist_s2 <--- F4 0.378 0.093 4.076 *** 0.432 
Seg_sist_s5 <--- F4 0.451 0.107 4.213 *** 0.551 
Seg_sist_s4 <--- F4 0.436 0.109 4.002 *** 0.426 
Info_pessoal 5 <--- F3 0.435 0.100 4.339 *** 0.402 
EstimateSECRpλ
Error 2 <--- F1 1.000    0.812 
Error 1 <--- F1 1.022 0.071 14.334 *** 0.774 
Error 3 <--- F1 0.966 0.061 15.846 *** 0.791 
Error 4 <--- F1 1.064 0.078 13.591 *** 0.770 
Error 6 <--- F1 0.994 0.090 11.007 *** 0.671 
Error 5 <--- F1 1.057 0.091 11.567 *** 0.702 
Error 7 <--- F1 0.949 0.219 4.342 *** 0.265 
Personal_actv 2 <--- F2 1.000    0.800 
Personal_actv3 <--- F2 0.742 0.065 11.390 *** 0.720 
Personal_actv 4 <--- F2 0.989 0.091 10.826 *** 0.679 
Personal_actv 1 <--- F2 0.701 0.063 11.090 *** 0.675 
Personal_info 1 <--- F3 1.000    0.730 
Personal_info 7 <--- F3 0.567 0.064 8.856 *** 0.680 
Personal_info 14 <--- F3 0.551 0.089 6.203 *** 0.462 
Info2_inv <--- F3 0.748 0.118 6.340 *** 0.444 
Personal_info 6 <--- F3 0.540 0.113 4.791 *** 0.334 
Personal_info 13 <--- F3 0.504 0.123 4.112 *** 0.273 
s1_inv <--- F4 1.000    0.531 
s3_inv <--- F4 0.800 0.108 7.389 *** 0.493 
Seg_sist_s2 <--- F4 0.378 0.093 4.076 *** 0.432 
Seg_sist_s5 <--- F4 0.451 0.107 4.213 *** 0.551 
Seg_sist_s4 <--- F4 0.436 0.109 4.002 *** 0.426 
Info_pessoal 5 <--- F3 0.435 0.100 4.339 *** 0.402 

We note that the trajectories of the items for their corresponding factors are statistically significant, which leads to their maintenance. However, the items that correspond to “error” 7 (“What is the role of healthcare organizations (e.g., hospitals, general practitioners) in error reporting”?) of factor 1 – Error and Patient Safety – and items 3 (“I feel confident talking to someone who is showing a lack of concern for patient safety”) and 6 (“I believe that filling out reporting forms will help improve patient safety”) of factor 3 – Personal Influence on Safety (“info_personal”) are eliminated in further analysis models because their saturations are less than 0.40. We note that, since this is a preliminary study, factor loadings equal to or greater than 0.40 are considered.

Integrating descriptive and inferential statistics with SEM allows for a comprehensive understanding of the relationships among variables and the underlying structure of the dataset, providing valuable insights for data interpretation and hypothesis testing. All items have saturations above the reference values in the four subscales except for the items previously mentioned, which demonstrates the relevance of the factor to predict the items. Nevertheless, the overall goodness-of-fit indexes only proved to be adequate for χ2/df = 3.631 and RMR = 0.064, acceptable for the GFI = 0.807; RMSEA = 0.094; and SRMR = 0.083, and inadequate for CFI = 0.743.

For improvement purposes, we adjusted the goodness-of-fit indexes of the model based on the modification indexes proposed by AMOS. To this end, some errors were correlated, and items with multicollinearity problems and with saturations below 0.40 were eliminated. We verified that factor 1 was composed of 6 items configuring correlations between model errors 1 and 3 and 5 and 6. Factor 2, factor 3, and factor 4 comprise four items each. The overall adjustment became adequate for χ2/df = 2.602; RMSEA = 0.073; RMR = 0.038; and SRMR = 0.062, and the indexes (GFI = 0.895; CFI = 0.898) are at the threshold of adequate.

In summary, it can be stated that after the hypothesized structure was submitted to confirmatory factor analysis, and after refinement, the following items were eliminated due to multicollinearity problems: item 7 (“The role of health organizations (e.g., hospitals, general practitioners) in error reporting”?) from factor 1 – Error and Patient Safety; item 5 (“The healthcare team receives training in patient safety”) from factor 2 – Health System Safety; and items 3, 4, 5, and 6 (“I feel confident talking to someone who is showing a lack of concern for patient safety,” “I know how to talk to people who have made an error,” I am always able to ensure that patient safety is not compromised,” and “I believe that filling out report sheets will help improve patient safety,” respectively) from factor 3 – Personal Influence on Safety” from factor 3 – Personal Influence on Safety. It should be noted that item 5 (“I am always able to ensure that patient safety is not compromised”) of factor 3 – Personal Influence on Safety became part of factor 4 – Personal Attitudes towards Patient Safety.

The moderate correlations between the various subscales presuppose the existence of a final second-order model, which was designated as the original scale, KIT TOOL-S2. We found that the highest correlation is obtained with factor 1 “error” which explains 60% of the variability and the lowest with factor 2 “safety system” with 22.0% of variability. In this model, the overall adjustment quality values showed slight oscillations. However, the factorial validity (χ2/df = 2.614; RMSEA = 0.073; RMR = 0.040; SRMR = 0.065; GFI = 0.892; CFI = 0.896) was not affected (Fig. 1).

Fig. 1.

Second-order model.

Fig. 1.

Second-order model.

Close modal

Table 2 summarizes the global adjustment indexes obtained in the different models. Note that the initial model has the least robust goodness-of-fit indexes, but after model refinement and item elimination, they became adequate.

Table 2.

Goodness-of-fit indexes in the different models

Modelχ2/dfGFICFIRMSEARMRSRMR
Initial model 3.631 0.807 0.743 0.094 0.064 0.083 
Model with eliminated modification item 2.602 0.895 0.898 0.073 0.038 0.062 
Second-order model 2.614 0.892 0.896 0.073 0.040 0.065 
Modelχ2/dfGFICFIRMSEARMRSRMR
Initial model 3.631 0.807 0.743 0.094 0.064 0.083 
Model with eliminated modification item 2.602 0.895 0.898 0.073 0.038 0.062 
Second-order model 2.614 0.892 0.896 0.073 0.040 0.065 

Table 3 presents the results of CR, MEV, and DV. The CR in factors 1 and 2 can be classified as adequate since it presents values higher than the reference (0.70), but in factors 3 and 4, the CR is acceptable since it is slightly lower than recommended. As for the MEV, the log indicates that the validity is convergent for factors 1 and 2, and divergent for factors 2 and 3. However, the existence of DV is evident among all factors since the squared correlational values are lower than the MEV. The stratified CR is found to be adequate (0.928), although the MEV, with a value of 0.430, is slightly lower than recommended.

Table 3.

CR, mean extracted variance (MEV) and DV of the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide”

FactorsCRMEVDV
f1 vs. f2f1 vs. f3f1 vs. f4f2 vs. f3f2 vs. f4f3 vs. f4
Factor 2 – Health System Safety and Security 0.873 0.537 0.152 0.136 0.184    
Factor 3 – Personal Influence on Safety and Security 0.811 0.518    0.078 0.012  
Factor 4 – Personal Attitudes towards Patient Safety 0.677 0.358      0.136 
Total factor 0.568 0.252       
FactorsCRMEVDV
f1 vs. f2f1 vs. f3f1 vs. f4f2 vs. f3f2 vs. f4f3 vs. f4
Factor 2 – Health System Safety and Security 0.873 0.537 0.152 0.136 0.184    
Factor 3 – Personal Influence on Safety and Security 0.811 0.518    0.078 0.012  
Factor 4 – Personal Attitudes towards Patient Safety 0.677 0.358      0.136 
Total factor 0.568 0.252       

Stratified CR 0.928, MEV= 0.430.

In summary, the values obtained indicate that the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” may be a valuable resource for studying the literacy of health professionals on care safety.

Finally, reference is made to Pearson’s correlation matrix between the different subscales and their overall value. The existence of positive and significantly correlated values is denoted, which allows stating that the increase or decrease in the scores of one of the subscales is associated with increases or decreases in the scores of the other subscales with which they correlate. With the global factor and the different dimensions, the correlation coefficients are higher, revealing variability above 29.0% (see Table 4).

Table 4.

Pearson’s correlation matrix between the factors

SubscalesFactor 1 – Error and Patient SafetyFactor 2 – Health System Safety and SecurityFactor 3 – Personal Influence on Safety and SecurityFactor 4 – Personal Attitudes towards Patient Safety
Factor 2 – Health System Safety and Security 0.293**   
Factor 3 – Personal Influence on Safety and Security 0.313** 0.221**  
Factor 4 – Personal Attitudes towards Patient Safety 0.321** 0.660 0.212** 
Factor total 0.813** 0.605** 0.631** 0.544** 
SubscalesFactor 1 – Error and Patient SafetyFactor 2 – Health System Safety and SecurityFactor 3 – Personal Influence on Safety and SecurityFactor 4 – Personal Attitudes towards Patient Safety
Factor 2 – Health System Safety and Security 0.293**   
Factor 3 – Personal Influence on Safety and Security 0.313** 0.221**  
Factor 4 – Personal Attitudes towards Patient Safety 0.321** 0.660 0.212** 
Factor total 0.813** 0.605** 0.631** 0.544** 

**p < 0.001.

The Multiprofessional Guide aims to encourage both health educators and students around the world to share common understanding, knowledge, and skills on patient safety to ensure the delivery of high-quality, safe services as members of local teams operating within a safety culture and globally connected members by advancing patient safety definitions, content, and solutions. The content of the Multiprofessional Guide considers the wide variety of contexts in which healthcare educators and students teach and learn. The World Health Organization’s Expert Working Group of more than 50 international experts was involved in the development of the curriculum to ensure its cultural appropriateness and global reach. Teaching and assessment strategies are designed to consider both the diversity of available resources and the cultural differences that may affect the learning environment. As a freely available resource for all healthcare systems worldwide, the Multiprofessional Guide reinforces the ethos that safety is all patients’ right and not a “luxury” confined to better resourced healthcare systems [16].

Although the importance of patient safety training has been widely recognized for over a decade, it remains underutilized and undervalued in many countries [17]. Therefore, the World Health Organization developed the Multi-professional Patient Safety Curriculum Guide to provide schools with the requirements and tools to incorporate patient safety into curricula. It was validated by Farley et al. [17] in a study with 12 participating schools in six World Health Organization regions evaluating its effectiveness in teaching about patient safety to students in a variety of settings. The results of this study indicate that all participants were positively satisfied with the Multiprofessional Guide, stating that it emphasizes patient safety topics of universal importance, that it was culturally appropriate for their countries, and that they gave it credibility and created a patient safety focus in their schools. It was also shown that students’ perceptions and attitudes about patient safety improved substantially during the on-site application, and their knowledge of the topics they were taught doubled from 10.7% to 20.8% correct answers.

In view of the above, the present study aimed at translating into European Portuguese, cross-culturally adapting, and validating for Portugal the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” of the World Health Organization. Translations and back translations were carried out since the original instrument is written in English and required a panel of experts for semantic corrections, especially in terms of sentence construction and replacement of some terms adapted for European Portuguese. The results were obtained from a sample of 300 health professionals working in a hospital center in the central region of the country.

The translation and cultural adaptation of assessment scales should follow the “emic-etic” paradigm, that is, the term “emic” refers to looking at the phenomenon from the very context or culture in which it occurs, while the term “etic” refers to generalizing the observed phenomenon for the purpose of comparison across different cultures [8]. In this context, the translation of a scale requires linguistic caution, to the extent that certain terms may have different meanings and specificities intrinsic to each language. On the other hand, semantic validation is essential to certify that the instrument is understandable by the target population. Thus, for the translation, cross-cultural adaptation into Portuguese, and back translation into English of the scale “KIT TOOL-S2 TEXT”, we chose researchers with knowledge in the area. The back translation was essential to ensure that the contents, from the conceptual point of view, did not have to be modified in the translation phase, and this was the key to establishing the semantic equivalence of the scale.

The validity of an instrument is probably the best indicator of its quality. It indicates whether the instrument measures what it is intended to measure, that is, whether the results obtained reflect or translate reality, in the sense of accepting them as indisputable facts. Thus, validity is a complement of reliability because before we can ensure the validity of the instrument, we must ensure its reliability. The literature indicates that the validity of an instrument cannot be taken as an intrinsic characteristic but as a characteristic of the instrument itself when applied to a sample. Thus, it is inferred that the characteristics of the study population can directly influence the structure of the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” of the World Health Organization. Our exploratory factor analysis of this scale resulted in a tetra factor structure that accounts for 43.0% of the variance and with an overall Cronbach’s alpha coefficient of 0.759. The overall goodness-of-fit indexes were found to be reliable, confirming the quality of the model fit to the empirical data (χ2/df = 2.602; GFI = 0.895; CFI = 0.898; RMR = 0.038; RMSEA = 0.073) and SRMR = 0.062. The evidence for the validity of the scale was presented, showing all the possibilities of its use in the Portuguese context, allowing the establishment of bases for the construction of a new instrument that assesses the literacy of health professionals on care safety.

The World Health Organization’s Multiprofessional Guide has been instrumental in shaping patient safety training initiatives worldwide. It has inspired various frameworks, including the Australian Patient Safety Education Framework and the Canadian Framework, which identified main areas of activity contributing to patient safety. The Multiprofessional Guide aims to promote a common understanding of patient safety among health educators and students worldwide to ensure safe, high-quality healthcare services. The curriculum considers cultural appropriateness and global reach and is designed to consider both the diversity of available resources and the cultural differences that may affect the learning environment. Although patient safety training has been recognized for over a decade, it remains underutilized and undervalued in many countries. To address this, the World Health Organization developed the Multiprofessional Patient Safety Curriculum Guide to provide schools with requirements and tools to incorporate patient safety into their curricula. The present study aimed to cross-culturally adapt and validate the scale “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” for Portugal. The validity of the instrument was established by exploring its factor structure, accounting for 43.0% of the variance, and with an overall Cronbach’s alpha coefficient of 0.759. The goodness-of-fit indexes confirmed the quality of the model fit to the empirical data. Overall, this study contributes to the establishment of a new instrument that assesses the literacy of health professionals on patient safety in the Portuguese context.

This study’s investigation into the “KIT TOOL-S2 TEXT Questionnaire to Assess the Implementation of the Multiprofessional Guide” scale underscores its efficacy as a psychometric tool. The scale’s satisfactory validation underscores its utility in assessing healthcare professionals’ literacy regarding patient and healthcare safety. Our findings highlight the scale’s robust factorial structure and its ability to accurately represent the underlying constructs of interest while maintaining its psychometric integrity.

The confirmatory analyses conducted support the scale’s construct validity, suggesting its applicability in diverse healthcare settings. This validation endorses the scale as a reliable instrument for future research endeavors focused on patient safety.

The “KIT TOOL-S2 TEXT” scale offers a promising avenue for developing and evaluating interventions aimed at enhancing patient safety attitudes and behaviors among healthcare professionals. Its application in longitudinal studies presents an opportunity to observe temporal changes in these attitudes and behaviors. Additionally, it facilitates the assessment of the impact of patient safety training on healthcare outcomes, an area ripe for exploration in future research endeavors.

In summary, the validation of this scale marks a significant contribution to the field of patient safety and healthcare quality. It opens new possibilities for research focused on improving patient care and safety across various healthcare contexts, both within Portugal and internationally.

  • 1.

    The KIT TOOL-S2 TEXT questionnaire is a verified and dependable tool that measures Portuguese healthcare professionals’ understanding of patient safety, encompassing significant facets of healthcare safety.

  • 2.

    This tool possesses the capability to detect changes in attitudes toward safety over a period; hence, it becomes an integral component in assessing the influence of safety training programs on patient outcomes.

  • 3.

    The application of this questionnaire in forthcoming research and in the practical field could potentially lead to substantial advancements in patient care and safety.

This research was conducted following the ethical standards and guidelines of our institution. All procedures involving human participants were reviewed and approved by the Institutional Review Board (IRB) at our institution, with reference number 04/21/10/2019. Informed consent was obtained from all individual participants involved in the study. Adequate measures were taken to ensure the confidentiality and privacy of the participants. The data collected have been used solely for this research study.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

We affirm that all authors have made substantial contributions to the accomplishment of this manuscript. All authors were involved in the conception and design of the study, the collection and analysis of data, the writing of the manuscript, and the critical review of its intellectual content. Moreover, all authors have approved the final version to be published and agreed to be accountable for all aspects of the work, ensuring that questions related to the accuracy or completeness of any part of the work are appropriately investigated and resolved.

This study adheres to the principles of data transparency and accessibility, in alignment with the journal’s data sharing policies. We hereby declare that the datasets underpinning the conclusions of this manuscript are available without undue restriction, within the ethical and legal boundaries. Data access is granted upon direct request to the primary authors, who will provide the data responsibly following ethical research practices. Where legal or ethical limitations preclude public data sharing, these are explicitly stated, detailing the conditions under which data access may be permitted. The research team ensures the privacy and confidentiality of participants in compliance with institutional guidelines and regulatory standards for data protection.

1.
Manzanera
R
,
Mira
JJ
,
Plana
M
,
Moya
D
,
Guilabert
M
,
Ortner
J
.
Patient safety culture in mutual insurance companies in Spain
.
J Patient Saf
.
2021
;
17
(
3
):
175
81
.
2.
Schouten
B
,
Merten
H
,
Spreeuwenberg
PMM
,
Nanayakkara
PWB
,
Wagner
C
.
The incidence and preventability of adverse events in older acutely admitted patients: a longitudinal study with 4292 patient records
.
J Patient Saf
.
2021
;
17
(
3
):
166
73
.
3.
World Health Organization
.
Fifty-fifth World Health Assembly, Geneva, 13-18 May 2002: resolutions and decisions: annexes
.
Geneva
:
World Health Organization
;
2002
.
4.
Cauduro
GMR
,
Magnago
TSBDS
,
Andolhe
R
,
Lanes
TC
,
Ongaro
JD
.
Patient safety in the understanding of health care students
.
Rev Gaucha Enferm
.
2017
;
38
(
2
):
e64818
.
5.
World Health Organization
.
WHO patient safety curriculum guide: multi-professional edition
.
Geneva
:
World Health Organization
;
2011
.
6.
Walton
MM
,
Shaw
T
,
Barnet
S
,
Ross
J
.
Developing a national patient safety education framework for Australia
.
Qual Saf Health Care
.
2006
;
15
(
6
):
437
42
.
7.
Canadian Patient Safety Institute
.
The safety competencies: enhancing patient safety across the health professions
.
Edmonton
:
Canadian Patient Safety Institute
;
2009
.
8.
Coutinho
CP
.
Metodologias de investigação em ciências sociais e humanas: teoria e prática
.
Coimbra
:
Almedina
;
2021
.
9.
Beaton
DE
,
Bombardier
C
,
Guillemin
F
,
Ferraz
MB
.
Guidelines for the process of cross-cultural adaptation of self-report measures
.
Spine
.
2000
;
25
(
24
):
3186
91
.
10.
Muñiz
J
,
Elosua
P
,
Hambleton
RK
;
International Test Commission
.
[International Test Commission Guidelines for test translation and adaptation: second edition]
.
Psicothema
.
2013
;
25
(
2
):
151
7
.
11.
Marôco
J
.
Análise estatística com o SPSS Statistics
. 6th ed.
Pêro Pinheiro
: ReportNumber;
2018
.
12.
Hair
JF
,
Black
WC
,
Babin
BJ
,
Anderson
RE
.
Multivariate data analysis
. 8th ed.
Hampshire
:
Cengage Learning
;
2018
.
13.
Kline
RB
.
Principles and practice of structural equation modeling
. 4th ed.
New York, NY
:
The Guilford Press
;
2015
.
14.
Byrne
BM
.
Structural equation modeling with AMOS: basic concepts, applications, and programming
. 3rd ed.
New York, NY
:
Routledge
;
2016
.
15.
Schumacker
RE
,
Lomax
RG
.
A beginner’s guide to structural equation modeling
. 4th ed.
New York, NY
:
Routledge
;
2016
.
16.
Leotsakos
A
,
Ardolino
A
,
Cheung
R
,
Zheng
H
,
Barraclough
B
,
Walton
M
.
Educating future leaders in patient safety
.
J Multidiscip Healthc
.
2014
;
7
:
381
8
.
17.
Farley
D
,
Zheng
H
,
Rousi
E
,
Leotsakos
A
.
Field test of the World Health Organization multi-professional patient safety curriculum guide
.
PLoS One
.
2015
;
10
(
9
):
e0138510
.