Abstract
Objectives: This study aimed to systematically identify and evaluate the readability and document complexity of currently available family history tools for the general public. Methods: Three steps were undertaken to identify family history tools for evaluation: (a) Internet searches, (b) expert consultation, and (c) literature searches. Tools identified were assessed for readability using the Simple Measure of Gobbledygook (SMOG) readability formula. The complexity of documents (i.e., forms collecting family history information) was assessed using the PMOSE/IKIRSCH document readability formula. Results: A total of 78 tools were identified, 47 of which met the criteria for inclusion. SMOG reading grade levels for multimedia-based tools ranged from 10.1 to 18.3, with an average score of 13.6. For print-based tools, SMOG ranged from 8.7 to 14.1, with an average score of 12.0. Document complexity ranged from very low complexity (level 1 proficiency) to high complexity (level 4 proficiency). Conclusion: The majority of tools are written at a reading grade level that is beyond the 8th grade average reading level in the United States. The lack of family history tools that are easy to read or use may compromise their potential effectiveness in identifying individuals at increased risk for chronic diseases in the general population.
Introduction
Health literacy has been defined as ‘the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions’ [1]. Health literacy consists of a constellation of skills, which include the ability to perform basic reading tasks and comprehend text, such as written instructions (prose literacy) as well as the ability to locate information and use documents such as medical forms, charts, and tables (document literacy) [2,3]. Approximately 90 million adults in the United States have limited literacy skills, or difficulty understanding health information [1].
Evidence for the negative impact of poor health literacy is accumulating. Adults with limited health literacy have less knowledge of disease management and of health-promoting behaviors, are less likely to use preventive services, and report poorer health status [1,2]. Not surprising, health literacy levels are lower among minority populations, those with limited English proficiency, the elderly, poor, and less educated [4,5].
Related to health literacy, the issue of genetic literacy has received much attention in recent years. Genetic literacy pertains to one’s capacity to understand and apply genetic information to make appropriate health and lifestyle decisions. Advances in genomics have led to exciting possibilities for the future of medicine, health care, and public health initiatives. The growing understanding of how genetics play a role in chronic diseases has spurred an ambitious vision of genomics for medical science emphasizing the potential health benefits for all [6]. Preliminary studies, however, have found that the general public has limited knowledge of basic concepts related to genetics [7,8]. As such, genomic advances have prompted concerns about the general public’s readiness to process and utilize the knowledge gained to make informed decisions about their health [8].
To facilitate the use of genomic information for health promotion and disease prevention, several government agencies, including the Office of the Surgeon General and the Centers for Disease Control and Prevention (CDC), have advocated for the use of family history assessment as a tool to increase genetic literacy [9,10,11]. Family history offers an ideal proxy to assess genomic risk and is the simplest applied ‘genomic tool’ available [9]. An individual’s family history is more than genetics alone, but rather reflects the consequences of genetic susceptibilities, shared environmental and cultural factors, and common behaviors [10]. All of these factors are important when estimating disease risk.
In spite of public health communication efforts that have been undertaken to promote increased awareness of family history, several factors may severely limit the impact of these endeavors. Complex health information that is traditionally conveyed on family history tools (e.g., genetic predisposition, heredity, blood relative, index patient, pedigree) are inherently difficult to understand and may not lend themselves to easy simplification. Moreover, little is known about how the public responds to family history tools. For example, it is unclear whether the public is able to access available tools, understand how to use them, and apply that information in terms of taking health protective actions [10,11], which may compromise their potential effectiveness. Readability, comprehension and cultural/linguistic appropriatenessof available family history tools are rarely, if ever, assessed. Underserved populations, in particular, may face distinctive literacy-related barriers for the effective use of such tools because of less access to culturally and linguistically appropriate health information. By not addressing literacy-related barriers, public health efforts to encourage the use of family history as a tool for prevention may inadvertently serve to magnify health disparities [12].
To better delineate the barriers to the effective use of family health history tools, the present study was designed to systematically identify and evaluate the readability and document complexity of currently available family history tools for the general public.
Materials and Methods
Identification of Family History Tools
We undertook 3 steps to systematically identify existing family history tools to be included in the present study. First, we conducted general Internet searches using search terms such as ‘family history tool’, ‘family health history’, ‘family history risk assessment’ as well as targeted searches on government websites focused on disseminating family history information: CDC (www.cdc.gov/genomics/public/famhix/links.htm) and National Human Genome Research Institute (www.genome.gov/11510372). Second, upon generating an initial list of print and multimedia-based tools, we contacted a total of 9 experts in family history or genetics for feedback and to identify tools that were not included on our original list. These were identified by the study team because of their prior work either in promoting the importance of family history or developing family history tools. Experts identified were from various backgrounds including public health professionals, genetic counselors, nurses, and physicians. Third, we examined the published literature using similar search terms as noted earlier for other family history tools to be included. The reference section of published articles was also examined for additional tools.
Our definition of what constituted a ‘tool’ per se was broad in scope. Criteria were established by the research team to determine what constituted an appropriate tool for literacy assessment, including (a) availability and appropriateness for general public (vs. targeting provider), (b) focus on family history assessment and not behavior risk factor assessment (e.g., traditional health risk appraisals), and (c) availability of an English language version. Tools that did not meet these criteria were not included in the literacy assessment (see complete list of tools identified in table 1). For example, questionnaires or single-page forms used to collect family history information were included in our literacy assessment(s). We also included web sites that provided information on the importance of knowing one’s family health history and detailed how to collect the information (e.g., instructions for which family members to include, information to include). However, we limited these web sites that provided family history guidelines to those representing national organizations (e.g., National Society of Genetic Counselors) or that were referenced by other government websites such as the CDC (e.g., Mayo Clinic).
In contrast, we did not include family history tools that were embedded in existing electronic medical record systems of a particular medical or hospital establishment even though several exist (e.g., Hughes riskApps [13], Centricity™ EMR) or are in development (e.g., Fox Chase Cancer Center [14], Intermountain Healthcare [15]). Other tools that were identified but no longer accessible either online or through the developer were also not included. In addition, we did not include medical research surveys that included a family history component (e.g., National Health Interview Survey – NHIS; National Health and Nutrition Examination Survey – NHANES). Publications that discussed a family history tool but either (a) did not present the actual tool (i.e., in appendix) or (b) pertained to family history questionnaires administered via face-to-face or telephone interviewers, were also not included. Family history tools provided through commercial companies that required a fee to access were also not included in our assessment.
All tools were identified and assessed over the period of January 2008 to June 2009.
Literacy Assessment
Family history tools exist in many different formats and as such, we undertook different literacy assessment approaches to accommodate the various formats. We assessed reading grade level for all family history tools that contained sentence-type text. For all print-based tools that contained questionnaires, charts, or family history tree diagrams to be ‘filled in’, we assessed document complexity. As a result, tools were assessed for either readability or document complexity, although some tools (N = 16) were assessed for both.
To assess readability, we used the Simple Measure of Gobbledygook (SMOG) readability formula [16] calculated through 2 methods: (1)‘Readability Plus’ (a computer software program designed by Micro Power & Light Co. to assess the reading level of materials) and (2) by hand – as a quality check to confirm results from the software program. The SMOG formula calculates for the reading grade level based on the number of polysyllabic words, or words with 3 or more syllables, for every 30 sentences. The SMOG reading grade level assumes 100% comprehension and is a more stringent criterion for comprehension compared to other readability formulas available [17]. We chose this formula because of its robustness and accuracy, strict criterion for comprehension, and widespread use in prior published literature assessing health education materials [17,18,19].
To prepare materials for readability assessments, we omitted numbered and bulleted lists, removed any punctuation including periods, colons, semi-colons, exclamation and question marks not indicating the end of a sentence, and omitted headings and sub-headings. Using ‘Readability Plus’ we took samples of approximately 300 words from each document (100 from 3 different sections). Text samples were taken from the beginning, middle and end of each family history tool. For tools that contained 300 words or less, we assessed all the available text. For hand calculations of SMOG, 10 sentences were taken from the beginning, middle, and end of every tool for a total of 30 sentences. For tools with less than 30 sentences, all of the material was used.
Some of the interactive family history tools calculated risk assessments based on family history and provided detailed summary reports based on risk level. For these tools, we separately assessed and reported the readability assessments for the instructions (i.e., front matter including description of tool, purpose, and instructions on how to complete) and actual tool for collecting family history information versus the summary report generated based on the information provided by an individual. This was undertaken to allow for more consistent comparison across tools and to capture any differences in language used between the 2 sections.
Document complexity was assessed by hand on all print-based family history tools that included ‘fill in’ sections, including questionnaires, charts, or family tree diagrams. Often, print-based family history tools contained insufficient text for traditional readability assessments (N = 9). Document complexity was particularly useful for assessing the tools that did not have sufficient text to perform SMOG readability assessments and allowed for another form of literacy comparisons across available tools. To assess document complexity, we used the PMOSE/IKIRSCH document readability formula [20]. Assessment of document complexity entails identifying and scoring documents based on the complexity of their structure (e.g., simple list, nested lists, etc) and their density (e.g., number of labels and items on the list). The majority of tools assessed for document complexity were print-based questionnaires or family history tree forms that are used to collect family history information (with the exception of one multimedia tool that consisted of a single screen or ‘page’ to complete family history information). Scoring on the PMOSE/IKIRSCH ranges from Level 1 proficiency – very low complexity (range including grade 4 equivalent to less than 8 years of schooling) to Level 5 proficiency – very high complexity (range including 16 years of schooling to more advanced postgraduate degree).
Results
A total of 78 tools were identified, 47 of which met the criteria for inclusion (table 1). Overall, 13 of the family history tools were multimedia-based, 5 were websites that detailed how to collect family history information, and 29 were print-based. A total of 8 of the multimedia-based tools were capable of generating summary assessment reports (e.g., risk assessments and/or preventive messages) and, as such, SMOG was assessed separately for the instructions/tool versus the summary report.
Several tools included in our assessment were developed for specific disease conditions, especially cancer (N = 13, tool #1, 2, 10, 11, 12, 16, 50, 57, 60, 63, 64, 67, 77). Five tools focused on metabolic conditions including coronary heart disease, stroke, and diabetes (#28, 29, 30, 55, 69). Some tools were developed for specific target populations such as African Americans (N = 3, #24, 47, 55). Seven tools were available in Spanish (#18, 28, 35, 40, 52, 66, 76), whereas only two tools have been translated into multiple languages for use (#40, 66). Only 3 of the tools included in our assessment collected family history information and generated risk estimates for multiple diseases (#3, 15, 21).
A list of the tools assessed for readability is included in table 2, organized by type of tool (i.e., multimedia, website, print). Overall, SMOG reading grade levels for the multimedia-based tools ranged from 10.1 to 18.3, with an average score of 13.6 that is equivalent to a college level reading ability. Differences in SMOG reading grade level between the instructions/tool and summary reports ranged from 1 to 5 (i.e., less than 1 grade level to 5 grade levels). Websites that contained information on how to collect family history information contained text written at grade levels ranging from 11.1 to 16.1, with an average score of 12.9. For the print-based family history tools, we were able to conduct a SMOG assessment on 20 of them. Reading grade levels for these tools ranged from 8.7 to 14.1. The average score was 12.0, which is equivalent to a 12th grade or high school level reading ability.
A total of 25 family history tools included fill-in sections or questions that could be assessed for document complexity (table 3). Overall, document complexity for the tools ranged from level 1 proficiency to level 4 proficiency. Level 1 is equivalent to a range of 4th grade to less than 8th grade reading ability. Level 4 is equivalent to a range of 15 years of schooling to college degree reading ability. Almost half of the tools (48%) scored at level 2 proficiency, which is equivalent to a range of 8th grade to high school diploma reading ability. Over a quarter (28%) of the tools scored at level 4 proficiency and are considered to be highly complex.
Discussion
The overarching goal of this study was to systematically identify and evaluate family history tools available to the general public for readability and document complexity. Although readability assessments on either a single family history tool [21,22] or a group of familial cancer risk assessment tools [23] have previously been conducted, this is the first study to our knowledge that systematically conducts literacy assessments of available family history tools for the general public focused on all diseases to allow for comparison across tools. A better understanding of the readability and document complexity across family history tools may help health practitioners and public health educators to better chose a tool that is appropriate for the audience they are working with.
Overall, our findings suggest that most tools are written on average at a 12-13 grade level that is beyond the 8th grade average reading level in the United States. This is consistent with evaluations of patient education materials in general which are often written at a 10th grade reading level or higher [1,4,24], and raises concerns about whether patients and individuals in the general public understand the materials they are provided with. Developers of patient materials and education products are often faced with the challenge of producing materials at a low reading grade level despite the fact that the majority of individuals, including those who are highly educated, prefer simple materials [25]. However, the design and evaluation of simple and easy to use materials is an increasing necessity given the prevalence of limited health literacy in the U.S. population [1,26,27].
The family history tools assessed for document complexity varied widely in their scores, with tools ranging from very low to high complexity. These results highlight the need for the developers of family history questionnaires to take into consideration how information is structured and organized. Highly complex forms that have multiple subheadings and nested columns have a greater likelihood of confusing the respondent and may result in inaccurate reporting of important family history information (fig. 1 illustrating the difference between low versus high complexity forms). Although it is desirable for family history questionnaires to collect the most detailed and accurate information, developers should also take into consideration how the forms will be used. For example, if the goal of the family history questionnaire is to obtain a quick ‘screen’ of a patient’s family history that can be taken to a provider who can then prompt for more detailed information (e.g., age of diagnosis, other family members), then a simple tool may be all that is necessary to serve that purpose.
There were several limitations to the present study. First, the rapid proliferation of family history tools posed a challenge to this study as tools are constantly being developed, adapted, and modified on a regular basis. As such, the findings reported in this study are only accurate to the extent that the tools listed have not been modified since the time of submission (June 2009). Second, all the experts included in the study were based in American academic, non-profit, or government settings. Although efforts were made to identify family history tools from other countries including the United Kingdom (tool #19, 67, 68, 72, 73), Canada (#36, 54, 62), and Australia (#23, 57, 65, 74), the study likely has an American bias.
All the literacy assessments were performed by a single coder (R.E.G.), which serves as a limitation to this study. However, SMOG calculations conducted by hand were compared to the computer derived SMOG and were highly correlated (r = 0.95, p < 0.001). Moreover, extensive discussions occurred between the coder and the lead author when there were ambiguities to be resolved while coding family history tools using PMOSE/IKIRSCH.
Our study used the SMOG to assess readability and may have provided grade reading level estimates that are higher than estimates provided by other readability formulas (e.g., Fry, Fog, Flesch). We calculated readability using the Fry formula [28] on a subset of family history tools and noted that scores on average were one grade level lower than the SMOG results. Although researchers have noted that readability estimates using different formulas can vary widely for the same material [17], others have shown strong positive correlations between commonly used readability formulas [19]. In spite of potential discrepancies across readability formulas, our use of a single approach, nonetheless, enables us to compare across family history tools using the same criteria. Developers of family history tools may want to assess materials using multiple readability formulas to ensure a more reliable readability score and help to guide text revision until readability is at a suitable level [17].
All family history tools were assessed without any prior editing for jargon. For example, ‘family history’ was counted as 2 polysyllabic words each time it was presented. Because the words ‘family history’ would be used repeatedly in family history tools, this may inflate the SMOG estimates calculated. To account for the potential impact of jargon, we performed an additional set of SMOG assessments on a subset of 6 tools where the words ‘family history’ was counted only once. The results of our assessment demonstrated that SMOG levels were reduced by an average of 0.8 (range 0.1–1.4). Nonetheless, SMOG reading grade levels for these tools remained high, ranging from 9.7 to 12.8, suggesting that family history tools would benefit from additional efforts to reduce reading grade levels.
Although we used the PMOSE/IKIRSCH document readability formula to assess print-based family history tools, we were unable to apply the same approach to assess the complexity of interactive multimedia-based family history tools. This approach was developed specifically to assess the readability of static print-based documents. Hence, it was not possible to capture the unique features and dynamic nature of electronic media, given its interactivity capabilities (e.g., prompts to correct for entry errors) and the extent to which the reader helps to determine the text (Irwin Kirsch, personal communication). As such, we were unable to provide comparative data on document complexity between print and multimedia-based family history tools.
In a related vein, this study did not address the usability and navigation challenges of multimedia-based family history tools. Although SMOG assessments can provide researchers with a sense of prose literacy (i.e., knowledge and skills needed to search, comprehend, and use information from continuous texts), the dynamic nature of multimedia-based programs may facilitate the process of collecting family history information and overcome some of the challenges of using more complicated text (e.g., glossary pop up for complex words). On the other hand, multimedia-based family history tools may usher in a new set of challenges that will need to be addressed, such as unfamiliarity and lack of skills in using computers, especially among older target audiences.
This study does not address issues related to the use of plain language among the family history tools. Plain language (also called plain English) is communication an audience can understand the first time they read or hear it (www.plainlanguage.gov). Materials are considered to be written in plain language if audience members can find what they need, understand what they find, and use what they find to meet their needs. Elements of plain language include appearance/appeal (i.e., text layout, illustrations), organization (i.e., headings, short paragraphs), and writing style (i.e., active voice). Although approaches to assessing plain language are currently available (e.g., Suitability Assessment of Materials – SAM [29]), we chose not to use this approach in this study. Rating materials using SAM is a subjective process and therefore not as objective of a measurement compared to either SMOG or PMOSE/IKIRSCH. Further, the categories for rating on the SAM are more applicable to health education type materials and are less appropriate for evaluating forms that collect (family history) information from a respondent. For example, SAM requires scoring for categories including graphics and learning stimulation. Although this would be useful for some of the family history tools included in our list (in particular, some print-based tools that are structured like health education materials and contain personal stories), it would be less useful for tools that simply ask respondents to complete information about their family members. The lack of comparative information across family history tools on the elements of plain language is a limitation in this study.
Finally, we should note that the assessment of readability and document literacy, while important, does not address issues related to the accuracy or value of a family history tool. Future work is needed to examine the clinical validity and utility of these tools on identifying individuals at increased disease risk and improving health outcomes. We refer readers to other references for further discussion on this issue [10,30].
In sum, the findings from this study suggest the need to consider ways to simplify family history tools. We recommend some of the following strategies, based on the findings from the study, and that are commonly used by plain language experts (www.plainlanguage.gov) when developing education materials for the general public: (a) use simpler language (e.g., ‘use’ instead of ‘utilize’, ‘doctor’ instead of ‘physician’), (b) use shorter sentences, (c) use active voice, (d) avoid unnecessary words, (e) avoid technical jargon, (f) write short sections to break up material, using clear headings and sub-headings, (g) use an easy-to-read layout by including lists, illustrations or tables to simplify information, (h) identify and write for a specific target audience, and (i) pretest the tools with individuals in target audience. To reduce document complexity, developers of family history tools may want to consider using a simpler structure for fill-in forms (e.g. avoid using a nested list structure) and reducing the density or the total number of headings/labels in tables. In addition, shortening the length of a form and reducing the number of items in general that people are asked to fill in would also reduce the complexity of documents.
Conclusion
The proliferation of tools in recent years to facilitate the collection of family history information may serve to enhance genetic literacy among the general public and thereby assist in the translation of discoveries stemming from the Human Genome Project. Disparities in health literacy, however, may serve as a critical barrier to the effective use of family history tools, particularly among underserved populations. The lack of family history tools that are easy to read or use may compromise their potential effectiveness in identifying individuals at increased risk for chronic diseases in the general population. Future efforts are needed to examine strategies for overcoming barriers to the effective use of family history tools, in particular, by those in the general population who may face the greatest challenges to benefiting from genomic advances.
Acknowledgements
This work was supported in part by grants from the National Human Genome Research Institute (R03 HG004216) and the National Cancer Institute (NCI) (P30 CA06927). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NIH. Dr. Wang is also supported by a career development award from the NCI (K07 CA131103). We are indebted to the various individuals who assisted in our efforts to identify the list of tools evaluated in the present study: Louise Acheson, Robin Bennett, Wendy Cohn, Diane Gross, Kevin Hughes, Jean Jenkins, Jenny Johnson, Sheran McNiff, Melanie Myers, James O’Leary, Suzanne M. O’Neill, and Paula Yoon. We would also like to acknowledge and thank Stephanie Raivitch, Rima Rudd, and Irwin Kirsch for their advice and contributions to this research, as well as the Fox Chase Cancer Center Behavioral Research Core Facility.