Giovanni Fava’s Contributions to the Conceptualization and Evidence Base of Clinimetrics

Many of the papers reviewed below also include advances in conceptualizing clinimetrics. Fava’s recent editorial, entitled Forty years of clinimetrics [3], encapsulates well his contributions to the conceptualization of clinimetrics.

Fava has been consistent in his support for the wider application of clinimetrics. In 2003–4, the Journal of Clinical Epidemiology published four papers on clinimetrics in its Variance and Dissent series. De Vet et al. [4] argued that clinimetrics was perhaps not as different from psychometrics as had been proposed to date. Streiner [5] went further, arguing that clinimetrics was not distinct from psychometrics. In response, Fava and Belaise [6] presented evidence supporting the relevance and need for wider implementation of clinimetrics, focusing particularly on mental health. Following Feinstein, they noted that psychometrics was largely developed by disciplines outside clinical medicine and some of its underlying assumptions and methods did not serve clinical assessments well. For example, reiterating Feinstein [2, 7], they pointed out that a measure which fulfilled the psychometric requirement of internal consistency assessed by high Cronbach’s alpha was very likely to include redundant items which would make the measure less responsive to change than required of an optimal clinical outcome measure. Fava and Belaise also noted that in psychopharmacology trials, severity is determined by the number of symptoms, not their intensity or quality, to the same extent that a score in a depression self-rating scale depends on the number of symptoms that are scored positive. They highlighted the importance of assessing incremental validity, the extent to which a new measure will increase predictive ability beyond that provided by currently used measures. Incremental validity applies not only to clinimetric measures but is a requisite of all clinical assessment measures [8]. Fava and Belaise also stressed a point made repeatedly by Feinstein in his book on clinimetrics [2] that psychometric methods should not be abandoned altogether but required integration within a clinimetric framework to minimize the problems noted above, although an earlier paper had suggested a more strident opposition to psychometrics [9].

In a later review, Fava and colleagues outlined the wider context of clinimetrics [10]. Again following Feinstein [11], they highlighted the tendency in clinical medicine to emphasize “hard data” such as laboratory investigations at the expense of “soft information” such as distress and impairment. Echoing Tinetti and Fried [12], they noted that when disease became the focus of medicine in the past two centuries, the average life expectation was 47 years and most clinical encounters were for acute illness. The changed spectrum of health conditions (shifted towards ageing and chronicity) and the interindividual variabilities in health priorities suggest that the aim of treatment should refer to personal goals that may range from attainment of cure to prevention of recurrence, from removal of functional impairment to alleviation of symptoms. The clear implication is that to serve the needs of patients, modern clinical practice must focus more than ever before on “soft information.” Clinicians and clinical researchers have a responsibility to develop optimal outcome measures and clinimetrics offers the methodologies to achieve this.

The points highlighted above were supported by a review of the clinimetric approach to the clinical process in psychiatry [13]. Noting that in clinical practice problems often arise from a single assessment not only yielding a diagnosis but also then being extrapolated to give therapeutic and prognostic judgements, the review highlighted the importance of adopting a longitudinal perspective. A single DSM or ICD diagnosis, taken alone, is likely to be of very limited assistance in clinical decision-making [14]. Furthermore, while single diagnoses can be made reliably, their validity is suspect because they usually encompass heterogeneous clinical states [15]. Clinical decisions will be more effective if all important aspects of the patient’s health are measured and therapeutic priorities based on considering all these aspects together – the concept of macroanalysis [16]. Furthermore, because interventions and/or disorder pass through different stages, therapeutic priorities are likely to change, and these assessments should therefore be repeated. The value in clinical decision-making of macroanalysis and staging was elaborated in further papers [13, 17‒19] and also more recently in Fava’s treatment manual for well-being therapy [20].

Fava has been critical of evidence-based medicine, implying that if clinical research had adopted clinimetric principles more fully, the situation of clinical research would be different [21]. In this paper, he noted the inadequacy of current diagnostic practice in clinical decision-making. However, this inadequacy is due more to the limitations, noted above, of single DSM or ICD diagnoses than their applications. Extending diagnosis to include evidence about other conditions relevant to the patient is expected to improve this deficit substantially, provided that these conditions are measured using valid and reliable methods. Fava pointed out that Feinstein had compared meta-analyses to the alchemy that existed before modern scientific chemistry. This is a particularly apt analogy because those who use meta-analyses risk being seduced by the apparent precision of the outputs of the analytic procedures employed, at the expense of critically scrutinising their hypotheses and data collection. Thus, for example, in two meta-analyses of cognitive-behaviour therapy for people with schizophrenia, trials of interventions during acute relapse were combined in the same meta-analysis with interventions for people with chronic and persistent symptoms – two very different patient samples with different expected outcomes [22]. Another example – an early meta-analysis of the efficacy of cognitive-behaviour therapy for depression selected only studies in which depression had been assessed using the Beck Depression Inventory [23], a clear risk of bias given that researchers who adopted other outcome measures might not have been as closely aligned to Beck’s cognitive model as those who used the Beck Depression Inventory. These examples illustrate the adage “rubbish in, rubbish out.” However, there is no reason why clinimetric measures should be immune from these biases, or why those investigating clinimetric measures might be more ethically rigorous than others.

A major contribution of Fava’s has been his leadership in the development of the Diagnostic Criteria for Psychosomatic Research (DCPR). These are a set of clinimetric measures to provide assessments of conditions commonly encountered in psychosomatic medicine but not covered by the ICD or DSM diagnostic classification systems, including health anxiety, thanatophobia, disease phobia, illness denial, persistent somatization, conversion symptoms, functional somatic symptoms secondary to a psychiatric disorder, anniversary reaction, demoralization, irritable mood, type A behaviour and alexithymia [24, 25]. Following field testing, two further conditions were added – allostatic overload and hypochondriasis [19]. Numerous studies have supported the clinical utility of the original version of the DCPR [26] particularly in identifying these psychosomatic conditions independently of formal diagnoses. For the revised version of the measures (DCPR-R), data have been published supporting criterion-related validity (measured against PsychoSocial Index scores – see below) and incremental validity [27, 28].

An early publication of a brief clinimetric measure illustrated well some key features of validation, including criterion validity, sensitivity to change and interrater reliability [29]. This measure – the Cushing’s syndrome severity index (CSI) – was not directly related to mental health and had the great advantage that illness severity in the people participating could be assessed directly by other means, allowing direct measurement of criterion validity. More recent measures developed by Fava and his colleagues aimed to quantify clinically important aspects of the illness experience for which there is no “gold standard” measure, making assessment of criterion validity more difficult. Feinstein certainly supported use of clinimetrics to develop of such measures despite such difficulties with validation [2].

The PsychoSocial Index (PSI) was derived using clinimetric principles from other lengthier questionnaires, with the aim of developing a practical measure of stress factors among patients which could be applied in busy clinical settings [30]. It was developed in self-report and observer rating forms. Rather than relying on scores derived directly from the patient’s responses, the clinician reviews the patient’s responses then makes overall ratings on four aspects of the patient’s life – stress, psychological distress, abnormal illness behaviour, and well-being. The authors demonstrated that these assessments could be made very reliably. A more recent review identified twenty studies reporting results from the use of the PSI and reported further data on its performance [31].

Healthy functioning requires allostasis – continual physiological adjustments are required in response to circumstances and the environment. The allostatic load is the cost of chronic and/or fluctuating heightened bodily responses to repeated or persistent environmental influences appraised as challenging. Fava and colleagues highlighted allostatic load as an important aspect of the illness experience particularly in psychosomatic medicine but pointed out that standard diagnostic classifications lacked a clinically useful measure of allostatic load [32]. They formulated a clinimetric definition of allostatic load. A later paper provided further validation of the measure [33] and a systematic review was published more recently [34].

Fava and colleagues have developed measures for euthymia and mental pain. Following a review of the concept of euthymia, Fava and Bech developed a euthymia scale [35] and later work has supported its validity [36]. Recent reviews have stressed the importance of measuring euthymia in clinical practice [37, 38]. Regarding mental pain, a brief clinimetric measure was first published in Fava’s book on well-being therapy [39] and the concept was substantially elaborated in a subsequent review paper [40]. Evidence has been reviewed that mental pain is likely to fulfil the key requirements of a global person-centred outcome measure [41].

Under Fava’s editorship, Psychotherapy and Psychosomatics have published 43 papers with “clinimetrics” appearing in the title or abstract of the paper. The same PubMed search identified no equivalent papers in other psychiatry or psychology journals with similar Impact Factors to Psychotherapy and Psychosomatics, including Lancet Psychiatry, JAMA Psychiatry (including Archives of General Psychiatry), American Journal of Psychiatry, Psychological Bulletin, and Annual Review of Clinical Psychology.

In 2004, the journal devoted an issue to clinimetrics to honour the work of Alvan Feinstein [42]. Faravelli discussed the inadequacies of rating scales developed using psychometric methods, stressing that an illness is not the sum of its symptoms and that operational definitions do not necessarily reflect clinical reality [43]. Bech contributed a critical discussion of the impact of clinimetrics on clinical trials of antidepressants [44]. Bech noted that Rasch item analysis, based on item response theory, was a valuable tool in determining the validity of clinimetric measures. This had not been discussed hitherto by either Feinstein or Fava. In fact, Bech first wrote on this some 9 years before Feinstein’s book on clinimetrics was published [45]. Feinstein recommended that items should be chosen for inclusion in a clinimetric measure on the basis of clinical sensibility, based on what an experienced clinician considers the most important attributes of the condition being measured. Devoting a whole chapter of his clinimetrics book to this topic, Feinstein reviewed requisite features of sensibility, noting that this reflected a qualitative judgement, appraised by what he termed “enlightened common sense,” which often conflicted with standard measures of validity and reliability. Bech made the case that item analysis could provide additional support for a measure’s validity. Bech’s own substantial contribution to the development of clinimetrics in mental health was recognized in a tribute article in the journal [46].

A number of the editorials in Psychotherapy and Psychosomatics on clinimetric topics have already been cited above. Numerous papers have reported on the clinimetric properties of a range of measures, established as well as new, including allostatic load [32, 47], euthymia [48], mental pain and euthymia in primary care [49], the Family Assessment Device [50], the Drug Attitude Inventory [51], and the Global Symptom Change Rating in psychotherapy [52].

Systematic reviews of clinimetric measures, some of which have been widely cited, have included papers on the WHO-5 well-being index [53], allostatic load [34], the Clinical Interview for Depression [54], the Hamilton Depression Scale [55, 56], Kellner’s Symptom Questionnaire [57], mental pain [40], and the PsychoSocial Index [31]. Recent reviews have focused on the clinimetric properties of the Charlson Comorbidity Index [58] and the Medication Appropriateness Index [59].

Two papers have been published in the journal giving pertinent guidelines for research into psychological interventions. The earlier paper [60] noted the importance and value of using clinimetric principles in developing assessment measures. The more recent paper focused particularly on the evaluation of clinimetric criteria for patient-reported outcome measures [61].

A personal anecdote perhaps provides some insight into some aspects of Fava’s approach as an editor. In 1998, he accepted for publication a pilot study on a novel measure of suffering, the Pictorial Representation of Illness and Self Measure (PRISM) [62]. A subsequent paper also published in Psychotherapy and Psychosomatics provided more evidence of the validity and reliability of PRISM [63]. PRISM is unusual in requiring few words and instead functioning as a visual metaphor. When these two papers were accepted by the journal, PRISM would not have fulfilled Feinstein’s criterion of sensibility, not least because the measure was identified by serendipity and at the time, the authors had little idea of how and why PRISM worked. This illustrates Fava’s willingness to take risks in the papers published in the journal and his avowed aim to support ideas and research outside the mainstream. These qualities and his sound intuition have undoubtedly contributed to the very impressive growth over the years of the journal’s Impact Factor. Regarding PRISM, a systematic review in 2016 identified 89 papers which provided data using the measure [64]. These not only demonstrated consistent results from use of the measure but have also led to an understanding of how PRISM works.

The outline above highlights Fava’s very substantial contributions to advancing and disseminating the evidence base of clinimetrics in mental health. In some ways, the account above describes a microcosm of Giovanni Fava’s academic and professional life – always being open to learning from patients and colleagues, having an enduring interest in fostering knowledge and understanding (including in topics which might initially have appeared peripheral to the mainstream of research and practice), and nurturing the professional development of those he has trained and mentored. It is typical of Fava’s approach to science and knowledge that he recognized the importance to psychiatry and psychology of clinimetrics and began advocating for it, and developing its conceptualization, long before it became widely understood and accepted.

The work summarized above highlights advances made possible by clinimetrics in our understanding of the patient experience not only in psychosomatics but also in medicine and health psychology more generally. However, there is more to be done. For example, clinimetric measures are generally designed to yield quantitative results. This is particularly helpful in comparing results from different samples or settings but requires that the measures used are generalizable. Using a given measure in a new setting is likely to meet the requirement of clinimetric sensibility, particularly because this is subjective. If incremental validity has already been demonstrated, the measure is also likely to demonstrate this in a different setting. However, neither of these criteria confirms that the measure’s performance is optimal. Particularly because clinimetric measures are designed to have only the minimum required items, if any of the items behave differently in the second or third setting in which the measure is applied compared to the initial validation, these items need to be reviewed to optimize generalizability. Assessing whether the items of a measure behave consistently in different settings can be done by assessing differential item functioning, derived from item response theory [65]. If a measure behaves similarly in several settings, one can have greater confidence that the items are robust and the measure generalizable.

While the development of clinimetric measures has undoubtedly contributed substantially to research, what have been its benefits for individual patients? Fava has argued that clinimetrics should be an integral component of clinical judgement [66]. Macroanalysis and the concept of staging are clearly intended to be used clinically with patients, but how can clinimetric measures contribute to their optimal application? At the very least, measures such as those outlined above can contribute to a more nuanced formulation of a patient’s problems, which should lead to a more comprehensive intervention plan. Some psychosomatic concepts influence treatment, such as alexithymia [67, 68], or have specific treatment guidelines or protocols associated with them, such as abnormal illness behaviour [69, 70]. Perhaps over time, further intervention protocols will be developed for others of the measures above? This raises another question – whose purposes do clinimetric measures serve? To date, it would appear that these measures have directly served the needs of clinicians, with patients benefitting only by association. Perhaps this is inevitable – should patients be expected to understand psychosomatic concepts, any more than psychiatric diagnoses, without collaborative discussion with their clinicians? Unlike psychometrically developed scales, clinimetric measures have the capacity to incorporate individual experiences as described by patients. Some types of psychological intervention, such as cognitive and/or behavioural psychotherapies, measure change based on personal targets individually agreed between patient and therapist [71], targets which the patient can recognize as sensible in Feinstein’s sense. There have been attempts to make quantifying change in such targets more systematic [72], but these have not been widely adopted, probably reflecting that they are as yet too cumbersome for routine clinical application. Rigorous assessment of the patient’s concerns, and giving the patient understanding and control of how they are evaluated, would certainly meet with Giovanni Fava’s approval.

The author has no conflict of interest to declare.

No funding was received.

Tom Sensky is the sole author of this editorial.