Clinimetrics and Clinical Psychometrics: Macro- and Micro-Analysis

Clinical judgment in psychiatry includes markers or indexes of high clinical validity, but from a scientific measurement point of view some clinical markers, even when performed by experienced psychiatrists, have been found to be questionable due to low inter-rater reliability. However, we have very few studies focusing directly on clinical judgment analysis [1].

When the Diagnostic and Statistical Manual of Mental Disorders, second revision [2], was released, Spitzer and Wilson [3] remarked that, compared to the DSM-I [4], the ten major diagnostic categories were no longer termed ‘reactions’, in favor of a return to a more Kraepelinian approach by considering shared phenomenology with the clinical syndromes.

At a symposium in 1977 at which Spitzer released the guiding principles for DSM-III, Feinstein [5] gave a critical review of diagnoses in psychiatry. In this presentation Feinstein [5] introduced his clinimetric ideas, going on to develop these thoughts a decade later in his monograph on clinimetrics [6]. In 1977 Feinstein [5] concluded that both clinical medicine and clinical psychiatry were trying to escape the clinical challenges that confront them. These escapes take a form in which we hope that investigators working in some nonclinical domains, i.e. the biophysicist, the biostatistician, or the pharmacologist, will take care of our classification problem. Feinstein’s advice was therefore to avoid these kinds of traditional clinical escapes and to consider all clinical markers as scientific objects, although their magnitude and complexity are enormous. In his monograph on clinimetrics, Feinstein [6] made a major attempt to cover the diverse clinical markers we have to take into account. His emphasis on the concept of clinimetrics should be seen as an attempt to refer to measurement although many of the clinical markers Feinstein collected might have a questionable measurement aspect. In this review we will try to discuss the measurement aspects in clinimetrics in a dialogue between clinical judgment and psychometrics.

In daily clinical practice when making an initial clinical assessment of the patient, the experienced psychiatrist takes the major diagnostic categories (dementia, schizophrenia, mania, depression, anxiety, neuroticism, and extraversion/introversion) into consideration.

With DSM-III [7] the full Kraeplinian approach was accepted, as the major diagnostic categories were defined by their shared symptoms. By focusing on the shared course of symptoms, Kraepelin introduced clinical psychometrics in psychiatry [8].

The initial clinical judgment made by experienced psychiatrists has been referred to by several authors [9,10,11] as a clinimetric macro-analysis. This macro-analysis rests on the evaluation performed by the experienced psychiatrist. In the next step of the assessment of the patient, we have to do a micro-analysis, i.e. a detailed analysis of the symptoms or other clinical markers in order to perform a measurement using unidimensional scales [11,12,13].

The term clinimetrics was introduced by Feinstein three decades ago [6,14] to indicate a domain in clinical medicine concerned with indexes or rating scales. According to Feinstein the clinical index of global improvement during treatment is the most unique component in clinical medicine. Thus, when we ask our patients about any change in terms of mental activity since the start of treatment, we are asking whether the patients are much better, slightly better, the same, slightly worse, or much worse. Such questions are considered very important by the patients and consequently give them the feeling that the clinician is thoroughly familiar with the kinds of problems that confront them. However, according to Wright and Feinstein [15], this single Likert item of global judgment has too many potential problems of measurement to be regarded as a psychometrically valid scale although it is still clinically meaningful.

By placing the evaluation of global improvement within the macro-analysis field in clinical markers, we have no need to make a psychometric analysis, this has to be dealt with using micro-analysis.

Against this background the term clinimetrics seems to rely too much on the psychometric validation procedure for clinical markers. At the same time it seems clear that clinical markers have to be examined using both macro- and micro-analysis.

Macro- and micro-analysis might offer a conceptual and methodological platform for a substantial revision of clinical assessment. In the following sections we will deal with some of the major issues in this respect.

The Hamilton Depression Scale (HAM-D) was introduced by Hamilton [16,17] as a 17-item scale for the measurement of depression severity. A macro-analysis of the HAM-D was, however, first performed by Bech et al. [18], using experienced psychiatrists’ global assessment of depression severity on a Likert scale from 0 (= no depression) to 10 (= extreme degree of depression) as an index of clinical validity. The experienced psychiatrists had grown up with Kraepelin’s diagnostic categories as their ‘ground schooling’. It was shown that only 6 of the 17 items were used by these experienced psychiatrists.

In our micro-analysis of this 6-item HAM-D₆ we used the item response theory model developed by Rasch [19] as the psychometric validation procedure [20,21]. We were then able to show that the rank order of prevalence of these 6 items (depressed mood, work and interests, somatic general, psychic anxiety, feelings of guilt, and psychomotor retardation) was invariant from week to week during treatment with antidepressants, and that this rank order of prevalence was invariant when females were compared with males or when younger patients were compared with elderly patients. This invariant item ordering is often referred to as transferability. The total score of the HAM-D₆ was therefore a sufficient statistic, in contrast to the HAM-D₁₇ which is a multi-dimensional scale. Thus, we often need to give a profile score rather than a total score of the complete HAM-D_17. This is illustrated in the ABC version of the HAM-D (fig. 1). We need to score not only the HAM-D₆ for the pure depression symptoms but also the HAM-D₉ (covering the physiological arousal symptoms, e.g. sleep, agitation, somatic anxiety, sexual problems, appetite, and weight loss). The remaining 2 items, the HAM-D₂, encompass the idiographic symptoms of suicidal thoughts and insight [8].

Besides transferability, there is another important micro-analytic principle in the item response theory model, namely local independence of items. Statistically, local independence is concerned with how different items are stochastically related through independence for any fixed level of depression [22]. However, clinically we are dealing with local independence as a measure (correlation) of the extent to which the score on one item can automatically predict the score on another item [23,24]. In the Montgomery Åsberg Rating Scale for Depression (MADRS) [24] the score on the item of reported sadness will have a clear local dependence on the item of apparent sadness [25]. It is such a dialogue between the statistical and the clinical aspects of clinimetrics that might improve the scientific level of the clinical markers we are confronted with. Another example of problems with regard to the local dependence of items is to be found in the Beck Depression Inventory (BDI) [26], recently discussed by Greenberg [27].

Careful symptom discrimination via interviewing using an observer-rated instrument may allow us to attribute differential emphasis on specific symptoms. In clinimetrics, major and minor symptoms may be discriminated, unlike in psychometrics, where all items are weighed the same [11]. A recent example was provided by the use of an item of the Clinical Interview for Depression [28], reactivity to social environment, to characterize the clinical features [29] and response to treatment [30] of cyclothymic disorder.

The Bech-Rafaelsen Melancholia Scale (MES) and the Mania Scale (MAS) as well as the schizophrenicity subscale of the Brief Psychiatric Rating Scale (BPRS) and the Hamilton Anxiety Scale (HAS) have all been validated by use of the macro- and micro-analysis [8,31].

Among the comprehensive distress evaluation scales, a suitable self-rating instrument is the Symptom Questionnaire (SQ). It has been used in psychiatric and psychosomatic settings to assess both symptomatology and emotional well-being and their relationship. Kellner’s SQ [32] is a simple and brief self-report questionnaire which contains single-word items instead of questions. The total number of items (92 items) has been increased in order to improve the scale’s sensitivity [33]. Responders should answer yes/no or true/false to the item, instead of measuring the frequency and severity of symptoms. The main novelty introduced by Kellner is the presence of items assessing emotional well-being. They were introduced to obtain a more complete evaluation of the patients’ clinical status and to improve the scale’s internal consistency. The final SQ version yields 4 principal scales: anxiety, depression, somatization and hostility-irritability, which can be divided into 8 subscales: 4 symptom subscales (anxiety, depression, somatization, hostility) and 4 corresponding well-being subscales (relaxation, contentment, physical well-being and friendliness). The depression scale includes items such as ‘hopeless’ from the depressive symptoms subscale and items such as ‘happy’ from the contentment subscale. However, a single high score cannot be equated with a diagnosis of psychopathology, without further clinical assessment. High scores on the somatization scale can indicate the presence of somatization only after exclusion of other medical conditions [34]. In double-blind, cross-over studies, negatively formulated items tended to be more sensitive in discriminating between the effects of a psychotropic drug and placebo [35] (see table 1[32,36,37,38,39,40]).

The macro versus micro analysis focuses not only on the evaluation of the severity of symptoms but also on their sequence, rate of progression of illness, and co-morbidity, and on problems of functional life, such as well-being [41]. Two patients may both meet the diagnostic criteria for major depression, but they may present very different scores on a rating scale. What they do not share may be as important as what they do share: their different scores may reflect different symptom severities, perceptions, and illness attitudes, and these variables may affect the clinical course of the disorder in these two patients.

As discussed elsewhere [8], the neo-Kraepelinic paradigm in DSM-III, DSM-IV or ICD-10 refers to a thorough symptom-based description of mental disorders. However, Kraepelin’s description was not completely atheoretical, as the medical model of disorders also ‘hovered’ in the back of his mind. Table 2 shows the ‘medical model’ in operation when the experienced psychiatrist performs his or her prognostic or etiological judgment. For example, in (F2a) there is a long-term psycho-social stressor which can explain the symptoms while in (F2b) a single, but exceptional event can explain the symptoms.

Emmelkamp [12] provided an example of the macro-analysis when confronted with patients who had developed depressive episodes. A substantial number of depressed patients had been through a chronic (F2a) marital stressor. According to Emmelkamp [12] the data suggest that depression and marital stressors are closely linked. Furthermore, persons who after treatment for depression return to distressed marriages are more likely to experience relapse [42]. When viewing the results of studies on marital stress-induced depression, behavioral marital therapy in maritally distressed couples seems to have an exclusive effect on the marital relationship. This is not found in individual cognitive behavioral therapy, while it is as effective as cognitive therapy in reducing depression [43]. Not surprisingly, behavioral marital therapy was hardly effective in depressed patients who did not experience marital problems [10].

An example of macro-analysis can be found in table 2; in this example, F12 is a macro-analytic attempt to evaluate hidden bipolarity by asking about previous hypomanic ups. We recently made an analysis of the Hypomania Checklist (HCL-32) and showed that around 50% of unipolar depressed patients not responding to antidepressants had a hidden bipolarity [44].

F17 in table 2 is another example. This is an attempt to identify drug-induced disorders by macro-analysis. We recently showed that patients with hepatitis C might develop a major depression when treated with interferon [45].

Comorbidity is the co-occurrence of multiple disorders of macro-analytic significance. In table 2 somatic illness is indicated (F16) when it is considered to have etiological significance, i.e. the mental condition would not have developed without this somatic illness, e.g. poststroke depression.

The co-occurrence of several mental disorders has been investigated in longitudinal observation [46]. In major depression, two thirds of the patients meet criteria for another axis I disorder (anxiety disorders in particular) and one third have two or more disorders [47]. The presence of anxiety disorders appears to predict the persistence and recurrence of depressive illness [48,49]. Some forms of comorbidity may be masked by manifestations of the disorder and only become evident when the most severe symptoms have abated [50]. The current emphasis in psychiatry is on assessment of symptoms resulting in syndromes identified by diagnostic criteria (DSM). However, an emerging awareness that also psychological symptoms which do not reach the threshold of a psychiatric disorder may affect the quality of life and entail pathophysiological and therapeutic implications led to the development of the Diagnostic Criteria for Psychosomatic Research (DCPR) [51,52,53] together with a specific interview to assess patients [54,55]. The DCPR were introduced in 1995 and tested in various clinical settings [51,52,53]. Diagnostic criteria for the following syndromes are included: abnormal illness behavior (disease phobia, thanatophobia, health anxiety, and illness denial), somatization syndromes (persistent somatization, functional somatic symptoms secondary to a psychiatric disorder, conversion symptoms, and anniversary reactions), irritability, type A behavior, demoralization, and alexithymia. The DCPR allow a far more sophisticated qualitative assessment of patients than the one dimensional DSM checklist of psychological symptoms.

The DCPR interview encompassing these various diagnostic rubrics has shown excellent inter-rater reliability, construct validity, and predictive validity for psychosocial functioning and treatment outcomes [54,56]. It has yielded important information as the clinical characteristics of anxiety disorders [57], major depressive disorders [58], psychiatric disorders in the setting of endocrine disease [59] and cardiovascular illness [60]. Whenever the DCPR have been used in conjunction with the DSM criteria, they have been found to carry additional clinical information [52].

Residual symptoms, despite successful response to therapy, appear to be the rule after completion of drug or psychotherapeutic treatment in both mood and anxiety disorders [50,61]. The presence of residual symptoms has been correlated with poor long-term outcomes [50,61]. There appears to be a relationship between residual and prodromal symptomatology: in fact most of the residual symptoms occur in the prodromal phase of illness. These findings have led to the hypothesis that upon recovery residual symptoms may progress to become prodromal symptoms of relapse and that treatment directed toward residual symptoms may yield long-term benefits [62]. Treatments which potentially aim to achieve different effects (e.g. pharmacotherapy and psychotherapy) may thus be used in a sequential order. One type of treatment (e.g. psychotherapy) may be employed to improve symptoms which the other type of treatment (e.g. pharmacotherapy) was unable to affect. The aim of the sequential approach is to add therapeutic ingredients as long as they are needed. In this sense, it introduces a conceptual shift in clinical practice. Therapeutic targets are not predetermined but depend on the response of patients to the first course of treatment [63].

Clinimetrics also provides relevant clinical implications in the definition of recovery. Commonly, the concept of recovery reflects that of ‘improvement’ which refers to the clinical distance along which the current state of the patient is compared to the pretreatment position [64]. In this sense, recovery can be expressed either as a categorical variable (present/absent) or as a comparative category (nonrecovered, slightly recovered, moderately recovered, greatly recovered). Both expressions require arbitrary cut-off points related to the amount of improvement.

Feinstein [6] differentiates between the monadic transition index (devoted exclusively to the rating of change during treatment), the dyadic component index (e.g. before and after treatment ratings) and the polyadic component index (when more than two ratings, such as with the sequential model, have been performed). A depressed patient who is asked how he or she feels after 3 weeks of treatment with an antidepressant drug and replies ‘just fine’ (instead of ‘better’) uses a self-monadic component. The amount of change induced by treatment, however, may make him/her overlook the distance from an intended goal, such as the preepisode state. The physician may collude with the patient in this illusion of wellness, since he/she may be more gratified by the amount of improvement induced in the patient than by the current distance from an intended goal [65]. Clinicians may choose recovery as a target that is negotiated between the doctor and the patient. The doctor can insist that the target be reasonable (e.g. not asking to be better than before the illness). Nevertheless, the idea of successful recovery may differ from one patient to the next and should not be constrained too much by the doctor’s ideas. We should accept the possibility that a treatment may determine abatement of symptoms in some patients, leave a substantial residual symptomatology in others, yield an unsatisfactory response in others, and provide no benefit or even cause harm in a few. The type of residual symptomatology varies widely from patient to patient and needs to be assessed individually [65]. Frequently, in fact, patients with affective disorders, despite being treated effectively with psychotherapy or pharmacological interventions, can continue to present high subclinical symptomatology and impaired levels of psychological well-being compared to healthy control subjects [66,67]. The persistence of this impairment after drug treatment of depression is likely to be a poor prognostic index [34,68].

In a survey on factors identified by depressed outpatients as important in determining remission, those most frequently judged as such were the presence of features of positive mental health, such as optimism and self-confidence, a return to one’s usual, normal self, and a return to the usual level of functioning [69]. Ryff’s psychological dimensions [70] may be instrumental in assessing both the process and the definition of recovery [71].

When Hans Selye developed his theory of the concept of stress, he discriminated between stressors (strain), stress (the bodily reactions to such strain), and distress (the mental reaction to the strain). As discussed elsewhere [8], Selye’s original stress model had the daily strain at work and at home as stressors, not the single catastrophic stressor that completely dominates the picture in posttraumatic stress disorder (PTSD).

The ABC version of the HAM-D₁₇ (fig. 1) actually covers the development of PTSD through the nine arousal symptoms in HAM-D₉ (disturbed sleep, somatic anxiety, and agitation) which develop over the first 1–2 months. After this the pure depression symptoms appear (HAM-D₆) with depressed mood, tiredness, lack of interests, guilt, and psychic anxiety. If untreated, PTSD might be more chronic, demonstrating the HAM-D₂ symptoms (feelings of emptiness, lack of insight, and suicidal thoughts).

In work-related stress conditions the symptoms basically follow the same course, but less clearly [72]. Cumulative exposures to stressful experiences, often subsumed under the term ‘allostatic load’, may increase vulnerability to disease onset [63,73]. The concept of allostatic load represents the cost of the continual adjustment of the internal milieu required by the organism to adapt to different environmental demands [74]. The majority of studies on allostatic load have focused mainly on identifying its biological components, which are believed to mediate the relationship between stress and illness. Recently, according to the clinimetric approach, Fava et al. [41,75] have introduced specific criteria for the evaluation of allostatic overload (AO) in the clinical setting. The first criterion (A) deals with the specification of the stressor. Stress is measured by evaluating the occurrence of stressful situations in areas of major importance to the patient, resulting in major changes in the patient’s living conditions, social and family circle, and work, as well as subtle and long-standing life situations, such as those occurring at work. In both cases the situation needs to be experienced by the individual as taxing or exceeding his/her coping skills. The second criterion (B) is concerned with the clinical manifestations ranging from psychiatric to psychosomatic/subclinical symptoms, as defined by the DSM-IV [76] and/or the Diagnostic Criteria for Psychosomatic Research (DCPR) [51], and from impairment in social and occupational functioning to a decrease in well-being [71].

A latent effect of adversities during childhood, when the developing brain is more sensitive to a variety of environmental signals, has been demonstrated and has created a need to classify the degree of stress children are exposed to for health and disease prevention uses [77].

A number of clinimetric instruments based on structured methods of data collected and used in controlled studies have been developed (table 3) [78,79,80,81].

An increasing body of evidence links the initiation and progression of several medical disorders, such as diabetes, obesity, and cardiovascular illness [82,83,84]. The benefits of modifying the lifestyle were demonstrated in several randomized controlled trials. Table 4 illustrates some instruments which may assess lifestyle [85,86,87,88].

It is commonly considered that Bentham (1748–1832) made the first attempt to measure psychological well-being via questionnaires [8,89]. He defined subjective well-being as the difference between the sum of all kinds of pleasure and the sum of all kinds of pain experienced by the individual in a given period of time. Psychometrically, it is very important to defined the time frame, the ‘given period of time’, often referred to clinimetrically as the ‘window’ [8].

Macro-analytically we can classify psychological well-being markers into the ‘hedonic’ scales and the ‘eudemonic’ scales. When going through Murray’s basic human needs and their hierarchic arrangements [90], the hedonic needs might be considered as a global index of measurement [8]. Ryff’s Psychological Well-Being Scale (PWB) [70] should be consulted when attempting to measure eudemonia, covering such elements as autonomy, personal growth, purpose of life, interpersonal relationships, and self-acceptance. Micro-analytically the PWB is a multi-dimensional approach implying that a profile score rather then a global score should be used (table 5) [32,70,91].

Among the hedonic well-being scales, Hall et al. [91] identified WHO-5 as a clinically very valid scale by judgment analysis. The WHO-5 questionnaire has been widely used and translated into around 30 languages. WHO-5 is a general psychological well-being scale based on five positively phrased items covering the ‘window’ of the past 2 weeks. Its total score is a sufficient statistic because the scale fulfills the item response theory models [8]. The theoretic score range goes from 0 (worst imaginable quality of life) to 100 (best imaginable quality of life).

Furthermore, the SQ [32] also represents a suitable clinimetric instrument for assessing psychological well-being. It yields 4 principal scales: anxiety, depression, somatization, and hostility-irritability, which can be divided into 8 subscales: 4 symptom subscales (anxiety, depression, somatization, hostility) and 4 corresponding well-being subscales (relaxation, contentment, physical well-being, and friendliness). The SQ well-being scales can, however, also be scored separately (scoring 1 when the answer indicates the presence of well-being), so that the higher the score is, the higher the well-being is. The total scale internal consistency can therefore be computed by comparing the symptom subscale score with the corresponding well-being one. Through this scoring procedure, it is possible to measure also the balance of positive and negative effects, which has been described in major depression [92] and found to be affected by cognitive-behavioral therapy [93]. In another clinical trial [32], the treatment of psychiatric symptoms induced improvement of well-being and, in this case, the SQ subscales describing well-being were more sensitive to drug effects than subscales describing symptoms [34].

Several studies have suggested that psychological well-being plays a buffering role in coping with stress and has a favorable impact on disease [71,94,95,96]. Table 5 illustrates some methods for assessing psychological well-being, both eudaimonic and hedonic [32,70,91].

Once the symptoms of a medical disease are perceived by a person, or he/she has been told by a doctor that he/she is ill even if symptoms are absent, then this disease-related information gives rise to psychological responses which influence the patient’s experience and behavior, as well as the course and therapeutic outcome of a given illness episode [97]. The study of illness behavior, the ways in which individuals experience, perceive, evaluate and respond to their own health status, has yielded important information in medical patients [98,99]. There are many manifestations of abnormal illness behavior, which range from hypochondriasis and disease phobia to illness denial and lack of compliance [100].

Table 6 indicates different clinimetric strategies for assessing illness behavior [101,102,103,104].

With the introduction of the term ‘clinimetrics’ Feinstein made an attempt to avoid the temptation to ask other scientists in the nonclinical domains of medicine (e.g. biophysicists or pharmacologists) for help when evaluating patients because the clinicians found the clinical world too complex and too difficult to handle in a scientific way.

We have followed Feinstein’s strict clinimetric approach by using macro- and micro-analyses of the various clinical markers. With this pure clinical approach we have attempted to identify clinical assessment tools covering severity scales, prognostic markers, issues of comorbidity, longitudinal assessments, recovery, stressors, lifestyle, psychological well-being, and illness behavior. In this process we have relied on a constructive dialogue between the clinical and the purely psychometric aspects of clinimetrics. Such dialogue may generate important developments: the staging of mental disorders with particular reference to their longitudinal development [105] and their previous treatment history [106]; the construction of subtypes of illness, such as has been suggested in major depression [28,107,108]; the broadening of assessment areas, such as demoralization and irritable mood [109,110,111,112], and psychological well-being [91].