Background: Human and animal work suggests a shift from goal-directed to habitual decision-making in addiction. However, the evidence for this in human alcohol dependence is as yet inconclusive. Methods: Twenty-six healthy controls and 26 recently detoxified alcohol-dependent patients underwent behavioral testing with a 2-step task designed to disentangle goal-directed and habitual response patterns. Results: Alcohol-dependent patients showed less evidence of goal-directed choices than healthy controls, particularly after losses. There was no difference in the strength of the habitual component. The group differences did not survive controlling for performance on the Digit Symbol Substitution Task. Conclusion: Chronic alcohol use appears to selectively impair goal-directed function, rather than promoting habitual responding. It appears to do so particularly after nonrewards, and this may be mediated by the effects of alcohol on more general cognitive functions subserved by the prefrontal cortex. © 2014 S. Karger AG, Basel

Keywords:
Alcohol dependence, Decision-making, Reinforcement learning, Dopamine, Computational psychiatry
1.
Balleine BW, Dickinson A: Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 1998;37:407-419.
2.
Sutton RS, Barto AG: Reinforcement Learning: An Introduction. Cambridge, MIT Press, 1998, p 1.
3.
Daw ND, Niv Y, Dayan P: Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 2005;8:1704-1711.
4.
Dickinson A, Wood N, Smith JW: Alcohol seeking by rats: action or habit? Q J Exp Psychol B Comp Physiol Psychol 2002;55:331-348.
5.
Hopf FW, Chang S-J, Sparta DR, Bowers MS, Bonci A: Motivation for alcohol becomes resistant to quinine adulteration after 3 to 4 months of intermittent alcohol self-administration. Alcohol Clin Exp Res 2010;34:1565-1573.
6.
Wolffgramm J, Heyne A: From controlled drug intake to loss of control: the irreversible development of drug addiction in the rat. Behav Brain Res 1995;70:77-94.
7.
Ostlund SB, Maidment NT, Balleine BW: Alcohol-paired contextual cues produce an immediate and selective loss of goal-directed action in rats. Frontiers in Integrative Neuroscience 2010;4:19.
8.
Robbins TW, Everitt BJ: Drug addiction: bad habits add up. Nature 1999;398:567-570.
9.
Jentsch JD, Taylor JR: Impulsivity resulting from frontostriatal dysfunction in drug abuse: implications for the control of behavior by reward-related stimuli. Psychopharmacology 1999;146:373-390.
10.
Bechara A: Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective. Nat Neurosci 2005;8:1458-1463.
11.
Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ: Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res 2009;199:89-102.
12.
Jentsch JD, Olausson P, De La Garza R 2nd, Taylor JR: Impairments of reversal learning and response perseveration after repeated, intermittent cocaine administrations to monkeys. Neuropsychopharmacology 2002;26:183-190.
13.
Schoenbaum G, Setlow B: Cocaine makes actions insensitive to outcomes but not extinction: implications for altered orbitofrontal-amygdalar function. Cereb Cortex 2005;15:1162-1169.
14.
Park SQ, Kahnt T, Beck A, Cohen MX, Dolan RJ, Wrase J, Heinz A: Prefrontal cortex fails to learn from reward prediction errors in alcohol dependence. J Neurosci 2010;30:7749-7753.
15.
Ersche KD, Roiser JP, Robbins TW, Sahakian BJ: Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans. Psychopharmacology 2008;197:421-431.
16.
Fillmore MT, Rush CR: Polydrug abusers display impaired discrimination-reversal learning in a model of behavioural control. J Psychopharmacol 2006;20:24-32.
17.
Ersche KD, Roiser JP, Abbott S, Craig KJ, Muller U, Suckling J, Ooi C, Shabbir SS, Clark L, Sahakian BJ, Fineberg NA, Merlo-Pich EV, Robbins TW, Bullmore ET: Response perseveration in stimulant dependence is associated with striatal dysfunction and can be ameliorated by a D2/3 receptor agonist. Biol Psychiatry 2011;70:754-762.
18.
Lawrence AJ, Luty J, Bogdan NA, Sahakian BJ, Clark L: Impulsivity and response inhibition in alcohol dependence and problem gambling. Psychopharmacology 2009;207:163-172.
19.
Morein-Zamir S, Simon Jones P, Bullmore ET, Robbins TW, Ersche KD: Prefrontal hypoactivity associated with impaired inhibition in stimulant-dependent individuals but evidence for hyperactivation in their unaffected siblings. Neuropsychopharmacology 2013;38:1945-1953.
20.
Marhe R, Waters AJ, van de Wetering BJ, Franken IH: Implicit and explicit drug-related cognitions during detoxification treatment are associated with drug relapse: an ecological momentary assessment study. J Consult Clin Psychol 2013;81:1-12.
21.
Wiers RW, Eberl C, Rinck M, Becker ES, Lindenmeyer J: Retraining automatic action tendencies changes alcoholic patients' approach bias for alcohol and improves treatment outcome. Psychol Sci 2011;22:490-497.
22.
Hogarth L, Chase HW: Parallel goal-directed and habitual control of human drug-seeking: implications for dependence vulnerability. J Exp Psychol Anim Behav Processes 2011; 37:261-276.
23.
Hogarth L, Field M, Rose AK: Phasic transition from goal-directed to habitual control over drug-seeking produced by conflicting reinforcer expectancy. Addict Biol 2013;18:88-97.
24.
Hogarth L, Attwood AS, Bate HA, Munafo MR: Acute alcohol impairs human goal-directed action. Biol Psychol 2012;90:154-160.
25.
Otto AR, Gershman SJ, Markman AB, Daw ND: The curse of planning: dissecting multiple reinforcement-learning systems by taxing the central executive. Psychol Sci 2013;24:751-761.
26.
Guillot CR, Fanning JR, Bullock JS, McCloskey MS, Berman ME: Effects of alcohol on tests of executive functioning in men and women: a dose response examination. Exp Clin Psychopharmacol 2010;18:409-417.
27.
Sjoerds Z, de Wit S, van den Brink W, Robbins TW, Beekman AT, Penninx BW, Veltman DJ: Behavioral and neuroimaging evidence for overreliance on habit learning in alcohol-dependent patients. Translat Psychiatry 2013;3:e337.
28.
Tricomi E, Balleine BW, O'Doherty JP: A specific role for posterior dorsolateral striatum in human habit learning. Eur J Neurosci 2009;29:2225-2232.
29.
Bornstein AM, Daw ND: Multiplicity of control in the basal ganglia: computational roles of striatal subregions. Curr Opin Neurobiol 2011;21:374-380.
30.
Rangel A, Hare T: Neural computations associated with goal-directed choice. Curr Opin Neurobiol 2010;20:262-270.
31.
Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ: Model-based influences on humans' choices and striatal prediction errors. Neuron 2011;69:1204-1215.
32.
Corbit LH, Nie H, Janak PH: Habitual alcohol seeking: time course and the contribution of subregions of the dorsal striatum. Biol Psychiatry 2012;72:389-395.
33.
Otto AR, Raio CM, Chiang A, Phelps EA, Daw ND: Working-memory capacity protects model-based learning from stress. Proc Natl Acad Sci USA 2013;110:20941-20946.
34.
Smittenaar P, FitzGerald TH, Romei V, Wright ND, Dolan RJ: Disruption of dorsolateral prefrontal cortex decreases model-based in favor of model-free control in humans. Neuron 2013;80:914-919.
35.
First M, Spitzer R, Gibbon M, Williams J: Structured clinical interview for DSM-IV-TR axis I disorders, research version, patient edition with psychotic screen (SCID-I/P W/PSY SCREEN). New York, New York State Psychiatric Institute, Biometrics Research, 2001.
36.
Brainard DH: The psychophysics toolbox. Spatial Vision 1997;10:433-436.
37.
Pelli DG: The videotoolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision 1997;10:437-442.
38.
Matlab: Version 7.4.0 (r2007a). Natick, The MathWorks Inc, 2007.
39.
Schmidt K-HMP: Wortschatztest (WST). Weinheim, Beltz, 1992.
40.
Wechsler D: WAIS-R manual. New York, Psychol Corp, 1981.
41.
Army individual test battery. Manual of directions and scoring. Washington, War Department, Adjutant's Generals Office, 1944.
42.
Morris JC, Heyman A, Mohs RC, Hughes JP, van Belle G, Fillenbaum G, Mellits ED, Clark C: The Consortium to Establish a Registry for Alzheimer's Disease (CERAD). I. Clinical and neuropsychological assessment of Alzheimer's disease. Neurology 1989;39:1159-1165.
43.
Simon D, Daw N: Dual-system learning models and drugs of abuse; in Gutkin B, Ahmed SH (eds): Computational Neuroscience of Drug Addiction. New York, Springer, 2012, vol 10, pp 145-161.
44.
Redish AD, Jensen S, Johnson A: A unified framework for addiction: vulnerabilities in the decision process. Behav Brain Sci 2008;31:415-437.
45.
Lucantonio F, Stalnaker TA, Shaham Y, Niv Y, Schoenbaum G: The impact of orbitofrontal dysfunction on cocaine addiction. Nat Neurosci 2012;15:358-366.
46.
Lucantonio F, Caprioli D, Schoenbaum G: Transition from ‘model-based' to ‘model-free' behavioral control in addiction: involvement of the orbitofrontal cortex and dorsolateral striatum. Neuropharmacology 2014;76:407-415.
47.
Chen BT, Yau HJ, Hatch C, Kusumoto-Yoshida I, Cho SL, Hopf FW, Bonci A: Rescuing cocaine-induced prefrontal cortex hypoactivity prevents compulsive cocaine seeking. Nature 2013;496:359-362.
48.
Vanderschuren LJ, Everitt BJ: Drug seeking becomes compulsive after prolonged cocaine self-administration. Science 2004;305:1017-1019.
49.
Deroche-Gamonet V, Belin D, Piazza PV: Evidence for addiction-like behavior in the rat. Science 2004;305:1014-1017.
50.
Wunderlich K, Smittenaar P, Dolan RJ: Dopamine enhances model-based over model-free choice behavior. Neuron 2012;75:418-424.
51.
Heinz A, Dufeu P, Kuhn S, Dettling M, Gräf K, Kürten I, Rommelspacher H, Schmidt LG: Psychopathological and behavioral correlates of dopaminergic sensitivity in alcohol-dependent patients. Arch Gen Psychiatry 1996;53:1123-1128.
52.
Heinz A, Siessmeier T, Wrase J, Buchholz HG, Gründer G, Kumakura Y, Cumming P, Schreckenberger M, Smolka MN, Rösch F, Mann K, Bartenstein P: Correlation of alcohol craving with striatal dopamine synthesis capacity and D2/3 receptor availability: a combined [18F]DOPA and [18F]DMFP pet study in detoxified alcoholic patients. Am J Psychiatry 2005;162:1515-1520.
53.
Volkow ND, Fowler JS, Wang GJ, Baler R, Telang F: Imaging dopamine's role in drug abuse and addiction. Neuropharmacology 2009;56(suppl 1):3-8.
54.
Frank MJ, Seeberger LC, O'Reilly RC: By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 2004;306:1940-1943.
55.
Maia TV, Frank MJ: From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosc 2011;14:154-162.
56.
Frank MJ: Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism. J Cogn Neurosci 2005;17:51-72.
57.
Davies SJ, Pandit SA, Feeney A, Stevenson BJ, Kerwin RW, Nutt DJ, Marshall EJ, Boddington S, Lingford-Hughes A: Is there cognitive impairment in clinically ‘healthy' abstinent alcohol dependence? Alcohol Alcohol 2005;40:498-503.
58.
Moselhy HF, Georgiou G, Kahn A: Frontal lobe changes in alcoholism: a review of the literature. Alcohol Alcohol 2001;36:357-368.
59.
Killcross S, Coutureau E: Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex 2003;13:400-408.
60.
Duncan J, Emslie H, Williams P, Johnson R, Freer C: Intelligence and the frontal lobe: the organization of goal-directed behavior. Cogn Psychol 1996;30:257-303.
61.
Schwabe L, Wolf OT: Stress prompts habit behavior in humans. J Neuroscience 2009;29:7191-7198.
62.
Schwabe L, Tegenthoff M, Hoffken O, Wolf OT: Simultaneous glucocorticoid and noradrenergic activity disrupts the neural basis of goal-directed action in the human brain. J Neurosci 2012;32:10146-10155.
63.
Hogarth L, Chase HW, Baess K: Impaired goal-directed behavioural control in human impulsivity. Q J Exp Psychol 2012;65:305-316.
64.
Beck A, Schlagenhauf F, Wüstenberg T, Hein J, Kienast T, Kahnt T, Schmack K, Hägele C, Knutson B, Heinz A, Wrase J: Ventral striatal activation during reward anticipation correlates with impulsivity in alcoholics. Biol Psychiatry 2009;66:734-742.
65.
De Wit H: Impulsivity as a determinant and consequence of drug use: a review of underlying processes. Addict Biol 2009;14:22-31.
66.
Montague PR, Dayan P, Sejnowski TJ: A framework for mesencephalic dopamine systems based on predictive hebbian learning. J Neurosci 1996;16:1936-1947.
67.
Schultz W: Dopamine neurons and their role in reward mechanisms. Curr Opin Neurobiol 1997;7:191-197.
68.
Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH: A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 2013;16:966-973.
69.
Bayer HM, Glimcher PW: Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 2005;47:129-141.
70.
Dayan P: Dopamine, reinforcement learning, and addiction. Pharmacopsychiatry 2009;42(suppl 1):S56-S65.
71.
Valentin VV, Dickinson A, O'Doherty JP: Determining the neural substrates of goal-directed learning in the human brain. J Neurosci 2007;27:4019-4026.
72.
De Wit S, Corlett PR, Aitken MR, Dickinson A, Fletcher PC: Differential engagement of the ventromedial prefrontal cortex by goal-directed and habitual behavior toward food pictures in humans. J Neurosci 2009;29:11330-11338.
73.
Lee SW, Shimojo S, O'Doherty JP: Neural computations underlying arbitration between model-based and model-free learning. Neuron 2014;81:687-699.
74.
Field M, Wiers RW, Christiansen P, Fillmore MT, Verster JC: Acute alcohol effects on inhibitory control and implicit cognition: implications for loss of control over drinking. Alcohol Clin Exp Res 2010;34:1346-1352.
75.
Ersche KD, Jones PS, Williams GB, Turton AJ, Robbins TW, Bullmore ET: Abnormal brain structure implicated in stimulant drug addiction. Science 2012;335:601-604.
76.
Dalley JW, Fryer TD, Brichard L, Robinson ES, Theobald DE, Laane K, Pena Y, Murphy ER, Shah Y, Probst K, Abakumova I, Aigbirhio FI, Richards HK, Hong Y, Baron JC, Everitt BJ, Robbins TW: Nucleus accumbens D2/3 receptors predict trait impulsivity and cocaine reinforcement. Science 2007;315:1267-1270.
77.
Flagel SB, Robinson TE, Clark JJ, Clinton SM, Watson SJ, Seeman P, Phillips PE, Akil H: An animal model of genetic vulnerability to behavioral disinhibition and responsiveness to reward-related cues: implications for addiction. Neuropsychopharmacology 2010;35:388-400.
© 2014 S. Karger AG, Basel
2014
Copyright / Drug Dosage / Disclaimer
Copyright: All rights reserved. No part of this publication may be translated into other languages, reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, microcopying, or by any information storage and retrieval system, without permission in writing from the publisher.
Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug.
Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.
You do not currently have access to this content.