Learning & Decision Making
Cognitive, Neuroscientific and Psychiatric Research on Learning and Decision Making
How do humans and animals make decisions? How do they learn to make better decisions from trial and error? And how does "artificial intelligence" learn? In this seminar, we read about classic and recent research into the cognitive, computational and neural processes that underlie learning and decision making, building towards how these processes are studied in computational psychiatry.
At a glance
Requirements
- Read the assigned research articles each week
- Give one 45-min presentation in class, followed by a 30-min discussion
- Come to each class with one question about the topic, ready to raise it
Presentation guidelines
Each presentation (40–45 min, one presenter per session) should cover:
- Relevant background and motivation for the study — what the question is and why it is interesting
- The study design and methodological aspects (number of subjects, task description, conditions, duration) — what was done
- An explanation of any concepts that may be unfamiliar to your colleagues
- Hypotheses — what specifically is expected in the data
- Results — what they are and how they were obtained
- A one-slide summary that flags open questions or issues
Discussion guidelines
Each discussion (25–30 min) should be an interactive in-class session that engages with the topic of the presentation and uses the submitted question as input. For example: round tables for different questions, hands-on demonstrations, short presentations, or discussion of a related paper.
Sessions & readings
Session 1
Foundations & review of basic knowledge
- Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction (2nd ed.). MIT Press.
- Dayan, P., & Niv, Y. (2008). Reinforcement learning: the good, the bad and the ugly. Current Opinion in Neurobiology, 18(2), 185–196.
Session 2
Foundations cont'd & prediction errors
- Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599.
Session 3
Prediction errors in the brain
- O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38(2), 329–337.
- Background: O'Doherty, J. P., Hampton, A., & Kim, H. (2007). Model-based fMRI and its application to reward learning and decision making. Annals of the New York Academy of Sciences, 1104(1), 35–53.
Session 4
Value signals in the brain
- Padoa-Schioppa, C., & Assad, J. A. (2006). Neurons in the orbitofrontal cortex encode economic value. Nature, 441(7090), 223–226.
- O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001). Abstract reward and punishment representations in the human orbitofrontal cortex. Nature Neuroscience, 4(1), 95–102.
Session 5
Exploration vs. exploitation
- Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), 876–879.
Session 6
Model-free vs. model-based decision making
- Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711.
Session 7
Task states
- Schuck, N. W., Cai, M. B., Wilson, R. C., & Niv, Y. (2016). Human orbitofrontal cortex represents a cognitive map of state space. Neuron, 91(6), 1402–1412.
Session 8
Reinforcement learning in mood and anxiety disorders
- Pike, A. C., & Robinson, O. J. (2022). Reinforcement learning in patients with mood and anxiety disorders vs. control individuals: A systematic review and meta-analysis. JAMA Psychiatry, 79(4), 313–322.
Session 9
Model-based / model-free: psychiatric dimensions
- Gillan, C. M., Kosinski, M., Whelan, R., Phelps, E. A., & Daw, N. D. (2016). Characterizing a psychiatric symptom dimension related to deficits in goal-directed control. eLife, 5, e11305.
Session 10
Model-basedness & compulsivity / intrusive thought
- Donegan, K. R., Brown, V. M., Price, R. B., Gallagher, E., Pringle, A., Hanlon, A. K., & Gillan, C. M. (2023). Using smartphones to optimise and scale-up the assessment of model-based planning. Communications Psychology, 1(1), 31.
Session 11
Value learning and uncertainty in OCD
- Aberg, K. C., Toren, I., & Paz, R. (2022). A neural and behavioral trade-off between value and uncertainty underlies exploratory decisions in normative anxiety. Molecular Psychiatry, 27(3), 1573–1587.
Session 12
Decision temperature and psychopathology
- Moutoussis, M., Garzón, B., Neufeld, S., Bach, D. R., Rigoli, F., Goodyer, I., ... & Dolan, R. J. (2021). Decision-making ability, psychopathology, and brain connectivity. Neuron, 109(12), 2025–2040.
Session 13
Reinforcement learning in schizophrenia
- Geana, A., Barch, D. M., Gold, J. M., Carter, C. S., MacDonald III, A. W., Ragland, J. D., ... & Frank, M. J. (2022). Using computational modeling to capture schizophrenia-specific reinforcement learning differences and their implications on patient classification. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 7(10), 1035–1046.
Session 14
Discussion
Closing discussion of the themes covered across the seminar.