Foundations of AI

Intelligence in Biological and Artificial Networks

AI has begun to make an impact on a small and big scale. How do ChatGPT and co. work, and does it have anything to do with how the brain works? This seminar focuses on artificial neural networks, the key technology underlying ChatGPT and co. What are neural networks? How were they developed, and what is the relationship between neural networks and psychological and neuroscientific research? The seminar traces the arc from early connectionist models through deep learning to today's large language models, with an emphasis on understanding the key ideas and their connections to psychology and neuroscience.

At a glance

  • Programme: Cognitive Neuroscience module, MSc Psychology, Institute of Psychology, Universität Hamburg
  • Instructor: Prof. Schuck
  • Format: weekly 90-minute seminar during the summer term

Requirements

  • Read the assigned research articles each week
  • Give one 25–30 min presentation together with a co-presenter
  • Moderate one 25–30 min discussion
  • Actively participate in discussions

Presentation guidelines

Each presentation (25–30 min) should cover:

  • Relevant background and motivation — what the question is and why it is interesting
  • The design and methodological aspects (computational methods, participants, where applicable) — what was done
  • An explanation of any concepts that may be unfamiliar to your colleagues
  • Results — what they are and how they were obtained
  • A one-slide summary that flags open questions or issues

Tips: Less is more — take your time to break ideas down carefully, even at the expense of not getting everything across; avoid complex terminology where possible; pause to give room for questions.

Discussion guidelines

Each discussion (25–30 min) should be an interactive in-class session that engages with the topic of the presentation:

  • Design an interactive session and coordinate with the presenter
  • For example: round tables for different questions, or hands-on demonstrations
  • If you run group discussions, engage with participants — have prompts and counter-arguments ready, and actively moderate

Sessions & readings

Session 1

Introduction & short history of AI

Basic concepts in AI; differences between neural networks and other approaches.

  • Muggleton, S. (2014). Alan Turing and the development of Artificial Intelligence. AI Communications, 27(1).
  • Turing, A. M. (1936). On computable numbers, with an application to the Entscheidungsproblem. Journal of Mathematics, 58, 345–363.
  • Dennett, D. (2009). Darwin's "strange inversion of reasoning." PNAS, 106(S1), 10061–10065.
Session 2

Perceptrons & the connectionist vision

McCulloch–Pitts neurons, perceptrons, their limits, and the PDP response.

  • Minsky, M., & Papert, S. (1988). Perceptrons: An Introduction to Computational Geometry. MIT Press. [Ch. 1–2]
  • McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, 115–133.
Session 3

NeuroAI: psychology, neuroscience, and AI

Critical reflection on the relationship between these disciplines.

  • Gershman, S. J. (2024). What have we learned about artificial intelligence from studying the brain? Biological Cybernetics, 118(1), 1–5.
  • Zador, A., Escola, S., Richards, B. et al. (2023). Catalyzing next-generation Artificial Intelligence through NeuroAI. Nature Communications, 14, 1597.
Session 4

Parallel distributed processing: distributed representations

Understand distributed representations and how neural networks are trained.

  • Rumelhart, D. E., Hinton, G. E., & McClelland, J. L. (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1. [Ch. 3]
Session 5

Deep networks & representation learning

Understand what deep neural nets are, why depth is useful, and what representations emerge.

  • Hinton, G. E. (2007). Learning multiple layers of representation. Trends in Cognitive Sciences, 11(10), 428–434.
  • LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
Session 6

Convolutional neural networks

CNNs as an inductive bias for spatial structure; image classification; visualizing what networks learn.

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. NeurIPS, 25.
  • Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. ECCV 2014, 818–833.
Session 7

Recurrent networks & the sequence problem

Understand how networks process sequences; why long-range dependencies are hard; LSTMs.

  • Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14(2), 179–211.
  • Schmidt, R. M. (2019). Recurrent neural networks (RNNs): A gentle introduction and overview. arXiv:1912.05911.
Session 8

Attention & the transformer architecture

Understand the attention mechanism and the transformer architecture.

  • Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. NeurIPS, 30.
  • Shanahan, M. (2024). Talking about large language models. Communications of the ACM, 67(2), 68–79.
  • Supplementary video (visual introduction to attention): youtube.com
Session 9

Large language models

Understand how large language models are trained.

  • Bommasani, R., Hudson, D. A., Adeli, E., et al. (2021). On the opportunities and risks of foundation models. arXiv:2108.07258.
  • Ouyang, L., Wu, J., Jiang, X., et al. (2022). Training language models to follow instructions with human feedback. NeurIPS, 35, 27730–27744.
  • Background — how LLMs work, from pre-training to inference: towardsdatascience.com
  • Supplementary video (LLM overview): youtube.com
Session 10

Strengths and problems of LLMs

Understand the current state of AI and key trends.

  • Stanford HAI. AI Index Report, Chapter 2 (highlights and selected benchmarks): stanford.edu
Session 11

LLMs in psychology & neuroscience research

Understand how LLMs are used as tools and models in psychological and neuroscience research.

  • Binz, M., & Schulz, E. (2023). Using cognitive psychology to understand GPT-3. PNAS, 120(6).
  • Demszky, D., Yang, D., Yeager, D. S. et al. (2023). Using large language models in psychology. Nature Reviews Psychology, 2, 688–701.
Session 12

LLMs and deep neural networks as models of the brain

Understand how deep neural networks are used as models in neuroscience and cognitive science.

  • Cichy, R. M., & Kaiser, D. (2019). Deep neural networks as scientific models. Trends in Cognitive Sciences, 23(4), 305–317.
  • Storrs, K. R., Kietzmann, T. C., Walther, A., Mehrer, J., & Kriegeskorte, N. (2021). Diverse deep neural networks all predict neural responses in higher visual cortex equally well. Frontiers in Computational Neuroscience, 15, 605.
Session 13

AI safety & ethics

Major ethical considerations around AI; public attitudes toward AI.

  • Stanford HAI. AI Index Report, Chapter 3 (highlights): stanford.edu
  • Gyevnár, B., & Kasirzadeh, A. (2025). AI safety for everyone. Nature Machine Intelligence, 7, 531–542.
Back to Home