The aim of this course is to provide an overview of some key theoretical concepts commonly used in computational neuroscience. The course consists of two parts, given by Bert Kappen and Paul Tiesinga. This page describes the part of the course taught by Bert Kappen.
Week |
Topic | Material | Weekly exercises | Take home exercises | |
| 5 | Lecture 1: Stochastic neural processes Poisson processes First passage time model |
DA chapter 1, chapter 5.4 handouts chapter 2 Gerstein and Mandelbrot, 1964 |
DA
1 Ex. 1 (here is the answer of 1.1) Handouts ch.2 Ex. 1, 2, 3a DA 5 Ex. 3 |
The drift diffusion model for reaction times driftdiffusion.pdf | |
| 0 | McCulloch-Pitts neurons, IF neuron, Perceptron Gradient descent rules, logistic regression Multi-layered perceptrons Deep neural networks |
slides CDS ML 66-90 slides CDS ML 98-106, 112-116 slides CDS ML 117-127 Handouts chapter 6 (based on HKP chapters 5 and 6) |
Handouts ch.6 Ex. 2,3,4 |
Choose one of these two exercises:
Or: |
|
| 6a | Lecture 2:
Sparse visual coding PCA, Sparse coding, Lasso |
Olshausen, Field 1996 Emergence of simple-cell receptive field properties by learning a sparse code for natural images |
Sparse coding exercise program template Natural image |
||
| 6b | Lecture 3: Stochastic neural networks Stochastic binary neurons and networks Markov Processes Mean field and linear response approximation |
handouts cns chapter 3 | |||
| 7 | Lecture 4: Boltzmann Machine Boltzmann machine learning Application to salamander retina data Restricted BM, contrasted divergence and collaborative filtering |
HKP chapter 7.1 DA 7.6 handouts cns chapter 4 Schneidmann et al. Nature 2006 |
handouts cns chapter 4 exercises 1a |
Boltzmann Machine
exercise salamander retina data. |
|
| 8 | Lecture 5: Attractor neural networks: The Hopfield model Delayed neural activity Evidence for attractor dynamics The Hopfield model |
HKP chapter 2 DA pg. 322-324 handounts cns chapter 5 Trappenberg 8.2 and 8.3 Further reading: Wills et al. Science 2005. Attractor in Hippocampus |
Capacity Hopfield network Attractor dynamics in hippocampus |
||
| 9 | RL 1 Trial level RL (DA 9.1,9.3) Real time RL (DA 9.2) Introduction to the theory of RL Introduction to control theory |
Sutton Barto Intro to theory of RL, Kaelbling Intro to control theory, Kappen slides animal learning slides RL and control theory Further reading: Montague et al. Waelti 2001 |
Reproduce fig. 17 in Sutton Barto DA 9.5 |
Construct the Bayesian solution of the two armed bandit problem using dynamic programming | |
| 10 | RL 2 Exploration and exploitation in the bandit problem Value iteration, policy iteration Model free and model based approaches Actor critic, TD learning Illustration of policy iteration and direct actor for animal learning (DA 9.4) Q learning and Dyna Linear function approximation Foster water maze |
Intro to theory of RL, Kaelbling Foster et al. slides RL and control theory |
DA 9.6, DA 9.8 and DA 9.9 | Reproduce the reinforcement learning result as presented in fig. 3 of Foster et al.. The more complex coordination based navigation model also discussed in the paper is not required. |
Exercises will be handed in one week after the assignment has been given. Exercises that are handed in on time will contribute to a total average. This total average on the scale of 0-1 will be added to the final grade.
Hand in the final result of your take home exercises of week 5,6,7 before March 15 2022 and the remaining exercises before April 15 2022.
There is overlap of this course with CDS Machine learning and with advanced Machine Learning. Very few students have not passed or followed CDS Machine learning are advised to study the material and exercises of the perceptron (week 0) on their own. This will not add to their grade.