Inference, neural networks and control

The course provides an advanced introduction to the modern view on Bayesian inference with application in neural networks and control theory.

Course material: The course is mainly based on

Presentation schedule:
Lecture Topic Material Exercises
1 Probability, entropy and inference MK 2,3 MK 2.4, 2.6 + continued, 2.7, 2.8 to be discussed in the class.
Exercises: MK 2.10, 2.14, 2.16ab, 2.18, 2.19, 2.26
MK 3.3, 3.4, 3.15 to be discussed in the class.
Exercises: MK 3.1, 3.2, 3.5, 3.6, (3.7 Bonus), 3.8, 3.9
2 Model comparison and Occam's raisor
Graphical models
MK 28
Bishop Chapter 8 [slides]
Skip 8.1.3
Exercises: MK 28.1, 28.2 only for model H_1, (28.3 Bonus)
8.1, 8.14 to be discussed in the class.
Exercises: 8.3, 8.4, 8.7, 8.10, 8.11, 8.15
3 Approximate inference cvm_sheets
nips paper
Run Belief Propagation for a discrete optimization problem see: opgave Ising model.
NB: Download LIBDAI from: LIBDAI.
4 Networks of binary neurons
Markov processes
Boltzmann Machines
Mean field approximation
Linear response approximation
sheets Boltzmann Machines
reader chapter 3, 4
reader chapter 3 exercise 2
reader chapter 4 exercises 1, 2
5 Monte Carlo Methods (2), HMC
Bayesian inference with perceptron
MK 29.1-29.5
MK 29.6,29.9 ,30.1, 30.3
MK 38, 39, 41
Exercise to compare MCMC with Belief Propagation discrete optimization problem see: opgave Ising model.
NB: Download LIBDAI from: LIBDAI.
An example of Baysian inference in perceptron learning using MCMC methods. The files (Matlabfiles and instructions) needed to do this exercise can be found here: [mcmc_mackay.tar.gz].
6 Exercises and recap sofar MK 2.10, MK 3.15 3.8 3.9
Example message passing in chain and loopy graph
Example sequential dynamics
Detailed balance, MF examples
7 Discrete time control
dynamic programming
Bellman equation
Bertsekas 2-5, 13-14, 18, 21-32 (2nd ed.)
Bertsekas 2-5, 10-12, 16-27, 30-32 (1nd ed.)
Kappen ICML tutorial 1.2
slides up to 28
Ex: Carry out the calculations needed to verify that J0(1)=2.7 and J0(2)=2.818 in Bertsekas Example 3.2 on pg. 23-25 in Copies 1b
extra exercise 1, 2a,b
8 Continuous time control
Hamilton-Jacobi-Bellman Equation
Pontryagin Minimum Principle
Stochastic differential equations
Stochastic optimal control
LQ examples, Portfolio management
Kappen ICML tutorial 1.3, 1.4
slides up to 69
extra exercise 2a,b
9 Path integral control theory
Kappen ICML tutorial 1.5, 1.6, 1.7
slides up to 93
extra exercise 2c, 3

If time permits:
Lecture Topic Material Exercises
11 Path integral control theory
MC Sampling solution
Numerical examples (particle in a box, N joint arm, Robot learning)
Kappen ICML tutorial 1.7
slides up to 127
extra exercise 4,5 Matlab code for n joint problem
Here is a directory of matlab files, which allows you to run and inspect the variational approximation for the n joint stochastic control problem as discussed in the tutorial text section 1.6.7. Type tar xvf njoints.tar to unpack the directory and simply run file1.m. In file1.m you can select demo1 (3 joint arm) or demo2 (10 joint arm). You can also try larger n but be sure to adjust eta for the smoothing of the variational fixed point equations. You can compare the results with exact cmputation (only recommendable for 2 joints) by setting METHOD='exact'. There is also an implementation of importance sampling (does not work very well) and Metropolis Hastings sampling (works nice, but not as stable as the variational approximation).
12 Lasso lasso slides
Sparse regression computer exercise
  • Derive the sequential Gauss-Seidel update rule from lasso slides Eq. 1.
  • Write your own Lasso method using coordinate descent.
  • Test your algorithm on data set 1 lasso data Reproduce a figure similar to slide 17. Find the optimal value of gamma by cross validation. Compare the Lasso result with ridge regression (with optimized ridge regression parameter found by cross validation).
  • Consider the example of correlated inputs on slide 21. Reproduce these results with your software using data generated by correlated_data.m. Compute the input output correlations b_i and use this to explain the observed phenomenon.
Write a brief report on your findings and include your source code.
13 Spike and slab
Variational Garrote
L0 slides
George and McCulloch 1993
Kappen 2011

6 Ising model MK 31 MK 31.1, 31.3
8a Attractor neural networks sheets attractors
5 Perceptrons DA 8.4
sheets supervised 1
sheets supervised 2
DA 8.8, 8.9