Short course on control theory for advanced CNS
Lecturer: Bert Kappen
The aim of lectures is to give some examples of control problems in
neuroscience.
General background material:
Topics:

The course starts with the notions of dynamic programming, Bellman equation and path integral control.

Differential Dynamic Programming or Iterative LQG. I show the optimal control computation for the linear
quadratic problem; I show how this solution can be used to iteratively compute the solution for a deterministic nonlinear control problem using a method
called Differential Dynamic Programming (DDP, Mayne 1966). DDP is very similar to a method called Iterative LQG (ILQG), developed by Todorov and Li in 2005.
This latter method is applied to control of a biological arm in a reaching task.
 MAYNE, D., A SecondOrder Gradient Method for Determining Optimal Trajectories
of Nonlinear DiscreteTime Systems, International Journal on Control, Vol. 3, pp. 8595, 1966.
 D. M. Murray, S.J. Yakowitz, Differential Dynamic Programming and Newton's Method for Discrete Optimal Control Problems
pdf. This paper outlines the DDP method, which is similar to ILQG.
 D. Todorov, W. Li, A generalized iterative LQG method for locally optimal feedback control of constrained nonlinear stochastic systems
pdf. This paper outlines the ILQG method and applies to biological motor control task.
 Y. Tassa, T. Erez, E. Todorov, Fast Model Predictive Control for Reactive
Robotic Swimming pdf. This paper outlines the DDP
method for robotic swimming.
 Model free path integral control as described in Kappen notes.

Consider the motor control problem of the acrobot Kappen notes.
 Implement a controller based on ILQG for this problem
using the software given on Todorov software.
 Compare the performance with the model free path integral control solution described in Kappen
notes and implemented in this software.
 Goal directed planning in hippocampus Recently, it has been shown that rats hippocampal place cell show activity
to previously visited goal locations when the rat is planning its trajector.
Build a model to explain these findings using KL control theory KL Learning for rat