Short course on control theory for advanced CNS

The aim of lectures is to give some examples of control problems in neuroscience.

General background material:

• chapter 1 from the book Dynamic programming and optimal control by Dimitri Bertsekas. Copies 1a Copies 1b (from 1st edition, 2nd edition is current).
• my ICML 2008 tutorial text will be published in a book Inference and Learning in Dynamical Models (Cambridge University Press 2010), edited by David Barber, Taylan Cemgil and Sylvia Chiappa.
• Kappen Gomez Opper 2012
  
• slides

• The course starts with the notions of dynamic programming, Bellman equation and path integral control.
• Differential Dynamic Programming or Iterative LQG. I show the optimal control computation for the linear quadratic problem; I show how this solution can be used to iteratively compute the solution for a deterministic non-linear control problem using a method called Differential Dynamic Programming (DDP, Mayne 1966). DDP is very similar to a method called Iterative LQG (ILQG), developed by Todorov and Li in 2005. This latter method is applied to control of a biological arm in a reaching task.
  • MAYNE, D., A Second-Order Gradient Method for Determining Optimal Trajectories of Nonlinear Discrete-Time Systems, International Journal on Control, Vol. 3, pp. 85-95, 1966.
  • D. M. Murray, S.J. Yakowitz, Differential Dynamic Programming and Newton's Method for Discrete Optimal Control Problems pdf. This paper outlines the DDP method, which is similar to ILQG.
  • D. Todorov, W. Li, A generalized iterative LQG method for locally optimal feedback control of constrained nonlinear stochastic systems pdf. This paper outlines the ILQG method and applies to biological motor control task.
  • Y. Tassa, T. Erez, E. Todorov, Fast Model Predictive Control for Reactive Robotic Swimming pdf. This paper outlines the DDP method for robotic swimming.
  • Model free path integral control as described in Kappen notes.
  • Consider the motor control problem of the acrobot Kappen notes.
    • Implement a controller based on ILQG for this problem using the software given on Todorov software.
    • Compare the performance with the model free path integral control solution described in Kappen notes and implemented in this software.
• Goal directed planning in hippocampus Recently, it has been shown that rats hippocampal place cell show activity to previously visited goal locations when the rat is planning its trajector.
  • Johnson and Redish 2007
  • Pfeiffer and Foster 2013
  Build a model to explain these findings using KL control theory KL Learning for rat