Short course on Machine Learning spring 2003*
Pompeu Fabra, Department of Technology
Lecturer: Bert Kappen

This course gives the basic foundation for probabilistic inference, which is currently the dominant paradigm for Machine Learning and Artificial Intelligence with uncertainty.
The course is based on the following two books, neither of which is finished, but both of which are available electronically:

Information Theory, Inference and Learning Algorithms from David MacKay, Cambridge UK

An introduction to probabilisitic graphical models from Michael Jordan, Berkeley USA.

The course is held on 3, 4 and 5 March from 3 pm to 7 pm (two lectures per day). The course is intended for Master's students and PhD students with some mathematical background, as well as anyone else that is interested in this topic.

The course outline is as follows:
TopicMaterialExcersizes
Probability, Entropy and inference
Balls and vases
Statistical inference, Bayes' Rule
Bent coin, legal evidence, Model Comparison
MacKay Ch 2
MacKay Ch 3
Further reading:
MacKay 30
Mackay 2.4,2.8,2.21,2.40,3.1
Exact inference in graphs
conditional independence
Elimination, Probability propagation, Junction tree
Jordan 2 (pg. 2-16)
Sheets (pg. 18-25)
MacKay 23.1
MacKay 28.1
Further reading:
Jordan 3
Jordan 4
Jordan 17
MacKay 23.2
MacKay 28.2
MacKay 28.5
Clustering,
maximum likelihood estimation,
EM
MacKay 22 23.2 24
Jordan 11
exercise EM
Hidden Markov Models
Multi-variate Gaussian
Jordan 12
Jordan 13
exercise HMM
Factor Analysis, Kalman Filtering and smoothing
Jordan 14
Jordan 15
Linear perceptron
Laplace's Method
Monte Carlo methods
Sheets
MacKay 31
MacKay 42
MacKay 44
This lecture will be illustrated with a computer practical, using Matlab. To start this, download this file.
Then do unzip mcmc (or let windows figure it out) which creates a directory mcmc with a number of
matlab files in it. Do cd mcmc, and type matlab, and we are ready to go! In the same directory, you
will find a file readme.ps that give further instructions and excersizes.

*This course is described in the 'Doctorat en Informatica i comunicacio digital' as 'Aspectes de recuperacio de la informacio'.