CDS Machine Learning - autumn 2021

Course information

This course is part of the minor track Computational Data Science. It provides an advanced introduction to machine learning. The course is intended for Master's students in physics and mathematics. Other students are advised to take the course Statistical machine learning prior to this course.
For physics and math students, this course is the follow-up of the bachelor course Inleiding Machine Learning

Format: The course will be weekly sessions. Emphasis is on learning the material through mathematical derivation and computer exercises.

Course material:

Week Topic Material Exercises
1 36 Probability, entropy and inference MacKay Chapter 2

Further reading: exponential_families.pdf
Exercises: Mackay 2.10, extra_opgaven 2.1, 2.2, 2.3, 2.4
Other recommended exercises (not for the grade, solutions are in the book): Mackay 2.14, 2.16ab, 2.18, 2.19, 2.26
2 37 More about inference
Model comparison and Occam's Razor
MacKay Chapter 3, Chapter 27, Chapter 28
Exercise MacKay 3.12, 28.1
extra_opgaven 3.1, 27.1, 28.1, 28.2
Other recommended exercises (not for the grade, solutions are in the book): MacKay 3.1, 3.2, 3.5, 3.8, 3.9
3 38 Classification
  • Perceptrons
  • Generalization and VC dimension
See slides
Further reading: VC_dimension
extra_opgaven/Other exercises/Perceptron 1, 2, 3, 4
4 39 Gradient methods
MLPs
See slides Gradient descent exercise
program template
MNIST data
5 40 Deep networks See slides Deep learning tensorflow exercise
6 41 Graphical models Murphy Chapter 10
sections: 10.1, 10.2.1, 10.2.2, 10.3, 10.4
extra_opgaven/Other exercises/Graphical models/1
7 42 Mixture models and EM
Autoencoders
Murphy Chapter 11
sections: 11.1, 11.2 (not 11.2.4), 11.3 (not 11.3.1), 11.4, 11.4.1, 11.4.2 (not 11.4.2.6), 11.4.7
extra_opgaven/Other exercises/Mixture models and EM/1,2

Reproduce Murphy fig. 11.11 using the EM algorithm. Here are the data Old Faithful Geyser


Examination:
There will be no final examination. The students will work in groups of maximum 3 persons. The grade will be based on these exercises.