CDS Machine Learning - autumn 2023

Course information

Half semester course (3 ec).
For: Master and PhD students in physics or mathematics
Teacher lectures: Bert Kappen
Teacher Practicals: Eduardo Dominguez and Aaron Villanueva and Peyman Najafi

This course is part of the minor track Computational Data Science. It provides an advanced introduction to machine learning. The course is intended for Master's students in physics and mathematics. Other students are advised to take the course Statistical machine learning prior to this course.
For physics and math students, this course is the follow-up of the bachelor course Inleiding Machine Learning

Format: The course will be weekly sessions. Emphasis is on learning the material through mathematical derivation and computer exercises.

Course material:

Some chapters from the book Information Theory, Inference and Learning Algorithms from David MacKay, which is online available free of charge.
Some chapters from the book Machine Learning: a probabilistic perspective from Kevin Murphy
These slides.pdf

Week Topic Material Exercises
1 36 Probability, entropy and inference MacKay Chapter 2

Further reading: exponential_families.pdf Exercises: Mackay 2.10, extra_opgaven 2.1, 2.2, 2.3, 2.4
Other recommended exercises (not for the grade, solutions are in the book): Mackay 2.14, 2.16ab, 2.18, 2.19, 2.26
2 37 More about inference
Model comparison and Occam's Razor MacKay Chapter 3, Chapter 27, Chapter 28
Exercise MacKay 3.12, 28.1
extra_opgaven 3.1, 27.1, 28.1, 28.2
Other recommended exercises (not for the grade, solutions are in the book): MacKay 3.1, 3.2, 3.5, 3.8, 3.9
3 38 Classification

Perceptrons
Generalization and VC dimension
See slides
Further reading: VC_dimension extra_opgaven/Other exercises/Perceptron 1, 2, 3, 4

4 39 Gradient methods
MLPs
See slides Gradient descent exercise
program template
MNIST data
5 40 Deep networks See slides Deep learning tensorflow exercise
6 41 Graphical models Murphy Chapter 10
sections: 10.1, 10.2.1, 10.2.2, 10.3, 10.4 extra_opgaven/Other exercises/Graphical models/1
7 42 Mixture models and EM
Autoencoders Murphy Chapter 11
sections: 11.1, 11.2 (not 11.2.4), 11.3 (not 11.3.1), 11.4, 11.4.1, 11.4.2 (not 11.4.2.6), 11.4.7 extra_opgaven/Other exercises/Mixture models and EM/1,2

Reproduce Murphy fig. 11.11 using the EM algorithm. Here are the data Old Faithful Geyser

Examination:
There will be no final examination. The students will work in groups of maximum 3 persons. The grade will be based on these exercises.

	Week	Topic	Material	Exercises
1	36	Probability, entropy and inference	MacKay Chapter 2 Further reading: exponential_families.pdf	Exercises: Mackay 2.10, extra_opgaven 2.1, 2.2, 2.3, 2.4 Other recommended exercises (not for the grade, solutions are in the book): Mackay 2.14, 2.16ab, 2.18, 2.19, 2.26
2	37	More about inference Model comparison and Occam's Razor	MacKay Chapter 3, Chapter 27, Chapter 28	Exercise MacKay 3.12, 28.1 extra_opgaven 3.1, 27.1, 28.1, 28.2 Other recommended exercises (not for the grade, solutions are in the book): MacKay 3.1, 3.2, 3.5, 3.8, 3.9
3	38	Classification Perceptrons Generalization and VC dimension	See slides Further reading: VC_dimension	extra_opgaven/Other exercises/Perceptron 1, 2, 3, 4
4	39	Gradient methods MLPs	See slides	Gradient descent exercise program template MNIST data
5	40	Deep networks	See slides	Deep learning tensorflow exercise
6	41	Graphical models	Murphy Chapter 10 sections: 10.1, 10.2.1, 10.2.2, 10.3, 10.4	extra_opgaven/Other exercises/Graphical models/1
7	42	Mixture models and EM Autoencoders	Murphy Chapter 11 sections: 11.1, 11.2 (not 11.2.4), 11.3 (not 11.3.1), 11.4, 11.4.1, 11.4.2 (not 11.4.2.6), 11.4.7	extra_opgaven/Other exercises/Mixture models and EM/1,2 Reproduce Murphy fig. 11.11 using the EM algorithm. Here are the data Old Faithful Geyser