1. Mathematics and Applications of Machine Learning

  • Institution: Mathematical Institute, LMU Munich
  • Term: Winter semester 2016/17
  • Lecturer: Dirk - Andre Deckert, deckert@math.lmu.de
  • Time: Wednesday, 14:00-16:00
  • Location: Lecture Hall A 027

1.1. News & Changelog

  • 2017-03-28: Exam results are available.

    Statistics:

    Average Grade Var Min Max
    2.1 1.2 1.0 5.0

    Weighting:

    Percent of each subtask was weighted and summed according to the following table to give a weighted percent score for the exam.

    Task 1 Task 2 Task 3
    Weight = .15 Weight = .15 Weight = .2
    a b c d a b a b c d
    .5 .1 .2 .2 .5 .5 .25 .25 .25 .25
    Task 4 Task 5
    Weight = .25 Weight = .25
    a b c d a b c d e
    .3 .2 .3 .2 .2 .2 .15 .15 .3

    A weighted percent score of 87% and above was graded 1.0 “Sehr gut”, < 50% was a fail, everything in between was linearly interpolated.

    Physics and TMP students:

    Please collect your “Schein” from

    Frau Heinemann, Room B 219 Mathematics Tower, Theresienstrasse 39.

    Mathematics students:

    Your results have been submitted electronically.

    All:

    Contact me by email for a copy of the marked exam.

  • 2017-01-31: Template structure for all missing sections with several reminders about some of the homework questions. At least until Feb 8, 2017 there will be a copy of my rough course notes available under: [PDF]. Step by step the online notes will be completed in the near future.

  • 2017-01-18: Exam is schedules for 14:00 February 8, 2017, Room A 027.

  • 2016-12-28: The handwritten number recognition challenge is open: [Link].

  • 2016-10-27: First source code available in Section First steps: Linear Classifiers.

  • 2016-10-26: Discussion session date and location updated.

  • 2016-10-19: Online.

1.2. Description

This course will give an introduction to selected topics on machine learning. We will start from the basic perceptron and proceed with support vector machines, multi-layer networks, and aspects of deep learning. The mathematical discussion will focus on machine learning as an optimization problem. As regards applications, it is the goal of this lecture and its tutorials to implement several applications of the discussed algorithms in Python. Therefore, basic knowledge in Python programming and access to a computer with a Python development environment is expected – and will be required to complete the exercises. If time permits and depending on the interest, we may furthermore discuss aspects of recurrent networks and reinforcement learning.

1.3. About this course material

  • Purpose: I am not a specialist in artificial intelligence or machine learning. My main area of research is mathematical physics. Naturally, It should then be asked why someone like me should give a course about machine learning. My intention is this: With today’s computer resources and the amount of available data, machine learning has regained its significance during the last decade. On the one hand, regarding the huge amounts of data that for instance scattering experiments at the CERN or astronomical observations produce, machine learning will soon have an impact on physics and other sciences requiring efficient numerics and data analysis tools. On the other hand, mathematics has yet only scratched the surface of understanding some of the successful machine learning algorithms while many of these mathematical problems are closely related to ones that have been studied in, e.g., statistical mechanics and probability theory. In this sense, I want to advertise for the mathematics and applications of machine learning in our field with an introduction using ‘our jargon’.

    “Ich kann es nun einmal nicht lassen, in diesem Drama von Mathematik und Physik – die sich im Dunkeln befruchten, aber von Angesicht zu Angesicht so gerne einander verkennen und verleugnen – die Rolle des (wie ich genugsam erfuhr, oft unerwünschten) Boten zu spielen.”

    —Hermann Weyl, Gruppentheorie und Quantenmechanik, Hirzel, Leipzig 1928

  • Selection of material: This lecture will not follow one particular text-book. Rather we will pick out topics from various sources here and there. I will try to cite these sources and references to the best of my knowledge. Please let me know if you feel an appropriate citation to books, papers, source code, media files, etc., is missing in which case I will add it.

  • Style: As regards style, these notes are written in the form of presentations which are discussed in class. Hence, in most parts this material is less detailed than our discussion, however, should serve as a good guide through the topics. I would be happy to received feedback concerning if and where you felt the notes came too short.

  • Typos: As it is always the case, these notes have been written in quite a haste during the semester and will contain lots of typos. If you find some please help to improve these notes by reporting them including precise references (URL, equation number, etc.) to my email address above. I will add a hall of fame for everyone who got involved improving these notes.

1.4. License

The Mathematics and Applications of Machine Learning course material by Dirk - André Deckert is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.