Mathematics and Applications of Machine Learning

Description

This course will give an introduction to selected topics on machine learning. We will start from the basic perceptron and proceed with support vector machines, multi-layer networks, and aspects of deep learning. The mathematical discussion will focus on machine learning as an optimization problem. As regards applications, it is the goal of this lecture and its tutorials to implement several applications of the discussed algorithms in Python. Therefore, basic knowledge in Python programming and access to a computer with a Python development environment is expected – and will be required to complete the exercises. If time permits and depending on the interest, we may furthermore discuss aspects of recurrent networks and reinforcement learning.

About this course material

  • Purpose: Originally I do not come from the field of artificial intelligence or machine learning. My main field of research is mathematical physics. Naturally, it may be asked why someone like me would give a course about machine learning. My intention is this: On the one hand, with today’s computer resources and the access to large amounts of data, machine learning has regained its significance during the last decade. By now there are even many interesting applications in physics and likewise in other sciences. On the other hand, mathematics has yet only scratched the surface of understanding some of the successful machine learning algorithms while many of these mathematical problems are closely related to ones that have been studied in great depth, e.g., in statistical mechanics, probability and optimization theory. In view of this, I want to advertise for the mathematics and applications of machine learning in the fields of mathematics and physics and give an introduction in terms of ‘our jargon’. Let us take as an example the following quote and sneak in the term computer science:

    “Ich kann es nun einmal nicht lassen, in diesem Drama von Mathematik und Physik – die sich im Dunkeln befruchten, aber von Angesicht zu Angesicht so gerne einander verkennen und verleugnen – die Rolle des (wie ich genugsam erfuhr, oft unerwünschten) Boten zu spielen.”

    —Hermann Weyl, Gruppentheorie und Quantenmechanik, Hirzel, Leipzig 1928

  • Selection of material: This lecture will not follow one particular text-book. Rather we will pick out topics from various sources here and there. I will try to cite these sources and references to the best of my knowledge. Please let me know if you feel an appropriate citation to books, papers, source code, media files, etc., is missing in which case I will add it.

  • Style: As regards style, these notes are written in the form of roadmaps for the material discussed in class. Hence, in most parts this written material is less detailed than our discussion in class, however, should serve as a guide through the topics. I am happy about every feedback concerning if and where you felt the notes came too short.

  • Typos: As always, also these notes have been written in quite a haste during the semester and will contain lots of typos. If you find some please help to improve these notes by reporting them including precise references (URL, equation number, etc.) to my email address above. I will add a hall of fame for everyone who got involved improving these notes.

License

The Mathematics and Applications of Machine Learning course material by Dirk - André Deckert is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.