Advanced Topics in Machine Learning

Created April 2017

Dirk - André Deckert

Mathematical Institute, LMU Munich, deckert@math.lmu.de
http://www.math.lmu.de/light-and-matter

1 Location and Time

2 News

3 Program

Date	Title	Speaker
03.05.17	no seminar	-
10.05.17	PAC Learning Model (blackboard, notes)	Francesco Romano
17.05.17	VC-Dimension (blackboard talk, notes)	Jukia Kraus
24.05.17	Boosting (blackboard talk)	Christoph Becker
31.05.17	word2vec (slides)	Yannick Couzinié
07.06.17	no seminar	-
14.06.17	no seminar	-
21.06.17	Unsupervised Learning (slides)	Shane Shang
28.06.17	Haar Cascades (slides)	Nora Kassner
05.07.17	Reinforcement Learning (slides)	Christian Hanauer
12.07.17	Recurrent Neural Networks (blackboard talk)	Dominik Meindl
19.07.17	no seminar	-
27.07.17	no seminar	-

4 Suggestion for topics

1. Mathematical background

2. PAC Learning Model

   • Book: http://www.cs.nyu.edu/~mohri/mlbook/

3. Complexity Analysis and VC-Dimension

   • Book: http://www.cs.nyu.edu/~mohri/mlbook/

4. Complexity analysis of support vector machines

   • Book: http://www.cs.nyu.edu/~mohri/mlbook/

5. Boosting methods

   • Book: http://www.cs.nyu.edu/~mohri/mlbook/

6. Approximation

   • Papers:
     • https://pdfs.semanticscholar.org/05ce/b32839c26c8d2cb38d5529cf7720a68c3fab.pdf
     • http://zmjones.com/static/statistical-learning/hornik-nn-1991.pdf

7. Architecture

8. Deep learning and convolution networks

   • Paper: https://arxiv.org/abs/1409.1556
   • Background:
     • https://github.com/janishar/mit-deep-learning-book-pdf/blob/master/complete-book-pdf/deeplearningbook.pdf
     • http://cs231n.github.io/
   • Tutorial: http://cs231n.github.io/convolutional-networks/

9. Neural network training

   • Background:
     • http://sebastianruder.com/optimizing-gradient-descent/
     • http://www.maths.ed.ac.uk/~prichtar/papers/Papamakarios.pdf
     • http://users.iems.northwestern.edu/~nocedal/PDFfiles/limited.pdf

10. Training of support vector machines

    • Book: http://www.cs.nyu.edu/~mohri/mlbook/
    • Implementation: http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf

11. Recurrent networks

    • Paper: https://arxiv.org/abs/1506.00019
    • Background:
      • http://karpathy.github.io/2015/05/21/rnn-effectiveness/
      • http://colah.github.io/posts/2015-08-Understanding-LSTMs/
      • https://www.tensorflow.org/tutorials/recurrent

12. Unsupervised learning

    • Books:
      • http://www.springer.com/de/book/9780387310732?referer=www.springer.de
      • http://www-users.cs.umn.edu/~kumar/dmbook/ch8.pdf
      • http://aima.cs.berkeley.edu
      • http://ciml.info/dl/v0_8/ciml-v0_8-all.pdf

13. Reinforcement learning

    • Paper: http://www.nature.com/nature/journal/v518/n7540/full/nature14236.html
    • Books:
      • http://incompleteideas.net/sutton/book/bookdraft2016sep.pdf
      • http://aima.cs.berkeley.edu
    • Tutorial: https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0

14. Special topics

15. Neural Turing machines

    • Papers:
      • https://arxiv.org/abs/1410.5401
      • https://arxiv.org/pdf/1505.00521.pdf

16. Image style transfer

    • Papers:
      • http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf
      • https://arxiv.org/abs/1505.07376
      • https://arxiv.org/abs/1508.06576
    • Tutorial: https://github.com/cysmith/neural-style-tf

17. Real-time face recognition with Haar cascades

    • Papers:
      • http://wearables.cc.gatech.edu/paper_of_week/viola01rapid.pdf
      • https://pdfs.semanticscholar.org/72e0/8cf12730135c5ccd7234036e04536218b6c1.pdf
      • http://www.florian-oeser.de/wordpress/wp-content/2012/10/crow-1984.pdf
    • Background: http://etd.dtu.dk/thesis/223656/ep08_93.pdf
    • Tutorial: https://www.cs.auckland.ac.nz/~m.rezaei/Tutorials/Face%20and%20Eye%20Detection%20using%20OpenCV-%20Step%20by%20Step.pdf

18. Natural language processing

19. Parts of speech tagging

    • Book: http://www.springer.com/de/book/9783827420237
    • Blog: https://explosion.ai/blog/part-of-speech-pos-tagger-in-python

20. word2vec

    • Paper: https://arxiv.org/pdf/1301.3781.pdf
    • Background:
      • https://arxiv.org/pdf/1411.2738.pdf
      • http://www.1-4-5.net/~dmm/ml/how_does_word2vec_work.pdf
    • Tutorials:
      • https://www.tensorflow.org/tutorials/word2vec
      • https://github.com/LasseRegin/gensim-word2vec-model

21. doc2vec

    • Paper: https://cs.stanford.edu/~quocle/paragraph_vector.pdf
    • Paper: https://arxiv.org/pdf/1607.05368.pdf
    • Tutorial: https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/doc2vec-lee.ipynb