6. Neural networks

Todo

Under construction. See for example [vapnik_statistical_1998].

6.1. Multi-layer networks

6.2. Learning and update rule

6.2.1. Iris flower classification

6.3. An efficient training algorithm: Backpropagation

Todo

Under construction. See for example. [vapnik_statistical_1998].

• Derive backpropagation either by hand or apply KKT conditions to neural network optimization program.

6.3.1. Handwritten number classification

6.4. Learning behavior and strategies

Todo

Under construction.

• Example for slow learning problem.
• Is cross-entropy always better?
• Derive learning & update rule for loss function with regularization.