4. Optimization theory¶
Todo
Under construction. See for example [vapnik_statistical_1998].
4.1. Existence of optimal solutions¶
Todo
Under construction. See for example [boyd_convex_2004].
- Proof that SVMs have a unique optimal solution.
4.2. Convex optimization¶
Todo
Under construction. See for example [boyd_convex_2004], [jarre_optimierung_2013].
4.3. Optimal conditions for constraint convex programs¶
Todo
Under construction. See for example [boyd_convex_2004], [jarre_optimierung_2013].
- Derive KKT conditions from hyperplane separation theorems
- Apply KKT to example constraint optimization program
- Apply KKT to SVM: Meaning of the support vectors
4.4. Lagrange function¶
Todo
Under construction. See for example [boyd_convex_2004], [jarre_optimierung_2013].
- Saddle points
- Relation between saddle point of $L$ and KKT conditions
4.5. Primal and dual form¶
Todo
Under construction. See for example [boyd_convex_2004], [jarre_optimierung_2013], [mohri_foundations_2012].
- Derive dual form for SVM: Why is this dual form advantageous?