Study notes for Shai Shalev-Shwartz and Shai Ben-David’s Understanding Machine Learning: From Theory to Algorithms.
Notes
| # | Note | Status |
|---|---|---|
| 1 | Introduction to Generalization | polished |
| 3 | A Formal Learning Model | draft |
| 4 | Learning via Uniform Convergence | draft |
| 5 | The Bias-Complexity Tradeoff | draft |
Concept Map
- Statistical learning framework: domain, labels, data distribution, target function, learner, and hypothesis space.
- Generalization: population loss, empirical loss, ERM, overfitting, and inductive bias.
- Finite hypothesis classes: realizability, misleading samples, union bounds, and sample complexity.
- Uniform convergence: representative samples, Hoeffding bounds, and agnostic PAC learnability of finite classes.
- Bias-complexity tradeoff: the No-Free-Lunch theorem, prior knowledge as hypothesis-class choice, and the decomposition of error into approximation and estimation terms.
References
- Book: Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms.