Deep learning notes centered on neural network architectures, D2L, 李宏毅’s lectures, and related model families.
Notes
| # | Note | Status |
|---|---|---|
| 1 | Feed-forward Neural Networks | draft |
| 2 | Convolutional Neural Networks | draft |
| 3 | CNN Architecture | draft |
| 4 | Modern CNNs | draft |
| 5 | Recurrent Neural Networks | draft |
| 6 | Modern RNNs | draft |
| 7 | Attention and Transformer | draft |
| 8 | Transformer and Variants | draft |
Concept Map
- Feed-forward networks and basic neural network training
- CNNs and modern convolutional architectures
- RNNs, sequence modeling, attention, and Transformer architectures
- Transformer variants and state-space-style sequence models
References
- Reading: Dive into Deep Learning (D2L), https://d2l.ai/
- Lecture: 李宏毅, Machine Learning, https://speech.ee.ntu.edu.tw/~hylee/ml/2021-spring.php
- Lecture: 李宏毅, 生成式 AI 時代下的機器學習 (2025)
- Reading: 邱锡鹏, 《神经网络与深度学习》
Planned Notes
- Backpropagation details
- Batch Normalization
- Computer Vision
- Diffusion Models
- Mixture of Experts