Calendar

Week #1: Jan 20–23

1/21

• Syllabus
• Chapter 1: 30,000-feet overview

Week #2: Jan 26–30

1/28

• Hands-On Intro to Deep Learning using Keras on MNIST

  1st Assignment

Week #3: Feb 2–6

2/2

Chapter 2: Linear algebra. Vectors and tensors.

Week #4: Feb 9–13

2/9

Chapter 2: Linear algebra. Vectors and tensors (continued). Singular Value Decomposition & Intro to NumPy.

2nd Assignment

Week #5: Feb 16–20

2/16

Chapter 3: Probability spaces, probability measures and distributions. Joint random variables (random vectors). Conditional probability and independence.

Week #6: Feb 23–27

2/23

Chapter 3 (continued): Probability density functions and change of variables (chain rule). Normal (Gaussian) jointly distributed random variables/vectors. Bayes’ Rule. Information theory: Shannon entropy and Kullback-Leibler (KL) Divergence.

3rd Assignment

Week #7: Mar 2–6

3/2

Chapter 4: Vector calculus. Chain rule. Gradient-based optimization. Gradient descent and learning rate.

Supplementary reading: Deisenroth-Faisal-Ong §5.1–§5.4

Week #8: Mar 16–20

3/16

Midterm Presentations: Download handouts

Week #9: Mar 23–27

3/23

Sections 5.1–5.2: Machine Learning Basics: Types of ML tasks. Supervised and unsupervised learning. Capacity, overfitting and underfitting. “No Free Lunch” theorem.

Week #10: Mar 30–April 3

3/30

Section 5.4: Machine Learning Basics: Statistical parameter estimation. Bias-Variance Trade-off. Overtraining and “Grokking”. Colab lesson.

Week #11: April 6–10

4/6

Section 5.7: Machine Learning Basics: Stochastic Gradient Descent (SGD) (Colab lesson.)

Week #12: April 13–17

4/13

Section 5.7: Machine Learning Basics: Support-Vector Machines.

(Colab lesson.)

Overview of Deep Network architectures. (Colab lesson.)

4th Assignment

Week #13: April 20–24

4/20

Overview of Deep Network architectures (continued). (Colab lesson.)

Sections 6.2 & 6.3 Feedforward networks in detail: Forward pass and activation functions (“units”). (Colab lesson.)

4/22

Section 6.5 The Back-Propagation Algorithm. Automatic Differentiation. (Colab lesson.)

Week #14: April 27–May 1

5th Assignment (Due Monday 4 May)

4/27

Chapter 20 [Primarily §20.10–12] Generative Models. Variational Autoencoders (VA). Autoregressive models & LLMs. (Colab lesson.)

Supplementary reading:

• D. P. Kingma & M. Auto-Encoding Variational Bayes.
• D. J. Rezende & S. Mohamed Variational Inference with Normalizing Flows.

4/29

Memory architectures for deep networks. Colab lesson.

Reading:

• Li et al. A Survey on Large Language Model Acceleration based on KV Cache Management.

• Fichtl et al. The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures.

: 5th Assignment

Week #15: May 4–8

5/4

Survey of further topics and wrap-up.

• Reinforcement Learning (Colab lesson.)

• Geometric Deep Learning (Colab lesson.)

• Diffusion Models (Colab lesson.)