Under construction. This area is a framework being built: most courses have no teaching material yet, and things may change or break. Private self-study. Not an official programme and not recognised by any ministry or school. No qualification, mark or document is issued, and “years 1 to 3” and “advanced” are names of study paths only.
E36 Optimization and generalization in deep learning
Advanced pathE: Proposed extensionsplannedabout 120 h (planning estimate)
Planned: syllabus only, no teaching material yet. Nothing on this page can be studied; it describes what the course would cover.
Named in: Advanced applied mathematics pathway
Why this course
Explains why training deep networks works in practice despite non-convexity.
Prerequisites
- Required
- A02 Convex & numerical optimization, A09 Mathematics of statistical learning, A18 Neural-network mathematics
- Recommended
- none
- Can be taken at the same time
- none
- Everything required, transitively (24 courses)
- A02, A09, A18, B01, B02, B03, B04, B05, B06, B08, B09, B11, B12, B13, B15, B17, B19, B20, F01, F02, F03, G01, G03, U09
- Courses that require this one
- none
Concept-level units
Units follow the syllabus order, as a recommended sequence rather than a gate.
- Stochastic gradient descent and its noise
- Momentum, Adam and adaptive methods
- Loss landscapes, saddle points, flat minima
- Overparameterization, double descent and benign overfitting
- Implicit bias of gradient descent
- Normalization, initialization and residual connections
- Neural tangent kernel and the lazy regime
- Scaling laws
Assessable outcome
Explain, with a theorem or a worked example each, why a given training behaviour occurs.
Assessment design
- topic exercises
- mixed problem sets
- unseen proof or derivation problems
- cumulative examination
cumulative; uses spaced retrieval
Resources
Every title here is candidate, unverified unless marked verified. Titles were suggested while planning. Nobody has checked that they exist, what they cost or how they are licensed.
- Goodfellow, Bengio, Courville · Deep Learning (free)
- Belkin et al · Reconciling modern machine-learning practice and the bias-variance trade-off (paper)
- Zhang et al · Understanding deep learning requires rethinking generalization (paper)
Content record
- Version
- 0.0.0
- Review
- no review of any kind
- Provenance
- PROPOSED by the assistant on 4 Nov 2026 so that a person with a mathematics degree can read AI articles fluently; not in the owner's brief; APPROVED by the owner on 4 Nov 2026 as an elective; still needs a curriculum reviewer
- Known issues
- Approved as an elective on 4 Nov 2026; syllabus only, no lessons, candidate resources unverified, needs a curriculum reviewer.
- Type
- course
Bridges and laboratories
No bridge or laboratory is attached to this course yet.
You and this course
Reading your plan on this device…