How to start learning mechanistic interpretability
Main courses
ARENA (Alignment Research Engineer Accelerator)
arena.education. A technical training program (originally run in person in London) whose full curriculum is published openly. It includes chapters on transformers from scratch, introduction to mechanistic interpretability, superposition and sparse autoencoders, circuits such as indirect object identification, and more, with coding exercises and solutions. Created by Callum McDougall and collaborators. It is one of the best structured free routes into the field. Check the website for current programs and materials.
Who it is for: people who know basic Python and PyTorch and want to code it themselves.
Neel Nanda — starting materials
neelnanda.io. Practical guides on how to get started in mechanistic interpretability, a long glossary, video walkthroughs of papers and coding, and lists of open problems. Free.
Karpathy — "Neural Networks: Zero to Hero"
karpathy.ai/zero-to-hero, including Let's build GPT. Builds from backpropagation to a GPT. Ideal prerequisite.
3Blue1Brown — Neural Networks
3blue1brown.com/topics/neural-networks. Best visual intuition for beginners.
MATS (ML Alignment & Theory Scholars)
matsprogram.org. A selective research program that matches participants with mentors, including mechanistic interpretability researchers. For people ready to do real research. Check current applications.
BlueDot Impact — AI Safety courses
bluedot.org. Introductory courses on AI safety, useful context for why interpretability matters. Not specific to mech interp.
Step by step (with time as a guide)
| Step | What to do | Time, as a guide |
|---|---|---|
| 0. Prerequisites | Python, basic PyTorch, linear algebra (vectors, matrices, dot products). On MEJ7: Beginner. | 2–6 weeks |
| 1. AI basics | MEJ7 lab (tokens, embeddings, attention), 3Blue1Brown | 1–2 weeks |
| 2. A transformer from scratch | Karpathy "Let's build GPT" and/or ARENA "Transformers from scratch" | 2–3 weeks |
| 3. First tools and experiments | TransformerLens, logit lens, following an "Intro to Mech Interp" tutorial | 1–2 weeks |
| 4. Core papers | "A Mathematical Framework", Induction Heads, Toy Models of Superposition, IOI | 3–6 weeks |
| 5. Repeat a result | Reproduce induction heads or IOI or grokking yourself | 2–4 weeks |
| 6. Your own project | Take a problem from "200 Concrete Open Problems", write up the results | 1–3 months |
These are rough estimates, not promises. The pace depends on your experience.
Author: mej7.com, an independent educational project. Last reviewed: 6 November 2026.