Notebook
Blog
Mathematics, robots, models and whatever else I end up poking at.
-
Sesame: building a quadruped, breaking it, and bringing it to life
Rebuilding Dorian Todd's open-source Sesame robot on my own electronics: a month of sourcing, a loose wire, a servo that stopped at 160 degrees, and a robot nicknamed Cockroach.
- Under the Hood: CUDA, MatMul, and the Engine of AI Moving closer to the hardware: why matrix multiplication is the core computation of modern AI, and how CUDA runs it on a GPU.
- Lecture 20: RL Debugging, Diagnostics and Policy Search Stanford CS229 lecture 20 notes: RL Debugging, Diagnostics and Policy Search.
- Lecture 19: Reward Models and Linear Dynamical Systems Stanford CS229 lecture 19 notes: Reward Models and Linear Dynamical Systems.
- Lecture 18: Continuous State MDPs and Model Simulation Stanford CS229 lecture 18 notes: Continuous State MDPs and Model Simulation.
- Lecture 17: MDPs, Value Iteration and Policy Iteration Stanford CS229 lecture 17 notes: MDPs, Value Iteration and Policy Iteration.
- Lecture 16: ICA Wrap-up and Introduction to Reinforcement Learning Stanford CS229 lecture 16 notes: ICA Wrap-up and Introduction to Reinforcement Learning.
- Lecture 15: PCA and ICA Stanford CS229 lecture 15 notes: PCA and ICA.
- Lecture 14: Expectation-Maximization and Factor Analysis Stanford CS229 lecture 14 notes: Expectation-Maximization and Factor Analysis.
- Lecture 13: Debugging ML Models and Error Analysis Stanford CS229 lecture 13 notes: Debugging ML Models and Error Analysis.
- Lecture 12: Backpropagation and Improving Neural Networks Stanford CS229 lecture 12 notes: Backpropagation and Improving Neural Networks.
- Lecture 11: Introduction to Neural Networks Stanford CS229 lecture 11 notes: Introduction to Neural Networks.
- Lecture 10: Decision Trees and Ensemble Methods Stanford CS229 lecture 10 notes: Decision Trees and Ensemble Methods.
- Lecture 9: Approximation/Estimation Error and ERM Stanford CS229 lecture 9 notes: Approximation/Estimation Error and ERM.
- Lecture 8: Data Splits, Models and Cross-Validation Stanford CS229 lecture 8 notes: Data Splits, Models and Cross-Validation.
- Lecture 7: Kernels Stanford CS229 lecture 7 notes: Kernels.
- Lecture 6: Support Vector Machines Stanford CS229 lecture 6 notes: Support Vector Machines.
- Lecture 5: GDA and Naive Bayes Stanford CS229 lecture 5 notes: GDA and Naive Bayes.
- Lecture 4: Perceptron and Generalized Linear Models Stanford CS229 lecture 4 notes: Perceptron and Generalized Linear Models.
- Lecture 3: Locally Weighted Regression and Logistic Regression Stanford CS229 lecture 3 notes: Locally Weighted Regression and Logistic Regression.
- Lecture 2: Linear Regression and Gradient Descent Stanford CS229 lecture 2 notes: Linear Regression and Gradient Descent.
- Lecture 1: Introduction to Machine Learning Stanford CS229 lecture 1 notes: Introduction to Machine Learning.
- Lecture 13: Reasoning and Agents Stanford CS224N lecture 13 notes: Reasoning and Agents.
- Lecture 12: Efficient Adaptation and Training Stanford CS224N lecture 12 notes: Efficient Adaptation and Training.
- Lecture 11: Benchmarking and Evaluation Stanford CS224N lecture 11 notes: Benchmarking and Evaluation.
- Lecture 10: Post-training (Instruction Tuning and RLHF) Stanford CS224N lecture 10 notes: Post-training (Instruction Tuning and RLHF).
- Lecture 9: Pretraining Stanford CS224N lecture 9 notes: Pretraining.
- Lecture 8: Self-Attention and Transformers Stanford CS224N lecture 8 notes: Self-Attention and Transformers.
- Lecture 7: Attention Stanford CS224N lecture 7 notes: Attention.
- Lecture 6: LSTMs, GRUs, Seq2Seq and Machine Translation Stanford CS224N lecture 6 notes: LSTMs, GRUs, Seq2Seq and Machine Translation.
- Lecture 5 - Recurrent Neural Networks Stanford CS224N lecture 5 notes: recurrent neural networks and language models, and why very large networks still generalise.
- Lecture 4: Dependency Parsing Stanford CS224N lecture 4 notes: phrase structure and dependency parsing, and how words combine into phrases and sentences.
- Lecture 3: Backpropagation, Neural Network Stanford CS224N lecture 3 notes: neural nets and backpropagation. Short, since the full derivation is in the neural networks post.
- Lecture 2: Word vectors and Language Models Stanford CS224N lecture 2 notes: bag-of-words, how word vectors are learned, and language models.
- Neural Networks A four-part walkthrough of neural networks: matrix-calculus review, forward propagation, the derivatives behind a neuron, and the four equations of backpropagation derived by hand.
- Logistic Regression Logistic regression as linear regression adapted to separate clusters with a line, or a curve, and how that boundary is learned.
- Linear Regression Linear regression as the backbone of older ML models: decision boundaries of the form wx + b, and how the fit is found.
- Decision Trees Classifier Decision trees for classification, built one split at a time on a two-feature dataset, with a live Desmos graph for every step.
- Decision Tree Regressor How a decision tree does regression: splitting the feature space into regions and predicting a value per region, with interactive 3D Desmos graphs.
- Attention is all you need! Building a GPT-2 style decoder-only transformer from scratch on Tiny Shakespeare, step by step: data, tokenizing, batching, self-attention and training.
- Lecture 1: Introduction and Word Vectors Stanford CS224N lecture 1 notes: why human language is hard for machines, and word vectors as the starting point.
Nothing matches that. Try fewer words.