Lecture 1 March 21, 2026

Diffusion Policy

Visuo-motor policy learning via action diffusion. Trace a complete forward pass from camera image to robot joint commands — then build every component from scratch in Colab.

ResNet18 Spatial Softmax 1D U-Net FiLM DDPM
Open Lecture Slides

📓 Practical Hands-on Notebooks

End-to-end diffusion policy training on three environments — from simple 2D to real-world bimanual manipulation.

📗

Week 1: PushT

Train a diffusion policy from scratch on PushT — noise scheduling, denoising, and action prediction.

Open in Colab
📘

Week 2: ALOHA Bimanual

Scale diffusion policy to real-world ALOHA bimanual manipulation with camera observations and delta actions.

Open in Colab
📙

Week 3: LIBERO Single-Task

Train a diffusion policy on a single LIBERO manipulation task with image observations.

Open in Colab
📕

Week 3: LIBERO Multi-Task

Multi-task learning with LIBERO benchmark — one diffusion policy across diverse manipulation tasks.

Open in Colab

🔧 Component Deep-Dives

Build each component of the diffusion policy architecture from scratch — one focused notebook per module.

🎲

DDPM Basics

🧱

ResNet-18

📍

Spatial Softmax

⏱️

Timestep Embed

🏗️

1D U-Net

🎬

FiLM

🎛️

Conditioning

🔭

Receding Horizon


🎲 Interactive Visualizers

Explore the diffusion policy architecture through animated 3D and scroll-driven 2D walkthroughs.

Three.js · 3D
🫖

Diffusion Policy 3D

SO-101 arm performing teapot pour with DDIM denoising steps, noise visualization, and action chunks.

Launch 3D
Scroll-Driven · 2D
📊

Training Walkthrough

Step-by-step training walkthrough with real dataset frames, noise schedules, and loss curves.

Open
Scroll-Driven · 2D
🔄

Inference Pipeline

Interactive scroll-based visualizer showing the complete diffusion policy inference pipeline.

Open