MIT CSAIL6.s953 Embodied Intelligence |
||
Spring 2024 |
||
This schedule is preliminary and subject to change as the term evolves.
| Date | Topics | Course Materials | Assignments | ||
| Week 1: Cognitive architectures and paradigms | |||||
| Tues 2/6/2024 | Domains, agent architecture, factory/wild |
Intro lecture slides Russell & Norvig slides |
|||
| Thurs 2/8/2024 | Classic papers |
Readings slides Assigned readings: 1. Shakey (video) (optional: Technical Report) 2. Brooks, Flynn: Fast, Cheap, and Out of Control 3. Sutton: Dyna Optional: Sims: Evolved Virtual Creatures (video) |
Assignment #1 | ||
| Week 2: The problem of long-term decision making | |||||
| Tues 2/13/2024 | Deterministic domains: pddl planning, IW(k), power of heuristics, factoring |
Assigned readings: 1. Bonet & Geffner: HSP (through section 5) 2. Lipovetsky & Geffner: IW(k) 3. Bandres, Bonet & Geffner: Planning with Pixels |
Assignment #2 | ||
| Thurs 2/15/2024 | Future-state uncertainty: value iteration, MCTS, MPC / replanning |
Assigned readings: 1. Kocsis & Szepesvari: Bandit-based Monte-Carlo Planning (skim section 2.4) 2. Yoon, Fern & Givan: FF-Replan: A Baseline for Probabilistic Planning 3. Riedmiller: Neural Fitted Q Iteration |
Assignment #3 | ||
| Week 3: The problem of continuous action spaces | |||||
| Thurs 2/22/2024 | Motion planning, optimization, control in continuous action space |
Assigned readings: 1. Kuffner & LaValle: RRT-Connect (section 4 optional) (animations) 2. Toussaint: Logic-Geometric Programming 3. Diffusion Policy |
Assignment #4 | ||
| Week 4: The problem of situation awareness | |||||
| Tues 2/27/2024 | Perception as state estimation, object-oriented perception |
Assigned readings: 1. Dense Object Nets 2. Object-Oriented Prediction and Planning 3. F3RM |
Assignment #5 | ||
| Thurs 2/29/2024 | Sensory representation learning, state representation learning |
Assigned readings: 1. Contrastive Predictive Coding (optional: Self-Organizing Neural Network that Discovers Surface in Random Dot Spectrograms) 2. Learning image representations tied to ego-motion 3. Quasimetric Reinforcement Learning |
Assignment #6 | ||
| Week 5: The problem of partial observability | |||||
| Tues 3/5/2024 | POMDPs: search and policies |
Assigned readings: 1. POMDP Tutorial (you don't need to read the last 4 sections) 2. POMCP 3. Belief-space Planning |
Assignment #7 | ||
| Thurs 3/7/2024 | POMDPs: Learning |
Assigned readings: 1. RL in Robotics: A Survey (just review sections 2.2.2 and 2.3) 2. Solving Deep Memory POMDPs 3. Bridging State and History Representations: Understanding Self-Predictive RL |
Assignment #8 | ||
| Week 6: The problem of learning for long horizons | |||||
| Tues 3/12/2024 | Exploration, novelty search / curiosity, intrinsic motivation, open-endedness |
Assigned readings: 1. Novelty Search 2. Curiosity-Driven Exploration 3. Open-endedness using LLMs |
Assignment #9 | ||
| Thurs 3/14/2024 | HRL |
Assigned readings: 1. Feudal RL 2. Diversity is All You Need 3. Code as Policies |
Assignment #10 | ||
| Week 7: The problem of very little data | |||||
| Tues 3/19/2024 | Model-based intelligence | Assigned readings: 1. World Models 2. Building Machines That Learn and Think Like People (read Section 4, skim rest) 3. UniSim |
Assignment #11 | ||
| Thurs 3/21/2024 | Meta-learning | Assigned readings: 1. MAML 2. Modular Meta-Learning 3. RL^2 |
Assignment #12 | ||
| Week 8: The problem of nonstationarity | |||||
| Tues 4/2/2024 | Methodology | Assigned readings: 1. Deep RL that Matters 2. Assessing Generalization in Deep RL 3. Machine Learning that Matters |
Assignment #13 Final project proposal due |
||
| Thurs 4/4/2021 | Cooperative Multiagent RL | Assigned readings: 1. QMIX 2. Learning to Communicate with Deep MARL 3. CICERO Optional: Comprehensive survey of MARL |
Assignment #14 | ||
| Week 9: The problem of adversaries | |||||
| Tues 4/9/2024 | Adversaries/Self-play | Assigned readings: 1. Markov games 2. TD-Gammon 3. AlphaZero |
Assignment #15 | ||
| Thurs 4/11/2024 | Artificial life | Assigned readings: 1. Karl Sims (evolving by competition) (video) 2. Hide and Seek (video) 3. Lenia — Biology of Artificial Life (video) |
Assignment #16 | ||
| Week 10: Concrete problems | |||||
| Tues 4/16/2024 | A general-purpose robot hand | Assigned Reading: 1. Learning Dexterous In-Hand Manipulation (Movie) Optional: The follow-up paper on solving a Rubick's Cube |
|||
| Thurs 4/18/2024 | A general-purpose robot | Assigned Reading: 1. SayCan |
|||
| Week 11: Project milestone presentations | |||||
| Tues 4/23/2019 | Project milestone presentations | ||||
| Thurs 4/25/2019 | Project milestone presentations | ||||
| Week 12: Concrete Problems II | |||||
| Tues 4/30/2024 | Is scaling enough? What is the best path to artificial general intelligence? | Assigned Reading: 1. A Path Towards Autonomous Machine Intelligence |
|||
| Thurs 5/2/2024 | No class, work on project | ||||
| Week 13: Final project presentations | |||||
| Tues 5/7/2024 | No class, work on project | ||||
| Tues 5/9/2024 | Final project presentations | ||||
| Week 14: Final project presentations | |||||
| Tues 5/14/2024 | Final project presentations | Final project due | |||