Provide a short discussion of each of the assigned papers (listed under Course Materials). Below are some questions to think about.
MAML
Questions
Meta-learning learns a learning algorithm. In MAML, which aspects of the learning algorithm are being meta-learned? Which are hardcoded?
In Figure 3, MAML (green curve) improves very rapidly after just one step of gradient descent, but for additional steps the improvements are no better than the pretrained baseline (blue curve). Explain this behavior. (Similar behavior can be observed in Figs 4 and 5.)
Modular Meta-Learning
Questions
Which aspects of the learning algorithm are being meta-learned? Which are hardcoded?
What's an advantage of meta-learned modules over the approach in MAML? What's a disadvantage?
General questions:
Describe a case where meta-learning will be detrimental (i.e. perform worse than learning from scratch).
Meta-learning is learning to learn. This idea can be applied recursively: we can learn to learn to learn. Can you think of an example (in humans or machines) of learning to learn to learn? Where might this be useful?
Upload a single PDF file through Stellar by April 2 at 10 am.