Assignment #11

Provide a short discussion of each of the assigned papers (listed under Course Materials). Below are some questions to think about.

Building Machines That Learn and Think Like People

Read Section 4, feel free to skim the rest.

Questions

• This paper argues that human learning is often more sample efficient (i.e. requires less data) than 2016-era machine learning. Did you find this argument compelling? A possible complaint is that the comparison is not apples to apples: humans adults have extensive prior experience, while most machine algorithms do not. How might we make the comparison more fair? The footnote on page 8 provides one attempt. Is it convincing? Given equivalent priors, do you think the visual concept learning algorithm in our brain is more sample efficient than SGD? (note: I don't think anyone knows for sure, this is an open question)
• Human babies seem to come with some knowledge baked in (see Section 4.1 in particular). If we, as engineers, want to build a generalist robot, what knowledge should we bake in? What should we let emerge?

Deep Visual Foresight

Questions

• Why was it useful to factor the predictions as future frame = flow * current frame?
• How could the method be extended to handle occlusion (the problem of occlusion is discussed on page 7)?
• Suppose the goal is to pick up an object, rather than to push it. What would be some difficulties the current method would encounter? Could the method be used to move a designated 3D point to a 3D goal position? Remember that the video frames are 2D arrays of pixels.

World Models

Questions

For the following three questions, please see the interative demos here:

• Try playing with the z sliders. What are these sliders doing?
• Try changing the Tau parameter (second set of demos). What happens and why? The paper says that high Tau makes the model harder to exploit. Why is that?
• In the VizDoom interactive demo, you can play the game entirely inside the network's hallucination. Can you beat the policy reported in Table 2 of the paper? What's your highest score?

Additional questions:

• Compare and contrast this method with Dyna.
• Do you think this is what our brains are doing when we dream? Notice that the model needed thousands of mental rollouts to play a really simple version of Doom. I feel like I only have a couple dreams per night. Are our brains really that much more sample efficient at learning from mental simulation? Or is dreaming not related to learning? Also, are our dreams so strange because Tau is ramped up?

Upload a single PDF file through Stellar by Mar 19 at 10 am.