Assignment #5

Provide a short discussion of each of the assigned papers (listed under Course Materials). Below are some questions to think about (you don't have to answer all of them):

Dense Object Nets

• This paper learns descriptors that map corresponding points to the same feature vector, and non-corresponding points to different feature vectors. They identify corresponding points (the "data association problem") via the training procedure in Secton 3.1, where they take multiple photos of an object on a table and get correspondence between the photos from an explicit 3D reconstruction. This procedure is quite expensive, as it requires a calibrated lab setup and robot arm. What might be some cheaper ways of solving the data association problem? How might an agent solve this problem in the wild?
• Figure 2 shows false color images where similar colors represent similar descriptors. What are some desirable properties of the descriptors that are apparent in these visualizations. Do you notice any undesirable properties?
• Section 5.3 discusses class-specific versus instance-specific descriptors. What are the relative advantages and disadvantages of these two kinds of descriptors?
• What _is_ the right object representation for robotic perception? Does it differ if the task is, e.g., navigation rather than manipulation?

Reasoning About Physical Interactions with Object-Oriented Prediction and Planning

• This paper argues for a factored, "object-oriented" representation of visual state. How does this factorization make prediction and planning easier? Think about the way goals are defined and the relationship to ideas in the papers we read on planning (e.g., "Planning As Heuristic Search"). What might be some other useful factored representations of visual scenes?
• The paper argues against direct supervision of state (object attributes). Instead object representations emerge somewhat indirectly, in service of a visual prediction task. What are some advantages and disadvantages of this indirect approach over the alternatives presented in Figure 2.
• Algorithm 1 presents a simple planning algorithm. Can you think of a goal tower configuration for which this planner would fail? How could the planner or representation be improved to handle your example?

Upload a single PDF file through Stellar by Feb 25 at 10 am.