Assignment #7

Background for both papers

• We'll be discussing planning methods for Partially Observable Markov Decision Processes (POMDPs). Next class we'll look at learning methods.

Provide a short discussion of each of the assigned papers (listed under Course Materials). Below are some questions to think about.

POMDP Tutorial
You don't need to read the last 4 sections. Or, if you prefer equations, you can read this paper, skipping the details of the Witness algorithm.
Questions

• Which problems that we have seen in class can be modeled as a POMDP? What is a problem that cannot be modeled as an MDP or a POMDP (and why)?
• Why can a POMDP be seen as an MDP in belief space?
• Wnat can we say about the shape of the value function of a POMDP? Intuitively, why are the highest values along the outside edges?

Belief-space Planning
Skip the analysis in last part of section IV.
Questions

• In what way is the planned mean trajectory better than the one found by B-LQR (Fig 1)?
• Why does the executed trajectory differ from the planned one (Fig 2)?
• Would it be possible to use an RRT, rather than trajectory optimization, within this method?

Upload a single PDF file through Stellar by Mar 3 at 10 am.