Assignment #9

Provide a short discussion of each of the assigned papers (listed under Course Materials). Below are some questions to think about.

Novelty Search
This paper is a bit long, but not hard. It's okay to read sections 5-8 quickly.
This recent paper evaluates the approach on modern RL benchmarks; it has a bit more technical detail.
Questions

• What are some situations where novelty search will struggle?
• Where might a novelty bonus lead to worse results than simply optimizing fitness?
• Draw a maze where novelty search will perform worse than a "distance to goal" fitness function.

Both papers
Questions

• We have seen many ways to incentivize exploration. Compare and constrast:
  • Novelty search,
  • Curiosity,
  • UCB,
  • IW,
  • Information seeking in POMDPs

Upload a single PDF file through Stellar by Mar 10 at 10 am.