Assignment #9

Provide a short discussion of each of the assigned papers (listed under Course Materials). Below are some questions to think about.

Novelty Search
This paper is a bit long, but not hard. Don't try to understand in detail the discussion of related work and skim the experiments (Sections 5-8).
This recent paper evaluates the approach on modern RL benchmarks; it has a bit more technical detail.
Questions

• What are some situations where novelty search will struggle?
• Where might a novelty bonus lead to worse results than simply optimizing fitness?
• Draw a maze where novelty search will perform worse than a "distance to goal" fitness function.

All the papers
Questions

• We have seen many ways to incentivize exploration. Compare and constrast:
  • Novelty search,
  • #Exploration,
  • Curiosity,
  • UCB,
  • IW

Upload a single PDF file through Stellar by Mar 12 at 10 am.