Assignment #9

Provide a short discussion of each of the assigned papers (listed under Course Materials). Below are questions to think about (you should discuss at least some of these but you don't have to address them all):

Novelty Search
This paper is a bit long, but not hard. It's okay to read sections 5-8 quickly. This more recent paper evaluates the approach on modern RL benchmarks; it has a bit more technical detail.
Questions

• What are some situations where novelty search will struggle?
• Where might a novelty bonus lead to worse results than simply optimizing fitness?
• Draw a maze where novelty search will perform worse than a "distance to goal" fitness function.

All three papers
Questions

• We have seen many ways to incentivize exploration. Compare and contrast:
  • Novelty search
  • Curiosity
  • Open-Endedness using LLMs
  • UCB/UCT
  • IW
  • Information-seeking in POMDPs

Upload a single PDF file through Canvas by March 12th at 1 pm.