Assignment #13

Provide a short discussion of each of the assigned papers (listed under Course Materials). Below are some questions to think about.

Comparing Classifiers

Questions

• Would meta-learning fit into Dietterich's taxonomy? If so, where?
• Give an example deep learning paper that is trying to "sell" a classifier. Given an example that is trying to "sell" a learning algorithm.
• Pick the modern supervised learning paper that you are most familiar with and discuss how they handle the four sources of variance that affect the results of a single learning experiment.

ML that matters

Questions

• What is a concrete example practice in current deep learning literature that Wagstaff would not approve of?

Deep RL that matters

Questions

• Compare the sources of variance that appear in an RL experiment to Dietterich's four. Does RL have more? Fewer?
• What might "random seed" govern in an RL experiment in a simulator?
• When (if ever) is it reasonable methodological practice to maximize over random seeds?
• When a plot shows a dark (mean) line and a lighter band around it, what is the meaning of that lighter band supposed to be?

Upload a single PDF file through Stellar by Apr 2 at 10 am.