The OpenAI Retro Contest⁠(opens in a new window) gives you a training set of levels from the Sonic The Hedgehog™ series of games, and we evaluate your algorithm on a test set of custom levels that we have created for this contest. The contest will run from April 5 to June 5. To get people started we’re releasing retro-baselines⁠(opens in a new window), which shows how to run several RL algorithms on the contest tasks.

Baseline results on the Retro Contest (test set) show that RL algorithms fall far below human performance, even when using transfer learning. Human performance is shown as a dashed horizontal line. The humans only played for one hour, versus eighteen for the algorithms.

You can use any environments or datasets you want at training time, but at test time you only get about 18 hours (1 million timesteps) on each never-before-seen level. 18 hours may sound like a long time to play a single game level, but existing RL algorithms perform far worse than humans given this training budget.

To describe the benchmark in detail, as well as provide some baseline results, we are releasing a technical report: Gotta Learn Fast: A New Benchmark for Generalization in RL⁠(opens in a new window). This report contains details about the benchmark as well as results from running Rainbow DQN⁠(opens in a new window), PPO⁠, and a simple random guessing algorithm called JERK. JERK samples random action sequences in a way that is optimized for Sonic, and as training progresses it replays the top-scoring sequence of actions more frequently.