OpenAI Five plays 180 years worth of games against itself every day, learning via self-play. It trains using a scaled-up version of Proximal Policy Optimization⁠ running on 256 GPUs and 128,000 CPU cores—a larger-scale version of the system we built to play the much-simpler solo variant⁠ of the game last year. Using a separate LSTM⁠(opens in a new window) for each hero and no human data, it learns recognizable strategies. This indicates that reinforcement learning⁠(opens in a new window) can yield long-term planning with large but achievable scale—without fundamental advances, contrary to our own expectations upon starting the project.

To benchmark our progress, we’ll host a match versus top players on August 5th. Follow⁠(opens in a new window) us on Twitch to view the live broadcast, or request⁠(opens in a new window) an invite to attend in person!

One AI milestone is to exceed human capabilities in a complex video game like StarCraft⁠(opens in a new window) or Dota. Relative to previous AI milestones like Chess⁠(opens in a new window) or Go⁠(opens in a new window), complex video games start to capture the messiness and continuous nature of the real world. The hope is that systems which solve complex video games will be highly general, with applications outside of games.

Dota 2 is a real-time strategy game played between two teams of five players, with each player controlling a character called a “hero”. A Dota-playing AI must master the following: