TrueSkill⁠(opens in a new window) rating (similar to the ELO rating in chess) of our best bot over time, computed by simulating games between the bots and observing the win ratios. Improvements came from every part of the system, from adding new features to algorithmic improvements to scaling things up. The graph is surprisingly linear, meaning the team improved the bot exponentially over time.

The project’s timeline is the following. For some perspective, 15% of players are below 1.5k MMR⁠(opens in a new window); 58% of players are below 3k; 99.99% are below 7.5k.

The full game is 5v5, but 1v1 also appears⁠(opens in a new window) in some tournaments⁠(opens in a new window). Our bot played under standard tournament rules—we did not add AI-specific simplifications to 1v1.

We whitelisted a few dozen item builds that bots could use, and picked one for evaluation. We also separately trained the initial creep block using traditional RL techniques, as it happens before the opponent appears.

Our approach, combining small amounts of “coaching” with self-play, allowed us to massively improve our agent between the Monday and Thursday of The International. On Monday evening, Pajkatt won using an unusual item build (buying an early magic wand). We added this item build to the training whitelist.