������ý

An AI just claimed another gaming victory over humans by winning a 20-day poker tournament. The AI, called Libratus, took on four of the world’s best Heads-Up No-Limit Texas Hold ‘Em poker players at a Pennsylvania casino. After 120,000 hands, Libratus won with a lead of over $1.7 million in chips.

“I’m feeling great,” says , a computer scientist at Carnegie Mellon University who was part of the team that created the AI. “This is a David versus Goliath story, and Libratus was able to throw a pebble.”

A poker-proficient AI is remarkable because poker is a game of “imperfect information”: players don’t know what cards their opponents have, so never have a full view of the state of play. This means the AI has to take into account how its opponent is playing and rework its approach so it doesn’t give away when it has a good hand or is bluffing.

“It’s a really important milestone for artificial intelligence,” says at the University of Malta. “This is like reality. The real world is a game of imperfect information, so by solving poker we become one step closer to general artificial intelligence.”

Libratus’s algorithms are not specific to poker, or even just to games. The AI has not been taught any strategies and instead has to work out its own way to play based on the information it’s given – in this case, the rules of poker. This means that Libratus could be applied to any situation that requires a response based on imperfect information.

“There are applications in cybersecurity, negotiations, military settings, auctions and more,” says Sandholm. His lab has also been looking at how AI can bolster the fight against infections, by viewing treatment plans as game strategies. “You can learn to battle diseases better even if you have no extra medicines at your disposal – you just use them smarter,” says Sandholm.

Spilling the beans

The Carnegie Mellon team has previously been tight-lipped about Libratus’s methods, fearing that any explanation could assist its human competitors. But now Sandholm is willing to say more about how it works.

Libratus has three main parts. The first has not changed much since 2015 when Sandholm’s team first entered its AI in a similar tournament against professional players (that time, humans won). This part computed a big list of strategies the AI could use when play began. At the outset of the tournament, Libratus had spent the equivalent of 15 million hours of computation honing its strategies.

The second part, now completely redesigned by Sandholm and his PhD student Noam Brown, worked to improve Libratus’s strategy with each hand. Called the “endgame solver”, it took into account “mistakes” the AI’s opponents made – instances where they left themselves open to exploitation – to predict the result of each hand. The team couldn’t tell from statistical analysis if the earlier version of the endgame solver improved the AI’s play at all, says Sandholm. “But this new one is just awesome.”

The final part of the AI looked for its own strategic weaknesses so it could change how it played before the next session. This sought to identify things its opponents were exploiting, such as a giveaway “tell” that another player had noticed.

This was important as, in the last tournament, the human players were able to work out how the AI played when it had different cards and change the way they bet accordingly.

Tougher to play every day

“It’s insanely good this time around – quite remarkable,” said Jason Les, one of the professional players, as the tournament entered the final days. “It seems to have some sort of strategy update component that is learning how to best play us. Its strategy seems to be improving as time goes on and it is tougher and tougher every day.”

Despite their loss, the professional players will split a $200,000 prize pot based on their performances – and the researchers won’t actually take home any winnings. Following its victory, the Libratus team plans to publish the AI’s algorithms in a peer-reviewed journal.

There is still a long way to go before AI can take on the real world, says at the University of Essex, UK. “In the real world, you often have a lot more choices than in a card game. The possibilities are more open-ended,” he says.

But it’s still a fantastic achievement as poker is a complex game, he says. “It’s an impressive step forward and a big deal.”