Artificial intelligence (AI) has reached a new level, with an AI poker program beating four of the top specialists in “heads-up, no-limit Texas hold’em” poker. “Libratus,” powered by PSC’s Bridges supercomputer, is a first step in AIs that can handle “imperfect information,” offering lessons in important problems such as security, negotiation and even medicine.
“We didn’t actually look at any data, just the rules of the game … It was like practicing by shadow boxing and then stepping into the ring with Mike Tyson.” —Tuomas Sandholm, Carnegie Mellon University
Why it’s important.
While computers have beaten human champions at a number of games—like checkers, chess or Go—these games offer perfect information. No information is hidden from the contestants and they have limited ability to deceive each other. Poker, on the other hand, is an imperfect-information game. Unlike in chess, in an imperfect-information game the opponent’s hand is secret, and he or she is trying to mislead. Imperfect-information games are like many real-world problems, including cybersecurity, terror defense, negotiation—even cancer treatment, because the tumor is actively evading both the patient’s immune system and any treatments. Tuomas Sandholm of Carnegie Mellon University and his PhD student Noam Brown have created a series of artificial intelligence programs (AIs) capable of optimizing how to play essentially any imperfect information game. The scientists wanted to find out whether the AI’s strategic reasoning had finally reached the point at which humans couldn’t beat it, even under imperfect information. To find out, they took on the world’s best specialists at heads-up, no-limit Texas hold’em poker—a benchmark game in which hidden information and deception are paramount. The AI would have to learn how to deceive—and how to win despite deception. In 2015, the team’s earlier AI, called Claudico, narrowly lost to top human players.
“We didn’t test [Libratus] ahead of time against professional poker players … They did everything I was afraid they’d do.” —Noam Brown, Carnegie Mellon University
How PSC Helped
No-limit Texas hold’em has 10161 possible situations—more than there are atoms in the known Universe, and far more than any computer can directly calculate. To master the game, the scientists’ new AI, called Libratus, had to find the optimal way to simplify the game to be computable and how to bluff in ways that tricked some of the most expert human players into holding, and folding, when they shouldn’t. For their January 2017 “Brains vs. AI” rematch with the pros, they turned to PSC’s Bridges system, using about 600 of Bridges’ compute nodes. This raw power gave Libratus the capacity to plot each move in real time—and reformulate its strategy each night, even as the four human experts—Dong Kim, Jason Les, Jimmy Chou and Daniel McAulay—were doing the same to try to expose any weaknesses in the AI.
In the end, Libratus scored a resounding victory, beating the pros by more than $1.7 million in chips at the Rivers Casino in Pittsburgh. The scientists calculate that an outcome this lopsided or larger was only 0.5 percent likely to happen just by chance. The win marked the first time that an AI had beaten the world’s best players at a game that had emerged over the years as the leading benchmark for solving imperfect-information games. This paves the way for AIs to help transform how humans approach real-life problems in security, negotiation, and medicine.
“The improvement they’ve been able to make over the past 18 months … has been quite extraordinary … [Libratus] is very beastly—super gangster … Whatever we throw at it, it just figures out … something that shuts us down … We were going after weaknesses, and we’d think, ‘Wow, this is it, we found something’—and as the time went on, those exploits just disappeared.” —Jason Les, Texas hold’em pro who played both Claudico and Libratus