First chess, then Jeopardy, then Go. Now poker too has fallen to AI

If you were about to start playing a game of online poker, you might want to think again. Humankind has just been beaten at yet another game, this time, Heads-Up No-Limit Texas Hold'em poker. This is a milestone moment for artificial intelligence (AI).

The first game that humans lost to machines was backgammon. In 1979, the world backgammon champion was beaten by Hans Berliner's BKG 9.8 program.

In 1997, Gary Kasparov, who was the reigning world chess champion, lost to IBM's Deep Blue program. Kasparov remarked that he could "smell" a new form of intelligence across the table from him.

Other games have since fallen to the machines: Checkers, Othello, Scrabble, the general knowledge quiz Jeopardy, even the classic arcade game Pong.

Most recently, the ancient Chinese board game of Go fell to the machines. Last March, one of the world's leading Go players, Lee Sedol, was beaten 4-1 by Google's AlphaGo program.

And to rub our faces in it, over the Christmas break, AlphaGo anonymously played against dozens of the world's leading Go players online and won convincingly.

WHY POKER?

Go has been described as the Mount Everest of board games. It is far more complex than chess or many other games. However, it is less of a challenge than poker.

Compared with other games, poker is a big challenge for AI. It is a game of uncertainty as players don't know other players' cards. It also requires an understanding of the psychology of other players.
Compared with other games, poker is a big challenge for AI. It is a game of uncertainty as players don't know other players' cards. It also requires an understanding of the psychology of other players. PHOTO: AGENCE FRANCE-PRESSE

Like the real world, poker is a game of uncertainty. Players don't know what cards the other players have, or what cards will be dealt in the future. In a game like chess or Go, by comparison, all the players can see the board. Everyone has complete information. This makes chess and Go much easier to program than poker.

To keep ahead of the bots, humans will need to play to their strengths, such as creativity and emotional intelligence. We should also look to augment rather than replace humans.

Poker also requires an understanding of the psychology of the other players. Are they bluffing? Should you fold? Should you bluff?

Finally, poker involves betting. When should you bet? What should you bet? This again adds to the challenge of writing a poker program that plays as well as or better than humans.

Over the past three weeks, four of the world's top poker players were locked in an exhausting 120,000 game match at the Rivers Casino in Pittsburgh.

Their opponent was Carnegie Mellon University's Libratus program, written by my colleague, Professor Tuomas Sandholm, and his PhD student Noam Brown.

Libratus won the tournament on Monday, with more than US$1 million (S$1.4 million) in notional winnings. The pros could be consoled by sharing the actual US$200,000 prize pot.

In order to reduce the influence of sheer luck on the result, the tournament used duplicate hands. This means that two decks of identically shuffled cards are used at two separate tables. On one table, a human player is dealt a hand, call it Hand A, and the AI player is dealt Hand B. On the other table (situated in another room), the AI player is dealt Hand A and the human player is dealt Hand B.

This means that even if one player receives an unusual number of lucky hands, this will be mirrored for the other player in the duplicate game.

This also explains why so many games were played. The end result is that we can say with statistical confidence that Libratus is better than the human players.

HOW TO WIN AT POKER

The details of how Libratus plays are still secret. But we can make some educated guesses based on the Carnegie Mellon University team's previous work.

Perhaps most interesting is that the victory depends more on Good Old Fashioned AI (Gofai) than on the currently fashionable deep learning processes.

Like IBM's Deep Blue in chess, Libratus used a lot of brute force calculation on how to play best. We know it called upon Pittsburgh's Supercomputing Centre to play out every end game.

And each night, Libratus used this supercomputer to refine its strategy. In case you think this is unfair to the humans, the pros also got together at night after each match to compare their performance and plan for the next day.

Libratus also takes advantage of game theory, the branch of mathematics made famous by the movie A Beautiful Mind, about mathematician John Nash. Libratus looks to play strategic moves that cannot be bettered, whatever its opponent does.

WHAT NEXT?

Poker is still not solved. Libratus plays only the two-player version of Heads-Up No-Limit Texas Hold'em poker. Adding more players greatly increases the complexity. So it will be a few years yet before computers can well play against four or more players.

But this is another example of how, in narrow-focused domains, AI is starting to take over from humans: reading mammograms, transcribing Chinese, beating human pilots in dogfights... the list increases almost weekly.

Not surprisingly, many people are wondering where this all ends. Will computers eventually take over all the jobs?

A widely reported study by the University of Oxford in 2013 estimated that 47 per cent of jobs in the United States were at risk from automation in the next two decades.

There were several limitations in the Oxford study. Ironically, one limitation was that the study automated the task of predicting which jobs were under risk. The study used machine learning and a small training set of 70 hand- labelled jobs to predict which of over 700 professions were at risk.

It is clear that some jobs, such as taxi driver, truck driver, radiographer and now poker pro, are under threat. Of course, technology will also create other new jobs. But whether as many get created or destroyed remains an interesting open question.

To keep ahead of the bots, humans will need to play to their strengths, such as creativity and emotional intelligence. We should also look to augment rather than replace humans.

Together, humans and machines can outperform machines or humans alone. The best chess player today is a human working with a computer. Together, we can be superhuman.

• The writer is professor of AI at the University of New South Wales, and research group leader at Data61, Australia's digital and data innovation group.

• This article first appeared in theconversation.com, a website of analysis from academics and researchers.

A version of this article appeared in the print edition of The Straits Times on February 01, 2017, with the headline 'First chess, then Jeopardy, then Go. Now poker too has fallen to AI'. Print Edition | Subscribe