Poker Bots: The Beginning of the End? Um, No

Phil Laak

Most serious poker players, especially those who play online, are aware of the existence of "bots."

Bot is short for robot, and the subspecies of interest here are those designed to play poker.

A bot is, properly, an artificial intelligence (AI) - a sophisticated piece of software that is programmed not only to make optimal decisions but also to learn from its experiences.

There are many phony bots on the market, pieces of programming junk that you can buy or lease. None plays poker better than you (at least I hope not).

However, there is one poker bot that has achieved considerable fame. It's a genuine AI, dubbed Polaris, and was developed by the members of the Computer Poker Research Group (CPRG) at the University of Alberta.

Polaris has won several contests against other poker bots and made headlines recently when it outplayed a group of online pros.

You can visit the CPRG site and follow any of the links, including one that will let you play heads-up against Poki, a "baby bot" whose game is good enough to be used in a poker training program. Other links will take you to tech reports and scientific publications.

Bill Chen
Geeks love sophisticated software.

Bots in general, poker bots in particular and the very notion of AI are topics of endless fascination. Computer geeks love the sophisticated software. Mathematicians revel in the formal properties of the systems that underlie them.

Applied scientists envision extensions into "partial information" domains like bidding auctions, commodities trading and currency exchanges.

Poker players, of course, view them from a host of perspectives - from envy to fear and loathing laced with heavy doses of paranoia.

The success of Polaris also seems to have fired the imagination of the media. Some called it the beginning of the end of poker. Others likened Polaris to Deep Blue, the chess AI that beat Gary Kasparov.

Still others warned ominously about mad scientists with clandestine bots lurking on the Internet, running roughshod over mere mortals - tidbits that inflame the onliners with paranoid tendencies. This is provocative stuff and we need to understand what's really going on.

So let's break it down and see just what the parameters of play are when a human being goes up against Polaris.

Phil Laak
Laak: If you look closely, you can see he's also playing Minesweeper and posting on Antonio Esfandiari's Facebook wall.

The game is:

Limit Hold'em (LH): Polaris won't sit down (metaphorically speaking) in a game of Stud or Omaha. It only plays this one game. On the occasions where it was programmed for No-Limit Hold'em, skilled human opponents consistently beat it.

Limit Hold'em, of course, is more algorithmic than No-Limit Hold'em. This is not to say that LH is not a complex game that demands a high level of skill; it's merely an acknowledgement that it is easier to develop effective strategic generalizations in Limit than No-Limit.

It is also says nothing about the possibility of future bots playing world-class NLH, although this is a task of another order of magnitude. No one knows what the optimal strategy is for NLH, and one may not exist.

Heads-up: Polaris only plays against a single opponent. Heads-up play has a reduced number of variables compared with a game with multiple opponents.

The computational burden on a bot that plays against more than one opponent is daunting and, worse, it isn't clear what the maximally effective strategies are. Again, this isn't an in-principle argument against developing such a bot, merely an acknowledgment of the difficulties.

Duplicate poker: The games were played in a version of poker modeled on duplicate bridge. The same cards are dealt to opponents at different times and each must play them from both sides.

For example, one time you will play A K against your opponent's T 9. Later, you will hold T 9 against your opponent's A K.

Ali Eslami
Ali Eslami: Hand-picked by Laak for first Polaris battle, but came up short.

Duplicate play lowers variance by reducing the impact of luck. It doesn't eliminate it, of course. For example, you may (correctly) fold a hand that someone calls with and a magical river presents your opponent with a pot you never got to see.

However, compared with random dealing, duplicate is known to reduce variance by about two-thirds. This increases statistical power so that only one-ninth as many hands are needed to yield significant results, which is why the CPRG used it.

Some see duplicate poker as the way of the future. I don't. Because it reduces the luck element, weaker players will have fewer winning sessions and lose too regularly. The balance between luck and skill in poker as currently played fits my Goldilocks Rule - it's "just right."

The opponents: The "pros" in the Pros vs. Polaris competition were a group of young, experienced online players. After 3,000 hands Polaris was up 195 small bets, a statistically significant result.

In an earlier contest, Polaris took on two prominent pros, Phil Laak and Ali Eslami. It beat Eslami but Laak won enough so they eked out a small combined win. Our species (assuming that Laak is one of us) hailed this as a victory.

The stakes: An important but oft-unnoted feature is that Polaris only plays for "cybercash," not real money. While there is little doubt that the pros are possessed of outsized egos (what top poker player is without one of these?), the fact that no actual harm could come to their bankrolls surely had an impact.

The online hotshots lost a combined total of 195,000 cyber dollars. Would their play have been different if they were confronting the possibility of losing that much "real" money? Almost certainly. Would they have played better? Perhaps. Worse? Perhaps.

Matt Hawrilenko
Matt "Hoss_TBF" Hawrilenko: Noted Limit specialist and one of the online hotshots who got bested by bot.

A more sober assessment: Given these factors, many of the concerns over Polaris's triumphs seem unwarranted.

You paranoids out there can retire to ruminating about hackers who can see your hole cards. But the success of the CPRG is significant and has implications for both science and poker.

For one, it is at the cutting edge of AI programs that learn from feedback in a very complex game. And, importantly, it shows that a set of heuristics exists for optimal play of heads-up Limit Hold'em. This is enough to make any serious poker player think - a lot.

There is more to discuss, but my editors get antsy when I go on and on. So, let's stop here.

Next time I'll examine a number of related issues that have popped up in the blogs, Web sites and other media about Polaris's recent success.

Author Bio:

Arthur Reber has been a poker player and serious handicapper of thoroughbred horses for four decades. He is the author of The New Gambler's Bible and coauthor of Gambling for Dummies. Formerly a regular columnist for Poker Pro Magazine and Fun 'N' Games magazine, he has also contributed to Card Player (with Lou Krieger), Poker Digest, Casino Player, Strictly Slots and Titan Poker. He outlined a new framework for evaluating the ethical and moral issues that emerge in gambling for an invited address to the International Conference of Gaming and Risk Taking.

Until recently he was the Broeklundian Professor of Psychology at The Graduate Center, City University of New York. Among his various visiting professorships was a Fulbright fellowship at the University of Innsbruck, Austria. Now semi-retired, Reber is a visiting scholar at the University of British Columbia in Vancouver, Canada.

More strategy articles from Arthur S. Reber:

Best Poker Sites - Editor`s Pick

Latest Blogs »