Poker Bots: The Beginning of the End? Um, No

Phil Laak
Phil Laak: Not a bot. But played one on TV.

Most serious poker players, especially those who play online, are aware of the existence of "bots."

Bot is short for robot, and the subspecies of interest here are those designed to play poker.

A bot is, properly, an artificial intelligence (AI) - a sophisticated piece of software that is programmed not only to make optimal decisions but also to learn from its experiences.

There are many phony bots on the market, pieces of programming junk that you can buy or lease. None plays poker better than you (at least I hope not).

Polaris, King of Poker Bots

However, there is one poker bot that has achieved considerable fame. It's a genuine AI, dubbed Polaris, and was developed by the members of the Computer Poker Research Group (CPRG) at the University of Alberta.

Polaris has won several contests against other poker bots and made headlines recently when it outplayed a group of online pros.

You can visit the CPRG site and follow any of the links, including one that will let you play heads-up against Poki, a "baby bot" whose game is good enough to be used in a poker training program. Other links will take you to tech reports and scientific publications.

poker bots
Geeks love sophisticated software.

Bots in general, poker bots in particular and the very notion of AI are topics of endless fascination. Computer geeks love the sophisticated software. Mathematicians revel in the formal properties of the systems that underlie them.

Applied scientists envision extensions into "partial information" domains like bidding auctions, commodities trading and currency exchanges.

Poker players, of course, view them from a host of perspectives - from envy to fear and loathing laced with heavy doses of paranoia.

The success of Polaris also seems to have fired the imagination of the media. Some called it the beginning of the end of poker. Others likened Polaris to Deep Blue, the chess AI that beat Gary Kasparov.

What's the Truth About Poker Bots Online?

Still others warned ominously about mad scientists with clandestine bots lurking on the Internet, running roughshod over mere mortals - tidbits that inflame the onliners with paranoid tendencies.

This is provocative stuff and we need to understand what's really going on.

So let's break it down and see just what the parameters of play are when a human being goes up against Polaris.

1) The Game is Limit Hold'em (LH)

Polaris won't sit down (metaphorically speaking) in a game of Stud or Omaha. It only plays this one game. On the occasions where it was programmed for No-Limit Hold'em, skilled human opponents consistently beat it.

Limit Hold'em, of course, is more algorithmic than No-Limit Hold'em.

poker bot
Laak: If you look closely you can see he's also playing Minesweeper and posting on Antonio Esfandiari's Facebook wall.

This is not to say that LH is not a complex game that demands a high level of skill; it's merely an acknowledgement that it is easier to develop effective strategic generalizations in Limit than No-Limit.

It is also says nothing about the possibility of future bots playing world-class NLH, although this is a task of another order of magnitude. No one knows what the optimal strategy is for NLH, and one may not exist.

2) The Game is Heads-up Polaris only plays against a single opponent. Heads-up play has a reduced number of variables compared with a game with multiple opponents.

The computational burden on a bot that plays against more than one opponent is daunting and, worse, it isn't clear what the maximally effective strategies are. Again, this isn't an in-principle argument against developing such a bot, merely an acknowledgment of the difficulties.

3) The Game is Duplicate Poker: The games were played in a version of poker modeled on duplicate bridge. The same cards are dealt to opponents at different times and each must play them from both sides.

For example, one time you will play A K against your opponent's T 9. Later, you will hold T 9 against your opponent's A K.

are poker bots real
Ali Eslami: Hand-picked by Laak for first Polaris battle.

Duplicate play lowers variance by reducing the impact of luck. It doesn't eliminate it, of course.

For example, you may (correctly) fold a hand that someone calls with and a magical river presents your opponent with a pot you never got to see.

However, compared with random dealing, duplicate is known to reduce variance by about two-thirds. This increases statistical power so that only one-ninth as many hands are needed to yield significant results, which is why the CPRG used it.

Some see duplicate poker as the way of the future. I don't. Because it reduces the luck element, weaker players will have fewer winning sessions and lose too regularly. The balance between luck and skill in poker as currently played fits my Goldilocks Rule - it's "just right."

4) The opponents: The "pros" in the Pros vs. Polaris competition were a group of young, experienced online players. After 3,000 hands Polaris was up 195 small bets, a statistically significant result.

In an earlier contest, Polaris took on two prominent pros, Phil Laak and Ali Eslami. It beat Eslami but Laak won enough so they eked out a small combined win. Our species (assuming that Laak is one of us) hailed this as a victory.

5) The stakes: An important but oft-unnoted feature is that Polaris only plays for "cybercash," not real money. While there is little doubt that the pros are possessed of outsized egos (what top poker player is without one of these?), the fact that no actual harm could come to their bankrolls surely had an impact.

The online hotshots lost a combined total of 195,000 cyber dollars. Would their play have been different if they were confronting the possibility of losing that much "real" money?

Almost certainly. Would they have played better? Perhaps. Worse? Perhaps.

how to beat poker bots
Matt "Hoss_TBF" Hawrilenko: Noted Limit specialist and one of the online hotshots who got bested by bot.

A more sober assessment: Given these factors, many of the concerns over Polaris's triumphs seem unwarranted.

You paranoids out there can retire to ruminating about hackers who can see your hole cards. But the success of the CPRG is significant and has implications for both science and poker.

For one, it is at the cutting edge of AI programs that learn from feedback in a very complex game. And, importantly, it shows that a set of heuristics exists for optimal play of heads-up Limit Hold'em.

This is enough to make any serious poker player think - a lot.

Poker Bots Don't Have Girlfriends

When a bot plays against a human there is a compelling affective asymmetry. Humans feel. Bots do not.

Humans experience the pain of loss and the euphoria of a win. They alter their games in reaction to emotional stress.

A run of bad cards can make some feel insecure and they gear down their aggression; others are provoked and become hyperaggressive. Some react strongly to being challenged by an opponent; others ignore such affronts.

If your girlfriend just dumped you, it probably won't do much for your game. Polaris doesn't have a girlfriend. It is devoid of affective states; it's as dead as a post.

poker bots online
Allen Cunningham: So good that if he didn't have a girlfriend he'd be suspected of being AI.

In various circles, this lack of emotional response in an AI is a topic of considerable discussion.

Debates range from discourses among neuroscientists and philosophers on the links between cognition and emotion to musings among sci-fi enthusiasts over whether androids should be portrayed as less than human by virtue of being bereft of emotions.

Is Polaris's lack of emotional reaction a long-term plus or a long-term minus? Frankly, I have no idea. It could be a bonus because its game won't get derailed by two or three horrific and mathematically unlikely beats.

But this lack of emotion could hurt because Polaris never gets "stoked" by events and take its game to a higher level as a result. This line of argument, of course, depends on there being a higher level to the game that bots can't attain (yet).

The accepted wisdom is that the absence of emotional reaction in an AI is a benefit. This may be right today; tomorrow, it may not be.

Poker Bots Don't Know Anything About Anything

Cognition is thinking; cognitive functions are those that are involved in deliberation, decision making and analysis - processes critical to any intellectually complex task. They include those that are overt and conscious, like calculating pot odds to determine the expected value of a call.

They also include processes that are covert and unconscious, such as experiencing a vague, intuitive sense that you're just beat in a hand. But, no matter how you cut it, these cognitive functions involve knowing, in any of the several senses of the word.

online poker bots
Phil Laak: If he didn't have a girlfriend, he might be legally common-law with Antonio Esfandiari.

Well, one of the signature features of Polaris is that it doesn't know anything about poker! Despite its ability to outplay some of the Limit Hold'em players in the world, it's just a collection of on-off gates.

In fact, it doesn't know anything about anything. Just like feeling, knowing isn't part of what it does. An AI is just a program running on a silicone-based device we call a computer. It's affectively, epistemically empty.

Oh, sure, you could program Polaris to say things like "Hmmm, I've got to think this one through," or to laugh when it steals a pot or throw a tantrum when it ends a session with a big loss, but it wouldn't be thoughtful, happy, sad or angry.

It would just be a bunch of on-off switches instantiated in a sea of transistors simulating these states. This raises questions about exactly what we mean by thinking or feeling, not to mention whether it is possible to ever build an AI that can become truly aware of itself and the world about it.

Such speculations, of course, go somewhat beyond poker sites but they are worth contemplating.

Can Humans Read Poker Bots?

Can a human player "read" a bot? Perhaps. 

If you can ascertain the patterns of play that have been programmed in, you ought to be able to put the device on a range of hands, just as you would a human opponent.

When chess champion Gary Kasparov defeated Deep Blue I, this was his strategy. Deep Blue II was made less transparent and Kasparov, no longer able to make such inductions, lost. It's worth noting that one of Polaris's programmers (who plays high-stakes poker) says he cannot beat it.

Online, where "tells" are usually timing tells, it's going to be "advantage Polaris." I suspect that some of the difficulty that professionals have had playing Polaris can be traced to the hazards of trying to read its silicon "mind."

poker bots
Daniel Negreanu: If anyone could read Polaris' mind, it's Daniel.

Can a Poker Bot Read You?

The flip side here is also important. Can Polaris read you?

Actually, it's likely to be better at this than you think. Because of its enormous computational capacity, it will divine patterns in your game faster and more accurately than you will in its.

And, because it is an AI, it has subroutines that enable it to learn from experience. In order to have a chance to beat Polaris, a player is going to have to take the adage "mix up your game" to new heights.

Poker Bots Create Paranoia

Bots like Polaris generate paranoia for two reasons. One, they play very good Limit Hold'em.

Two, they would likely pass a restricted version of the Turing Test. Alan Turing argued, famously, that if a computer were switched with a person with whom you were conversing and you didn't realize it, then the computer could be called a genuine "artificial intelligence."

A full Turing test doesn't place limits on the topics so Polaris couldn't meet that sort of challenge, but it does appear to satisfy such a test so long as the topic is Limit poker, played heads-up.

You can't get a copy of Polaris and the designers won't allow it to be used by anyone. But there are other bots around, many available commercially.

None are very good so keep your paranoia bottled up. Their main use is making pre-flop "fold" decisions, enabling one to play more tables. But the future will be different; it usually is. 

Play at 888poker Now and Get an $888 Bonus!

Author Bio:

Arthur Reber has been a poker player and serious handicapper of thoroughbred horses for four decades. He is the author of The New Gambler's Bible and coauthor of Gambling for Dummies. Formerly a regular columnist for Poker Pro Magazine and Fun 'N' Games magazine, he has also contributed to Card Player (with Lou Krieger), Poker Digest, Casino Player, Strictly Slots and Limit Hold'em. He outlined a new framework for evaluating the ethical and moral issues that emerge in gambling for an invited address to the International Conference of Gaming and Risk Taking.

Until recently he was the Broeklundian Professor of Psychology at The Graduate Center, City University of New York. Among his various visiting professorships was a Fulbright fellowship at the University of Innsbruck, Austria. Now semi-retired, Reber is a visiting scholar at the University of British Columbia in Vancouver, Canada.

More strategy articles from Arthur S. Reber:

Please fill the required fields correctly!

Error saving comment!

You need to wait 3 minutes before posting another comment.

Darse Billings 2010-10-20 21:37:49

Good article. One correction though: Ali Eslami was not the weaker half of the team in 2007 -- he put on a clinic to win the final match. Phil would say the same thing. If Phil won more overall, it only means he held the strong side of the cards more often.

The second session cards were particularly lopsided, and Phil's more defensive style would have been better placed on the weak side (to lose less). There is still a lot of luck involved in the match outcomes. There are lots of hands where the duplicate format fails to balance (e.g. Polaris folds 92o, Ali decides to play it and wins a big pot with a full house -- random noise, mostly).

Post-match analysis of the 2008 matches (using several variance reduction techniques) indicates that Polaris was full credit for the win. The 6000 hands (two-sided) get magnified to about 80,000 hands, which isn't too bad a sample, and the margin of victory was quite large.

BTW, the redirect for the CPRG link is broken. The direct URL is:

http://webdocs.cs.ualberta.ca/~games/poker/

Cheers, - Darse.

Avner Mart 2008-09-01 22:01:00

This guy knows what he is talking about. I am of the opinion that 5 years from today, or even sooner, bots will be playing online, undisguised.

Best Poker Sites - Editor`s Pick


×

Sorry, this room is not available in your country.

Please try the best alternative which is available for your location:

Close and visit page