Skill vs. Luck: Let's Get it Right

The roll of the dice.
You've got to ask yourself one question: Do I feel lucky? Well, do ya, punk?

The beginnings of an important movement in poker are detectable; scientific studies examining the statistical nature of the game as it is actually played are appearing.

Not just the mathematics of the game (e.g., Chen & Ankenman's The Mathematics of Poker) or applications of game theory (e.g., Ferguson & Ferguson's papers which are available here), but research that approaches the game from a practical point of view based on statistical analyses of the outcomes of actual poker games.

Until the advent of online poker these data could not easily be collected. Few players keep accurate records of their play and those that do are not anxious to share them with researchers.

Two reports appeared recently that have attracted a good deal of attention. The first was the large-scale examination of over 100 million hands carried out by the Cigital group, a consulting firm in Washington, DC.

The two main findings, both of which fit nicely with the intuitions of most experienced poker players, were:

  1. Three-quarters of all hands never go to showdown.
  2. Only about twelve percent of hands are actually won by the best hand.

These results, many maintained, showed that poker is a game of skill for the chance element (the cards actually dealt) played a vanishingly small role in the typical hand.

While this is certainly one conclusion that can be drawn from these data, it is, unfortunately, not a logically necessary one. Go here to see why.

What is needed is more than an analysis of hands, for that statistical base will always be contaminated by uncontrolled factors, but an analysis of players.

The first, solid start in this direction was a recent analysis in Gaming Law Review and Economics by Ingo Fiedler and Jan-Philipp Rock at the University of Hamburg's Institute of Law and Economics.

Bill Chen
How many times do I have to tell them? Hypotheses must be tested!

They examined the records of over 50,000 individual online players. They began with the assumption that poker is a game of skill but pointed out that no matter what the poker community may believe, this is still a hypothesis subject to test.

Importantly, they note that no specific statistical feature of the game has been identified that could be used in this kind of analysis.

Their candidate for this measure is the Critical Repetition Frequency (CRF) or the number of hands needed before a player can be confident that their results reflect skill level, and that a scientist evaluating the data can be confident that these numbers are reliable.

In short, the CRF for an individual player is the point in their playing career where the impact of skill crosses that of the random turn of a card.

By the way, the word "confident" in these analyses means "95% certain." There is always room for error in statistical analyses.

They crunched the numbers from the play of nearly 55,000 online poker players over several millions of hands of hold 'em from mid-level games and the answers they found may surprise you.

First, a large proportion of the individuals whose data they had access to were winners, fully one-third!

This number flies in the face of standard wisdom, which has it as around 5% to 7%. The difference lies in the games being analyzed. The 'standard wisdom' estimates are based on live play and individual intuitions.

Tom Dwan
It takes skill to lose millions of dollars.

The Fiedler and Rock data are from the online game where the vast majority of players in their sample played fewer than 100 hands, went broke and never logged on again.

As Fiedler put it, "No serious player can win as fast as a highly unskilled player can lose."

The impact of this infusion of "dead money" is to make many players winners, although most of them don't win much and the 'standard wisdom' is undoubtedly far closer to the truth.

Second, they discovered that determining who the winners actually are in these games is a complex problem, but to make an exceedingly long story ridiculously short, their data pointed to an answer: a CRF of 1000 hands --- or only some 30 or 35 hours on live play or perhaps 12 - 14 online.

This, of course, is silly. But it is the number that pops out of their analysis. However, there is a good reason it is so low.

Their sample contained that vast sea of horrible players who drifted in and then quickly back out of the games. If you eliminate these from the analysis, things change and they move toward more sensible estimates.

However, even the sensible estimates only make sense when the relative skills of the participants are factored in. Here's what they found:

An exceedingly skilled player, one averaging 100BB/100hands, has a CRF = 300. That is, if they can keep this up for a mere 300 hands they can feel confident in their results.

However, basically no one can sustain this win rate so this number has little to do with reality.

For skilled players with more modest (and realistic) win rates, the number changes dramatically. A (still unrealistic) win rate of 30BB/100h has a CRF = 3,300; one of 5BB/100h is 118,000 and 1BB/100h = 295,000.

If you're averaging +1BB per 100 hands don't trust your results and even if you're averaging 5BB/100h you probably shouldn't either, unless you're multi-tabling and doing it for many hours.

The Fiedler and Rock approach is important. It provides strong support that poker is a game of skill.

David Pham, Bertrand Grospellier
LOL at not factoring temporal elements into the analysis.

The CRF statistic emphasizes what many of us have been harping on forever, the need to take a temporal element into the analysis.

Combined with the Cigital study, it presents converging lines of evidence for the skill-based argument, one from the "hands" perspective and one from the "players."

Of course, the impact of these lines of research on judges, juries and legislators has yet to be seen. Those of us who take an objective, scientific view would like to believe that these groups will concur.

However, we can't lose sight of the fact that many who object to poker do so on moral and social grounds and will fight the fight, not in pages of scientific journals, but in the political arena.

Author Bio:

Arthur Reber has been a poker player and serious handicapper of thoroughbred horses for four decades. He is the author of The New Gambler's Bible and coauthor of Gambling for Dummies. Formerly a regular columnist for Poker Pro Magazine and Fun 'N' Games magazine, he has also contributed to Card Player (with Lou Krieger), Poker Digest, Casino Player, Strictly Slots and Titan Poker. He outlined a new framework for evaluating the ethical and moral issues that emerge in gambling for an invited address to the International Conference of Gaming and Risk Taking.

Until recently he was the Broeklundian Professor of Psychology at The Graduate Center, City University of New York. Among his various visiting professorships was a Fulbright fellowship at the University of Innsbruck, Austria. Now semiretired, Reber is a visiting scholar at the University of British Columbia in Vancouver, Canada.

More poker strategy articles from Arthur S. Reber:

Please fill the required fields correctly!

Error saving comment!

You need to wait 3 minutes before posting another comment.

Comment added. Thank you!

jerry pippin 2013-12-04 18:19:58

Luck obviously plays a much bigger roll than skill does. If you don't get the hands, you can have all the skill in the world and still not win.

David 2013-02-19 19:35:21

Hi,

Only about twelve percent of hands are actually won by the best hand.

Is this gleaned from the 1/4 that go to showdown or from the whole sample ... so includes hands that didn't go to showdown but where the hero was outplayed (or possibly overtaken on the flop/turn)

D

phil grimes 2011-08-31 22:01:30

what?? CFR....100bb/100 hands=300 30bb/100h=3,300 1bb/100hands=295,000 what does this shit mean? you average win 1 bb per 100 hands? 30bb per 100 hands? what does 3,300 mean? what 295,000 mean? i read this over a couple times, makes no sence. what r those numbers?

bennie99 2009-10-09 09:29:00

I'd be interested to know if the 12 per cent best hand ratio was preflop. It would almost certainly be, I would say.

Arthur S Reber 2009-05-02 19:08:00

Wesley:

The 95% "confidence level" is the "gold standard" in the social sciences. It allows for reaching a wrong conclusion some 5% of the time ... a level of error that we're willing to live with.

If you lower it the possibility of error goes up, which is unacceptable.

Don't know about the SnG's, but I bet someone is collecting the data as we write.

Arthur

Wesley Gibson 2009-05-02 17:38:00

That 95% certainty is an excessively high number, in my opinion there is absolutely no need for such amount of played hands.


BTW has anyone made some similar sort of statistics for SNG players?

Wesley