What are we doing about AI, Bots, and RTA?

To understand why, I first want to take a walk through history…

The Sad Demise of Online Backgammon

In 2005, backgammon (another skill-based game with an element of chance) had a boom not dissimilar to the one we had in poker. New online backgammon sites like GamesGrid, PartyGammon and Play65 were popping up all the time, with generous welcome bonuses and splashy marketing. 

Major backgammon tournaments like the World Backgammon Championships and PartyGammon Millions took place in glamorous destinations like Monte Carlo, with equally glamorous hosts like Kara Scott, and with huge prizes for the winners.

Three years later, backgammon was dead.

The main culprit was Snowie. Snowie was a computer program that ‘solved’ the game of backgammon by evaluating all the possible moves that could be made, looking 3 rolls of the dice ahead, and determining which one had the highest expected value. You could play against it as a learning tool, and it would highlight the mistakes (‘blunders’) you made so you could improve. (There were other tools as well, like GnuBG, but these were much less popular). Unfortunately, it could also be turned into the brain for real-time advice, and unscrupulous players did this, as a way to beat the online games.

Backgammon sites tried to defend themselves. One way they did this was to analyse every game in Snowie themselves, and look for players who played exactly the same way as Snowie would. Unfortunately, having learned by playing against solvers, the best human players played pretty similarly to Snowie at this point, so this wasn’t a perfect defence.

Most importantly, they were too slow. By the time the sites had started to take action, it was too late. Players’ confidence in online backgammon was destroyed. They felt unsafe, and they voted with their feet - overwhelmingly choosing to spend their entertainment money elsewhere. 

Does any of this sound worryingly familiar?

The Story of Bots and RTA in Poker

Bots have existed in online poker, in one form or another, for over twenty years. Here’s a post about WinHold’em, one of the earliest commercial poker bots, from 2004.

Below is a screenshot from that post about WinHold'em

It’s so old, they refer to dial-up modems in the post.

Around the same time, early real-time assistance, tools such as Texas Calculatem, came to market. Texas Calculatem attached itself to your poker table while you played online, showed you the odds and the strength of your hand, and gave advice on whether to call, raise or fold.

Both were laughably primitive by today’s standards, in every respect. They didn’t use AI, they were ‘rules-based’, i.e. they made decisions or provided advice by using simple rules such as ‘if I have 7-2 fold’ and ‘if I have a flush draw with 2:1 pot odds or better, call’.

They never caused much damage because they were terrible at poker and could only beat players who were even more terrible. Both played an absurdly weak-tight strategy that would be destroyed even at the lowest stakes today.

They were also easy to detect. Back then, Windows would happily return a full list of all the running processes on a user’s computer if any program asked for it. So, your poker client could just check if WinHold’em was running, and close your account and confiscate all your money if it was. The makers of WinHold’em tried to get around this, first by changing the name of the WinHold’em program to something innocuous, and secondly by coming up with a technique whereby WinHold’em was run on a second computer. But we were one step ahead each time, and so the arms race began.

The Evolving Brain and the Arms Race

Research into ‘solving’ poker isn’t always done by bad actors. In fact, game theory is widely taught in universities and many academic institutions like the University of Alberta and Carnegie Mellon have done extensive research on poker. The University of Alberta Computer Poker Research Group (CPRG) in particular is responsible for some significant advances in solving the game of poker, including Polaris (which beat some of the best players in the world at Duplicate Heads-Up Limit Hold’em in 2008), and Cepheus (which all-but-solved, i.e. was essentially unbeatable at Heads Up Limit Hold’em by 2015).

(As an aside, you don’t see much Heads Up Limit Hold’em played these days).

It was the CPRG who first published research on Counter-Factual Regret Minimisation (CFR) in 2008. This technique is used in most modern-day solvers. It recursively solves the game tree, playing against itself, and iterates by playing different strategies, over and over again, until the optimal strategy is found (GTO Wizard has a good explanation, if you’re interested in more technical detail). Because poker is an incredibly complex game - far more complex than Backgammon - this was dreadfully slow on 2008-era hardware. With the advancement in computing power that we’ve seen since 2008, it’s now possible to solve hands much more quickly, depending on complexity. 

By 2019, the bot Pluribus developed by Carnegie Mellon University was able to defeat some of the best human players in the world at No Limit Hold’em, and in 2020, GTO Wizard was released, bringing GTO to the masses with a relatively friendly UI and good marketing.

Today, you cannot escape the exponential rise of AI. AI is already being used all around us. On WPT Global, we use AI both in our daily work, and to detect and prevent cheating in our games. 

You can see then, that development of the poker AI brain was driven mainly by legitimate research, but of course, some of these advances were also picked up on by bad actors. For example, not long after the CPRG published its research on CFR, we started to see the first online poker bots using it, such as ‘Poker Snowie’ - quite a foreboding name, considering backgammon’s ‘Death by Snowie’ was still in our short-term memories. Unlike WinHold’em, Poker Snowie was actually able to win in the games, and was only unprofitable because we repeatedly caught, banned and confiscated funds from its users. Around this time, we also started to see bots that were able to win in more complex games like Pot Limit Omaha.

The arms race also continued in prevention and detection. Bot makers found new ways to hide themselves and avoid detection (aided by developments like mobile poker software and new versions of Windows that made detection more difficult). We quickly followed up with new ways to detect them. I recently spoke about one of my favourite illustrative examples of this arms race at the EDGE Conference in Dublin, Ireland. Here’s the full presentation.

Chased by a Lion

You and Elon Musk are being chased by a hungry lion. How fast do you have to run? Well, you don’t have to be faster than the lion. You just have to be faster than Elon.

This analogy works nicely to describe Game Integrity in online poker as well. Sites don’t have to be perfect at stopping cheaters. They just have to be either:

• Good enough to stop cheaters before they can make a profit
• Or, better at catching cheats than the competition

I first realised this at MPN, the poker network I ran earlier in my career. We had infiltrated a botting forum online, and the admins had categorised poker sites into green, amber, and red. Users were warned away from trying to run their bots on amber and red sites, because the likelihood of detection was too high, and were encouraged to try on the green sites instead. One of my happy moments was when our investments and efforts in bot detection led the forum to recategorise us from green to amber.

Because to deter bot users, you simply have to be better than the competition, this causes sites who don’t invest in game integrity to get attacked disproportionately more. It is not difficult to figure out who is investing and who is not. Check out the jobs adverts and LinkedIn pages of a few online poker sites and their suppliers, and you can get a good idea of who is hiring people to work on game integrity problems and who is not.

If the use of bots and AI scales up significantly, then I am not frightened for WPT Global because we have invested heavily in both technology and people to protect ourselves and our players, and we continue to do that on a larger scale every day. We have turned AI against AI - researching existing and new AI models and their applications to poker, so that we can spot not only the old-style-CFR-based bots and RTA programs, but also new systems that we haven’t seen before (including exploitative systems and those based on LLMs). Every hand played on WPT Global helps us to train our AI models to detect and prevent cheating even more quickly.

We are undoubtedly a red site. 

However, I am slightly scared for the industry as a whole. The smaller sites of today, who are either unwilling or unable to invest the huge sums required, are likely to become very unsafe places to play. As a result, players will not stick around on those sites. The poker industry thrives on competition and even though this outcome might benefit WPT Global in the short term, the long term effect might be to decrease the size of the market as a whole. That does worry me.

In this post I wanted to do two things. First, I wanted to reassure WPT Global players that we are on top of this problem, and investing to stay ahead in the long term. But secondly, I want to send out a rallying cry to the industry as a whole - invest, or lose everything.

Until next time.

– Alex Scott, President, WPT Global

 PS. If you’re interested in getting involved in the fight against bots and AI, then check out the job openings at our tech supplier A5 Labs as well as the roles at WPT Global.