Brandon Sheils (twitter: @brandonsheils) is a professional poker player and poker coach who recently did a scientific study of poker behavior (aka “poker tells) as part of his seeking a Masters degree in Psychology at the University of Nottingham. Brandon also has a poker-focused YouTube channel.
Topics discussed in our talk include: the challenges of studying poker tells; how he set up his study and the reasons behind the structure; what the results were; the meaning of something being “not statistically significant”; speculations on what AI and machine learning might hold for the analysis of poker tells; some times Brandon has used opponent behavior in poker hands.
Links to this episode:
Things discussed in this episode:
- Brandon Sheil’s Youtube channel
- Zach Elwood’s poker tells books and videos
- Previous episode about poker tells with Dara O’Kearney
- Previous episode with Abe Rutchick, which touched on poker tells research
TRANSCRIPT
Zach: Welcome to the People Who Read People podcast hosted by me, Zachary Elwood. This is a podcast about better understanding why people do what they do. You can learn more about it at www.behavior-podcast.com.
On today’s episode, I talk to Brandon Sheils, that’s SHEILS, a professional poker player who recently did a scientific study of poker tells. We talk about the challenges with studying poker tells, the structure of Brandon’s study and why he set it up that way, what the study found, some talk about general scientific concepts, some speculating on what AI and machine learning approaches might hold for analyzing poker behavior, and then at the end we talk about some poker hands where Brandon used behavior in his decision process.
If you didn’t already know, my own main claim to fame is that I’m the author of some respected books on poker tells; my first book, Reading Poker Tells, has been translated into 8 languages. I’m most proud of my second book, Verbal Poker Tells, which attempts to find the hidden meanings in the things poker players often say during a hand of poker. If you’d like to learn more about that work, you can go to www.readingpokertells.com. If you’re interested in this subject, you might also enjoy a previous episode of this podcast where I talk to Dara O’Kearney about poker tells.
My work on poker tells is what led Brandon Sheils to reach out to me when he was starting work on his study. I helped him a bit in brainstorming the setup of the study, the criteria of what poker hands would be included, and the behaviors he’d examine. And I helped him a bit in going through footage and finding poker hands that met the criteria that he’d later log.
A little more about Brandon and his work: He’s a professional poker player and coach who plays both online and live. He has a poker-focused Youtube channel at brandon sheils, again that’s SHEILS. If you’re curious about his poker tournament scores, you can check out his profile on HendonMob.com, which is a site that tracks tournament results. His Twitter handle is at @brandonsheils.
Brandon did his poker tells research as part of his pursuing a Masters degree in Psychology at the University of Nottingham. That study is not yet published.
One thing that might be important to emphasize before the interview is that good poker players are generally only infrequently basing decisions on poker tells. I want to emphasize this because I think the importance of poker tells is quite exaggerated in the public’s understanding, based on depictions of poker in movies like Rounders or James Bond movies and such. The ability to read poker tells well has been called the “icing on the cake” by some, in terms of it being much less important than having a strong strategy. Poker is a tremendously complicated game; it is a much tougher to solve game computationally than chess and strategy is so much more important than tells. In my poker tells book, I’ve given the estimate that being strong at reading poker tells might add anywhere between 1-15% to a live poker player’s win rate. Put another way: you can be a hugely successful professional poker player without ever thinking about poker tells. As someone who considers themselves quite good at reading tells, if I were playing a full day of poker against somewhat decent players at decent stakes, I might base a decision on a tell only a few times during that session, with some of those spots being pretty small decision points early in a hand, like whether to raise or fold pre-flop. I like to emphasize all this because I think for a lay audience there can be exaggerated ideas about poker tells and how often good players are using them, and all this is especially relevant for Brandon and I’s talk about his study.
Ok, here’s the talk with Brandon Sheils.
Zach: Hey Brandon. Thanks for coming on.
Brandon: Hi Zach. How are you?
Zach: Good. Thanks for joining me. I guess we’ll start with maybe you can go into a little bit about how you got interested in poker and maybe go into how you’ve gotten to playing for a living and what led to your interest in doing this study. I know that’s a lot of questions I just threw out there, but maybe a brief summary of that stuff.
Brandon: Yeah, I’ll try and condense my life of poker into a paragraph as I can, I guess. I’ve always been interested in strategy games, so growing up I played different card games and different games that were more around trying to out-strategize your opponent. And then my parents played poker for a living when I was very young. My first memory, this is when I was seven or eight, and they’re playing like home games sometimes or I’d see them playing on TV. So I knew poker as a game we would play as a family at home sometimes. And I wasn’t like super into it, but I enjoyed playing as a family. And the strategy elements obviously I was clearly very bad at it compared to two professional players and my brother, who’s four years older than me. I think that kind of started my interest. I used to watch the World Series on TV and I liked the idea that I could watch these people playing for a lot of money, it seemed like infinite money at the time. And I could spot mistakes people were making on TV at the time when I was seven or eight years old, and then I started to play home games and pub poker games, and I would win against my dad’s friends even, maybe they were actually drunk people at the pub or whatever. I didn’t understand variants at the time either, it is possible I was just super lucky, but it felt like I was making good decisions and they were making very bad decisions already at the start of playing poker. And I was getting money for this at an age where even 20 pound is infinite money or $25 is going to be infinite money at that age. I didn’t play it too much as I was growing up because there’s no opportunity obviously, but if there was a home game or a pub poker game and I could play, I’d gravitate towards that. And then my brother became a professional player when he was 18 or 19 or as he was finishing uni, and that’s when I was still underage because he’s three or four years older. But that again, continued my interest, my family have made a living at this all at different points. And then I went to the casino for my 18th birthday pretty much during that period of time, I was doing my A Levels, which is the exams that get you into uni in the UK, and it’s like the most important study phase effectively. And I would bring my A Level revision to the casino and do it between hands, because I was that interested in just playing as much poker as I possibly could. And it went pretty well, and then I’ve kind of had a fun relationship with it and having a normal career. And I enjoyed the fact that when I eventually started doing a psychology masters for probably lots of reasons, that I could combine my love for poker with creating a study of my own. And I have a lot of passion for psychology, I have a lot of passion for poker, and it seemed like the natural culmination that I’d end up doing this study.
Zach: So maybe before we get too much into the details of the study, maybe you can talk a little bit about the difficulty of setting the study up and the difficulties you ran into of trying to study poker tells in a scientific manner. What stands out as the major obstacles you encountered?
Brandon: Yeah, so the most forefront problem is creating uniformity across what we’re measuring, because if you try and measure turn decisions or anytime someone bets or times where someone has 10 big blinds or 20 big blinds, there’s so many different factors in poker where the decision making process is completely different, so it wouldn’t be kind of right to compare them. So first was picking one specific area of poker where there’d be a lot to learn from, but also a lot of data available. So the first hurdle I met because I allocate straight away that when someone bets the river, this is the point when they’re waiting for their opponent to make their decision. The voice in their head is either saying, “Please call or please fold.” I thought that’s going to be the perfect point, but it just wasn’t possible to get data on that area because the majority of the stream data footage for poker is they’ll keep the camera on whoever’s turn it is and then as soon as it swaps turn the majority of the time it will be on that player. And as soon as the camera swaps once or twice, as soon as you’ve sort of lost some data, it’s just too hard to have a uniform sample based on that. So sometimes it would go back and forth a little bit, but because it would swap at different rates and sometimes it wouldn’t swap at all, it just wasn’t going to be possible to get data for post-bet river analysis, and I think that would’ve been the most ideal. So that was the first big obstacle and that’s why I ended up doing pre-bet if that’s the right word on the river. And I tried to determine, being a poker player myself, what are the exact situations where there’s the most pressure and therefore the most– if they would be given away based on what I’ve read obviously about human psychology. If someone is betting one big blind on the flop, I think you wrote about this in your book as well, it’s just going to be a completely different subjective experience for them because the risk reward, invariantly, they don’t necessarily care that much if it doesn’t work yet because they can bluff later or they can give up and it’s a small pop. So I decided to choose parts that were at least 10 big blinds, and I just looked at tournaments just because that’s the uniformity that I went for there as well. I could just looked at cash games, I think that that would be interesting as well. And I thought once the part is at least that big and it’s a river decision, that’s a good starting point to say they’re going to care about the result of this bet and therefore they’re going to have to try and balance their emotions a lot more, people are going to take longer to make their decisions. It’s just a more important decision for both players. So I think that was nice to hone in on.
Zach: Yeah. The footage issue is a big problem, and that’s something I’ve dealt with a lot because I’ve created my Poker Tells video series. And ideally you would have those post-bet spots, those spots after someone’s made a significant bet. So I think it was a great decision you made to focus on that slightly before bet and then the actual during bet as they’re placing the bet, because those are usually things the camera stays on them for when it starts their turn the camera’s on them and then up until they place the bet the camera’s on them typically for that stretch of time. So that all made a lot of sense.
Zach: A small edit here, Brandon took a while to explain all the various elements he had logged for the experiment, there were 22 factors in all. But to speed this up a bit, I’ll just name a few of the specific verbal and nonverbal behaviors that he logged. One was the amount of time a player thought before betting, another behavior was the amount of time a player spent placing a bet once they’d either declared the bet or started putting together the bet. Another behavior was whether the bet was verbalized or not. Another behavior was whether the player looked back at their whole cards before placing their bet. Another behavior was whether the player was playing with their chips or not. Again, that was just a few of the aspects that Brandon logged.
Another aspect of Brandon’s study that made a lot of sense was in how he approached ranking, whether a better’s hand was a bluff or a value bet. And a value bet is a way of saying that it’s done for value with a hand that will usually be the best hand. In other words, a value bet is not a bluff. Brandon sent each hand in the study to several skilled poker players who then ranked the hand as either a value bet or a bluff. This was an improvement on a method of categorizing hand strength that Michael Slepian had done in his poker behavior study. In that study, they’d apparently, from what I could tell, relied on the onscreen percentage graphics which are displayed beside a player’s hand graphics. Those graphics show the likelihood of a player’s hand winning, it does this by comparing it to the opponents known hand. This makes it a pretty bad way to categorize whether a player believes that they’re betting a weak hand or a strong hand. In other words, a player could have a hand they believe will tend to be a winner, but in that specific hand their opponent happens to have an even stronger hand. In that instance, the first player’s strong hand would be presumably classified as a bluff. I confess I’m not sure if there was some way Slepian adjusted things to account for that, but my understanding was based on reading their paper. So it’s possible there was more to it. But in any case, Brandon’s decision to get skilled players to rank hands as either bluffs or value bets makes a lot of sense and may I think be the best way to easily make that categorization.
Okay, back to the interview where Brandon talks about this a little bit more.
Brandon: Yeah, I completely agree. I think it was your critique of that past study that helped me get onto that. So thank you as well.
Zach: Oh, that’s awesome. So let’s see. After naming all of those factors that you looked in, it might be anti-climatic they didn’t say what were your findings.
Brandon: Essentially, that none of these factors alone were statistically significant. And that does not mean that they aren’t potentially significant with a bigger sample, but with the sample I had, some of these factors didn’t actually occur that often, even though obviously I tried to measure all of them. But it’s not actually that often that someone double checks their cards. If I find the exact number here, we had in the 400 and–
Zach: 24.
Brandon: Yeah, only 24 times someone double checked their cards and four times it couldn’t be determined based on where the camera was or something else. So 24 out of 416 is a really small amount. If you looked at the ratio there, it would look like it is statistically significant to the human eye, just based on the maths behind figuring out statistical significance, it has to be. The confidence rate is 95%. We have to be able to say with 95% certainty that it’s the case that this makes this more likely, and we just didn’t have the sample. And I think with a bigger sample, I’m quite sure that that would’ve been significant.
Zach: That gets into a question I have, just a general scientific question, which is when I see a study that says, “This wasn’t statistically significant,” is my takeaway supposed to be just that the study cannot tell that? Because sometimes I feel like it’s framed as if there’s no correlation, but maybe that’s just a misreading on my part, and should the takeaway for me be this study can just not determine that?
Brandon: Generally, yes. It’s like saying we failed to prove it, it doesn’t mean that it’s not true. But it depends on the actual science that went into it. So if I had a sample here of a hundred thousand, it’d be pretty hard to argue with it assuming that my practice and how I recorded data is fine, which is kind of a whole other ballgame. But it is determined by there’s something called statistical power, which generally you want to get to above a rate of like 0.8, and it’s determined by basically the sample size. And so some studies are going to have really good statistical power and they’ll talk about that. And if they’ve got good power and they’ve got good science behind what they did, for example, if you don’t use the equity in our case, because that’s kind of a flaw in the science. If you’ve got good power which is due to the good sample and you’ve got good science, then it’s pretty hard to argue with. But you still can’t say for sure that it’s false, you can only say that, “With all of this, it’s not true.” And it’s almost as good as sometimes if there’s enough data though.
Zach: So it might be getting too far in the weeds, and if it is, feel free to say. But for that, say we were looking at that double checking whole cards before a bet behavior, how much more data do you think you need to be able to confidently say like, “There’s no correlation there,” if that makes sense?
Brandon: Well, I can say I, I ran the statistical power test on the sample that I would need total, but I didn’t run it on individual factors. So the initial based on the timeframe I had only had like a month or two of collecting data. It would’ve been impossible for me to get true statistical power on all of these individual factors just because they’re so infrequent. I knew that some of them would hit the benchmark, some of them wouldn’t, and it’s just kind of a starting point. It’s better I recorded it than not, but didn’t become the main focus of the study as good as it would be to have more data on them. So to answer your question about the amount, bear in mind we had 24 where it was true across 392, I’d have to plug it into a statistical computer to run all the exact equations. But in fact, to be honest, I don’t want to make any false claims by coming up with an exact number, but 24 out of that is very small. So I would imagine based on the ratio we’ve got, I’d need at least 4,000 data points maybe more.
Zach: Yeah, I think it really shows the difficulty in this because I think I was going to say too, I think some people would expect me to be disappointed or surprised by not finding anything, and actually these things are so hard to study because as you say, you collected 400 hands and only in a few of those hands is each behavior that you’re studying found, which means you’ve got to get a lot more hands to find a lot of those behaviors. And then you’ve even got more complexity too because you’ve got the fact that the situational context is important, you named a few of the factors involved, but there’s the fact that skilled players can behave quite differently from recreational players and most tells are found from more recreational players. So there’s even this thing of if you were able to zero in on the more recreational players and chop out the pro players, then that would also be an interesting way to analyze it. And this is just to say that this is massively complex to study these things, and I actually had considered setting up a local game to try to study this myself because they had poker rooms there and I thought about putting some effort into it. I started thinking about the same things that you’re thinking about here where I was like, “This would be like a life’s work almost, and I would have to invest a lot of time to really to do this right.” And then I would still be left with these things where I’m like I’m still running into these situational factors where there’s so many things to take into account. And it was kind of just really probably what you ran into yourself actually doing it, was it’s kind of daunting to set it up and to try to distinguish to get the situation down to a specific consistent situation that you’re comparing.
Brandon: Yeah, I completely agree about what you said about recreational players. And I think it’s a factor of self-awareness. So even though obviously being professional kind of comes with that, still there could be a recreational player that has read all about the leading poker tells and spoke to pros about tells that they’ve seen on them, and then they can reverse tell them all the time and be complete outliers. But when it comes to just people playing and not thinking too in-depthly, then the less experienced you are or the less that this is your profession and you’ve really fallen into this sort of stuff, you are going to just naturally give more stuff away or not realize that saying certain things is indicative of strength or weakness because you haven’t just got the sample size or you just don’t kind of care enough about that, you’re just there to have fun and you just do what you’re feeling at the time as opposed to thinking, “I need to recreate this situation all the time and not be exploitable.”
Zach: Yeah. The other one that I was, it was the double checking of whole cards, one that I was expecting a little something from, and then the other one was the length of time thinking before a bet. I kind of thought we’d see a little something there, but then I was thinking about it after you said you didn’t find anything. And I started thinking well, maybe it’d be hard to find it anyway, but I was thinking the fact that a lot of good players like to tank a long time with their good hands and their bluffs regardless and good players tend to take a long time in general, and I kind of wondered if that would throw off the timing averages. That thing, again, if we were just studying recreational players and you had this similar sample size of just recreational players, I kind of feel like there’d be a little something there. But anyway, that was all just stuff I was thinking of when you told me the results.
Brandon: I did notice in the stream games in… I forget the city now, is it Chicago? The Wind City, was it?
Zach: Oh yeah, Chicago, Windy City.
Brandon: Windy City, yeah. In those games, people acted so quickly, and it did make it really hard to gather data because I wanted to use more of a breadth of not just these big tournaments and more these were still tournaments in different environments. So I thought it’d be good to have a wider range of players, and there was so much more often that people acted in even less than two seconds. And because my whole analysis is on how long they take to make a bet, it almost became… You can’t get most of the factors when someone takes less than say 10 seconds. So I think I actually started excluding anything less than five seconds because you just couldn’t really determine anything and that didn’t feel so great to do either, because a lot of people snap at with bluffs or with value based on what they’re thinking too which you don’t want to exclude from the data. So it can change a lot based on the environment.
Zach: That gets into another thing here where the thing about using poker tells is applying a player specific filter to it. So, for example, if you’re playing with a few people who you noticed are pretty quirky and they’re always betting quickly or just betting weirdly or doing other weird things, it’s kind of like if you can’t find anything noticeable on them, you’re not going to apply the common general tells that you might apply to somebody else and other recreational player to those kind of quirky weird ones who are doing unusual things. And that kind of played into it too. And when you said that about the Windy City games, which I used in my Poker Tells series, you are right. Because some of those games they satellite into those games from lower level tournaments and there can be almost like a home game feel to those. And when you said that about those games, I was like, “Yeah, now that you mention it, those games are really quirky and people do all sorts of weird things and act really quick for spots you wouldn’t typically see that for.” Because it’s a lot of the same player pool and I think that kind of lends it to this kind of home game feel. So anyway, that was just to say, yeah, it’s tough to study these things basically.
Brandon: Yeah. I had two lines of four from what you just said. The first was the other thing with the Windy City games was there was a lot more times where there was not a unanimous opinion on whether it was bluff or value because there was a lot more times they would bet and I would say they didn’t know why they were betting. Or even sometimes the way they turned their hand over would be that like, “I don’t know if I win. You’ve called, here’s my hand, I don’t know. Maybe I win.” Well, it just meant if I know they don’t know, then it’s harder to get into their psychology because they might be almost free rolling it psychologically to think what they do is what they do. If they’re not thinking in-depthly about the strategy, you can’t really get the same data from them because they don’t know what they’re thinking. The other thing I was going to say is because my study was mainly focused on universal tells, I wanted a wide array of players, I didn’t get more than 10 samples on one player. So the whole 420 data points, the most I got on one player was 10 because I didn’t want it to be kind of too many of one player. But as you’ve wrote about and touched on already, I think if you just looked at one player and across many situations, that would be the best way to actually determine what stuff they do, what their tendencies are when they’re bluffing or value betting. And you very clearly see that the pros are much more balanced in that case and a recreational player I imagine it is not. So having like a hundred hand sample on one player in just these spots I think would’ve been very interesting. Not as useful for universal tells, but just to see that stuff is clearly different.
Zach: Yeah, it’s interesting because when I think about applying poker tells, so much of it is knowing which tells are likely to be true for someone you’ve kind of classified as a certain type of player and then there’s those kind of general tells for different player segments and then it’s also noticing the player-specific tells for things that you wouldn’t apply general tells for. So I mean there’s definitely some tells that I would use cold just because they are so common, assuming I peg someone as fairly recreational. And then there’s other tells that I would never use cold where I’m like, “I need to know more about this person.” So it’s kind of this intersection of universal, which I think is interesting too, and then the player-specific, which is almost like, “Let me study someone for a while and build up a little bit of information.”
Brandon: That’s pretty much what I’d say, I do want them at the table as well. The more information they give away in the hand, whether they’re talking or they’re doing certain things with their body language, if their hand gets to show down, I’m like, “Oh, that’s a data point in my memory about this player.” I can’t use it yet, but if they’re doing something completely different in another hand, then they get to show down and the hand is opposite, I’m like, “This is already quite a lot of data.” They’ve done two opposite things or two opposite ends of spectrum of hands, and some people are really smart and can kind of duck and dive around that, but a lot of people don’t realize how much they give away in the moment.
Zach: And the other interesting thing too is sometimes people say like, “Well you didn’t get to see their hand, they didn’t show it down,” but in practice so many players are only making big bets with value bets. So you can often just assume, even if you’ve seen them not show down, if they’ve made only a handful of big bets in a few hours time or something, you can safely assume that those were value bets if they don’t seem like a bluffy kind of player or whatever. So there’s even that kind of correlation you can draw, which is a little obviously not certain, but it’s kind of in the realm of assuming it’s probably true which can be helpful in some spots too.
Brandon: I was just going to add, another thing people forget with river bets, just based on how the pot odds and the maths works is that, for example, if someone bets the size of the pot, the price you’re laying for opponent is two to one. They’re calling the original size of the pot to win three times the size of the pot. They need to be right one in three times. So the person that bets is supposed to have it most of the time even with a perfect strategy, which obviously no one has and people are normally quite bad at bluffing or they do too much depending on the player, but they’re supposed to be making you indifferent, which means if they bet the size of the pot, they’ll lay you two to one. In theory, depending on obviously the different ranges and perceptions, they’re going to be bluffing between 60 and 70% of the time just as a factor of the pots. No, sorry, they’re going to have value that’s 67% of the time, so they’re only going to be bluffing–
Zach: Yeah, the game theory fundamentals, yeah. And yet that’s what you found in your study too, it was like 70% value bets, right? Something like that, yeah.
Brandon: I have the exact number here. Yeah, 71.2% value bets across 420 points. I don’t have the average bet size here which would be really useful as well actually to see the difference in what it should be. But clearly people always have it, that’s always been true across pretty much every focus environment.
Zach: I was going to ask too, I wasn’t exactly sure how to interpret it, but in your paper you had written that I think you did find something, it was like depending on controlling for a few variables, there were a few things that were interesting or was I misreading that?
Brandon: Yeah, so this is exploratory analysis, which again, I’m no expert on the nuances of how this works and this is using regression. So my understanding is if I find one of the statistically significant results we got, player verbalized the bet when controlling for total turn time, best size percentage, and if the player raised, if they went all in and if they were protecting their cards. So because I have 21, 22 factors, 21 independent variables, when you use the regression to see if anything’s significant, it’s using all of them in a different way to say like it uses the data such that it can isolate certain variables, whereas if you did the test just with a one on one variable, it would be different. So it’s almost accounting for the other variables. It’s hard to give a direct example, but the fact that someone raised is kind of its own area of data. And if you looked at just this when they raise or just this when they don’t raise, it’s almost like controlling for if they raised. So if you kind of play around with the data and just control for specific things, then you can get statistical significance. It’s not as useful because in theory you could kind of cherry pick it and work around with it and there’s always going to be significance you can find if you… Well, I guess not always, but if you really go into every possible permutation, you’re going to find significance. And this is one of the problems with [paper] sometimes as well, is that if you don’t pre-register your hypotheses and what you’re actually looking for, in theory afterwards you could have hypothesized the result you got was the one you wanted and then this is now a big headline, people are going to share it.
Zach: Yeah, I was reading something about that recently where they were making some point about finding something completely ridiculous, correlation between, I can’t remember what it was, it was something about DNA and something completely unrelated, but it was to make the point for what you’re saying, you can theoretically find significance if you look across so many different permutations and combinations, you’re able to find some correlation in something.
Brandon: Yeah. The more times you look for it, you’re supposed to adjust to the fact you’ve looked more times because it’s more likely you’re going to find it. So that actually affects the significance rate as well. So there’s more maths you’re supposed to apply to it. But if someone doesn’t pre-register the hypotheses and doesn’t use correct sound science, then… Like, if my story for this paper going into it I’m blindsided in the fact that I think when people, let’s take all of the random points, when people play with their chips, they’re always bluffing. And I really went into this paper thinking that and I write my whole paper such that I’m kind of looking to prove that it’s true, then if I do a test such that it doesn’t come out that way and I then want to like kind of move around the different data points and say, “Oh, what if this data was never recorded for? What if this data was controlled for in this aspect?” Until I find something significant and make out that was the first test I did, in theory I can then publish a paper and then say, “Yeah, this was statistically significant because of this.” And that’s why most papers today, I don’t know if all journals do this now, but I think most reputable ones, you have to pre-submit the paper and basically make sure that that can’t happen because there’s too many cases in the past where people have been doing this. And so that’s what we did with this paper as well for all it’s worth.
Zach: Nice. So in your case, in the one you mentioned, the verbalizing bet, so that would mean that depending on those other factors that you named, there was theoretically something there with the verbalizing bet, and maybe that points to like further study basically. Is that what that would tell you?
Brandon: Essentially, yeah. Another one here is thinking time percentage when controlling for if the player raised and if the player went all in. So already that’s a really specific area that they raised and went all in because I think generally people take longer as well then because it’s a second decision and it’s a bigger decision. So it’s like almost going too far off the path sometimes when you look into these specific areas, because I’m now honing in on one, I’m honing in on something which is kind of a small sample, and the data might be significant for this same as to give an extreme example, if I controlled for if the player had the nuts, obviously it’s going to be really significant that they’re never bluffing because I’m only looking at times where they’ve got the nuts. So you do kind of have to be careful with it. But I mean it does show that there is stuff going on in some permutations of the tree.
Zach: So are you interested in doing more in that space? Are you theoretically interested in adding more to your sample size, that set of hands that you have or any plans like that?
Brandon: I’d say that I am and I’m not at the same time. I enjoyed the process a lot and I really enjoy poker psychology, but I had deadlines and a timeframe and I was working on my own so it was a very different type of study as opposed to a full fledged study with a lot of people at it and a lot more sophisticated team and more people to collect data. I would enjoy being a part of that I’m sure, especially as, I don’t want to say poker expert, but I guess in that context that’s what I would be because I’ve played professionally for so long. But I’d be happy to be involved in and help out in these studies as they go forward or potentially have more of a role depending on the opportunity. But I think if new technology comes out where there’s new ways to analyze the data or it becomes easier or there’s new ways to think about it where there’s a lot more we can learn in a different way, that could reignite my interest as well or I think as you spoke about I really like the idea of someone creating a game which is the psychology poker game. Oh, sorry, just to go back to one we’re speaking about hurdles, the problem initially with this is if you bring people into a lab to play poker, the data’s almost worthless because they don’t have any risks. I played Play Money with people before and it’s not poker, unless you have a league or something that means something. People just don’t care, they’ve got to have their own risk. I really like the idea of if hypothetically I had infinite money to make this study, I can put on like a big tournament, whether it’s a league or whatever, have some pros, have some maybe athletes, have some famous people in different areas or purely recreational players and just analyze as much data as possible, but just give away actual prizes as well like prize money that means something to people or a title or a trophy, and it becomes kind of prestigious to be able to win this game where maybe it’s one table of six max every week and we record heart rate, we record breathing rate, we record the eye shiftiness directly–
Zach: Skin conductance.
Brandon: Yeah, yeah, as many things as you can possibly record without being too intrusive such that people can still relax enough to play the game and gather all the data as possible and then kind of have a… I can imagine if we recorded this and people watched it, you could have experts generally saying like, “This is leading to this. We can say that this is more likely because of this or here’s the science behind this,” and I think I would find something like that really fascinating to watch as a poker player. So I imagine other people would find it interesting.
Zach: No, I think it’s a great idea, and I think it’s like using the entertainment factor as almost like an excuse to do the science, because you’re creating that real environment. And that’s what I was struggling with too, because actually I spent a good few weeks brainstorming this a while back in Portland where I was because it was that same challenge of it needs to be real obviously, but then what am I doing to induce people to be willing to do this with a bunch of cameras and detectors and stuff? And it’s like I would have to pay them a good amount, so for many reasons, it had a lot of obstacles. But I think your idea’s great because it would be using the entertainment and the money involved that would come with that to do some cool science. And actually I don’t know if you ever saw that show, I can’t remember what it was called, it was very short-lived.
Zach: A small edit here, I talked a little bit about a poker TV show here that I couldn’t remember the name of. In the show, they had recorded the players’ heart rates. The show I was thinking of was from 2006, it was called Poker Dome Challenge. And it was only on air, I think a few weeks. Back to the interview.
All I remember was that it was only a few episodes I think, and they recorded I think it was heart rate, but maybe it was something else. But does that ring a bell at all with the heart rate?
Brandon: I mean, I’ve seen streamers do it now, but they have the heart rate on the screen while they’re playing it.
Zach: I haven’t seen that.
Brandon: There’s a guy called BBZ, I’ve seen him on his streams where his heart gets like 140, 150 when he is doing a huge bluffing like a 10K tournament. You can see it going up.
Zach: I haven’t seen that. Okay, I got to check that out because I always thought it would be cool to wear those monitors on yourself when you go to play or when you’re playing at home or whatever. I think that stuff is really interesting. And you can also buy the EKG skin conductance things too if you really wanted to get into that.
Brandon: Well, maybe as a starting point, hypothetically, if there’s a game that already runs then imagine if you could say to those people before they play like, “We’re doing this study, do you want to have your [data] measured? I don’t know what incentive you can necessarily give people. But if we could start to get data from that from games that already exist before creating a full-fledged game, that seems like a good kind of stepping stone. I would be happy to do that. I’d be interested in how my own physiology reacts when I’m playing if the whole cards and everything’s already streamed.
Zach: Totally, yeah. No, and if there’s anybody listening who’s into that idea, contact me and/or Brandon, we’ll look into it. It’s interesting too thinking about how AI, machine learning, video detection stuff can play into this too, because you can imagine usually heart rate can be kind of hard to see, but you can imagine hooking up something where it’s like recording a specific person’s heart rate or even indicators of like flushing at a very minute, detailed level and then correlating that in some way and noticing things that wouldn’t be obvious to people. And that’s something I think is interesting too because, for example, there was a recent Israeli study that found facial movements pretty high frequency ability to detect deception by minute facial movements detect when people were lying, which struck me as like these kinds of things that are not obvious to human eyes, but that a video recognition AI could pick up gets into a kind of a scary area where you can imagine somebody making some really awesome advancement and using that to really take advantage of that at the poker tables without anybody knowing. Because if you had something like that, that would be a way I would be using some advanced technology like that if I was trying to make the most money and willing to cheat basically. So it’s something to think about.
Brandon: Some super glasses. I was going to say that I know you’re saying if you could record or AI detected all these extra features of someone’s face, the stuff we don’t see, I actually think there’s a lot of stuff that we pick up subconsciously. Because when you’re playing, if you look at someone, it’s almost like you can’t pinpoint why, but you can just sense discomfort sometimes or sense, comfort. And you won’t be able to put it into words it’s because of X, Y, and Z, but well this is kind of an ongoing debate in the psychology world that we have an area of our brain that is either really, really good at just detecting objects or it’s really, really good at detecting faces. And we don’t know if it’s either that we see so many faces that that’s why we’re so good at determining faces or that we have a specific area for faces and it’s still kind of up for contention, but either way we are much better at reading faces than we realize. We pick up so many sort of cues as well just as humans, even if we can’t document it. It would be really cool to pinpoint that into a big AI super learning machine that you just tell it it’s bluffing. You just say like, “Watch this guy’s face for this period of time,” and it comes out, you can plug in the next day and it’s like, “Yep, they’re bluffing. Yep, they’re not bluffing…”
Zach: No, totally. And I think that is not far away. They have an app for analyzing video for various things and you could plug that into some machine learning stuff and study a bunch of footage. They have these black box machine learning things that can just spit out correlations and you don’t really know how it’s working. And I think that’s stuff that I think you could theoretically do now if you were so inclined. And I think like you were saying, it’s like the things that we often don’t notice consciously or just don’t notice at all are these kind of like when someone’s relaxed they might have little tiny micro movements that are not really that obvious to us, but that might stand out as like the things that we pick up as a feeling or a vibe or things that the machine would be able to get down to a really fine grain detail very exact.
Brandon: The only issue is just going to be similar to my study is sample size. I know how poker solvers work, they play against themselves millions of times. I don’t know how many times you’ll have to but someone doing this like river action or whatever before it can be statistically significantly correct that much percentage of the time. It might need hundreds of thousands or millions of bits of data. If we did create hypothetically in this parallel world, if we had infinite money to just make this game, then you’d have the camera exactly on everyone’s face such that they can’t move between so far or you can always see their entire face and you get a pretty big sample pretty quick just doing that. Because every time you’ve got everyone’s face in every game and they play every day for six hours, then you start building a sample pretty quick. Obviously not compared to the numbers you might need, it would take a very long time, but if there was more games and more people doing that, then that’d be a really good starting point.
Zach: So it sounds like we need multiple numbers of these games set up around the world going 24 hours a day. So, yeah, we’ll get started on that. So I wanted to ask you too, are there any tells that stand out to you that you use live when it comes to maybe a recent hand you played where a tell made a decision for you? Anything stand out in that regard?
Brandon: Definitely yes. I try not to base… I say never, it’s very rare that I’ll make a super, super export base purely on a tell. I’d have to be really confident which is very, very rare situation. It’s a dangerous place to be if you’re so confident in something like that, but it does happen. I play a lot of hands, and it’s very, very rare that will happen. But obviously I won’t go directly into this means this, because then I’m going to get leveled very easily next time I see that.
Zach: Yeah, I hear you, I hear you. I get you.
Brandon: I mean, there also isn’t a direct thing. I can give you one actually. I’ll give you two examples that come to mind for playing in the last 12 months. There’s one where I had a friend who’s a very good online player and I know he was new to the live poker game, but he’s a very good theoretical player. And there’s something which I call card apex, which I can’t remember if you also wrote about, but I’ve seen it in a few places, which is when you look at your hand for the first time, if you see that it’s like ACEs or Kings or like a really good hand, you naturally put it down quickly because your body’s like, “Oh, shit, good hand.” People just put it down quicker. Whereas if you see more of a marginal decision where you need to think about it, people look at it for longer. So if you see a Jack-Ten suited, Jack-Nine suited, Ace-Five suited, something which is you want to play, but it might be dependent on the action, whereas compared to a hand that you’re always playing no matter what, people tend to look at it a little bit longer. I played a hand where I’d raised first to act, and this guy was on the button. And he looked at his hand and he was still looking at it for like a few seconds and I was watching him and they put it down, and then he re-raised and it came back to me and I had like the worst hand I could possibly have. So I was like I really want to just go all in here, but if I’m wrong, I’m just a complete idiot. And I’m really sure that he’s bluffing based on this one tell, but I know it would still be too out of line for me to go in with this hand. If I had a hand I’m supposed to bluff here with sometimes, I’d just do it every time. But that’s how I’d calibrate. I wouldn’t then go in with a hand I should never go in with just to control my frequencies. And so I just said to him, “You’re bluffing, aren’t you? I’m so sure you’re bluffing here. Please show me and I’ll fold it.” And he showed me he was bluffing. So that’s just one nice one which can be quite reliable for people.
Zach: No, I like that one. I like that one a lot. I write about that a good amount and I talk about the kind of psychological reasons behind that, and I think I wrote a good amount that in Exploiting Poker Tells, my last one. And I will use that one a good amount to decide whether to three bet somebody preflop if they raise and they stare at their cards a little bit longer than normal, longer than average. I’ll use that as a decision point to make a looser three bet.
Brandon: Yeah, I think it can be a really nice one, but the more important part like my decision process there is that I don’t know he’s bluffing there, it’s a strong indicator. So I can use that to make smaller exploits by saying… Let’s say I fold bet all in as a bluff there with– I don’t know, 5 percent– maybe that’s not the right number, but instead I go to 5.5 and all the hands that I’m supposed to mix, I just always use. And maybe there’s one hand which I don’t use that I then use, but as soon as I start, I just go on a limb with everything. It feels like it’s too far away from a strategy, so to speak. The example you just gave I think is a great indicator, but you don’t just re-raise the seven two off suit because it’s–
Zach: No, exactly. Because they’re still going to call you some percentage of time or whatever, it’s not… And like you said, it’s far from certain anyway, it’s just making it slightly more or even significantly more likely. But yeah, you’re right, you have to keep in the factors of what’s good play too.
Brandon: The way I talk about it is you’ve got to give weightings to your assumptions. So my assumptions in some spots are not worth much because I don’t know much about the player, I don’t have much info, but in other spots they’re worth a bit more. And this is an example where based on my history of playing with people and the psychology I’ve read behind it, my assumption that that meant he was bluffing is worth something. It’s not worth everything, but it just allows me to expand my range a little bit. And the other example that came to my head was a hand I played in Vegas against someone who was a recreational player. To some extent I think he probably had a job but was just out for the World Series, and he plays poker for fun, but he’s not necessarily terrible, but he’s not professional. So there’s a hammer I raised with Ace King and he called in the big line. I’ve gone too poker technical I guess. I bet on 7, 7, 6, 2 or flush draw and I just have Ace King and he raised. So this is a point I know where I always continue with Ace King against someone that raises correctly, but my assumption tells me he’s not raising correctly because he’s not professional. He’s not going to know which bluffs to use, and it’s quite counterintuitive to find some of the bluffs. But obviously some people easily overdo it too, but I’ve not seen him do anything crazy. So my head’s playing back all these different features like, “Do I defend my Ace King versus the raise maybe he’s always got two power set and I’m just losing loads of money or maybe I’ll just keep him honest for one straight and then over fold the turn. I just started staring at him, and it was just clear that he was uncomfortable. And I can’t necessarily explain why, but something about his eyes and the way he had his movement, everything. Because I’d seen him play a few of my hands where he had good hands and just his body language was just completely different. And it was almost enough for me to say, “I’m not going to fold this hand at any point unless his body language changes. And if I’m wrong, I’m wrong, I’ll die by the sword at this point. I’m so confident in the fact this guy doesn’t look comfortable.” So I called, the turn was nicely at two, so nothing changed. And then he bet again, and it was the same story. And I went as far as to… I don’t know if this actually made a difference, but I tried to make myself look as weak as possible when I called the turn because I really wanted him to bluff the river. So I really made it look like really begrudging call like, “Ah, this is a close spot for me.” And then I got one of the best rivers in the deck, another two, so every single bluff became the same hand by the river. And he went all in, and I called him, he had a really strong bluff. He had open-ended straight flush draw, but it would’ve been really easy for me to just over fold that flop against the other players or over fold the turn without that extra I think he’s uncomfortable so I’m going to go closer to theory here.
Zach: That’s a real interesting one.
Brandon: But I couldn’t pinpoint his hand was in this place or he had this brief. Whatever it was, it’s a combination of lots of things.
Zach: Yeah, I was going to say it gets into that, like you were saying, sometimes there may be things that we feel that may be based on, for example, you subconsciously noticing something about how he was acting in previous hands that was like his eye contact was completely different that didn’t really consciously register to you because I think eye contact’s really big and an underrated behavior. But some of these things can be things that we’re kind of slightly aware of which gets into that realm of… I actually had a really good interview with Brian Rust about this kind of stuff about poker tells and he was talking about–
Brandon: I like Brian Rast.
Zach: Yeah, he’s great. I respect him a lot, poker-wise. And yeah, he was talking about playing draw games and the fact that there’s so little information in draw games. And so a lot of it comes down to these like, “Well, do I feel one way or other about this?” There’s a lot of these borderline spots where you’re put in where you’re like, “Well, this could go either way,” more than other games because you have less information. And he was saying he really does trust those feelings sometimes and that he thinks that’s a source of a big edge where you’re just like, “I just feel like even if I can’t put my finger on it, I think this guy’s bluffing or this guy’s got it this time.”
Brandon: Yeah. I think especially in the single draw games where it’s decision draw, decision hands over, then you get so much less information about how to range your opponents and that becomes a much bigger component of the strategy used.
Zach: Well, this has been great. Anything else you wanted to throw in here before we signed off?
Brandon: I guess that the only other thing we didn’t touch on that I had one note on was determining player skill, a way to do that. I was just going to mention that I was going to use Hendon Mob as a reference point to say, for example, if someone has 10 million in cashes and they’re playing a 5,000 pound buy in, it’d be good to use that as a metric to say obviously that I played a lot of big tournaments and maybe the amount of total cashes they’ve got could go into that and we could have a formula and a rating. So you could have a degree of live professional poker player based on that. And there is a lot of problems with it because if someone is a business man with millions of pounds and plays high rollers and then wants to play a small tournament, it might bias the data, but I’m sure there’s a way to do it to make it correlated to skill level. So I think that’d be really good to incorporate into future studies if we could create it, some sort of system of recreational to pro, maybe a scale of one to 10, and then we could use that to determine how useful some of the data is or to see if there is a lot more indicators when it comes to more recreational players which I think we both agree is intuitive that that makes sense that it’s true.
Zach: Yeah, that sounds great because even if it wasn’t perfect, it would still be something that you could filter through.
Brandon: Yeah. I guess other than that, I just wanted to say thanks, you helped me determine the hypothesis of this study, you helped me kind of plan it out in a really nice way and incorporate much better science in some ways, learn from past mistakes of other studies. I didn’t know so much about the other poker study that happened in the past, the sleeping one, but as we touched on today, there was some issues with it and I think my study became the next step from there in some ways. We improved on a lot of the problems of that and it’s going to make for better science in the future for the next study in this space. Reading your books and speaking to you helped me learn a lot about the space and make a lot of good decisions when it came to studying it and recording the data. So thanks.
Zach: Yeah. Thanks Brandon. I appreciate you saying that and thanks for talking to me and look forward to seeing what else you do. Yeah, thanks for coming out.
Brandon: No problem.
Zach: That was a talk with Brandon Sheils. You can find him on his youtube channel, which is titled Brandon Sheils, or on Twitter at @brandonsheils.
If you’re interested in poker, you might like to check out my poker tells work, which you can learn about at www.readingpokertells.com.
If you like this podcast, I’d very much appreciate you sharing it on social media and giving it a rating on iTunes or another platform. You can learn more about this podcast at behavior-podcast.com You can follow me on Twitter at @apokerplayer.
Thanks for listening.
One reply on “The scientific study of poker tells, with Brandon Sheils”
[…] Episode about poker tells research […]