Categories
podcast

Indicators of fake Amazon reviews, with linguistic researcher Olu Popoola

An interview with Olu Popoola, a forensic linguistic researcher focused on studying indicators of deception. He’s a doctoral researcher at University of Birmingham U.K. and he also consults as a fraud investigator and corporate trainer on deception detection. Here’s his Twitter account: @oepopoola. Here’s his bio and here’s his blog at Outliar.blog.

A transcript is below. 

Links to the episode:

Topics discussed include:

  • What does linguistic analysis for deception entail?
  • The definition of “coherence relations” (a basic principle of linguistic analysis)
  • Indicators of fake Amazon book reviews and true/honest ones
  • Popoola’s work examining Amazon reviews of Hillary Clinton’s 2017 book What Happened?
  • Why is detection deception important in our modern age?

Here are some resources of Popoola’s or on subjects that came up in our talk:

TRANSCRIPT

[Note: transcripts will contain errors.]

Zach: My guest today is Olu Popoola. He’s a forensic linguistics researcher, corporate trainer, and learning developer. He conducts workshops and does consulting for fraud investigators and solicitor-advocates on subjects of deception detection and credibility assessment. He’s a doctoral researcher at the University of Birmingham in UK. He studied Law at King’s College London and then did postgraduate research at the Centre for Forensic Linguistics at Aston University. He’s a member of the International Association of Forensic Linguists and a Fellow of the Higher Education Academy

Regarding his deception detection research, here are a few of the things he’s worked on:

Detecting deception in Amazon reviews
Detecting deception in the statements of athletes who talked about use of performance enhancing drugs
Detecting indicators of fake news and propaganda

Olu has a blog he calls Outliar, where he writes about his work. That blog is at outliar.blog. That’s OutLIAR.

You can find him on Twitter at @oepopoola; he shares a lot of cool and interesting stuff about language and deception on there and I recommend following him.

Hi, Olu. Thanks for coming on. Hi, Zach.

Thanks for having me. Oh yeah, my pleasure. Thanks. Thanks for joining me and, and being willing to talk about your work. Uh, it’s very interesting stuff. So, uh, was that a pretty good synopsis there of your work? Anything else major you’d like to add to that?

Olu: Yeah, that’s a good synopsis and I’ve also worked on a couple of other things.

Uh, [00:02:00] actually I worked on a child abuse, uh, case, which. Was actually what I first got me into deception detection. I can’t really talk about the details of that because there’s an embargo on that, but, um, that was a, a quite a big, big project. And I also, I. Worked on a native language identification project, uh, with regard related to the Gucci hacker.

Myself and a couple of colleagues at University of Birmingham were trying to figure out what was the language of origin, and there was a bit of deception detection work in that as well. Because, uh, we, we identified that there was more than one author,

Zach: right? Remind, uh, could you remind the viewers what, uh, Guer or Guer, um, what, what the story was behind that really quickly?

Olu: Um, that was, uh, regarding the hack of the, um, democratic. [00:03:00] Convention emails, and it was a question, who was this? This person, this was Gucci for 2.0 because the original Gucci for was, was a Romanian and was, um, had been caught. Mm. So this person was pretending to be related, but um, according to our, um, analysis.

We figured that the person was not Russian, was not Romanian, but was Russian, likely Russian, and possibly with a Dutch influence. Hmm. Which, which, which sounds a bit odd, but, um, we did subsequently find that there were reports of, um, Russians in Holland working on hacking in this area. So. We think we might have been, um, on point with that.

Zach: Did they figure out who that was or that was not solved?

Olu: They didn’t figure out who it was, but they um, they did narrow it down to a Russian [00:04:00] intelligence cell based in Holland.

Zach: And you had, uh, your work helped figure out, uh, narrow down some of the language clues to Holland related and Russia related.

Olu: Um, yes.

Uh, we were not asked to, our research wasn’t given to the FBI or anything like that, but, um mm-hmm. It’s, it, it, it’s out there. And it did corroborate a, a lot of, um, the theories that were going on at the time.

Zach: So let’s talk about, uh, linguistic analysis in general. I know that the, you had told me that the concept of coherence is a very important one when analyzing text.

Can you explain a little bit about what coherence means in that context?

Olu: Okay. So can coherence is how a, a text is put together, how a speaker or a writer puts a text together. So. Writer might put a text together as a series of, um, [00:05:00] elaborations. You talk about something, then you add some more information and you add some more information.

Um, another text might be put together, um, as a series of contrasts. You talk about something and you compare it or contrast it with something else. Another text might be put together as a. Um, a series of phenomena and results, causes and effects.

Zach: Right. So it’s like something that’s in incoherent, a text that, in that’s incoherent would just be a bunch of random sentences that don’t relate to each other.

But yes. When, when you study coherence, you’re studying like how the phrases and the, you know, the clauses and the phrases and the sentences relate to each other as a whole.

Olu: Exactly. And so. Very different linguistic systems, um, that I use and I’m developing, uh, one myself. They’re called coherence relations.

They’re different ways [00:06:00] of, of, um, gelling your text together. Exactly. That,

Zach: how these things interact basically as like, sort of like a machine almost. Yes,

Olu: yes, exactly that.

Zach: Yeah. That’s, that’s really interesting ’cause I first read about that when reading about your work, that there was this. Kind of, you know, more scientific, uh, way to approach that in a, in an objective way.

And, and it sounds like, like you said, there’s different ways people have constructed different systems, people have constructed for how to compare those, um, coherence relations.

Olu: Yes. So, um, I’ve been using one particular, uh, system. It is called, uh, rhetorical Structure theory, RST. And that was my initial foray, if you like, into using coherence relations for deception detection, which is a new, um, way of looking at, uh, deception.

I thought it would be, could be a useful way because coherence it gets at the, the mentor representations [00:07:00] that a speaker or a writer, um, is holding. So, and deception is a, is a, is a cognitive act. So it’s another way of getting into the mind of the potential deceivers. So I thought coherence relations would be good for that.

Um, it, RST is good, but uh, it has a, a, a lot of relations about 30, and I don’t need them all. So I’m working on a, on a, on a stripped down version where we just look at some basics. Like, like if you think about how people talk. It’s almost like basic, you know, logical operators. You have and, and you have Oz and you have nots.

You know, think about your, your, um, your basic search queries, things like that. So, um, mm-hmm. I’m now looking at just very basic. Ways of gelling of text and relating parts of text together. Are they related with and, and relations? Are they related [00:08:00] with, or like alternative or contrast relations? Are they related with not relations?

Um, are they, um, therefore, or because relations, you know, very simple. Mm-hmm. Ones like this.

Zach: One thing I wanted to ask was about the rhetorical structured theory. Was that something that was, had been used before for deception detection or was that, uh, new with you to, to try to use that system

Olu: for that? It didn’t.

It was new of me to, to use that system on naturally occurring. Text on real world texts. Um mm for deception detection. It’s been used for a whole lot of things. In education, for example, it’s been used a lot. It’s been used to compare argumentation in different languages, so it’s been used a lot, but it had not been used for, it had been used once.

For deception detection, um, where people had been invited to make [00:09:00] up true and false stories, you know, that kind of psychological experiment, but it had not been used on real world actual texts. That was an innovation on, on my part. The, the reason why people have not been rushing to use it is because it’s something that computers still cannot be automated.

It relies on.

Zach: You have to parse it and, and analyze it a bit. Yeah.

Olu: Yeah.

Zach: So yeah. One quick aside on that, that rhetorical structure theory, kind of a neat background because it was created for trying to create an algorithm for understanding, uh, speech and making machine speech more natural. Isn’t that how it was created?

Olu: Yes, exactly. Exactly that. And so that machines could communicate themselves.

Zach: Yeah. And then, and then when they, when they were making it for that purpose of, of making machine speech more human, it led to them having to analyze speech in general and then reach this kind of coherence relations, uh, [00:10:00] system for structuring and, and analyzing language, which I thought was kind of interesting.

Like the, the trying to create it for one thing and then that required creating a whole system for analyzing language.

Olu: Yeah. Yeah. It, there’s a bunch of, um. Early computational linguists in the, in the, in the eighties, who thought, okay, well we’ve gotta figure out, in order to make machines speak, we’ve gotta figure out how do humans put words together so that it doesn’t sound like, like gibberish.

And yeah, they did, uh, um, uh, a pretty good job.

Zach: Yeah, it’s the kind of thing where, you know, if you’re, if you haven’t examined it or thought about the rules that govern it, it, you know, we tend to take a lot of things in human life for granted, but then when you look beneath the hood, it’s tremendously, insanely complex.

And yeah, we just intuitively understand all of these things that. Or very complex. So when you did the Amazon review research, we can talk a little bit about that. What were some of the major [00:11:00] pattern indicators you found in those?

Olu: So the, with the Amazon, it was Amazon book reviews. The, the first thing I found is that the real.

Positive reviews, they always contained some caveats. They’ll say how much they loved it in the five star review, but they will try to anticipate what a reader might not like.

Zach: So it would be something like, uh, I, I thought this book was great, but something, something.

Olu: Yes, exactly. So, um, you might not, um, you might not like, um, the violence.

It might be a bit violent for some people, but mm-hmm. Um, it’s a fantastic book and I did a reading, a bit of reading around this, and I found that it’s a convention not just in book reviews, but in, in, um, movie reviews. And it’s a convention in English and in also in Japanese. I talked to some European friends and they [00:12:00] said, oh, in Spanish, it’s not like that.

You just, uh, wax lyrical. You just are really positive and you don’t worry about, um, hedging your positivity. So I think, um, you know, Japanese and English, there’s, there’s a lot of politeness. In those languages. And I think that spills over into conventions around evaluating something. You always try and give both sides.

So it’s not a universal trend, but definitely in, in in English language, you know, you hedge your, your, your praise. But the, the fake reviews. And, um, I did the, this research on, on this data set, which was not put together by me. It was put together by, um, a couple of, uh, researchers. Um, Massimo Pio, [00:13:00] who is a professor at Queen Mary’s, uh, university in London.

Uh, he worked with, um, the investigative research of David Strait Field. Who’s a New York Times journalist who’d done an investigation into fake book review factories sometime in the mid and sometime in 20 12, 20 13. So using is is very, very nicely done because the thing about deception research is finding the lies.

’cause you need to be able to compare known lies with, with unknown truths. And, um, what Professor Pio did, he worked out okay. These people confessed to buying fake reviews. Um, these authors confessed to buying fake reviews. These writers confessed to writing them. So with a small bunch of confessions, he was able to find thousands of rev of reviews.

That were very likely to be fake. [00:14:00] It was police work to put together a hit list of very likely fake reviews, and it’s the best list. That’s available. So I wrote to Dr. Ari, who is actually an Italian police officer, and, uh, professor Perio and I got the data set and I said to them, have you, they were, they were looking at it in terms of machine learning and um, Ingram and this kind of, um, very computational work.

And I said, oh, has anybody looked at it in terms of coherence relations? And they said, no, go ahead. Good. So I did that. And the fake reviews, they were all positive. There was no hedging. That was the first thing I found.

Zach: Mm-hmm.

Olu: Second thing I found was that they were very, um, they, they retold the plot a lot.

This makes it, look, this made them look real. Because, oh, they must have read the story, but they kind of, the information [00:15:00] was what you could glean from press release or from the back of a book.

Zach: Right. No real interesting detail. Like people would honestly have, if they’d read the book and give, you know, feedback or opinion along the way.

It was just kind of a retelling of the Yes. A boring retelling of the story.

Olu: Yes, it was a retelling, there was no opinion, there was no nuance. It was a retelling, like a trailer for a movie that was interesting often, and they were very well written. You know, often written in a, quite a professional way.

Zach: Mm-hmm.

Olu: And I did some tests and I’ve got on my blog, a, a, a version of the test that I, that I did, but I did some tests showing these reviews to some people. Which ones do you think are real? Which ones do you think are fake? And people, naturally, they go for the professionally written one with the retelling of the story.

They go, they go for that. Somebody’s writing, you know, not very well, and they said, oh, I really loved it. You might not like this. And it’s kind of broken English and the [00:16:00] punctuation, like when I think, oh, that must be fake. But, but, um,

Zach: mm-hmm. Mm-hmm.

Olu: It, it wasn’t like that. The, the retelling was fake. The professionality, the high standard of the writing was fake.

Zach: It makes you think of the, uh, yeah. In interrogations they say something similar, right? Because the liars will have really rehearsed their, their stories and the people telling the truth will have a lot of, uh, you know, starts and stops and various diversions when they tell their story. But the liar is just telling what.

Trying to tell what they re rehearsed, you know? Yes. Not, not the same, but it kind of reminded me of that.

Olu: It’s, yeah. It, it, it’s like that. Other things that, that, that, that came out were you would get sub stories or stories about family. One fake, one fake reviewer had a very similar pattern of starting with a.

By relating the purchase of the book to something in her family life.

Zach: Yeah, I saw that one. Is that the one where I thought [00:17:00] it would be the perfect gift to my husband? Yeah. Yeah. That one. Exactly. Yeah. I actually, I’ll just read that now because I thought that was an interesting one. It start, the review goes, the cover of Circle of Lies by Douglas.

Allen really caught my eye first as what I thought would be the perfect gift to my husband. I started reading it myself and could not put it down.

Olu: Yeah, exactly that. That’s one. And the same if you wrote one about, there was some book about bullying and she started off with a, oh, my, my son was bullied at school.

So that’s what drew me to this book. And then she talks about the book,

Zach: right? It’s almost like they’re it self-conscious because they know they’re making it up and I’m, it’s almost like the. Unconsciously wanna prove that they really got this book and they belabor the point too much about how they got the book or why they have the book.

Olu: Yes, that’s a, a really good point. And this is what helps in deception detection because liars try to appear truthful, [00:18:00] deceivers try and prepare truthful, but truth tellers usually are not trying to appear, not trying really hard to appear truthful.

Zach: Mm-hmm.

Olu: Perceived as a, trying harder than truth tellers to appear truthful.

Zach: Right. It’s like the, looking at the, some of the examples there were almost like, like you said, it was almost like a, what they thought a book review should look like based on, you know, going to school. It was really neat and uh, like something they were gonna turn in as a paper almost, you know? Yeah. Which, you know, most, most people writing a real review are not in that mode of constructing it like that.

They’re just writing what they thought as they go. ’cause they don’t care, you know? Yeah,

Olu: that’s absolutely right.

Zach: So one other, uh, interesting pattern in there in your, in your study was about the nuclear discourse unit and the placement of it. Can you talk a little bit about that pattern?

Olu: Yes. So in, um, coherence, uh, analysis, the nuclear discourse unit [00:19:00] is the central part of a text, meaning it’s the part of the text that most nodes.

Or most other parts of a text relate

Zach: to this one. So is it almost like a, uh, kind of a synopsis or like a summary almost kind of,

Olu: yeah. So I could start by saying, um, this book, this book is fantastic. And then I give reasons why it’s fantastic. Then the sentence, this book is fantastic is the central node in that text, because the way that the text is structured is evaluation followed by reasons.

So that’s the structure of a text and you can draw that and the central node will be the initial evaluation. Another, just take a text from a, from a different genre. There’s a, there’s a, there’s a funny, um, genre of stories which, uh, you, you get in [00:20:00] the UK they end with, and then I found five pounds. So it is a really boring story.

Zach: So that’s a, like a joke story you tell you say is a funny Yeah,

Olu: yeah. You know, like, oh, what’s the point of this story? What’s the point of this story? You know, I woke up in the morning and I, and I, and I cleaned my teeth and I had a coffee and I put on my slippers and then I found five pounds.

Zach: Wait, wait, is that, is that like the English version of Cool Story, bro?

Have you ever heard that?

Olu: No, I haven’t. What’s, what’s that?

Zach: People say that when somebody tells a boring story, they say, cool story, bro. Uh, so I kind, I kind of thought maybe, maybe it was like a self-effacing version, uh, of the same idea. It sounds, it,

Olu: yeah. It sounds like, it sounds like that, you know, you use it to rescue your story.

So in, in that case, and then I find five pounds is the central unit, the nuclear discourse unit of that, of that text because, um, all the, uh, other sentences and [00:21:00] phrases are only have a point. In relation to, to that phrase, and then I found five pounds. So Nuclear Discourse unit is the central part of a text.

Um, and what I found with the, with the reviews is, um, that the central part of a text is different for fake reviews than real reviews for fake reviews. It tended to come at the beginning of a text and it tended to be name, the author or the name

Zach: of the book, right? Like, here’s an example from one of them.

It says, this was the first sentence in the, uh, review. It says, Hobe Secret Study Guide Your Key to Exam Success. It. That’s the name of the book. The entire name of the book right there. So they say, yeah, the name of the book is a no-nonsense approach to studying for and taking the Hoed exam.

Olu: Yes.

Zach: So it’s yeah, a [00:22:00] summary right there.

Yeah,

Olu: exactly. So it’s, and it starts off with that. And they name the, the name, they name the author, you know, it’s, it’s almost like you’ve gotta write about who’s paying you.

Zach: And it, and it, it kind of gets back to the, uh, they, they’re almost writing it like a, like they’ve been taught to write papers. It’s like you have to put your summary statement at the top, you know, say what you’re gonna say and then say, you know,

Olu: yeah.

Then say,

Zach: wrap it up at the end, you know, with the summary, you know, and they kind of reminds me of that.

Olu: Yes, it is like that. And, you know, you’ve gotta say. You know, you’ve gotta show that you’ve done a good job for your pay master.

Zach: Mm-hmm. Mm-hmm.

Olu: So you name check, it’s like if you’re going to an interview in a, in a, in a, in a, um, to work for a company and they say, oh, how would you present this company to, you know, prospective to prospective customers?

You would say, oh, okay, such and such company is this, you know, you would write, um, a speech and you would name all the, [00:23:00] the main people in the company and you’d make sure the logo was in there. You know, you’re giving yourself away as somebody who is, who is working for somebody. Right.

Zach: And that would explain the lack of caveats too.

’cause they’d be, even if they thought that was. Potentially a good thing to include for realism. They’d be unlikely to do it ’cause they wouldn’t wanna, you know, hurt perception.

Olu: Yeah. You know, you know, you can’t, you can’t imagine your pay master is saying, well, uh, the, the violence is a bit cheesy, but. You know, the deal author is not gonna be necessarily best impressed.

So, um, and also the, what I found with the, with the caveats was, is backed up by, um, a lot of research done on deception de detection in investigative settings, um, where they found that liars can’t hold. Contrary opinions about something they haven’t experienced. [00:24:00] So a liar cannot talk about the good points and bad points about something that they haven’t done or haven’t experienced that’s been found in, um, in other research and is used in, in investigations.

For example, you would ask them, well, what do you think about x? What do you think about race? Do you think there’s any, any difference between whites and blacks? Do you think, um, what do you think about, about the, uh, difference between whites and blacks? So they say, oh, um, no, I think, um, whites and blacks are equal, so, uh, why would somebody think that whites are better?

A racist would be able to talk about that without um, saying, oh, these are not my thoughts, but people would say whites are better because such and such. Okay. Why would you think that? Um, why would people say that blacks are better? They wouldn’t be able to answer that. [00:25:00] Um, so this is a strategy that’s used in, in investigations to kind of find out if people have strong views or fanatics in a, in a, in a certain area.

A fanatic cannot talk about the other side, the point of view. Somebody that is, um, neutral or less biased can talk about both sides.

Zach: Right. And are they not savvy enough to know that they shouldn’t be talking about those things? Like they just don’t know that they might be giving that information away, I guess?

Olu: Um, yeah. Yeah. Um, and they also might think that they, you know, that they can, you know, they can do it. Yeah. That they, that they can do it. You know, tests have been, have been done on people. They’ve been asked to lie about their attitudes to abortion. For example, there’s a series of experiments done by James Penn Baker in the early two [00:26:00] thousands, um, looking at, um, attitudes to abortion.

If you believe that abortion is not okay, it is hard for you to argue that, um, convincing me that abortion is okay.

Zach: That makes sense. I can imagine them trying to do it and one side would be the side they believe would be very detailed and then the side they didn’t believe would just be very cursory and, um, you know, no details.

That’s exactly it.

Olu: So you can use that, you know, in, in interrogations and, and in my work with, um, for investigators, this is one of the techniques that I introduced to them with the aim of. Enhancing the differences between lies and truthtellers. So there are some strategies for put for making the differences more clear.

The linguistic differences between lies and truthtellers more clear. One of the strategies is to tell the story. Get somebody to tell a story backwards. [00:27:00] Another strategy is to get somebody to give both sides. In both situations, liars, they can’t tell a story backwards, something that they haven’t experienced.

They find that very difficult and, and as as we just discussed, people who don’t really hold a certain opinion, find it hard to argue about it.

Zach: Mm-hmm. Yeah. It makes you think, uh, a better Amazon review format would be like a two columns of, uh, pros and cons or something.

Olu: That’s That’s absolutely right. Do you know the website?

I don’t know if you have it in the us Glassdoor,

Zach: yeah. Yeah. Uhhuh I use, I’ve used that. Yeah.

Olu: Right. So they, they have a template. Pros.

Zach: Yeah, I forgot about that.

Olu: That’s, that is what Amazon should do now. They probably have thought about it and they, maybe it doesn’t make, um, it’s not such a, so attractive, uh,

Zach: yeah, yeah.

I imagine they, they try to keep everything [00:28:00] as, uh, yeah. As light and, and friendly as possible, as opposed to Glassdoor, which is more. In depth, their brand is more in depth and

Olu: yeah. But that’s, that’s exactly what would, um, would really help.

Zach: Hmm.

Olu: You know, if I, if I were to have a meeting with Amazon, that is what I would be, um, recommending.

Absolutely.

Zach: Though, I’d imagine. Would you agree that, do you sometimes feel that the deceivers are just getting better at their jobs? You know, like fake account? Creators, like on Facebook are getting better at their those jobs. And then deceivers like. Oh on Amazon, I’m sure have learned the major tipping points, but then again, maybe it, maybe it doesn’t matter that much to fake review creators, because I don’t think Amazon really polices that stuff very well to begin with.

So maybe that doesn’t matter.

Olu: Yeah, I think it, this, it’s, it’s a bit of an arm’s race. You know, we have this, we do this interview, people listen, they adapt [00:29:00] their strategies, you know? People used to be able to tell scam emails by the low quality of English, by the spelling mistakes and things like that.

Zach: Yeah. Now they’re quite good. Yeah.

Olu: Yeah.

Zach: I’ve, I’ve, I remember them ramping up in quality just over the last few years, where now I’ll see ’em a good one, a good phishing attempt. And I’m, and I’m like, Hey, that’s pretty good. Good job, guys.

Olu: Yeah. Yeah. And comments in, um, newspapers, like the just paid comments, you know, you used to be able to tell.

The, the Russian bots, if you like, from the poor language and the, the incoherence, the non-sequitur that you would find in the comments. But, you know, that’s improved. And people, they integrate themselves into the previous comments very well, and they write very well. And it becomes mm-hmm. Harder to detect.

Zach: Yeah, and, and I don’t know if you know this, but I worked on some analysis of [00:30:00] fake Facebook accounts and I wrote a article that got, gets pretty good traffic about recognizing fake Facebook accounts, all the different, different indicators, ones that are deceptive and operated by, you know, using fake pictures and fake names and, um, yeah.

And while I did that, I was just wondering like, am I just, you know, am I just making it easier? Uh. For these guys, they’re just gonna take that and be like, oh, okay, well we won’t put, we will make sure the URL doesn’t have a, a different name than our display name. You know, that’s a, a new checkbox for them, you know.

Um, but what can you do though? You got, you gotta try to out the lies though.

Olu: I think it’s making it easier, but I think that fundamentally because deceivers have different aims, you will. They will hit a brick wall. They will not be able to be completely, uh, truthful ’cause they have different aims. For example, a deceiver aim is to create false belief.

There are certain language that you [00:31:00] need. I. In order to create belief you need to use in order to create belief, if I just describe, um, something in like an encyclopedia, I’m not creating any belief. In order to create belief, I need to foreground certain information. I need to background other information.

I need to be slightly persuasive. You know, there’s, there’s, it is a different function from, you know, giving directions or explaining a recipe or describing a room. So you need to draw on, on different language and, um. This is the, the main direction of my research. Just kind of identify this grammar of deception, if you like.

Um, and I believe that, that that exists because deception is the basic functions involved, which is, um, the creation of [00:32:00] belief.

Zach: I mean, for, for certain use cases though, like say Amazon reviews, I mean, it seems like. The fake review creator can succeed at his job with, uh, some basic set of skills. Because even if, even if you come in later and say, I think there’s a good chance that’s deceptive, like Amazon or all these services are not gonna re remove these things, if it, if it meets a, a few basic things, like they’re not gonna take the risk of review, removing, uh, something, you know, if it has like a.

You know, even a, a pretty high percentage of being fake because then they’d just be removing a lot of real reviews accidentally, sometimes, and, and, and pissing, pissing people off. So it’s kind of like, for some of these applications anyway, it, it kind of seems like I. Even figuring out a lot of these patterns won’t result in too many fixes.

Would you agree with that? Um,

Olu: yeah, I would, I I, I would agree with that. I think that you’ve gotta, um, to solve it on, on [00:33:00] platforms like Amazon, you’ve gotta look at it at a macro level. So you have to look not at individual reviews, but you have to look at the whole app products. It’s the product discourse, IE the whole set of reviews and consumer communication about a product that is what is corrupt.

So you’ve gotta take down the product. Rather than take down individual reviews and leave other reviews, you’ve gotta think of the whole system, the whole ecosystem. And what’s happening is that fake reviews have contaminated that system. You take the whole, take the whole thing down, and you are able to.

Detect fake reviews. I think more easily if you look at the whole product together, because the balance of reviews and language in the reviews in a sequence of fake reviews is gonna be different to [00:34:00] that of, um, genuine reviews. This is what I found in when I looked at the, um, I found hints of this when I was looking at the Clinton reviews.

Zach: And that was Hillary Clinton’s book, right? That came out about a year ago. Yeah. Yeah. So what did you notice with, with that?

Olu: Okay, so with that, um, I took all the reviews that were available from the launch day for, for one month. What I found was that in the first day. There were like 500 reviews, and then in the next six days there were 500 reviews.

And then in the next three weeks there were 500 reviews. So obviously the rate of reviewing per day rapidly decreased. Now that’s, uh, a trend that. You’ll find to some extent with, um, with other books, but not, um, as marked as that. To go from 500 reviews a day to 20 reviews a day in two weeks suggests some intentional, [00:35:00] organized reviewing.

Then what you are looking for is evidence of an organized campaign. You see, rather than looking at individual reviews for signs, deception, you’re looking at the whole set of reviews for organized campaign.

Zach: Right. So you’d be, you would notice you’d be looking for Yeah. If they all, if a bunch of the ones that came in on the first day all had a similar, some similar aspects, that would be a pretty fishy sign.

Olu: Yeah. Yes, yes. Exactly. And, and if periodically, like every weekend, midnight for the first four weeks, there are a bunch of similar reviews that’s gonna be suspicious.

Zach: Oh, right. So you’re looking at the schedule of it too, right? Yes. That makes sense. Yeah. So, and edit here. I cut out a good portion of the interview here because I realized I was talking about something in a bad, wrong way.

So I’m gonna rephrase the gist of what I was saying in a more accurate way. When it comes to publishers and [00:36:00] authors promoting their books on Amazon, it’s thought that Amazon’s algorithm promotes you based on large numbers in short time periods. So for example, it’s known that getting a lot of sales in a short one day period of time can result in someone being a bestseller in that book category.

And it’s thought that the number of reviews is also important. In a similar way, increasing both sales and reviews is thought to help create a snowball like effect to more and more notice and more and more sales. Having my own books on Amazon and knowing other people who’ve published books, I know it’s a common strategy to give away advanced copies to fans and interested bloggers and to try to strongly encourage reviews on that first couple of days.

I haven’t done this myself because actually I think it’s a bit deceptive, but I know that this is a common practice. In short, reviews are obviously important, but it’s thought that getting a big spike of reviews in that first few days is very important. And then this helps explain so many reviews from those first few days on Amazon.

I. Even though objectively these reviews don’t make much [00:37:00] sense because what kind of normal reader has had time to read the book at that point and review it. And those are just some of my thoughts on the subject. I’m not an Amazon, um, algorithm expert or anything like that, and there’s other people that write about those kinds of things online.

Okay, back to the interview. So, um, with the Hillary Clinton one, what was your final take? I know you, I don’t think you were able to confirm No. Uh, which ones were fake, but you probably had some, some opinions about it, right? Yeah.

Olu: So the reviews in the first, um, few days were very different in style. They were tribute reviews where they were just praising Hillary Clinton.

Zach: No, no caveats.

Olu: No, no, no. Um. Were either stories of, um, you know, the, the experience of the reader with the book. You know, it was such a, a topical and emotive, um, issue that, um, [00:38:00] commentary, you know, comment, people wanting to comment on it. So, um, and I, mm-hmm. It’s, it’s, it’s nonfiction. So the later reviews contain a lot of commentary.

You know, I agree with this. I. Didn’t know about such and such, so it was more like pundits. Um. You know, everybody became an expert. So the later reviews contain a lot of commentary, not, uh, not so much on the quality of the book, but on the events of what happened, you know, so they’re adding to the, to the discourse of, of, of what happened.

So they might not actually be about the book. You know, um, uh, uh, public conversation about, um, the events of, of that election. But the early ones were just tributes. And, um, what I think happened was, um, and I read somewhere, um, something that might corroborate this, maybe the, the Democratic Party or, um, people working for Hillary Clinton.

They have a remaining [00:39:00] list. They send books out. You know? Mm-hmm. A couple of, um, reviews actually said, oh, I received my copy today. You know, a couple of reviews said that. So, you know, is there anything wrong with that? Is that deceptive? You know, people are not gonna have time to read it. They’re going to review it.

Some have said, oh, I haven’t finished it yet, but, you know, some have said that. So is it deceptive if you send, um, books out to, to your fan base? And asked them to review it early because some mean people are writing bad reviews.

Zach: Hmm. Yeah, I mean I definitely think it’s on the very low, low scale of what’s, uh, deceptive and bad.

But yeah, it’s always, you know, for me, ’cause I, I consider doing something like that ’cause I, I easily could have gotten, you know, if I’d done a lot of early mailings of the, of, of my books and tried to get people to review it early, I could have done something like, something like that and have a lot more Amazon reviews.

But yeah, it always just felt a little bit, um. [00:40:00] A bit deceptive to me, just to a, a bit coercive, to ask people to do that for one thing and pressure them. And also the sense that like then, then they’re writing reviews that are a bit pressured and then that’s kind of deceiving other people. So, yeah.

Olu: Yeah. It depends on how we moved.

You can be from it if you could like talk to an agent. You know, you’ve gotta be quite big. If you can talk to an agency. And they organize it for you, and you just pay them a bunch of money,

Zach: then you’re absolved Yeah. Of responsibility.

Olu: You can step away from it, step away from it a bit.

Zach: Right, right. Yeah.

It’s easier to, to use a gun than a, than a knife. Right. So would you ever take a job at Amazon or, or a similar company like that, working on those kinds of issues? Or are you not, not really interested in,

Olu: in that kind of

Zach: thing?

Olu: I was asked this, I have a, I have a, um, a friend working, uh, at Amazon. He’s a computational linguist guy.

Oh. Only got some jobs going, [00:41:00] do you wanna get a job? You know, we were, we were working on some, some research together. And when he got the job at Amazon. We had to stop because basically. Anything, a Amazon review, reviews and anything on their website as their product. We, the public, we, we view, you know, Amazon reviews as, as a, you know, information for the public, but for them it’s, this is their product.

So in that respect, you know, I said to them, oh, I’ve got these, you know, I’ve done this research and stuff. Maybe you can get me a meeting at Amazon. And um, he said, you know, if you come in to Amazon and talk about this stuff, they’ll just take it. So in, in, in that respect, I thought, well, you know, to make a difference, maybe the only way is to work for Amazon.

And you know, the same, the same with Facebook. You know, if you want to figure out content moderation and make a difference, you’ve gotta go and work with them. You can’t do it from the [00:42:00] outside. So I think about it in those terms, but ultimately I like Ed. You know, I think that education. Is the way forward.

You know, I work, um, with, with students and researchers at university on critical thinking, and my approach to it is human deception detection. And um, as you mentioned earlier, I, I work with, um, in a corporate. World we would like fraud investigators and, and and such training. Again, it’s training to raise awareness of, um, how to detect deception using linguistics.

So I’m into training and education and I think I prefer to do that than become a corporate mo. Although I have respect for people who try to do it from the inside, I’m not sure that that’s the most effective way for me, I think education. So, you know, and I, mm-hmm. I put, that’s why I do the blog. I put this information out there.

I want people to read it and figure it out [00:43:00] and, and, and use it themselves.

Zach: Yeah. It’s a great blog. And again, that’s at outlier, O-U-T-L-I-A-R blog. Yeah. Speaking of education, I’ve. It really played around with, in my just brainstorming, uh, an idea for mobilizing people to do one-on-one, kind of talking to people about the fake news they’re sharing, like on Facebook, you know, because I’ve seen, I’m in, I’m in a lot of pro-Trump Facebook groups that where there’s just like, I.

Complete ignorance and fake news shared, and I see people responding in the comments, you know, that believe it. And I’ve actually had a good amount of traction, like explaining to them one-on-one in a polite way, here’s the real story, or here’s how this was skewed. Or you’re believing you’re reading a, a Macedonian fake news site that makes money on clicks and they just wanna rally you up.

You know? So I’ve, I’ve thought about. Ways to, you know, kind of mobilize people as in sort of a nonprofit way or maybe just write a standard for people who wanna help and then go out there and talk to these people on these Facebook groups. Like, [00:44:00] because I really do feel like that has shown traction. Like I’ll have people respond and be like, oh thanks, I didn’t know that.

And, and then I’ll, then I’ll share something else. Like, oh, look at this article about all the fake news on Facebook. You know? So yeah, I just wanted to mention that. ’cause I feel like there, I think there’s a lot of stuff when people. You know, when people are frustrated these days about all the fake news and, and ignorance.

I think, I think there’s a lot you can do one-on-one, you know, and, and if you had, if somebody wanted to devote a little bit of time to that as like a, I wanna spend a few hours a week, uh, talking to people one-on-one in a polite manner on Facebook and just making them more skeptical about the things that they’re seeing in the, on these sites, I think that would be, um, would be valuable.

Olu: I think it’s, it’s really worthwhile and you know, it. It’s like with climate change, not just in terms of the, the denial, but with actually, you know, with, with saving the planet, every individual person thinks that they can’t make a difference, but you can make a difference if you do your recycling. Each individual person can make a difference.

And I think it’s [00:45:00] the same with, um, educating people to critical thinking and deception detection. You know, one conversation and making one person think again. Can make a difference. Absolutely agree.

Zach: Yeah. And then it’s also the, you know, sometimes people feel like they can’t make a difference and, but they forget that they’re, the things they do influence other people.

So you talk to one person and that person talks to other people, you know, maybe you. You talk to somebody online, a stranger, and then they tell their friends like, that’s fake man. You know? Hmm. Uh, I think that people discount how quickly change can happen, I think ’cause they forget about that impact.

Olu: Yes.

Um, I remember the, the, the flat earth thing. Around. Do you remember that a few years ago?

Zach: Oh, I think it’s still, still with us.

Olu: Yeah. Periodically it gets shared around and I remember it happening a few years ago and think, and um, having some friends of mine, you [00:46:00] know, discussing it in a credible way, saying that, you know, well, I don’t know if the earth is flat, but it’s definitely not completely round.

There is a straight line somewhere and. Showing me all these different images. Um, and you know, maybe that’s why the Bermuda Triangle exists, because that bit is straight, you know, and all these Oh yeah, oh boy. And, and, and all this stuff. And, um, I had conversations with them and gave them a different perspective.

And I said, have you thought about how, you know, this comes in waves. And, um, this was actually before the, the Cambridge Analytica thing and the whole Facebook, uh, data collection thing broke. But at the time I thought, you know what, this is a, like a siop. This is a way of, they’re finding out all the believers of all the flat earth believers.

What a great database. Yeah. You know?

Zach: Well, very, very valuable. Yeah.

Olu: You know, because you basically, you’ve got, you’ve got a database of gullible [00:47:00] falls.

Zach: Yeah. I’ve, uh. Yeah, I’ve, I’ve thought about that too. I was saying the, uh, the Trump email marketing list must be one of the most valuable, uh, sub substances in the, in the known universe.

Olu: Yeah.

Zach: Uh, yeah. What makes deception detection important in your opinion?

Olu: I think that right thinking is important, and we as a society need to be able to, uh, think clearly, think critically, and make the right decisions. And deception is like a, a corruption. It’s like a, a lack of quality of information, which it stops us making the right democratic decisions.

It stops us, right? Making the right economic, political, social decisions. And without that quality of information, which veracity gives, [00:48:00] I think that, you know, society can. Can disintegrate. Do we base our whole, um, communication and, and life around the fact that people are supposed to be telling the truth?

Zach: You, you’re talking about the biggest problem, which is, you know, these ways that fake news and fake communications hold us back, you know, as a, as as countries or, you know, on large scale things. But then there’s also all the numerous ways and small ways like Amazon reviews, product reviews, or online deceptions or cheating on tests that all these various small ways also, right.

Olu: I don’t even think that they are that small. They are equally important, you know, um, decision making in the consumer sphere is crucial. The education system, you know, you get people who become lawyers, doctors and engineers who don’t have the actual skills to be able to do that job. And, and you know, it, it happens.

For example, there are, there are doctors who are recommending. [00:49:00] That people don’t vaccinate their children. Mm-hmm. And they are using as evidence bad research. So these doctors themselves are deceptive. They, they often get a, a. A financial advantage from their recommendations. Maybe they have a sideline in Doty health products, which they want to push themselves.

Or maybe they’re just selling, you know, vitamins or something. They, they usually have a, a benefit. They’re using research that has, uh, deceptive foundation either has been funded by people who don’t want, um, who wanna encourage non vaccination. Or, uh, for another reason. And, uh, it leads to we’ve had, uh, we’ve had stories of, um, needing to, or threatening to quarantine groups of people in order to stop, uh, measles [00:50:00] outbreaks.

And this all starts from deception in laboratory or deception in a, in a research institution. And it spreads.

Zach: Yeah, I see so many random ideas that are, you know, bad science, uh, and bad nutrition, kind of news, dietary things like the blood type diet kind of ideas that are, have no basis in science, but seem to be so popular these days.

Kind of reminded me of that.

Olu: Yes. And, and this is, this is deception. And it’s, you know, without sounding like a, like a crazy loon, it does really, um. Attack the, the fabric of, uh, society in the world that we, that we live in. So

Zach: yeah, definitely it seems like a, uh, it’s corroding, uh, things that would otherwise be working pretty well.

Uh, yeah. Any current events recently where you thought something someone said or wrote was pretty clearly deceptive based on, you know, the kinds of [00:51:00] linguistic clues you use?

Olu: Tell you an interesting story. Um, in one of the, um, corporate training, um, workshops I did with Ford investigators, one I did a few years ago, it was in 2015, I did this one and um, Alan Dershowitz’s statement at that time.

I forget the name of the lady, but she made an accusation about, um, Alan Dershowitz and Jerry, um, having sex with her on Jerry Epstein’s plane, or on his island or something like that. And these, um, these accusations were, were quickly thrown out. And I looked at the Ditz State statement back in 2015 and I looked at it as a, as in these, I was looking for examples of something that was true.

You know, I was, I was probably a bit more, um, naive in this, this research then. Um, and I just thought, oh, ditz, he’s like one of the most famous lawyers in the [00:52:00] world. He must have what he said must have been true. So I’m gonna have a look, look at that, um, as a, and use that. I’m trying to pull together things.

I’m gonna use that as example or two. Let me have a look first. And I had a look at a statement and did my analysis. I’m like, oh. Maybe the, this bit here about the plane, about the airplane seems a bit, um, is coming up as, as potentially deceptive what he’s saying about the airplane and, uh, other areas are coming up as potentially deceptive.

And, and in particular, there’s one statement where he says in this, in this statement, oh, um, there were no young women on the island. None at all. We were show shown around by an older woman. And in this statement it just says older woman. And subsequently, now with all the um. You know, Jerry Epstein is in court and, and stuff like that.

I, I had a look at that again. And [00:53:00] the older woman, apparently, according to reports, is actually a very important person. There’s a, a woman who, I forget her last, I think it’s somebody, Maxwell, one of the youngest daughters of the late Robert Maxwell, was like a madam who procured, um, girls for Jerry Epstein.

So. This statement, which, uh, my analysis flagged the airplane and this older woman. I am now looking at it and thinking, oh, actually, you know, Deitz is now under a bit of, um, scrutiny. What I did at the time was I just parked it. I thought, oh, well I’m can’t use this as an example of truth. I don’t think it is true, but I’m not gonna say anything.

So I parked it there. Now I’m looking at it again and thinking, well, you know, you were thinking then maybe there’s something suspicious about him. Now a few people are saying that, um, and that’s happened to me a few times ’cause you have to be careful about accusing people. What I [00:54:00] found with the, with the, with the sports stoking, when I started looking into that, I found this connection between sports stoking and asthma and that all these athletes who are denying.

Four Stiping. They have asthma, which they get, they’re allowed to take medicine for, and the medicine that they take just happens to be performance enhancing, but they get permission.

Zach: Hmm.

Olu: To take them is called a chewy therapeutic use exemption. Hmm. And, um, you know, Brad Bradley Wiggins has asthma, Mora has asthma, and a whole bunch of, uh, um, a athletes at, um, this kind of, uh, institute run by Alberto Salazar who’s this, uh, trainer who was involved in the, the drug campaign.

A lot of them had asthma. My point with that was, um, I have to be, I do look at a few things and I’ve been a bit careful about not blogging accusations, and then a year or two or a few [00:55:00] years later, stuff has come out and I kick myself and I think be more brave. Just say what you say, what you think. And then I think, well, maybe you need like an anonymous, a blog or something where you can just, you know, like these guys on Reddit, you just, they just

Zach: Right.

And then, and then somebody can be, uh, analyzing your, your statements Exactly. And trying to figure out who you are. Exactly.

Olu: Yeah. Sometimes I. You were talking about, oh, maybe go and work for Amazon and I’m saying, oh, education. But another way to like make a difference is to have an anonymous

Zach: soothsayer account.

Yeah. You could become the next Q anon. Like the Q anon for good. Exactly.

Olu: Exactly. You know, make one, why not? Yeah. Yeah. Let’s work together on that. Okay. Don’t put it in the pod. Don’t put that in the podcast. We’ll keep that one, two.

Zach: Okay. Are you being serious now? Should I take it? No. [00:56:00]

Olu: Um, I think we can do both.

I think we can do both. Okay. I think, you know, that’s, um, I think we should, we should think about it. Um, and, you know, the, the, the podcast can be a clue.

Zach: There you go. Yeah. Are we, you know, I, I, I’m mostly kidding, but maybe there’s a little bit of truth, so Yeah. We’ll let them figure it out. Yeah. Okay, that’s it for the interview.

You can get in touch with Ulu via the contact page of a site, which is at outlier blog. That’s O-U-T-L-I-A-R blog, and he’s on Twitter at oe, P-O-P-O-O-L-A. If you like this interview, I think you’d also like an early podcast interview of mine where I talk to Mark ish about statement analysis. That’s one of my favorites of the interviews I’ve done.

If you like poker at all, I’ve got my own books on analyzing verbal behavior in poker. It’s called Verbal Poker Tells, and it’s on Amazon. Or you can get the ebook along with some bonus [00:57:00] materials at my website, reading poker tells.com. And as always, if you like this podcast, I highly appreciate if you leave a rating or review on the platform you listen to.

Much appreciated. Thanks. Bye.

One reply on “Indicators of fake Amazon reviews, with linguistic researcher Olu Popoola”