Vanessa Otero is the creator of a popular and well respected media bias chart that ranks the bias of many news outlets, and she’s the founder of Ad Fontes Media. Topics discussed include: the process her team uses to determine media bias; recognizing that everyone is biased and that the best we can do is try to reduce our bias; the ambiguity in what is “left” and “right” in a polarized, fast-changing political landscape; how recognizing and reducing media bias can help reduce us-vs-them polarization; perceptions of the word ‘misinformation’ being liberal-leaning; liberal-leaning journalists’ pushback to Trump being elected; and more.
Transcript is below.
Episode links:
Resources related to or mentioned in our talk:
- A talk Vanessa did on David French’s podcast, which includes talking about how she got started doing this work, and the receptions to it of people/outlets who were ranked
- The book The Myth of Left and Right
- A talk with Michael Macy about the randomness of political group stance formations
- Ad Fontes announces AI algorithm for ranking bias
- A criticism of liberal-side “polarization is not the problem” stances
TRANSCRIPT
Zach: Welcome to the People Who Read People podcast with me, Zach Elwood. This is a podcast aimed at better understanding other people , and better understanding ourselves. On this podcast I also sometimes focus on the topic of political polarization, because I think that’s the biggest danger humanity faces, and because it is directly related to our difficulties in understanding each other.
You can learn more about this podcast at www.peoplewhoreadpeople.com. If you like the work I do and want to encourage me to spend more time on these things, you can sign up for a premium subscription to this podcast, which gets you a few benefits. If you’re interested in reducing us-vs-them polarization, you might also like to check out my book Defusing American Anger, which is at american-anger.com.
On this episode, I talk to Vanessa Otero, who is the founder of Ad Fontes Media. Vanessa is most well known for creating a chart ranking the bias of various news outlets. This chart has gotten a lot of attention; there’s a good chance you’ve seen it if you’ve spent much time looking at political content online. In 2016, when she created her chart and the methodology behind it, she was working as a patent attorney. Due to the respect she got for that work, she was able to turn that into a full time career; she founded Ad Fontes Media in 2018. As you’ll hear us talk about later in this interview, one of the reasons she started working on analyzing media bias was to combat polarization and our divergent narratives; she sees reducing bias in media, and encouraging people to read less biased news sources, as one of the ways we can combat toxic polarization.
You can learn more about Vanessa and her work at AdFontesMedia.com; that’s AD FONTES.
A big part of our talk is focused on the question: how can we try to reduce bias when all of us are biased? If all of us is biased, as we all clearly are, how can we try to analyze and mitigate bias?
Other topics discussed include:
- What’s the process look like for their analysis of bias?
- How do they categorize something as “left” or “right”, especially considering how turbulent and quickly changing some of the group stances can be?
- How media bias relates to polarization
- The word ‘misinformation’ and why the use of it can be seen as liberal-leaning
- The pushback of liberal-leaning news outlets to Trump, and how that can result in more bias, and more perceptions of bias
- AI approaches to algorithmically detecting bias
If you’d like to hear more background about how Vanessa got into this work, I’d recommend listening to an interview with her on David French’s podcast Advisory Opinions. It was a talk in early September 2023. I’ll include a link to that, and to other resources mentioned in this talk, on the entry for this episode on my site peoplewhoreadpeople.com.
Okay here’s the talk with Vanessa Otero.
Zach Elwood: Hi, Vanessa, thanks for joining me.
Vanessa Otero: Thanks so much for having me, Zach.
Zach: Oh, my pleasure. I realized this would be a big question, and you also go into detail about this on your site, is it possible to give an overview of your approach for determining a news outlet’s bias? What does the general start-to-finish process look like?
Vanessa: Great question. It’s always top of folks’ minds how we actually come up with news ratings for reliability and bias. So, we have a content analysis approach. It’s really all about looking at the content, the rhetoric that’s in there on the page or in the episode when it’s audio or video. We have a big team of analysts. We have almost 60 analysts right now on staff at Ad Fontes Media who are politically balanced, left, right, and center. It’s a really important part of mitigating our own biases. And they go through pretty extensive training on this methodology we’ve developed, where we look at sub-factors that determine overall reliability and sub-factors that determine overall bias. And to come up with an overall news source rating for a big national publication, we will rate a representative sample of those articles.
To date, we usually have several hundred articles from big publications, and each one of them will be looked at by a panel of our analysts; one left, one right, one centre. And they’ll score it on a rubric. We have this grading rubric that goes from top to bottom and from left to right, and it’s a point scale, and they’re looking for various things throughout that. And because they’re looking at a very granular piece of content, they can usually come to a pretty close consensus within a score range right off the bat. They can look at a left-leaning opinion article and say this is a left-leaning opinion article, even if one person’s left and one person’s right. So we’ve done that over the years, over the last four years with over 70,000 individual pieces of content, and that’s how we come up with our overall ratings.
Zach: And what was it that you’d say you brought to the table that was different? Because there have been other attempts to read bias, right? I was just looking at all sides and their attempts to rate bias, and maybe you could talk a little bit about the value you want to bring to that space.
Vanessa: Yeah. The complaints that folks commonly bring when you’re trying to measure something that seems just so subjective is as much objectivity as possible, right? I mean, these are things written by humans and assessed by humans. So to me, we have a unique taxonomy. Rating something from top to bottom on a media bias chart, you’ve got to make decisions on what makes something good. Like, what puts something at the very top? What puts something a little under that? What puts something in the middle? What makes something okay, and what makes it problematic or misleading or inaccurate? What makes something not just centrist or a little left or a little right, but what makes it really far left or right bias? And having really specific definitions for each one of those, that’s one unique thing we brought to the table, and scoring on a very granular article-by-article basis. The primary complaint I would get when I first put out the media bias chart when it was just me was like, “Hey, you’re biased.” And I was like, “Well, how do I mitigate that? How do I… I know we can’t eliminate it.” So, mitigating had to be the most important question and it seemed reasonable to recruit people with different political perspectives and include them as a part of the process. But to be as objective as possible, that’s a different question. It’s not just about the taxonomy, it’s about the methodology for how do you select what’s a representative sample of content? How do you do it fairly across tens of thousands of information sources? Our approach is the only one where we’re trying to replicate it as consistently as possible across all the different formats.
Zach: Did I understand correctly that part of what you were doing differently was the different axes? Like, you rate the accuracy separately from the bias, because theoretically, you can have an outlet be very accurate but be biassed and what they focused on. Am I understanding that correctly?
Vanessa: Right. I mean, left-right perspectives, left-right ideology, left-right moral foundations, there’s nothing inherently wrong with those. There’s nothing inherently wrong with advocating or focusing on things that are of more interest to folks who are more left-leaning or right-leaning, and things can be highly accurate and highly reliable and you still be left or right. When you get really, really biased, there are typically other factors that will bring down reliability. So if you look at the shape of how things are plotted on our chart, there’s a high but imperfect correlation between things that are in the middle and things that are really far left and right, and how reliable those are. One way that you can tell they’re somewhat independent variables is if you look at them just straight down the middle of the chart, you’ll see stuff that’s way at the top. But then there’s some stuff that’s kind of in the middle or at the bottom. So just being middle doesn’t— We’re not saying middle is good. And if you look at especially the skews left and the skews right categories, those columns, you can see stuff that’s way closer to the top and stuff that’s way closer to the bottom. Those can vary even if they have the same actual bias score.
Zach: Yeah. The thing that strikes me there is very much related to polarization and conflict because I think the focus on wrong information or distorted information gets a lot of attention. But people have also written and researched how polarization us-versus-them feelings make us basically filter for true things like real things, but it’s just in the way we filter for them. So you can imagine a far-right outlet that’s just writing only about true things, but they’re all things that are supporting the conservative point of view. It’s just a matter of filtering for those things and leaving out the other things, right?
Vanessa: Exactly.
Zach: I’m understanding that you would factor in that kind of filtering for only one that unilateral filter would be a reason to call something biased if I’m understanding correctly.
Vanessa: Definitely. We liken it to how lawyers argue in court. Lawyers can be very opinionated and very persuasive, and content that we read in our political landscape can be very opinionated. But lawyers can use just the facts that are helpful to their side, right? They’re not required to also include all the other facts that aren’t helpful. But there’s a line, right? A lawyer will get in trouble if they’re leaving out things that are so critical to the story that it totally changes the meaning of things. Right? So folks have a hard time with what that line is. And sometimes folks have a hard time reading a highly reliable piece that it is factual reporting, and yes, it’s just focusing on the stuff that’s more favorable to one side or the other, but they have a hard time seeing it.
Zach: Getting back to what you said about bias is always present, how do I mitigate bias? I really liked the part on your website where you had an FAQ section with the question, “Aren’t you biased? Isn’t Ad Fontes biased?” Maybe you could talk a little bit about what you wrote for that and why you saw that as important to point out.
Vanessa: Yeah, and the short answer to the FAQ Are you biased is, yes. [chuckles] We’re all biassed. We all have our life experiences, and the sample of content that we’ve read in our lifetimes is all different. All the things that you’ve read are very different from all the things that I’ve read. And they’re even more different from folks who you disagree with politically, right? So, we don’t think that being a centrist or being of a particular political lean is the most important requirement to do this work. Because in society, there are certain positions that we ask people to set aside their preconceived notions and just be as fair as possible. Judges and referees, they have their political views, they have their personal views and feelings, but we’re asking them to be fair. We’re not asking them to be neutral, though. And I like to make this distinction because if you’re a judge, if we’re asking a judge to be neutral, you know, there’s two sides in every case. If we asked them to be neutral, at the end of the case, they’d be like, “Okay. Well, you know, you both have a point.” We’re asking them to be fair, we’re asking them to make a call. In view of all the facts presented, make a call. That’s what we’re endeavoring to do here. And mitigation of bias, just trying to have as little bias as possible, is so critical in not just the work that we do, but the work that journalists do. I mean, we give people credit. We give journalists credit for trying because there are outlets out there that don’t try. I mean, they specifically say it like, “This is from a socialist perspective, or we are fighting the leftist narrative or whatever.” Distinguishing between those things is important, I think.
Zach: Yeah, you get bonus points for trying. [chuckles]
Vanessa: Yes, and we definitely try.
Zach: I don’t mean you. I mean, the journalists. Yeah.
Vanessa: The journalists do. And we think we do, right? [crosstalk] I want to emphasize that we try. [chuckles]
Zach: Yeah. I think when point making that point to people that everyone is biased and we can only try to mitigate bias, I think some people mistakenly perceive that as saying truth is relative and that there is no truth. And I’ve seen that confusion before and I try to emphasize that no, it’s not saying truth doesn’t exist. It’s just saying it’s very hard for us to know that we have the truth as individual humans, you know? There’s always these layers and it’s very hard to define the truth or know what the truth is. The truth may exist for XYZ, but whether we know the truth, you know? There’s also even just the element of different values that can result in different truths. Like the trolley problem, right? There’s no answer to the trolley problem because it depends on different biases or values you bring to the table to try to answer these questions about morality or what the best outcomes are. So just to say yeah, the landscape can be very confusing and we all have our biases that we bring to the table, but all we can do is try to reach something that resembles removing ourselves from it.
Vanessa: What I say that we aim for is an objective-as-possible evaluation. I always say as possible. Because when we get comments on a particular news source, it’s always funny to me when people accuse us of being biased. Especially if you see our Facebook comments, they’ll say, “Well, as a longtime listener of NPR, I can assure you that they’re much further left than that.” And then somebody else right underneath that says, “Well, as a longtime listener of NPR, I assure you they are a lot further right than that.” Just looking at that, who has the truth there? If they talk to each other, they’re obviously seeing this very same thing quite differently. So having a little bit of self-awareness that everyone who’s accusing us of being biased is also biassed, let’s recognize that too. We refrain from saying, “Well, so are you.” We just admit that we are. And I tell folks that no one agrees with every individual rating on our chart, including myself because it’s impossible. Because my sample of all the things that I’ve read and listened to and watched is much smaller than the sample and the composite view of our analysts. Nearly a hundred people have participated in our ratings as analysts over the years. A hundred people who are trained, having read 70,000 articles together, came up with this composite view of the truth, objective as possible truth. So from any one person’s vantage point, it’s going to be off from their own.
Zach: That’s like the wisdom of the group kind of thing. If you combine the group’s take as a whole can be better than any one person’s.
Vanessa: Yeah. And I like to distinguish that, though, from opinion polling. Because opinion polling can be not very granular. And that’s been the very common way of measuring media bias and trust. Like, do you trust Fox News? Or do you trust MSNBC? And that typically just tells you… You get a 50/50 answer because of your audience.
Zach: Speaking of the difficulty of removing bias, I’m somebody who for years now has been working on removing bias from my language because I wrote a book aimed at conservatives and liberals, and that just naturally progressed to where I was like I need to really speak in persuasive ways. So I’ve been thinking about this for a while and then I noticed something fairly recently, where I noticed I was more likely to use the word expert for people that I agreed with when I described them. And if I didn’t agree with them, I might choose another word, like a professional or something else. But that was just a case where I was like, “Oh, I didn’t realize that I was doing that, but now it’s completely obvious.” But that’s just getting at even when we try, there can be various ways that our biases creep in.
Vanessa: I noticed the same thing because I speak to audiences that are left, right, and center all the time. So I’m very cognizant of how my words will land. Because if I am perceived as biased, I’d lose credibility.
Zach: Yeah, part of the process is trying to speak in ways that do not seem biased, to show that you’re trying to remove the bias. You’ve talked in your process about your process involving someone from the left, the right, and the center. And one thing that strikes me there is it can be hard to define those labels, especially as the landscape becomes more and more kind of turbulent and chaotic as polarization becomes worse. For example, what is viewed as conservative views could be seen to shift abruptly under Trump as he kind of changed things about more protectionism and more policies that previously had been seen as more liberal and these kinds of things. I’m wondering how you arrive on who represents the conservative view and who represents the middle and who represents the liberal view.
Vanessa: That’s something that we’ve thought a lot about. Because when people first look at the chart and when I was first putting it together– you know, left and right, like in the United States we just have these broad concepts– I actually had the words liberal and conservative on the first version of the chart and realized that left and right are broader. Because liberal and conservative don’t quite capture the whole range. You know, we have so many subdivisions of what people will call themselves and what people call each other on leftist categories, like progressives versus moderates, center-rights, and there’s more pejorative terms like MAGA or socialist or communist or fascist or whatever. But they’re reasonable gradations. So, left and right is meant to capture the broadest. But the more you look at it, the more politically savvy you are, the more you realize that these things do shift over time. So many folks look at the chart and they say, “Oh, well, that’s just the Overton window.” This concept that what’s acceptable in political discourse at especially the fringes changes over time. People talk about folks moving the goalposts. And that certainly does happen.
So what we’re trying to capture is how media publications present things in relation to what the current US left-right spectrum is at that time. It’s US, so country-based, its contemporary now, but what we anchor it on is the policy positions and actions of elected officials. Because how do you measure what’s left and right in the United States right now, like what people think but the citizens think? That’s something you can measure with polling, but there’s a lot of different issues is and there’s a lot of squishiness there. We couldn’t anchor it on what the media is calling is left and right because we’re trying to measure the media. But a thing that you can consistently point to is the policy positions and actions and statements of elected officials. They have power to legislate and influence and they say things on cable television and on social media and on their platforms and their websites, so you can gauge what a particular elected official is saying and you can kind of put them on a spectrum. You and I could identify who are the furthest right and furthest left elected members of Congress who are moderate members of Congress or governors or what have you. And we have an idea of who those people are, and those anchor spots on the chart. So, however the media is talking about political positions relative to those is how we anchor it. And issue by issue, it varies. Some issues are very… They stay similar over time. Like abortion, you know, what’s left and constitutes left and right about that stay similar over time. But on things like immigration, I’d say it moves a little bit quicker when you have fast-moving situations. For example, what’s a centrist right, a moderate right, and a far-right position on whether Trump should have been impeached for January 6th? That’s something that wasn’t a question before but you could figure out what the moderate versus strong versus extreme positions were on that just by looking at the elected officials.
Zach: Yeah, or COVID for example. It’s possible to imagine COVID going a different way if Trump had initially taken really hard stances against COVID and such, sort of like Republicans did with Ebola a few years before. They were the ones more hardcore about fighting Ebola a few years before that and these kinds of things. I interviewed Michael Macy, I don’t know if you know him, but he’s researched opinion cascades and how there’s a chaotic element to some of the things that are new emerging sources of polarization and how they could theoretically go different ways depending on who the early trendsetters are. So I think that’s a really important point to getting to the difficulty of using labels like conservative and Republican or conservative and liberal to describe some of the stance formations basically.
Vanessa: We polarize everything. It’s funny you mentioned COVID. I’m sure you’ve read Ezra Klein’s “Why We’re Polarized” that was pre-COVID. I think an example of things he said that we tend to not polarize was epidemics because they’re so science-based. And it was right when COVID was starting and I was like, “Oh, no.” You could see how that was getting polarized in news coverage. And when it wasn’t on our shores, when it was still just in China or just in Italy before it had really gotten to the United States, the news coverage wasn’t polarized about it. But as soon as it was like it’s going to come here, we decided to take polarized positions about like, “Is it going to come here? Yes or no. Based on my guy. And how bad is it going to be? What should we do about it?” Every single question, because that’s what we do I think better than anyone else right now in the world.
Zach: Well, yeah, Americans are always leading the world. We’re leaders at polarization and we are great at it.
Vanessa: We can polarize literally anything.
Zach: I do sometimes think that it seems relatively unexamined but the idea that America, because in the same way that we’ve spread media and entertainment and our culture to the rest of the world, we’re spreading our polarization to the rest of the world. I think there’s something. But anyway, that’s a topic for another time. Yeah. And to your point, I think of it much in the same way because I’ve seen the idea that the conservative and liberal labels aren’t very good and it’s much more about the Republican-Democrat kind of tribe. And so whatever those political groups do kind of creates the stances associated with those tribes. And I almost went back to in my book because in my book I use liberal and conservative a lot, and I’m thinking about going back and changing it more to Republican and Democrat just to emphasize that point, you know? And I had some caveats at the beginning of how rough the language was anyway but that seems like a better… Or left and right, which is kind of what you were getting at. I think that’s the same concept you were getting at, it’s just a rough approximation of this Republican slash Democrat divide.
Vanessa: Yes. Well, when we look at it in other countries, we can still use left and right. Because we’ve done some work with NGOs in other countries that want to replicate the media bias chart in their countries. And so, you know, in English-speaking Western democracies, the left-right spectrum maps are set fairly closely. There’s obviously intricacies in the local politics. But so many of these other countries don’t have this fixed two-party system that our Constitution has entrenched in the United States. So it’s especially strong here, but when you have parliamentary systems where there’s multiple parties that are along a spectrum, you can still see media follow left and right. So, there are some things about left and rightness that are somewhat universal just like fascism or communism. Folks associate communism with left and fascism with the right, even though they’re both types of authoritarianism. Authoritarianism and populism, but you can’t see lots of populism on the far left and the far right. There’s so many different factors that universally being able to… Most issues, you can map on the left to right spectrum. Which is why we use it.
Zach: Yeah, totally. As a grouping of stances, it can make a lot of sense because a lot of things do slot in there. Yeah, I think it becomes more confusing for the individual stances. It’s when you get into like, “Oh, this is a liberal stance, or this is a conservative stance.” Because some of those things can vary across the world. There’s countries where, as I understand, that the more conservative stances in a lot of countries is more pro-choice, because they’re more for individual liberty so that can actually vary. So there’s these variations like that. It’s tougher for the individual stances, basically.
A quick note here: One book that examines the ambiguity in terms like ‘liberal’ and ‘conservative’ is the book The Myth of Left and Right, written by Verlan and Hyrum Lewis. If you’re curious what people are talking about when they talk about the ambiguity in these terms, that might be a book you’d enjoy. Back to the talk.
Vanessa: Getting back to your question about how we determine who’s left, right, and center, we do a bunch of different evaluations. They’re all self-evaluations, but one of the principal ones we do is have people list about 20 issues. And issue by issue, how strongly left or right are you on LGBT civil rights, K–12 education, taxes, foreign policy which is itself a big one, gun rights or gun control? The reason we do that is because most articles are about one or two political issues at a time. Usually, one article isn’t about everything that you can be conservative or liberal about. It’s like one, two, or maybe three issues. Like, it’s about immigration and maybe it will touch on national security and budget issues, but it’s not usually also going to talk about gun rights and abortion in the same article.
Zach: Let’s see, one second I look at my notes here.
Vanessa: I did very much like your article about Polarization is not the Problem, the critique of that argument. You had some interesting thoughts on the use of the term misinformation.
Zach: Oh, yeah, I was going to ask you about that for sure. What are your thoughts on that word? Because I think other people have written about this, but the perception is that it’s a very liberal-leaning word.
Vanessa: I purposely avoid using the term misinformation when we talk about our work, and disinformation. There are academic circles in which those terms have been formally defined, and that has to do with intent, the difference between mis and disinformation. And there’s other different definitions like malinformation. There’s a taxonomy, there’s like a glossary that some academics will use. But that is not how it’s used by the general public. And you can’t really force your definitions of something onto other people, people are going to use terms how they use the terms. And now people use the term really loosely and flippantly. They use it for information that they disagree with, for information that they just think is biased. It’s just not a precise use in most cases. And if you accuse somebody of misinformation and then that thing turns out to be true, then the word loses power. Folks often talk about how we need a common terminology or a common lexicon of how we talk about these issues of information quality and I sort of disagree. I think our language should evolve. Because talking points, like words that people come up with to define an issue, they tend to lose power over time, especially as they get co-opted by another side. So when a word gets co-opted, when a word loses its power, when the definition of the word is so broad in usage, I try to stop using that word and just say the idea in more precise language.
Zach: Yeah, like something’s wrong. A piece of information is wrong or distorted or whatever. That’s easier to understand.
Vanessa: Yeah, something specific. Like, this sentence is misleading because, or this fact is inaccurate because… So on our chart, even at the bottom, it doesn’t say misinformation or disinformation. It says, contains misleading information, or contains inaccurate or fabricated information.
Zach: Yeah, I avoid that word for that reason, too. There’s just too much misuse of it. And yeah, I think it’s very important to speak in ways that speak to a broad range of people. If our goal is to persuade people of our ideas, it’s up to us to use persuasive language. We can’t just say like, “Oh, well, it’s their fault for not understanding what we’re saying.” I think that’s a lot of people’s default is like, “Well, if they don’t get it, that’s on them.” But to me, it’s very important. For example, if I care about polarization and trying to depolarize and reduce contempt, it’s up to me to make the case for that in persuasive terms, right?
Vanessa: Yeah, exactly. Persuasion is the primary reason I got into this work. I was upset that people would use very biased content from their side to try to persuade people on the other side.
Zach: [chuckles] Exercise in futility there. Yeah.
Vanessa: Exercise in futility. If somebody on the right tries to convince somebody on the left that this is correct because look at this article from Breitbart, the person on the left just totally discounts it. And the person on the right is like, “What? This is a perfectly valid compelling explanation to me.” And then the person on the left will do the same thing with Alternate or HuffPost or something like that, you know?
Zach: Right, speaking different languages.
Vanessa: Yes. And it sounds like you have experienced this a lot. If it’s just not persuasive, I’m not interested in that line or in that talk track. And around the issue of the word misinformation, sometimes people use it to refer to news content that’s in a grey area. So there’s opinion, there’s a section on our chart, and then there’s misleading, but there’s a section in the middle that’s where we score things in the score range of 16 to 24. Our blanket term for that is a problematic area. But there’s all sorts of things that are worse than opinion, but they’re not misleading. They are vilification and dehumanization, or there’s a mismatch between the headline and the article, or there’s some material facts that are not corroborated, or there’s extreme speculation. All of those things, we view them as, “Hey, that’s not ideal in this news content. There’s a bit of a problem with this so-called reporting. It’s not just something you can chalk up to opinion.” But if you call that stuff misinformation, it’s not compelling. It’s not convincing to folks who find it compelling.
Zach: And you’re driving people away and making them more likely to view you as misinformation because they view you as taking a bias stance or a too aggressive stance on that.
Vanessa: Exactly.
Zach: I’ll throw in a note here because I think this is so important. Some people will speak as if political passion is at odds with the work of reducing us-vs-them political animosity. There can be an instinct people have that goes something like, “By telling us to reduce our us-vs-them anger and contempt you are preventing us from working effectively towards our political goals. You are attempting to neuter us and take our power away.” This is a common objection to depolarization-aimed work and it is such a huge misunderstanding of this work. I would say that, in a bit counter-intuitive a way, the work of effective political activism is completely aligned with depolarization-aimed work: by attempting to actually persuade people on the “other side,” to really try to form persuasive, non-insulting arguments to convince the other side, you are also taking a depolarizing, de-escalating approach. The act of trying to persuade people who don’t agree with you requires really trying to understand those people’s beliefs and values, and what their motivations are; it requires removing the biases you have about what they believe; it requires a lot of self-examination and a lot of thinking about how to best make connections. And doing that work naturally aligns with reducing us-vs-them contempt, because you’ll usually find, once you try to understand the quote “other side”, you’ll find the people in that group are not as alien or reprehensible as you previously imagined. You may still strongly disagree with them; you may still find the things they support dangerous and believe they’re doing a lot of harm; but you will have a better view of their humanity and have lowered your contempt. This is what I mean when I say the work of political activism is completely aligned with depolarization endeavors. Okay back to the talk.
I want to ask you about speaking of dehumanization. I actually was just looking at your LinkedIn post you put up about dehumanizing language. Could you talk a little bit about how the more ad hominem and contemptuous language factors in to your attempt to analyze bias?
Vanessa: Yes, it definitely is one of the main reasons that we’ll score something lower than our opinion section. We have this binary framing of news versus opinion, but there are opinions that are better supported and then there are opinions that use things like ad hominem arguments. But what we find so problematic is dehumanization and vilification. And what we mean specifically by dehumanization is using words like somebody’s indicating that somebody’s not a human. You know, they’re subhuman. They’re an animal, they’re a monster, they’re a creature. To me, that makes it harder to break out of cycles of violence. And the reason you see this so much, especially in the current like Israeli and Hamas conflict, folks are calling folks on the other side not humans. When you do that, it creates this layer of abstraction and creates this mental barrier to seeing other people’s humanity. And anytime you have a barrier to seeing people’s humanity, it just makes it easier to convince people to be indifferent to their suffering or be violent towards them. And people don’t often mean it that way. Sometimes people say they’re monsters, they’re animals, because they want to express extreme condemnation. They’re so disgusted with the thing that they did. When people do truly disgusting things, we reach for these words that try to express how outraged we are. But it has this side effect of making other people okay with justifying whatever retaliation or whatever violence in return. I encourage people to avoid doing that. And we score content lower for doing that, just because it’s definitely not convincing to the other side, and it worsens opinions that are supported by actual facts.
Zach: Yeah, and I see that. I talk about that a good amount in my book of trying to get people to see basically their own role in these things, like in the American divide. Millions of people are issuing insults to the other group on social media, and all this condescension and insults and threats and all these things play a role in our divides. We can be very much a part without realizing it of creating the atmosphere that drives our divides because all that animosity and contempt and dehumanization bubbles up. That’s what creates the support for more dehumanizing approaches from leaders and media and such. So I was trying to make the case that it starts with us in a way, that we have to see the power that we have. And often, that’s all that we can really change, right? Like, how much can we really change? But we do have control over how we act and how we speak on social media and see how those things contribute to the environment around us and how that all bubbles up. Anyway, yeah, I think it’s important to see it.
Vanessa: Well, social media dehumanizes us a bit because we don’t see the humanity of the person behind the keyboard. There’s this layer in between us and them and we will say things to them on social media that we wouldn’t say in person. One of the real gifts of the work that we do that I get to experience is we have these unusual environments where every day somebody left, right, and center, it’s like they’re all talking to each other about politics and coming to agreements and they’re face to face on Zoom. Right? That’s rare. And it’s really a privilege to see people working through this and working through disagreements in good faith. Like, assuming the good faith of the other side, really listening, having people show you their perspective and having it convince you, this happens all the time. It’s not perfect. If people do have real disagreements– I tell my analysts the least we can do, like if we want to have peace in the world, I’ve heard this saying, “Let there be peace on earth and let it begin with me.” The least we can do is give our other analysts some grace and that peace and to try to create it between us. Because if we ever expect it to be out in the world, we have to model it, we have to do it, we have to practice it.
Zach: Are you investigating or have you seen any algorithmic approaches that are good for determining bias? For example, one thing I’ve thought about was when I was reading some academic papers that I thought were pretty bad and they were using language like never and always, and I was like, well, maybe you could create an algorithm that scans for all these different words that don’t really have objective and that aren’t very objective, and then you could also throw in words that are associated with bias and see how frequently they occur. Are those things you have explored?
Vanessa: Yeah. Actually, we’ve done quite a bit of work in that space and around using machine learning modeling to score articles. And we’ve actually just recently released a model that works quite well for scoring articles at scale.
Zach: Oh, cool.
Vanessa: Yeah, we’ve been doing this work for quite a long time. So for the last four years, we’ve scored over 70,000 articles by hand with like three people each. So, it’s 200,000 pieces of human labeled data for reliability and bias with a score attached. And we capture the content. So we’ve taken traditional machine learning approaches, plus added our own features of things that we’ve noticed that characterizes these types of content. So actually our machine scoring model scores our articles on average within the standard deviation that our analysts do.
Zach: Wow.
Vanessa: This is a new capability for us, we were really excited about it. For us, it’s so important to blend human ratings with AI. Because humans obviously aren’t scalable enough for the millions of pieces of content out there, but AI is not accurate enough by itself. And these things change, right? And the machine is still a machine, right? If you’re looking at breaking news articles from the last couple of days, we want to make sure that we’re doing the lateral reading and whatnot that goes into determining the reliability on these very very important stories, and keeping tabs like continually training our model. Because the words that are biased, like what constitutes biased words changes over time. Critical race theory wasn’t a word indicating bias in articles 18 months ago. Hard to believe.
Zach: Things can change quickly. Yeah. Which shows the importance of the human element. Yeah. I want to get your take on something that I think is pretty bad. Matt Taibbi talked about it in his book, “Hate Inc.”. He talked about the reaction that many journalists and news outlets had when Trump was elected. They took it as a sign that they basically interpreted it as, “Oh, we weren’t doing our jobs well enough, we were being too soft.” And so they saw it as important to call out Trump and be more aggressive on these kinds of things. That resulted in a lot of bad journalism, for example, around the Trump-Russia stuff. There was a whole host of articles that were just clearly bad and irresponsible and rushed. Another example, I was watching MSNBC the other day– which I seldom watch cable news– but a show’s anchor was slinging insults at Trump calling him the disgraced former president and these kinds of things, and they just seemed like giving up at even attempting, even for the more opinion approaches. You would think that people would attempt more persuasive language that would be more likely to speak to people on the other side, but it seems like the reaction of a lot of journalists was that, “No, we have to get really tough and make our stances more well known, basically, and more obvious.” Which I think, to me, was a pretty bad misstep because I think they didn’t really see that so many conservatives already thought that there was an extreme liberal bias in the media. Which sometimes was more subtle and not as obvious to people, but it manifested as condescension about conservative points of view and often just an ignoring of the more rational aspects of conservative side views. So to me, this reaction by journalists was really a big part of the problem in exacerbating the polarization problem. Which ties back into my thoughts on polarization that our instincts on how to fix the problem that we perceive are often just driving us deeper into the problem. So I’m curious what your take on all of that is.
Vanessa: I totally agree. I don’t know if you recall during the run-up to the 2016 election that HuffPost made the editorial decision that at the end of every article on Donald Trump, they’d put an editor’s note like Donald Trump is a racist, xenophobic flop blah, blah, blah, who cheats on his taxes and whatever. Like, it was a note.
Zach: Wow, I didn’t know that. [chuckles]
Vanessa: Yeah, they’re like, “This is an editorial decision we’re making to do this hard calling out of Donald Trump, like this pushing back and this fighting back.” And you hear that a lot in really polarized camps. The problem is you got to call the other side out. But in how many cases does calling the other side out, specifically calling people wrong to their face as the beginning of your argument, how often does that result in them being like, “Oh, yeah, you’re totally right. I am stupid and wrong. And you have shown me the error of your ways.” That’s just not how persuasion works. And people are doubling down. It feels good, it feeds the confirmation bias. And I think you can see within cable news shows and very left-leaning or very right-leaning publications, people just sort of buy their own BS. They’re talking to each other, they’re saying the same talking points, they’re in violent agreement with each other. And the folks that they would want to convince in order to advance a policy, which are people they disagree with, are not even watching. There’s no way they would watch. If they did, they’d just dismiss it completely. So I think it was a big mistake. Especially when you’ve got outlets that do have wider audiences, like folks from the left and right read them. Well-known publications with large readership bases, you know, CNN, Washington Post, New York Times, to the extent that sections of their content in each of those outlets, they’ve got opinion sections. For CNN, like the TV and evening divisions, that really double down on this pushback kind of content, this opinion stuff. And that just erodes trust of anybody that they’re trying to convince. The presence of opinion generally in reputable publications erodes trust. The format of it. The primary reason local news is so trusted is because it doesn’t have an opinion section. Like the evening show, it’s like 13 stories, took them all day to gather it, and they deliver it to you in half an hour. Some of them are about sports and weather. They don’t have time to have four pundits on telling you what they think about the horse race stuff in Washington.
Zach: Regarding the opinion versus the straight news, I think a lot of people just don’t really make the distinction between those. I’ve heard people complain about Fox News shows and they’re like, “That news is just completely false,” and I’m like, “Well, a lot of that stuff is just their opinions.” Which as much as we can disagree with it, it’s like they aren’t even attempting. They do a better job on the shows that are trying an actual… It still might be biased, but they’re doing a better job at reporting actual news when they’re attempting to do it. But just the perception on both sides across the board is that most people just turn on these stations and they just see these opinions being lobbed and they’re like, “That’s their version of the news.” When it’s not, it’s just the fact that there’s so many opinions shows.
Vanessa: We tend to see the other side’s news through a very small aperture or very small filter. We usually just see an example of the other side’s news, it’s a really terrible filter through somebody that’s on our side that we like. And we’re like, “Ah, I know exactly what people are watching over on Fox or over on MSNBC.” But if you actually watch all the content every day, we go on our interactive media bias chart and just look up Fox shows. We have dozens of Fox shows. And some of them are top middle, and some of them are bottom right on the chart, and a bunch of them are in between that. So there’s this huge variation and people don’t make those distinctions. If somebody on the left says, “I can’t believe people watch Fox News,” people who watch Fox News and are familiar with it’s reputable content are like, “What are you talking about?” Same thing with MSNBC, right? It’s the same thing.
Zach: Right. There’s all these systemic and emotional reasons why we focus on the most extreme or weird stuff on the other side. Yeah.
Vanessa: Of course.
Zach: Do you see part of your work or part of your motivations as helping with the polarization issue?
Vanessa: Oh, a hundred percent. I mean, that’s sort of our reason for being. The reason we exist is to bring people together. The reason I started this company is because if we are this divided on the facts that we share and we can’t talk to each other about politics, we can’t solve our major problems. How can we hope to address some things that are as difficult as immigration when there aren’t really a lot of good answers when we’re just so far apart on what the facts are on the ground? The fact that people would lose friends and family members over discussions about politics was disturbing to me. The fact that people can’t talk about politics in their workplaces, these all damaged interpersonal relationships and our ability to function as families, as companies, and as a country. So I think that it’s fundamental to what we do is to reduce polarization.
Zach: Is there anything else you’d like to mention about what you’re working on before we sign off?
Vanessa: People often are aware that we put out this media bias chart because they see the infographics that circulate frequently on social media, but it really encourages folks to come to our website and explore and see what we do for the various stakeholders in the media ecosystem. Because how we’re trying to affect society and positively transform it and reduce polarization is bringing in all the stakeholders to the media ecosystem. The stakeholders are individual citizens, they’re educators that are teaching media literacy, they’re the brands that fund journalism, they’re the publishers themselves, the social media companies that distribute them. Everyone has a role to play. And so we designed our company and what we do is we rate the news, we do all these ratings. But we provide tools for each of those kinds of stakeholders like media literacy education for high school and college and your data sets and APIs for researchers and your commercial enterprises. So I just encourage people to learn more about what we do.
Zach: Awesome. Thanks a lot, Vanessa. Thanks for joining me and thanks for your work.
Vanessa: Thank you, my pleasure. Thanks for having me.
Zach: That was a talk with Vanessa Otero, founder of Ad Fontes Media, which you can learn more about at adfontesmedia.com.
That was a talk with Vanessa Otero, founder of Ad Fontes Media, which you can learn more about at adfontesmedia.com.
If you want to learn more about political polarization and why it’s such a big problem, and what we can all do about it, consider signing up for my depolarization-aimed substack, or getting my book Defusing American Anger. You can learn more about that work at my site american-anger.com.
This has been the People Who Read People podcast, with me, Zach Elwood.
Thanks for listening.
Music by Small Skies.