SIO327: Does Paying Fox News Viewers to Watch CNN Pull Them Back Into Reality?

Transcript

This transcript was generated automatically and may contain errors — the audio is the record.

0:00:11 Thomas: Hello and welcome to Serious Inquiries Only. This is episode 327. I'm Thomas and that's Dr. Lindsey Osterman. How you doing?

0:00:19 Lindsey: How are you?

0:00:20 Thomas: I'm good. I can't wait to find out that everything's fixed. It's all solved. We're good to go. Everybody's been talking about this study and the coverage. They've been sharing it. You pay Fox News viewers to watch CNN, of all things, and It's cured. They're just, it's all better, it seems to be the indication. No, or at least it seems to help. And definitely we thought we'd put this study through the Lindsay treatment and see if we're saved. Oh,

0:00:25 Lindsey: Mm -hmm. Mm -hm. Yes. Yes!

0:00:51 Lindsey: Yeah. This paper, it's a manuscript by the way. Some of the outlets have been reporting this as though it's actually published, but it's not. Like, currently undergoing peer review, so this is still manuscript. Uh,

0:00:55 Thomas: Why? Did they jump the gun on that? Just wanted to share the good news. Okay.

0:01:07 Lindsey: Yeah, I mean, it's not, it's not totally uncommon for authors to share pre, preprints of their work, and so they, this was a study that they had, um, -registered in terms of the analyses and design and things, and so they uploaded the preprint to an open science website before it actually gets published, but, um, don't think it's entirely uncommon to do that. Yeah.

0:01:12 Thomas: Okay. Really?

0:01:27 Lindsey: So this manuscript is by David Brookman and Joshua Kala, and it's called "The Manifold Effects of Partisan Media on Viewers' Beliefs and Attitudes, a Field Experiment with Fox News Viewers." So to start us off, at the core of Brookman & Kala's interest in this study is the large real-world impact of partisan media. So news outlets can be more or less partisan. Some of them at least try to balance partisanship a little bit, whereas others, like Fox News, for instance, and MSNBC, right, clearly presenting news in a way that is intended to be flattering to one side and unflattering to another. And there's a large body of research in the area of media studies that demonstrates that partisan media actually do have strong effects on voting behavior. And so they have important effects on elections. But interestingly, and this is where the current study is going to situate itself, empirical work on the mechanisms by which these outlets affect attitudes and behavior is not as thoroughly studied. So there was a quote in the article where they said something like, it's very well documented, but poorly understood, or something to that effect.

0:02:28 Thomas: It's not just as simple as expose people to this constant fake news, or or whatever, constant slanted coverage, and they just start to believe those things. Oh,

0:02:36 Lindsey: I mean, that's one of the mechanisms that they're going to investigate. Yeah. So maybe it is that simple. Just that empirically, those things haven't been documented. So there are, yeah, at least not as as thoroughly as the effect itself, descriptively. So their study is designed to investigate two mechanisms that have been put forward as explanations for how partisan media might influence attitudes and behaviors of viewers. And so the first potential mechanism is essentially what you just described. It's called partisan coverage filtering, and it describes selective reporting, essentially.

0:02:43 Thomas: Oh, Okay. Mm. -Hmm.

0:03:09 Lindsey: So reporting some topics and deliberately excluding others, according to which stories cast the favorite, favorite party in the most flattering light. Um, the thing that jumped to mind immediately for me was, uh, few weeks ago I was following a story about the Missouri House Bill 28 28 10 have one of the many anti -abortion bills that they're passing. Um, this one in particular was a little disturbing, disturbing because, you, from the horrible thing of making, you, to do with abortion a felony, essentially, a class B felony, it also had a provision specifically to make it a class A felony if the abortion was performed or induced on a woman who has an ectopic pregnancy. That's obviously ridiculous and evil, right? That's just straight up murdering a person that is pregnant. Ectotic pregnancies, like, no good, good outcome to that, none of those are viable.

0:03:09 Thomas: Uh, Right, huh.

0:03:59 Thomas: What?

0:03:59 Lindsey: The only thing it does

0:04:00 Thomas: I didn't I didn 'y even see this, this is, that's crazy.

0:04:01 Lindsey: Okay. So this is interesting, um, yeah, so this was awful, it was like written into the bill, and I, I think that at least some, some people have said that maybe this is just included so that they can take it out later and then claim that the law is now fixed, right, long as they take out the ectopic pregnancy part, then it's fine, even though they're completely banning abortions. I don't know, but my, my point of bringing this up is that I was, you know, finding stories about this bill on essentially exclusively left wing outlets and some local media outlets in Missouri. And I was super curious to see how a right wing outlet would be spinning that story. So I went specifically to Fox News, and I searched a variety of combination of search terms to try to bring this story up, like, you know, Missouri abortion and Missouri with the house bill number, and, you Missouri ectopic pregnancy and all these things.

0:04:03 Thomas: Hmm.

0:04:51 Thomas: They Just don't even cover it. Yeah.

0:04:53 Lindsey: Zero stories. And I searched on Breitbart and The Blaze and like a bunch of other things, zero, exactly zero articles, which, yes.

0:04:53 Thomas: Yeah. The common, which is interesting because, because, you know, the common meme about this is whenever anything bad was happening, you know, to Trump or whatever, it was like, you show all the news networks and what they're covering, and then Fox News would be like, a hippo escapes, you know, it'd just be totally unrelated. But I guess you wouldn't necessarily expect that with an abortion bill. But maybe because this one's so obviously, like, going to be deeply unpopular, if people get the details, maybe it's just like, nah, don't even bother with it.

0:05:16 Lindsey: Yeah. Yeah. I think that that must be the case here. Yeah, so exactly the kind of thing that's being described when Brookman and Kala describe partisan news filtering, know, leaving out things that would be unflattering, only focusing on things that would be flattering. And also counted as partisan coverage, be choices to report on certain details of a story and not others in order to cast their side in a better light. So in addition to some stories just being entirely omitted, some stories might be covered and just include entirely different sets of details. So for instance, as we'll see in much finer detail later on, when more progressive outlets covered COVID-19 during 2020, they focused on things like the deadliness of the virus, rising death tolls, you know, stories about people dying from the virus. And outlets like Fox News almost entirely left this out of their reporting and instead focused on downplaying the severity of the disease and playing up Trump's, you know, quote unquote, competent handling of the pandemic response. So they were both covering COVID-19, but just talking about completely different things, essentially.

0:06:33 Thomas: That sounds right.

0:06:34 Lindsey: It does. Right. So that's partisan coverage filtering, the selective reporting to flatter one side over the other. And the reason that the authors think that partisan coverage filter may be a uniquely important part of how partisan media coverage impacts voting behavior is that obviously, it changes the information to which people have access. If folks only watch Fox News, they might have no idea, for instance, that Trump privately admitted very early on that he knew the virus was more deadly than the flu and then said the exact opposite thing to his supporters. And so this is obviously a problem. Maybe that doesn't bear repeating.

0:06:59 Thomas: Right. Yeah.

0:07:11 Lindsey: I don't know.

0:07:11 Thomas: And then other stuff like just reporting on, you know, violence in Chicago or whatever, and then conservatives think it's like a, you type hellscape that's just all out killing and death at all times.

0:07:24 Lindsey: Yeah. Okay. All right. So that actually is a beautiful segue. People should know you don't have my notes. A friend of mine started listening and he was like, yeah, this is a really good bet you guys do or Thomas doesn't have your notes, but you really don't.

0:07:35 Thomas: I can't. I don't read notes. Sorry.

0:07:37 Lindsey: No, I don't read either. It's boring. But yeah, so that is a Really Nice segue though. So the second mechanism they talk about here is called agenda setting, which describes when partisan media outlets modulate the amount of coverage of particular topics in order to manipulate viewers' perceived importance of that topic. And it also has the consequence of priming viewers' pre -existing attitudes on topics when they evaluate politicians. The, you relentlessly covering violence as an example of this, people get the impression that it's much more common than it actually is, and that this is a much more important issue than it Actually is a repressing issue. You, exactly.

0:08:13 Thomas: And I'm curious because the The baseline of just news is already going to be skewed because stuff that's news, you know, dog bites man kind of thing, like is going to be that kind of violent or whatever attention grabbing thing. So already, even if you had a quote unquote actual balanced news source, it's still going to skew a certain way, isn't it?

0:08:21 Lindsey: Oh. Yeah.

0:08:36 Lindsey: No, I think that's true. Um, the question is whether it's skewing specifically in the direction of trying to flatter a particular political party.

0:08:42 Thomas: Right, right. No, it'll just make you more scared of everything, which could still lead you to, be you more conservative because conservatives are terrified of everything. Thing I

0:08:42 Lindsey: Yeah. Yeah. Sure. Yeah. Yeah, that's, um, yeah, I mean the, another example that came to mind in terms of the agenda setting was that the weird moment when all Fox News was covering for days was the coverage of how Dr. Dr. Seuss was canceled. And this, oh, yeah, right. Yeah, same thing, right?

0:09:03 Thomas: Thought you're gonna say the caravan but same thing, yeah, that Dr. Seuss caravan that was coming to our borders bringing books with odd rhymes, no, different, different issue, yeah.

0:09:13 Lindsey: Right. Yeah. Right. And so that's obviously going to inflate the impression about how, how critical, how pressing, how important that topic is, and how people should feel about it. Another interesting point that they brought up about agenda setting was that it also has the consequence of causing that topic to be much more salient when people are evaluating candidates who either align with that preferred partisan side or not. And the first thing that came to mind for me was the effectiveness of agenda setting with regard to race and CRT in schools in terms of getting my current governor elected. Was tons of coverage of this stuff, like relentless coverage of it. Youngkin definitely used that to his advantage in his campaign. And that seemed to be persuasive, not just to folks that were already on the right, but to a lot of folks who described themselves as moderates, liberals and independents. So I think that this kind of agenda setting piece is also very important. And the authors agree with that.

So there's one other mechanism called framing, which is not really part of their study. So I won't talk too much about it. But framing is essentially describing the fact that two sides will talk about an issue using different language. So thinking about describing something as deaths of unarmed civilians versus collateral damage, or undocumented immigrants versus illegal aliens.

0:09:44 Thomas: Right Well.

0:10:26 Lindsey: And those kinds of choices in terms of framing the message change what is salient to viewers about the topic that's being reported. So that's less of a focus of their study. So, you know, we'll leave that to the side.

So they investigate the role of two of these three mechanisms, so agenda setting and partisan filterage covering, in how partisan media outlets shape viewers' attitudes by doing a field experiment in which they paid people who at baseline were very, very heavy Fox News viewers to watch CNN for a month and then seeing whether it changed their beliefs and attitudes. And so I do think that this is a very intriguing study, created a very rich data set, and it has potentially very important implications.

Because if they get any movement in beliefs, it would contradict, I think, a very common fatalistic belief that it's essentially pointless to try to persuade anyone, and especially someone who's hyper -partisan, because they'll just reject any contradicting information out of hand, right? So you're better off trying to convince people who either are not very engaged or more middle of the road. And it's not it's just not worth talking to people who already

0:11:28 Thomas: Yeah.

0:11:28 Lindsey: Hold strong attitudes.

0:11:30 Thomas: But I mean the whole thing of this experiment, I thought, was that you actually make them start absorbing stuff from a different angle, which is much different than like hey, let's have a conversation and I'll try to persuade you. Like, I think it requires this, um, constant, consistent, um, over time, I think. Right.

0:11:50 Lindsey: I think that I thinkthat's fair. And, you know, when we get when we gets to the end of the results, Ithink that the take home there is exactly that you need to get people away from the constant exposure to Fox News. I think thats true. But I think that there are implications that this has for the attitudes and beliefs that people hold. I take your point that it's not exactly the same as like me sitting down with someone and trying to persuade them on, you know, some sort of general economic issue or something. But I do think that if you can chip away at the credibility of a news source that is pushing a particular set of values to the point where somebody actually considers that to be an unreliable source of information, that that could be an avenue to eventual attitude change.

0:12:01 Thomas: Hmm.

0:12:36 Thomas: Right, I guess the only distinction I was making is it sounded like, it sounded like you were saying, oh, this, this will tell you whether or not we can persuade people. And I, I just was thinking, um don't know that the average person has this ability to do whatever. Yeah. I don't know what the experiment was, but my indication from all this is that they had to watch, as you said, CNN for a period of time. And I don'T know that you or I can make someone watch CNN for a period or do the equivalent thing ourselves.

0:13:02 Lindsey: Oh, I see. Okay. Let's come back to that point. Because I think that like what they think is actually happening as a result of having people watch CNN is something that maybe you could replicate. Maybe. Maybe. See where we are by the end of this, you won't agree. I don't know, I think maybe I'm more optimistic about this stuff than some people. But so let's talk about the details of their study. So this was a pre -registered experiment in which which they randomly assigned some Fox News viewers to watch CNN for about an hour per day, um, think pretty much every day. There was some variability in terms of how much people agreed to watch, um, they, they watched a lot of CNN, right, so it was most, most days of the week, um, they did this for almost the entirety of September 2020, is when this, this treatment period was taking place, um.

0:13:06 Thomas: Okay. Okay, I love it.

0:13:47 Lindsey: They were either randomly assigned, huh.

0:13:47 Thomas: That's interesting because that's like, you leading up to the election, right?

0:13:48 Lindsey: Yeah, Yeah. Yeah, so there's a, there's a lot of stuff that they're tapping into in the questions that, uh, yeah, will take us back to exactly what was happening during that time, I know, yeah, it was good times. Um, so they, they assigned people either to be watching CNN during September 2020, or the control group just watched TV as they normally would, and then filled out the same surveys as the people in the treatment group. Um, the sample selection is pretty interesting for this study. They were, we're looking for a sample that watched a lot of Fox News and not much CNN. And apparently, it's actually pretty difficult to get access to a population of folks who frequently watch partisan media.

0:13:51 Thomas: Yeah. Oh, fun.

0:14:26 Thomas: Yeah,

0:14:27 Lindsey: Yeah, so like, usually what I would do, if I was going to recruit a sample like this that was specialized, is I would be like, I would go to Fox News, and I would do like a short screen or survey in which I asked people something like, how many Fox, how many hours of Fox do they watch each week? And then those who answer above a certain threshold old would qualify for my study. But unfortunately, previous media research suggests that this would not be a good strategy because partisans vastly overestimate how much partisan media they consume.

0:14:53 Thomas: Overestimate?

0:14:53 Lindsey: Yeah. Yeah. Certainly unintentionally. But yeah, I read part of the paper that they cited for this effect and it's pretty interesting. It looks like self -reports are not a great way to assess how much Fox News somebody is actually viewing.

0:15:05 Thomas: Oh, because I thought you were going to go down the avenue of the same thing with polling where it's like, well, they won't be honest with you about their views or that, but it sounds like it's the opposite. Like they're like, no, I'm definitely a Fox News viewer. And then they don't watch as much as they think. Huh?

0:15:14 Lindsey: Yeah, yeah, Yeah. Right. Right. Interesting. I mean, people don't have perfect insight into this stuff. And it might be the case that you can get like an ordinal sense that if somebody says that they watch more Fox News than anybody else in your sample, they probably do watch more Fox News and anybody else on your sample. But it's important here that they get access to a group of people that actually watch like a certain minimum absolute amount of Fox News, because they're trying to replace that with CNN viewing. Yeah.

0:15:34 Thomas: Mm hmm. Hmm. Right.

0:15:48 Lindsey: So given that they need that, right, they can't rely on the sort of like pre-screener thing. Another strategy would be to just recruit like a representative sample of the U.S. and then look at the amount of partisan media viewing as a factor instead of just targeting people who only use it a lot. So in other words, I could compare the effects of heavy Fox News viewers in my sample to people who aren't heavy Fox News viewers. But but unfortunately, if you're really interested in that one group that's at the extreme tail, that strategy would require you to obtain like a massive sample of people in order to get enough people in that comparison group. So apparently the proportion of folks who watch a lot of Fox News is really small, like smaller than I would have thought. So one estimate suggests that only about 10 % of U.S. adults watch Fox News for an hour or more during a two week period.

0:16:19 Thomas: Hmm, right. Really?

0:16:37 Lindsey: Yeah, that is lower than I thought.

0:16:39 Thomas: What the hell?

0:16:40 Lindsey: I don't know.

0:16:40 Thomas: Where are all their viewer numbers coming from? I mean, an hour is like one one episode of Hannity, right? Or whatever that other Tucker Carlson.

0:16:43 Lindsey: I don't know. Yeah. Yeah, so the the population of the US is what is like 350 million. Is that right? So

0:16:54 Thomas: Yeah. And we're looking at like adults, right.

0:16:55 Lindsey: Yeah, okay. So I mean, it's still like 10s of millions of viewers if it's 10%. I So you can imagine that if you want to recruit a sample that watches Fox News, like daily, then you're looking at a very small percentage of a general population sample, so they can't afford to do that. So here's the creepy way that they managed to identify their sample of heavy users. They made use of data from a media analytics company. So it will surprise no one to learn that for users who opt in, many smart TVs collect data on which programs are viewed on that TV and then report that back to the manufacturer. And then the manufacturer sells that data to marketers. And so the TV's IP address can then be matched to other databases like voter VoterFiles. So neat.

0:17:02 Thomas: Yeah, true. Yeah wow. Voter

0:17:43 Lindsey: Yeah. Voter Files.

0:17:43 Thomas: Files. You have the IP address of my, of the TV can be matched to a voter file.

0:17:47 Lindsey: I, I guess.

0:17:50 Thomas: Yeah I usually bring my TV into vote and then just like, Like, here, let me grab that IP address.

0:17:55 Lindsey: Yeah, I genuinely do not know how this worked. But they they got this data from from one of these marketing companies. And then they match this to Voterfiles in order to get addresses to send their their study recruitment materials to Yeah. That's creepy. Probably not the worst thing that's being done with their data, but it's pretty bad.

0:18:08 Thomas: Hmm. Right. Yeah.

0:18:13 Lindsey: So they got data from one particular smart TV brand, and they identified nearly a quarter of a million registered US voters who regularly watched Fox News. So these folks watched Fox news between eight and 240 hours a month, which It 240 hours works out to eight hours per day. So I'm guessing this is people who just like, have it on in the background, is kind of like a yeah

0:18:30 Thomas: God. I was going to say that's like when Trump at all the government buildings mandated that it be Fox News all the time. Is this going to be like a lot of Department of Defense officials or something? You're watching Fox News, you know, 24 hours a day, seven days a week.

0:18:45 Lindsey: Yeah. Yeah. Exactly, it must be, I don't know, um, yeah. So, so about a quarter million people, they sent their initial recruitment material letters to this quarter million voters and invited them to participate in a paid online survey and included some questions about relevant demographics and self-reports, their TV viewing, and some specific questions about how frequently they watched Fox News in particular. And then in a series of stages, they winnowed down this quarter of a million viewers to a final sample of 763 individuals from 695 unique households who reported watching Fox News regularly, were willing to participate in a study in which they got paid to watch TV, like a pretty good rate too. It's like $15 an hour to watch TV, and then $10 each for the surveys they had them do. So it's pretty good. So yeah, we're willing.

0:19:39 Thomas: I don't think you could pay me $15 to watch an hour of CNN. I'm just saying.

0:19:44 Lindsey: Well, these folks were willing. Yeah, so they were willing to be in the study. And they committed to watching CNN during specific times during the regular week. And all of the times options that they gave them that they could choose were during prime time. And they chose these times specifically because they figured that there would be more compliance during evening hours. But also, the programming during primetime is the most likely to engage in partisan coverage filtering compared to the daytime shows. So their final, right, that

0:20:12 Thomas: Oh. Oh. So, Like the Fox News that they're, they're not watching would have been more partisan, right? Or, are You saying also the CNN coverage would be at

0:20:23 Lindsey: Is their argument that both the Fox News that they're not watching and the CNN that they are watching, like that those have the biggest differences in terms of this, these sort of like partisan filtering processes.

0:20:34 Thomas: That time, right?

0:20:35 Lindsey: Yeah, during primetime. Yeah.

0:20:36 Thomas: So it's just like when all those, you know, dumpster fire white supremacy shows like Laura Ingram and all those ones, it's like you're trying to pick that time allotment so that the, the 763 worst people in the country that they, they got found for this survey are, are not watching those things instead.

0:20:52 Lindsey: Jesus. Right.

0:20:55 Thomas: Yeah. Okay. But, um, I was curious, the only thing I was wondering about this part was if there was like variability and when they had them watch CNN, cause I, I'm not super familiar with CNN, but I imagine like most cable networks, there's like, you know, there'll be like, okay, well, there's a bit of a right leaning show. And then there's been a bit of a left leaning show and I'm just wondering like, if you're always watching it the same same hour and it happened to be the same show or if they're like varying kind of what they're watching.

0:20:56 Lindsey: Yeah. Yeah. And they, they, so different people watched at different times. Like they gave them, you know, a certain chunk of time that they could choose slots out of, but like the particular days that people were watching and the particular times that people are watching varied. So they should have coverage across that different programming. Yeah. And so one, one thing that I think is important to note is that they didn't specifically instruct them not to watch Fox News. So people absolutely could have, yeah, or they could have like, you know, T -voted and watched it later if they wanted to. They But did collect information on, you Know, their viewing patterns. And it did seem like people watch less Fox News during the observation period. So I just thought that was worth mentioning.

0:21:30 Thomas: Gotcha.

0:21:47 Thomas: People with a two TV setup were, yeah, yeah.

0:22:03 Thomas: Yeah.

0:22:04 Lindsey: So this final sample that they ended up with was actually, surprisingly, surprisingly representative, representative of the starting sample of that quarter million in most respects, except that it was even more conservative. And these people were reported watching even more Fox News than the original sample, which was sort of, you know, it was chosen on that basis. So to pick, so to paint a picture relative to the average American, this sample was on average older with a mean age of 54 years, whiter, 95 % white, more Republican, 92 % Republican, and more active voters than the general population.

0:22:37 Lindsey: So 90 % of the sample voted in the most recent presidential election. Yeah. Election, I should say, right? Recent at that point. And obviously they were more frequent Fox News viewers than that starting sample. So they self -reported watching an average of 14 hours of Fox News per week or two hours per day. So in other words, as a group, they have selected a sample that is particularly unlikely to show change in their views according to conventional wisdom, right?

0:22:36 Thomas: Yeah, yeah yeah.

0:23:04 Thomas: You would think yeah these are all our family members that we don't go to Thanksgiving for anymore like that that kind of, it's those people.

0:23:06 Lindsey: Yeah. That's right. Yeah, so they randomly assigned 40% of their sample to watch CNN and complete surveys, and the remaining 60% of their sample to just complete surveys. They did not give them an instruction to change their TV viewing, yeah.

0:23:24 Thomas: Oh, that's a, is that, that seems like a lot for control. So it was like 60 % was kind of a control group or is that normal?

0:23:31 Lindsey: Well, so I'm guessing this was a funding issue, so I think that ideally you'd want to do 50-50, but the

0:23:36 Thomas: Ah, like we can only afford so much CNN.

0:23:39 Lindsey: Yeah, pretty much, I mean, so the, you know, $15 per hour of watching, and on average people watch six hours of CNN per week, this was an expensive study, um, was a month yeah, yeah, Most of September, October was the study period.

0:23:50 Thomas: Yeah. How, what was the time period? Was it a month? What did you say? A month. Okay. Right.

0:23:59 Lindsey: Okay, so 304 individuals were paid $15 an hour to watch CNN. They agreed to watch an average of 5.8 hours of CNN per week. And they also completed five surveys about the TV that they watched, which I'll get into a little bit more in a minute. Were paid 10 dollars for each survey. The remaining people in the control group were not paid for watching TV, but they were made paid for the survey. So, they were paying for completing the five surveys.

So everybody got a fair amount of money for doing this study. Now I want to give just a brief overview of the timeline of the study and the general kinds of questions that were being asked. And then for the sake of time, I'm going to go through some of the detailed content about the questions as we work through the results. They obtained measures from participants at multiple time points here.

So they took some measures at baseline, which were obtained when they were going through that sort sort of screening process with their sample. Also had a standardized set of measures about political attitudes and beliefs that they administered five times over this four-week observation period in September, during which people were either watching the CNN or TV as usual.

And then after that treatment period, after September, they had two somewhat longer follow-up surveys, one of which was administered right after the treatment period ended, ended like within three weeks. And then another one that was administered two months after the treatment period ended. And so with all this data, we can observe the differences in political attitudes between the treatment and control group at baseline and then while they're consuming this media actively.

But then we can also see if there are any lasting effects of this treatment by seeing if, you know, the initial differences persist two months later. In case anybody's wondering how they ensured that people actually watched the CNN that they were paid to watch instead of just saying that they were watching it.

0:25:41 Thomas: Right. Yeah.

0:25:42 Lindsey: They use the hidden cameras in their smart TVs. No, I'm just kidding.

0:25:42 Thomas: Oh God. Yeah, I would have believed you if you said that. But

0:25:48 Lindsey: No, no, they told participants that at randomized times during the week, they would get a pop quiz on the content of a program that they had said they And that they had to get a majority of the questions correct in order to be compensated. And so because

0:25:58 Thomas: Wow, that takes it. That would be pretty, I don't know what the word is, but if you cheat on your getting paid to watch TV job, that would be a pretty legendary. I would like that, yeah.

0:26:09 Lindsey: Participants didn't know when these were coming, this is a pretty effective way of ensuring that they actually watch everything that they say they're going to watch. Right, quiz performance indicated the compliance was very high, so the treatment group answered an average of 12.4 out of 15 pop questions correctly, um, the median was 14 out of 15, indicating that like almost everybody got almost all of these rights, there was probably a small tail of folks who weren't complying, but they

0:26:33 Thomas: Wow. I imagine they must have been pretty basic questions, right? Because I don't remember a lot about what I watch.

0:26:41 Lindsey: Yeah, yeah, and it was also right after the program. So, boy, man, the research assistants in this study had a hell of a time doing this.

0:26:46 Thomas: Ah, gotcha. No kidding.

0:26:49 Lindsey: So they, like, they had to watch this stuff live, then they had to come up with quiz questions, and then they had to be administered within 30 minutes at the end of the period. This is nuts. So, but an example question was, you know, in the program that you just watched, which member of Congress appeared on the program to discuss racism and social justice? Was that Mitch McConnell or AOC or Karen Bass? Yeah.

0:27:10 Thomas: Mitch McConnell. He's getting all woke on CNN. And I

0:27:12 Lindsey: You Know, Mitch. Detail to mention before we move on to results and things is to note the very impressive retention rate that they had in the study. When you're collecting measures over time, it is very common for a chunk of your sample to drop out as the study goes along. And the longer the study period is, the more attrition you tend to get. Participation rate by the very last survey that they administered was 95.3% of the original sample. It's very good.

0:27:40 Thomas: Would have thought just like that many of them would have heart attacks or something. We're dealing with very elderly folks or by the way. Well, yeah.

0:27:46 Lindsey: It is true. Nope, they all made it. They did it. I That so, I mean, part of this is they just badgered the hell out of people if they didn't respond to the original requests. And also, their last-ditch effort if they couldn't get a response with someone is they sent them a letter that said, okay, you haven't responded yet, we'll give you $30 to do the final survey instead of 10. And I bet that the respondents who responded immediately and then read this pre-print were like pretty mad that they didn't hold out a little bit. But anyway, okay. Okay, so let's get into what they found. Addition to their experimental results, they also performed a content analysis of how Fox News and CNN covered stories during September 2020.

0:28:28 Thomas: Okay.

0:28:29 Lindsey: Yeah. And their goal here, obviously, is to provide context for the information that participants were exposed to during the study period. Kind of have to know that to understand the questions they were asking. But also to provide some descriptive evidence for agenda setting and partisan coverage filtering on both networks. So during the observation period, damn, these research assistants seriously had a hard job.

0:28:52 Thomas: Yeah.

0:28:52 Lindsey: During the observation period, a research assistant watched Fox News and CNN during the study period and kept notes on the topics that were being covered by each of the networks. From those notes, the lead researchers compiled a list of broad topics that Were being covered By either Fox or CNN or both, and then made a list Of subtopics as well, right? So like under the general topic of COVID -19, you might have subtoprics like long COVID exists or Trump's treatment or, you know, handling of the pandemic. And so from that list, they research assistant back this list and then had the research assistant read and code every Fox News and CNN transcript during the incentivized period. This is hours and hours of transcripts from Fox News in CNN.

0:29:37 Thomas: Yeah. Are any gains in the target subject going to be offset by the fact that these poor assistants like went, QAnon you or something?

0:29:46 Lindsey: Would like to know how that research assistant is doing. This has got to do a number on you.

0:29:50 Thomas: Yeah, they're in the hospital.

0:29:51 Lindsey: Yeah, no kidding. Every one of these transcripts. And every time a subtopic on the list came up, the research assistant copied and pasted all of the relevant direct quotes from the transcript into a spreadsheet. And then the researchers counted the number of words that Fox News and CNN dedicated to each topic and subtopic, right? The results of their content analysis suggest strong agenda setting and partisan coverage filtering on both networks. So here are the the top five CNN subtopics as measured by word count. Trump's failures to protect the U.S. and his supporters from COVID -19, that the most word count associated with it, followed by information indicating the severity of COVID - 19, racism being a problem in the U -S., mail -in voting being secure, and Trump's opposition to wearing masks. Those were their top five.

0:30:40 Thomas: So all pretty much things that happened in reality. Okay.

0:30:41 Lindsey: Oh, Yeah, correct. But Fox spent a fraction of the time on each of those topics that CNN did. So, for example, CNN spent over 21,000 words discussing Trump's failures to protect the US and his supporters from COVID-19, whereas Fox spent just over 2,000 words, so less than 10% of the amount of words on that topic. CNN spent over 10,000 words discussing the severity of COVID, while Fox devoted only about 700 words to that topic. 700 words, this is, this is again, about a month period. 700 words.

0:31:17 Thomas: I'm wondering who even snuck in those 700 words. Brett, who's the other guy who left and a couple of them leave? Maybe it's them. Oh, wait, but this is during primetime, right? Or was this at all times?

0:31:17 Lindsey: Right. Yeah, I Yeah, uh, that Chris Wallace? Yeah, just primetime.

0:31:32 Thomas: Yeah, okay. Yeah, I'm surprised.

0:31:33 Lindsey: Yeah, yeah. Likewise, right, CNN spent a fraction of the time that Fox did on Fox's top five subtopics, which were Biden and Democrats support for quote unquote, extreme racial ideologies and protests. Protests. One was negative consequences of extreme racial ideology and protests. So they're talking about violence of the protests. Third was information downplaying the severity of COVID -19. Was Biden embracing the far left. The fifth was Trump's...

0:32:01 Thomas: Fifth was the turtle that holds up the earth. CNN didn't spend any time on that.

0:32:06 Lindsey: I know, it is kind of hard to read these lists with a straight face And be like, "See both sides." Um, the last one was Trump's actions to advance racial equality. This was in the top five things that Fox spent time on, and don't you know, it CNN spent way less time on those things.

0:32:10 Thomas: Yeah.

0:32:23 Thomas: Well, I don't know. I'm not surprised that Fox proactively went after that. That's interesting. Thing. Wonder what got coded as Trump improving race. There was a time where didn't you have that like, I don't remember if this was at that time period, but there's that time where they tried to run on like, well, Trump's been the best for black Americans because of like, you know, something about the unemployment rate or something. But I feel like that was not this time. I feel like that pre -COVID, but I could be wrong.

0:32:28 Lindsey: Yeah. Yeah. Sure. I that some of these things are perennial, and and, you know, September 2020, this is right after, uh, the shooting of Jacob Blake, right? So this is when, when all the Kenosha protests are going on, I'm sure that some of that messaging got recycled. So descriptively speaking, I think it's pretty obvious that there is agenda setting here, meaning that these networks are spending different amounts of time on different topics that correspond to different views of the world, right? Or different views of these parties specifically. Also evidence of partisan news filtering, meaning that the specific details that Fox and CNN reported were chosen in a partisan way, right?

So Fox was covering the violence that happened during the Kenosha protests, whereas CNN was covering the problem of racism that motivated the protests, right, that's what they mean, bipartisan coverage filtering. Also, Fox News and CNN both covered voting by mail extensively, but CNN described it largely as secure and covered the Russian attempts to undermine confidence in the 2020 election, whereas Fox News barely covered these subtopics at all and instead spent time on how voting by mail would in fact be susceptible to fraud.

0:32:58 Thomas: Right. Yeah.

0:33:56 Lindsey: Yeah. That's the results of their sort of content analysis, which I think are interesting and useful to set us up for these experimental results. Now we get to the primary results to whether adding CNN to a Fox News viewer's media diet has any effects on that individual's beliefs, attitudes, or policy preferences, or perceptions of importance of particular issues, or broader political views. All of the results we're going to be talking about were obtained through regression analyses. So they use experimental condition, treatment, or control group.

0:34:01 Thomas: Yeah, yeah.

0:34:25 Thomas: Love regression analyses.

0:34:26 Lindsey: I know you do.

0:34:27 Thomas: You you know, official show position, SIO loves regression analysis.

0:34:30 Lindsey: It does seem to be our favorite type of analyses.

0:34:32 Thomas: This show brought to you by regression analysis, right?

0:34:35 Lindsey: Yeah, no one way ANOVA's here. Nonsense. So all regression analyses, they use experimental condition to predict beliefs and attitudes, controlling for baseline covariates, meaning measures taken prior to the treatments. So their covariates included things like how much CNN, Fox News, and Total TV people said they watched at baseline, their partisanship at baseline their attitudes towards CNN and Fox News at baseline household size. And additionally, they controlled for baseline responses to all of the attitudinal items that they ended up administering in these surveys. Is an important detail because if you are using the experimental condition to predict change on, say, attitudes toward Trump, in order to directly demonstrate change in your groups, you need to account for people's baseline attitude toward Trump. So if you don't include that in your model, then you're just essentially looking at straight up differences in attitudes toward Trump at time two. In order to see change, you have to include the baseline attitudes.

0:35:33 Thomas: Yeah, and I imagine the baseline for this 763 people is going to be pretty high on Trump,

0:35:34 Lindsey: Yeah. Yes, it is. Yes. They do include like the descriptives on that and some of their supplemental materials. And it was like on a feeling thermometer of zero to 100 on how much they like Trump. I think the average was like, it's around 90, I think. So it's pretty, yeah, it's pretty high.

0:35:55 Thomas: Yeah, he already bungled COVID, and this wasn't like 2017 or something.

0:36:01 Lindsey: Yeah, But I mean, as we're going to see, it'S possible that people didn't know that he bungled it. Yeah.

0:36:04 Thomas: No, I know, that's kind of the point I'm making, like that's how strong this effect of partisan coverage is.

0:36:08 Lindsey: Yeah, exactly. Yeah So let's start with the effects of CNN exposure that they found like in real time, essentially. So during the period that people were actually watching CNN actively. Compared to the control group, participants in the treatment group were significantly more likely to report seeing negative news about Trump and positive news about Biden, and far less likely to recall seeing positive news about Trump or negative news About Biden. So this makes sense. I mean, it's almost like a manipulation check, right?

0:36:38 Thomas: Mm-hmm.

0:36:38 Lindsey: They're showing that when they watch CNN, they're seeing more partisan coverage, essentially, about these two individuals. Sizes across the five surveys during this period was small to moderate on these perceptions. So between a 10th and a quarter of a standard deviation difference between the treatment and control, but they were consistent. And small but consistent effect there in terms of partisan valence of the news exposure. Also found, consistent with their hypotheses regarding partisan news filtering, that the treatment changed what individuals knew about current events. So folks who are watching CNN learned more unfavorable information about Republicans and more favorable news about Democrats compared to the control group. For example, folks in the treatment group were more likely to know that the following things were true, that Trump admitted privately in February of 2020 that he knew COVID-19 was more dangerous than the flu, and that Trump said, another thing, that people have seen planes full of, quote, looters and thugs who are flying into cities that are experiencing protests and looking for trouble, quote-unquote, looking for trouble.

0:37:44 Thomas: Yeah. Looters tend to jump on air. You know, I was going to make a joke that looters wouldn't jump on planes to go loot cities, but I guess they will do that to go riot in the Capitol, so I don't know. Is that technically looting, though? They looted democracy.

0:37:54 Lindsey: And Well, Yeah. I know, it right? I mean, yeah, talk about, talk about a framing effect, like I did the whole thing about that being reasonable political speech versus you know, armed. Yeah, that's a framing. Okay.

0:38:06 Thomas: Yeah, yeah. Versus people protesting where an unarmed Black person was murdered by cops is not legitimate protest. Yeah.

0:38:09 Lindsey: Yes. Exactly, yeah. In the treatment group were also more likely to know that the The Department of Homeland Security warned that the Russian government was promoting claims that mail-in voting encourages fraud in order to undermine confidence in the 2020 election. Treatment group more likely to know that than the control group. So maybe these aren't super surprising results, but it's not something that I always direct attention toward when we talk about how Republicans and Democrats live in different universes.

0:38:42 Lindsey: But yeah, of course, that's almost literally true. True. I believe that different things are happening in the world than most Fox News viewers.

0:38:42 Thomas: Mm -hm. Yeah. Yeah, absolutely. Had these conversations with my family. When I've heard these conversations about COVID, it's interesting because my Fox News mom, a at certain point, you know, we're talking about stuff and I was trying to just, you know, give like science and facts and anything normal that happens in the world. And then she's like, my news just doesn't, it says a different thing. It was like, yeah, I know we both kind of knew like, all right, she knows my news, which she probably thinks I watched some cable bullshit that I don't watch, but she's like, yeah, my news doesn't say that. And, and like kind of aware that it's like this is the impasse, you know, of like, I'm not gonna, you know, her, her news says a different thing. And I, she's trusting that and there's not really much I can say.

0:38:50 Lindsey: Yeah. Yeah. Mm -hmm. Yeah, yeah, I've definitely experienced that conversation. So importantly, it was what individuals knew, new, but it wasn't just what individuals knew that changed. Their actual attitudes toward current events also changed significantly after watching CNN as opposed to Fox News. So compared to the control group, people in the treatment group were less likely to agree with statements like, quote, it is an overreaction to go out and protest in response to the police shooting of Jacob Blake. Right. Their, their actual attitudes to where those things change. I do think that is very interesting.

0:39:56 Thomas: Hmm. Interesting. Yeah. It's also, I wonder, maybe this is too philosophical, but I I wonder how much difference there is between knowledge and attitudes, you that attitude come from, does it come from nothing or does it come from the knowledge that something really shitty happened? And so therefore it's not that unreasonable to protest it, you know?

0:40:20 Lindsey: Yeah, I'm trying to resist answering this like a social psychologist, because, like, you know, in terms of the way we treat these constructs, they are very different things. But, um, I take your point. Right, I think that how you feel about something is very heavily influenced by what you know about it, especially when we're talking about events like this.

0:40:28 Thomas: Okay. Yeah, sure.

0:40:37 Thomas: And it may just be like a cumulative effect of past knowledge kind of thing, you You know, like that might build your attitude toward something.

0:40:44 Lindsey: Yeah. Yeah, fair enough, yeah. Um, right, so we've got beliefs changing, we've got attitudes changing, um, there's some interesting ways in which people didn't change, change. So they had, they had some, right?

0:40:57 Thomas: They're still white. No, I'm just kidding.

0:41:00 Lindsey: So they, they had some more general media attitude measures, and these did not seem to change differently between the treatment and control group. So they had questions like, how much do you trust the media in general to present the news in an unbiased way? They had, like, a feeling thermometer thing toward the media in general. So, you know, zero to a hundred, how cold or warm, how how much do you like them or dislike them, that kind of thing. Um, and those things didn't shift, and that's not super surprising to me. I mean, generally, when you, there's, if you try to look at mapping between attitudes and behavior, the more general the attitude gets, the weaker that relationship gets. So the fact that they're, you know, they're changing these very specific things and it's not affecting the general attitude does not surprise me. Right. Yeah. Trust of CNN and Fox News also did not seem to consistently or significantly shift during the treatment period, which is kind of interesting.

So you look at the graphs, it looks like, you know, maybe there's a very small trend for CNN sort of trending in a more positive direction, but it's not consistent. It is consistent in the direction, but it is not significant, right? Yeah, there's basically nothing happening there. There's absolutely nothing happening in terms of attitudes toward Fox News during the treatment period.

So, like, out of the five effects from these surveys, two of the effects lean toward having reduced trust in Fox News, three of them lean toward having increased trust, and none of them were significant. So this is hovering around zero, right, from just the surveys that were administered during the TV viewing period, right. Now let's see what they found in terms of those surveys that they administered after the treatment period. Period.

They administered one survey that was taken within three weeks of the end of their CNN viewing month, and then they completed the same survey again about two months after that to really look at whether any of these things were long-term changes. We're going to see if anything, you know, sort of stuck from this treatment. So one new set of items that they added to these post -treatment surveys pertains to how important participants perceived the issues that CNN versus versus Fox News covered during that month.

And again, this is speaking to the idea of agenda setting. The volume of media coverage on a topic changes viewers' perceptions of how important the issue is, right? So they found that in the first post -treatment survey, the treatment group rated the issues that Fox News cover as less important and the issues CNN covered as more important.

So, for example, folks in the treatment groups were far less likely than those in the control group to believe that it was more important for the president to focus on violent protests than the COVID -19 pandemic, right? Kind of a weird item. It, like, put it, asked people, like, which is more important for a president to focus on, COVID - 19 or the violent protests.

And so a one was like violent protests are much more important, 10 was COVID is much more important, and if people had a five, that meant both were equally important. So they sort of pitted those two things against each other, and I think they did that because those were the two, uh like, sort of top covered things on CNN and Fox News respectively in terms of word count.

When they asked that item, they did get a significant difference in terms of people's importance when those two things were pitted against each other. I think it's very important to acknowledge, though, that on other items that they put in here where they just had, like, standalones, so they had, they gave people a list of some of the the big topics during this month, and then they just have them rate, how important is this, how how important is this, how important is this?

Almost none of those showed up as having significant differences between treatment and control groups. They can, they can get it when they're pitting them against each other, but I just, I think it's important to acknowledge this isn't, a it's not a huge effect, and it didn't show up for their other items. Yeah, yeah. Additionally, they found that these differences regarding beliefs about current events remain stable at this first follow up.

So the treatment group was still significantly less likely to believe that Trump's campaign was, for instance, taking COVID precautions at rallies. They were less likely to believe that Biden supporters are happy when they see police officers get shot. Less likely to believe that Biden supports eliminating all funding for police departments. And also less likely to believe that Democrats were trying to steal the 2020 election. Seem like pretty consequential

0:41:30 Thomas: Yeah. Yeah.

0:45:04 Lindsey: Beliefs. Glad we got some persistence on that. Group was also more likely to know that long COVID is a thing, and more likely to be aware that other countries did a better job of controlling the pandemic relative to the U.S. Also found that the size of the difference between the treatment and control on those beliefs depended on the amount that the stories were being covered, right? So some of the beliefs that they asked about showed smaller differences between the experimental control groups, and those tended to be items on topics that didn't receive much coverage from either CNN or Fox News.

0:45:05 Thomas: Yeah.

0:45:36 Thomas: Check that word count.

0:45:38 Lindsey: Exactly. Yeah. And so they didn't do a formal mediation analysis. I kind of wish they had to see if, like, the coverage actually statistically explains the signs of those differences, but it seems like an intuitive explanation. I wouldn't be surprised if that's the case. Um, important set of findings regarded participants' political preferences. So compared to the control group, the treatment group was significantly more supportive of vote by mail, and their attitudes toward Trump and Republican politicians became significantly more negative.

So some of the example items here about Trump, for instance, they had questions about whether Trump was encouraging Americans to say stay safe during COVID, uh, the word selfish describes Trump, whether Trump is ensuring that the election is fair, whether Trump is encouraging good race relations, whether he's patriotic. And on all of those items, the attitudes toward Trump became significantly less or more negative in the treatment group relative to the control.

There were no differences between the treatment and control on racial attitudes, climate, climate change, policing, attitudes toward Biden. There was no difference, there was a change in terms of partisan identification, right? So we didn't change anybody from being a strong Republican to being a moderate or a Democrat.

0:45:38 Thomas: Ah.

0:46:51 Thomas: Yet.

0:46:51 Lindsey: Right? Yeah, I like that attitude. Right. So the next results provide a potential explanation for that asymmetry, the fact that, you know, Trump evaluations got worse, but evaluations of Biden and CNN didn't get better, essentially. So when they evaluated attitudes toward Fox News and CNN, they found that attitudes toward FOX News were significantly more unfavorable in the treatment group than the control, but they were not significantly more favorable toward CNN. And So here's one potential interpretation of these findings. I think it's possible that watching CNN essentially cast a spotlight on the fact that Fox is not fair and balanced as a news network, which might not have have been obvious to people if they were watching almost nothing except Fox. And don't think that people like being lied to or manipulated.

So I think it's possible that this month made it easier for them to sort of see behind the curtain in terms of the narrative that Fox is trying to sell. But because CNN is also a partisan news network, it makes sense to me that their attitudes towards CNN wouldn't necessarily become more positive, right?

I'm guessing that these folks are, they're probably sort of left with the feeling that Fox is more biased and less credible than they thought previously, but maybe they haven't really – maybe they don't really feel like they've been given a viable alternative in CNN, right? Like, yeah, Fox is bad, but, you know,

0:47:17 Thomas: Okay.

0:48:10 Lindsey: Democrats are doing the same thing.

0:48:10 Thomas: What does this study think that CNN is? It think it's like MSNBC or something?

0:48:16 Lindsey: So they actually – they talk more about MSNBC in the intro as an example of a partisan network, but they are sort of casting CNN as being partisan. Um, Democrats.

0:48:27 Thomas: That's weird. I don't really. I

0:48:29 Lindsey: I don't watch CNN, so I don't have a great reading on this, but when I look at the, you know, those, those little graphs that they, you see sometimes where they're sort of plotting the, the news networks.

0:48:40 Thomas: Yeah, but I just worry that what happened was Trumpism, you know, again, it takes you towards stuff that's so batshit insane that merely presenting facts gets read as, you biased toward Democrats.

0:48:51 Lindsey: I Hear that. I, I don't want to fully both sides the coverage that's being, that's been described here in this paper, because yeah, yeah, the stuff that Fox is doing is blatantly misinformation. And I don't think the same thing was happening on CNN. But I think in terms of it being partisan, right, I think that that is fair, right? Do think that the from that content analysis that they did, that the stories that they're reporting on are in fact designed to cast Trump in a negative light and to cast Biden in a a positive light, right?

0:49:22 Thomas: Just weird that there's no fact of the matter, I guess. I, you know, if CNN does a special on Jeffrey Dahmer, it's like, well, they're just an anti-Jeffrey, that they're just biased, you know, they Got a part, they're partisan against Jeffrey Dahmer.

0:49:34 Lindsey: No, no I get it. I mean when the state of the world is what it was during this time period, I get that it can be sort of hard to imagine like what coverage from a non, a quote-unquote non-partisan network might look like.

0:49:49 Thomas: Yeah, what I was going to ask, like, what do you think would, would, have, what hypothetical coverage would read to these people as, like, this is just straight on objective or something? Yeah.

0:50:00 Lindsey: I mean, but take this for example. The fact that CNN's top topic in terms of word count was Trump's mishandling of the the pandemic, right? I do think that that is evidence of like the kind of thing that they're talking about. It's not that it's wrong, but it is devoting the most space to this one thing, right? And that's, I, I think, by design. I, I don't t Know.

0:50:22 Thomas: I Am just kind of wondering, I'm trying to think, like, what would, would, if there is such a thing, what coverage would they rate as, like, this is just straight up and down, straight balance.

0:50:33 Lindsey: I, I don't know, I was looking at, uh,

0:50:35 Thomas: Spend half the time on his mismanagement COVID and the other half on how well he did on COVID, I think. Objective. Seriously, I don't know what their scoring system would say is like objective coverage.

0:50:40 Lindsey: No, I I don't think I have a good answer to this. I Mean, I Think I I think it's a fair point, yeah. So but yeah, so I think an interesting point here though is that like, if if a if a Fox News viewer is paid to watch CNN, right, the fact that, I mean, I mean, I think at the very least we can, we can that CNN is kind of designed for not conservative audiences. Is that fair in of what they devote time to?

0:50:49 Thomas: That is all right. We will move on. I Guess. I mean, again, the whole thing with that producer that finally got fired was how much he helped Trump win in 2016 by covering Trump's empty podium for four hours waiting for him to come talk. I don't know. I guess that's why I struggle with what the hell this is supposed to mean. I hear you. I also think another part of that is that Jeff Zucker, I know that part of it was trying to profit off of both sides, you know, like he helped Trump get elected and then kind of also, you know, drove numbers with the anti-Trump coverage. So maybe that's part of what's going on here.

0:51:47 Lindsey: Yeah. And so I don't want to claim that like CNN is the same is the same as Fox News, just just leaning in a different direction. I don't think that's true. Um, Like CNN does have more of sort of like the opinion stuff than, and I know NPR is not a um program or whatever, but has more of that than like NPR news coverage does, right? Maybe that's a place where you could find variability in networks that would be meaningful here in terms this point is important, right, because I think that exposing Fox News is untrustworthy to viewers is pretty valuable in and of itself. It potentially is like a follow up.

It could be interesting instead of CNN to provide them with something that is, you know, reporting fully on these stories instead of omitting really important things, but maybe not doing as much of the partisan analysis or sort of manipulation of volume and stuff. Right. Like that could give people an actual alternative to Fox News if they felt kind of disillusioned with the coverage that was happening there. So, so those are the main results from that first follow up study.

The results from the last follow up survey that they administered, we can cover very quickly. So again, this was administered about two months after the end of the treatment period. And to get right to the point, they suggested that the changes that they found in that first follow up diminished considerably over time. So the treatment group, like, first of all, they didn't report any long term change in terms of their television viewing behavior.

So the amount of Fox News that they watched compared to baseline did not change. The amount of CNN did not go up right at the two month follow up compared to the control condition. And the effects in terms of beliefs and attitudes that they found at the first follow-up diminished in size considerably, virtually all of them were non-significant. So unfortunately it will probably take more than a month of like clockwork oranging your uncle Frank to watch CNN to get any long-term change here.

Um, I think that there are multiple interpretations of why these effects got smaller, you know, like some of the stuff that kind of was particularly disappointing to me that I thought could have had more staying power is like just sort of like the factual knowledge stuff. And even those effects seem to drop to non-significance, which is, I feel like there's something to be explained there. I want to investigate that a little bit more.

Like, why would you suddenly say that you, that no, you don't remember that Trump said that COVID-19 was more deadly than the flu in February, right? Doesn't that seem weird?

0:51:53 Thomas: Yeah. Yeah, I

0:54:11 Lindsey: I don't know.

0:54:11 Thomas: I Mean, That specific one does, but I could imagine that just, again, people, people don't have perfect memories, and people, these are people who are already being swayed by a fake entire world constructed by Fox News. I don't know, I guess they, so they fall back into that, yeah.

0:54:26 Lindsey: I guess so, right. I mean, Maybe it's active interference from going back to consuming Fox News. If people are just going back to old habits, maybe they're hearing actual counter -arguments or excuses that reference that story and then justify it or something. I don't know. Does seem like there's a parallel there to the capital right was just legitimate political speech rather than a violent insurrection. That could rewrite things in people's minds. So the these were not huge effects, but also Also, right, the manipulation is kind of modest in terms of trying to change these pretty established beliefs in a group of people with very, very strong beliefs.

Like all of these were political, politically active folks. All of them were extremely right wing. So I don't know. I find the fact that they got any movement here, even temporarily, somewhat Encouraging. Lots of prior research suggests that when people encounter information contrary to their viewpoints, they have a bias toward rejecting it out of hand. That's one of the mechanisms that underlies the confirmation bias, and they kind of, they, they found the opposite here, right? They found that people did believe those, those things when, uh, when they saw them presented, right?

Also, a lot of prior research suggests that people are especially likely to reject counter-attitudinal information if it's from a source that they consider untrustworthy, which surely Fox News viewers are likely to consider CNN to be, right? Yet this still had an impact, at least during the period that they were watching it, right? I don't know. I've got a couple of maybe analysis thoughts, but I know that we're already sort of close on time. So that's, that's the main things that

0:54:33 Thomas: Yeah. Yeah.

0:55:13 Thomas: That's Your sense of how this coverage stacked up? I, the coverage too positive, you think? Because it still sounded like there's some positives in there. It's just that maybe the effect didn't last. Right.

0:56:03 Lindsey: I think I think there are. Yeah. Right. I mean, I think that I think. So to me, this suggests that the effect that partisan media has on attitudes and ultimately behavior might just in general be short-term effects. So if you can sort of break people's attachment to Fox News, I think that these results actually do provide some evidence that you could get long-term change if you could convince people to stop watching Fox News. News.

0:56:33 Thomas: Yeah.

0:56:33 Lindsey: And I think that the results also provide some evidence that we can do that, right? To, I don't know, make this a longer study period or make it around issues that were more central to, you know, feelings that Fox News is a reasonable outlet, I think that using this kind of strategy, you really could undermine confidence in Fox News enough for people to want to go to a different sort of news source. This is optimistic of me, but I think that ultimately people want to have an accurate sense of what's going on in the world. So again, like if I were to do a follow-up on this study, like an extension, I would replace Fox News with something a little more neutral, right?

0:56:34 Thomas: Yeah, yeah.

0:57:15 Thomas: About just replacing it with a hobby? Do you think that would be interesting? That'd be interesting. I don't know. It'd be interesting to study that. Like if you just try to reduce, pay people to just not watch Fox News and do I don't know anything else. Yeah.

0:57:28 Lindsey: Hmm. No, probably so. Honestly.

0:57:32 Thomas: Because it's like you said, I mean, the results were like, they didn't love CNN or anything. Yeah.

0:57:36 Lindsey: They didn't. But, you know, I think the thing that you would lose if you just had people get a hobby is that the things they were seeing on CNN, I think, did it again, at least temporarily make them say, oh, man, Fox News is not fair and balanced. They completely, they completely omitted this thing in their reporting. Like people said, people answered more in agreement with an item that said that like Fox News would not report on something if it made Trump look bad. So in the treatment group, they were more likely to say, yep, that's true.

0:58:05 Thomas: Yeah.

0:58:06 Lindsey: Think that piece of it is valuable. We just got to figure out how to make that strong enough for people not to watch the network anymore. Closing thought is that, you know, we've had several conversations at this point about how policy doesn't seem to matter. It doesn't change anybody's views about Biden or anything. And I think that one interesting thing that they brought, brought up in this paper is that, you know, maybe the reason for that is that the coverage that people are getting from media Sources is so consistent, so it's not actually that people are resistant to change, but maybe they're not hearing about these policy things. Yeah.

0:58:32 Thomas: Oh. Yeah, yeah, no. I, I do think that's been kind of the mechanism there, which is people just don't find out about it and It doesn't really matter if you rhetoric Biden uses, if he says a certain thing at a certain time, it's just these Fox News viewers, just they don't hear it, it never Gets to them, but It also, but not only that, just talking about that general problem with policy not mattering, I also think that even people on the left don't get most of the information about what Biden is doing. I mean, cause that's not what drives any news. If he, if he's doing like, it's hard to make news with ordinary run of the mill government stuff now.

0:58:38 Lindsey: Yeah. Yeah. Yeah. Yeah. Right. Mm.

0:59:15 Thomas: Um, it has to be usually something extreme or something, you know, something with a, with a lot of disagreement or a lot OF flashy attention, grabby stuff. So often I would be curious if we looked at that, you know, if we try to do a study on that question, like why doesn't policy seem to make much of a difference, it would be interesting to see the numbers on like how, where is even the left of certain policies that That are happening, so I think what you're saying is the government should mandate that we all watch a certain amount of hours of state-run TV every Day and We put an eyeball in the TV to make sure that we are watching it all at The, yeah, okay.

0:59:24 Lindsey: -Hmm. Yeah.

0:59:40 Lindsey: Nope. I agree.

0:59:50 Lindsey: Yes. Yeah. Uh. It's It's funny that you say that because, because they end the paper on the note that, like, this should be disturbing to everyone because it completely undermines democracy. Like, it undermines, you know, democratic sort of, like being able to hold elected officials accountable if our news is being filtered this dramatically.

0:59:58 Thomas: Cool, oh Yeah, yeah. Yeah. Yeah. Welcome to 1998, this Study, yeah. No. No kidding, I, let's

1:00:19 Lindsey: Yeah. Mm -hmm. Good stuff. I know it's probably not the silver bullet some folks are looking for. But I think it was an interesting study.

1:00:27 Thomas: All Pinch Our Pennies and pay everybody to not watch Fox News, I Guess we all Gotta come Together We get, let's, yeah, Beijing goal, fix the entire country by, I, that's got to be like a billion dollars an episode, maybe, that, that'll Be the number, I'll put that in the Patreon.

1:00:32 Lindsey: I'm in. Patreon goal. Yeah. Nice. I'm excited.

1:00:47 Thomas: Well. Speaking of Adrian, yeah, so support the show over on patreon.com, but thanks for giving us the breakdown, Lindsey. I, I was so excited to hear your, uh, study of this, to hear your breakdown of this, because it's been everywhere, I keep seeing everybody sharing it, so I mean, it doesn't really sound like the kind of study where you're like, this is a total bullshit thing That's getting Covered wrong, it's just maybe Yeah, I mean, honestly. Honestly, it seems like the coverage, at least that I've seen through osmosis, has been mostly right. It's just not emphasized, some of the maybe some of the drawbacks or the shortcomings.

1:01:05 Lindsey: Oh, no. No.

1:01:18 Lindsey: Yeah. I mean, if anything, like I haven't read a bunch of the popular media coverage. It seems like most of them are like, yeah, they found effects at first and then they went away. That's what happened. Um, possible that people aren't sort of adequately talking about how small these effects are, like, they, they really are, even the ones that are consistent during the study period are like very small, right? They're small to moderate effects, but, but otherwise, yeah, I don't, I don't think that this has been covered in a way that's irresponsible. Yeah.

1:01:44 Thomas: Well, there you go. A good day for science. So, yeah, either donate to our Patreon or, you know, pay your family to not watch Fox News. Either one of those is a good use of your money.

1:01:47 Lindsey: Thank You.

1:02:00 Thomas: Thanks so much, Dr. Lindsay Osterman. Thanks, Thanks, everybody, for listening, and we'll see you next time.

1:02:14 Clip: Thank you so much for listening to Serious Inquiries Only. If you like the show, the absolute best thing you can do is support us at patreon.com/seriouspod. The second best thing you can do, is please share it on social media or by word of mouth. Speaking of social media, follow me on Twitter at Serious Pod. If would like to be Facebook friends, send me a request at facebook.com Thomas podcast. If not up for that level of intimacy, that's fine. Maybe you want to join the discussion at facebook.com slash groups slash SIO members. Thank you again for listening. We'll see you next time.