SIO276: Is the Dunning-Kruger Effect Real?

Transcript

This transcript was generated automatically and may contain errors — the audio is the record.

0:00:28 Thomas: Hello and welcome to Serious Inquiries Only. This is episode 276. I'm Thomas and I'm joined today by Lindsey. How you doing, Lindsey?

0:00:37 Lindsey: Pretty good, pretty good. Like, yeah.

0:00:38 Thomas: Alright. Like, really good? Like as good as I'm not doing?

0:00:42 Lindsey: I, it sounds like it. You've had, you've had some rough times.

0:00:46 Thomas: Stop the show. Gotta talk about the tornado, hurricane thing that came through. Unbelievable climate change event. California, the whole state blew, blew, it blew. Just one state, every object in the universe that was to the southeast of me is now to the northwest of me. So if anyone's missing anything, keys, you know, any, if you can't find something, the wind blew it all the way to California. It was unbelievable. Gale force, hurricane winds, our fence got knocked over, trees, power, quarter million people without out power. And I'm hoping I get, I hope, I, I help this, uh, this recording goes. Okay. It's, it's, it's not a sure thing. Power has been in and out, but so that's me. And then we think we're all, it's all resolved. We're good. We recovered this morning, you know, and, uh. Wouldn't you know it? Phoebe has a fever, which means we can't, both kids can't go to daycare for, I don't know, a calendar year or something until it's safe.

0:00:47 Lindsey: Yeah. Oh, God. Oh my God. Yeah, that sounds about right. Yeah

0:01:45 Thomas: Yeah. So that's my day. Anyway, I'm really excited to have you on, though, because this is going to cheer me up. This is going to be a really cool episode because there was an article by a friend of us. His name is Jonathan Jerry, and he wrote an about the fact that the Dunning-Kruger effect might not be real and made some waves on the Twitters, on the interwebs. Super curious about this. Lindsay was kind enough to check this out, apply her, uh, apply spider dick knowledge. Yeah.

0:01:45 Lindsey: Me too.

0:02:04 Lindsey: Yeah my expertise. Yes.

0:02:16 Thomas: Sorry. I was, yeah.

0:02:16 Lindsey: Yeah spider dick knowledge really did play a large role in my ability to evaluate this.

0:02:20 Thomas: Yeah. Just a quick plug to the Twitch stream where we have been streaming here and there and talking about all kinds of cool reproductive facts about other species. It's been a lot of fun. You, be thankful that you're not a spider, everybody. Be thankful that your not a spider. Well, specifically a male spider. I guess the women have it pretty good.

0:02:37 Lindsey: Yes, yeah, Yeah, they seem, they seem to be doing okay, they get a lot of snacks, uh, it's uh

0:02:44 Thomas: Yeah. Feel a little bit peckish? They remedy that problem.

0:02:48 Lindsey: Yeah the plugging doesn't sound like that much fun though, so I, I wouldn't want to be a spider of either sex, personally but

0:02:55 Thomas: Yeah, more details on the twitch.tv slash SeriousPod stream if you go check out those videos. It's been a lot of fun. Maybe don't bring your kids. I I don't know, unless they're spiders.

0:03:06 Lindsey: Definitely don't bring your kids, yeah, no

0:03:08 Thomas: Actually, especially if they're spiders. It's better that they don't know. All right, now that we've covered that, let's talk about Dunning-Kruger. I figure since you are so good at science-plaining everything, I thought maybe, since I read this article, maybe I can try to explain what I think this is saying. You can tell me if I got it right, and then, you critique it or confirm it. What do think?

0:03:09 Lindsey: Yeah yeah No, that sounds fun. Mm-hmm.

0:03:30 Thomas: Okay, so this article seems pretty plausible to me as not a scientist and I can't wait for you to tell me why I'm wrong, but it, it shows the classic chart, the Dunning-Kruger effect, you know, in the common understanding is that the, the less, you know, roughly speaking, the Less, you Know about a topic, the more you're going to overestimate your knowledge of a topic or a thing or a task or whatever. And so the classic experiment involved a self-assessment and like comparing a score on a test on something. Now, what was the it grammar? No, that was just one of the examples.

0:04:04 Lindsey: Grammar was one of them, yeah so

0:04:05 Thomas: Well, okay, it wasn't. Yeah, So it's taking some sort like asking people, you know, how do you think you'll score on a grammar test, for example, and they estimate what they think they'll score, and then they take the test, and the classic chart that I'm sure a lot of us are familiar with, the bottom quartile or, you know, the lower people in terms of actual score have all overestimated their abilities. And actually the top, the people who score highest have slightly underestimated their abilities. And so I think the common conception has been that like the less you know, the more you overestimate your abilities. And then as you become an expert, then you're maybe humbled a little bit. And so you underestimate your abilities, and so this article though suggests that perhaps perhaps this is just like a, I don't know what you want to call it, a data artifact or a problem with because like apparently if you just replace the data with like randomness, random samples, you reproduce the effect.

0:04:06 Lindsey: Yeah yeah

0:05:05 Thomas: And so the thing and when I looked at that, it kind of made sense in a way like, oh, it's it's almost as though everybody basically estimates that they are average, you because like the the the perceived ability is pretty close to average, You know, like within a little bit. And again, you're going to science explain this to us in just a minute. But it made a bit of intuitive sense to me like, oh, yeah, it does kind of seem like I could I could see where that might come about from just random data.

Like if everybody imagine, you know, you're going to be assessed a score on a random thing, you know, some sort of test on average, people will estimate that they are somewhere around average, you know? And then the people who are at the bottom of the test, by definition, they are, will, will have overestimated themselves because they, you Know, it averages out. So to me, that seemed kind of compelling. And the, the, at the end of the day on this article, it's like, well, this might not be a real effect. So how did I do? And then what do you think about this article?

0:05:23 Lindsey: Yeah You did A wonderful job, that's so yeah, so I thinkthat that's good summary of of what Jonathan Jerry was was, um, was summarizing here, um, right. So I, I have like two bottom lines that we can circle back to, after I, I think it's very important to actually go through the original demonstration of the Dunning-Kruger to talk about some of the more nuanced results. Because this critique is sort of aimed at like the basic demonstration that they did of the effect, but it actually doesn't address sort of the more nuanced evidence that provides more conclusive support for the mechanisms that they're proposing. So that's where I want to get to eventually.

There's a lot more to this paper than just that graph that they're sort of, you know, critiquing there. And also, Newfer is the lead researcher that authored the paper that Jonathan Jerry is talking about here. And the critique that he's leveling here is not a new one. So this critique was presented pretty soon after the publication of the original Kruger and Dunning 1999 article. And it has been thoroughly addressed, like they've gone back and, yeah, they've done new tests and and sort of reanalyze the data in a way that is consistent with um, the recommendations of people who have made these kind of statistical artifact claims, and they found that like it doesn't make the effect go away

0:06:43 Thomas: Sure. Oh, really? Huh.

0:07:23 Thomas: Oh, interesting.

0:07:24 Lindsey: Yeah, let's talk about what the Dunning-Kruger effects is, right. I think you summarize sort of the broad strokes really well, but there's some some sort of wrinkles here that I think are important, um, and I do want to say up front that like this was a wilder ride than I expected it to be, going through all this literature and the critiques and the responses and all this stuff, and I, you know, I think I read enough to do like a competent job here, but especially given the content, I'm like very aware of the fact that I may be missing something, so yeah Yeah, so I want to state that up front, you know, um, and also if I'm right about that, if I am missing something, then that's just further support of the Dunning-Kruger effect.

So there's that. There's a couple of misconceptions here that I think are important to address. So let me lay out what the effect is, and then we'll talk about some of those misconceptions, because I think those get distorted and how people talk about this. So what is the Dunning-Kruger effect, as Kruger and Dunning described it? By the way, authors on this article, it's Kruger and Dunning, 1999. I don't know why everybody calls this the Dunning-Kruger

0:07:51 Thomas: Yeah. So meta. Yeah, yeah.

0:08:20 Lindsey: Effect. It should be the Kruger-Dunning effect.

0:08:09 Thomas: Okay. Yeah I was that I was gonna make a joke, like that would be the perfect Dunning-Kruger thing, as if like it actually was supposed to be Kruger-Dunning the whole time and we all thought, like, like thought we knew what we were talking about and said it the wrong way.

0:08:34 Lindsey: Yeah, I mean, I'm guessing it's because, so David Dunning is, I am guessing was the faculty mentor, maybe Justin Kruger was his grad student at the time that they published this. I didn't have time to check that, but

0:08:44 Thomas: Are there, yeah, is it like an English, like a soundy thing? What are the rules on this guy? When you name an effect, when you name a thingy, is that like Dunning-Kruger? It has a nice ring to it. Do they just go back and forth or is it alphabetical? D, you know, K.

0:08:59 Lindsey: Yeah, it could be. I don't know.

0:09:00 Thomas: Yeah. Anyway.

0:09:02 Lindsey: I don't' know, we should find out. Okay, so here's the basic claim underlying the Dunning-Kruger effect. Basic claim is that lack of expertise in many domains imposes a double burden on people, so if you if you don't have expertise, you're likely to be less competent in that domain because of your lack of experience, and because of the lack of lack of expertize, it renders you incapable of recognizing the fact that you are incompetent because you you don't have sort of the metacognitive awareness that you would need in order to recognize when you make errors, and metacognitive means like thinking about thinking, right? So ability to sort of reflect on your performance and recognize errors essentially in domains where expertise confers skills, and that includes things like grammar and mathematics. It doesn't include things like physical skills perhaps, like I could, I could maybe learn a lot about golf swings and still not be able to perform a good golf swing, right? But in a lot of domains, learning more about it per se confers skill, right? If I learn about the rules of grammar, I'm going to be better at grammar.

0:09:03 Thomas: Okay. Hmm.

0:10:04 Thomas: As someone who plays golf, absolutely. Yes. That has, there's almost no correlation between knowing how a golf swing works and how well you hit the ball.

0:10:11 Lindsey: Yeah, right. So that's kind of an interesting thing. And they talk about that in the discussion, that there are these domains in which you're not going to find the Dunning and Kruger effects, you're only going to find it in areas where expertise actually confers skill. So, so in those those domains, right, it's the, again, it's the very knowledge that makes you competent, that also renders you able to accurately recognize competence.

0:10:35 Thomas: Yeah.

0:10:36 Lindsey: So the prediction of Dunning-Kruger effect is that people are going to make biased judgments of their own performance that are related to their expertise. So those without expertise are gonna consistently overestimate how well they're performing on a task, essentially because they're unable to notice errors, right? And kids provide cute examples of this all the time, don't they? Because they're basically not experts in anything. Thing, back when I could still listen to Louis CK, the Pig Newton thing gets me every time.

0:10:55 Thomas: Every yeah.

0:11:01 Lindsey: That's one of my favorite things

0:11:02 Thomas: Yeah, I have that Pig Newton moment a lot with Phoebe where Phoebe tells me a thing. Actually, no, Arlo's starting to do it. Well, this is whole side story I won't get into, but Arlo has a 50-50 chance of being colorblind and we still don't know. And he's shown signs either way that I thought, oh, he can see the colors and then he can't. It's a topic for another time. But anyway, I've been trying to go through colors with him here and there just to see, or maybe just to teach him. And sometimes I'll be like, and this is green. He's like, no, yellow. I was like, like, no, this is green. No yellow. Like he's so sure of himself. No, yellow, like red, just no zero hesitation. It's totally true. He's a Dunning Kruger color kid.

0:11:04 Lindsey: Yeah that's awesome, that's awesome, and we all are right, that's that's where we're going next, but um, but yeah, yeah, so so first prediction is about how that lower quartile performs, they're going to overestimate the, the second prediction is about um what people with greater expertise are going to do, so those with with expertise are going to be actually relatively accurate, according to these predictions, and judging their performance, because they have greater metacognitive skill conferred by that expertise. We're going to come back to the underestimation piece in a little bit. It's actually a little bit separate.

So that's the Dunning-Kruger effect. And before we get to the studies, I want to talk about some of these common misconceptions. So you don't want to conflate incompetence with stupidity, right? And I think that's important to say, because, you know, people who have sort of a passing familiarity with Dunning-Kruger and also like to make shitty comments on the internet have developed a real fondness for using the Dunning-Kruger effect as shorthand for "you're an idiot and I'm smarter than you," and David Dunning is an incredibly kind and humble man, and his research is, could not be further from like gotcha research to demonstrate how stupid

0:11:50 Thomas: Yeah. Yeah, I

0:12:53 Lindsey: Everybody is, right? The warning that he is clearly issuing is to have humility and caution when you are examining yourself and the limits of your own expertise. And, you know, all of us are not experts in way more things than we are experts in. So I think that's really important caution.

0:13:09 Thomas: Think at the end of this episode, though, we need to know whether it's appropriate to call Trump the Dunning Kruger president or not. And maybe it is. Maybe it isn't. I'll remain open.

0:13:09 Lindsey: Yeah, well, so we were talking on the stream about the interview with Fauci. And there were a couple of moments where Fauci is describing conversations that he's had with Trump that sound like very clearly like Dunning-Kruger in the realm of understanding like medical research

0:13:30 Thomas: Yeah? Everything though. But in the realm of everything, can you be Dunning-Kruger? Can you think that you're just a genius? And so everything you say is going to be right. Would that still follow?

0:13:33 Lindsey: Yeah that's

0:13:42 Thomas: Yeah. But would that still fall under technically Dunning Kruger or not? Is that too broad?

0:13:41 Lindsey: Called narcissism, yeah well no, no, it interacts with Dunning-Kruger, so they, they've done several studies where they look at like personality differences and how much people underestimate when they're low performers, and people who are high on narcissism or narcissistic traits um over-inflate their performance estimates more than people who are low on narcissism. So I'd say it interacts with it.

0:13:49 Thomas: Okay.

0:14:05 Thomas: Gotcha. Okay.

0:14:07 Lindsey: Yeah. So yes, Trump is probably the Dunning-Kruger president as well as the narcissistic president. But anyway, so yeah, so the Dunning-Kruger effect, definitely not to point out how stupid people are, even more stupid than we suppose they were. That's not the point at all. And also, and this comes back to the point you made a minute ago, the Dunning-Kruger effect also doesn't suggest that people are totally clueless about their competencies. So one thing that you're going to see across the studies that we talk about is that people's estimates of their own performance are positively correlated with their actual performance. It's just that they're off in these systematic ways, but people who judge themselves to be less competent or like the least competent in the set were in fact the least confident people in the data set yeah, so in a sort of ordinal or relative way, people do have some um, some insight there, it's just that they lack sort of like the detailed insight into how bad they are or whatever

0:14:52 Thomas: Yeah. Yeah.

0:15:05 Thomas: I think I've made this joke before, but you always, you don't, and I think it's even even referenced in the article maybe, but there's always the like people overestimate how smart they are. Did you know that, you know, 80% of people think they're smarter than average and that's mathematically impossible. And then I've always thought like, well, doesn't that just mean the people who are less smart or bad at estimating what, you Know, how intelligent they are? Like if, if it's 80, you Now, like whatever percentage it is, it sounds like some people are right. And then some people that are wrong about that.

0:15:21 Lindsey: Yeah yeah, yeah, I mean that you, you just described the Dunning-Kruger effect right I mean, so there's also, there's a better than average effect, like there's... there's evidence that um, that people self-inflate it it yeah it is, it is a separate thing, right so that

0:15:37 Thomas: Oh, okay. Yeah. I thought that was a separate thing that he referenced in the article. Okay. Like more people than mathematically should think they are better than average at things.

0:15:53 Lindsey: Is correct yeah at the group level, right? And there's... there's potentially like different mechanisms that can explain that under different circumstances but

0:15:53 Thomas: Okay, yeah. Well, but like just 49% of them are right, aren't they? Like, so if some 49.99% of people are better than average.

0:16:04 Lindsey: Yeah no, you're, you're absolutely right yeah, I'd say that like some, some amount of the better than average effect maybe is accounted for processes like we're going to talk about, um, but I don't think that that's all that goes into it, like, so uh, this original paper, Kruger and Dunning 1999, they did four studies, and those four studies tested four different research predictions. So first prediction was that incompetent individuals, compared with their competent peers, would dramatically overestimate their ability and performance. Uh, second prediction is that incompetant individuals will suffer from a deficient metacognitive skills, and that they will be less able than their more competent peers, peers, to recognize competence, both in themselves and, interestingly, in other people's. So we'll get to that study. That was interesting. Third prediction, so incompetent individuals be less able than their competent peers to gain insight into their true level of performance by means of social comparison information, so by directly comparing their performance to somebody else's.

0:16:11 Thomas: Okay? Okay.

0:17:06 Lindsey: And finally, fourth prediction, incompetent people can gain insight about their shortcomings by becoming more competent, which will provide the metacognitive skills necessary to recognize their poor performance. Okay.

0:17:17 Thomas: Let me make sure I understood that, because I was thinking some, some amount of this could be just a lack of exposure to other people, but it sounded like maybe I got it wrong, but it sounds like you're saying they have an inability to like understand or compare themselves to other people, are understand. 'Cause I, 'cause like, for example, this is really common whenever you try to learn an instrument, like this is something that, you know, it's actually kind of, kind of fun and interesting. I, I, you Know, I play a number of instruments and there's a really, probably the most fun you have when you're playing an instrument is when you, not when you first start, but when you kind of start learning a little bit enough to mess around and then you feel like you're amazing at it. You're like, Oh my God, I can play this. This is so cool. I can pay this stuff.

It's awesome. Whatever. And then what can often change that is if you, you know, like go on YouTube or like here, here, anybody else played and you're like Oh my god, I thought I was a mayor. I thought it was Mozart or something. Or here I thought it was Ingway or you know it's Rachmaninoff, whatever.

0:17:18 Lindsey: Yeah yeah

0:18:20 Thomas: And good. And you're like, oh my God, I had no idea the whole world of the range of like skill that there even is. So I'm just wondering, is it a lack of exposure, or is it there's something in some way that you're describing that people just don't have the ability to like make that observation?

0:18:27 Lindsey: So we're going to talk about both, um, there's... sort of two different things that you're bringing up here. So one is, is like the domain that we're talking about, because there's certain domains, and I think that things like playing an instrument may be one, one of them, where you, you can get information about your own competence via social comparison, right?

0:18:54 Thomas: Right.

0:18:55 Lindsey: Um, but if you're talking about something like, uh, English grammar, if you don't know, yeah, if you don't know the rules of English grammar, then you're not going to be able to recognize whether somebody else's performance on a test is competent or not. Um, yeah.

0:19:00 Thomas: Yeah. Okay. That makes sense. Okay, cool.

0:19:10 Lindsey: The other thing that you said, right, so, well, we'll get into it, but, like, they took two different measures. One was comparative and one was objective to sort of like tease apart what is driving the Dunning-Kruger effect. Is it just that people don't have access to knowledge about how well other people perform? Or is it really that they just don't have the competence to see their own errors?

0:19:29 Thomas: Right.

0:19:29 Lindsey: And so they test both of those. All right. So overview, four studies to test these predictions. And they tested them in three different domains. So the first study evaluated people's abilities to recognize how funny a joke was. Yeah, I'm gonna spend less time on that.

0:19:30 Thomas: Okay?

0:19:47 Thomas: Are you about to explain all of dad jokes? Oh, man. Dunning-Kruger dad jokes. Everybody, every dad thinks they're freaking hilarious, okay?

0:19:53 Lindsey: Yeah. Yeah, I'm going to spend a little bit less time on study because it's like, it doesn't, it doesn't provide as much information as the last three do. But yeah, we'll talk about it a little. So yeah, ability to recognize the funniness of jokes, uh, studies two and four looked at performance on different logical reasoning tasks, and then there's the third study, was that English grammar study. Um, so, so again, I'm going gloss over this first one, but here, here are the highlights. So they assessed how good people were at judging the funniest of jokes by comparing the ratings of funniness that their participants gave for a range of jokes with the ratingsof professional comedians, and it was that gap that they were, were using to assess, like, essentially competence. So yeah. I know I, it

0:20:38 Thomas: That seems really subjective that's like, I, I wonder why they picked that particular thing. Like, it seems like there's about much more objective measures or objective fields, because like, often there's also a thing, like, how do you rate how funny something is? Like, how funny something is, it might be that a bunch of snobby comedians would think something more edgy or interesting or different is funny. Whereas, you the show Friends was highly successful, and that, like, for by a by a measure of like how many syndication numbers. Friends is funny.

0:20:42 Lindsey: Yes. Yeah.

0:21:05 Lindsey: No. No, you're totally right. And that, that's so, the reason that they, that they decided to use humor as one of the three domains that they looked at is because it's, it's kind of a, a complex, socially relevant kind of skill, to, like, recognize or produce humor, and that sets it apart from, you know, these logical reasoning tasks and English grammar texts, which have the objective piece that you're, you're wanting here, um, but you know, maybe lack that sort of, uh, it's a different, it's a different kind of skill, right, to recognize when something's funny. Um, and in terms of the, the reliability that you're talking about, so they got eight different professional comedians and confirmed that they had sort of reliable judgments of the, the funniness of the jokes, to just concert, uh, confirm that there was convergence there in those ratings.

But anyway, so yeah, so they, they got people's ratings of the funniest of the jokes, they compared it to the comedians for, you know, their measure of performance, and they also had people self-assess how good they were at recognizing when something is funny in general. And the first important thing that they found in this first study was that as a group, people's perceived performance was significantly correlated with their actual performance, so they had some degree of accuracy there. But it was inflated at the group level.

0:21:34 Thomas: OK,

0:22:22 Lindsey: So on average, people thought that they were at 66th percentile. And as you mentioned a minute ago, mathematically, the average has to be at the 50th percentiles.

0:22:30 Thomas: Gotcha.

0:22:30 Lindsey: So as a group, it's inflated. But more importantly, this study was the first one to document the now famous Dunning-Kruger effect. So they showed that participants in the bottom quartiles, the bottom 25% of performers, vastly overestimated their relative performance. So how good they thought they were relative to other participants. So they were actually on average at the 15th percentile, but they rated themselves to be on average at the 58th percentiles. So that's a pretty big overestimation. And then the top 25 performers were more accurate in judging their ability to recognize humor. And to the extent that they made errors, they were in the other direction. So, they slightly underestimated their performance relative to the other participants of the effect.

0:22:31 Thomas: Mm

0:23:17 Lindsey: In study two, their goal was to replicate study one in a different domain. So this one used logical reasoning task. And they addressed in the article that like, yeah, you know, the humor thing's a little bit subjective, it's a bit squishy. You know, there's nothing subjective about scoring this test of logical reasoning. And additionally, and this was super important, in the humor study, in terms of self assessments, all they could measure was participants sort of socially relative self-assessments. So how good do you think you are at this task relative to other people? But in study two and for the rest of the studies, they were also able to get a sense of people's perceived objective performance. So just how many questions do you got right out of 20? And that allows them, in addition to the social comparison piece, to look at accuracy in sort of two different domains, right?

0:24:08 Lindsey: How accurate are they in judging their relative standing relative to other people, and how good are they at recognizing like how many errors they made in an objective sense? Cool, so they had uh 45 participants in this study, and everybody completed a 20-item logical reasoning test from the LSAT. It's the the law law school admission test.

0:24:26 Thomas: I would have done this just for fun. They could have gotten me. They could've kept their five bucks or whatever. They paid people to do this.

0:24:28 Lindsey: Yeah. But yeah, so the logical reasoning section just has people read like a brief argument, and then they have answer questions that gauge ability to understand, criticize, and draw reasonable conclusions from those arguments, right? So everybody completed 20 20-item test of that taken from the LSAT. And then they also completed questions assessing their relative performance estimates. So they were asked to estimate their own percentile rank on logical reasoning in general as an ability, and their estimated percentile rank on this specific test of logical reasoning. So, they estimated what they thought their performance was relative to others on that test. And, then, they got an estimate of objective performance. So they asked participants to estimate how many questions they thought they got correct out of 20.

0:25:20 Lindsey: So here's what they found. Participants overestimated their performance relative to their peers as a group. So that replicates study one. So on average, they thought we're at the 66th percentile when in actuality, of course, they have to be at the 50th. Interestingly, and this is a difference from the last two studies, participants did not in this study overestimate how many questions they answered correctly at the group level.

0:25:35 Thomas: Right.

0:25:45 Lindsey: So on, yeah, right. So there's going to be a qualification in terms of performance here, but at the Group level after,

0:25:44 Thomas: That's interesting. Oh and sorry if I missed it, did they ask them that before or after they took it? They asked like, hey, you're gonna take it. Oh, okay. Gotcha. Hmm. And

0:25:55 Lindsey: Yeah. So, so Yeah, that's an important detail, right? So they just took a test and then they're asking people to reflect on what they just did and estimate how many errors they made. Yeah.

0:26:07 Thomas: So as a group, they pretty much nailed that.

0:26:10 Lindsey: Well, again, that is the average though, so right, so also they did get overestimation at the group level for the other two studies, so it's it's possible that this is a fluke, right? So yeah, so at the group level, on average, people thought they got 13.3 questions correct when it actually, when in actuality they got 12.9 questions correct, that's not a significant difference. But when we break things out by quartile, so participants in the bottom quartile to average 12th percentile, those individuals thought that they were at the 68th percentiles for general ability and logical reasoning, and they thought they were the 62nd percentile on test performance. And furthermore, participants in bottom quartiles overestimated how many questions they thought they'd answered correctly. So they thought they got 14 correct, when in actuality they'd only gotten like nine and a half questions correct on average. And that's more so than the group average, like that's kind of the effect that you're looking for with Dunning-Kruger, right?

0:26:17 Thomas: Okay

0:27:06 Lindsey: It's not that everybody would overestimate how many questions they got correct, it's specifically those folks. That's the bottom quartile. And then participants in the top quartile, who on average were at the 86th percentile, slightly underestimated their percentile rank. So they thought they were around the 70th percentiles. And they slightly underestimated how many questions they got right. So they got 14 on average, and they actually got like 16.9 on average. So it's a smaller error than was being made in the bottom quartile, but they're still slightly underestiminating that. That's another thing that's different in the last two studies. Okay, so study two, they replicated the general pattern from study one. But again, they were able to look at overestimation in two different ways this time.

0:27:51 Lindsey: So they got a sense of whether people are overestimating only in a sort of socially relevant or socially relative way, or if they're also overestimating their objective performance. And again, this is important for the reasons that you kind of brought up before, right? It could be the case that everybody has like like, a very accurate sense of how many questions they're getting right or wrong, but they just don't have a well-calibrated sense of what they're others perform.

0:28:16 Lindsey: And that would suggest that, like, the Dunning-Kruger effect is not produced by a metacognitive deficit because everybody knows how many errors they're making, but rather it's the product of, like a false consensus effect. So I know I did shitty, but I assume that everybody else did shitty too. But that's not what they found, right? They found that people are, seem to not be recognizing the number of errors that they're making in the lower quartile. So

0:28:01 Thomas: Yeah Yeah Right Okay, so even as a group, even though it averages out, it's because the people at the top are underestimating and people at the bottom are, yeah, okay.

0:28:42 Lindsey: Yeah Yeah. Yeah, and and also, so as I mentioned, um, these patterns look slightly different in the last two studies, and I have a feeling, um, that it has something to do with like test difficulty, which is something that can sort of change the pattern in the Dunning-Kruger effect a little bit, like tweak the level of miscalibration on some things, so yeah Um, yeah. Yeah. So, well,

0:28:43 Thomas: Hmm. Oh interesting yeah Like if you give people a super hard test, will that kind of slap the Dunning-Kruger out of them a little bit? But if you ask them Afterward, like, how well do you think you did? And it was impossible. Maybe they'd be like, geez, I guess I didn't do that well.

0:29:12 Lindsey: Yeah, no, actually. So the, the interesting thing about, about the Dunning-Kruger effect is that you don't get it in domains where people know absolutely nothing. Yeah. You only get it if, if it's like an area that people feel like they're familiar with and they just fail to recognize like, yes.

0:29:29 Thomas: Right, right, right. Yeah. Gotcha. Yeah. So that would be consistent with that then. Like, if you give them something impossible, they don't have even the little bit of knowledge that would suggest they think they know more than they do.

0:29:45 Lindsey: Yeah, absolutely. Yeah. Mean, if you, if you give me a quantum mechanics test, like there's no way I'm going to say like, like, yeah, I'll do better than average. So totally.

0:29:46 Thomas: Okay. Yeah. I probably got 14 of those. Yeah.

0:29:55 Lindsey: Yeah. Right, so study three, very similar procedure, but this time, instead of logical reasoning, they're evaluating English grammar skills. So we got 84 participants, they complete a 20 item test of grammar. And this is taken from the National Teacher Examination Prep Guide. And so each question requires participants to evaluate an underlying portion of a sentence and judge whether or not it's grammatically correct or needs to be changed to one of four rewordings to make it grammatical. And in addition, the participants also again rated their general ability to identify grammatically correct standard English and their performance on this particular test, both its percentiles, and then they also estimated how many questions they thought they'd answered correctly. And overall, replicated the effect again.

0:30:42 Lindsey: So participants overestimated their ability and performance as a group. On average, participants thought they were at 71st percentile on ability and 68th percentile. On test performance, and they also overestimated as a group, the number of items that they thought they'd answered correctly. So on average, people thought they answered 15.2 correctly, when in actuality, they'd only answered 13.3 correctly. Those effects were exaggerated in the bottom quartile. So people who were at at the 10th percentile on average, that's the bottom 25% of performers, estimated their ability at the 67th percentiles and their test performance at the 61st percentile. And they also overestimated their raw score by 3.7 points. So 12.9 perceived items correct versus 9.2 actually correct.

0:31:29 Lindsey: And the top quartile, this is interesting, slightly underestimated their relative performance, performance, so they thought that they were like at the 72nd percentile for ability and 70th percentile on performance, but they were actually accurate regarding the number of questions that they thought they got correct, so 16.9 perceived correct versus 16.4 actual correct, and it's not a significant difference, and so these results again are more consistent with a metacognitive deficit account of the DKE, right, of the Dunning-Kruger effect, top performance are more accurate and judging their objective performance. But as with the first two studies, their judgment of performance relative to others estimated or underestimated their true relative standing. So study three had a second phase that I think is super interesting, important.

0:32:15 Lindsey: And they're sort of further probing these mechanisms of like metacognitive deficits at the lower end versus like this sort of false consensus kind of thing that seems to be happening at the higher end where people are knowing how well they did but thinking it's not very good relative to others, you know, um, so, um, they did a second wave where they had the bottom and top performers, uh, come back to do a second way of data collection, um, and they asked participants to grade the tests, right, from the same study, so this English grammar test, of five other participants and assess how competent each of those five other participants was and then your

0:32:53 Thomas: Are they getting the answer key or like you graded based on what you think is the right your knowledge? OK.

0:32:57 Lindsey: Knowledge yeah yeah yeah so if you have the ability to sort of recognize competence here then seeing other people's performance potentially like could provide you new information about how well you're doing

0:32:59 Thomas: Right.

0:33:11 Thomas: So maybe the social part will be more accurate after that. Potentially.

0:33:12 Lindsey: Yes well potentially right if you can use that as a source of information, right, right? So probably for the top performers. But for the people at the bottom, right? If it's really this metacognitive deficit that's causing them to inflate their self-views, right, then looking at the tests of other people isn't going to change anything for you because that doesn't, you're not competent to judge whether or not somebody else did well or poorly because you don't know those rules, right. So they're expecting this exposure to the tests of other people to change the social perceptions for the top performers, but not to change anything for the bottom performers yes

0:33:18 Thomas: Yeah. Yeah, yeah. Because, again, importantly, they don't have the answer keys. So it's not like they're taking the bottom performance, performers, and being like, here, look at, see how these people objectively got something wrong. They're using their not their potentially incorrect knowledge to still judge other people, so they should still, as presumably, should still see an effect there interesting drum

0:33:59 Lindsey: Yes that's right, um yeah, right, and so the, and the other, the other piece that this allows them to examine, right, is whether the inflation among bottom performers is just like a self-enhancement bias, right, because if it is, and it's not really about this metacognitive deficit, then low performers should be able to accurately report when other people's performance is poor, but just show inflated self-ratings, right, um, and so looking, looking, looking at whether seeing other people's tests changes anything really bolsters support for whether or not this is a metacognitive deficit or some other kind of effect. So they had people grade five tests from other participants. And then after that, they had participants reassess their own performance after grading the tests of their peers. And so again, they're thinking that the top quartile is going to become more accurate in terms of their relative rankings, kinks of themselves, but that the bottom quartile is not because they wouldn't know enough to recognize the superior performance of their peers.

0:35:06 Thomas: Roll, let's find out what they found, right

0:35:08 Lindsey: Yeah. So participants in the bottom quartile were just, this is just to establish things at baseline, they were less accurate in grading the test of the top quartile. So they were not competent to evaluate when somebody else made an error. They're less accurate compared to people in the top quintile. And participants in the bottom quartile also did not adjust their self-assessments of their own competence after seeing their peers' tests. But the top quartile participants became more accurate in their relative self-ratings. So their percentile ranking estimates of themselves went up because they were able to see like, oh man, like I compared to these people, I actually did pretty well. So their self rankings became more inaccurate. And so this really does suggest that the reason for inflation in the bottom quartile is metacognitive. They have a deficit. They can't evaluate their own own competence. And the reason for the underestimation in the relative performance of the top quartile is due to a false consensus effect. They know that they're good at grammar, but they assume everybody else is too.

0:36:05 Thomas: Well that is a super cool, so that, that seems like strong evidence then I don't

0:36:09 Lindsey: Yeah, it does. Totally agreed. Right. And for the mechanisms, right? I mean, this is, we'll come back to this, but these statistical critiques that have been launched are fine, right. You would want to establish that the overall effect is not just a statistical artifact. So it's good that those criticisms been brought up. It's good that Kruger and Dunning responded to them and showed that the effect is robust to those effects. But also those effects ignore evidence like this, right? They're only addressing that basic graph.

They're not addressing the evidence about mechanisms here, which is actually very persuasive, right. It is very strong. So yeah. All right. Last study, study for, this is another logical reasoning test, different one than the first one, um, actually I'm curious if you've run into this before, you heard of the the ways in selection task? Okay. It's like you present people with four cards, and in the in the original version of the ways and each card has a letter on one side and a number on the other, the task is you are supposed to figure out, out whether the cards violate a rule that's given to you. That's, it's like a logical reasoning thing. So if I give, if I place four cards in front of you, and on the top face, it says A on one card, seven on the second, B on the third, and four on the fourth. So A7B4. And then I give you a rule. I say, if a card has a vowel on one side, then it must have an odd number on the other.

0:36:58 Thomas: Know by name, hmm cool

0:37:39 Lindsey: Right. And Then I'm going to ask you to indicate, all right, which cards do you have to flip over in order to test if the rule is violated or followed, right?

0:37:45 Thomas: Yeah, I don't remember if I've heard of that, but yeah, sure. Sure. Sounds cool.

0:37:48 Lindsey: Yeah. So this is a very difficult problem to solve without training. There's a heuristic that once you learn it, it becomes fairly easy. But without training, it's very, very difficult. So if you think through it, right, so A7B4, and we're testing the rule, if a card has a vowel on one side, it has to have an odd number on the other, right? So what most people say to solve this problem is that you have to flip over the cards that say A and 7, because they're thinking, well, I got to flip over A to make sure that the vowel has an odd number on the other side. And I have to flip over 7, the odd number to make sure that there's a vowel on the other side. But that's like a logical fallacy, right?

0:38:29 Thomas: Right. Is it because it doesn't go the other way? Is it because you're trying to find the something that disconfirms it?

0:38:33 Lindsey: Yeah, it that is precisely it, right, you need to use a disconfirmatory strategy. And So the right answer to this problem is A and 4, because if 4 has a vowel on the other side, that would violate the rule. Yeah. So they used this task, and they had participants complete 10 of these ways and problems.

0:38:36 Thomas: Okay.

0:38:57 Lindsey: And so they completed this task. 10 things. They estimated their percentile rank in logical reasoning, their percentiles rank on the test that they just took, and then they estimated the number of questions they thought they got correct out of 10.

0:39:11 Thomas: Sorry, what's the 10?

0:39:12 Lindsey: That's

0:39:12 Thomas: Questions then? Is that a different set of rules each time? Or is it just a different sort of cards where you're like, you need to pick one to flip?

0:39:16 Lindsey: Right, no. It's, it's, uh, it's a different set of cards, but it's also a different sort of rules, or Different Versions Of the ways, yeah.

0:39:23 Thomas: Different set of rules. Okay. And so it's kind of interesting, because essentially, it just matters whether or not you figured out this one particular thing. Like the fact that you need to disconfirm it rather than confirm it?

0:39:25 Lindsey: Uh, It seems to be, yeah. I mean, Actually, I'll qualify that, because this is, this is the version of the ways and problem that I always see given as examples and articles, but I've never done the full thing, so it's possible that there's other logical rules in there, um, other than the disconfirmatory thing. But yeah, I will state my ignorance on that one. But nonetheless, right, like it's, uh, Actually,

0:39:36 Thomas: Oh, okay.

0:39:55 Thomas: I'm Just wondering what kind of scores we should, are we expecting that people who aren't good at this get zero and people who are good at it get a hundred percent? Or is there A certain amount Of chance? Okay.

0:40:03 Lindsey: Yes. Yeah. So, yeah, it's not, it's not perfectly that, but there, it's a, it's an Unusual distribution.

0:40:08 Thomas: Like people might luckily grab, you know, pick the right one. Gotcha.

0:40:09 Lindsey: Yeah. Right, yeah, so it's got real heavy tails, and there are fewer people that score in the middle on this test, um, which is, yeah, it's unusual, um, but yeah, okay, all right. So that's the same as the previous study. The thing that they added here that's important is that they added a manipulation, experimental manipulation, to see if they could get the Dunning-Kruger effect to go away. And so after they did the test and made their self-ratings, they randomly assigned their 140 participants to either receive a training packet in solving logical reasoning problems, like the Wazen, or they didn't receive that training. They got some sort of filler task. And then after they went through that training, they were able to go back to their tests and sort of score them, indicate which problems they thought they had answered correctly and incorrectly again.

0:41:01 Lindsey: And then they re-answered the questions from before about percentile rank of logical reasoning in general, estimated percentile ranked on the test, and they were able to give a new estimate of the number of questions they thought got correct out of 10. Right, and so what they're testing here is like, okay, uh, before the training we had, you know, a bunch of people varying levels of competence, and presumably we have a tail of people that are not very competent at this task, and so we're going to do is make those people more competent and see if that gets rid of the Dunning-Kruger effect. If we give them, you know, If it's really about this sort of metacognitive deficit, and then we make them competent, then they should become very accurate it in their self-assessments after sort of gaining that metacognitive skill.

0:41:35 Thomas: Yeah.

0:41:46 Thomas: Assuming it. So because it's kind of a finite set of skill, it's not something you have to become like an expert in studying thousands of years pouring over these problems. It's like something that you can become virtually an expert in pretty easily. Is that why this might work? Okay.

0:41:52 Lindsey: Yeah. Yeah. Right, it's, it's a fairly simple rule, and once you're trained on it, then you're real good at the ways and selection task, um, but importantly, you're then able to recognize your own errors, and you're able to make a more accurate judgment about how well you did and how well you were likely to have done relative to other people, um, so prior to Training.

0:42:18 Thomas: SIO drum roll. We got it. We might have to start coming up with a sound effect. If you're going keep coming on here and presenting us these awesome studies that we're all waiting to find out, like drum roll, tension, what did they find?

0:42:18 Lindsey: Yeah.

0:42:30 Lindsey: I would like a drum roll, that'd Be great, um, Yeah.

0:42:31 Thomas: Yeah. We'll work on it.

0:42:33 Lindsey: So, so yeah, so the results were, were as follows, um, so prior to training, they replicated the previous pattern, right? So, the, we got the bottom quartile overestimating themselves, both relative to others and in terms of objective performance, we've got the top quartile being like fairly accurate about how well they performed in an objective sense, but underestimating their relative standing. And then there's the, the new stuff, right? They found that training improved accuracy dramatically overall as a group, but it was especially striking in the bottom quartile. So participants who received the training graded their tests more accurately than those who did not, and the increase in accuracy was especially dramatic in that bottom quartyle. So, like, without, without training, people in the bottom quartile only graded 3.5% of the problems accurately.

0:43:25 Lindsey: But after training, they were indistinguishable from the people who were originally high performers. The upshot is that once they have that indication of, like, improved metacognitive skill on this task, then they can use that to figure out if that statistically explains the effect that they're getting. And so that provides more evidence for this, this mechanism that they're postulating underlies this effect. So, okay, all right, so, uh, so after training, people in the bottom quartile became better calibrated regarding their performance on ability, um, so they, they were still a little bit inflated, they still thought that they were higher as a percentile rank than they actually were, but the, the margin by which they were wrong shrunk significantly. So, like, their estimated percentile, um, without training was like at the 51st percentile, and with training dropped to like the 32nd percentile. So that's like a 20 point estimation difference, That's Pretty large.

0:44:14 Thomas: So were they asked to to reassess how they originally did or how they would do? OK, gotcha. That makes sense.

0:44:15 Lindsey: Yes. Yeah. Yeah Exactly. Um, Yeah, uh, their relative estimates became more accurate, and their estimates of, like, objective accuracy improved as well. So, without training, they thought they'd gotten an average of 5.3 questions correct. And after training, they were essentially accurate in knowing how many questions they got correct. So their estimate dropped to an average of one, which was consistent with how they actually did. So, after training, people in the top quartile also became better calibrated in the other direction. So their perceived percentile about the test went up after they, they received the training, right? And their estimate of the number of items they got correct went up, not as much, because they were already pretty accurate before the training. But nonetheless, in the note, that

0:45:06 Thomas: Interesting. I was curious about that because it doesn't seem like that would necessarily make sense. Like if you you know, like if you're doing a thing and now we're on to the people who are better at the thing. Right. And we're saying, OK, you're receiving a packet that teaches you how to do these things. And then that, you know, like, couldn't you see it going the other way, which is like, oh, yeah, this is easy. So therefore, other people will have done this right already. Or is it or is it that? So I guess on average, people see that and think, wow, I was even better than I thought getting it all right before this packet. Is that what it is?

0:45:41 Lindsey: Seems reasonable to me. Yeah, I recognize what you're pointing out there. But because, I mean, from study three, that was kind of the take home, right. They were assuming that, but I think that like, I think the Wazen is sort of on its face a more difficult task than the grammar test is. And So it's possible that that's part of why the intuition is a little bit different with this task. All right, so those are, you know, replications of the previous studies. And then we've also got this experimental piece showing that you can get rid of the Dunning-Kruger effect when you improve competence, right, which is, it's very strong evidence that the mechanism that they're suggesting is actually the mechanismthat's underlying the effect, right? And so the last piece of what they did with this fourth study is they conducted an actual mediation analysis to see if the increase in metacognitive skills explain the relationship between objective performance and inflated self-assessment. And it did. So that's statistical evidence that like, yeah, it's this metacoggnitive skill that is causing the miscalibration in the lower quartile. So, that's the studies.

0:45:42 Thomas: Hmm. Yeah. OK, That's the study. So how do we go about trying to see, or if they already did, trying to see like what amount of random noise could account for any of this? Is it some amount of it or none amount of it or is there any way they separate it?

0:46:56 Lindsey: It is, yep, there is some amount of this. So after they published this original paper, as I said, Neufer's not the first person to say like, yeah, it looks like you've got a noisy measure. And so that could be creating sort of an illusory effect. And so they published a response after these criticisms and were like, hey, you know what? That was a reasonable thing to bring up. We collected some more data. We ran some new analyses that are consistent with your suggestions here. And we've controlled for things that would be related to statistical noise, right? And they find that when they control for that, the effect is attenuated a little bit, right? So part of the size of the effect is likely because there's measurement error, right, but it does not get rid of the affect. It is robust after controlling for those things. Yeah,

0:47:30 Thomas: Yeah. Huh. Interesting. Yeah. Cause it does seem like the graph itself, like we talked about in the beginning of the episode, it does seem like there there would be some amount of you should expect a little bit of this just by the way numbers work, like if everybody thinks they're, you know, might on average people might think they're on average around average skill, so therefore people who are less average skill, you know, would have that effect. And people, you know, so but I guess if if you're saying they're able to try to control for that somewhat and it does make sense that when you've tested and done on experiments to show like the actual mechanism, as you say, that then it's like, OK, it would be hard to when you when you're going through those experiments, it's hard to sometimes it almost seemed tautological, like, well, yeah, obviously, you know, so it's kind of interesting to be like either this doesn't exist or it's so obvious that, of course, it exists. I'm not sure which. Which.

0:48:14 Lindsey: Yes, yeah pretty Much. Yeah, so one of the, I always send you the articles afterwards, and one of things I'm going to send is a chapter that David Dunning wrote in 2011, and it's just, he is one of the clearest writers and most interesting writers ever, right, and it, it's written, it's, you know, it's it's a chapter, so it's written in a slightly different way than a a research article. And he goes through, um, a lot of the research that came out after this original 1999 paper was published that demonstrates exactly what we're talking about here, like the mechanisms that prevent people from having perfect self-knowledge, and especially if they don't have have competence.

And it's it's just, it's very, it's very convincing, and the thing, the problem that I have, so, you know, the the criticism about the noise, right, there's a related criticism about regression to the mean and how that might produce this effect. And as I said at the beginning, the problem that I have with it is that it's addressing like one very small thing, right, which is that the this effect exists, and it's not addressing, it's not even intended to address, I don't think, the the the more nuanced mechanistic findings that have been presented. And and they're they're huge, right, like this is the number of times that this effect has been documented, the number of mechanisms that have lended support to the idea that this is produced by a metacognitive deficit, is huge, um, and

0:48:47 Thomas: Wow. Yeah. Yeah, yeah.

0:49:54 Lindsey: So

0:49:54 Thomas: I'm always confused as a non-science person how, a lot of times when we're breaking down studies and you're explaining these things to us, and just everywhere that that you might, you know, somebody who's not a scientist might consume things like this, I never know what to make of the the effect exists thing. That just means that statistically. But like how much of our, I think there's a big difference between saying like this exists versus how to, to what degree it might exist, or to, to how much we should go looking for it in the wild. You know, like if it's something that's just, well, once you take out the random noise, it actually, it does exist, it's statistically significant, but you know, barely. So how do you measure that?

0:50:32 Lindsey: It's I this is a pretty large

0:50:33 Thomas: Or how do, you know. Just by effect size, like how large of the effect is. Okay.

0:50:34 Lindsey: Yeah So So, um, Yeah That's exactly right. Right. And the effect size of this is not, is not small. Like this is, this is not just a thing that like, yes, you can get it if you have a large enough sample, but it's not practically significant. I think that this is a practically significant effect.

0:50:42 Thomas: Okay. Cool. Wow. Well, what went wrong with this article then?

0:50:53 Lindsey: Yeah. Well, okay. So I, I didn't want to lead with this because this is a heuristic, right? But I looked up the lead author on this, the series of papers that that came out, Edward Neufer. And he's not a psychologist. He's a professor of geology, and he's apparently a director of like educational assessment at a university. And so he has some interest in, you know, whether people can accurately self-assess.

0:51:19 Thomas: Interesting.

0:51:20 Lindsey: But the problem is, it's not like, it's like it's impossible for somebody to, to lodge a valid critique if they're not a psychiatrist. But I mean, this is kind of the the point of the effect, is that he's more likely to be missing something important if he's trying to criticize something that he doesn't have expertise in. And the impression I got from the article is that there are some things that he is not understanding about the effect.

0:51:50 Thomas: So you're saying he's done and crude. You're trying to not say he's Dunning-Kruger.

0:51:53 Lindsey: You know, this, Well, but it's a thing that we, it's thing that we have to deal with as psychologists a lot because everybody has lay intuitions about psychology, and there's a lot of people who have expertise in statistics in areas outside of psychology, and so I feel like people from other disciplines who do not have expertise in psychology uh feel kind of permitted to critique stuff without actual psychological expertise because I don't think they value it very much, and that's that's kind of how this critique read to me, like

0:52:07 Thomas: Mm-hmm.

0:52:22 Thomas: Interesting. So you don't think it's the case that, like, the stats are the stats, the math is the math?

0:52:29 Lindsey: It's not the whole picture, like the statistics are very important. And I'm not saying that it wasn't important for these critiques to be brought up and evaluated as alternative explanations, because that was possible.

0:52:40 Thomas: Mm-mm.

0:52:40 Lindsey: But they've evaluated those alternative explanations and there's still an effect here. And I think that, yeah, I think the expertise in psychology and being able to understand like how these mechanisms are tested and how to evaluate the evidence for them is relevant to evaluating whether the Dunning-Kruger effect exists.

0:52:44 Thomas: Right.

0:52:55 Thomas: Yeah. But it just seems like that's still something you could do with the statistics knowledge. Like, it doesn't seem like you need, you know? Isn't that well within the realm of what you should already be familiar with if you're good at stats is saying, oh, they controlled for this effect and it still came through.

0:53:11 Lindsey: Not necessarily. Let me, let me, um, give you an analogy, like I, I think that you would be immediately suspicious if somebody, um, like with expertise in biology, say, was like, came in and and tried to criticize the the math of of somebody who did a demonstration of some quantum mechanics principle, I think it would would be immediately obvious that, like, okay, like I'm glad you know about math and statistics, but maybe there are some pieces that you're missing because you don't have expertise in quantum mechanics, and that, I, I feel like that's a reasonable comparison to what's going on here.

0:53:14 Thomas: Okay. Okay. Okay. Oh, Huh. All

0:53:47 Lindsey: Like there there are, there are things that you develop expertise in when you, when you have to deal with human data, um, and training that you get in terms of like these theoretical conceptual relationships and how they, how they relate to operational definitions in your study design that I doubt this person has expertise in. And so I feel like he's ignoring, and I'm not basing this on just the fact that he's a geologist, right, I'm basing it on the fact that I read two of his papers very carefully, and he, he ignored things that I feel like you would have to address in order to strongly make the claim that the Dunning-Kruger effect is a statistical artifact, acknowledging the experimental data or acknowledging study three about the social comparison and the the um the lack of symmetry on either side, so that

0:54:32 Thomas: Right. Well, you've convinced me. I feel, but that, that isn't saying a lot, but I see what you're saying. And that's, so I guess we can continue going along in our lives thinking the Dunning-Kruger effect is real and it makes sense. All

0:54:45 Lindsey: Is my position. I will qualify once again, though, that he probably has expertise in things I don't, so maybe I'm missing something here, that's totally possible, but people have been trying to debunk this effect for a long time and it has stood up to a lot of criticism, and I think that says something, you know, there's a lot of psychological effects for which that is not the case. Oh,

0:54:47 Thomas: Right. Yeah. Huh? Well, thanks so much for the breakdown, Lindsay. You're such a good science talking person that I bet you would underestimate your skill at talking science. That's how good you are.

0:55:12 Lindsey: That's so nice. Yeah.

0:55:15 Thomas: I love hearing these breakdowns from you. That's so cool. I know a lot of people were talking about this online, so I hope maybe they will refer to this and believe in the Dunning-Kruger effect once again.

0:55:28 Lindsey: Me too.

0:55:29 Thomas: Well, thanks so much for listening, everybody. Thanks so much, Lindsay, again, for coming on. I love these Sciency episodes. They're just so much fun, and you explain it so clearly, and I really enjoy it. And I'm sure we will be talking again in the very near future. Everybody else, I'll see you whenever you come watch or listen to one of our things.

0:55:55 Clip: Thank you so much for listening to Serious Inquiries Only. If you like the show, the absolute best thing you can do is support us at patreon.com slash SeriousPod. The second best thing you can do, is please share it on social media or by word of mouth. Speaking of social media, follow me on Twitter at Serious Pod. If would like to be Facebook friends, send me a request at facebook.com Thomas podcast. If not up for that level of intimacy, that's fine. Maybe you want to join the discussion at facebook.com/groups/SIO members. Thank you again for listening. We'll see you next time.