SIO528: The Marshmallow Test Is Mostly BS, and It's Not the Only Classic Psych Test Like That

Transcript

This transcript was generated automatically and may contain errors — the audio is the record.

0:00:14 Thomas: Hello and welcome to Serious Inquiries Only. This is episode 528, I'm Thomas, and joining me today is Jenessa. How you doing?

0:00:21 Jenessa: Doing pretty good, a little stressful over here in voting land, but luckily, we're not talking about that today, we are talking about much more fun things, hmm,

0:00:30 Thomas: You know, in happier news, I gotta say, not gets to join for like the intro to the show. Out. So I just, I hope, you know, it's how appreciated you are here. Yeah, voting is a little scary. Talked a lot about that on OA And the Supreme Court, just effing with us for fun, just being like, "Well, you not going to say there's no rule a dog can't play basketball yet, but because no dog has quite, like, they haven't signed up." Well, they did try to sign but it hasn't been processed yet. It's the equivalent basically

0:00:38 Jenessa: Yeah, hmmm, a little busy, hmm , keeping me on my toes, you know, just making sure I'm literally waking up in the middle of the night, like drafting letters that I have to send to people.

0:01:03 Thomas: Yeah. Oh. Wow. You send letters? That's a grown-up. You are. I wouldn't even know.

0:01:13 Jenessa: Yeah. Oh, yeah, that's me. Just, you know,

0:01:17 Thomas: How that's not true.

0:01:17 Jenessa: It's got like fancy letterhead and everything.

0:01:21 Thomas: Holy shit. And realize I was talking to the president over here. Well, Mr. President, what are we, uh, talking about today? I

0:01:22 Jenessa: Yeah, not that we're going to talk about kids and the goofy stuff they get up to. Too.

0:01:34 Thomas: Love that. Cell, what's, we're being blippy today or something? That's

0:01:38 Jenessa: Kind of. Child development is like such a fun psych topic, especially because I'm just going to show you videos of kids being weird and doing weird stuff.

0:01:48 Thomas: Most of my life, though, to be fair. What? But also, to be clear, I love it all the time, so I can't wait. Oh,

0:01:50 Jenessa: That's true. Yeah. It's just so fun to see the ways their little weird brains work. I always start child development when I taught this class as like, they're not short adults. They behave in some very unique ways. So super fun. And we can talk about how depending when you went to school, if you took some like intro psych or whatever, everything you learned about child development is wrong. Uh, yeah.

0:02:22 Thomas: We should have started with that, I guess. That'll be the clickbait title and then get ready for We'll do our really surprised faces to put on the video version. God, I hate the world we live in. Anyway, we're a podcast, we don't have to do that, so that's good. Um, No, that's interesting. This was kind of the, the selling point for the episode and I was really excited to hear it because I, while I wasn't a psych major, I know I've absorbed some of that stuff, uh, especially, you know, doing the show throughout the years.

And I, I'm always curious to hear, like, often we don't do that with things we learn. We don'T update when sometime later someone came up with a study that's like, oh, actually, this thing, like, didn't really make sense. It's rare that we correct those things. That's honestly, like, just as important as knowing things is knowing to revise a false thing, possibly. Uh, and so this is, is gonna be a lot of fun. So we're kind of taking a look at some child development stuff, uh, some advances and some, I don't know, not like regressions, but things that were changed there.

0:02:26 Jenessa: Yeah. Um. Yeah.

0:03:02 Jenessa: Updates.

0:03:14 Jenessa: Better understanding. I

0:03:16 Thomas: You go all

0:03:17 Jenessa: Guess is a charitable way to say it. Oh.

0:03:20 Thomas: Right, well, I'm excited to do that. Why don't we take our first break? Support the show, patreon.com/seriouspod, and, uh, on the other side of the break, we'll learn everything there is to know about being a parent from Jenessa. No, sorry, that's not what's gonna be Where do you want, want to start us off?

0:03:31 Jenessa: Yeah, so First one I want to do is, you may have heard of the, the marshmallow test?

0:03:53 Thomas: Yeah, sure have. I have maybe even heard of some amount of debunking of that. That might be one I have heard a debunk of.

0:03:54 Jenessa: Yeah. So I've got a video for you. It's actually my niece and my nephew doing it. Only did a whole minute. It is normally 15 minutes. But yeah, going to wait that long.

0:04:03 Thomas: Nice. Oh my God. If I did this with Remy, it would be zero seconds. There wouldn't be any amount of time that that kid could not eat the marshmallow.

0:04:18 Jenessa: Niece is two and a half and my nephew is five in this video, right?

0:04:21 Thomas: The way, unless we said it's marshmallows for dinner. And then he would be like, fuck no, I'm not eating this thing. Cause that's dinner that someone made for me. And I would never eat that, even though I like exactly this thing that you made me. That's Remy. Anyway, sorry. Let's get back to the show.

0:04:26 Jenessa: No. Now I won't eat ever again.

0:04:42 Jenessa: God, um.

0:04:43 Clip: Stay right there, so you don't touch that one.

0:04:45 Thomas: It's a Hershey's Kiss. That's way better.

0:04:47 Clip: If you don t eat it, then you get two pieces of chocolate, okay? Don't eat it. Alright. Don t eat this cake.

0:04:54 Thomas: They're sitting there with their hands covering their

0:04:56 Clip: Leave it there until Mommy and Daddy are done with the party. And if you don d eat it and they saw that when

0:04:56 Thomas: Mouths. Okay, the older one is telling the younger one what to do. The younger one's already way ahead of this. You know what I mean? So I'm looking at two kids. It looks like a -year-old boy, five and two And a half. And the two and half -year old already her hands on her, covering her mouth, looking at an unwrapped, ready -to -go Hershey's Kiss. And the the older brother's mansplaining, like, hey, you gotta not eat this because, and she's already way ahead, she's way ahead of him here. We go. Let's see what happens.

0:05:10 Jenessa: Yeah. Five. And two. Yeah.

0:05:37 Clip: They re done with their party, and when they see if you didn t eat, you get another one. Only if you don't eat that chocolate. You have to not eat it, okay? Not. Her

0:05:52 Thomas: She's just sitting there with her hands over her mouth the whole time. I feel like the, oh, the ultimate perfect comedy is if the older brother eats it right now, that's the, that is that would be a plus. We got, this is too, what Oh What if it melts, yeah. It's maybe you have to eat it in case, case it melts. Yeah. Oh, she's reaching out for it, she's losing strength, she's, oh man, I just picture the Emperor: you want this, don't you? He's got a little Hershey's Kiss at his little throne thing. Return of the Jedi.

0:06:01 Jenessa: What if it melts, she says

0:06:05 Clip: Mouth. No,

0:06:29 Thomas: Now of course she has to touch it, but she touched it in a delightful goofy way, as a joke. Oh, that's so funny. Oh, these are some cute kids. It's still going. This is two minutes. It's is hold on, hold on, I'll this is getting close. Oh, sheet, boy, this is like those marble races that, uh, his name did during the pandemic they didn't need it and the older brother like, like went to take hers.

0:06:25 Clip: Don't touch it. Did you eat it? No! Good job! So that means you each get another piece of chocolate.

0:06:52 Jenessa: Know

0:06:56 Thomas: You didn't say it was going to be a Reese's.

0:06:57 Clip: Good job listening.

0:06:58 Thomas: Oh wait, is it a Reese? Dude, I would have not eaten so many Hershey's for a Reese.

0:06:59 Clip: Open it! Nice

0:06:59 Jenessa: Yeah. Yeah, yeah. My favorite is when she whispers a second time, "What if it melts? I just can't resist dying." Never

0:07:12 Thomas: Oh man. That was so good.

0:07:14 Jenessa: Never would have made it for 15 minutes. Like impossible. Possible. Love these. They're the cutest. Every time you see the marshmallow test videos, just, I mean, especially, it obviously varies by age, but you'll see one of my favorites, a kid peels it open and eats the inside of the marshmallow and tries to like glue it back together.

0:07:17 Thomas: Yeah. Oh wow. Yeah,

0:07:38 Jenessa: They're not going to know.

0:07:37 Thomas: That's a future entrepreneur right there. So remind us what the typical, like, the classical conception of what this proved was, or whatever, yeah.

0:07:42 Jenessa: Yeah. So the idea was certainly there is an age effect. Two and a half, Hazel is expected to fail this test. At five, Teddy is getting closer to an age where you might be expected to pass, and essentially you wait, and then you get more, So can you delay gratification? Can you control your impulses and wait? And if just looked at it purely from that perspective, like, yeah, kids slowly develop that skill under a certain age. I mean, at some point, they don't even understand the instructions, right? Like they don't even know what's going on, but it got taken to a pretty extreme degree. The first study was in 1990, and then very quickly, the ability to do this was correlated with better social skills, better SAT scores, lower rates of obesity, higher income, better relationships, lower rate of crime.

Like, if you could wait, even waiting 20 seconds longer before you ate the the marshmallow, that meant you were gonna have a better income, you were going to be a better person. Like, essentially just blamed poor people for their own poverty. That you just can't wait, you can't delay gratification. You jump right for rewards. And that sounds like, geez, how could waiting 20 seconds to grab a marshmallow when you're three predict crime when you're 40? It doesn't.

0:08:22 Thomas: Yeah.

0:09:22 Thomas: It seems pretty extreme. I

0:09:22 Jenessa: Yeah, it does, doesn't it? There's a couple of things. First of all, later replications, Every time we've replicated this, the effect size shrinks and shrinks and shranks. Statistical methods are getting better at

0:09:36 Thomas: Picture a disheveled scientist, okay, press press conference, okay, year 30 of this. So it turns out if you can turn down a marshmallow for 35 seconds, that means you're slightly more likely to turn down a marshmallow when you're 42 for seven seconds. That's the best we got right now. It's all it's it's actually all it means. It's down to nothing.

0:09:59 Jenessa: That's kind of what's happening. Like, the better we design the study and the more we control for different factors. It started to shrink and shrink. I think the biggest breakthrough, the thing you probably could have predicted is like how much socioeconomic status affects this. And people real eugenics -y real fast about that. That it's like, oh yeah, you have poor parents because they're bad at this and you're going to be bad at it too. Like it's genetically bad at resisting the marshmallow, um.

0:10:19 Thomas: Right. Right. It really is the perfect, like, two, what's it, what's the thing I'm thinking of where where you just see two different worlds, like two people see two different worlds. It's constant where you're like, oh, well, if people have no resources, then that's going to cause them to maybe make different decisions based on, you know, resource tests like this. And then the other side can be like, no, you see, they don't have any resources because they're their parents are the kind of people who aren't good at prioritizing the bullshit.

0:11:03 Jenessa: Mm.

0:11:03 Thomas: And it's like, you can kind of perfectly, if you want to, you could kind of make either story work in your mind, and it's really frustrating to me, because it pretty pretty obvious to me, like, how much it would affect you if you grew up with nothing, even if you're the same exact, like, everything else, you know, genetically or something. Like, it seems like that's going to have a big effect, right?

0:11:10 Jenessa: -hmm. Mm. -hm. Yeah. Uh, exactly that. It's food scarcity is really what's doing a lot of this. So you think about a kid who who not only food is scarce, food is not promised, things are not guaranteed, but they can't trust that things are coming. Food is not always on the table. Yeah, it kind of makes sense when somebody is like, hey, you got to wait, I'll give you two. And you're like, will you though? It suddenly changes things. So what they did in a follow -up study, this was in 2013.

0:11:43 Thomas: Yeah.

0:11:54 Thomas: Interesting.

0:12:01 Jenessa: Beforehand, kids came in and there was a very similar kind of task, but it didn't feel like the experiment. So you come in and they're like, okay, I'm going to do this. And they're and there were some like kind of busted up crayons in a jar with like the lid screwed on really tight. So the kid was never going to actually be able to get into it and make the the wrong choice here.

The experimenter's like, you can use those if you want, but if you wait, I'm going to bring you like a brand new nice box of crayons. Hypothetically, a kid could try to get to the crappy crayons, but they were all going to fail it anyway. Nobody actually opened it. Then half of the kids, the experimenter did in fact come in with a nice new box of crayons and the other half, he comes back in, he's like I'm so sorry, it turns out I don't have any.

I don't know. I must not have understood. But you can use those. And then he opens it and gives them the like old, you know, dull, crappy crayons. And the kids who were in, they call it the reliable condition. I trust the experimenter. You're a reliable person. They wait on average 12 minutes. And the kids in the unreliable condition wait on average three minutes. Don't trust you anymore. When you said, I'll give you two if you wait, they're like, you're a liar. I don't believe you. And one of the most important things is the effect of age disappeared. Younger kids, older kids, they all did kind of the same thing.

0:13:34 Thomas: Yeah.

0:13:35 Jenessa: So like if this ability to resist, oh, I got to wait, I got to Wait, don't eat the marshmallow. If that was mostly predicted by impulse control, the reliability and the trust with the experimenter shouldn't matter that much. Like whether they trust you or not, they should just be staring at the marshmallow and I can't resist and then they take it. It When they knew they could trust you, little kids waited like 12 minutes, which is a long time if you're a a little kid, you know? So that really blew this out of the water that, That's what's going on is kids from lower socioeconomic status households.

They don't have trust built up around the idea that when you offer me a piece of food, I know a second piece of food will come if you promise me. They dive right on it. There was four children of non-degreed mothers, what a weird way of saying that, um, in one study they waited less than 20 seconds. Was a single mother household, mom doesn't have a degree, the kid just dives on it, they don't wait at all. And don't think the difference between waiting 20 seconds and waiting 30 seconds is like, that's that's what'll tell me about your SAT scores. Like, no.

0:14:07 Thomas: Yeah.

0:14:51 Thomas: Or If it does, it would, probably do it through an alternate means where it's like, yeah, where it's like, well, that means that you didn't have the resources and therefore you didn't get the SAT prep and you didn't, you know, all that stuff.

0:14:52 Jenessa: Right. And mediating Right. The causal relationship is not the direction people are thinking of it. It certainly does have a correlation. Yeah, It's just not because it's impulse control that the kid who can wait 30 seconds, that's the real go-getter. Um, it's that's the kid who is supported at home and whose family can afford to support them at home. So um it's cute, it's fun, but like it has a super racist history of how it's been weaponized.

0:15:12 Thomas: -hmm. Yeah. I bet.

0:15:36 Jenessa: So um, good for seeing at young ages, watching impulse control develop, but it is so deeply confounded by other factors that I struggle to imagine the utility beyond that really basic early development of both impulse control and trust. A tough one.

0:16:00 Thomas: One for the eugenicists, I guess. False

0:16:03 Jenessa: Yeah, big minus one on that one. I another one for you. You might have heard of the Sally Ann test, or sometimes it's called the false belief test.

0:16:14 Thomas: Oh, it's someone going to look for this thing and they think it's somewhere. Yeah. Okay.

0:16:18 Jenessa: Yup. Yup, yup.

0:16:19 Clip: This is Anne and this is Sally. This is Sally in Anne's basket and box. One day, Sally and Anne were playing with a block in their house. Then Sally decided to put the block in the box.

0:16:39 Clip: Then she decided to go outside and play where she couldn't see or hear anything, right? Yeah. Okay. Now Ann, that tricky Ann, decided to take the block and put it in the basket.

0:16:56 Clip: Now since Sally is outside, did she see Ann move the block? No, she didn't, did you? So when Sally comes inside, where will she think the block is? Do you think it's in there? Why will she thinks it's there? Because it's out there.

0:17:17 Clip: Because it is in there. Because it in there, okay. Okay.

0:17:20 Thomas: Yeah. That's exactly what I remembered. Oh

0:17:24 Jenessa: Keep playing that, you would see a four and a Half year old pass the test clearly. And the first one we hear is a three year old not doing so good. She says, where is she going to look? She says in the basket. Why? Why? It's in there. I don't know.

0:17:40 Thomas: Do you want from me, lady?

0:17:42 Jenessa: So the initial perspective on this is this demonstrates theory of mind. Can you take the perspective of another person? I'm going to put myself in the mind of Sally. And if I'm older, I can say, oh, yes, I know where the thing is. But Sally doesn't. So what does Sally know? Sally knows it's in the box. So Sally would look in the box, even though I know it's the basket. Some problems here.

0:17:48 Thomas: Makes sense. Really? Hmm.

0:18:13 Jenessa: Yeah, quite a lot. So the biggest thing is this is extremely linguistically challenging.

0:18:20 Thomas: Yeah. I was thinking that as we were watching that, I was like, like Marty.

0:18:24 Jenessa: A lot of instructions here, right? One of the things that kind of tips us off to like, oh, this might be a language problem. Deaf children born to hearing parents often have language delays. They pass this much later than other children. And there's not a lot of reason to think they should have a theory of mind problem. Problem. They're just seeming to struggle with the language part of the question.

So then we want to step back and say, okay, so is this three-year-old not answering it right because she can't put herself in the mind of Sally, because she cannot understand Sally's thinking process? Or is there something else going on possibly? We have to get away from these explicit measures, tell me your thoughts. A pretty common thing in child development, that they can't tell us their thoughts.

They're not very good at it, or they're terrible at it because they can'T even talk at all when they're really little. So we do a lot of things with infants where we have to get implicit measures, right? To do it is this thing with preferential looking.

Looking, we do this with infants a lot. Um, example, you can show that they have a basic understanding of physics pretty early in life, so will look longer, and like, you know, we can quantify this, but also if you're a parent, you can just watch the surprise face. Um, look longer at a ball that rolls up a hill instead of down a hill, or if it rolls behind an object and and doesn't roll out the other side, the look longer, like, whoa, something happened.

What happened behind the tree? So it's that they are starting to understand how objects should behave. That's interesting. And clearly, they understand that long before we can ask them verbally. So we had to find some way to get at it. So they did this with literally infants.

Infants, I can't ask you anything, I'm just going to move the dolls around. I'm going to show Sally put the marble in the box, I'm gonna show Sally leaving, Anne is going to come in and move the marble, and then if Sally comes in and looks for it where Anne put it, as if Sally is somehow psychic and knows that it was moved, infants stare at it longer, like, that's weird, how did she know? Of tipped us off that, like, huh, that

0:18:24 Thomas: Yeah, exactly.

0:20:54 Thomas: That is interesting.

0:20:54 Jenessa: Doesn't seem right, right? If you have to be four and a half to answer it verbally correctly, but an infant is showing me cues that maybe they get it. It. How? What?

0:21:07 Thomas: Boy, that is really surprising to me.

0:21:09 Jenessa: Isn't it? It's really weird. Also found that instead of saying answer it verbally, can you just point? You get way more three -year -olds who pass it. So is pretty common in child development that kids will like say one thing and do another thing with their body. Body. Common to have them correctly answer questions with everything but their words, their body showing you that they understand. They will gesture correct answers very often. Super frustrating if you're collecting data with kids. I've had this before where it's like, put your hand up when you see the thing and then say, they're also supposed to say yes or no. And they put their hand up and say no. You're like, damn it.

I'm to to be writing down what you said, not what you did, but like we were using the hand as like a way you could try to remember. And like, I don't know what to write down now. Are frustrating in the lab. One way they got at this, somebody a clever hypothesis. This is something we know even adults have called the curse of knowledge. So if I were to to ask you before a hockey game, who do you think is going to win?

0:21:10 Thomas: Yeah.

0:21:27 Thomas: All right.

0:22:22 Jenessa: Could guess and you could tell me all kinds of reasons why. And if I said, um, do you Think the average hockey fan thinks? You might give me a different answer, you if I ask you for the first time ever after the game, when you already know the outcome, what Do you think the average hockey fan thought was going to happen.

0:22:31 Thomas: Maybe.

0:22:42 Thomas: Right, right.

0:22:44 Jenessa: Changes people's answers. It shouldn't, right? The question didn't change. And supposed to be answering based on what people knew before the outcome was known, but it does change it. Even like total upset surprise outcomes, people will be more More likely to say that was predictable when it, like, you was not predictable.

0:22:45 Thomas: Yeah.

0:23:07 Thomas: Just, I mean, for many reasons, but like, for it sounds like, you know, people are really bad about remembering what, what they thought when, you know, it's, that's really bad, but also availability heuristic. I guess once once you see something actually was possible and happened, you're definitely going to wait it differently.

0:23:13 Jenessa: Oh, sure. Yeah. Mm -hmm. Yup. Right. Exactly. So we know adults struggle with this, too.

0:23:26 Thomas: This is called the, how everybody predicted Trump in the first election syndrome.

0:23:30 Jenessa: Yeah. Right. Right. And we also know it affects memory. We go back and edit our own memories that way, too, too. So what hypothesized was, I wonder if this is happening to children, the curse of knowledge. They know where the thing is. So maybe it's just so hard to resist the impulse to give the correct answer. She says, where is she going to look? And she's like, in the basket, because that's where it is. I know where The Ball is. It's in the the basket. Maybe that's what's happening. It's a cognitively complicated task. I'm trying to remember everything that happened. I am three. I don't even want to be here.

0:23:31 Thomas: Yeah. Yeah.

0:24:12 Thomas: I mean, that'S kind of part of it though. Or is it just that the experiment is claiming that it's theory of mind and you're just saying there's other explanatory factors.

0:24:18 Jenessa: Right, right. It's certainly interesting to see when they can and can't fully verbally explain. It is not like like nothing. But to make the claim that that means the kid can't put themselves out of their own mental state, that's a pretty bold claim if the problem is actually language. So what they did here is four locations. So instead of just the box or the basket, it gets taken out of the box and it could have gone in four possible occasions. They did four different colored boxes.

In one example that, you know, you hear there the experimenter has a script they're following and they always do that. They say the same script every time. In one group, they're going to say she could have put it in here or here or hier or hier. But we know it's in the blue box. And in the other group ,they say it could have been here or her or her. And we don't know what box it'sin.

They don't know where it is and then they say, where will Sally look? Three -year -olds pass the test. If they know, if they were told, we know it's in the blue box, not only do the three -year olds almost all fail, but 80 % of the time they say she's going to look in the Blue Box, not the green, not the red, not purple.

She's going look in a blue box because that's where it, it is. So the problem seems to be this curse of knowledge thing. When they know, they're three, they just can't resist this urge to tell you the true, correct answer. They can't get over the fact that they know, and they are trying to predict what someone will do knowing what they should do. But if they don't have the information, they will pass the test. They will say, "Oh, she'll look in the the basket because that is where she put it, not where it got moved to later." If was they don't

0:24:21 Thomas: Okay.

0:26:19 Jenessa: Know, they should guess randomly. They shouldn't all of a sudden be passing the test. Tried a lot of variations on this and seen it really matters how you set the test up, how you You set the question up. What if I do this? What if I put it in my pocket? A really interesting one where the earliest studies of this were very specifically looking at the differences between neurotypical and autistic children. Critically, this is in the 90s, 80s, more in the 80s.

That time, we still had an Asperger's diagnosis. So really, when we talk about the early studies, you should think, when they say like autistic people, what they mean is white male children with an autism diagnosis and most likely an intellectual disability and a language delay. They were almost always going to take the kids with normal or above average IQ and give them the Asperger's diagnosis and put them in another group.

You do always want to check what the sample was for any of those old studies, or just look in new studies, because what did they do? Autistic kids from those 80s studies did tend to pass later in life. Like, if, you know, you're supposed to pass when you're like somewhere around like four and a half, five and a half, somewhere in there. Autistic kids were passing at like six or seven or eight. Okay. Do they struggle to get theory of mind?

Except when you had the kids with normal or high IQ, they were passing, or normal or high verbal skills. They were passing at about the same age you would expect. And then some researchers were saying, like, "Oh, actually, what they're doing, they're using verbal reasoning as an atypical strategy to, like, hack their way through it with logic. They're not actually doing theory of mind," which

0:26:19 Thomas: Interesting, what is theory of, don't mind don't the force, they're

0:28:06 Jenessa: I mean, like, that to me feels like such a cop-out to be like, well, they did the behavior I expected, but they're still wrong somehow. Um, The one that for for me really sank it. One researcher said, "I wonder if the problem is just that this is a social situation. There's a human telling you a story." So they had one group do it the traditional way and another group, the task was presented on a computer. And when it was presented on a supercomputer, the difference in age for autistic and neurotypical children disappeared.

0:28:13 Thomas: Still autistic about it,

0:28:38 Jenessa: Yeah, it was really just the problem all along was that a person was sitting there telling them a story with dolls and trying to be like, "Look at the doll." And they were like, "No." So that a 2014 study. Computer changed everything. It's unclear exactly why. Maybe the computer is more engaging. Maybe the social situation is very stressful and interrupts cognitive ability, that you're stressed out. Um, it is literally not wanting to, like, look and engage in the social situation, unclear. Um, yeah, that was a pretty big development, like, "Oh, they do it at four or five if you just use the computer instead." Um, yeah, for like decades we were like, "Oh, this means they can't take your perspective until they're eight or but also so not when they're adults." Then it turns out that just needed a computer to do it.

0:28:38 Thomas: Yeah, huh, that's a lot lot you threw at me. So there's the, you're saying there's the Asperger's autism distinction, but didn't apply to the 2014 thing, right?

0:29:54 Jenessa: That is mainly to, if we're trying to understand why in the 80s, were we like getting this big distinction, the kids with normal or high IQ and normal or high verbal skills were excluded from a lot of those studies.

0:30:09 Thomas: You're Saying they wouldn't be counted as autistic in those studies. Like if, If you're like, hey, we're studying autistic you're saying it would be a selection of autistic kids who might have had like learning challenges or something.

0:30:21 Jenessa: And especially if we know this is a very difficult language problem, like it's possible when kids, if they point instead OF answering verbally, they're more likely to pass. And then we have a group of kids that we know on average have language delays, like, oh, well, it's not super surprising that they didn't pass it until their language skills caught up. Might be part of it. And then when we took the social situation out of it, that also really helped for a lot of autistic kids. For younger kids who like neurotypical autistic doesn't matter, or three -year -olds generally fail in the traditional original setup, the video I showed you, the three -year -old failed it. If we tweak that setup to point instead, to have no knowledge of where it actually is, just knowledge that it moved, three -years -olds do start passing it. So this might not be a like four and a half, that's the year that you pass the theory of mind test. Like, oh,

0:31:04 Thomas: OK, I was going to say, Because, like, ultimately this is a little bit different than the last one, where it's like, well, yeah, but there is something here. Like, there's, there is an ability to put yourself in someone else's mind that obviously isn't there immediately and eventually is there. So it's not like it's studying, like, an insane thing. It's just maybe the, the previous, like, the first study was, it, pretty, like, direct and, like, oh, it's this age where this officially starts, kind of thing. Okay.

0:31:49 Jenessa: It was like three year olds don't pass. You should start passing somewhere between five and six. If you don't press it by seven, you have a disability. Something's wrong with you. And it's like, hmm, might just have a language problem. Like, it's children born to hearing parents are passing it late, and no reason to think they have a theory of mind problem. They have a language problem getting in the way of that, on average, of course. So yeah, there's a really big language confound on this one, and many, many child development tests. This is one example. We will see this happen with the other tests we're going to talk about, and many child development tests, there's just something about trying to get a kid to verbalize what they're thinking that is so hard.

0:32:39 Thomas: No.

0:32:40 Jenessa: And they get halfway through saying it, and then they start talking about something else, and it's all over.

0:32:40 Thomas: Freaking kidding. Boy.

0:32:47 Jenessa: So mostly that this test isn't the best way to get at this question of there is a very complicated skill and a multifaceted skill of understanding how other people think, but purely isolating it down to being able to say, well, I know X, but you do not know X, is interesting. Interesting. It seems like infants even do it with preferential looking. We seem to see them start getting it, I Know, yeah.

0:33:24 Thomas: That's, I will say, I doubt that one more than the one that you're debunking, you know what I mean? Like that, I, I get it, but like that result is so crazy to me, or so, like, it just seems really, really, it seems really slight, seems really tough. I don't know. I'd want to, I'd want to dig into that more.

0:33:39 Jenessa: Sure, yeah, of course. I'm always happy to send these over. Um, interesting to think that it's, um, like if we could distill it down to its purest form of, of just like, if I could somehow not need to get you to say something and I could just know what was happening in your brain, how soon on average do kids get it? And then more importantly, how does that evolve into like something way more complex, like empathy, you know, like your ability to, because I mean, really, who gives a shit about if I know that you know that I know? It's more, more about caring that I know that you know, right? Mean, I guess it's incognitively important, but it's part of the process, right. So yeah, I think the biggest thing, the biggest takeaway as we've slowly evolved our understanding of this one is the OG test development, the one that I showed you, it's just too simplistic. We have extrapolated from that one test, very similar to of the marshmallow test. Like somehow I'm going to predict the likelihood that you'll commit a crime from resisting the marshmallow for an extra minute.

Trying to predict so much from the exact age that they verbally answer this correctly with this one very specific design. And what we end up finding is, yeah, when we tweak the design a little bit, we get a a different answer. Same thing is going to happen with the other one I got for you is water conservation. You're familiar with that one?

0:33:55 Thomas: Right. Yeah.

0:34:21 Thomas: It's just part of the process. Yeah.

0:35:15 Thomas: Conservation. I don't, I don't know.

0:35:17 Jenessa: It's not the environmentalism. It's conservation of matter. When do they figure this out? So I have a video for you that's rather long. We can cut it wherever you feel. Super cute, though. This kid's about four.

0:35:23 Thomas: Oh, okay.

0:35:40 Clip: Okay, does this room have more quarters? Does this room has more quarters, or are they the same? The same. The same? Okay, now why?

0:35:50 Clip: Now, does THIS room have MORE quarters? Does THIS room has MORE quarters, or are THEY the same. That one has more quarter. That one HAS more quarters. Yeah. Why does that one have more quarter?

0:35:57 Thomas: So all she did was like space them out more.

0:35:59 Clip: Because it's stretched out. Because it is stretched out? Correct. Okay, so how many are in this room?

0:36:08 Clip: 3, 4, 5. Okay, how many in this one? So, are there more in this row or are they the same? The same. The same? Yeah. Okay, so we have two sticks.

0:36:15 Thomas: Is that the same or different? You idiot.

0:36:23 Thomas: Well, that's weird. Cause you told me me a second again the point of this is just to insult kids.

0:36:25 Clip: Is this stick longer? Is this still longer or are we the same or are you the same, the same.

0:36:28 Thomas: Okay.

0:36:32 Clip: What about now? Is this longer? Is this the longer or the same stick is longer? The stick is the longer?

0:36:41 Thomas: Well, this one this experiment I don't like because you're just kids are just fucking they're just there trying to figure out why you're making them sit still and you're like, is this one? So all that happened in that, sorry to use some theory of mind for the listener, um, two sticks that are the same length and And she asks, is the same, is it the same? Then she just moves one to her left. And it's like, okay, which one's bigger? And for some reason, this kid is like, that one's bigger. Which doesn't make any sense on any theory of any anything.

0:36:41 Clip: Yeah.

0:36:45 Jenessa: Yes. Yeah. Yeah.

0:37:12 Thomas: And I know from having kids that this is just they're trying to figure out what the hell you're doing to them right now. They're just like, I don't know, when is this going to stop? Can I eat something? Can I go play? It seems like they want me to say that one is longer probably. If anything, it's more theory of mind, even though that's not this, uh, experiment. Yeah.

0:37:32 Jenessa: Well, They are trying to figure out what you're up to. I think you're exactly right.

0:37:34 Thomas: They're trying what you want to get out of this. Hmm.

0:37:35 Jenessa: Yeah. So they go through so many iterations. They do it with a ball of play-doh and then I squish it flat. Classic The one is water in a glass. So you put two glasses of water.

0:37:43 Thomas: The the meme or whatever, where it's like the taller cylinder and

0:37:50 Jenessa: Yeah, that's exactly what this is. So when it's two glasses water that are the same shape and the same level, and they say it's the same, and then you pour it into a taller, skinnier glass, so the level is higher, they'll tell you, oh, there's more water in that one. And they watched you do it. And then you poor it back and they go now they're the same again. So it's supposed to to be saying like, oh, they don't understand it's the same amount. Um, you've ever worked with kids, you know, like, oh, she has two and I have one, and you crack their graham cracker in half and they're like, now I have two. This is fair.

0:37:52 Thomas: Yeah. Yeah.

0:38:21 Jenessa: Um, One thing that leans me towards a little bit of they might actually be misunderstanding is the fairness part of, like, if deep in their heart they're like, it's It's fair now.

0:38:21 Thomas: Yeah. Oh No, I don't get me wrong, they definitely are idiots when it comes to some physics stuff. I'm just saying that stick one is dumb, like I don'T, that's the only thing I'm saying, like a lot of these it kind of makes sense, but like just, I just know it's hard to explain, but as someone who has these maniacs every day, kind of the cadence and ordering of how she's asking that, like, I knew he was gonna say that was longer, like, just no matter what.

0:38:41 Jenessa: Yeah, yeah, yeah. Hmm. So this is what's super interesting. Like on the one hand, you would think, oh, my God, they care so much about fairness. They must actually be confused. On the other hand, studies have done exactly what you are talking about to try to reveal this is a misunderstanding issue.

0:39:04 Thomas: Oh, really? Okay, so I was so I Was saying that this stick one is dumb, but I actually believe that they don't understand a lot Of this stuff because of the fairness thing that you're talking, but you're saying that might not be true.

0:39:25 Jenessa: A little bit of both. Once again, it's in the land of like, if you took intro psych, you were probably, depending on when you went to school, you're probably taught that like four -year -olds cannot do this. Five -year olds can do this, their brain has developed to where they can do this. And it is so much more complicated than that.

0:39:26 Thomas: Okay.

0:39:44 Thomas: Yeah. Well, that, I think that's an important frame to put on this though, because I do think I, I maybe went into like the typical, like debunking, like, this is all like, you know, I just, as a podcast listener, I went into the, like, Jenessa is going to debunk all of psychology or whatever, but it's more like, no, you were taught like, this experiment established this exact skill happens at four and a half. And it's, it's like, yeah, not really, but it doesn't mean that the whole thing is bunk or whatever except the marshmallow thing, which I kind of think is, yeah,

0:39:48 Jenessa: Yeah. Right, right. That one I lean more towards it being like holy nonsense. Mean it's still useful for figuring out when they can resist an impulse but with 500 other caveats. This one too. So for For example, kids who regularly interact with water in vessels, like kids who have to go get water from the well because it doesn't come out of the tap, they pass this way sooner because they have poured the water from vessel A to vessel B over and over again. They at four will be like, yeah, I get it.

It's the same because it moved back and forth. If this was a brain development thing that just goes in a certain timely pattern, there's certain brain development things that really are just like, they happen at a certain stage and it's really hard to speed it up or slow it down. This one, if you can speed it, it might be an exposure type of skill rather than just progressing ...through, like, language is one of those where they go through phases that you just can't seem to get them out of. Um, might not be one of of those. Not as religiously, at least.

0:40:15 Thomas: Yeah. Sure.

0:41:06 Thomas: Okay, that makes sense, that's

0:41:29 Jenessa: Also did it where instead of the experimenter manipulates the object, the kid does it. So if they mash the ball of Play -Doh, way more likely to pass.

0:41:39 Thomas: It, I love that, that's such a good, like, thought is it, I let, I love, you scientists, I love when they think of the different ways to test stuff. Sometimes you're like, oh, that's really smart. Yeah.

0:41:49 Jenessa: Got to think about what's going on in that little brain. Of the issues is very simply, and I think this is what you were getting at, if you ask the question twice, twice. They think, I must have gotten it wrong.

0:42:00 Thomas: Yeah. They think something needs to change or something. Yeah. Or

0:42:05 Jenessa: Quiz. I must've done it wrong, I'm going to give a new answer.

0:42:08 Thomas: Something had, like, if you asked it and I gave an answer and then you did something and asked again, like, they just expect, like, well, things are different now, so I guess it's probably the other answer, you know? Like, there's a lot, kids just, they do a lot of that stuff. It's hard to explain, you've kind of said it, but they do say a lot of stuff that's not really, it's not really what they're thinking. It's, I think so many of the parents listening, I think will probably identify with that, where it's like, yeah, you can't, this isn't gonna work. I mean, there's some form of it can, and I think you doing the different experiments is cool, and there's probably better and worse ways to get at it.

0:42:19 Jenessa: Yeah, yeah, yeah.

0:42:41 Jenessa: Right. Something going on where almost wonder sometimes if it's like they're a little too smart for their own good, because you couldn't possibly be asking me a question this stupid. Like, I have to say something.

0:42:54 Thomas: Yeah. That's where I wonder if autistic kids would beat the shit out of normal kids on this. They'd be like, no.

0:43:01 Jenessa: No.

0:43:01 Thomas: It's still, they're still the same. Like, honestly, that was, I was kind of that kind of kid a little bit. I don't have autism that I know of, but I have ADHD and, you know, kind of related. I feel like Arlo's that same way. Son with, autism, with, I just feel like there's a certain amount of, like, not getting distracted by those kind of parts of the interaction that I think is really interesting and can be, I think can be a real asset in, in a lot of situations. Conversations.

0:43:24 Jenessa: Yeah, there's like this sort of cooperativeness of like, not assessing what is the true answer, but what is that answer you're trying to elicit from me? I will give it.

0:43:29 Thomas: Hmm. Yeah. Right. We're playing a little game here of some kind. It's an interaction.

0:43:37 Jenessa: Yes. Yes. And some kids are just less likely to play along. They're like, I don't know what you're up to, but

0:43:38 Thomas: Yeah. Yeah, but these sticks are the fucking same asshole. Anyway, I want to go play video games. I'm Arlo. No, I'm just kidding.

0:43:52 Jenessa: So one of the ways they got around this that sort of avoided producing that like social desirability kind of response, they set it up. So it seemed more like an accident. Rather than like, I watch me, I am intentionally moving the stick. They just like, oops. And then they're like, oh, are they the same? And they're like, yeah.

0:43:53 Thomas: Oh, I love that. That's so cool. I love that. I had such a cool way to get at it. Yeah I did it.

0:44:16 Jenessa: Isn't it? Yeah.

0:44:18 Thomas: So are you saying it like drastically change the results or, okay.

0:44:20 Jenessa: Yeah, four -year -olds can pass it now.

0:44:23 Thomas: So it wasn't just like a minor change. It was like night and day.

0:44:25 Jenessa: It's on average, right? So already some four -year -olds were passing with the original setup, but with this new setup, four -years -olds are significantly more likely to pass.

0:44:26 Thomas: Sure.

0:44:35 Thomas: Right. Yeah. I was trying to get at the size of the, uh, I always forget if it's effect size or significant, I guess the size.

0:44:41 Jenessa: Size is probably what you're looking for. Definitely statistically significant. What is the...

0:44:41 Thomas: Yeah, yeah. Otherwise we wouldn't be talking about it. Yeah

0:44:46 Jenessa: Right, right. Yeah. I don't actually know what the effect size was on that one. It was running through a list of all the different ways they've manipulated this. One of my favorite ones was instead of the experimenter doing it where you've set up this social I like you relationship, the naughty teddy bear did it and it worked. The kids were more likely to pass because they didn't think you were trying to get me to go along with whatever you're doing.

0:44:52 Thomas: Hmm. Sure. Sure, yeah,

0:45:17 Jenessa: So there's definitely something going on.

0:45:16 Thomas: I like that. I also love that because it's sort of like, I mean, I, and this could be, I'm just pontificating here, but there's something interesting in that, you know, I instantly was reminded of was like church where you're like, if you're in the social situation where everybody's like loving one another and you're like, and you know also Jesus's blood and is whatever. And you're like, yeah, okay, sure. Yeah. All right. But then it's like, you've set it up where the evil bear is like, you you're, you're more critical in situations where there's, it's like adversarial potentially, or like it's not your in -group or something. I don't know if that's anything, but it just feels like it makes sense to me where they'd be like more clear -eyed about it.

0:45:59 Jenessa: Know, that actually, I wonder, um, I didn't come across this, but I would not be surprised to see if the experimenter was outgroup, like that would change this. If that relationship between you and the experimenters is somehow breached, that suddenly they're like, I am not participating in whatever you're up to. Um, I have seen some criticism of these specific manipulations, the either it's an accident or the naughty teddy bear shows up and does it. Um, might be like such a delay between is this one the same or this one? Okay, slight movement. Now is this this one the same or different, or that maybe the delay is just like a forgetting thing that they're, Don't remember what you did. But like, I don't know, at the end of the day, they're still looking at two sticks and giving me an answer. So didn't feel like the strongest criticism to me. But I do take their point that like, you have quite significantly changed the setup at that point.

0:46:09 Thomas: Hmm. Yeah. Yeah, yeah, yeah.

0:47:01 Thomas: No, I could see that.

0:47:03 Jenessa: It's interesting. The biggest things with that is you probably saw the kid do it on the stick one. He's gesturing the shape. He kind of holds his hands out and he holds it up to the one stick and then he moves it over to the other one.

0:47:14 Thomas: Don't think I got that far. Does he do that after?

0:47:17 Jenessa: Maybe it was on a different one, 'cause he goes through, he does it with Play-Doh and water and graham crackers and every object you can think of to really drive it home that he's, he's not getting it. Is a tendency to once again, answer verbally incorrectly, but gesture correctly. So with his hands, I didn't change the shape. I, they're the same and say they're different. So that is a thing that will show up again and again in child development. It'll often be called embodied cognition, showing with the body that you understand. Will sometimes even be a predictor. So when kids finally start, they're still verbally answering wrong, but they start gesturing the correct answer. A predicter that they're going to start verbally answering correctly soon. They're like almost there. We're getting there any minute now.

0:48:13 Jenessa: Cognition might be evidence of the development, or it might be evidenced that they do know the, the answer and we're just working our way linguistically up to it. So essentially, I mean, this is one, it's been in the textbooks since the 70s, it's a famous test and it kind of still holds up that there probably is some level of a physics misunderstanding happening that they will eventually figure out

0:48:40 Thomas: I Well, yeah. I mean, why wouldn't there be?

0:48:41 Jenessa: Sure. Um, the, like, religion around it that it's going to happen and at age five is like, no way, they are definitely also verbally struggling here and they're definitely socially participating and you can also teach it. You can teach them out of it in a way that, like, certain other developmental things you absolutely cannot teach your way out of, that they just have to go through it or not. So it's an interesting one for sure and all, but it is, is just so cute.

0:48:49 Thomas: Right. Yeah.

0:49:15 Jenessa: I mean, adorable.

0:49:15 Thomas: Yeah, these are these are fun. Well, this has been really fascinating. So it's not as though all of, you know, child psychology or psychology is out the window or anything. But interesting to what you might have learned in undergrad or whatever, is not quite as certain as it used to be. Anything else we can take from all this for

0:49:17 Jenessa: Yeah. No, no. Yeah I I would say anything that you learn in an intro level class, even the teacher knows they are teaching you oversimplified things and there's more to it. There's more nuance. There's more layers. You just want to always be open to hearing about updates and nuance and complications because we can't cram all of those details into like 12 class periods in a college course.

0:50:01 Thomas: Sure. Well, this was cute and a lot of fun. Thanks, Jenessa. Oh, shit. Thank you. Actually, that's really good that you said that, because I blew it like an asshole last year. I think I didn't get any of my shots, or maybe I got them really late. I might have gotten really late.

0:50:06 Jenessa: Here we go. I'll just say random bullshit. It's that time of year. Get your flu shot, everybody.

0:50:16 Jenessa: Better get it done. Like an updated COVID vaccine is out. Like get your booster. Shot. It's time.

0:50:22 Thomas: And I'm not going to download that until I know that the patch worked. No, I'm just kidding. Like it's a software update. No.

0:50:30 Jenessa: Oh no.

0:50:31 Thomas: Uh, really good reminder, actually. Thanks again, all let- I love you, scientists. I

0:50:34 Jenessa: Talking to you.