SIO321: Does Giving Money to Mothers Boost Baby Brain Activity?

Transcript

This transcript was generated automatically and may contain errors — the audio is the record.

0:00:11 Thomas: Hello and welcome to Serious Inquiries Only. This is episode 321. I'm Thomas and my rapid test tells me I don't have COVID. Just a slight cold for now. We'll see. Yeah. How are you doing?

0:00:20 Lindsey: Congratulations. I'm a loser today, so I'm not feeling great, but yeah. I

0:00:24 Thomas: Yeah. An inside reference that... See, here's the problem with that is only you and I know what you're talking about. So if we have a really interesting episode about giving money to people, which is a thing I'm a fan of. In fact, I read a book by, [FLAG] wrote a book called Give People Money. It's how universal basic income would end poverty, revolutionize work and remake the world in 2018. I read that back then.

0:00:45 Lindsey: Yeah. Are you sure about that? Are you certain? Are you, are you sure it wouldn't just make people do more drugs? I don't know.

0:00:55 Thomas: Yeah. Well, but think about the jobs that creates. No one ever talks about that. The drug jobs. Anyway, I remember reading that and liking it. And I generally as a progressive person, and I'm pretty in favor of just giving people money. So that's where my biases are. But there's a study thingy for us to look at today.

0:00:59 Lindsey: Oh, that's true.

0:01:16 Lindsey: Yes, there is. There was a New York Times write-up about this. Most obviously the relevance of the study we're going to be talking about is to the, you making the case that the expanded child tax credit should be retained. And that's the angle that the New York Times took on reporting it. It's why I was interested in digging into it. So this study is titled The Impact of a Poverty Reduction Intervention on Infant Brain Activity. And it was published just this year, 2022, by Sonia Trawler-Renfree and colleagues.

This paper describes one preliminary set of analyses from a fairly large, So there's a thousand families represented in the study, longitudinal study called Babies First Years, which focuses on children from low income families. And the most exciting thing about the study is that they are investigating experimentally questions that up till now have only been tested correlationally. So, they're looking at the effect of poverty on child development. And, you up until now, the research on this has been correlational because how do you manipulate poverty, right? Like in order for something to be experimental, you have to have an experimental control group and randomly assign people to each group. And that's not easy to do if

0:01:27 Thomas: Yeah.

0:02:06 Thomas: What you mean. Yeah.

0:02:25 Thomas: Yeah.

0:02:25 Lindsey: You're talking about something like poverty. So, know, you there's a big literature on this question, but yeah, it's all correlational. So anyway, we'll talk about how they approached this as an experimental question a little bit. But this is a potentially very exciting study because they can add the strengths of experimental evidence, which allows you to make causal inferences to complement the existing correlational evidence, which has demonstrated meaningful associations in the real world between poverty and a variety of outcomes. Right. But they can't ultimately demonstrate causal impact. So this study really could end up informing policy in a pretty meaningful way and potentially help progressives make stronger cases for a variety of social welfare programs, including the continuation of the expanded child tax credit. Right.

0:02:26 Thomas: Yeah.

0:03:12 Thomas: We'll get to the politics part of this later. Should do the science first.

0:03:14 Lindsey: Okay. Okay, all right, we can do that.

0:03:18 Thomas: Note that pessimist Thomas rates that previous statement as not going to happen. But more on that later. They,

0:03:25 Lindsey: Yeah, unfortunately, that's true all too often that the, you know, the evidence doesn't inform these discussions as much as it should. But yeah, we'll see what happens. So I'm really interested to follow the progress of this study as they continue to release reports. Because again, this is, you one of only four publications that have come out so far from baby's first years. And it's the only one so far that presents actual data from their participants. So the first, like two of the reports are just about methodology. One of them describes a pilot study. So this is the first study that they actually published from these participants.

0:03:56 Thomas: Issue a whole report that's just methodology? Who reads that? But, you, it's just, you isn't it? Did, They hand deliver It to you? They're like, "Here, I know we only made one copy, it's for you." That,

0:04:02 Lindsey: It's, uh, they, they are definitely for the benefit of other researchers who are, who are tackling these, these questions, right? Like one of them is about how do you,

0:04:11 Thomas: One, Didn't, Have a New York Times write-up? I'm guessing. Oh,

0:04:14 Lindsey: No, No, like, one of them's about how, how do you use these, like, mobile EEG things to do to collect data from babies? You know, because that's a hard thing to do, right? Yeah, so this is like very sort of initial findings, but the study itself I'm really excited about. So, so anyway, the, the New York Times reported on this study in an article headlined "Cash Aid to Poor Mothers Increases Brain Activity in Babies, Study Finds." So let's see if they actually found that. Um, think that they recruited their sample, um, people who are pregnant, so that they could take, uh, like, take measurements right after the baby is born, um,

0:04:23 Thomas: Nice. Yeah. Yeah.

0:04:42 Thomas: Right away why poor mothers? I'm just wondering about the gendering of that, so, It wasn't like, like, were they targeting, like, single mothers or anything? Or is that, okay, so it's just, okay. Interesting.

0:05:02 Lindsey: No, Yeah, Yeah, it's just their, just their sampling method, I think, recruiting people who are pregnant.

0:05:09 Thomas: Would that headline, could it also just say, like households, essentially? And would it still be true? Or is the study specifically like, no, you have to you gotta hand the money to the mother. Like, it's gotta be the baby has to see it. And then they're like, oh, okay

0:05:19 Lindsey: They're the more responsible ones. Yeah. Yeah, no. Yeah that's the mechanism. First just a little bit of study background. So there's this large existing literature that suggests an association between poverty and a variety of negative outcomes. I'm sure that's a shocker to people. So economic disadvantage has been linked to lower school achievement, educational attainment, and adult earnings. And in childhood and adolescence particularly, it's been linked with lower scores on assessments of things like language and memory, self-regulation, and social-emotional processing.

Some studies have directly focused on these cognitive and behavioral outcomes, but others have focused on documenting the neural underpinnings of some of these negative effects and have found a couple of interesting differences in terms of both brain structure and function. So growing up in a lower income household is associated with less surface area of the cerebral cortex, particularly in regions that have been associated with language and executive functioning. So presumably that could underlie some of the cognitive and behavioral differences that have been found.

In this, you know, the mechanisms here, right, there could be a variety of mechanisms that link poverty to to these, to these differences. But one interpretation is that children in poorer households are like receiving reduced stimulation in critical areas,

0:05:27 Thomas: Yeah. Yeah.

0:06:44 Lindsey: Right? So poorer parents are going to have less expendable time to do things with their kids that might facilitate the development of those skills. You know, talking to kids is really important. Reading to kids.

0:06:53 Thomas: Yeah. Seems Obviously true there. And so many possible mechanisms that when people come along and point to things like race or other things that might be causing These differences, it just makes my skin crawl.

0:06:54 Lindsey: Yeah. Yep. Same here, right? Even like even if you don't take sort of a racial angle on these things, you know, I do hear sort of conservative talking points about like, you know, it's something about these parents That made them poor in the first place. And that's The same thing That makes them bad

0:07:17 Thomas: Yeah. Oh, a genetic laziness. Yeah. Which I think is just coded for racism, but they're just trying to not say it.

0:07:21 Lindsey: Yeah, yeah. Exactly. Yeah. Nope. You're probably right. Yeah, um, so these associations between economic disadvantage and structural brain differences are not linear, which is an interesting thing. So they're strongest among the very most disadvantaged. So that means that below a certain threshold, you see more pronounced impacts on development. And above that threshold, you still see an association between income and development, but the association is weaker. So it's about having enough here, the marginal gains sort of decline As income increases. And the importance of noticing or noting these diminishing returns is that it suggests that measures to increase income among the most disadvantaged Is Likely to result in the biggest payoffs in Terms

0:07:36 Thomas: Huh. Oh, okay. Right. Right. So it's

0:08:07 Lindsey: Of Child Outcomes.

0:08:07 Thomas: Like you're talking like kind of like a hockey stick or something like where the bottom of it goes. So it goes up and then it sort of tapers. It goes straight after. OK, gotcha. So It makes it so like, hey, if we just help those people in the and the blade of the hockey stick get to the, you know, handle of theockey stick, then uh, it might be uh, very effective. Oh.

0:08:13 Lindsey: Yeah. Yeah, exactly. Yeah?

0:08:24 Lindsey: Right, the payoffs of that would be would be pretty large according to the research that's been done so far. Yeah, and also in addition to being associated with structural brain differences, economic disadvantage has also been associated with functional brain differences, like patterns of electrical activity. Right. So childhood. Yeah. Electrical Brain Activity apparently follows a specific developmental trajectory. This is not something that I was aware of before. And I think it's it's worth spending just a couple of minutes talking about EEGs because that's their primary measure in this report. So understanding that it's probably important. So EEGs or electroencephalograms measure activity in the brain, electrical activity in the brain. So if you've ever seen a picture of someone with one of those electrode caps on their heads, right, that's an EEG.

0:08:39 Thomas: They're Oh, that's not just testing if they can talk to a ghost or something? I always thought.

0:09:09 Lindsey: It's been used for that purpose, but there's others. Yeah. Yeah, so when networks of neurons fire, you can detect that electrical activity via those electrodes, right, via an EEGs, because part of neuronal communication is actually electrical, which is a cool thing, right? When a neuron receives a chemical signal from a neighboring neuron. This triggers an electrical impulse that propagates down the neuron itself. And so the EEGs measure that electrical activity, and it measures it in two dimensions. So first, it measures the frequency of electrical oscillations. So there's like a frequency spectrum ranging from low to high. And it also measures power, which is the amount of electrical activity in a particular region of the brain. So just to give folks a picture of this a little bit, most people have have maybe seen graphs depicting EEGs of different sleep stages. Have you seen this?

0:09:11 Thomas: Okay. Oh

0:10:03 Lindsey: Like the spindle graphs?

0:10:03 Thomas: Yeah. Like, okay. Maybe. Yeah. I can think, I can imagine that. Oh, I might be thinking of a functional MRI though. That's a different thing. Nevermind. I think that's what I was thinking about. So that's a difference. Okay. That is a different test. Okay, the one where you see the like actual brain, you know, in almost like x -ray -esque, but not, but better. Okay so not that.

0:10:07 Lindsey: Yeah, right. Okay. Yeah, yeah. So, that's right. Not that.

0:10:24 Thomas: What does this look, look like? Okay.

0:10:24 Lindsey: Yeah. EEGs spit out these little like spindle looking graphs, like line graphs, jagged lines, similar to like a heart rate monitor is what it kind of looks like, right? Yeah. So if you've seen these graphs of like electrical brain activity during sleep, you'll see is that in the periods where you're awake or in REM sleep, the line graphs produced by the EEG have peaks that are very, very close together. So they're like higher frequency oscillations compared to what you see in the deepest stages of sleep, also called slow wave sleep. And this is why, which are characterized rise by lower frequency electrical activity. So the peaks of the oscillations are further apart, that's frequency. And then the power is just like total the total amount of electrical activity that's being produced, right? Did all that good?

0:10:34 Thomas: Yeah. Hmm. Yeah.

0:11:10 Thomas: I mean, I don't know how to read one of those as a result of your explainer, but I'm interested. The thing I didn't think about was I didn't realize there's two different things you're talking about, like structure and also activity. I guess I would have figured that was all kind of one measurement. Hmm.

0:11:14 Lindsey: Yeah. No, no, uh. Pretty, pretty different, right. So if we're talking about structural differences, We're talking about impacts on things like cortical volume, right, which is impacted by both like the thickness of the cerebral cortex as well as the surface area, which they're two different things. I don't know how worth it is getting into these things.

0:11:29 Thomas: Yeah.

0:11:45 Thomas: But there So there's... there's different outcomes associated or different, know, I manifestations of those physical measurements.

0:11:53 Lindsey: I mean, I don'T know how to say this exactly. They're just measuring two different thing. Like one's telling you about, yeah,

0:12:01 Thomas: I know, I Know that, but like, if all this is taking data for, you experiment of like what happens to kids' brains or whatever, if get the brain activity in general, but, but at this detail, I guess I don'T totally understand what it's doing. If I, if, if I found that like, oh yeah, kids have a structural difference here, what would that mean for those kids? Like, what does that do to them?

0:12:23 Lindsey: I understand. Yeah, sorry. Yeah. So essentially, they're just reviewing like a bunch of different neural indicators that have all been associated with these cognitive and behavioral outcomes. So some people have looked at just structural differences. Others have have tapped into this via functional differences. And both of them are associated with These outcomes.

0:12:34 Thomas: Okay. Right. So it's not like a one -to -one like, run as fast because this thing and you, it's like, there's, you Know, because this is a incredibly complex thing. These are, There are a number of measurements that are all kind of associated with certain, uh, outcome. Okay, gotcha.

0:12:41 Lindsey: Yeah. Yeah, mm hmm. That's Exactly right. Yeah, okay. So, so getting back to the study, from infancy through middle childhood, there is apparently a decrease in power or amount of electrical activity in certain frequency bands and an increase in the power of other bands. So specifically, you see a decrease in power in low frequency bands from infancy to middle childhood and an increase in power and the mid and high frequency bands from infancy to middle childhood. So on average, if we took these measurements from just a big sample of kids, that's the typical developmental pattern that you see in terms of electrical activity.

And apparently, individual differences in these developmental neural patterns have been associated with differential development of various skills, like language and cognitive, cognitive skills, and social emotional abilities. So children who show greater increases in the mid to high frequency bands also show greater gains in language, cognitive skills and social emotional abilities, whereas children who show more power in the low frequency portion of the spectrum are more likely to have behavioral, attentional or learning problems. Right? This difference in electrical activity is what they are going to use as their indicator of of whether kids in disadvantaged households are worse off than their more advantaged counterparts.

They're not using any of those direct cognitive or behavioral measures yet. That's going to be something they do later in the study. But in this paper, all they have is this EEG stuff, right? And so when I read this, you know, I'm not a neuroscientist. I don't do EEG work. And so I was a little wary of just taking the author's word on whether this is a reasonable proxy because I don't have the expertise in this area. So I did consult with my fra-league, if you will, my friend colleague. Yeah. So David

0:12:55 Thomas: Right.

0:13:44 Thomas: Are

0:14:43 Lindsey: Nichols. Yeah, I think we're fra-leagues.

0:14:44 Thomas: We For all leagues, I Feel

0:14:46 Lindsey: Yeah I

0:14:47 Thomas: Like I don't have an important enough job to have a colleague. I always thought as a kid that a call you had to be specifically a college professor or something to have a colleague, but I guess not. Maybe we're for all weeks. Okay, well Thanks for giving me that one. I like that you could have said no, sorry, you're a podcaster, you dummy. You You don't have Colleagues is what you could have said.

0:14:58 Lindsey: Think were fra-ligues. Yeah we're fraudcasters. So that's not good.

0:15:11 Thomas: Yeah, we're not going to go with that.

0:15:13 Lindsey: Yeah nobody keep that one. One.

0:15:15 Thomas: So what did your froligs say?

0:15:16 Lindsey: So my colleague, David Nichols, he is a neuroscientist and he does a lot of EEG research and he did not see this as a red flag, right? I only had him look at a couple of specific parts of the paper that I had questions about. So if I'm missing some other issue that I didn't ask him about, I don't want to claim that like Dave gave his 100% seal of approval on everything. I asked him about this and a couple other things and he didn't think it was a problem, right? Did share some concerns about some other issues that we'll talk about in a minute, though.

So anyway, so this apparently is a reasonable proxy. This is what they're using. And because correlational research has shown that kids from disadvantaged backgrounds show this different developmental pattern in electrical activity, this is what their using as their proxy, right? Here's where the methodology of the current study really sets it apart from past research, right? So as I've said already, previous research on poverty and its relationship to brain function and cognitive development and so on has been correlational.

And this is unsurprising because you just can't take a bunch of infants, separate them from their families, and then randomly assign them to lower high income families for the sake of

0:15:23 Thomas: Oh, okay. Yeah.

0:16:19 Lindsey: An experiment, right. Unfortunately, I can't say that's never been done. It has been done, but you fucking shouldn't because it's terrible. Anyway.

0:16:24 Thomas: Yeah. Yeah. Like you need a prince and the pauper situation. Although I guess they didn't actually, they weren't like twins or something. Or were they? I don't remember. It's been a million years. They just looked alike.

0:16:34 Lindsey: No, they just happen to look alike, I think.

0:16:37 Thomas: Yeah. You need a, you need to wait, man in the iron mask. Situation. That's what you need. Although I feel like that test is pretty easy. The guy with the iron thing on his head, that's probably not good for development, I think. And then the guy who gets to be the king or whatever, I Think that's Probably good for that person. So that's what we can conclude based on that study.

0:16:37 Lindsey: Yeah.

0:16:51 Lindsey: Not. Yes, we can.

0:17:01 Thomas: Okay. But what about this study?

0:17:03 Lindsey: Yeah, right. Yeah. So because all this past research has been correlational, you can't make strong causal claims because there could be something associated with poverty that's actually accounting for this relationship. So this study employs an actual experimental manipulation. And that's pretty impressive. So these data were collected as part of the baby's first year study. This is the first randomized control trial of poverty reduction in early childhood. And its purpose is to address whether poverty reduction causes changes in children's brain development and in associated cognitive and behavioral outcomes. So talk about worthwhile research, right? And research that I think can directly speak to policy disagreements about whether it's a good thing to give people quote unquote free money, right. Their participants were 1000 low income mothers living in four metropolitan areas, New York City, New Orleans, Omaha, and the Twin Cities, Minneapolis and St. Paul. 1000 mothers, low -income mothers were enrolled in the study during 2018 and 2019. on average, these mothers had incomes of just over $20,000 a year, which is below the poverty threshold.

0:18:15 Thomas: Yeah.

0:18:16 Lindsey: Shortly after giving birth, these 1,000 mothers were randomly assigned to receive one of two amounts of money every month for 52 months, so just over four years. 40% were assigned to received $333 per month. So this was the poverty reduction group. And 60% were assigned to receive $20 per month. So this was the control group, essentially, because $20 isn't going to make a meaningful difference in someone's finances.

0:18:41 Thomas: Yeah.

0:18:41 Lindsey: You know, the choice to give them something in this group is smart here, because it rules out the possibility that the effects they get are simply due to like receiving something per month, rather than the amount they're receiving.

0:18:51 Thomas: Like just the the joy you get from a transaction on your bank statement, maybe? Yeah, okay. It's science.

0:18:55 Lindsey: Yeah. I mean, it could be,

0:18:58 Thomas: You got to rule everything out.

0:18:58 Lindsey: Right? Yeah. You're getting a free present every month. So we need to make sure that it's not that, that it'S actually that you're getting, you know.

0:19:04 Thomas: That's true. If you found a 20 on the street every month, maybe that would just make you happy and you'd be like, wow, I'm just a charmed person.

0:19:09 Lindsey: Uh -huh. Exactly. Right. Yeah. So it's a good piece of experimental control here that they're keeping everything similar between the two groups, except for the amount of money. And right? Just one note here, this has probably already occurred to people, but I think it's worth pointing out that the sum of money that these mothers are receiving in the poverty reduction group is pretty similar to the amount that families are receiving under the expanded child tax credit. Yeah, that's

0:19:32 Thomas: Right. But important to note, I actually didn't know before getting into this, they weren't studying the actual child tax credit, just an equivalent artificial kind of program. Got it.

0:19:42 Lindsey: Right. Yeah and apparently it was coincidental, right? Because I think that those amounts were chosen, I mean, they recruited their participants in 2018 and 2019, which suggests that they probably chose that amount prior to the child tax credit, right?

0:19:45 Thomas: Oh, right. Maybe the way they chose that amount for both things might've been similar. Maybe there's some underlying research or something that would tell them. Who knows?

0:20:05 Lindsey: It could be, I don't know. Yeah. Yeah, so just to highlight, like aside from the awesomeness of having funding to just give 400 people $4,000 a year for more than four years, the uniqueness of this study comes from the fact that they're able to use random assignment to actually manipulate poverty in a way, right? So like whatever characteristics these thousand mothers might have that that might be distinctive compared to folks that are more more economically advantaged, those are shared characteristics between the two groups that they have. And because they made the assignment to the two group randomly, those characteristics should be equivalent between their experimental and control groups. So that does allow you to make pretty confident causal claims that any differences that are observed between those two groups are because of your manipulation.

0:20:48 Thomas: Right.

0:20:49 Lindsey: So over the course of the study, children were assessed annually, or I guess more properly, they are being assessed annually because data collection is still ongoing. The current study focuses on data that were collected during the one year visit. So the infants here are around 12 months of age. From about two thirds of the participants was collected in person. Unfortunately, they had to switch to phone interviews in March 2020. Yeah, that sucks. COVID really disrupted their data collection for this first wave.

In total, the number of mothers from whom they had data across in-person interviews and phone interviews was 931 out of 1,000, but they only had data from 435 families who had completed usable in-person EEG data before the onset of the pandemic. The sizes in the data that we're talking about today are 184 in the $300 group, the poverty reduction group, and 251 in the $20 group, the control group.

Because of this significant reduction in their original pool of participants, we do want to look carefully at whether there are any differences at baseline between the experimental and control group in this particular analysis that could account for differences that they found. You look at those data, it appears that the groups were in fact equivalent on most dimensions, including gender and birth weight and marital status and mother's education and so on, right? There were a couple of significant pre-existing differences, including infant age. So the mean age in the $20 group, the control group, was 12.9 months versus 12.6 months in the 300-dollar group.

0:22:28 Thomas: Okay.

0:22:28 Lindsey: 0.3 months. Yeah. It's not a huge difference, but it was significant. So also differences in the percentage of participants who were white in the two groups. So that was 12% in the control group versus 6% in the poverty reduction group. And there was a significant difference in the percent of mothers who were black in the two groups, so that was 39% in the control group versus 47% in the $300 group, right?

0:22:51 Thomas: All this is again because of the limitations from COVID you're saying they had to like, okay, so originally these would have been basically identical numbers, right?

0:22:54 Lindsey: That's right. Yeah, right. Think, yeah, in the entire sample, right, I'll have to go back and look and see if there were any pre -existing differences, but in the entire sample I don't think that they had these problems. Yeah. So since you've got this handful of differences, you would need to account for those in your analyses, and they do, right. They account for for these as covariates, and they also account for a whole bunch of other covariates. Right.

0:23:09 Thomas: Hmm. Actually, let me ask you something just about studies in general, uh, like let's say you say you got your sample of people and you got got to essentially divide it up into a controlled group and a study group or whatever it is. You just completely make sure to do that randomly and then risk that like, oh, we just happen because, you know, random is random. We happen to like have some differences, like, potentially large differences just from luck in certain demographics. Or do you try to split them in a demographically equal way or like, like artificially be like, Like, okay, let's try to make these as identical as possible. Or is that too like manipulative?

0:23:22 Lindsey: Yeah.

0:23:35 Lindsey: It depends on the study. So most of the time, people use random assignment. And if you have large enough groups, then random assignment actually does work to even out individual differences in your two groups. And you can always, it's good practice to do a check on that and make sure that your groups are equivalent on important dimensions. But random assignment does work if you have a big enough sample. There are some types of studies where they will, like on one or two important dimensions, they will match participants in The two groups in order to make sure that those dimensions are equivalent.

So like if you're doing a study of some, say, like new math program, if existing math achievement is a a really important individual difference to account for there, which it probably is, you can do like a baseline, you can take like a baseline measure of your participants, and then take the two highest scores and randomly assign one of them to be in the experimental and one of them to be in control. And then you go on down the line, and then you end up with two perfectly matched groups on that dimension. So some studies do that. It's not as common as just using random assignment, though.

0:24:11 Thomas: Right. Okay. Right.

0:25:07 Lindsey: Yeah. So last, last note before we get to results. Study's methodology and analyses were pre-registered. So this is an added layer of rigor that has been more and more widely encouraged in psychology in recent years. So researchers can publicly pre-register what their plan is for conducting a study and collecting data, as well as the specific statistical analyses that they plan to conduct before they actually collect their data. So doing this removes concerns that if researchers report some significant difference that it was just obtained through phishing or p-hacking because they've said prior to data collection this is the analysis we want to do, right? That's worth saying, I'm not surprised that they did this because this is clearly a study that took a tremendous amount of funding, so You know

0:25:48 Thomas: Yeah. They didn't throw this together quickly and be like, let us waste whatever that was like $1.6 million or whatever the math would be there.

0:25:50 Lindsey: No Um, yeah, so it's not not surprising they're taking steps like this for sure. Right, so let's, uh, let's see what they found, so they used regression analyses here. Boy, have we talked about regression analyses a lot on this show.

0:26:07 Thomas: Yeah. Love them.

0:26:10 Lindsey: Yeah, they're the best.

0:26:10 Thomas: Huge fan. No idea how you do it, but it sounds cool.

0:26:11 Lindsey: Yeah. Yeah you just you click some buttons

0:26:15 Thomas: Cool. It actually sounds like a mathy thing that I would really like, but I've never looked into it.

0:26:15 Lindsey: Is most of it. Yeah mm -hmm. Yeah I bet I bet you would like it. I mean you can find a Moodle or something.

0:26:21 Thomas: I want to just regress some numbers just by myself for fun.

0:26:26 Lindsey: You totally could. Yeah there's yeah, so they used regression analyses to see if their poverty reduction manipulation predicted different patterns in electrical brain activity in these one -year -olds. So their primary predictor of interest, obviously, is the treatment condition, whether or not the mothers received $333 per month or $20 per month. And they also controlled for a variety of covariates. So they have in their model education and income and net worth and maternal age and and infant age, race and ethnicity, parity of the mother, which means number of other children, whether the mother smoked or drank alcohol during the pregnancy, whether the father was living in the household. And that's not even all they controlled for. So they put in a lot of factors that might be associated with poverty and could also impact brain development. And they're seeing whether their experimental poverty reduction still predicts a difference beyond these other variables. Yeah.

0:27:16 Thomas: Right. That enough people to do all of that? It wasn't only 400 people.

0:27:22 Lindsey: Well, no, you're right. Yeah. So, so this, this reduced sample is 400 and something people. No, you can, you Can do that, right? There's regression is useful. If you like, there's certain types of analyses that once you get up to a certain number of variables, you have to have just like unwieldy large datasets. Regression is a little better for that. Um,

0:27:29 Thomas: Yeah. Yeah, yeah. Because I was just thinking like you need to have enough different instances of these particular characteristics in the samples and in the people, right? In order to kind of meaningfully differentiate. Yeah. Okay. But

0:27:53 Lindsey: That's true, right? Um, And Every every variable you you add sort of reduces power to detect your effect too, so yeah.

0:27:56 Thomas: Yeah. Right. Cause like, oh, we need someone who's, you know, father's not in the picture, but they also did drink. And then like all the, you might need some different numbers of those combinations to have the right data you might needs. Right?

0:28:10 Lindsey: Yes. Yeah, um, we should do an episode just just on regression that would be really fun. We could get we could

0:28:14 Thomas: Ooh, I would like it. I have no idea if our listeners would, but, but well, We'll let it suffice to say that doesn't seem to be an issue here to you, there's enough big enough sample

0:28:17 Lindsey: Yeah, yeah. Maybe a patron only, yeah. I I don't I don't think so, if somebody thinks I'm wrong on that, I'll be, I'll, I'll be open to that. But I don't think so. So I'm going to, I'm going to preview the overall gist and then we'll talk about some specifics. Did find patterns here that were in the hypothesized direction, but none of their pre-registered analyses reached statistical significance, is at odds with the New York Times headline. And it's also at odds with the summary of findings that the authors gave in their abstract. And that doesn't sit super well with me, did not sit well with Dave either.

0:28:24 Thomas: Okay uh Oh, Yeah.

0:28:56 Lindsey: Yeah, it's a bummer, right? So why did they say that they did find differences? And there's a couple of things to say here. So first, if we look at their pre-registered analyses, they reported two indicators of statistical significance. One was adjusted and one was unadjusted. And so I don't want to spend too much time on this, but just briefly, if you run an analysis with multiple predictors, you're going to get a value for each of your predictors indicating whether it's statistically significant.

As you add predictors to your model, it becomes increasingly more likely that by chance alone, one of them is going to be statistically significant, even if in actuality, none of the predictors have real relationships with the outcome in the real world. And so, like, I demonstrate this to my research students sometimes by generating a data set with like 20 fake variables from 100 fake people. And we populate those variables with just purely randomly generated values. So there should be no relationships between these. And then we run a correlation matrix between those 20 variables. And invariably, there are a handful of relationships in that set that are statistically significant just by chance.

0:29:48 Thomas: Yeah.

0:29:59 Thomas: Gotcha.

0:30:00 Lindsey: So returning to this analysis, because they had so many predictors, they used a significance value that adjusted for the number of relationships they were evaluating in order to lower the likelihood of getting false positives.

0:30:14 Lindsey: So, essentially, they made their criteria for considering a relationship significant more stringent, right? And they're specifically referring, I don't want to confuse people, it wasn't about adjusting for the number of covariates, because what they really care about is the EEG data, but they had several different values from the EEJ data that they were looking at, right. Right. So one of those EEG outcomes or variables could be significant just by chance since they're evaluating lots of them.

Okay, so so essentially, they're just, they're making their standards here, the threshold more stringent. So if we look at the unadjusted significance values, the infants from the high cash group show significantly greater power in the mid and high frequency bands, but not in the lower frequency bands. And that is what they hypothesized. But once they adjust those significance values to this more stringent standards, the differences no longer reach significance, right?

And when we look at the effect sizes here, which is like the absolute size of the difference between their two groups, the differences are pretty modest. So among the ones that were even close to reaching significance, it was between like a fifth of a standard deviation and a quarter of a standard deviation. So it's not huge, right? It's possibly not nothing, but it's not a very large effect size. What they do note later that actually these effect sizes are pretty similar in magnitude to more resource-intensive educational interventions that have been done to improve

0:31:41 Thomas: Interesting.

0:31:41 Lindsey: Cognitive functioning. So we'll come back to that. All right. So, unfortunately, if we're just looking at the pre-registered analyses, this really isn't as clear or decisive as I was hoping. I found this a little bit disappointing. May, in fact, be, be something real here that they're detecting. I really hope that's true. But I think that we really need to wait to see additional demonstrations of the effect in order to say that confidently.

0:32:02 Thomas: Well, you, so how much time had elapsed in this, like that was, that, that was that number. Okay. Sorry.

0:32:06 Lindsey: 12 months. Yeah.

0:32:09 Thomas: The 12, 12. Yeah. I mean, I guess maybe take a step back and at what a, what a high burden that would be like for one, this is to say nothing of the question of like, are those people happier and doing better in life because they have money? This is just purely is literally the, the household having 300 more dollars a month, changing the brain chemistry of a child within 12 months, which is like, okay, that would be cool if it was, but also that's a, seems like a very, very high bar.

0:32:17 Lindsey: Yes. Mm hmm. Yes, it's a high bar. Mm-hmm.

0:32:43 Thomas: Maybe give it another year or maybe like it, rather than changing their brain, it just has other really great, obviously good effects that we should still want, even if it doesn't change like like people's brain functioning.

0:32:44 Lindsey: Yeah, and the absolutely and so no pun intended. I am not throwing the baby out with the bathwater here. I'm still very excited to follow the progress of this study and see what they come out with next. I mean, one of the things they talk about in their discussion is exactly what you're talking about, that like a 12 -month intervention in the first year of life, right? It may be possible that you only see really modest effects at that point, but I want to see the rest of their data. Absolutely.

0:33:12 Thomas: Yeah, right.

0:33:23 Thomas: Yeah. Guess I don't fully understand why did this get reported wrong? Is it just because of those prior to being adjusted values?

0:33:23 Lindsey: Yeah.

0:33:32 Lindsey: That, that and they also, in addition to their pre-registered analyses, they also did some exploratory comparisons that examined the differences in electrical activity in different regions. So they essentially broke out the EEG readings into those taken from the frontal part of the scalp, so toward the forehead, and those in the center, the top of the head, and separated those into like sort of three chunks, right? So rear readings, frontal readings, and center readings. And they did find some significant differences once they broke them out in that way, right? And the patterns were consistent with their original predictions. Again, it's like those exploratory findings may be meaningful, but we would need to see them replicated in order to really be comfortable interpreting them, because that's the kind of stuff where you get potential p-hacking, right, you can slice up your data a bunch of different ways.

0:34:16 Thomas: Yeah. Yeah?

0:34:24 Lindsey: I don't know, it well could be meaningful. We just got to wait and see, essentially. Yeah. I know.

0:34:27 Thomas: Well, that's a little sad, but again, yeah. So are they going to do a report every year or what's their plan?

0:34:34 Lindsey: I would expect us to be seeing potentially multiple reports, right?

0:34:40 Thomas: Wait. Yeah. How long were they funded to do this?

0:34:42 Lindsey: So at least four years, they've got guaranteed funding for like 52 months or something.

0:34:44 Thomas: Okay. Gotcha.

0:34:48 Lindsey: And so they might extend it for longer if they can get more funding and, you know, retain their participants. So yeah, no, I think I'm definitely going to keep tabs on what they're doing. And maybe we can do a follow up at some point. Think that would be fun.

0:34:52 Thomas: Yeah, yeah, definitely. Definitely.

0:35:02 Lindsey: Yeah. So I think the biggest thing to ding them on is, is really their abstract here. You know, they, they reported this as saying that they found differences in their abstract, which is supposed to be the overall summary of their paper. And like, they did, but you kind of need to lead there with, you know, none of our pre -registered things were, were significant. Right. Right.

0:35:19 Thomas: That's frustrating. I wonder, you it be the pressure of like, they need funding to do that? This? Do you think that might play into this?

0:35:30 Lindsey: Yeah. Yeah, it's a delicate balance in studies. And I think that it is higher stakes since this is such a since it obviously required so much funding. But when you're when a publication, it'S always a delicate balanced between qualifying your results and emphasizing, you primary points of interest or the value of your results. And and so. So like, I actually think that in their discussion, they did a really nice job of qualifying their findings and being very cautious in terms of how they said they should be interpreted and talking about implications and saying, you know, we don't want to make strong policy implications yet. We need to wait and look at the results that are going to come out over the next few years. But like, yeah, putting that in the abstract seemed like a bad decision to me.

And of course, that's what ends up in the headline, right? So I don't know. Not sure what the decision making was, but it's their discussion, they give this very cautious, tempered summary of their findings. They do say that the patterns they found were consistent with both their hypotheses and with this body of correlational literature, which it is, and I think that that's meaningful.

And highlight the fact that using several different analytical strategies in addition to their pre-registered analyses, that the overall pattern of differences in electrical brain activity between the experimental and control groups remained consistent, right? So that's also probably meaningful. But then they rightly go into a discussion conceding that not all the results that they obtained survived the most stringent tests in employing this correction procedure that they employed. And they say that the results need to be interpreted very cautiously until they can be replicated. And this I found really interesting, they contextualized the size of the effects that they got relative to large-scale environmental interventions, like resource-intensive educational interventions, and

0:35:50 Thomas: Yeah.

0:37:18 Lindsey: Said that they were very similar. Similar being that if if they can replicate these effects and you know and show that the effects hold for like actual direct measures of cognitive and behavioral development, this suggests that like this relatively simple intervention, just giving these direct cash payments every month, may have relatively large payoffs, right.

0:37:38 Lindsey: And that's if you can do that instead of like I'll, I mean, educational Interventions.

0:37:38 Thomas: It won't make a difference, Lindsay.

0:37:43 Lindsey: Are a pain in the ass, right? I, I know.

0:37:46 Thomas: Totally right. And I don't want to interrupt your point. Yeah, this is a cheaper way to obviously help people as much or better. But I think we already know this based on, again, a lot of the research referenced in Give People Money, a book from 2018 and Earlier. And it's it's come up on the show before, almost every homelessness intervention is like, no, just give them money and it would be fine. But what if we make them work in a program? No, Just Literally just give him like a thousand dollars and they'll be, it'll be much cheaper, it'll save us money in the long run because of the services they, you know, use if they don't, they're like hospitals, you know, whatever, all that stuff, saves us money the long run. But our society is Uh, crap, and doesn't want to just give people money because they might not deserve it.

0:37:59 Lindsey: Yeah. Yeah, mm Mm-hmm. Well, but maybe they were just waiting for the experimental evidence.

0:38:33 Thomas: No, we already have it. It's already, every analysis is like, it's way cheaper. Like literally, I think there was, no man, we should actually, should do a review of that evidence at some point. Maybe it might be interesting, you know? Cause like from my memory, I don't know. I think it's pretty stark. It's like, no, just give them like 40, even if you paid them a lot of money per year, it's still cheaper than the amount they would use in services otherwise. But our society cannot deal with that information. They just can't.

0:38:44 Lindsey: Okay. Right. You don't think it's because they were worried about confounding variables in the analyses that were done?

0:39:03 Thomas: No, I don't think it is.

0:39:04 Lindsey: Damn. I thought it was that the policymakers wanted random assignment and now they would have it. It's a bummer.

0:39:11 Thomas: Well, this conversation is a little different than I thought based on, cause you know, I just took the headline at its word. Cause the New York Times, but yeah, no, this sucks. It's I guess, again, as you say, we could see stuff later on down the road. But yeah, that, that's not great that it was reported this way. Okay.

0:39:28 Lindsey: It - it's not. It is not. But, like, I don't - I don - I'm trying not to be too disappointed about this because, you know, I - they They did a nice job of making the case that, you know, even though they're not meeting the standard, that these results are unlikely to be spurious, right? And I think that they're probably right, you know, but we cannot say that yet. But like, Again, I'm just - I'm very interested to follow the progress of the study as we - as these kids get older and we can get some better measures from them and get measures from the entire sample, right.

0:39:49 Thomas: Hmm. Right.

0:39:59 Lindsey: Is 40 % Of the sample or something. Thing, so I - I have not lost hope here. I think that there's probably a lot of good stuff to come.

0:40:00 Thomas: Yeah. Time for, you know, again, briefly, maybe some of the political stuff. I do think I, you know, sorry to blow your optimism and feel free to retain your optimism, but I don't really think the evidence - like, one thing that's super for disappointing about this expanded child tax credit that has, again, maybe it doesn't instantly change your child's brain chemistry within 12 months or something, you know, like maybe it does do that, who knows, but it cut child poverty in half or whatever, as they said it might do.

And this back to our conversation in our Patreon episode about kind of the little bit of the hopelessness both of us were feeling. This an example of an incredible policy that is undoubtedly helping. Like, it's - it's reducing child poverty - a thing, a sequence of words that shouldn't even exist. Child poverty, like, should not even be a thing that exists in our country. And people - it's not just that it would be bad enough if the problem was - yeah, but the politicians are canceling it despite popular support. Um, as - as just a review, this is one of the things that uh, Mansion has killed for now. Um,

0:40:11 Lindsey: Yeah, yeah. Right. Right, yeah.

0:41:17 Thomas: It's expired at this point. I think - I think it was set to expire this month, and so far it's not - you know, unless we - there's every chance that it could still back into a bill, uh, in some form. I think - I think Joe Mansion probably just wants means testing, you know, which obviously we're anti means testing or I'm anti means testing in this case to degree he wants, I'm sure. But like if as long as if we still get something good, like at least that's something. But the point is, it's not just that like, oh, fuck Joe Manchin. He's single handedly doing this despite everyone. You look at the poll numbers. This is the most disappointing thing. The people don't even give a shit about this policy. Like, yeah, it is. It's approval is like 50 ish percent.

0:41:21 Lindsey: Yeah. Really?

0:41:56 Lindsey: No. Shit, what is that about? That -

0:42:02 Thomas: Yeah. No, but I don't know. I don'T think that policy matters anymore. See, honestly, like, I think that like, it's an amazing policy, but people's overall approval rate, like it doesn't really change anything somehow. I, I don't know exactly what it means, but this amazing policy in which we're just giving people money that, uh, helps them. Um, it doesn't pull any better than like, It's lower than a lot of other stuff. It's just barely in the the 50s. I don't know if it's, I have to look at the detailing of how they asked it. I don't if the problem is polarization or what. Maybe, I don'T know. It's just disappointing.

0:42:39 Lindsey: Is really disappointing and and surprising. I thought that I I thought this was a popular policy. Is it used to be and then it dropped off for some reason? Huh.

0:42:48 Thomas: I don't know. I don't know if maybe inflation's the problem. Like people, policies like this, maybe people are like, oh, I know inflation's a problem and this might lead to inflation or something. It's also, I'll say this about the data. There's not a lot of disapproval, I guess. So like it's 50% approval versus like 28% disapproval or something. So it's still like net positive by 30, 22 or whatever. Maybe that's what people are referencing, but yeah, no, it just sucks that this, we, I don't know what the reason is. I guess it takes more of a political polling analysis that you or I don'T have right now, but for whatever reason, this isn't even in something. This policy is not that popular and apparently hasn't done much to make Biden more popular. I don't know. I don't know what to tell you.

0:42:51 Lindsey: Okay. Yeah.

0:43:33 Lindsey: Well, I knew that - that was true. It's nothing's apparently working to make Biden more popular, but yeah. Well, that's a bummer. Yeah.

0:43:37 Thomas: Yeah. Yeah, know, some speculation is already kind of what I referenced where people think people don't deserve it. That might be another thing that pulls this approval down a little bit. You know, people are like, "Ah, giving people money for nothing." Yeah, and so that might - you sway some people a little bit, I don't know.

0:43:58 Lindsey: Need to keep them apprised of the baby's first year study and obviously that'll change everybody's mind. Yeah.

0:44:01 Thomas: That'll do it that will do it. It won't - it won't hurt, I guess. Oh,

0:44:04 Lindsey: Yeah. Um, another thing - I only found this shortly - shortly before we started recording, but they - one of the other reports that they had on their website right now is the results of a pilot study. So this was prior to the sample that they were analyzing here. Um, it was a pilot city where they gave mothers like a hundred dollars a month versus $20 a month or something and looked at how they spent it and like did some qualitative sort of analyses of, um, you know, the impact it was making on their lives. And like, you know, again, I only had time to skim it, but it - it seemed very counter to the - to the narrative you hear sometimes that if you just give people free money, they're going to squander it, or it's going to have these - these sort of negative backfiring effects, um, which again is not surprising to anybody

0:44:47 Thomas: Yeah. I wasn't even entertaining that as a - that is something that probably - that I think that specifically Joe Manchin said he was like, "Oh, they'll spend it on drugs." So cool, yeah. Yeah. Awesome. That's a huge problem. You know, it's that I don't even entertain - person in my personal politics, I don't give a shit about that. But I get that probably makes a big difference nationally, like in political rating for the policy. I think a lot of people do think that.

0:44:49 Lindsey: Yeah. Yeah. I'm sure he did. Yeah, I - Yeah. Mm. -hmm. Yeah. I think a lot of people do. If - if we could get this into the hands of - of, you know, everybody in the country and they actually trusted it, then yeah, it might actually change opinion, but

0:45:20 Thomas: All in all, kind of a disappointing episode. Both the study wasn't quite what we hoped and the politics isn't what we hope. But I think it's important to keep track of this. And then when and if it shows results, nothing on that. Maybe we can, as a a progressive, somebody wants these kind of policies, if You Make Noise About These Results, maybe it'll have an effect. I don't know.

0:45:39 Lindsey: Maybe it will, and also I - don't, you know, it is - I was very disappointed to find out that the results weren't as strong as they were sort of pitched to be, um, really just on the back of the methodology that they're employing here and the scope of what they're doing. Like, on our - our science scale, you know, all helps down, bad science thingy to five chili peppers which science again. Um, I - I think this is a - I think it's a good science thing. Like, they deserve criticism for their abstract, but like, yeah, it's - it's good methodology. It's a good study, so yeah.

0:46:05 Thomas: Yeah. Oh yeah, I should have asked you that yeah But the - what about the fact that they kind of misrepresented it a little bit? Okay.

0:46:23 Lindsey: No, that's - they deserve criticism for that, um,

0:46:25 Thomas: But not like terrible science thingy, not we're not going two palps down. I

0:46:29 Lindsey: I honestly like, I was - I was kind of, after I read the results and got to the discussion section, which is where you do that analysis and you talk about the results mean, I was like, "Man, they're going to screw this up in the discussion." And actually, the discussion was a really fair treatment of how to interpret and understand the results. And I was, like, well, who wrote the abstract then?

0:46:42 Thomas: Want to make clear that this is entirely speculation on my part, but I wouldn't be surprised if that's somewhat political in the sense that, like, they - they need the funding to - to continue. I don't know. Like, who knows?

0:46:58 Lindsey: It could be. Yeah.

0:47:01 Thomas: Thanks for taking us through another science thingy. I'll be at a little, little disappointing, but you know, we, again, we'll keep an eye on this going forward. And

0:47:03 Lindsey: Yeah. Absolutely. I'd like to do a follow-up.

0:47:09 Thomas: Yeah, I give your science explanations two palps straight up.

0:47:16 Lindsey: Thank you. That means so much. It really does.

0:47:20 Thomas: Thanks for listening. Hey, support the show on Patreon, patreon.com/seriouspod. You know, studies show that when you support us, we make better stuff or something. I don't know. We'll get back to you.

0:47:32 Lindsey: It's good for our brain development.

0:47:33 Thomas: Yeah.

0:47:33 Lindsey: It's true.

0:47:34 Thomas: At least the abstract said that. I don't know what the actual data said, but it definitely would be good. And tell people about the show. Share the show! I especially think that, you comeback episode in which you broke down the political science study was just like such interesting information that I feel like people would want to know.

0:47:52 Lindsey: Oh, boy.

0:47:53 Thomas: So, you know, share the show around everybody. Appreciate you so much for listening and we'll see you next time.

0:48:07 Clip: Thank you so much for listening to Serious Inquiries Only. If you like the show, the absolute best thing you can do is support us at patreon.com/seriouspod. The second best thing you can do, is please share it on social media or by word of mouth. Speaking of social media, follow me on Twitter at Serious Pod. If would like to be Facebook friends, send me a request at facebook.com Thomas podcast. If not up for that level of intimacy, that's fine. Maybe you want to join the discussion at facebook.com slash groups slash SIO members. Thank you again for listening. We'll see you next time.