SIO463: Synesthesia - How Does It Work?

Transcript

This transcript was generated automatically and may contain errors — the audio is the record.

0:00:16 Thomas: Hello and welcome to Serious Inquiries Only. This is episode 463. I'm your host Thomas Smith, and I am, again, so excited for this episode. We're back with Dr. Jenessa Seymour, and we're going to be talking about, she's going to teach us all about synesthesia. That's something I'm sure most people are loosely familiar with. Senses activating or triggering when you wouldn't think they would. Seeing colors with seeing colors with music all kinds of combinations she teaches us all about it and this is so fascinating seriously she goes through something in this interview that I'm still thinking about and that blows my mind so much I cannot wait for you to hear it is so cool and it was so cool that this extended into two parts and it is trust me it is worth it like I loved doing this interview and learning more about synesthesia and it's topic a it's that Jenessa is passionate about too and has studied a lot uh she's a phd in neuroscience by the way in case you didn't remember um and this is in particular something she's looked at a lot and it is so interesting I can't wait for you to hear the thing that has been mind -blowing and stuck with me uh this entire time until now and so I just get to it after this break uh we'll get to jenessa please support the show leave the ads And all that at patreon.com/seriouspod. It means a lot. It's really important. Do so um, and after this We'll be on with jenessa about synesthesia And Jenessa, welcome back to the show.

0:01:38 Jenessa: Hey, how you doing?

0:01:40 Thomas: Great. This I is something I feel like has come up a number of times tangentially and stuff we've talked about and is so fascinating. And I feel like it's something you're also very interested in. So can't wait to get into it

0:01:52 Jenessa: Yeah, yeah. I, you know, I like this one especially because after all the things we've talked about, like, nobody is dying, no rats were harmed in these experiments, just fun, chill. Well, yeah, it would be tough to do this kind of perception with rats, I think.

0:02:03 Thomas: That we know of. But we can ask everyone what color is the podcast they are listening to right now in their ear holes.

0:02:20 Jenessa: Boy. We're going to have some people come out of the woodwork, let me tell you.

0:02:23 Thomas: Yep. Yep. That's cool. Where do we where do we begin right there?

0:02:27 Jenessa: Yeah, so I was thinking we could start with just to like crash course, what is regular old perception before we get into like weird perception?

0:02:38 Thomas: ,that's a tough topic. Are we going to talk about, yeah, the demon, not Laplace's demon. Who's the other one? Descartes. Yeah, we're going to start. No, we are not going philosophy. We are going just, yeah.

0:02:49 Jenessa: Okay, so we're gonna. Right. Yeah, going to set some definitions that you know, some philosophers might not love, but we're We're going to operate in the world of, this neuroscientists are approaching it. Yeah, only something that I spent a decade and a half studying, no big deal. We're gonna condense it into like five minutes. But yeah, we're venturing into the world of psychophysics, which a good word, right? Basically just the meeting of the physical world and psychological world. How does some physically measurable thing become a a psychological experience. So, So, yeah, hit your retina and you experience vision. Pressure wave gets into your ear, you experience sound. And want to differentiate differentiate those because, uh, is objectively measurable. We have devices for this. Um, once we agree on a definition, there's one right answer. What is the amplitude of the pressure wave in the air in decibels? But then you experience internally internally loudness that goes along with that and that experience is subjective in the sense that it's internal and in the since that I measure it by asking you not subjective in the sensethat it's like major individual differences like you're not going to find someone where the decibels go up and they tell you it gets quieter um there is a a very strong relationship and psychophysics is studying that relationship what is

0:04:37 Thomas: Interesting.

0:04:37 Jenessa: The physical stimulus what is the experience of the person that that's basically my whole bread bread and butter.

0:04:44 Thomas: Yeah, it's so fast. I mean, this is something, again, not to get into a bunch of the philosophy, but I have always, you know, like, my experience with a lot of this stuff is in reading philosophy. And also biology, it's crazy to think about. Like, it is really crazy to thing about the fact that, like, somehow with our nerves and our brain and our consciousness and all that stuff, we have these experiences that feel like things. Like, feels like stuff for us. It's so basic and yet so really impossible to even comprehend in a way. It's like, and you can't really just turn it off, you know? It's not like you can just be like, well, I'm tired of my leg hurting. I'll just not. But it's like, you in some sense, it's like you say, it'S subjective, but it'S also like, I wonder how much of that is us having all of the information. Like you, I guess you could imagine if we knew everything about someone's, you brain and nerve endings and all that, like all that stuff we could in a way in theory, I think we would have to be able to say that we could like know exactly how someone will perceive something or not I don't know like I think I think the answer has to be yes because there's nothing magical happening but at the same time there's so many unknowns

0:05:58 Thomas: Yeah

0:06:00 Jenessa: And I think to some degree neuroscience gives us the tools for that. Know in a lot of way I think of neuroscience not as it's at least you know from the direction I operate it's more of a tool than its own separate thing. If we use the tools of neuroscience, you know, I can look at the way your brain is activating and I can make some very good guesses about what you are perceiving right now. That's actually going to be a huge tool for us when we get into people who are having atypical perceptions um I'm going to measure it and also to some extent how I am going to I don't know if prove it is the right answer validate it is probably a better way to say it know I always find it interesting like to think about there's no real reason like like photons had to be light as the subjective experience. It could have been anything, right? Know, we want to perceive them, but sure, sure.

0:06:54 Thomas: Yes and no I mean it had to be something that objects give off all the time so I guess in that sense it could have been something else but I don'T know if there's like another good candidate for what that could have have been in our the physics as we know it you know I mean but then again I you know to bats the famous whatever that is that essay the what it's like to be a bat thingy whatever that thing is like in a way to them sound and vibrations might be quote -unquote light like it might actually be to them what they would consider light they might see it in away

0:07:27 Jenessa: Yeah, now you're in my exact bread and butter because, you we have people who lose their vision, they still have this area of the brain we would call visual cortex of course it's not doing that um they learn echolocation abilities they will often report uh of all we will see the visual brain areas activate as they use sound to navigate their environment and their subjective report is no I don't I don't mean like metaphorically I am like seeing my environment I literally when I make those little clicks and the click bounces back I am having a visual experience of my environment experiment described it like a like a camera flash lights up the room for a brief second so um there's really no reason it had to be it had to be experienced if we wanted to use the information in our brain but it didn't have to be experienced the way that it is you know um certainly philosophically there's so many little questions in there and And psychologically, neuroscience, we're kind of just landing on, here's the definition I'm going to work under because I got to be able to do an experiment, essentially.

0:08:37 Jenessa: And really, this does illustrate, I think, in a lot of ways how atypical experiences are so useful for research that it helps us to understand how certain things work. And it also challenges assumptions. When we're going to look at synesthesia and people who are having that experience, you have to come up with a theory and an explanation of sensory perception that accommodates them and allows for them because they exist.

0:09:06 Thomas: Right. Yeah unless a bunch of people are making it up which you know yeah

0:09:12 Jenessa: Well, that's exactly where we want to go, right? So, let's just start with like defining it. So what even is it, especially if people haven't heard of it? Like, one sentence I can give you is when a stimulus in one sensory modality is going to evoke a perception in that modality and a second illusory perception.

0:09:37 Thomas: Illusory interesting that's a loaded term in that like what what is an illusary perception and what's not you know know how to even define that yeah

0:09:42 Jenessa: It is. The definition we're going to you to stick with is that um there's a pressure wave hitting your ear and you hear a sound that is typical I'm going to stay away from like normal because that is like value laden you know kind it but it's typical it's it's normal statistically normal to have a pressurewave hit your ear and hear a sound. Not typical is to then see a color. Not photons hitting your eye and yet you are having a visual perception. And so in that sense, it is illusory.

0:10:18 Thomas: Interesting but it's also like how would you and I totally uh I'm not trying to fight fight you on any of this I just think it's already interesting to think about because I I feel like if it was, it's very, and admittedly so based on just kind of statistic norms or whatever, like it's It's like if everyone had that experience, I don't think it would be looked at as quote unquote illusory. It'd be like that's just part perceiving color or sound or whatever. And away, I'm not trying to fight or anything on the devil. It's just interesting to think about like what means it's illusary. If it's consistent for someone, like if that was the only person on the earth and they had synesthesia, they'd be like, well, that's juste normal perception.

0:11:00 Jenessa: Yeah I totally agree and I actually this is one of my biggest complaints about how we talk about this research I think in some ways this is informed by a lot of my background and my work with disability that way we define like what's normal and what isn't is very social social and, um, some ways subjective. Um, so yeah, I've got examples for you where I, I feel very strongly that all humans are having a thing I would call synesthetic and it's just statistically normal. So we're not calling it that.

0:11:37 Thomas: That's very interesting.

0:11:38 Jenessa: Um. Like I would say, for example, um. Empathy. I see you cry and I just get so, so sad. I might even cry myself and And like, I don't know, isn't that kind of, I'm not sad. I'm the thing didn't happen to me.

0:11:53 Thomas: Yeah I guess the only distinction would be is that a perception or is that you know do you call that a more complicated process but yeah I see your point though for sure

0:12:03 Jenessa: Yeah. And like there are other examples like that. So there's a lot of the distinction. The definition I gave you is specifically what we would call strong synesthesia. And we have a separate definition for weak synesthesia. It kind of fits what you're talking about. Out. So like strong synesthesia, what is it? Got my little table here from a paper that does a really thorough review. The prevalence, it's normally thought of as pretty uncommon. Hard to measure the prevalence because a lot of people who have this experience don't know that it's not normal.

0:12:40 Thomas: Yeah that's so funny yeah

0:12:42 Jenessa: You could think about like color pitch synesthesion. I hear a sound and I see a color, right? We all say things like, oh, this piece of music is like really blue, like the blues. I just mean that metaphorically, right.

0:12:58 Jenessa: I'm not literally seeing blue. But if you do, you'd be like, yeah, I agree. It's very literally blue.

0:12:54 Thomas: Well yeah okay sure yeah unless for them the synesthesia makes it red Maybe like, what is everyone talking about?

0:13:11 Jenessa: Right, like you guys are out of your mind. It's very common for people to get into their 20s, 30s, 40s before they finally figure out, like, wait, you don't?

0:13:23 Thomas: Well, we see that and I don't know how related this is, but we see that with the inner monologue or the a Fantasia stuff too, where it's like so many people are like, I did not know this was a thing at all.

0:13:34 Jenessa: And I think for a lot of those people, they're like, well, I mean, I think thoughts. I thought you meant, like, metaphorically, you're literally hearing a voice in your head? Like, what are you talking talking about?

0:13:45 Thomas: Man. Did you, okay. Did You Hear the Dear Old Dads where Eli was talking about this? Oh Oh man.

0:13:50 Jenessa: Oh, I don't think I've gotten to it.

0:13:52 Thomas: No. That's funny. Cause he, he has this whole thing about like, he thinks everyone, he says they don't hear it in inner monologue. He thinks they're all like wrong about what we mean by inner monolog. And then at the end, after a while, I'm like, dude, I think maybe you don't have an internal monologue or or it's at least different. Cause he's like, no, you don'T literally hear thoughts of, Oh, I'M going to go over.

0:14:16 Thomas: And it's It's like Tom and I were like, well, actually kind of, yeah, like we do literally hear a voice saying thoughts like that is how it works for us. And I still don't totally know. The reason I don't know with him is because he does seem to suggest that like he has it sometimes. And I feel like the people who have no internal monologue, it's not like it turns off and on. I don' know. So I don'T know. I DON'T know what's going on with Eli. But yeah, it was a really funny episode.

0:14:38 Jenessa: Yeah, I know for that aphantasia for the internal visualization, there is a spectrum. So some people experience that as like, absolutely not never what's wrong with you people. And some people are like, I have a friend who explains it as like he doesn't picture his home, but he can kind of picture like an architectural sort of of diagramming like a rough sketch might come into his mind

0:15:03 Thomas: But that would be different than saying, yes, sometimes I see the full entire house and sometimes I see nothing. And so I don't know.

0:15:11 Jenessa: I agree yeah yeah yeah absolutely like for me it is extremely literal for both of those like I could you guess close my eyes close my ears and like hear my grandmother's voice and I haven't spoken to her and you know she she passed away over a decade ago and I mean it literally I could hear her um if she was right next to me but not in a like schizophrenia hearing voices kind of way I I know it's I know its me right

0:15:37 Thomas: You're not using your literal ears to hear an internal, you know, like,

0:15:42 Jenessa: So so this is very similar I think um was one woman whose story was she was at a baby shower and people were talking about like names and you know they're all throwing out like opinions about pick a name? How do you pick a name? She was like, yeah. And I mean, like, what do you do if like, you really love this name, but like, What if your partner doesn't like the color? And they were like, What are you talking about? And was like 50 something. And that she was when she learned that for everyone else names don't have colors.

0:16:03 Thomas: That's awesome. That's so funny.

0:16:14 Jenessa: Um, yeah. And

0:16:15 Thomas: But now that's interesting now do people have it where it's name so not all words that

0:16:22 Jenessa: It can be very specific like that.

0:16:25 Thomas: Is weird okay that's that's hard to understand sure

0:16:29 Jenessa: Yeah, Yeah. Yeah. So, so like the most common, or at least the most commonly reported is color grapheme. And what that means is letters and numbers, like the, the letter itself having a color. A is red, B is blue, C is on green. I'm just making stuff up. Um, I don't have any of these experiences if anybody's noticing that I not speaking from personal experience here. Will usually have it as well for those individuals. They will usually have the whole alphabet will have something. Was a really cool thing where they taught people who had it for the the Latin alphabet, they taught them Cyrillic, it's like slightly different letters. And as they learned, this is roughly an A, it instantly adapted and picked up the colors. So it does transfer in

0:17:21 Thomas: Interesting

0:17:21 Jenessa: That way.

0:17:21 Thomas: And it feels like there's some sort of like that that feels like There's some sort intermediary process then like it has to you know like if you have to explain something for them to get like wow yeah that is mind -blowing How the hell does that work?

0:17:38 Jenessa: So like it's not just the shape of the thing because then like, oh, every circle would be the color, right? It is about the meaning.

0:17:44 Thomas: Interesting. That is interesting.

0:17:48 Jenessa: One's pretty common, you know, common -ish, right. There You is color pitch where different notes have a color.

0:17:57 Thomas: Here's the question I have that I don't know if you have the answer to. I have often kind of wondered if everyone who has perfect pitch, which is pretty rare, if they have some sort of synesthesia thing going on.

0:18:08 Jenessa: Statistically more common. Not 100 % of the time, but a lot know, it seems more common in musicians in general, and that could be, you Know, synesthesia, yeah. Artists in general. Chicken or the egg, hard to say.

0:18:16 Thomas: Synesthesia does? Oh, interesting.

0:18:23 Jenessa: You know, did it lead you to it? Did it make it easier? Or did did early exposure cause this, make it more likely to develop? To say. It's more common in autism. It is more common in savants. I don't know that that's even really a diagnosis anymore, but that form of autism, it was thought for a very long time to be significantly more common to the point that some people were proposing there must be a gene for it and it must be fatal to the male fetus. Then it turns out it's just about the way you ask the question that like men don't want to tell you that they're experiencing it.

0:19:06 Thomas: Because certain numbers are pink and they're like, I'm not telling anyone this because it's you know be too too feminine yeah no I don't I don' t see pink there's no way

0:19:11 Jenessa: I'm not gay. Like, yeah. No, music doesn't have a color. Yeah and if you ask the question the right way the gender bias drops significantly depending on the study it might become completely equal um there

0:19:26 Thomas: Interesting that's crazy that it went from that like what in the question made such a difference that like I actually I may I joke but I don''t understand what that would be hmm

0:19:39 Jenessa: There is a general thing that just overall many different domains women are more likely to report atypical experiences um that's report court, not have. Like, many, many times we find that if you ask the question the right way, all of a sudden men will disclose.

0:19:57 Thomas: Okay so it's like from an early age man like ah my name is a color rub some dirt on it toughen up

0:20:02 Jenessa: Yeah, I think so. There's a lot of things. You know, certainly I'm sure people are familiar with that from the medical perspective, that men are unlikely to report, you kinds of things they might be experiencing or to go to the doctor about it um know men might also be less likely to show up to studies um it not surprising from that perspective I don't know that it's fully understood I just jumped straight to like yeah it's probably a toxic masculinity thing I don' t know um

0:20:21 Thomas: That's interesting. Didn't think about the studies part because I imagine you would be, you trying to statistically control your results based on a participation bias or whatever. But yeah, maybe that's an imperfect process. I don't know.

0:20:43 Jenessa: Some authors do phrase it as like you know a methodological error that they they they treat it like this is a problem, this could have been done better. Sometimes it's thought of as more of like just a human thing. A little bit of both.

0:20:58 Thomas: You talking specifically when it comes to the difference in synesthesia or are you talking generally there.

0:21:03 Jenessa: Yeah, both, I would say. But I'm seeing it mainly in the context of synesthesia when authors are saying, well, why did we think there was this huge gender difference?

0:21:13 Thomas: Okay.

0:21:14 Jenessa: I do see some people phrase it as as methodological error specifically, that this is a problem. You can think of it a lot of different ways. Yeah. There's certain populations where it seems more common and it's not super clear why. And then there's certainly populations where we thought it was more common, like in women. And that seems to not be true as certainly not to to the extent we once thought. I lean towards it just being a reporting problem. We've got it relatively uncommon, probably, depending on reporting. Big The thing for this strong synesthesia, first of all, it's literal, not metaphorical.

0:21:18 Thomas: Gotcha.

0:21:55 Jenessa: I'm not saying I developed a sort of association that like, you know, I memorized colors for the letters for some sort of mnemonic. No, literally, like A is red. Identified and remembered. So this isn't like a mnemonics trick. Like this is just, it's involuntary. It is just always evoked. They They can't, like, choose not to experience it. It's unidirectional. So if they have color pitch, the sound causes a color. The color doesn't make you hear a sound. It direction. It's idiosyncratic. So for this person, middle C might be blue. And for this other person, it might be green. There's no real reason that, you should be.

0:22:44 Thomas: Now that's interesting. I was going to ask you about that. Is there any rhyme or reason? Because you would think if it's, again, it's something consistent, it'd almost think there'd be some rhyme or reason to these associations. Is that because

0:22:59 Jenessa: So there is, like, a trend. So, yes, there are wild individual differences. There does seem to be, like – the letter A is red for more people. Like, there – it's not, like completely – maybe?

0:23:12 Thomas: Red is the lowest in the spectrum of color? Yeah

0:23:17 Jenessa: Like, that's a reasonable association. Association.

0:23:19 Thomas: It's the lowest wavelength or is the highest no it's the lowest yeah because you redshift yeah yeah yeah that's interesting I was actually going to ask along those lines like you would think there could be some rhyme or reason to that like if numbers you well I guess it's hard because numbers are kind of infinite but yeah or if it's like the digits one through nine you know go in like ascending spectral order or if there's just no rhyme or reason to it interesting

0:23:22 Jenessa: Lowest for visible. Yep, absolutely. Right. So what you don't get is that. I haven't seen a case where the individual had like, oh, one through nine. Yeah, it goes red, orange, yellow, green, blue, purple. That's almost like a hallmark of, no, you mean it metaphorically. Um do have a case study of a woman who uh realizing this is what it was um and she fit the whole definition um she'll fit the tests I'm going to describe for how we um for this but they did find in her attic a childhood toy of like the little puzzle of put this letter over and it matched it was the right colors yes

0:24:16 Thomas: And you're saying this was genuine genuine synesthesia though. It wasn't like she had associated. Okay. Interesting.

0:24:19 Jenessa: This so she passed the test like we we believe her

0:24:25 Thomas: Yeah. I'm curious what this test is. Yeah. It's just stop lying. And that's the test. Don't be such a liar. Okay. You got me.

0:24:33 Jenessa: Well so that's the thing is people are going to treat it initially like um must be speaking in metaphor and and they're just artsy -fartsy or they're lying they're they're just trying to get attention, whatever. This doesn't fit my model of how cognition works or my model about sensory works. And rather than adapt that to fit, I'm going to just claim you don't exist. Sure, I mean, if it was all made up, then I guess that's fine. The way they end up doing it to how are we going to test this? Have you heard of the pop -out effect?

0:24:40 Thomas: Yeah, I get it. Yeah yeah. Uh, is it like the magic eye thing?

0:25:18 Jenessa: No, not exactly. I could send you a picture. I know this is not a visual medium, but it might be very informative.

0:25:24 Thomas: It is for synesthesia, people.

0:25:27 Jenessa: Yeah. Yeah, right. So let me, I'm going to send you real quick and hopefully we can describe this reasonably enough.

0:25:36 Thomas: It going to be related to the thing where if you give a person who knows a language a word that's like blue, but it says red, it's hard for them to say the color?

0:25:47 Jenessa: That is one of the tests. It's not this test, but, it is one the ones we're going to use.

0:25:49 Thomas: I was trying to think of how you could adapt that for synasthesia, sort of.

0:25:54 Jenessa: Yeah okay. So here's the, the the first one and

0:25:57 Thomas: Yeah. Okay. OK, so it's going to be like it's a bunch of similar stuff. And then it's typical people, I suppose I'll use the word. It's probably hard for them to see. Like you've sent me something that's all sort of the digital clock version of fives. But there's also some twos. And I can look around and be like, I think. OK, I see one, two, three, four, five. Like I can see them, but it takes a minute. And now instantly right when you send me this, I'm like, oh, if you had synesthesia, it'd be so easy. Like it'd just be like literally someone's taking a highlighter and saying, look at these parts of the thing. They're different.

0:26:33 Jenessa: So we have these they're they're little black icons on a white background and what you're describing is what we'd call a serial search you had to go through one character at a time and you find your first two and you're like oh okay and then you find the rest of them and eventually um you might have spotted they they make a triangle yeah

0:26:54 Thomas: All they do. Yes. I didn't spite that. No, no, no.

0:26:57 Jenessa: It takes it takes a while right and

0:27:00 Thomas: I admittedly, I didn'T try that hard, but I they make it wait. They make a triangle because I'm seeing a two, a two to two to oh, a triangle like pointing sort of to the lower right.

0:27:04 Jenessa: If uh no they're pointing towards the top see it's very very hard to see, right?

0:27:14 Thomas: Well, sorry. That was the stupidest thing to say. I'm so sorry. It was, if it's an equilateral triangle, then it's not really pointing anywhere. I thought it was like, no, I, for a second, I thought I only saw some of them. So I thought it was, like, an arrow, kind of, you know, like if it was skinnier on one side, you could say it's pointing somewhere, kindof. But then I saw that, I saw, that last two, that to the left more, I was like oh, that was a really dumb thing to say.

0:27:19 Jenessa: Jesus. It doesn't point any. Yeah, yeah. No. Oh, geez. I'm like, no,

0:27:36 Thomas: It's for sure an equilateral triangle, yeah.

0:27:37 Jenessa: It's pointing the other way. Yeah. Yeah OK. So it is an equilateral triangle of twos. So what's characteristic of this serial search, the more fives I put up there, the longer it would have taken you to go through and find it. What is characteristic for our pop out effect is, I've sent it to you here with the fives are green and the twos are red.

0:28:05 Thomas: Yeah, that's a lot easier. I could pass that test. Now, Arlo couldn't. You got to be careful with the colorblind folks now he actually does have synesthesia but he's colorblind so the test takers givers missed it you

0:28:18 Jenessa: Yeah, so here we have a parallel search. All

0:28:24 Thomas: Know that is in crumb sorry that's incredibly interesting statistically it's I'm sure it's not super likely but there must be some percentage of people who have colorblindness and synesthesion and then like oh okay I can't well we'll put a pin in that anyway point is this very festive christmas colored fives and twos incredibly easy to see and so you're saying for people with synesthesia they look at it like this they're like duh that's a triangle does not pointing anywhere thomas and Jenessa you idiots it's not so try it's an equilateral triangle you can't say pointing anywhere it's pointing all three directions

0:28:34 Jenessa: I have a case for you. Yeah! Yeah!

0:28:51 Jenessa: Yes! So what's critical here with this parallel search and this pop -out effect, no matter how many green fives I throw up on that background, in 50 milliseconds approximately, you are going to spot those twos and respond and tell me what you see. And it doesn't matter how Many Fives. It just pops out of the background. There it is.

0:29:19 Jenessa: You can't look at that black and white example and voluntarily do that for me. There's no amount going to practice that's a serial search this is a parallel search and for our synesthetes we will get a pop -out effect a parallel surge 50 milliseconds no problem and they that's not a thing they could have like practiced

0:29:29 Thomas: Interesting yeah you can'T fake it no that's that'd be and you know what if they they did fake it, fine. You have synesthesia. I'll just, whatever.

0:29:41 Jenessa: You can't fake it yeah

0:29:46 Thomas: You did such a good job. You worked so hard at, let's just say they have it.

0:29:50 Jenessa: Sure so so this is really was one of the most definitive tests of like, no, they mean literally, I have any other answer for you. This has to be.

0:29:55 Thomas: I bet. Yeah. And, And, but interesting is this also tells us, and maybe this was already known or something, but it also tells us that it's like, it's involuntary and happens. Now I, I don't, I'm not going to get probably the right terms for this, but I know there's, there's like processes in our brain that we're going to to be conscious of and unconscious of right and so it seems like this because because I could imagine a possible synesthesia world like I could imagine it being true potentially even though we know it's not that synestesia required me to first serial search out those twos and then my brain would assign them that color and sort of in the same way that you said a minute ago that someone had to be explained which letters of the whatever were equivalent to the letter a And then it changed. Like, I could imagine that that would have been the case. But to see that it's not almost mean so. So it happens kind of, if not unconsciously, maybe that's not quite the right word. But it does happen at a level kind of before we're like thinking about it deliberately. Deliberately. Is that right? Right.

0:31:00 Jenessa: Yeah, you're absolutely right that this not only told us this is real, this is involuntary, this has not a metaphor. It also tells us something thinking about where this is happening in the stream of processing, this has to be extremely early. This can't be happening up at some higher cognitive frontal lobe decision -making attentional process where I have to be wanting to look for and think carefully, like, no, oh, this is really basic visual processing.

0:31:35 Thomas: At the same time, it's names and not let you know, like there's other things about it. They're like, well, that's kind of weird. Does seem like it takes some amount of processing first.

0:31:46 Jenessa: Sure. Yeah. So, it's definitely not, like, happening in the retina, but it's certainly not happening all the way up at, like intent conscious decision making. So it's somewhere early -ish.

0:32:01 Thomas: This is so fascinating.

0:32:04 Jenessa: It's trying Trying to pinpoint this is really tough. They took in one of the studies that looked at things like this, they took some synesthetes and they did, for example, okay, you're telling me the number five is blue. I'm going to put it in your central vision. You're going to tell me that. Great. I'm gonna slowly move this into your periphery. And it would hit a point where I can still see in my periphery that is the number five. It has lost its color.

0:32:28 Thomas: Wait, because, yeah, your peripheral vision doesn't have color, but but isn't what? What? So it also lost color? That's crazy.

0:32:37 Jenessa: It lost color, but it was still perceived. Perceived so yeah

0:32:41 Thomas: So that's weird. So, that would tell us that it is still... But the thing with the names is throwing me off so much. Like, how could it be both those things? Because that sounds like it's way more in the early times of your visual cortex text processing whatever that stuff fancy science words yeah

0:33:02 Jenessa: So for example there was a woman who had um uh months had color you know january february march the word march can also be a verb and when it is in the context of a verb it did not have color for her so

0:33:11 Thomas: What okay I don't believe her let's get her back on this test and figure out we'll put here's what we'll do we'll make a thing that's a bunch of marches and some of them are the month and they'll be on a white background and some other month and some are the verb and we'll see if she can just pick them out sorry Sorry. That's really funny to me.

0:33:30 Jenessa: So clearly for her it is yeah there you you go. Clearly, context is relevant, right? For these higher order, more complex associations, context is going to have to be relevant. And some names are nouns. So some these are really simple. A pitch has a color, a shape, the letter A has a colour. But if the letter O has a a color. Not every circle has a color, so it's still a little higher up than just frog brain.

0:34:02 Thomas: That I so that's why this peripheral vision thing is like that. That actually seems like the outlier to me if that person's lying. Sorry. Not real.

0:34:10 Jenessa: Well, so they had a couple of people in this case study. Part of what's difficult is it does seem to be either rare enough or the reports are rare enough that you are often doing case studies or small samples. This is also where we get our colorblind person.

0:34:27 Thomas: Ooh, here we go.

0:34:29 Jenessa: This is the only example I have ever seen of a case study of a colorblind synesthete. And let me tell you, there was more where I was working on a paper on this. And if are other case studies on this, I'm very surprised I missed it. Um this individual um are color graphing so letters have colors they also have uh faces are producing colors for them um so you different types of faces photographs and cartoon faces produce some colors they have a type of color blindness called S cone weakness. So the S cone is the the one that perceives like, yeah, yeah. So you can have a type of colorblindness where of your three cones, one is missing. And that would more significantly impact your color perception.

0:35:05 Thomas: S cones are weak. Right?

0:35:17 Jenessa: In a weakness, it's shifted so that its sensitivity is closer to the the other cone. So if the one that perceives green and the one that perceieves blue, part of how you differentiate them is the wavelengths they're sensitive to are further apart. So, if one is firing a lot and one is fireing a little, it's more green or more blue. If their sensitivity, which wavelength is like their favorite to fire to, if are closer together, you're going to have a harder time distinguishing them. Them. So in this S -cone weakness, where the sensitivity of the blue cone is shifted towards the green, this is going to produce a yellow -blue type of colorblindness, where they have a harder time telling apart yellows, greens, and blues. They look much more similar. And the deepest blues and the purples are going to be not visible to them almost at all, because uh that cone because its sensitivity is shifted over um people with typical color vision that cone will be sensitive out to a certain degree of purple but like not ultraviolet light right it

0:36:15 Thomas: What do you mean not the difference won't be visible?

0:36:31 Thomas: Oh, it's all shifted down. You're saying? Sorry, I missed that. Yeah, yeah.

0:36:34 Jenessa: It's shifted towards the green and towards the red

0:36:36 Thomas: Yeah, so anyone's vision has a limit, sort of like how our hearing has a limit is what you're saying. So like 20 ,000 Hertz or roughly, I think is, or maybe it's 10 ,000. I don't know. It's something, one of those is like, we can't really hear.

0:36:42 Jenessa: It's pretty high yeah uh

0:36:48 Thomas: Yeah. We can't hear anything higher than that usually. And so it's same for vision. So you're saying they literally can't see the color at all.

0:36:55 Jenessa: So past a certain point into the deep blues and purples they're not going to get it really at all and for yellow blue and green they're going to have a hard time telling it apart um

0:37:05 Thomas: Does it look black? What does it look like?

0:37:08 Jenessa: If you go anybody can do this if you want to pop into google Google like colorblindness example, yellow, blue, red, green. They'll show you the picture of everybody that everybody does. It's like a little spread of like vegetables. It'll be like a bunch of different tomatoes. And they'll show like normal color vision, proto-anomalous, deuteranomalous. And these are like different versions of the red cone weakness and the green cone weakness. Green colorblindness is more common. And that's where telling, you know, my partner has this where basically reds are like a warm brown and greens are like cool brown. They're not super different.

0:37:48 Jenessa: They are not super pigmented. Whereas yellows and blues really pop for him. For yellow blue colorblindess, it's going to be kind of the other way. Reds and greens, I got this. Those two cones are in the typical position. And so looking at the difference between the firing of the red and the green cone will help me tell reds, oranges, yellows, greens. But when I'm trying to distinguish the other colors by telling the difference in firing rate between the green and the blue cone, I'm not going to do so good. Someday if you ever want to talk about those colorblindness glasses, there's like a whole method to get into

0:37:52 Thomas: Interesting. Interesting yeah I'm curious about that for Arlo maybe someday but I can't remember if they work or not yeah it's also interesting because I can't really get Arlo's too young I don't understand what it looks like to Arlo yeah I mean I know I can do that thing but but I used to always think like oh do they both look brown but but he kind of when I ask him that that's not what he says you know like he doesn't that doesn't make sense to him and I haven't quite figured out if we're struggling just on a, you normal five-year-old with words thing, or if he sees something other than that in a way that I don't get.

0:38:27 Jenessa: To this. A whole other dive. Yeah. Yeah, yeah. It can be really tough to communicate across any sort of perception barrier, right?

0:39:01 Thomas: Yeah, sure.

0:39:01 Jenessa: It And can be tough to communicate with kids um

0:39:03 Thomas: Yeah. And Arlo in particular, but it's also so sad because what he'll tell, he's told us a few times, oh no, I can see color now. I learned it. We'll be like, oh buddy, no, no. I know he's done that a lot. I'll be like, so does this? And like, Oh no ,I can see it because people in class tell me. And I'm like well yeah I mean it's good they're telling you but then I don't think that yeah whatever sure I

0:39:13 Jenessa: My heart uh by a definition I suppose um so so we've got our person here who have this S-cone weakness um and blues not so different uh blues and purples not perceived at all um they also just so happen to have a grapheme synesthesia where uh letters and numbers have color and

0:39:46 Thomas: Can't believe I predicted this

0:39:47 Jenessa: Yeah so what he describes uh I'll just quote, I'll... the paper intriguingly this subject is color anomalous S-cone weakness But he sees, quote, weird, number -evoked colors that are, quote extra-spectral for him, i.e. he cannot see them in the real world

0:40:08 Thomas: My god that is the coolest thing I've ever heard in my life because I was just thinking if you oh my God that's incredible that's like I'm genuinely getting chills thinking about that that's that's the old thing of like oh imagine a color you can't conceive of and and I are like yeah impossible but I can kind of like I can think of the concept of that like there's no reason that the exact number of colors that I—that the human brain tends to label there's No reason for it being exactly what it is it could be way more fine-grained and we could have all a color is is a mental label and wow that is that that might be like the only person in the world who's ever had, you know, the number of people who can literally in their brains, at least see a color that they can't otherwise see is gotta be a handful. Like that's crazy.

0:40:43 Jenessa: -hum, yeah Yeah, just the statistical coincidence of these two things co-occurring in someone one. And his retina is incapable of this.

0:41:07 Thomas: And Yeah. Oh my God. So just to make sure I have this right. So the issue with colorblindness is the rods and cones. It's well with whatever, I don't know if there's other ones, but like, yeah, we'll see it was the cones it's the cones the cones have done sure whatever uh parks and rec uh and it's not the mental label for it so it's not as though if they somehow if we were to bypass someone's science words for wherever in the visual thingy that happens and to like we could just matrix mode we could plug in a cable to where that happens then we're like all right instead of the the essential the reading you're getting from your cones like which is sort of what's happening in a way, right? You're getting

0:41:19 Jenessa: The cones. The cones. Yeah. Sure. Yeah, yeah. No. Yeah yeah yeah. Oh

0:41:50 Thomas: Like some sort of value. If it were a computer, you're getting that, like, you know, the numbers that they do where it's like the, do they call that? Where you can do it by like, okay, how much of this color is it? This color is it? And this color, is it, and that's the color you are on a computer.

0:42:03 Jenessa: It's very much works that way in a lot of ways. Yeah it

0:42:05 Thomas: And so hypothetically, if we could plug someone into that, who is colorblind and just feed them that data their brain would have the label for it and it would be a color and this person is able to see those things in his brain with his brain that he can't otherwise see because he doesn't get that data coming from his cones is that it is the coolest thing I've ever heard like I I, I really just am like, my mind is blown by that. Like, imagine that. That's so cool.

0:42:26 Jenessa: That's what what it appears to be going on.

0:42:37 Jenessa: Yeah. When he looks at the world and photons hit his retina, he is not capable of perceiving like purple. But when he looked at a letter, there is no reason that his brain that is producing colors seemingly randomly can't produce purple unless we were to imagine a sort of like, oh, well, the only way you see color is if you, like, get exposure to it and your brain learns it.

0:43:06 Jenessa: It appears that's not how color works. Like, we're learning how the brain works by looking at him. Like what they write here is this observation, again, supports the cross wiring hypothesis rather than the memory association hypothesis. You cannot remember a color you have never seen.

0:43:11 Thomas: Oh man. That is amazing. Never seen. Yeah.

0:43:29 Jenessa: Further, the study Study of this subject may throw light on the old philosophical conundrum raised by Molyneux, whether someone can experience completely novel qualia never before experienced. So it does touch that of like, it seems like your brain, you know, whether as an adult, maybe prune connections you don't use. To what extent I would quibble with some of my colleagues. Um you know there's like the old guard and the new guard on this um uh we are talking about someone who has had synesthesia uh this is thought of as lifelong seems like the brain comes a little bit pre -programmed with some of what to do about

0:43:33 Thomas: The, the room. Would

0:44:10 Jenessa: Colors. It would have to.

0:44:10 Thomas: Have to. It would, it would have, do you think?

0:44:13 Jenessa: Yeah.

0:44:14 Thomas: Otherwise, because, because now, like, maybe I am taking it it a step too far, but just as you were reading that, I was like, well, how could it work any other way?

0:44:22 Jenessa: Would think, right?

0:44:22 Thomas: Yeah. Cause like, it's not as though seeing, ah, I've seen purple. Now I can remember purple. No, you've seen that wavelength that take people out of the world. That wavelength doesn't contain purple. Like it just is a wavelength. So like how, yeah, that's interesting. And now I'm thinking like, I imagine the same thing could apply with other senses. So like, and even in a very plain, boring way. So let's say you have some sort of damage that causes you to lose. And this happens all the time, a band of hearing like, so like, oh, I can't hear in the, I don't know, 2k to 5k Hertz range anymore. That's gone.

0:44:35 Jenessa: Of course not. Yeah? Mm -hmm. Mm -huh.

0:45:05 Thomas: But you would still, Still, I'm sure be able to like imagine your favorite song, possibly if you're someone who's good at being able to hear kind of music in your head or or you might even maybe have auditory hallucinations, which can happen, I think, to pretty much everyone at some time. And it's not like that would be missing that just because you detecting instrument in your ear is no longer able to detect it doesn't mean the brain label for that has gone away. Or let's say you were born without the ability to detect it ever, like just maybe your ears, you know, different in a certain way. That wouldn't mean that had you no capable, capability in your brain of labeling that hurts, you know, value with, vibrational, you know, kind of thing with, with a sound in your head.

0:45:49 Jenessa: Sure, like you didn't come out a completely blank slate, that would be very to imagine. So you're exactly right, as when people lose hearing, you know, older adults especially, it's very common to start losing your hearing, and you usually lose the high frequencies first. Majority, almost everyone who loses their hearing will get some amount of tinnitus, some ringing in the ears. Many people, and it is hard to say how many because people are afraid to report it, it progresses to musical hallucination.

0:46:22 Thomas: Progresses to that.

0:46:23 Jenessa: Yeah, they'll start with tinnitous and then eventually they're like, so I'm hearing church music.

0:46:30 Thomas: Like once they've lost a bunch of their ability to hear.

0:46:33 Jenessa: So I've been, you know, only for like the last decade and I'll never finish it. I've been working on a theory paper that pulls together a bunch of these case studies, and it appears to progress for both losing your vision and then visually hallucinating, for losing your hearing and then auditory hallucinating for losing a limb and then getting phantom limb. It appears to follow a, a really stereotyped progression of instant loss, instant hallucination, slow loss.

0:47:04 Jenessa: It takes two to three years and you begin hallucinating. I've been working on this for a while and then I was like I have to I have to quit this job I'm dying um someday somebody please plagiarize me I will happily give it to you

0:47:14 Thomas: This is so fascinating, but wonder if this answers or could potentially say something about the question I had instantly to Eli because when he was talking about, I don't, nobody hears that. Nobody blah, blah, like not to rehash that conversation. But one thing I asked him was, do you think that Helen Keller had no mental experience? You know, like having never seen or heard anything ever, like hypothetically.

0:47:35 Jenessa: Which doesn't sound right and

0:47:39 Thomas: Yeah. Like, so now I'm thinking like someone who has never heard or seen anything could be having all kinds of visual or auditory experience that like, yeah. Like, it's not like as though it would only be. Yeah no label and maybe even no way to communicate about it because it's so like again if you tried to say what would a color look like that you've never seen it's like I don't know there's no way to say that

0:47:51 Jenessa: Not have a label foreign right he doesn't say he doesn't say to these researchers purple he says weird never seen it before don't know what it is yeah

0:48:08 Thomas: Weird yeah yeah indescribable yeah that like it would be that's so crazy that that is the coolest thing. Wow. So I think we went way longer than we intended because that was blowing my mind. And hopefully at least some of the other minds listening but there's way more on this. So we're going to have to regroup for a part two sometime, Jenessa but thank you so much for starting us off. I will be thinking about this for a while.

0:48:24 Jenessa: Of what I figured is I didn't want to structure it yeah

0:48:37 Thomas: And in part two, what can we maybe tease for people that they're going to learn further about this?

0:48:36 Jenessa: Cool yeah would love to start part two with some people who who weren't synesthetic until they lost other senses.

0:48:53 Thomas: That's awesome. Plus, uh, a little bit more about how this happens or yeah.

0:48:58 Jenessa: Yeah. Yeah. Some theory behind like how.

0:49:02 Thomas: Blowing my mind. I, this is so cool. Well, thank you so much. Uh, do you want to throw anything out there? I forget. Do you have anything to plug or it seems

0:49:09 Jenessa: Oh, hey, New Yorkers, flip your ballot over. You have ballot proposals. And I think it's like 31% of New Yorkers forget to flip their ballot over and vote on proposals. Yes, statewide Prop 1 amendment to the Constitution to extend equal protection to like like, age discrimination, disability discrimination, LGBTQ discrimination, would be real cool to not forget to vote on that.

0:49:34 Thomas: Like a good idea. Yeah. Flip your ballot over. I That would be something someone would need to tell me. So no shame, anyone.

0:49:36 Jenessa: Flip your ballot. Yeah, it's on the other side of your paper ballot.

0:49:43 Thomas: All right. Well, I love that use of the plug in order to do something pro -social. You're amazing. And I cannot wait for part two. So excited.

0:49:52 Jenessa: Sounds great.