SIO463: Synesthesia - How Does It Work?

Transcript

This transcript was generated automatically and may contain errors — the audio is the record.

0:00:16 Thomas: Hello and welcome to Serious Inquiries Only. This is episode 463. I'm your host Thomas Smith, and I am, again, so excited for this episode. We're back with Dr. Jenessa Seymour, and we're going to be talking about, she's going to teach us all about synesthesia. That's something I'm sure most people are loosely familiar with. Senses activating or triggering when you wouldn't think they would.

Seeing colors with, seeing colors with music, all kinds of combinations. She teaches us all about it, and this is so fascinating, seriously. She goes through something in this interview that I'm still thinking about, and that blows my mind so much. I cannot wait for you to hear It is so cool, and it was so cool that this extended into two parts, and it is, trust me, it is worth it. Like, I loved doing this interview and learning more about synesthesia, and it's topic a, it's that Jenessa is passionate about too and has studied a lot.

Uh, she's a PhD in neuroscience, by the way, in case you didn't remember. Um, and this is in particular something she's looked at a lot, and it is so interesting. I can't wait for you to hear the thing that has been mind -blowing and stuck with me, uh, this entire time until now. And so I just get to it after this break. Uh, we'll get to Jenessa. Please support the show, leave the ads and all that at patreon.com/seriouspod. It means a lot. It's really important. Do so um, and after this We'll be on with jenessa about synesthesia And Jenessa, welcome back to the show.

0:01:38 Jenessa: Hey, how you doing?

0:01:40 Thomas: Great. This I is something I feel like has come up a number of times tangentially and stuff we've talked about and is so fascinating. And I feel like it's something you're also very interested in. So can't wait to get into it

0:01:46 Jenessa: Mm -hmm. Yeah, yeah. I, you know, I like this one especially because after all the things we've talked about, like, nobody is dying, no rats were harmed in these experiments, just fun, chill. Well, yeah, it would be tough to do this kind of perception with rats, I think.

0:02:03 Thomas: That we know of.

0:02:13 Thomas: But we can ask everyone what color is the podcast they are listening to right now in their ear holes.

0:02:19 Jenessa: Oh, boy. We're going to have some people come out of the woodwork, let me tell you.

0:02:23 Thomas: Yep. Yep. That's cool. Where do we where do we begin right there?

0:02:27 Jenessa: Yeah, so I was thinking we could start with just to like crash course, what is regular old perception before we get into like weird perception?

0:02:38 Thomas: ,that's a tough topic. Are we going to talk about, yeah, the demon, not Laplace's demon. Who's the other one? Descartes. Yeah, we're going to start. No, we are not going philosophy. We are going just, yeah.

0:02:40 Jenessa: Yeah. Oh, yeah. Okay, so we're gonna. Right. Yeah, going to set some definitions that you know, some philosophers might not love, but we're We're going to operate in the world of, this neuroscientists are approaching it. Yeah, only something that I spent a decade and a half studying, no big deal. We're gonna condense it into like five minutes. But yeah, we're venturing into the world of psychophysics, which a good word, right? Basically just the meeting of the physical world and psychological world. How does some physically measurable thing become a a psychological experience.

So, So, yeah, hit your retina and you experience vision. Pressure wave gets into your ear, you experience sound. And want to differentiate differentiate those because, uh, is objectively measurable. We have devices for this. Um, once we agree on a definition, there's one right answer. What is the amplitude of the pressure wave in the air in decibels? But then you experience internally internally loudness that goes along with that, and that experience is subjective in the sense that it's internal and in the since that I measure it by asking you, not subjective in the sense that it's like major individual differences, like

0:03:30 Thomas: Gosh. Yeah. Right. Yeah.

0:04:23 Jenessa: You're not going to find someone where the decibels go up and they tell you it gets quieter. Um, there is a a very strong relationship, and psychophysics is studying that relationship. What is The physical stimulus, what is the experience of the person, that that's basically my whole bread bread and butter.

0:04:37 Thomas: Interesting. Yeah, it's so fast. I mean, this is something, again, not to get into a bunch of the philosophy, but I have always, you know, like, my experience with a lot of this stuff is in reading philosophy. And also biology, it's crazy to think about. Like, it is really crazy to thing about the fact that, like, somehow with our nerves and our brain and our consciousness and all that stuff, we have these experiences that feel like things. Like, feels like stuff for us. It's so basic and yet so really impossible to even comprehend in a way. It's like, and you can't really just turn it off, you know? It's not like you can just be like, well, I'm tired of my leg hurting. I'll just not. But it's like, you in some sense, it's like you say, it'S subjective, but it'S also like, I wonder how much of that is us having all of the information. Like you, I guess you could imagine if we

0:04:55 Jenessa: Yeah. Yeah, no, yeah. Sure,

0:05:38 Thomas: Knew everything about someone's, you brain and nerve endings and all that, like all that stuff, we could, in a way, in theory, I think we would have to be able to say that we could, like, know exactly how someone will perceive something or not. I don't know. Like, I think, I think the answer has to be yes because there's nothing magical happening, but at the same time there's so many unknowns.

0:05:58 Jenessa: Sure. And I think to some degree neuroscience gives us the tools for that. Know, in a lot of way, I think of neuroscience not as, it's at least, you know, from the direction I operate, it's more of a tool than its own separate thing. If we use the tools of neuroscience, you know, I can look at the way your brain is activating and I can make some very good guesses about what you are perceiving right now.

0:05:58 Thomas: Yeah. Yeah.

0:06:25 Jenessa: That's actually going to be a huge tool for us when we get into people who are having atypical perceptions. Um, I'm going to measure it, and also to some extent how I am going to, I don't know if prove it is the right answer, validate it is probably a better way to say it. Know, I always find it interesting, like, to think about there's no real reason, like, like photons had to be light as the subjective experience. It could have been anything, right?

0:06:42 Thomas: Yeah. Yes.

0:06:54 Jenessa: Know, we want to perceive them, but sure, sure.

0:06:54 Thomas: And No, I mean, it had to be something that objects give off all the time, so I guess in that sense it could have been something else, but I don't know if there's, like, another good candidate for what that could have have been in our, the physics as we know it. You know, I mean But then again, I, you know, to bats, the famous, whatever that is, that essay, the "What It's Like to Be a Bat" thingy, whatever that thing is, like in a way, to them sound and vibrations might be quote-unquote light, like it might actually be to them what they would consider light, they might see it in away.

0:07:01 Jenessa: Yep, absolutely. Sure, sure. Mm -hmm, mm -hmm.

0:07:27 Jenessa: Yeah, now you're in my exact bread and butter because, you, we have people who lose their vision, they still have this area of the brain we would call visual cortex. Of course it's not doing that. Um, they learn echolocation abilities. They will often report, uh, of all, we will see the visual brain areas activate as they use sound to navigate their environment.

0:07:47 Thomas: Yeah.

0:07:48 Jenessa: And their subjective report is, no, I don't, I don't mean, like, metaphorically, I am, like, seeing my environment. I literally, when I make those little clicks and the click bounces back, I am having a visual experience of my environment experiment. Described it like a, like a camera flash lights up the room for a brief second. So, um, there's really no reason it had to be, it had to be experienced if we wanted to use the information in our brain, but it didn't have to be experienced the way that it is, you know. Um, certainly philosophically there's so many little questions in there, and And psychologically, neuroscience, we're kind of just landing on, here's the definition I'm going to work under because I got to be able to do an experiment, essentially.

0:07:57 Thomas: Yeah. Yeah. Yeah. Huh. Yeah. Right.

0:08:37 Jenessa: And really, this does illustrate, I think, in a lot of ways how atypical experiences are so useful for research that it helps us to understand how certain things work. And it also challenges assumptions. When we're going to look at synesthesia and people who are having that experience, you have to come up with a theory and an explanation of sensory perception that accommodates them and allows for them because they exist.

0:09:06 Thomas: Yeah, Right. Yeah, unless a bunch of people are making it up, which, you know,

0:09:12 Jenessa: Well, that's exactly where we want to go, right? So, let's just start with, like, defining it. So what even is it, especially if people haven't heard of it? Like, one sentence I can give you is when a stimulus in one sensory modality is going to evoke a perception in that modality and a second illusory perception.

0:09:15 Thomas: Yeah.

0:09:37 Thomas: Illusory, interesting. That's a loaded term in that, like, what, what is an illusary perception and what's not, you know, know how to even define that. Yeah.

0:09:42 Jenessa: It is. The definition we're going to, you, to stick with is that, um, there's a pressure wave hitting your ear and you hear a sound that is typical. I'm going to stay away from, like, normal because that is, like, value laden, you know, kind it, but it's typical, it's, it's normal, statistically normal to have a pressure wave hit your ear and hear a sound. Not typical is to then see a color. Not photons hitting your eye and yet you are having a visual perception. And so in that sense, it is illusory.

0:09:58 Thomas: Exactly. Right. Yeah. Interesting. But it's also like, how would you, and I totally, uh, I'm not trying to fight, fight you on any of this. I just think it's already interesting to think about, because I, I feel like if it was, it's very, and admittedly so, based on just kind of statistic norms or whatever, like, it's, it's like if everyone had that experience, I don't think it would be looked at as quote-unquote illusory. It'd be like, that's just part perceiving color or sound or whatever. And away, I'm not trying to fight or anything on the devil. It's just interesting to think about, like, what means it's illusary. If it's consistent for someone, like if that was the only person on the earth and they had synesthesia, they'd be like, well, that's juste normal perception.

0:10:27 Jenessa: Sure. No, absolutely.

0:11:00 Jenessa: Yeah, I totally agree, and I actually, this is one of my biggest complaints about how we talk about this research. I think in some ways this is informed by a lot of my background and my work with disability, that way we define, like, what's normal and what isn't is very social, social, and, um, some ways subjective. Um, so yeah, I've got examples for you where I, I feel very strongly that all humans are having a thing I would call synesthetic and it's just statistically normal. So we're not calling it that.

0:11:20 Thomas: Mm hmm. Hmm. That's very interesting.

0:11:38 Jenessa: Um. Like I would say, for example, um. Empathy. I see you cry and I just get so, so sad. I might even cry myself and And like, I don't know, isn't that kind of, I'm not sad. I'm the thing didn't happen to me.

0:11:52 Thomas: Yeah. Yeah, I guess the only distinction would be, is that a perception, or is that, you know, do you call that a more complicated process? But yeah, I see your point though, for sure.

0:12:03 Jenessa: Yeah. And like there are other examples like that. So there's a lot of the distinction. The definition I gave you is specifically what we would call strong synesthesia. And we have a separate definition for weak synesthesia. It kind of fits what you're talking about. Out. So like strong synesthesia, what is it? Got my little table here from a paper that does a really thorough review. The prevalence, it's normally thought of as pretty uncommon. Hard to measure the prevalence because a lot of people who have this experience don't know that it's not normal.

0:12:40 Thomas: Yeah, that's so funny. Yeah.

0:12:42 Jenessa: You could think about like color pitch synesthesion. I hear a sound and I see a color, right? We all say things like, oh, this piece of music is like really blue, like the blues. I just mean that metaphorically, right.

0:12:54 Thomas: Well, Yeah. Okay. Sure.

0:12:58 Jenessa: I'm not literally seeing blue. But if you do, you'd be like, yeah, I agree. It's very literally blue.

0:12:59 Thomas: Yeah. Unless for them the synesthesia makes it red. Maybe, like, what is everyone talking about?

0:13:09 Jenessa: Right. Right, like you guys are out of your mind. It's very common for people to get into their 20s, 30s, 40s before they finally figure out, like, wait, you don't?

0:13:13 Thomas: Yeah. Yeah. Well, we see that and I don't know how related this is, but we see that with the inner monologue or the a Fantasia stuff too, where it's like so many people are like, I did not know this was a thing at all.

0:13:29 Jenessa: Oh, yeah. Absolutely. And I think for a lot of those people, they're like, well, I mean, I think thoughts. I thought you meant, like, metaphorically, you're literally hearing a voice in your head? Like, what are you talking talking about?

0:13:40 Thomas: Yeah, yeah. Oh man. Did you, okay. Did You Hear the Dear Old Dads where Eli was talking about this? Oh Oh man.

0:13:50 Jenessa: Oh, I don't think I've gotten to it.

0:13:52 Thomas: Oh no. That's funny. Cause he, he has this whole thing about like, he thinks everyone, he says they don't hear it in inner monologue. He thinks they're all like wrong about what we mean by inner monolog. And then at the end, after a while, I'm like, dude, I think maybe you don't have an internal monologue or or it's at least different. Cause he's like, no, you don'T literally hear thoughts of, Oh, I'M going to go over.

0:14:15 Jenessa: Yeah.

0:14:16 Thomas: And it's It's like Tom and I were like, well, actually kind of, yeah, like we do literally hear a voice saying thoughts like that is how it works for us. And I still don't totally know. The reason I don't know with him is because he does seem to suggest that like he has it sometimes. And I feel like the people who have no internal monologue, it's not like it turns off and on. I don' know. So I don'T know. I DON'T know what's going on with Eli. But yeah, it was a really funny episode.

0:14:38 Jenessa: Yeah, I know for that aphantasia for the internal visualization, there is a spectrum. So some People experience that as, like, absolutely not, never, what's wrong with you people. And some people are like, I have a friend who explains it as, like, he doesn't picture his home, but he can kind of picture, like, an architectural sort of, of diagramming, like a rough sketch might come into his mind

0:14:46 Thomas: Right. Yeah. Yeah. Yeah. But that would be different than saying, yes, sometimes I see the full entire house and sometimes I see nothing. And so I don't know.

0:15:04 Jenessa: Um Right. Yeah. I agree, yeah, yeah, yeah, absolutely. Like, for me it is extremely literal for both of those. Like, I could, you guess, close my eyes, close my ears and, like, hear my grandmother's voice and I haven't spoken to her and, you know, she, she passed away over a decade ago and I mean it literally, I could hear her, um, if she was right next to me but not in a, like, schizophrenia hearing voices kind of way I, I know It's, I know It's me, right?

0:15:24 Thomas: Yeah, yeah, exactly. You're not using your literal ears to hear an internal, you know, like,

0:15:40 Jenessa: Absolutely. So, so this is very similar, I think, um, was one woman whose story was she was at a baby shower and people were talking about, like, names and, you know, they're all throwing out, like, opinions about pick a name? How do you pick a name? She was like, yeah. And I mean, like, what do you do if, like, you really love this name, but, like, What if your partner doesn't like the color? And they were like, What are you talking about? And was like 50 something. And that she was when she learned that for everyone else names don't have colors.

0:16:03 Thomas: That's awesome. That's so funny.

0:16:14 Jenessa: Um, yeah. And

0:16:15 Thomas: But Now that's interesting. Now, do people have it where it's name, so not all words that is weird. Okay.

0:16:22 Jenessa: It can be very specific like that. Yeah.

0:16:26 Thomas: That's, that's hard to understand. Sure.

0:16:29 Jenessa: Yeah, Yeah. Yeah. So, so like the most common, or at least the most commonly reported is color grapheme. And what that means is letters and numbers, like the, the letter itself having a color. A is red, B is blue, C is on green. I'm just making stuff up. Um, I don't have any of these experiences if anybody's noticing that I not speaking from personal experience here. Will usually have it as well for those individuals. They will usually have the whole alphabet will have something. Was a really cool thing where they taught people who had it for the the Latin alphabet, they taught them Cyrillic, it's like slightly different letters. And as they learned, this is roughly an A, it instantly adapted and picked up the colors. So it does transfer in

0:16:47 Thomas: Yeah. Yeah. That's Interesting.

0:17:21 Jenessa: That way.

0:17:21 Thomas: And it feels like there's some sort of, like, that, that feels like there's some sort intermediary process then, like, it has to, you know, like If you have to explain something for them to get, like, wow, yeah, that is mind-blowing. How the hell does that work?

0:17:30 Jenessa: Yeah. So like it's not just the shape of the thing because then like, oh, every circle would be the color, right? It is about the meaning.

0:17:44 Thomas: Interesting. That is interesting.

0:17:48 Jenessa: One's pretty common, you know, common -ish, right. There You is color pitch where different notes have a color.

0:17:57 Thomas: Here's the question I have that I don't know if you have the answer to. I have often kind of wondered if everyone who has perfect pitch, which is pretty rare, if they have some sort of synesthesia thing going on.

0:18:05 Jenessa: Mm -hmm. Statistically more common. Not 100 % of the time, but a lot know, it seems more common in musicians in general, and that could be, you Know, synesthesia, yeah. Artists in general. Chicken or the egg, hard to say.

0:18:10 Thomas: Right. Synesthesia does? Oh, interesting.

0:18:23 Jenessa: You know, did it lead you to it? Did it make it easier? Or did did early exposure cause this, make it more likely to develop? To say, it's more common in autism. It is more common in savants. I don't know that that's even really a diagnosis anymore, but that form of autism, it was thought for a very long time to be significantly more common, to the point that some people were proposing there must be a gene for it and it must be fatal to the male fetus.

0:18:23 Thomas: Right. Wow.

0:18:59 Jenessa: Then it turns out it's just about the way you ask the question, that, like, men don't want to tell you that they're experiencing it.

0:19:06 Thomas: Yeah, Because certain numbers are pink and they're like, "I'm not telling anyone this because it's, you know, be too too feminine."

0:19:11 Jenessa: I'm not gay.

0:19:12 Thomas: Yeah. No. I don't I don't see pink, there's no way.

0:19:12 Jenessa: Like, yeah. No, music doesn't have a color. Yeah, and if you ask the question the right way, the gender bias drops significantly, depending on the study.

0:19:26 Thomas: Interesting.

0:19:26 Jenessa: It might become completely equal. Um, there

0:19:28 Thomas: That's crazy that it went from that, like, what in the question made such a difference that, like, I actually, I may, I joke, but I don't understand what that would be. Hmm.

0:19:39 Jenessa: There is a general thing that just overall, many different domains, women are more likely to report atypical experiences, um, that's report, court, not have. Like, many, many times we find that if you ask the question the right way, all of a sudden men will disclose.

0:19:48 Thomas: Yeah. Oh, Okay, so it's like from an early age, man, like, "Ah, my name is a color, rub some dirt on it, toughen up."

0:20:02 Jenessa: Yeah, yeah, I think so. There's a lot of things. You know, certainly I'm sure people are familiar with that from the medical perspective, that men are unlikely to report, you kinds of things they might be experiencing or to go to the doctor about it. Um, know men might also be less likely to show up to studies. Um, it not surprising from that perspective. I don't know that it's fully understood. I just jumped straight to, like, yeah, it's probably a toxic masculinity thing. I don't know. Um,

0:20:10 Thomas: Yeah. Oh, that's interesting.

0:20:32 Thomas: Didn't think about the studies part because I imagine you would be, you trying to statistically control your results based on a participation bias or whatever. But yeah, maybe that's an imperfect process. I don't know.

0:20:43 Jenessa: Some authors do phrase it as, like, you know, a methodological error that they they they treat it like this is a problem, this could have been done better. Sometimes it's thought of as more of, like, just a human thing. A little bit of both.

0:20:58 Thomas: You talking specifically when it comes to the difference in synesthesia or are you talking generally there.

0:21:03 Jenessa: Yeah, both, I would say. But I'm seeing it mainly in the context of synesthesia when authors are saying, well, why did we think there was this huge gender difference? I do see some people phrase it as as methodological error specifically, that this is a problem. You can think of it a lot of different ways. Yeah. There's certain populations where it seems more common and it's not super clear why. And then there's certainly populations where we thought it was more common, like in women. And that seems to not be true as certainly not to to the extent we once thought. I lean towards it just being a reporting problem. We've got it relatively uncommon, probably, depending on reporting. Big The thing for this strong synesthesia, first of all, it's literal, not metaphorical.

0:21:13 Thomas: Okay. Gotcha.

0:21:55 Jenessa: I'm not saying I developed a sort of association that like, you know, I memorized colors for the letters for some sort of mnemonic. No, literally, like A is red. Identified and remembered. So this isn't like a mnemonics trick. Like this is just, it's involuntary. It is just always evoked. They They can't, like, choose not to experience it. It's unidirectional. So if they have color pitch, the sound causes a color. The color doesn't make you hear a sound. It direction. It's idiosyncratic. So for this person, middle C might be blue. And for this other person, it might be green. There's no real reason that, you should be.

0:22:03 Thomas: Yeah. Yeah, yeah, yeah.

0:22:44 Thomas: Now that's interesting. I was going to ask you about that. Is there any rhyme or reason? Because you would think if it's, again, it's something consistent, it'd almost think there'd be some rhyme or reason to these associations. Is that because

0:22:59 Jenessa: So there is, like, a trend. So, yes, there are wild individual differences. There does seem to be, like – the letter A is red for more people. Like, there – it's not, like completely – maybe?

0:23:12 Thomas: Red is the lowest in the spectrum of Color? Yeah.

0:23:17 Jenessa: Like, that's a reasonable association. Association.

0:23:19 Thomas: It's the lowest wavelength, or is the highest? No, it's the lowest. Yeah. Because you Redshift.

0:23:22 Jenessa: Lowest for visible.

0:23:23 Thomas: Yeah, yeah, yeah. That's Interesting. I was actually going to ask along those lines, like, you would think there could be some rhyme or reason to that, like if numbers, you well I guess it's hard because numbers are kind of infinite, but yeah, or if it's like the digits one through nine, you know, go in, like, ascending spectral order, or if there's just no rhyme or reason to it. Interesting.

0:23:24 Jenessa: Yep, absolutely. Right. So what you don't get is that. I haven't seen a case where the individual had, like, oh, one through nine. Yeah, it goes red, orange, yellow, green, blue, purple. That's almost like a hallmark of, no, you mean it metaphorically. Um, do have a case study of a woman who, uh, realizing this is what it was, um, and she fit the whole definition, um, she'll fit the tests I'm going to describe for how we, um, for this, but they did find in her attic a childhood toy of, like, the little puzzle of put this letter over and it matched. It was the right colors.

0:24:16 Thomas: And You're saying this was genuine genuine synesthesia though. It wasn't like she had associated. Okay. Interesting.

0:24:16 Jenessa: Yes. Yes. This So she passed the test, like we We Believe Her

0:24:25 Thomas: Oh yeah. I'm curious what this test is. Yeah. It's just stop lying. And that's the test. Don't be such a liar. Okay. You got me.

0:24:26 Jenessa: Yeah. Yeah. Well, so that's the thing, is people are going to treat it initially like, um, must be speaking in metaphor and and they're just artsy-fartsy, or they're lying, they're they're just trying to get attention, whatever. This doesn't fit my model of how cognition works or my model about sensory works. And rather than adapt that to fit, I'm going to just claim you don't exist. Sure, I mean, if it was all made up, then I guess that's fine. The way they end up doing it to how are we going to test this? Have you heard of the pop-out effect?

0:24:40 Thomas: Yeah, I get it. Yeah yeah.

0:25:14 Thomas: Uh, is it like the magic eye thing?

0:25:18 Jenessa: No, not exactly. I could send you a picture. I know this is not a visual medium, but it might be very informative.

0:25:18 Thomas: No. It is for synesthesia, people.

0:25:27 Jenessa: Yeah. Yeah, right. So let me, I'm going to send you real quick and hopefully we can describe this reasonably enough.

0:25:34 Thomas: Sure. It going to be related to the thing where if you give a person who knows a language a word that's like blue, but it says red, it's hard for them to say the color?

0:25:47 Jenessa: That is one of the tests. It's not this test, but, it is one the ones we're going to use.

0:25:49 Thomas: Yeah. I was trying to think of how you could adapt that for synasthesia, sort of.

0:25:54 Jenessa: Yeah, okay. So here's the, the, the first one and

0:25:57 Thomas: Oh, yeah. Okay. OK, so it's going to be like it's a bunch of similar stuff. And then it's typical people, I suppose I'll use the word. It's probably hard for them to see. Like you've sent me something that's all sort of the digital clock version of fives. But there's also some twos. And I can look around and be like, I think. OK, I see one, two, three, four, five. Like I can see them, but it takes a minute. And now instantly right when you send me this, I'm like, oh, if you had synesthesia, it'd be so easy. Like it'd just be like literally someone's taking a highlighter and saying, look at these parts of the thing. They're different.

0:26:06 Jenessa: Sure. Yeah. Yeah. Yeah. So we have these, they're, they're little black icons on a white background, and what you're describing is what we'd call a serial search. You had to go through one character at a time, and you find your first two, and you're like, "Oh, okay," and then you find the rest of them, and eventually, um, you might have spotted they, they make a triangle. Yeah.

0:26:54 Thomas: All they do. Yes. I didn't spite that. No, no, no.

0:26:57 Jenessa: It takes, it takes a while, right? And

0:27:00 Thomas: I admittedly, I didn'T try that hard, but I they make it wait. They make a triangle because I'm seeing a two, a two to two to oh, a triangle like pointing sort of to the lower right.

0:27:02 Jenessa: Sure. Yeah. If Uh, No, they're pointing towards the top. See, it's very very hard to see, right?

0:27:13 Thomas: Oh, well, sorry. That was the stupidest thing to say. I'm so sorry. It was, if it's an equilateral triangle, then it's not really pointing anywhere. I thought it was like, no, I, for a second, I thought I only saw some of them. So I thought it was, like, an arrow, kind of, you know, like if it was skinnier on one side, you could say it's pointing somewhere, kindof. But then I saw that, I saw, that last two, that to the left more, I was like oh, that was a really dumb thing to say.

0:27:18 Jenessa: Oh, Jesus. It doesn't point any. Yeah, yeah. No. Oh, geez. I'm like, no,

0:27:36 Thomas: It's for sure an equilateral triangle, yeah.

0:27:37 Jenessa: It's pointing the other way. Yeah. Yeah OK. So it is an equilateral triangle of twos. So what's characteristic of this serial search, the more fives I put up there, the longer it would have taken you to go through and find it. What is characteristic for our pop out effect is, I've sent it to you here with the fives are green and the twos are red.

0:28:05 Thomas: Yeah, that's a lot easier. I could pass that test. Now, Arlo couldn't. You got to be careful with the colorblind folks now. He actually does have synesthesia, but he's colorblind, so the test takers givers missed it. You know, that is in crumb, sorry, that's incredibly interesting statistically. It's, I'm sure it's not super likely, but there must be some percentage of people who have colorblindness and synesthesion, and then, like, oh, okay, I can't. Well, we'll put a pin in that. Anyway, point

0:28:08 Jenessa: Yeah. Oh, yeah. Yeah. Yeah, so here we have a parallel search. All Oh, I have a case for you. Yeah!

0:28:39 Thomas: Is this very festive Christmas-colored fives and twos, incredibly easy to see. And so you're saying for people with synesthesia, they look at it like this, they're like, "Duh, that's a triangle, does not pointing anywhere, Thomas and Jenessa, you idiots, it's not so try, it's an equilateral triangle, you can't say pointing anywhere, it's pointing all three directions."

0:28:44 Jenessa: Yeah! Yes! So what's critical here with this parallel search and this pop -out effect, no matter how many green fives I throw up on that background, in 50 milliseconds approximately, you are going to spot those twos and respond and tell me what you see. And it doesn't matter how Many Fives. It just pops out of the background. There it is.

0:29:19 Thomas: Right.

0:29:19 Jenessa: You can't look at that black and white example and voluntarily do that for me. There's no amount going to practice. That's a serial search. This is a parallel search, and for our synesthetes, we will get a pop-out effect, a parallel surge, 50 milliseconds, no problem, and they, that's not a thing they could have, like, practiced.

0:29:29 Thomas: Interesting, yeah.

0:29:39 Thomas: You can't fake it. No.

0:29:40 Jenessa: No.

0:29:40 Thomas: That's, that'd be and You know what, if they, they did fake it, fine. You have synesthesia. I'll just, whatever.

0:29:41 Jenessa: You can't fake it. Yeah.

0:29:46 Thomas: You did such a good job. You worked so hard at, let's just say they have it.

0:29:47 Jenessa: Yeah. Sure. Sure. So, so this is really was one of the most definitive tests of like, no, they mean literally, I have any other answer for you. This has to be.

0:29:55 Thomas: Yeah, I bet. Yeah. And, And, but interesting is this also tells us, and maybe this was already known or something, but it also tells us that it's like, it's involuntary and happens. Now I, I don't, I'm not going to get probably the right terms for this, but I know there's, there's like processes in our brain that we're going to to be conscious of and unconscious of, right? And so it seems like this, because, because I could imagine a possible synesthesia world, like, I could imagine it being true potentially, even though we know it's not, that synestesia required me to first serial search out those twos, and then my brain would assign them that color, and sort of in the same way that you said a minute ago that someone had to be explained which letters of the whatever were equivalent to the letter A.

0:30:43 Thomas: And then it changed. Like, I could imagine that that would have been the case. But to see that it's not almost mean so. So it happens kind of, if not unconsciously, maybe that's not quite the right word. But it does happen at a level kind of before we're like thinking about it deliberately. Deliberately. Is that right? Right.

0:30:42 Jenessa: Yeah, yeah, you're absolutely right that this not only told us this is real, this is involuntary, this has not a metaphor. It also tells us something thinking about where this is happening in the stream of processing, this has to be extremely early. This can't be happening up at some higher cognitive frontal lobe decision-making attentional process where I have to be wanting to look for and think carefully, like, no, oh, this is really basic visual processing.

0:31:11 Thomas: Right. But at the same time, it's names and not let you know, like there's other things about it. They're like, well, that's kind of weird. Does seem like it takes some amount of processing first.

0:31:40 Jenessa: Mm -hmm. Sure. Yeah. So, it's definitely not, like, happening in the retina, but it's certainly not happening all the way up at, like intent conscious decision making. So it's somewhere early -ish.

0:31:51 Thomas: Right. Yeah. This is so fascinating.

0:32:04 Jenessa: It's trying Trying to pinpoint this is really tough. They took in one of the studies that looked at things like this, they took some synesthetes and they did, for example, okay, you're telling me the number five is blue. I'm going to put it in your central vision. You're going to tell me that. Great. I'm gonna slowly move this into your periphery. And it would hit a point where I can still see in my periphery that is the number five. It has lost its color.

0:32:28 Thomas: Wait, because, yeah, your peripheral vision doesn't have color, but but isn't what? What? So it also lost color? That's crazy.

0:32:31 Jenessa: Yeah. Yeah. It lost color, but it was still perceived. Perceived, so yeah.

0:32:40 Thomas: Right. So that's weird. So, That would tell us that it is still... But the thing with the names is throwing me off so much. Like, how could it be both those things? Because that sounds like it's way more in the early times of your visual cortex text processing, whatever that stuff, fancy science. Words, yeah.

0:32:44 Jenessa: Yeah. Yeah. Sure. Yeah. So for example, there was a woman who had, um, uh, months had color, you know, January, February. March. The word march can also be a verb, and when it is in the context of a verb, it did not have color for her.

0:33:08 Thomas: Right. What? Okay, I don't believe her. Let's get her back on this test and figure out, we'll put, here's what we'll do, we'll make a thing that's a bunch of marches and some of them are the month and they'll be on a white background and some other month and some are the verb and we'll see if she can just pick them out. Sorry.

0:33:30 Jenessa: So So clearly for her it is yeah There.

0:33:34 Thomas: Sorry. That's really funny to me.

0:33:34 Jenessa: You you go. Clearly, context is relevant, right? For these higher order, more complex associations, context is going to have to be relevant. And some names are nouns. So some these are really simple. A pitch has a color, a shape, the letter A has a colour. But if the letter O has a a color. Not every circle has a color, so it's still a little higher up than just frog brain.

0:33:47 Thomas: Yeah. Yeah, that I so that's why this peripheral vision thing is like that. That actually seems like the outlier to me if that person's lying. Sorry. Not real.

0:34:10 Jenessa: Well, So they had a couple of people in this case study. Part of what's difficult is it does seem to be either rare enough or the reports are rare enough that you are often doing case studies or small samples. This is also where we get our colorblind person.

0:34:23 Thomas: Yeah? Ooh, here we go.

0:34:29 Jenessa: This is the only example I have ever seen of a case study of a colorblind synesthete. And let me tell you, there was more where I was working on a paper on this. And if are other case studies on this, I'm very surprised I missed it. Um, this individual, um, are color graphing, so letters have colors. They also have, uh, faces are producing colors for them. Um, so you different types of faces, photographs and cartoon faces, produce some colors. They have a type of color blindness called S cone weakness. So the S cone is the the one that perceives like, yeah, yeah. So you can have a type of colorblindness where of your three cones, one is missing. And that would more significantly impact your color perception.

0:35:05 Thomas: S cones are weak.

0:35:17 Thomas: Right?

0:35:17 Jenessa: In a weakness, it's shifted so that its sensitivity is closer to the the other cone. So if the one that perceives green and the one that perceieves blue, part of how you differentiate them is the wavelengths they're sensitive to are further apart. So, if one is firing a lot and one is fireing a little, it's more green or more blue. If their sensitivity, which wavelength is like their favorite to fire to, if are closer together, you're going to have a harder time distinguishing them. Them. So in this S-cone weakness, where the sensitivity of the blue cone is shifted towards the green, this is going to produce a yellow-blue type of colorblindness, where they have a harder time telling apart yellows, greens, and blues. They look much more similar. And the deepest blues and the purples are going to be not visible to them almost at all, because

0:35:18 Thomas: Right?

0:36:15 Thomas: What do you mean not the difference won't be visible?

0:36:18 Jenessa: Uh, that cone, because its sensitivity is shifted over, um, people with typical color vision, that cone will be sensitive out to a certain degree of purple, but like, not ultraviolet light, right? It it's shifted towards the green and towards the red.

0:36:31 Thomas: Oh, it's all shifted down. You're saying? Sorry, I missed that. Yeah, yeah. Yeah, so anyone's vision has a limit, sort of like how our hearing has a limit is what you're saying. So like 20 ,000 Hertz or roughly, I think is, or maybe it's 10 ,000. I don't know. It's something, one of those is like, we can't really hear.

0:36:48 Thomas: Yeah. We can't hear anything higher than that usually. And so it's same for vision. So you're saying they literally can't see the color at all.

0:36:39 Jenessa: Yes. Yes. It's pretty high. Yeah, uh -huh. Yeah. So past a certain point, into the deep blues and purples, they're not going to get it really at all, and for yellow, blue, and green, they're going to have a hard time telling it apart. Um,

0:37:05 Thomas: Does it look black? What does it look like?

0:37:08 Jenessa: If you go, anybody can do this, if you want to pop into Google, Google, like, colorblindness example: yellow, blue, red, green. They'll show you the picture of everybody that everybody does. It's like a little spread of, like, vegetables. It'll be like a bunch of different tomatoes. And they'll show, like, normal color vision, proto-anomalous, deuteranomalous. And these are like different versions of the red cone weakness and the green cone weakness. Green colorblindness is more common. And that's where telling, you know, my partner has this, where basically reds are like a warm brown and greens are like cool brown. They're not super different.

0:37:47 Thomas: Yeah.

0:37:48 Jenessa: They are not super pigmented. Whereas yellows and blues really pop for him.

0:37:52 Thomas: Interesting. Interesting.

0:37:53 Jenessa: For yellow blue colorblindess, it's going to be kind of the other way. Reds and greens, I got this. Those two cones are in the typical position. And so looking at the difference between the firing of the red and the green cone will help me tell reds, oranges, yellows, greens. But when I'm trying to distinguish the other colors by telling the difference in firing rate between the green and the blue cone, I'm not going to do so good. Someday if you ever want to talk about those colorblindness glasses, there's like a whole method to get into to this.

0:38:21 Thomas: Yeah. I'm curious about that for Arlo, maybe someday, but I can't remember if they work or not. Yeah, it's also interesting because I can't really get Arlo's too young, I don't understand what it looks like to Arlo. Yeah, I mean, I know I can do that thing, but, but I used to always think like, oh, do they both look brown, but, but he kind of, when I ask him that, that's not what he says, you know, like he doesn't, that doesn't make sense to him, and I haven't quite figured out if we're struggling just on a, you normal five-year-old with words thing, or if he sees something other than that in a way that I don't get.

0:38:27 Jenessa: A whole other dive. Yeah.

0:38:44 Jenessa: Yeah, yeah. It can be really tough to communicate across any sort of perception barrier, right? It, and can be tough to communicate with kids. Um,

0:39:01 Thomas: Yeah, sure. Yeah. And Arlo in particular, but it's also so sad because what he'll tell, he's told us a few times, oh no, I can see color now. I learned it.

0:39:05 Jenessa: Oh, no.

0:39:12 Thomas: We'll be like, oh buddy, no, no.

0:39:12 Jenessa: Oh, my heart.

0:39:14 Thomas: I know, he's done that a lot. I'll be like, so does this? And like, "Oh no, I can see it, because people in class tell me." And I'm like, well, yeah, I mean, it's good they're telling you, but then I don't think that, yeah, whatever.

0:39:14 Jenessa: Uh, by a definition, I suppose. Um,

0:39:28 Thomas: Sure. I

0:39:29 Jenessa: So So we've got our person here who have this S-cone weakness, um, and blues not so different, uh, blues and purples not perceived at all, um, they also just so happen to have a grapheme synesthesia where, uh, letters and numbers have color and

0:39:46 Thomas: Can't believe I predicted this.

0:39:47 Jenessa: Yeah. So what he describes, uh, I'll just quote, I'll... the paper, intriguingly, this subject is color anomalous S-cone weakness, but he sees, quote, weird, number-evoked colors that are, quote, extra-spectral for him, i.e., he cannot see them in the real world

0:39:47 Thomas: Oh, my god, that is the coolest thing I've ever heard in my life, because I was just thinking if you, oh my God, that's incredible, that's like I'm genuinely getting chills thinking about that. That's, that's the old thing of, like, oh, imagine a color you can't conceive of, and, and I are like, yeah, impossible, but I can kind of, like, I can think of the concept of that, like, there's no reason that the exact number of colors that I—that the human brain tends to label, there's no reason for it being exactly what it is. It could be way more fine-grained, and we could have all a color is, is a mental label and wow, that is, that, that might be like the only person in the world who's ever had, you know, the number of people who can literally in their brains, at least see a color that they can't otherwise see is gotta be a handful. Like that's crazy.

0:40:10 Jenessa: Mm -hmm, yeah, yeah Mm -hm, mm -hm Mm -hum, yeah Yeah, just the statistical coincidence of these two things co-occurring in someone one. And his retina is incapable of this.

0:41:07 Thomas: And Yeah. Oh my God. So just to make sure I have this right. So the issue with colorblindness is the rods and cones. It's well with whatever, I don't know if there's other ones, but like, Yeah, we'll see.

0:41:14 Jenessa: Yes. The cones. The cones.

0:41:20 Thomas: It was the Cones, it's the cones, the cones have done, sure, whatever, uh, parks and rec uh, and it's not the mental label for it, so it's not as though if they somehow, if we were to bypass someone's science words for wherever in the visual thingy that happens and to, like, we could just matrix mode, we could plug in a cable to where that happens, then we're like, "All right, instead of the, the essential, the reading you're getting from your cones," like, which is sort of what's happening in a way, right? You're getting like some sort of value. If it were a computer, you're getting that, like, you know, the numbers that they do where it's like the, do they call that? Where you can do it by like, okay, how much of this color is it? This color is it? And this color, is it, and that's the color you are

0:41:20 Jenessa: Yeah. Sure. Yeah, yeah. No. Yeah yeah yeah. Oh yeah,

0:42:03 Thomas: On a computer.

0:42:03 Jenessa: It's very much works that way in a lot of ways. Yeah it

0:42:04 Thomas: Right. Yeah. And So hypothetically, if we could plug someone into that, who is colorblind and just feed them that data, their brain would have the label for it and it would be a color, and this person is able to see those things in his brain, with his brain, that he can't otherwise see because he doesn't get that data coming from his cones. Is that it?

0:42:26 Jenessa: That's what what it appears to be going on.

0:42:28 Thomas: Is the coolest thing I've ever heard, like, I I, I really just am like, my mind is blown by that. Like, imagine that. That's so cool.

0:42:37 Jenessa: Yeah. When he looks at the world and photons hit his retina, he is not capable of perceiving, like, purple. But when he looked at a letter, there is no reason that his brain that is producing colors, seemingly randomly, can't produce purple, unless we were to imagine a sort of like, oh, well, the only way you see color is if you, like, get exposure to it and your brain learns it.

0:42:51 Thomas: Right. Yeah.

0:43:06 Jenessa: It appears that's not how color works. Like, we're learning how the brain works by looking at him. Like what they write here is this observation, again, supports the cross wiring hypothesis rather than the memory association hypothesis. You cannot remember a color you have never seen.

0:43:06 Thomas: Right? Oh man. That is amazing.

0:43:26 Thomas: Never seen. Yeah.

0:43:29 Jenessa: Further, the study Study of this subject may throw light on the old philosophical conundrum raised by Molyneux, whether someone can experience completely novel qualia never before experienced. So it does touch that of, like, it seems like your brain, you know, whether as an adult, maybe prune connections you don't use. To what extent, I would quibble with some of my colleagues. Um, you know, there's like the old guard and the new guard on this. Um, uh, we are talking about someone who has had synesthesia. Uh, this is thought of as lifelong. Seems like the brain comes a little bit pre-programmed with some of what to do about

0:43:33 Thomas: Yeah, the, the room.

0:44:10 Thomas: Would

0:44:10 Jenessa: Colors. It would have to.

0:44:10 Thomas: Have to. It would, it would have, do you think? Otherwise, because, because now, like, maybe I am taking it it a step too far, but just as you were reading that, I was like, well, how could it work any other way?

0:44:13 Jenessa: Yeah.

0:44:22 Jenessa: Would think, right?

0:44:22 Thomas: Yeah. Cause like, it's not as though seeing, ah, I've seen purple. Now I can remember purple. No, you've seen that wavelength that take people out of the world. That wavelength doesn't contain purple. Like it just is a wavelength. So like how, yeah, that's interesting. And now I'm thinking like, I imagine the same thing could apply with other senses. So like, and even in a very plain, boring way. So let's say you have some sort of damage that causes you to lose. And this happens all the time, a band of hearing like, so like, oh, I can't hear in the, I don't know, 2k to 5k Hertz range anymore. That's gone.

0:44:29 Jenessa: Right. Sure. Of course not. Yeah?

0:44:58 Jenessa: Mm -hmm. Mm -huh.

0:45:05 Thomas: But you would still, still, I'm sure, be able to, like, imagine your favorite song, possibly if you're someone who's good at being able to hear kind of music in your head or, or you might even maybe have auditory hallucinations, which can happen, I think, to pretty much everyone at some time. And it's not like that would be missing, that just because you detecting instrument in your ear is no longer able to detect it doesn't mean the brain label for that has gone away. Or let's say you were born without the ability to detect it ever, like just maybe your ears, you know, different in a certain way. That wouldn't mean that had you no capable, capability in your brain of labeling that hurts, you know, value with, vibrational, you know, kind of thing with, with a sound in your head.

0:45:10 Jenessa: Sure? Mm hmm. Uh huh. Mm hmm? Mm hm. Mm hm? Sure, like you didn't come out a completely blank slate, that would be very to imagine. So you're exactly right, as when people lose hearing, you know, older adults especially, it's very common to start losing your hearing, and you usually lose the high frequencies first. Majority, almost everyone who loses their hearing will get some amount of tinnitus, some ringing in the ears. Many people, and it is hard to say how many because people are afraid to report it, it progresses to musical hallucination.

0:45:53 Thomas: Right. Yeah.

0:46:22 Thomas: Progresses to that.

0:46:23 Jenessa: Yeah, they'll start with tinnitous and then eventually they're like, so I'm hearing church music.

0:46:30 Thomas: Like once they've lost a bunch of their ability to hear.

0:46:33 Jenessa: So I've been, you know, only for like the last decade and I'll never finish it. I've been working on a theory paper that pulls together a bunch of these case studies, and it appears to progress for both losing your vision and then visually hallucinating, for losing your hearing and then auditory hallucinating for losing a limb and then getting phantom limb. It appears to follow a, a really stereotyped progression of instant loss, instant hallucination, slow loss.

0:47:04 Jenessa: It takes two to three years and you begin hallucinating.

0:47:09 Jenessa: I've been working on this for a while and then I was like, I have to, I have to quit this job. I'm dying. Um, Someday somebody please plagiarize me. I will happily give it to you.

0:46:58 Thomas: Hmm. This is so fascinating, but wonder if this answers or could potentially say something about the question I had instantly to Eli because when he was talking about, I don't, nobody hears that. Nobody blah, blah, like not to rehash that conversation. But one thing I asked him was, do you think that Helen Keller had no mental experience? You know, like having never seen or heard anything ever, like hypothetically.

0:47:18 Jenessa: Sure. Right.

0:47:39 Thomas: Yeah. Like, so now I'm thinking like someone who has never heard or seen anything could be having all kinds of visual or auditory experience that like, yeah. Like, it's not like as though it would only be. Yeah.

0:47:35 Jenessa: Which Doesn't sound right and not have a label Foreign

0:47:52 Thomas: No label and maybe even no way to communicate about it because it's so like, again If you tried to say what would a color look like that you've never seen, it's like, I don't know, there's no way to say that

0:47:59 Jenessa: Right. He doesn't say He doesn't say to these researchers "purple," he says "Weird." Never seen it before. Don't know what it is. Yeah.

0:48:05 Thomas: Right. Weird yeah. Yeah. Indescribable. Yeah, that, like, it would be that's so crazy that that is the coolest thing. Wow. So I think we went way longer than we intended because that was blowing my mind. And hopefully at least some of the other minds listening but there's way more on this. So we're going to have to regroup for a part two sometime, Jenessa but thank you so much for starting us off. I will be thinking about this for a while.

0:48:14 Jenessa: Yeah. Yeah.

0:48:24 Jenessa: Of what I figured is I didn't want to structure it yeah Cool.

0:48:37 Thomas: And in part two, what can we maybe tease for people that they're going to learn further about this?

0:48:38 Jenessa: Yeah. Would love to start part two with some people who who weren't synesthetic until they lost other senses.

0:48:53 Thomas: That's awesome. Plus, uh, a little bit more about how this happens or yeah.

0:48:58 Jenessa: Yeah. Yeah. Some theory behind like how.

0:49:02 Thomas: Blowing my mind. I, this is so cool. Well, thank you so much. Uh, do you want to throw anything out there? I forget. Do you have anything to plug or it seems

0:49:08 Jenessa: Yeah. Oh, hey, New Yorkers, flip your ballot over. You have ballot proposals. And I think it's like 31% of New Yorkers forget to flip their ballot over and vote on proposals. Yes, statewide Prop 1 amendment to the Constitution to extend equal protection to like like, age discrimination, disability discrimination, LGBTQ discrimination, would be real cool to not forget to vote on that.

0:49:34 Thomas: Like a good idea. Yeah. Flip your ballot over. I That would be something someone would need to tell me. So no shame, anyone.

0:49:36 Jenessa: Flip your ballot. Yeah, it's on the other side of your paper ballot.

0:49:41 Thomas: Yeah. All right. Well, I love that use of the plug in order to do something pro -social. You're amazing. And I cannot wait for part two. So excited.

0:49:52 Jenessa: Sounds great.