A talk with Dr. Marcel Just, a researcher at Carnegie Mellon University, about his work using function magnetic resonance imaging (fMRI) to look for brain activity related to specific thoughts. For example: differentiating the brain activity of someone thinking about an apple versus thinking about other words and concepts. Dr. Just and his work have been twice featured on the TV show 60 Minutes.
A transcript is below.
Links to this episode:
Topics discussed include:
- The nature of human thought; how much is it individual thoughts vs many things firing at once
- Why brain activity for specific thoughts and emotions trigger specific areas of the brain in consistent, predictable ways
- Why thoughts and emotions activate the same types of patterns for so many people, even across language divides
- How fMRI works and why it is a good choice for this type of work (versus EEG, for example)
- The state of this technology, its potential applications, and what its practical limitations are
Related links:
- Dr. Marcel Just’s TEDx Talk The New Science of Thought Imaging
- Dr. Just’s work identifying suicidal ideations from brain scans
- Use of this work to identify patterns associated with autism
- About similar brain imaging work done at Berkeley as people listened to The Moth Radio Hour
- Some skepticism about the repeatability of finding patterns in brains
TRANSCRIPT
[Note: transcripts will always contain some errors.]
Zach: This is the People Who Read People Podcast. I’m Zach Elwood. Today is June 3rd, 2020. Today I’ll be interviewing Dr. Marcel, just a researcher at Carnegie Mellon University In November, 2019, Dr. Just and his team were featured on 60 Minutes. I’m gonna play a couple clips from that episode because it serves as a good introduction to what this work is about and how exciting it’s.
Reporter: When we first visited Dr. Just Lab 10 years ago, he and his team had conducted a study. They put people in the scanner and asked them to think about 10 objects, five of them tools like screwdriver and hammer, and five of them dwellings. Like Igloo and castle, while measuring activity levels throughout their brains.
The idea was to crunch the data and try to identify distinctive patterns of activity for each object. You had them think about a screwdriver. Mm-hmm. In the [00:01:00] computer. Found the place in the brain where that person was thinking screwdriver.
Researcher: Screwdriver isn’t one place in the brain. It’s many places in the brain.
When you think of a screwdriver, you think about how you hold it, how you twist it, what it looks like, and each
Reporter: of those. Functions are in different places. Correct. He showed us that by dividing the brain into thousands of tiny cubes and analyzing the amount of activity in each one, his team was able to identify unique patterns for each object you’re reading.
Their mind
Researcher: we’re identifying the thought that’s occurring. Whoa. It’s incredible. Just incredible.
Reporter: Incredible, but only the beginning in the decade since Professor Just Lab has taken this technique and applied it far beyond hammers and igloos to increasingly complex thoughts.
Zach: You can watch the 60 Minutes episode if you search online for 60 minutes.
Marcel, just personally, I had no idea this kind of work was [00:02:00] going on, and I was pretty astounded to learn about it. If you like the podcast, please leave a rating or review on the podcast service. You listen on. If you’d like to send me ideas for the show, you can reach [email protected] or on Twitter where my handle is a poker player now onto the interview.
Okay. Thanks for coming on, Dr. Just
Marcel: pleasure to be with you.
Zach: Yes. Your work is very interesting and, and pretty mind blowing really. When I was reading about it, I had no idea that that kind of work was being done. Speaking of the 60 Minutes episode, considering that you’ve been doing that kind of work for 10 years or more, how is that advanced?
Like if I was gonna gonna go into your. Uh, your laboratory now and go into the FM MRI machine. Would you be able to quickly tell like the specific concepts and words that I was thinking of, or is, is it that exact at this point just doing a cold read of somebody like me?
Marcel: If you’re thinking of concepts that we have the template for, then yeah, it’s pretty accurate.
And I say pretty accurate. If I [00:03:00] let it take 10 guesses, it’ll get it right on the first two. Three, maybe 3, 2, 3 guesses out of 10. So it, it’s, um, yeah, it’s in the, it’s in the 80th percentile, roughly speaking in its accuracy. And it varies from person to person. You know, it varies with uninteresting things like how still you can keep your head in the scanner, which isn’t conceptually interesting, but it’s limited by that.
Zach: You’ve been doing that work for a while, how have you seen that that accuracy for, you know, say for the, the simplest, uh, words or concepts, change over time?
Marcel: You know, the accuracy, we, we, it sort of stays in that, in that range, you know, somewhere between 70 and 90. 90th percentile of the, of the guesses it takes, I think some of the biggest advances have to do with the, the scale of the knowledge that we can decode.
It used to be one concept at a time. You [00:04:00] think of some concept and we can identify the PO pattern. Now we’ve been able to get, uh, a configuration of concepts, so not just, you know, hammer, but I want that hammer to build a. Bird house or something like that, so we can get a set of actions, a scene, you know, the dog chasing the cat as opposed to the cat chasing the dog.
So thoughts and even up to. A short passage, say about 50, 75 words describing some, some technical area, you know how, I don’t know, radar works or something, and we can identify that well enough so that we can get the library call number for the kind of book it would’ve come from. You read a passage on radar and our system can tell you where to find the relevant book in the library.
Zach: Well, yeah, it just seems [00:05:00] phenomenal. I mean, so, and, and I’d seen another study, it wasn’t yours, but someone else had done a study. Similar to that, where they were having people read from the Moth, uh, audio show, uh, read and listen to the Moth. Uh, and, and, and they were analyzing that same thing where it was an entire, like sentence, an entire scene.
Had you seen much about that other study?
Marcel: Well, that, that you’re describing Layla web study. She was, she’s now a professor at Carnegie Mellon. Uh, and this work started with. Myself, I’m a cognitive neuroscientist, meaning studying brain function and thought, and her advisor was my collaborator on this. His name is Tom Mitchell.
He is an extremely prominent computer scientist who’s one of the leading figures in the field of machine learning, and Tom and I decided, actually it’s 20 years ago, we decided to, to see if we could use machine learning to use the power of computation to [00:06:00] recognize the patterns. A brain activation associated with thoughts.
Uh, Layla was, was a graduate student, uh, with, with Tom. And as I say, she’s now a prominent faculty member. And so she did this, uh, um, I think la um, study of continuous narrative where she tries to, her model tries to say something about what word should be, might be coming next based on what you’re thinking now.
It’s a, it’s a sister, sister approach. Again, using these machine learning techniques to look at the brain representations of concepts.
Zach: When you talk about, you know, listening or reading to a, a narrative, uh, and, and having those longer chain ideas, like a sentence for example, is that something you’re, you’re reading the pattern as like a more of a static shot, or is it definitely something that changes a lot over, you know, like a few seconds or does that make sense?
Marcel: Yes, I understand what you’re saying in the brain. [00:07:00] Maybe you could ask me about the time, pace of thinking. So, you know, you people read 200 words a minute or something like that. Mm-hmm. So that means every, I don’t know, let’s say every quarter of a second you read the next word, roughly speaking. So the pace of thought at, at that level of granularity is, let me just say three, four little thoughts a second.
So to read a, a 10 word sentence, you know, a couple of seconds and so on. FMRI has certain properties and that is, it’s slow, it’s sluggish, and it gloms together the brain activation patterns that occur over like five seconds. Some people consider that a detrimental, uh, a sort of a shortcoming of FMRI because you can’t watch things in real time like the way you can with EEG or other electrophysiological techniques.
But with FMRI, it gloms everything together Now. Yes, it’s a shortcoming if you wanna know [00:08:00] whether X occurred before Y or after Y. But if you wanna know the whole set of X and Ys that occurred in a sentence or passage, then having them all present at the same time is a boon. So we let people, for example, read a 10 word sentence and all the concepts from those, from that sentence are cove in the brain at the same time at the end of the sentence.
And we can decode all of them. You know when, when we’re thinking, we seldom just think of one concept at a time. Sometimes, you know, you focus on, I don’t know, a banana or something, really thinking banana, but most of the time it’s multiple concepts in combination with each other. How you’re gonna use the banana in your cereal, in the milk or something like that.
Zach: Do you think there’s, you know, there’s kind of shorthand thoughts that we have for things that we commonly think about, like, for example. Like if you read the sentence, it was a dark and stormy night. We’re not actually like going through multiple ideas there we’re just. It seems to me that there’s [00:09:00] probably just some preset activation in the brain that we all have for the idea of a dark and stormy night.
We don’t actually have to think through those separate ideas.
Marcel: Yes. Certainly when you come across a familiar configuration of thoughts, you know, like what I said, banana and cereal, you didn’t have to sort of like figure out how those go together because you mm-hmm. Presumably come across that before. So, yeah.
So more familiar. Thought patterns could, are probably processed faster, more easily, and so on. Mm-hmm. But I wanna, I wanna say the combination of, of concepts is a sort of a human thing. I think very much enabled by, um, the human capacity for language. You know, so a, a dog can probably think of a banana. I’m sure you could think of a banana and, uh, and maybe banana and milk maybe, but not, I’m gonna have banana and milk yesterday.
Uh, I’m gonna have banana milk tomorrow. And where does that [00:10:00] power to combine ideas come from? I think the lot, in large part it comes from human language. It’s sort of a, a, a fabulous mental tool that we have. Mm-hmm. And you see the language areas engaged when you’re. Uh, processing certainly when you’re reading or listening.
And so human language allows us to combine thoughts into meaningful structures, combinations. You know, roughly speaking, you could call this grammar or syntax and you get, you know, the dog chased the cat, the cat chased the dog, you know, in my backyard yesterday while I was sleeping or something or other.
So we, these combinatorics. One of the things that allow us to do complex thoughts, language allows us to build complex structural thoughts. It’s like having, you know, a Lego set.
Zach: Mm-hmm.
Marcel: A dog can play with [00:11:00] individual Lego pieces, maybe two. But a dog can’t build a Lego castle and we can
Zach: mm-hmm. Mm-hmm.
Marcel: A conceptual castle by, by building, and, you know, so a novel is a sort of a, a Lego castle that some author has written.
Mm-hmm. And we can’t read, and we can put those ideas together. So I think building, you know, the, the construction, the interlocking of ideas is a key. Human capability. Oh. And you see people who have damage to the brain area that does some of that building, that structure building have difficulty building structure and comprehending complex structure.
Zach: Mm-hmm. So, yeah. When you say, uh, Legos, that really struck me because it seems like one of our strengths is, is being able to take these. Complex things complex and then make that into a block that we don’t have to think that much about. It’s just a complex block, but, but it’s, it’s complexity is hidden from us and then we can use it.
Marcel: That’s [00:12:00] right. But we, and yes, and, and it empowers us, enables us to, to build these complex structures. So, you know, novelists do this, obviously scientists do this when they build theories everyday. People when they’re planning, you know, their, you know, their trip to the supermarket, whoever do some of it. Do this.
It’s, it’s, it’s this very powerful capability of the human brain enabled in large part by the language system. Um, when scientists first learned about what a part of the, what part of a brain does something, it was broker’s area and broke his area is exactly the part that’s centrally involved. It’s not the only part in building mental structures.
Brokers area, and for language. Some scientists think that it evolved from primates, motor areas, primates who could build up a complex set of motor actions. You know how to decide to grab onto a [00:13:00] vine and swing on it and. Land on the next tree, and so on and so forth. They had to be able to plan a complex sequence of motor actions and broker’s area is kind of geographically close to that part of the brain that plans physical actions.
Zach: How much of these, looking at these patterns and the images, how much of it is human readable versus something you need the machine to interpret?
Marcel: Most of the analysis is done automatically, and you know, we’re not holding up x-rays to the lights. But at, at the end of the day, we wanna see where that activation is and, and look at one pattern for, you know, banana versus peach.
We wanna wanna look at one person’s banana representation to another person’s banana representation. And we do both. You know, if you have, you know, 10 or 50. Participants in your study, it [00:14:00] just, there’s too much data involved to be doing it manually, but you sort of check on individual instances to make sure that the theory, the explanation you’re building corresponds to the reality of individual, um, activation patterns.
So it’s mostly done by machine,
Zach: right? Because there’s a lot of machine learning, uh, computation that that goes into this d identifying the patterns in the first place, right?
Marcel: Yes. We’re typically looking at the activation levels of, say, 150 little Foxs, foxholes or three dimensional pixels, volume elements.
So we look at the activation levels of a hundred, say 150 box holes. It’s kind of hard to wrap your head around 150 numbers.
Zach: How small are those voxels?
Marcel: The ones that we use are the size of a peppercorn. Mm-hmm. They’re three millimeters by three millimeters by six millimeters.
Zach: And, um, a [00:15:00] little bit about the FMRI, uh, technology that works basically by, um, inducing a magnetic charge in the blood, and then by, by that you can read, uh, where the heavier blood flow is, is, is that correct?
Marcel: Yeah. When your neurons start firing, say you’re, you’re deciding to talk and use brokers area to generate a sentence. Uh, then the neurons in that region start firing, and when they fire the vascular system within half a second, sends oxygenated blood. And the oxygenated blood contains hemoglobin. So that means it has a slightly different magnetic property.
And FMRI, the, the magnetic property of a scanner allows you to detect where the magnetic prop, where the magnetic field is slightly different. So if you’re using bros area, there’s mo more oxygenated blood there, um, with the hemoglobin, hemoglobin being [00:16:00] iron related. And so you can, you can measure that. Uh, typically in, in, uh, in our studies, we measure it once per second in about 20,000 voxels that cover the brain, oh, it’s called bold imaging, blood oxygen level dependent.
All things being equal, you’d rather directly measure the firing of the neurons. And you, you can do that with, you know, somebody happens to be in the middle of a neurosurgical procedure, you can actually have electrodes on their brain.
Zach: Mm-hmm.
Marcel: But that’s, it’s, it’s much more
Zach: harder.
Marcel: Yeah.
Zach: Uh, so quick question.
You had said something about viewing 20,000 voxels and it, and then you had said you were looking at the activation of 150 voxels. So I imagine that’s, you, you’re, you’re, you’re looking at, uh, the pattern for a much smaller amount than you actually can measure with an FM fmr.
Marcel: Oh, no, no. We can measure all one, all [00:17:00] 20,000.
Zach: Mm-hmm.
Marcel: But when you’re thinking about these concepts. Only a small handful are really players. Uh, how do we know that they’re players? We ask if they have, if they’re kind of consistent across that they’re, they always do the similar thing when you ask the person to think about a, about a banana or a hammer or a cup or a car.
So I want voxels that, you know, really like cars and hate bananas and other voxels that. Like bananas and hate cars and so on, but I want them to, to not be. Noise. I want them to be systematic. We refer though to those as stable voxels voxels that have a stable set of preferences over our, our, over the concepts that we’re examining.
We, we present the concepts. Usually around six times and we want that voxel to have the same set of preferences each, [00:18:00] each of the six times. And we pick, say the 150 voxels that have the most stable set of preferences from those voxels activation levels. We can then tell which of the concepts is being considered.
Zach: So does that mean, like, say I was gonna go in for a cold read, in that case, would it be looking at a much larger number of voxels to do that cold read? Uh, does, does that make sense? Or, or are you constraining it in some way based on what you’re looking for?
Marcel: Yeah, that’s a good question. Yes. So I think it’s somewhat constrained, but you know, it, you don’t really need, um, there’s certain areas that are, that tend to contain the players.
So it’s not just any old place. It’s, it’s certain places and if you look across our various studies, you know, whether we we’re studying, uh, tools or, or emotions, oh, emotions are, [00:19:00] uh, is a slightly different set of places. So that, that sort of speaks to your question. Mm-hmm. Is it the same set of voxel? I, it’s rather different voxels.
I wanna know whether you’re feeling envy or anger. Uh, then I wanna know if you’re thinking about a screwdriver or a banana,
Zach: right? So you’re saying depending on what the category of, of, you know. Emotions or, or tools or whatever that category was. You’d, you’d be looking in a different, constraining it to a different, uh, set of voxels.
That’s what you’re saying?
Marcel: Yeah. Different set of brain regions and, uh, now I think, you know, emotions are really, really different. But physical objects, let’s see. Um. Places and buildings and, uh, some actions, uh, some sort of your notion of, of where you are in space. Are you in an enclosed space or an open space?
Uh oh. Are you dealing with other people?
Zach: Social.
Marcel: Yeah, yeah, exactly. Yeah. So these are kinds of the, some [00:20:00] of the main underlying dimensions that, that underpin human thought. Uh, interaction with others, your own emotion, obviously perceptual things, what you’re seeing and hearing.
Zach: Mm-hmm.
Marcel: Uh, whether, oh, here’s an important one.
How does this relate to me? You know, am I on the spot here? Am I looking good? Am I looking bad? Does this make me feel good? And so on. So the self is one of the,
Zach: oh, yeah, that’s, that seems, that seems really interesting. Yeah. Like what is, when you’re self-conscious, what are the areas that are triggered?
Marcel: Yes.
And I, I, I think that plays a much larger role in human thought than, I don’t know, than nons psychiatrists realized. I think the self plays a big role in, in human thought and in, well, unsurprisingly, the way we are, our personalities.
Zach: Yeah, I mean, from an existential kind of perspective, it’s, you [00:21:00] know, when, because you boil it down and the world can be kind of boiled down to self and others and a, a very existential level, you know?
So it seems very fundamental to, uh, so many other things.
Marcel: Yes. And so there are regions of the brain that activate when you’re reflecting on yourself. You know, uh, I don’t know whether you were planning to ask me about our suicide work or our autism work, but the self plays. Uh, substantial role in the differentiation of people with various psychiatric, uh, disorders.
Zach: Yeah. Let’s, let’s come back to that. I wanted to ask you, uh, about the geographic areas of the brain. Uh, do, do the activation patterns you see, uh, make sense in terms of what, what you or what is already known about what specific brain areas do. For example, like if you’re. Doing a word, uh, recognition for something kind of social related and it [00:22:00] activates something that’s known to be, uh, a more social area of the brain.
Uh, have there been a lot of surprises in terms of where the activations show up or is it kind of adhering to what is known about. What the specific sections of the brain do.
Marcel: Well, it is adhering, but you know, knowing what the various brain parts of the brain do is a very recent set of knowledge that only started happening like 90 years ago when neurosurgeons specifically Wilder Penfield in Montreal.
While he had people’s brains open, he would stimulate various parts of the brain and he’d stimulate some, some place, and somebody’s left hand would jerk up and he’d say, oh, okay, so that’s where the left hand is controlled. And so he’d stimulate another place and they’d say, oh, I hear a whooshing sound.
See? Okay, that’s the auditory cortex. So the mapping out of which brain area does what is, you know, like. That hasn’t been known for hundred hundreds of years. I mentioned Broca’s area, I [00:23:00] believe that was first identified in the late 19th century when Paul Broka was informed about the autopsy of this patient who had trouble formulating sentences, and it was Broca’s area that had the big lesion in it.
And so that was the beginning of a mapping of the brain. Are there surprises? Well, nobody had that many expectations to start with, so it’s hard to be surprised. It’s, you know, it’s, it’s very interesting to think there are these, you know, these parcelization of the brain, and it’s extremely interesting to think about how they come about and what they really mean.
When an infant is born, the, the neurons are clearly there in Broca’s area, but it’s not really Broca’s area. The infant has to learn about language and word combinations and gradually. Broker’s area gets to do its thing. Mm. So it’s born, I think the infant is born with the capabilities to build for structure building, [00:24:00] but it’s only withdrew exposure to language that it actually gets to practice that.
And. Gets and, and captures the, I don’t know, wins the contract to use the theology of a prominent neuroscientist to see scientists, Liz Bates wins the contract for formulating sentence structure.
Zach: Hmm.
Marcel: And so we’re born with, you know, an 86 billion neurons say, but they’re not undifferentiated. They’re different, you know, they, they look different.
And they almost, and they almost certainly have different functions or, or I shouldn’t say, functions, different computational capabilities. So for example, the neurons in the, in the hearing, in the auditory cortex almost certainly have to be able to process. Very fine differences in timing. You know, the difference between one consonant and [00:25:00] another.
It could be like 10, 15 milliseconds of, of something, some, uh, some acoustic signal. I. And so on. Oh, but this, this all goes to say is we, we’re, we’re born with these 86 billion neurons, and they each gra gradually get contracts for this and that whatever we’re exposed to, whatever we care about, whatever we try to think about.
And, you know, we’ve learned a lot about what the various brain areas do. I, I’ve described some, some are obvious, some are less obvious. But what it is, is that the neurons have different computational. Capabilities. You know, there’s slightly different types of processors in one part of brain than the other, and some are suited to certain kinds of computations more than other kinds of computations.
And it gradually the right kind of, I dunno what to call it, task or processing ends up in the best neurons for that type of processing.
Zach: This brings [00:26:00] me to one of the most amazing things about this work that we haven’t even talked about yet, or I, I mentioned in the introduction, but, uh, so you’ve shown that these patterns are common, uh, in many cases across many people, which, uh.
The other, you know, astounding thing to me because I think a lot of people, myself included up until I saw that, you know, I would’ve thought that, yeah, sure. There’s some general function that is the same in specific areas, but your, your work is showing that it is very specific or very common across people, and can you talk a little bit about why that is?
Marcel: Yes. I think that’s one of the most fascinating outcomes of our work. The fact that. We’re all working with very similar brain infrastructure. And it’s hardware more than hardware, more than hardware. Um, you know, it’s, it’s also, uh, software connectivity networks and so on.
Zach: Mm-hmm. [00:27:00]
Marcel: So, yes. So we’re born and, you know, we, we, we recognize this commonality with respect to our bodies.
We all know that we walk uphill a certain way and we see somebody else walking uphill. We understand that they have to live. If their legs higher and so on, that it’s energy consuming. We don’t see other people thinking We know sometimes, you know, oh, that’s the thought that would’ve come to my mind in that situation, but the commonality is much greater than than anybody thought.
So, you know, in our programs are, are trained on the patterns of one set of people and we put a new person in the scanner and it could recognize their, their patterns. So, you know, when you think of a banana and I think of a banana, similar things happen in our brains. For example, we both think about how you hold a banana, you know, the, the fact that it’s a, a cylindrical thing and you fold your fingers around it a certain way, and it’ll be different for [00:28:00] Apple where you hold the two ends of the apple differently.
So the commonality of our, of our, of our brains that, you know, there’s a part of our brain that controls our hand, for example. And so everybody codes, tools, and fruits by how you call them and how you wield them. And the fact that there’s a, there’s a commonality of experience. You know, like maybe in in cultures where there are no bananas, maybe, maybe it’s slightly different.
And my work and my, my focus has been on the commonality. There’s also individuality, of course, you know, in some countries the bananas are much smaller than in other countries. You know, probably different types of bananas. So I’m not saying that everybody’s thoughts are identical. Mm-hmm. People obviously have unique experiences.
What we subjectively experience is the uniqueness of our experience and our, and our thoughts, and we assume that we’re sort of [00:29:00] unique and I think of banana and. God knows how you think of banana, but no, it’s rather similar. Also, the, we see the similarity when we, when we ask people to think of concepts when they’re presented in different languages or sentences presented in different languages.
In one study we had people reading some sentences and our program could tell which of which of many sentences they were reading. We then translated those sentences into Portuguese. Ask monolingual Portuguese speakers to read those sentences. We gave those resulting activation patterns to our program, and they could tell which sentence the person was reading.
Oh, the sentences were, were good, excellent translations. In other words, these aren’t. Brain representations of words, they’re representations of concepts.
Zach: Mm-hmm.
Marcel: So there, it’s like a universal language of the brain rather than any particular natural language.
Zach: Yeah. I mean this, this is [00:30:00] just so astounding because I think it.
I think so many people’s thoughts around, you know, variability between people or go against that idea. I mean, that, mine definitely did. I, I, I tended to have that concept of, oh, we’re all so different. I, you know, everyone’s so vastly different. It’s so hard to understand under other people. I mean, one thing that struck, uh, struck me reading this was this just really would seem to give so much support to, you know, evolutionary biology.
Um, you know, uh. Concepts around, uh, psychology. Have you written much about that or seen much talked about with this work?
Marcel: Well, I haven’t related it to evolutionary psychology, but more to en environmental. Mm-hmm. You know, it’s the fact that everybody sees a horizon. I. Nobody lives in a tilted, uh, in a tilted world except people in the space capsule perhaps.
So it’s
Zach: Oh, so it’s more experiential than it is, uh, biological you’re, you’re, you’re saying? [00:31:00]
Marcel: Well, uh,
Zach: or both, I guess. I think the
Marcel: biology. Interacts with the experience,
Zach: right. You, I guess, I guess you can’t separate ’em ’cause you have the underlying hardware and then you, all of our experiences are the same, mostly.
So it makes sense that they’re in certain regions. Yeah, I guess. Yeah, I see what you’re saying.
Marcel: And so we, we, we start off with 86 billion neurons and you know, it’s not. People’s neurons aren’t identical. And, you know, we know about, uh, uh, pathologies, for example, where brains aren’t formed, right? But if one has a normal brain, you’re gonna have a part of the brain that, you know, controls your motor action controls, how you, you know, grip things, how you punch, how you poke, and so on, how you walk.
And there’s a terrific commonality in that, you know, if you lob a tennis ball to somebody gently. They’re gonna probably stretch out their dominant hand with their fingers open and try to catch the ball, either underhand or [00:32:00] overhand or something. Everybody’s gonna do that. Similarly, when you lob an idea at someone, their mind is going, you know, the same part, you know?
Similar parts of their brain are going to reach out to grasp that idea.
Zach: Mm-hmm.
Marcel: And, and so obviously we have vastly different beliefs and values and so on, but the machinery with which we process those ideas, those values are, is, is remarkably similar.
Zach: Mm-hmm. Yeah. Made me think too of, uh, I’m not a, you know, young believer or anything like that.
I kind of find ’em silly, but it made me think of the young youngian ar archetypes of, you know, these are common, important, uh, concepts and, and your work kind of made me think of that as where there’s maybe these common, uh, just from our experiences and the way our brains are structured, there’s these common, uh.
You know, things that light up [00:33:00] for very, in very similar ways for very fundamental concepts.
Marcel: Uh, you know, I I, I’m unfamiliar with Young’s Works. I’m not gonna be able to comment on that, but I, I think, um, as I said this, this commonality enters into, you know, things about how we, how we learn stuff, how we, how we gain knowledge through everyday experience, how we gain it through formal schooling.
And, and we, we end up being similar kinds of, of, of thinkers.
Zach: Mm-hmm.
Marcel: Uh, similar kinds of feelers of emotion. Maybe. It’s not so surprising that everybody’s activation pattern for bananas is saying, but everybody’s pattern for anger and envy is very similar, maybe even more similar than for Banana and, you know, and yet it feels so unique.
It’s because you know what is there to anger? There’s the valence that’s negative. [00:34:00] There’s the intensity of it, there’s the sort of the aggression part of it. Uh, there’s the control of it. You know, that, that those are, those are some of the dimensions of emotion that, that we all share. You know, now some people, you know, exercise more control, some exercise, less control, and so on.
Mm-hmm. But the underlying dimensions are, are rather similar.
Zach: This made me think, uh, if you put somebody, uh, speaking of the emotion, more emotional, uh, psychol, psychological, uh, angle, if you put somebody in there, uh, who like, like a Norman Bates and, and told them to think of the word mother, would you see it light up just tremendously differently than other people’s, uh, concept of mother?
Marcel: Well. I, I understand that you’re, uh, using an example. Mm-hmm. But if you have somebody who has a psychiatric illness that changes the way they think about certain concepts, you can detect those [00:35:00] changes. You can detect those alterations associated with certain psychiatric illnesses. Which,
Zach: which, which leads into your, yeah.
The, your work on, um, suicide. And I, I don’t mean to be flippant about the Norman Bates example, but that was just an extreme. Pop culture idea of the idea of like, can we see our, our various, um, basically our issues in the brain and, and your work on suicide seems to show that too. Yeah.
Marcel: Yeah. So in the case of suicide, you know, the, the one concept that’s clearly altered is the concept of death in, in retrospect, of course, that’s the one that would be altered.
So yes. So you, you can detect, I don’t know. I’m not sure that you can detect just idiosyncrasies. But certainly detect out, certainly in certain cases you could detect pathological alterations.
Zach: Right. And getting back to that existential, you know, of, of yourself and others. I mean, death is one of [00:36:00] those, those existential things.
So it, it makes sense that that would be one of the more findable things to, to find. Is that one of the reasons you decided to focus on that? Because it was such a. A fundamental and, and important, uh, concept.
Marcel: Well, we get, you know, uh, there have been previous studies that show had it shown that. Emotional reaction to the concept of death was different among people who were thinking about suicide.
So we included death and you know, other words like funeral and so on. In our study, we also included mutual words that you wouldn’t think have anything to do with death. We also included positive words. Sort of a contrast. So certainly the death related words had been altered, but also some of the positive words had been altered.
The, the death related words were altered by virtue of greater self-identification with [00:37:00] it, uh, more activation of the self in relationship to death for people who are thinking about suicide. But for the positive concept, you got less self.
Zach: So that
Marcel: if we ask people to think about carefree, then the controls think about themselves more than people.
Who think about suicide.
Zach: Right. That makes sense. It’s a, it’s something that happens outside of you. It’s it’s another world. Yeah.
Marcel: Yeah. They’re not in it.
Zach: So that maybe gets to a question I had about applications. Ha. Have there already been, um, applications of this outside the, the laboratory that, that have been applied in, in some places, or, or is that, uh, more a work in progress?
I can see there’s a lot of things that need to be, you know. Before practical applications too, but can you talk a little bit about how it’s been applied or, or do you think it will be applied?
Marcel: So I, I think it will be applied in psychiatry and education, and the big challenge is [00:38:00] scalability. You can’t put, you know, a million people in a scanner.
I mean, it’s just too cumbersome, too expensive. And so the challenge is to port these kinds of measurements to some much more accessible technology. Like, you know, many, many clinics have an EEG system. An e an nice EEG system can cost, I don’t want say, $30,000 instead of two and a half million or 3 million.
And so if one could ever do this kind of work with EEG, that would just be a game changer. So the big challenge is scalability. I think FMRI can, can reveal the nature of the, I dunno, the, the neural and psychological processes that are at issue both in education and in psychiatry. But you know, at this point in time, you can’t put, you know, classrooms of people into an MRI scanner.
Except in, in, in, in small scale studies. [00:39:00] So I think that’s, that’s the challenge. Now, you know, FMRI and fm I. Fm I, you know, the first FMRI paper was published in 92, 19 92, so 28 years ago. And somebody, you know, clever scientists, figured out how to measure the blood oxygen level and that proves that the electromagnetic energy is absolutely there inside our, inside our skulls.
Now it’s hard to get inside our skulls, but all it would take is some brilliant biophysicist to find some way to measure the. That electromagnetic energy, you know, more, more cheaply, more widely than with, with FMRI, you know, maybe in 20 years from now we’ll just point our, our smartphones at somebody and we’ll get a readout of the next person’s brain or something.
The energy is definitely there. The signal clearly exists, and right now we need a big, cumbersome, expensive machine to get at it. [00:40:00] There’ll be, there’ll be a more, uh, a more accessible path to it.
Zach: So it sounds like you’re, you’re saying outside of the, there haven’t been, uh. Applications like diagnosing, um, uh, psychological issues or, or anything like that in a, in a practical sense outside of the laboratory?
Is that, is that correct?
Marcel: It’s hard to scale, but I, many people, you know, there’s lots of research that tries to say, how is the, how is the activation pattern and I don’t know, schizophrenia or bipolar, different, and how do we, and, and how does that help us understand the nature of the disorder? There’s a lot of research.
I don’t think we’re at the point where somebody goes to their clinic and their psychiatrist puts them in the scanner and out, out comes the diagnosis That’s not there now.
Zach: I did see something about, uh, I can’t remember her name. Maybe it was something like Jessup, where she had [00:41:00] started a company that was aimed at basically creating like a, you know, telepathy kind of technology where some, it was like a wearable FM RI device that would do similar to what.
Your work is, is doing and be able to pass it to someone else. And she seemed to be like some sort of reputable, uh, you know, she had worked at Google or something. Have you heard anything about that kind of work,
Marcel: what people are working on? Uh, I don’t know that specific project
Zach: mm-hmm.
Marcel: Referred to as brain computer interfaces.
The big challenge is, as I say, is getting that signal out. You know, in some patients who temporarily have a piece of skull removed, you can record from their brain and the, and stimulate, and then you could, then you, you can set up the communication can be quite good. You know, uh, there’s a, there’s a video.
Of a woman who is, I think, paralyzed, but she has these electrodes in her brain and the electrodes are hooked up [00:42:00] to a computer that controls a, a robot, a robotic arm. And when she wants to reach out for a candy bar and she thinks I wanna reach out for that candy bar, and she thinks reaching, and that robotic arm goes out and reaches and grasps that candy bar brings it into her.
So there are these attempts. To be able to, for the brain signals, to be externalized one way or the other. Now, you can’t go around removing pieces of people’s skulls, but, and, and their attempts with EEG, we we’re doing a little bit of that with, uh, EEG is, you know, has wonderful temporal resolution, but really the signal isn’t, isn’t quite as good.
You can’t, it’s very hard to tell where it’s coming from. You know, it’s nothing like which Voxel is activating. Uh, I mean, you get some information about where it’s coming from, but it’s coarser. Oh, can you tell what somebody’s thinking about from the EEG signal? So we’ve done a little bit of that, and you can do it.
It’s not as good [00:43:00] as FMRI. And maybe it’s the best portable technology and, and scalable technology now, I would hope in the next, in the next few decades, somebody will come up with something even better.
Zach: Mm-hmm.
Marcel: Elon Musk talks still talks about things in the brain. Very invasive. You wanna do it non-invasively, you know, no surgery.
Zach: One question I had was, and I think a lot of people on hearing, you know, this kind of astounding, uh, work that’s being done, is, do you see issues with the, um, the accuracy so far? Have you have it, has, has it been replicated by other, um, labs? Is it, is it something that’s easily replicated if you gave those, you know, instructions to somebody else?
And how much of it is, is, yeah. Is, is that a good question?
Marcel: Not that it’s, uh, so, you know, there aren’t, uh, you know, thousands of people doing this. You, you mentioned, [00:44:00] uh, Leila Wease work. So how many people, how many laboratories are doing this? Not 200, maybe, maybe 20, maybe 40. You know, I don’t, I don’t know of all of ’em, but people are doing, they call this decoding, sometimes brain decoding.
So they’re different names for it, so people are doing it and they’re reporting the results. The accuracies are, you know, they’re not a hundred percent. As I said, I think we do very well and we, and I told you the rank accuracy of the, the normalized rank accuracy, the correct answer. If I, we let our program take as many guesses as there are alternatives.
The correct guess is often above the 80th percentile.
Zach: Mm-hmm.
Marcel: So that that can be.
Zach: Depending on your perspective, that is impressive. Yeah.
Marcel: Chance level with this measure is 50% 50th percentile. So by chance the correct answer would on average end [00:45:00] up in the middle of the list if, if the machine knew nothing.
So it’s at 80 oh and a hundred percent of course. Perfect. Mm-hmm. So it’s a little closer to perfect than it is to. Nothing. And as I said, it depends on, on, on things like if you move around, you know, in the box so the box aren’t in the same location that degrades the signal and so on and so forth. And
Zach: it, it depends on yeah, human specific, individual variability I’m sure adds a little bit more, um, chaos in the mix too, I guess.
Marcel: Yes. You know, when, when we ask somebody to think about a banana, we need them, them to think about a banana and focus on that. They can’t be thinking about, you know, how their left hip hurts them at
Zach: Oh, right.
Marcel: You know, and they can’t let their mind wander even for a fraction of a second. They have to think Banana, banana, banana.
You know, and, and, you know, and they have to sort of smell, see. You know, taste a banana for three seconds and, you know, it’s, it’s not, it’s, it’s sort of [00:46:00] challenging to remain focused for half an hour thinking about all the concepts we ask them to think about.
Zach: That makes me wonder, does that mean you could, uh, could you classify some people as basically better focusers, uh, who, who are, you can more easily get a, a clear, you know, non-no.
Marcel: It’s certainly the case that some people systematically produce more decodable. Activation patterns and others and what’s it due to? So, as I said, head motion focusing and so on.
Zach: I’d imagine there must be something related to sort of test taking anxiety too, where some people you tell ’em to focus, they’re, they’re gonna have a hard time with that just because they’re on the spot.
Marcel: Yeah. And you’re lying in an MRI scanner in this sort of foreign looking environment and so on. You know, we try to make them comfortable. I think we have a Tempur-Pedic pad under them. But you know, you’re still in the scanner. Your, your head is sort of, uh, constrained not to move. You know, these [00:47:00] aren’t conceptually interesting issues, but they certainly limit the accuracy, the ability to decode
Zach: makes me think too, you could look into the, you know, some of these things that are harder to.
Otherwise test or research like a Fantasia, you know, the, the idea that some people don’t have the mind’s eye that other people have that don’t, don’t have mental images. It seems like theoretically you could test that idea a bit too.
Marcel: That kind of thing is studied more directly. You, you measure the degree of, of mental imagery that people use while they’re thinking.
And, and there are differences among people in that, um, you know, some people say they’re more spatial thinkers and some there are thinking styles and you, and you could detect that.
Zach: Oh, with, with brain brand imaging, you’re saying that’s already been looked into a bit.
Marcel: Yeah, there’s a part of the brain. The intraparietal sulcus that activates when you form a visual image of something.
So if I [00:48:00] ask you to think of what a banana looks like, you know, I, I, I think that that part, you’ll get activation in that part. Or if I ask you to think of a banana inside a black mug, I think you’ll see activation in the area. And, you know, some people tend to use it more, use that kind of thinking more than others.
Zach: So, uh, do you see in the future a way to induce ideas in people’s minds? I know there would be obviously ethical issues with that, but I was imagining if, if we know how something activates, could you induce the thought of, uh, you know, a dark and stormy night In my mind, by precisely activating those.
Those areas
Marcel: right now, one can in induce the thought of gripping your fingers around something by stimulating the appropriate motor area. I bet you you were thinking of slight slightly a more abstract thought than that.
Zach: Mm-hmm.
Marcel: But also people who are are blind in experimental studies. [00:49:00] There are devices that stimulate their visual cortex so they can see some patterns and shapes.
So it, it’s beginning, it’s, it’s kind of in, it’s, it’s not easy to do non-invasively. Again, if you have access to their brains, then, then you have a much better shot at it. Yeah. So, you know, and, and we, you know, cochlear implants about, which I don’t know very much, are sort of like a, a step in that direction.
So more generally, we’re getting closer to be able to interact with our brains more directly in some sort of interface. You know, the skull is a big gut. You know, it protects our brains and we’re really grateful for it. But it gets in the way if you really wanna communi, if you want to stimulate, people use transcranial magnetic stimulation, they use electrical stimulation.
It’s very coarse in terms of its spatial precision. You can’t like stimulate a voxel. Mm-hmm. You’re stimulating the equivalent of, I don’t know, probably thousands or tens of [00:50:00] thousands of voxels when you do a transcranial magnetic stimulation. You know, one study that a colleague of mine did, what, what they did was they had one person think about lifting their finger.
And that was translated by a computer into another person being stimulated in their motor area, and their finger would flick and, and they refer to this as brain to brain communication. I. The, our thoughts are obviously in our brains, we use language on our mouths and typing to get the ideas from one head into another head.
Mm-hmm. That’s what language does for us and another kinds of symbol systems. But you know, maybe someday there’ll be a way to get the thoughts from one person’s head into another person’s head without putting it into words. So if I’m thinking the bananas in the cereal. If that’s an activation pattern, maybe someday we’ll be able to put that activation pattern into another [00:51:00] person’s brain without my having to say the words.
Zach: It seems almost impossible to imagine that being done without a hugely invasive like series of electrodes being in someone’s brain to get that exact. Would you agree, you know, at, at least for the, uh, a more exact kind of concept or activity do, would you agree with that?
Marcel: Oh yes. Certainly in the near future it seems impossible, but a hundred years from now, who knows.
Zach: Right? They could induce electoral activity at a distance of a few inches from the outside or something like that.
Marcel: Yeah. Who knows? You know, magnetic resonance energy I think was first kind of, I don’t know to say formulated that theory. I think it’s like 1920s, 1930s. The on which the MRI is based. So who knows, maybe there are other kinds of forms of energy that we don’t know about yet and so on.
Hmm.
Zach: Well, yeah, this is fascinating. I think we’ve got, um, plenty here to, to, to leave it here, unless you had anything else you [00:52:00] wanted to say about, you know, the work that you’re doing and the near future things you’re excited by. Anything like that.
Marcel: Well, there’s, there’s a topic that I’d like to mention, and that has to do with the fact that all of our thoughts have multiple components.
You know, sometimes we say, oh, such and such a thought is a frontal task, a frontal thinking or spatial thinking. But every kind of thought, every kind of thought involves multiple parts of the brain. And thinking is in some ways a network function. A team sport. So if I think of a banana, I think about what it looks like, how I hold it, what it tastes like, and each one of those is in a separate brain area.
And those brain areas, the act activation in the, in the various brain areas, the information needs to be coordinated. Also, obviously there has to be communication among those brain areas. And the, the communication occurs along axons, the [00:53:00] parts of the brain that are, uh, ca wiring that goes from one set of neurons to another.
So 40% of our brains. White matter, which is very heavily, um, insulated cabling, running between brain areas 40%. It’s a gigantic amount. So Mother Nature obviously thinks that it’s important to have communication between these brain areas. So when we’re thinking from the simplest thoughts, the most complex thoughts, we use multiple brain areas.
And I think it’s extremely interesting to think about the fact that thought is so, every thought is multifaceted. The multiple facets get linked to each other by brain connectivity, white matter. So it, it’s an, an incredibly important part of the infrastructure of thought.
Zach: Mm-hmm. The
Marcel: gray matter that we’ve been discussing, you know, in the brokers area or the motor region, the sort of, in some [00:54:00] ways more intellectually interesting, just as if you open your desktop computer.
I don’t know. The processors are much more interesting than the cables between
Zach: the network. Yeah, the networking.
Marcel: Yes. Yeah. It’s sort of more interesting, but we all know when the network goes down, you’re dead. You know, the network is what a net allows our computers to do what they do. And similarly, the white matter in our brains allow us to do the thinking we do.
I think the white matter sort of is, is a sort of a silent hero of our brains and it allows these, these various parts of the brain to communicate so that the. You know, the holding of a banana and the side of a banana and the taste of a banana yet can come together in, in some way. To create the construct of a banana.
And, and, and of course, you know, there are neurological diseases that degrade the white matter and you know, and then [00:55:00] people have various neurological problems. So the human brain is sort of this magnificent thing that met Mother Nature’s created that’s allow us to think as well as we do, and human culture.
It’s a slightly different topic. Human culture has found a way to bootstrap up the brain to do bigger and better things. So in some ways, you know, biologically brains aren’t all that different than they were, uh, 5,000 years ago. But the thinking that goes on now is, I think, enormously different from the thinking from, uh, 500 years ago by virtue of formal education, schooling.
And science, we learn more and more and we communicate it to vast numbers of people. So the brain keeps change. The human brain keeps changing over time. Not so much biologically, but what it can do, [00:56:00] uh, let me give you an example. Right now, we take for granted that many, maybe most huge numbers of people can read.
They’re literate. But literacy is a very, very recent thing. First of all, written language was only developed around 5,000 years ago. So some, some very bright people develop written language, but it’s only in the past, you know, one or 200 years that we’ve learned to communicate the ability to read to lots and lots of people.
So now you know, there are millions of billions of people reading. Nobody was reading 5,000 years ago. So people are reading, and there are other kinds of things. People think about science and technology. They know how the world works, they understand about our solar system. They know how internal combustion engines work.
And so the human brain, while the biology, you know, that that mother nature gave us, is not very different a little bit. You know, we have better [00:57:00] nutrition and so on and so forth. But the, our culture has built up new knowledge. Methods to widely disseminate that knowledge. And so the brains of the 21st century are, are, you know, way ahead of brains of, of previous centuries in millennia.
And going forward, I think they’re gonna get better and better. We’re gonna develop new and better understandings and ideas, and we’re gonna find better and better ways to teach them to people. We are, you know, you see how so many people are tech savvy now, you know, that was unimaginable, you know, 50, a hundred years ago.
Anyway, we, we should be extremely grateful for what Mother Nature has given us. This, this capability to think unlimited thoughts and to convey them to others and to learn to think better and better and bigger and bigger.
Zach: [00:58:00] We’re, we’re very adaptable, uh, adaptable creatures. It’s, it’s kind of amazing. I.
Marcel: Yes.
Zach: Well it is amazing. Alright, uh, thanks Dr. Justice has been an awesome talk and, uh, just wanna say wrap up. This is Dr. Marcel, just, he does his research at Carnegie Mellon. And, uh, thanks a lot for coming on.
Marcel: Uh, my pleasure to be with you.
Zach: If you enjoy the people who read People podcast, please leave me a rating or review on iTunes or wherever you listen to podcasts.
Any questions or feedback or ideas for the show? Send them to reading poker tells.com and use the contact form.