Cortical Brain Implants Are Paving the Way for Visual Restorative Medicine

On Tech & Vision with Dr. Cal Roberts

Today’s big idea highlights how innovations don’t happen in a vacuum, but rather a long chain of science and research and developments that build on each other. Dr. Shelley Fried’s work exemplifies this process. It took him a career’s worth of experiments and adjustments to enable his cortical brain implants to bypass the eye and restore the patient’s ability to perceive light. He had a lot of obstacles to overcome, everything from circumventing the brain’s natural inflammatory response to getting the research published. One thing is clear, breakthroughs take time and you cannot give up in the process. Your work often becomes an iteration of an iteration. Dr. Fried took inspiration from the artificial retina, which was prototyped from a cochlear implant. Dr. Fried’s revolutionary technology is another step towards a world in which no person is limited by their visual capacity.

Podcast Transcription

Alexander:  During the beginning of the pandemic I became a mental health first responder for front line workers.  And that was definitely a really unique opportunity to be the one helping them, knowing that, for instance, if they saw me in person and they saw me with a cane that I might be the last person they would turn to for support or help.

I am Rebecca Alexander.  I have Usher Syndrome Type 3, which is the leading genetic cause of deaf blindness in the US and around the world.

Roberts:  Rebecca is an author, an extreme athlete, a keynote speaker, and a psychotherapist.  As you can imagine, with the pandemic she’s been very busy lately.

Alexander:  My practice, sadly, has not suffered during COVID because everybody is in a very difficult state these days in terms of their mental health.

Roberts:  Has Usher Syndrome prepared Rebecca to help people through difficult times?

Alexander:  To have Usher Syndrome means that I have a condition that I have no control over.  I was told that I would be completely blind almost over ten years ago, and I still have about ten degrees of central most vision.  So, it does give me perspective because I’m constantly living in this state that most people have been fortunate enough to not find themselves in of having to be keenly and acutely aware of all that is so completely out of our control.

Roberts:  So, how does she do it?  How does Rebecca counsel patients online despite vision and hearing loss?  Well, to compensate for her lost hearing due to her Usher Syndrome, Rebecca has two cochlear implants, one in each ear.

Alexander:  So, I was cochlear implanted in my right side in 2013 and I was cochlear implanted on my left side at the end of 2017.

Roberts:  Cochlear implants are an amazing technology.  They are a neuro-prosthetic device that is surgically implanted in the cochlea – the inner part of the ear that is responsible for the transmission of nerve impulses into the auditory cortex of the brain.  These implants were developed in their original form in the 1950s.  And in their modern form, with a little help from an engineer at NASA in the 1970s.  In fact, much science had to happen to develop the cochlear device that now helps Rebecca counsel her patients.

Alexander:  They implant a magnetic internal piece of the device that is essentially embedded in the side of your skull just outside of your cochlear.  The external piece is what magnetically adheres to the internal piece.  You have electrodes that are sort of snaked through the snail shell of the cochlea, and that is what gives you access to the range of sounds.

Roberts:  But, as innovative as the cochlear implant is on its own to restore lost hearing, its invention also unlocked a series of important leaps forward for neuro-prosthetics and ophthalmology and vision technology.

I’m Dr.  Cal Roberts and this is On Tech and Vision.  Today’s big idea is that innovations don’t happen in a vacuum.  Instead, they’re part of a long chain of science and research and development by others.  Science is a two-way street, and incidentally, so is this podcast.  We’re excited to share with our listeners that we have a voice mailbox, and we’d love to hear your big ideas.  The number is 646-791-6115.  So give us a call and you might just be on the show.

Our guest today is Dr. Shelley Fried, Associate Professor of Neuroscience at Harvard Medical School and Assistant in Neuroscience at the Department of Neurosurgery at Massachusetts General Hospital.  He is the developers of cortical brain implants, work that builds on the work of other scientists including a guest we talked to in episode 4.  We spoke with Dr. Mark Humayun who developed the Argus II retinal implant.  He took design cues from the cochlear implant.

Humayun:  Cochlear implants had taught us that if you even put somewhat of a rudimentary signal into the ear that the brain can start to use it.  So, we were really hoping that the brain will help us.

Roberts:  Dr. Humayun even used a cochlear implant to build his first prototype of the Argus II retinal prosthesis.

Humayun:  We sort of reconfigured a cochlear implant and used it to stimulate the retina.

Roberts:  It worked because the technology is analogous.  Both implants gather information externally either in sound or image, then convert that data to stimulate the brain so the patient can perceive it.

Humayun:  You put the electrode array where you need to to result in those electrical pulses and in this case we put it in the retina.  In the cochlear implant they put it into the cochlea.

Roberts:  But just like neuro-prosthetics don’t end at the cochlear implant, vision science doesn’t end at the artificial retina.  Vision science, like all science, keeps building on itself.  The Argus II was an inspiration for the new technology we’re profiling today, Dr. Fried’s cortical brain implants.

Imagine the electrode array that Dr. Humayun mentioned.  In a cochlear implant, it goes in the cochlea inside the ear translating sound from the world into digital sound.  In the Argus II it goes in the retina, recreating in pulses an image captured by a camera.  In a cortical brain implant, the electrode array goes directly into the brain, into the visual cortex.

Now, bring your hands up behind you and interlock them behind your head.  Your index fingers are pointing at your occipital cortex.  And right at the back pole of the occipital cortex, right at the back of your head is your visual cortex, that part of your brain that helps you to see.  What if we put the electrode array right there?  Right into your brain, and bypass your eyes all together.

Early research by my guest today suggests this can work.  Before we get into the science behind Dr. Fried’s innovation, I asked him why he chose the vision pathway as his area of expertise.

Fried:  I’ll tell you two things that I think really contributed to why I was fascinated.  When I was six years old I remember they thought I was kind of not to bright when I went into school.  It turned out that I couldn’t see the blackboard.  I couldn’t read what was going on.  So, I couldn’t see.  I’ve always been wearing glasses.  It wasn’t that common at the time.  I was the only one in my class for a number of years.  I became interested in that way.

And then, I remember reading a New York Times article, I grew in New York City reading the Sunday magazine, about Dobelle’s work trying to restore vision to blind Vietnam veterans, and I was just fascinated by that and said that’s what I wanted to do.

Roberts:  What do we mean when we talk about a retinal prosthesis or we talk about any kind of medical prosthesis that’s involved in the vision process?

Fried:  In it’s simplest form it’s an array of electrodes.  It’s a grid of electrodes – the Argus II, the common one is 6×10 electrodes.  Each electrode is designed to stimulate a small, narrow portion of the retina.  And just like pixelated signs we see everywhere in the elevator that tell you what floor you’re on, you activate the correct pixels and you try to recreate a pattern.  So you can recreate the number seven or the letter E or the letter I, depending on which ones you activate.  That’s the theory.  Argus II has run with that and folks have done that.

But the practice is that the individual electrodes don’t always activate a nice, neat little area of neural activity on the retina.  That’s what we want.  We want the neural activity to match the pattern that we see – the nice LED pattern that we see in the elevator.  The work that I do studies why neurons respond to this artificial stimulation, to this electric stimulation.  And we work to try to recreate the neural signals that occur naturally.

So, when we see the number seven, we have a bunch of neurons that essentially light up, that recreate that spatial pattern of 7.  The work that I do really focuses on what happens when neurons see this artificial stimulus.  Can we start to recreate the natural pattern?

Roberts:  When I think about the visual system, it starts with the eyes and then the eyes process the image into an electrical signal that then the optic nerve carries to the brain and then the brain interprets that to what we consider to be vision.

Fried:  That’s right.

Roberts:  Where are the sights that this system can break down?

Fried:  That’s a good question.  The front of the eye is designed to focus that light right on the back of the eye, back on the retina, and convert it into the electrical signals you described.  The retina has many, many different layers and many, many different support mechanisms.  All of those can break down.  We call diseases like macular degeneration or retinitis pigmentosa that can destroy some of those really, really highly specific metabolic functions and destroy the function of the retina.

Glaucoma can destroy the nerves in a different way.  There can be tumors along the optic nerve that can destroy the passage of the information from the retina to the brain.  So, practically anywhere along the pathway, up to and including the brain, any damage along that way could severely impact our ability to see.

Roberts:  Explain what you’re doing with cortical implants.

Fried:  The work done by Second Sight with their Argus device and by other retinal prothesis efforts is really wonderful.  It’s ground-breaking.  It has been amazing.  The vision that they provide are still limited, but there’s a lot of effort to improve the quality of vision by those devices here in the States and all over the world.

But, even when those devices work well, they won’t be able to treat all of the blind folks that are out there.  So, someone with advanced glaucoma has very little retina left to target, so a retinal prosthesis, even if it works perfectly, would have no effect because there are no retinal neurons left to stimulate.

The thought is, we can build a prosthesis that goes into a downstream visual pathway, which is the visual cortex or the occipital cortex right at the back of our head.

Roberts:  This is important.  What Dr. Fried is saying is that if cortical brain implants work – and his studies are promising – they could restore sight to many people who are blind.  I ask Dr. Fried to walk our listeners through what happens in the process of sight once the visual information gets from the front of the eye through the retina, down the optic nerve and into the brain.

Fried:  The early part of the visual system dissects the scene.  It figures out when we’re staring at the visual world okay, that’s color, that’s form, that’s shape, that’s an edge, that’s dark, that’s a contrast.  It breaks it down into all these different dimensions.  It transmits those individual signals to the back of the brain and then the visual cortex starts to reassemble all those dissected pathways into what we perceive as vision.

That actually makes a stop right in the very center of our brain at a region of the brain called the thalamus, more specifically called the lateral geniculate nucleus.  But it goes right from there into the primary visual cortex and then it keeps going.  It goes to higher and higher levels of cortex and the signal keeps processing, keeps getting more complex.

Roberts:  How did Dr. Fried learn to code the array, to make the pulses that will create the image?

Fried:  The work that I did while I was in Frank Werblin’s lab and the work that he did over many decades is exactly that – trying to understand the fundamental by which we see.

Roberts:  Many listeners will remember Dr. Frank Werblin.  We interviewed him in Episode 2 on his device, the IrisVision.  He is one of Dr. Fried’s closest mentors.  Again, remember this episode’s big idea.  Cortical brain implants are arriving on a long history of research and developments that came before.

Fried:  We have a hundred million neurons in each eye.  A hundred plus million.  But only a million of them send a signal out to the brain.  Over the course of our lives we can lose hundreds of thousands of those output neurons and we don’t even notice that they’re missing.  It’s part of the aging process.  So we can see almost as well with 200,000 neurons as we can with one million.

The questions we start to ask are, what’s so important about those 200,000?  Which ones are the important ones?  What exactly is that neural code that’s making us see?  What are the important electrical activities?  Which are the precise ones that convey specific aspects of vision?  That’s a big focus of my work and what we’re trying to understand.

Roberts:  In order to solve this problem, Dr. Fried and others in the medical community are trying to figure out what each electrical variation that each neuron transmits to the brain means, and how bunches of neurons signal together and what those signals mean.  But, even if they come to learn all of that they still have to figure out how to target a single neuron, smaller than a human hair.  For Dr. Fried, coils were the answer, at least to the question of how to isolate a neuron.

Fried: The reason I became interested in coils is that I think coils do a better job of confining activation – of getting one electrode – one coil to activate on neuron.  But it has to do more with that second half problem.  Even if we know the neural code, how do we replicate that neural code with prosthesis?

Roberts:  When you say a coil, to me a coil is like a twisted wire, like a Slinky.

Fried:  For years folks have been trying to develop cortical prostheses.  There’s papers out from 20-25 years ago.  Schmidt at the NIH had this beautiful study where he implanted a couple of electrodes in the visual cortex of a blind subject and showed that he could recreate these light percepts, similar to what the Argus II does.  Since then there has been a lot of technology development.  All the ability to miniaturize technology.  to get computers to shrink everything down.  Power requirements, advanced materials.

We can put many, many electrodes, we can condense them in a very small space and get those into the brain.  But we’ve run into additional problems when we go into the brain that don’t exist in the retina.  One of them is that the brain has a huge, inflammatory response to the implant.  It detects any implant into the cortex as an invasion and a scab will form around these electrodes.  That scab, that inflammation – they call it gliosis – essentially blocks that electrical signal.  So, the electrodes might be good on day one, and they might even be good on day seven or two or three weeks later, but over time they start to lose efficacy.

It’s just a fundamental limitation of electrodes.  There’s a lot of work trying to improve on that, but it’s a fundamental problem of electrode-based stimulation.  These tiny little Slinkys that you described are really just a way of overcoming some of those inherent limitations.

Roberts:  Dr. Fried has encountered a problem.  Gliosis, or brain swelling, inhibiting the function of the electrodes.  And the same way he built on Dr. Humayun’s work, which builds on the cochlear implant, Dr. Fried looked to an already existing technology to solve the problem.

Fried:  You may have heard of magnetic resonance imaging – MRI, where they use magnetic fields.  Magnetic fields go right through any biological material.  We can see into people’s skulls, their dense bone.  Magnetic fields pass through any biological material, so if we put a coil in, even if we get that gliosis around the coil, the coil continues to function.  That magnetic field goes right through that blockage created by the body in response to the implant and it continues to function over time.  That’s what we like about coils.

Roberts:  And with coils you can target individual neurons.

Fried:  That’s exactly right.  Coils not only are more stable over time, but they’re more selective.  They’re able to create a smaller region of activation, so we think we can get much higher acuity with coils than we can with conventional electrodes.  In other words, we can pack the individual channels of the array closer together and still get discrete regions of activation with the coil than we can with an electrode.

Roberts:  Did you borrow this technology from someone else or did you come up with this idea on your own?

Fried:  I’d love to say I came up with it on my own, but of course, it’s evolved over time.  Folks have known that magnetic stimulation can activate neurons for hundreds of years.  But even in the modern era there’s a technology called transcranial magnetic stimulation that’s used to treat depression and other diseases where they put large coils on the outside of the head.

Our advance was that we showed that we could really shrink down coils to the tens of microns size.  To the sub-millimeter size and that they would still be effective.  That they can still induce neural activation.  We did some modeling work, some engineering calculations to lay out the theory, but the real advance was we took it into the lab of of bright post-doc of mine at the time, a fellow by the name of Sun Wu Li.  Took it into the lab with help from a lot of others and was able to show that we could activate single neurons and then continue to do this in additional dish work and then eventually in laboratory animals.

Roberts:  I can’t show people a picture, so describe to me what a coil cortical implant looks like.

Fried:  The coil implant, the Slinky that we remember as a kid’s toy, they call that a helical shape.  You keep shrinking that down until it gets smaller and smaller.  It turns out from the basic electromagnets, from the basic physics theory, that all you really need to do is bend it once.  You can take a very, very thin wire, micron size, sub-millimeter size and bend it in half into a U shape or a V shape.  And it’s tiny.  It’s like the size of a human hair bent into a V shape.

We can manufacture these.  We have the technology to that.  We found that when we take that and put it up next to the neurons they’re small enough to activate.  And because these wires are so small they can be safely implanted into the brain and we can put many of them in an array and create the prosthesis that way.

Roberts:  To receive the coils, a patient must undergo brain surgery.  I asked Dr. Fried how accurate he has to be for these super tiny coils to spark the neuron’s reaction to the electrode array.  Do they have to be in exactly the right place in the brain?

Fried:  The complete answer is that we don’t really know.  But, we don’t think so.  We think we can be off by many tens of millimeters.  We know from a lot of basic vision work over many, many decades how the visual cortex is organized.  We talked about that back part of your head, the very, very back portion of the visual cortex is what accounts for our central most part of vision.  I won’t get into the detail – but different regions, it’s all very, very precisely mapped.  So we know we want to be close to that back central region.  That’ll give us the highest acuity vision.

We don’t know exactly yet whether it matters critically whether we’re in the exact very back, but we don’t think so.  We think that patients can see with high enough acuity even if we don’t find that perfect sweet spot.

Roberts:  You talked about the fact that you started with electrical stimulation of microchips that created an electrical signal.  And now you’re using a magnetic signal.  Was there anything in between?  Did you try something else that didn’t work?  Tell us about that.

Fried:  We make it sound like we went lickety-split from electrodes to coils.  We don’t talk about the year and a half it took us to get those experiments to work.  We don’t talk about how we tried to publish our first paper and it got rejected by the reviewers because they were so skeptical of our results and we had to do more controlled experiments.  We don’t talk about all of the challenges we had to develop some of those cortical experiments.  So, it’s not always a perfect linear process.  That’s really the key here.

We were buying tiny coils from Panasonic.  We know them as the TV and VCR company.  We were able to buy computer board-level chips they make from Panasonic.  They cost 39 cents a piece.  They were tiny.  That’s what we tried first.  But those were still too large so we had to develop our own.  We did this over a period of three or four or five years from the first concept to where we were really convinced that this was working reliably.

Roberts:  Dr. Fried’s cortical brain implants had positive results in laboratory tests.  Now they’re being testing in clinical trials.  But their aren’t enough results yet to determine what the impact will on human vision.  He hopes to achieve results similar to Dr. Mark Humayan’s, with the Argus II, the retinal prosthesis.

Fried:  There’s a lot of technological challenges to overcome, but the hope is that we’re going to get to 2100 or 2200 and right now that the best levels are around 2400 or 2500 vision.  We want to get better with those.  There are patients that can read 3-5 letter words and they can read them, a couple of words every second. Two, three, four – they can see it.  There’s hope that those folds can read, can see spatial vision.  They can get through and improve their activities of daily living.

Roberts:  You talk about 2400 or 2500.  With that kind of vision someone can navigate their home without assistance.  That is really a major improvement in vision.

Fried:  I think so.  I think there’s always the hope as the engineer, we always want to try to get more pixels in and higher acuity.  But for folks who have limited or no light perception whatsoever, to be able to have light perception and be able to navigate and get through some activities of daily living, I think it’s a very big advance.  Very, very exciting.

Roberts:  In the episode where we featured the Argus II, we also talked to Barbara Campbell, who was an early recipient of the artificial retina.  She had been completely blind before the retinal prosthesis.  But after the surgery…

Campbell:  I would see little flashes.  And there would be different kinds of tests that they would give me.  The more I used it, the more I improved.

Roberts:  And after significant training with the device…

Campbell:  I was able to very easily identify the pole for the bus stop.  That was really good.  That was really very good.

Roberts:  What fascinates me is that with cortical implants, Dr. Fried has bypassed the eye.  In theory, someone who has lost their eyes from trauma or otherwise might still be able to have some vision back.  Important work for someone who began his career inspired by veterans returning home from Vietnam.

Fried:  Part of our funding comes from the Department of Defense and the Department of Veterans Affairs who I also work for.  And they have a huge commitment to these veterans who are coming back, some of them from explosions, from traumatic eye injury.  They come back with little or no eye left so they’re not candidates for any of these retinal devices.  They’ve decided that they wanted to develop a way to treat these folks, so they’re funding this research and it’s really wonderful because it will be available for the widest range.  Anybody without an eye.  If the eye had to be removed for cancer or something like that.  Even bilaterally.  We’re developing a device to offer to these folks.

Roberts:  For Rebecca Alexander, the cochlear implant allows her to hear, and to listen so that she can maintain her psychotherapy practice despite hearing loss from User Syndrome.  But, just like Barbara Campbell needed training to interpret the sensations from the retinal implant, Rebecca underwent hundreds of hours of listening therapy to interpret the digital sounds she was experiencing.  And she had to lose her natural hearing.

Alexander:  Deciding to lose that last bit of my residual hearing was very much a mourning process.  Before I got implanted I did grieve.  I did have those times where I had to break down and cry and allow myself to really be able to let go of this time and place of being able to hear naturally.  And how many of the sounds that I missed hearing from when I was younger.  It was certainly a process.  A grieving process.  But, it was something that ultimately I knew would give me greater access to the world, and that it would make my life easier in terms of being able to navigate and hear other people.

Roberts:  Was it hard for Rebecca to make the choice to get the cochlear implant?

Alexander:  I was fortunate that I certainly was not one of the pioneers in terms of being one of the first people to be implanted.  So that gave me some level of confidence.  But, I do think that being able to rely on, I think science, and other people’s experiences and being able to trust the process was really helpful.

Roberts:  Just like Dr. Fried has built on the work of researchers and scientists and doctors who came before him to develop cortical brain implants, users of technology too have to build on other patients pioneering choices.  Barbara Campbell, the early adopter of the Argus II retinal implant was one such pioneer.

Campbell:  Somebody has to be first and try it and learn the lessons.  There will be improvements and there will be things that work and things that do not work.  So they’ll be able to move forward from the lessons learned from my experience.

Roberts:  In this episode we were able to tell the story of Dr. Fried’s cortical brain implants by reaching into our archives to include tape about Dr. Humayun’s artificial retinas.  This is the way learning works.  The way science works.  Every new innovation is made possible by the innovations that came before.  And someday, maybe some day soon, the new technologies we profiled here will be foundational technologies of tomorrow.  This is our greatest hope.

Did anything of this episode or in previous episodes spark ideas for you?  If so, we want to hear about it.  Leave us a voice memo at 646-791-6115.  Again, that’s 646-791-6115.  Give us a call and you just might be on the show.

On Tech & Vision will take a break for the summer, but we’ll be back with more episodes in the fall.

I’m Dr. Cal Roberts.  On Tech & Vision is produced by Lighthouse Guild.  For more information visit www.lighthouseguild.org.  On Tech & Vision with Dr. Cal Roberts is produced at Lighthouse Guild by my colleagues Jaine Schmidt and Annemarie O’Hearn with support in part from Biogen.  My thanks to Podfly for their production support.