New Device Lets Blind People See!
By Erin Froehlich More Blogs by This AuthorFrom the Id and Ego Blog Series
As a writer, there are few things more difficult or more interesting than using words to describe the sort of things that normally transcend them. A sound – English speaking people agree that cows make the sound “moo”, that dogs make the sound “woof” and that a car goes “honk!” – but what about a song?
Let’s use an example we’ll all know, mmm…. first one that comes to mind is “Mary Had a Little Lamb”. I’m hearing it inside my head right now, but if I had to describe with words on paper just how “Mary Had a Little Lamb” sounds different than any other song in the world, it would be a definite challenge.
Or a taste – how is honey’s flavor different than cane sugar? Tricky….
I came across a great question in this same vein once (and if you read my blog “The 15 Best Questions to Really Get to Know Someone”, you know I love great questions) which goes:
“If you had to describe the colors of the rainbow to a blind person – how would you?”
Colors – red, orange, yellow, green, blue... to most of us, they’re so basic to our own understanding of the world that we take their perception for granted. However, I’ll give a description a try:
- Red: the color of stop signs. Dangerous, sexy, ripe.
- Orange: the color of oranges. Playful, fun, exciting.
- Yellow: the color of sunshine. Joyful, lively, innocent.
- Green: the color of most plants. Fresh, natural, pure.
- Blue: the color of the sky. Peaceful, quiet, sad.
- Indigo: the color of deep ocean. Serious, cold, business-like.
- Violet: the color of many wild flowers. Mysterious, luxurious, spiritual.
Interesting maybe, but is it really a fair translation? I did my best using my own emotional associations with them still, I’m not really satisfied. Facebook friends! What do YOU think?
Me: Enlisting your help for my blog again - if you had to describe the colors of the rainbow to a blind person - how would you?
Dan H: "Oh, it's soooo coooool. Too bad you'll never see it, loser!"
… Dan. Go figure my friends would have some sarcastic comment to make....
Dan H: Seriously though; this is one of those imponderables. It's akin to a person who can see normally wondering whether the colors he/she sees appear the same to others. Does a person who's never been able to see able to interpret or picture color the same way people with normal vision do?
Really I'd just describe it in scientific terms; that the colors of the rainbow in ROYGBIV order are the components of white light. And that white light is the portion of the electromagnetic spectrum that's processed by human eyes. Etc. etc. etc.
So there’s two ways we found – emotional and scientific. Once again, interesting, clever, but the point I’m trying to make is that it’s really, REALLY difficult and yet - this whole idea, the need to describe visuals in a way a blind person could understand, is actually the basis of a new Sensory Substitution Device (SSD) designed by Dr. Amir Amedi.
The Sensory Substitution Device
Using a small camera fitted into a pair of glasses hooked up to a cellphone or small computer, he was able to design a program that can translate visuals into their sound equivalents in the brain! (…:O… I know, right?!) With it, blind people can open a door and “see” where furniture is sitting in a room, where people are, even if they are smiling or frowning. They can read with this device, even get a translation of illustrations in the book that they could understand.
Basically, they’re getting a sound-scape of the world around them much they way dolphins or bats do. Just incredible, but this device wasn’t actually what Dr. Amir originally set out to make when he first began his research. As he explained in a radio interview with Dr. Adam Goldstein, everything started with a question – what happens to visual portion of our brain if we can’t actually see?
“…We know that about 25 to 30% of the brain is dedicated to vision.…(but) what’s happening to all these unemployed areas in the brain (for) someone that was born blind and had no visual experience and people that became blind later in life and now don’t get regular input for 10 years, 15 years? What’s going on with all these billions and billions of neurons? What happens when we do the things we usually do by vision when we do it with other senses?”
In a controversial first paper on the subject he declared that based on the findings of his research he knew the old notions of our brain’s cortex to be wrong. In his study, he compared the brain activity of seeing people while reading with the brain activity of a blind people reading through touch with Braille and as he said “...It’s actually indistinguishable. …The brain doesn’t care where it’s getting the information from.”
The portion of our brain commonly described as the “visual cortex” was actually is capable of all kinds of environmental processing and understood them all in virtually the same manner. It was with that discovery that he thought, “I can help these people,” sought to do so and succeeded in for the lucky few that now use these specialized SSD glasses every day.
What The Images Sound Like To Me
Now though, as miraculous as the device really seems, it's not an instant fix. It requires months of training. To illustrate a point, Dr. Amir gave two simple examples of a sound translation in his interview – a diagonal line and the letter “w.” - interesting, but they were just gibberish to me until he explained them.
The diagonal line sound goes from a low to a high pitch and the line would start low and move higher. The W likewise uses pitch to draw the image – high then low, high then low then high again all fairly quickly. "Ah-ha!” I thought. “I get it now! Give me another! :D”
Next, he picked something far more complex – a photo of a person standing next to a horse.
Internet fogies, you remember how dial-up internet used to sound? It was lot like that - though thankfully less piercing and irritating. For all I know, it could have been a photo of him flipping me the bird. He literally could have told me it was anything. The interviewers had essentially the same reaction.
That’s when he explained that using the SSD was a lot like learning a new language. It WOULDall eventually make sense to the blind person, but it would take time and they would never be able to interpret those sounds as quickly as we could take in visual information. Where vision is practically instant for us, it would probably take a blind person with this device about 8 seconds to understand the layout of a room and the people inside it. Still, what a huge life change to be able to understand your environment without having to touch everything!
As Dr. Amir’s (pictured left in his device) new Sensory Substitution Device currently stands, developers are working on making the sounds more pleasant, even musical, but there has yet to be a serious funder for wide-spread distribution. It's particularly maddening as he explains the actual manufacture would be relatively inexpensive. The barrier, he says is the “fantasy” perception people have of it. They don’t think it’s possible. However, he’s optimistic that once people understand the real scientific basis it has and that it’s not just some strange fad or experiment, they’ll appreciate the potential it possesses to help many, many people.
The Stuff of Movie Magic
Researching this I got a lot of movie comparisons – Daredevil was one the interviewers used, though I’ve personally never seen it. “Mask” was another friend on Facebook mentioned:
Nancy T: “Erin have you ever see n the movie "Mask" with Cher and Sean Penn who plays a boy who has a bone disorder that causes him to have a lion like face?...anyway, he goes to work at a blind camp and falls for a blind gal and tries to do this very thing...he brings her to the kithen...places in her hand something hot (not too hot) and describes red...then something from the freezer and describes it as blue...cotton balls for white...”
But being one of my favorite movies, Amelie and the scene where she helps a blind beggar “see” by guiding him down the street describing all the sights to him, was what instantly came to my mind. As she leaves him, he looks up to sky in pure wonder, glowing light all around him. I’m not blind, so any guess I’d make would be purely speculation, but I imagine that having this device would be a little like that.
Amelie: “Let me help you. Step down. Here we go! The drum major's widow! She's worn his coat since the day he died. The horse's head has lost an ear! That's the florist laughing. He has crinkly eyes. In the bakery window, lollipops. Smell that! They're giving out melon slices! Sugarplum, ice cream! We're passing the park butcher. Ham, 79 francs. Spareribs, 45! Now the cheese shop. Picadors are 12.90. Cabecaus 23.50. A baby's watching a dog that's watching the chickens. Now we're at the kiosk by the metro. I'll leave you here. Bye!”