The blind may just be getting a step closer to being able to “see,” and it doesn’t involve any surgery or invasive procedures. All they need is a vOICe.
Dutch physicist Peter Meijer has developed software—the vOICe—that converts live camera images into corresponding soundscapes in a way that preserves pictorial information.
“I developed the vOICe with the hope that the human brain is smart enough to learn and make a mental reconstruction of the visual content of the soundscapes. Mental imagery would then offer blind users an approximate equivalent of eyesight,” he said.
The technology offers a form of sensory substitution. The way it works is by scanning video frames from left to right in a second while associating height with pitch and brightness with sound intensity, according to Meijer. “A bright rising line simply becomes a rising tone, and a bright rectangle becomes a noise burst, such that if you hear a rising tone or a noise burst you can visualize the corresponding shape.”
According to Meijer, some late-blind users who have memories of prior eyesight reported a visual experience; however, the percentage of users who experience visual perception with soundscapes is unknown at the moment. In a paper written by Meijer and psychology professor Jamie Ward titled “Visual experiences in the blind induced by an auditory sensory substitution device,” one late-blind user reported the following:
“Just sound? No, it is by far more: It is sight! There IS true light perception generated by The vOICe. When I am not wearing The vOICe, the light I perceive from a small slit in my left eye is a grey fog. When wearing The vOICe, the image is light with all the little greys and blacks… The light generated is very white and clear, then it erodes down the scale of color to the dark black. I don’t really see a difference in this light as compared to the ‘light phosphenes’ they are talking about.”
Currently there is a Windows and Android version of vOICe. The Windows version requires USB camera glasses and a Windows tablet. The technology supports all Windows operating back to Windows 95, as well as ME and NT.
The Android version is awaiting the arrival of augmented reality glasses like the Google Glass. The technology already runs on the Glass, but the user interface hasn’t adapted to it yet, according to Meijer. He explained that it is important to have a nonintrusive head-mounted device for hands-free use. All versions of the software are free for non-commercial and academic use.
There is also a free iOS app called EyeMusic, developed by a neuroscience research group in Israel, which used vOICe for its research on brain plasticity.
“They developed EyeMusic as an experimental tool to investigate what happens in the brain if you encode color in sound, as well as to try and make the soundscapes a bit more pleasant,” said Meijer. “However, this goes at the expense of effective visual resolution (visual acuity), which is what the vOICe focuses on for use by the blind.”
Future development of the technology depends on what is going to happen in the market for augmented reality devices.
“Possible future extensions include support for sonar, binocular vision, object recognition, eye-tracking, location-aware technology (with GPS), and brainwave entrainment as neural engineering ingredients in a highly modular, adaptive and preferably non-invasive and unobtrusive brain-computer interface for augmented vision, augmented reality, and augmented cognition,” according to vOICe’s website.
