Wearable device to show the way for blind people

A*Star scientist's prototype has 'memorised' three routes to help users navigate in office building

Dr Xu Qianli with the prototype of his wearable navigation assistance for the visually disabled. It comprises a Google Glass device, a mobile phone app and vibrating motors that can be embedded in a user's shoes.
Dr Xu Qianli with the prototype of his wearable navigation assistance for the visually disabled. It comprises a Google Glass device, a mobile phone app and vibrating motors that can be embedded in a user's shoes. ST PHOTO: ALICIA CHAN

Navigating an unfamiliar building can be daunting for those with visual disabilities.

But a scientist from the Agency for Science, Technology and Research (A*Star) may be able to help.

After two years of research, Dr Xu Qianli from A*Star's Institute for Infocomm Research has come up with a wearable device that guides a user in the right direction using computer vision technology, and verbal cues and vibrations.

Called Show Me The Way, it comprises a Google Glass device, a mobile phone app and vibrating motors that can be embedded in a user's shoes.

The hardware is still clunky at this prototype stage, said Dr Xu, 39. But the key lies in the software - the computer algorithms that extract and process data and, based on the information provided, give instructions to the user.

The first step to using Show Me The Way, now still in a research phase, is to train the app to "memorise" a route. This is done by getting another person to walk the desired route while wearing the kit.

The camera in the Google Glass picks up the information and sends the data to the app, which processes it and extracts important visual cues, such as landmarks and "decision points". These could be areas where a user has more than one option, such as a T-junction.

So, if the person turns right at this point, the Google Glass registers it, and the app will note the appropriate signal to send to the user later on.

It took Dr Xu and his team two to three months to program Show Me The Way to "memorise" its first route within their office building in Fusionopolis.

But with the initial barrier overcome, adding a new route takes just a few days, said Dr Xu. The prototype can now navigate three routes within the building.

Once a route has been "memorised", users would be directed by voice cues played through the speakers in the Google Glass, and vibrations in their shoes.

The motors in the shoes will vibrate for each step in the right direction, in a concept similar to the raised tactile paving in MRT stations, Dr Xu said. When a user veers off in the wrong direction, the vibrations stop.

Dr Xu had worked with the Singapore Association of the Visually Handicapped (SAVH) on the project to gather feedback. But the next step is to scale up and improve the technology so user trials can be carried out.

For one thing, the hardware of the device must be improved to make it more user-friendly.

Aside from the cumbersome haptic modules on the shoes, Dr Xu said the Google Glass is too expensive, with each costing about US$1,600 (S$2,265). So, one priority is to look for cheaper alternatives.

Another improvement would be to fine-tune the computer algorithms, so that the software can extract more information from an image, such as whether there are obstacles along the route. He also hopes to have a crowdsourcing element in the system to increase the number of registered routes.

Dr Xu said: "There is a huge potential for indoor navigation devices - the market potential is projected to hit $2.6 billion in 2018.

"But this will also provide social benefits to the visually impaired. There are about 285 million people with visual disabilities around the world, and mobility is a challenge that Show Me The Way could potentially help solve."

Mr Francis Tay of SAVH welcomed the innovation, saying it could help visually disabled people find their destination unaided.

One issue they usually face while navigating indoors is not knowing where amenities like washrooms, entrances and exits, rooms and offices are, he said.

Temasek Polytechnic student Edwin Tan, 21, who is blind, said he usually moves around by memorising routes. For unfamiliar places, he would ask for directions.

"But I think this is a good invention. It could help me be more independent," he said.

SEE SCIENCE

Join ST's WhatsApp Channel and get the latest news and must-reads.

A version of this article appeared in the print edition of The Straits Times on December 18, 2015, with the headline Wearable device to show the way for blind people. Subscribe