Watch: How “magic” apps are helping the blind to see

By Kitty Knowles 21 December 2015

The Aipoly app is changing lives by letting people use their smartphone to 'see' the world around them.

Whether we need to identify information around us, like a bus or door number, or we’re simply appreciating our changing surroundings, the joy of sight is something that’s easy to take for granted.

But when your sight deteriorates, doing the simplest of tasks like reading a book, getting to a friends house, or shopping on the high street, can become incredibly challenging – a reality faced by around 2m people in the UK.

Introducing the Aipoly Vision App

Posted by Aipoly on Saturday, December 19, 2015


Now though, a company called Aipoly is about to release an app that uses artificial intelligence to help the blind ‘see’ through their smartphone.

This uses your phone’s camera to identify objects and dictate what they are. It can recognise up to three objects per second, and doesn’t require internet connection to run.

Rob Turner, President of Silicon Valley for the blind identifying his Apple earbud earphones.

In the company’s latest video Rob Turner, President of Silicon Valley Council of the Blind, uses the app to correctly identify his ‘New Apple Earbud’ earphones.

“That’s like magic” he exclaims.

Clearly moved by the potential of the app he adds: “I don’t know how – that’s so impressive – I don’t even know what to say. I’m shocked.”

“It would be wonderful to have access to street signs, maybe even just being able to get a perspective – hey, what’s around here, what am I looking at? What building is in front of me? What kind of car is this?,” he said.

“As a blind person you don’t really think about the things that you might be able to see because you aren’t aware of them. So I might not even have all the possible ideas of how much this will be useful to me.”

Aipoly identifies 'bottle' and then brands like 'Pepsi' or 'Coca-Cola'. Pic: Aipoly.

The video also shows how the app can help in situations where using your hands to feel objects may not be ideal – to identify objects in a bathroom for example.

The app can also help colour blind people with an ability to name over 900 specific shades like “dirty orange”, “kiwi green”, or “wine red”.

Aipoly admits its “not perfect” (in the video it mistakes the large metallic doors of a lift for those of a ‘refrigerator’) but over time it will continue to learn thousands of new objects with the help of sighter users, making the potential benefits even more life-changing.

Part of a wider movement

Aipoly is just one tech company that is working to help the visually impaired regain the joys of sight.

Most recently, we featured Wayfinder, an app that uses blutooth tracking beacons to help blind people navigate the London Underground with audio directions.

Be My Eyes connects the visually impaired with sighter people who can personally talk them through what it in front of them.

NuEyes has created zoomable smart glasses that lets its users read, watch movies in the cinema, or play music.

Other wearable tech companies like OrCam read out visual information to users, like sign names, menus or whether a traffic crossing is red or green.

GiveVision's wearable headset can identify the faces of friends and loved ones. Pic: GiveVision

GiveVision, which uses GoogleGlass, can even be programmed to identify the faces of friends.

“This community feels powerless and is looking for inclusive and innovative solutions that will improve their quality of life and increase their independence,” Give Vision’s Elodie Draperi recently told The Memo.

“We want to give to both the severely sight impaired and totally blind access to visual information for their everyday life: going to a shop, paying for their goods, cooking, reading any printed text at work or home.”