When computer vision expert Oded Ben Dov was contacted by a quadriplegic engineer following a demonstration of his video game on Israeli TV, he had no idea that it would lead to the world’s first smartphone controlled by head movements.

“I got a phone call [the day after the demonstration] and the guy on the line said ‘Hi, I can’t move my hands or legs, could you make me a smartphone I could use?’”, Ben-Dov told IoT Hub.

“That was Giora Livne, who is today our partner and co-founder of Sesame. Giora realised the potential gesture technology has for people in his condition.”

The result of this fateful phone call is the Sesame Enable, touted as “the world’s first touch-free smartphone”.

The device works by utilising the front-facing camera of a Google Nexus 5 smartphone to detect the user’s head movements.

The movement is then processed by an algorithm which then translates the head movement into movement of an onscreen cursor.

Ben Dov said that the decision to use the front-facing camera to register head movements was a simple one.

“If you think about it, when someone looks at a smartphone, the front-facing camera is looking right back,” he explained.

Catering for different disabilities

At first, development of the smartphone was centred around creating a device to suit Livne’s needs as a quadriplegic who suffered a spinal cord injury.

However, this user scope expanded quickly, according to Ben Dov.

“Once we had a working prototype, we expanded our beta user group to include more people from different disability backgrounds, such as ALS (Amyotrophic Lateral Sclerosis), Cerebral Palsy, or Multiple Sclerosis, among other conditions,” he said.

“One of the things we discovered with special needs is that everyone has special needs of their own. Two people could have the same condition, but still vary a lot in posture, the ability to move their head, steadiness, and more.

“So with time, we made the solution more and more customisable to answer to all users’ needs. For example, you can adjust the sensitivity to apply to your range of head movements, and we even have a feature that measures this automatically and sets the right sensitivity settings.”

Life-changing technology

Ben Dov said that the response from users has been overwhelming.

“I don’t think there is a single user that hasn’t referred to it as life-changing,” he said.

“It’s not just an app, it’s a way for them to communicate again, to gain independence.

“For some, it’s the privacy of having a phone call by themselves, without someone holding or waiting to hang up for them. One user’s family referred to it as a safety blanket – a way for them to communicate if they need to leave the house or leave the user by himself.”

Ben Dov said that Sesame is investigating opportunities to integrate this technology into other devices, but added that there has been plenty of feedback from current Enable users requesting for more functionality from their smartphones.

“One interesting feature we keep being asked for is to take selfies or photos with the device,” he explained.

“Currently, since we rely on the device’s camera to track the user, the camera does not work in other apps, but we are looking into how to make this happen.”

Ben Dov also hopes that the less revolutionary but still useful voice control capability built into the device will also give users more freedom.

“We see voice control and computer vision as complementary technologies, each with its own strengths,” he said.

“We have voice commands for answering or dismissing incoming calls, and several more for easy navigation.

“We also advise our users to take advantage of the existing Google voice control which is very powerful, and could also be used for dictating long messages.”