Giving devices the ability to sense the world around them is fundamental to the Internet of Things, and according to a senior Intel executive, the human body itself is being used as a reference point for sensing technology development.

“The goal when adding senses to devices is to copy the human perceptual system, to make devices and machines sense, understand, interact and also learn,” said Achin Bhowmik, vice president and general manager of Intel’s perceptual computing group.

Bhowmik was speaking at this year's Intel Developer Forum in San Francisco, from where IoT Hub will report over the next few days.

He said that one of the key challenges of enabling human-like sensing is replicating a human’s ability to perceive the world in three dimensions.

“To see the world in three dimensions, you not only have the two eyes, but also a nice piece of hardware known as the visual cortex at the back of the brain,” he said.

“One other system that’s important is the human vestibular sensing system, which is located inside the inner ear, and is the body’s accelerometer, gyroscope and motion sensor.

“When it comes to virtual reality and robotics, for example, you have to combine the visual and vestibular tracking for accurate 3D tracking and navigation.”

Bhowmik said that technologies that mimic these human abilities can be paired with machine learning systems to perform object recognition.

What are the applications?

He said that Intel’s take on this initiative, dubbed RealSense, has “evolved into a core technology for human-like sensing, which is going into robotics, virtual reality, and drones.”

“We wanted to copy the whole process, ranging from the sensor to the software libraries for cognition, autonomous navigation and interaction,” he said.

“We can also teach our systems with deep learning to recognise hand motions, objects, people, and so on.

He said that a system’s ability to recognise objects is dependent on the training data set used, which forms the reference baseline for the sensing system.

“If I was building a flower detection system, for example, then I would find a thousand samples of flowers and I would train my system to recognise them,” he said.

Bhowmik said that RealSense is capable of tapping into existing image libraries such as Google’s TensorFlow, so that developers to not have to build a training data set from scratch.

“The image recognition field is advancing so quickly, and we’re doing a lot of great work there as well, to have special training hardware systems – both the software libraries and the training hardware are evolving quite quickly.” 

Peter Gutierrez travelled to the Intel Developer Forum as a guest of Intel.