The ideas and technologies developed by media agency tkm9 for a Coca-Coca Amatil pilot have carried over into ‘imHexa’, tkm9's digital display platform that’s underpinned by IoT technology.

imHexa turns a digital display in a store or shopping centre into a powerful data-collection device. It does this by using cameras to perform facial recognition that can determine a person’s gender, age range and mood when they view an on-screen advertisement.

It  also collects the MAC addresses of mobile devices that pass within the digital display’s vicinity, which can be used to  create foot traffic heat maps and detect multiple visits from the same person.

“imHexa has taken everything from IoT, and is utilising a number of components. It’s doing anonymous profiling of wifi, anonymous profiling of facial recognition, and it’s using facial recognition and wifi tracking to then serve the right content at a particular time,” tkm9’s global CEO Mark Hodgens told IoT Hub.

“It’s also anticipating that conversion [to a purchase] that might happen from interacting with that [advertising] content right through to point-of-sale.”

The Coca-Coca Amatil (CCA) trial involved integrating digital displays and cameras to 50 of the beverage maker’s refrigerators and vending machines. tkm9 developed a software platform to deliver augmented-reality applications to customers as they interacted with these assets.

The agency also created a number of FMCG (fast-moving consumer goods) interactions for CCA’s point-of-sale platform, making purchasing suggestions depending on the time of day – such as suggesting customers buy a meat pie with an energy drink at lunchtime.

Hodgens was quick to point out that no personally identifiable information is recorded or retrieved by tkm9’s imHexa platform. It only uses the MAC addresses of mobile devices for anonymous profiling and to gauge people’s reactions to the content displayed.

For example, the system is capable of determining the number of times a mobile device – and therefore a potential customer – has walked past a display.

If a given piece of advertising causes that customer to stop and react to the display in some way, it will record that nature and duration of that reaction along with the person’s gender and age range, and send an aggregated set of anonymous information to the retailer of the advertised product, according to Hodgens.

“What that allows tkm9 to do is say to a retailer that based on all of these profiles of people looking at their advertising, we can ensure the right content is served at the right time,” he said.

“We can also integrate this profile information into some front-of-store screens for that retailer, or potentially influence the layout of the store, based on the profile information collected.

“The output of us injecting IoT technology into physical displays has meant that we’ve been able to track and create heat maps of the locations that consumers are frequenting and what they’re doing based on our ability to serve advertising content and to detect their level of excitement or engagement of that content.”

Catering to behavioural change

Hodgens said the advertising industry is struggling to capture the attention of today’s consumers, because they are fixated on their mobile devices and desensitised to advertising.

He said that tkm9 has devised a way to engage with the customers, not by attempting to draw their attention away from their smartphones, but to use them to initiate contact.

“What we do with outdoor advertising is we use the mobile phone as the controller of content,” he said.

Hodgens pointed to a recent deployment of a three-projector overhead display in Westfield Garden City in Brisbane that delivers both advertising and content to smartphones.

“We allow people to open up their phone, where they can create a virtual lantern, write their name on it, and release it up to those digital screens,” he explained.

This, Hodgens said, allows people to control their interaction with the digital media and encourages them to look away from their phones to where the advertising is located.

He added that some agencies hold the mistaken belief that simply using large digital displays and presenting a “slideshow of ads” will be sufficient. Instead, he said, effective advertising today requires integration with personal devices and intelligent content to encourage interaction.