A team from Curtin University is using facial recognition technology to develop a platform that assesses pain experienced by patients that cannot verbally communicate.

Dubbed ‘ePAT (Electronic Pain Assessment Tool)’, the system works by recording a 10-second video of the patient’s face and micro-expressions are detected which then correlate to the amount of pain experienced by the patient.

The system is designed for patients who have difficulty communicating, such as people with moderate to severe dementia.

The team was recently awarded access to CSIRO’s second round of its ON Accelerator program, giving them the ability to perform market validation and rapid commercialisation of their system.

Professor Jeff Hughes, former head of school at Curtin University and CEO of ePAT Limited, spoke to IoT Hub about the technology behind the platform and his hopes for its deployment and use within the health industry.

“We engaged with a company called Nviso, who have been working in the area of facial micro-expression analysis, and identified them as a group that could turn our idea into a reality,” he said.

“They built the algorithm that we use in our technology to analyse those micro-expressions in real-time.”

Hughes said that the multi-dimensional nature of pain is such that the tool his team developed also had to recognise the number of ways that pain is expressed on a patient’s face.

“We have facial expression which is related to pain, and what we know is that there are certain action units, such as muscle contractions within the face, that are indicative of the presence of pain,” he explained.

Hughes said the video that captures the patient’s micro-expressions over the 10-second interval is used in combination with a smartphone or tablet app where the medical professional will record other non-facial pain cues such as vocalisation, movement, and other behaviours on a checklist.

“The tool will then collate all of the information and calculate a pain score for that patient,” he added.

He said that once an assessment of a patient’s pain is made, adequate treatment can be provided which should improve the results collected on subsequent assessments.

“What we’re aiming to do is allow the capture of pain treatment in-app, both pharmacological and non-pharmacological, and then to be able to graph a patient’s pain history over time and add comments to note down any factors which may have caused a worsening of pain,” he said.

“It would give people caring for patients with cognitive impairment more than just a tool to not only identify and measure the severity of pain but also the responsiveness to treatment.”

Catering to differing pain thresholds

Hughes said that there are a certain percentage of people who don’t outwardly express the pain they feel through facial expressions.

He also said that a range of things also influences pain thresholds, and much of it is learnt behaviour.

“There’s a misconception that people with dementia have increased pain thresholds, which is not true,” he explained.

“In actual fact, it’s likely that the reverse is true and they lose their learnt behaviours and revert to child-like expressions of pain.

“Whilst there are different levels of pain threshold between patients, there will be consistency for a given person, and it’s important to perform benchmarking for each patient to determine their threshold and pain markers applicable to that individual.”

The importance of going mobile

Hughes said the tool is being developed as an app for smartphones and tablets to provide caregivers with the mobility required to attend to patients who cannot attend a facility or who are being treated at home.

“Our patent covers the camera and processor, but in this day and age, we’re seeing more health care providers and people in general with portable smart devices with high-resolution cameras and excellent processing speeds.”

He said that remote connectivity is a key function that the team is looking to integrate into the app.

“We want to add connectivity within institutions such as residential aged-care facilities and hospitals to link with electronic patient records, and a lot of our discussions are also with aged care providers who have got patients who are remote to their facilities,” he said.

Hughes said that an Android prototype is currently being built and is scheduled for user testing this month, and an iOS version is also in the works.

He hopes to have the app approved by the Therapeutic Goods Administration – the government body that regulates the supply, import, export, manufacturing and advertising of therapeutic goods, including electronic medical devices – by mid next year.

The future of ePAT

Hughes would like to see future iterations of the ePAT platform integrate with other digital health devices to provide more data and a richer picture of pain assessment for its users.

“We see the future of having integration between platforms [such as heart rate monitors],” he said.

“Our first priority is to create an app that can function independently, then over time to integrate with other systems to collect other objective data as well.

“It’s very difficult to determine heart rate from visual observation alone, but if you’ve got someone who’s using a heart rate monitor, or even a wearable that measures heart rate, then you could conceivably feed that data into the app.”

He also hopes that continuous pain monitoring via wearables could become a reality and be used in conjunction with the episodic pain assessment for which the current platform is being built.