Fujitsu Laboratories of America, Inc (FLA) has announced the development of technology that enables smart sensing and adaptive coaching for people exhibiting restricted and repetitive behaviours (RRBs), such as those affected by autism.

Dubbed ‘Guided Play’, the technology works by measuring the lack of variation in object manipulation during play. It provides behavioural intervention using principles of Applied Behaviour Analysis (ABA), which essentially states that human behaviour can be shaped by the environment.

RRBs are considered a core symptom and treatment target of a wide variety of neurological conditions including autism, and early recognition and intervention of these symptoms can significantly improve treatment prognosis and lower long-term behavioural costs, particularly for children.

According to Cong Chen, research staff member at FLA, this technology is being designed to replicate the actions of a behavioural therapist.

“A human therapist would first observe the child’s behaviour, join the play, follow the child’s lead, and at the appropriate time, change and modify the behaviour a little at a time, then immediately reward any compliance,” he said during Fujitsu’s North American Technology Forum last year.

“As you can imagine, this process is labour-intensive, and actually requires 20 hours a week of such therapy in order to make significant progress, which is why behavioural therapy is so expensive.

“What we’re trying to do is use digital technology to perform similar things.”

FLA has chosen building blocks as the basis of the technology platform, as their structures can be recognised by machines, and the progression of play as a child grows – from unstructured solo play to the shared creation of three-dimensional shapes – is well-known.

How does it work?

Children on the autism spectrum often exhibit repetitive behaviours, such as lining up toys, stacking blocks, and spinning objects obsessively.

In this case, a set of smart blocks is provided to the child, which continuously reads and sends behavioural data to the decision-making component of the platform, known as the Guided Play Coach (GPC).

The GPC then measures various aspects of the child's behaviour, such as variability and size of the built structure, comparing data with similarly aged peers.

When a RRB is detected, the GPC can send instructions to the attending therapist to start behaviour shaping and encourage the child to vary their activity.

“One key technology we’re also using are [blocks with] transformable connections, where connections can be disabled, stopping further continuation of repetitive construction,” Chen explained.

“We can also pop up connections as a prompt for the child to do something different.”

When desired behaviour is observed, reinforcement in the form of sound or music will be played by the smart blocks to promote and maintain behavioural variability.

Early progress and testing

FLA has been testing its hypotheses with software systems, as they are more flexible and easier to do, according to Chen.

“Using software gives us more flexibility in the system, such as showing backgrounds and shape outlines as a general prompt for the kids to build something different,” he said.

“We can then turn the digital creation into a live, animated object as a reward.”

This software is now freely available as an app on the iOS app store called ‘Guided Play’.

The app measures attributes such as virtual block variability, novelty, and complexity, and allows the caregiver to track the child’s progress over time.

The technology is currently being trialled in collaboration with the Center for Autism and Related Disorders in the United States.

According to Fujitsu, “current data shows that the technology successfully captures distinctive behavioural profiles in children with autism” and “the trials will continue to evaluate its effectiveness for increasing behavioural variability and flexibility.”

What's next?

“Our ultimate goal for this technology is first and foremost for special education, where we hope it can drive the construction of smart toys,” Chen said.

“It could also be used as a screening, testing and assessment program tool for professionals and families, and we also think there’s some worth for senior people who have cognition disabilities as well.”

Chen also believes that Guided Play be a platform for individualised therapeutic programs.

“Content producers such as game makers, therapists and teachers could customise and create games on the system, which people can then download for their kids.”