Neurodevelopmental disorders are estimated to affect 150 million people worldwide. In the US the cost of care for people with autism alone was $268 billion in 2015 – a number expected to rise to $461 billion by 2025, in the absence of more-effective interventions and support.
For people with autism, the disconnect between what they are thinking and feeling and what they are outwardly expressing is a major problem for a variety of social interactions. Patients become isolated, more alone, and feel more helpless and hopeless.
Atlantic Productions, in collaboration with Harvard Medical School faculty and innovative tech partners across the world, have developed a system that will potentially change the lives of patients living with autism, schizophrenia, and other mental and emotional disorders. Moreover, the system has potentially significant broader mental health benefits with applications in both social and work-related environments, even for healthy individuals.
Working with the latest virtual reality (VR) technologies and cutting-edge artificial intelligence (AI), we have created an adaptation of proven treatment, which uses and combines physical exercise and cognitive training. In addition to AI and VR, we employ the latest in gesture control (Leap Motion), and heart beat sensors, facial and eye tracking, and are looking to voice analysis as an additional diagnostic and therapeutic tool. The aim is to package these technologies into a complete ecosystem for treatment – while simultaneously providing an entertaining, engaging, narrative-driven experience.
The new AI-based tool which tracks eye, face, and voice patterns presents VR-based scenarios to patients in ways that incrementally reduce the fear or pain they feel in those situations. It helps patients learn to interpret other people’s expressions and, in turn, helps them to develop mastery of what they themselves are feeling and what they are outwardly expressing.
The Atlantic-Harvard faculty partnership was created through an initial proof-of-concept, adapting a non-immersive game which combined physical exercise and cognitive training in a therapy scenario. In the initial concept the user wears a VR head-mounted display, with a heart-rate monitor strapped to their chest, becoming immersed in a 6-degrees-of-freedom game environment. While using a peddle exerciser, patients were simultaneously challenged with tasks that would test their working memory. The biosensor information, provided by the heartrate monitor, was used for controlling the game mechanics, ensuring an optimal range of difficulty was maintained. Tests revealed that this approach was highly engaging and improved cognitive function in patients.
Research indicates that immersive experiences, such as those provided by VR, are likely to increase engagement as well as effectiveness of psychological treatments. These fully immersive worlds are not only potentially fun and engaging, but physicians can comfortably put their patients into them confident that they are in a safe and controlled environment. These treatments have potentially much wider applications, including other neuro-developmental disorders, such as schizophrenia, bi-polar disorder, and depression. Advancing technology-based applications, such as VR and AR-based interventions, can potentially bridge the large ‘mental health gap’ we see today.
While much of current AI tends to focus on creating efficiencies or replicating human tasks, we believe there is real room for AI to help people create and build happier personal relationships. We use AI technology to detect patients’ facial expressions and emotions in real time as they experience their virtual world. This project is a fundamental game changer as it lets physicians prescribe individual, AI-informed treatment. It is at the heart of the new era of precision medicine, where the treatment is specific to the individual patient.
Yet this approach is highly scalable. This type of treatment can be extended not only to a wide range of psychiatric disorders, but also to healthy people. The benefits are likely to be far-reaching in terms of improvements in physical and mental health.
At this stage, our program is ready for further development and testing. In this next phase we plan to expand into new games that have additional applications. Through a range of new creative approaches these potential applications could include cognitive bias modification, criticism management, managing social dilemmas, practicing social skills, and learning to take the perspectives of other people. These immersive VR games will be developed using the latest technologies such as real-time game engines, next generation CGI, 3D ambisonic audio, and haptic feedback.
The intention is that everything from the system to the VR headset to the AI enable software will be packaged up into a single off-the-shelf unit, with a centrally controlled database, available globally to health care institutions, community mental-health centers, employers, and individuals of any age who value talent development and cognitive enhancement. In other words, a highly individualized therapeutic solution in a one-size-fits-all package.
