The rapid advancement of natural language processing (NLP) models and large language models (LLMs) has enabled the development of new use-specific conversational agents designed to answer specific types of queries. These range from AI agents that offer academic support to platforms offering general financial, legal or medical advice.

Researchers at Hefei University of Technology and Hefei Comprehensive National Science Center recently worked to develop an AI-based platform that can provide non-professional, but potentially helpful, psychological support. Their paper, presented at the International Conference on Multimedia Modeling held in Amsterdam from Jan. 29 to Feb. 2, introduces EmoAda, a conversational system trained to engage in emotional conversations, offering low-cost and basic psychological support.

"Our paper originated from a concern over the increasing prevalence of psychological disorders such as depression and anxiety, particularly following the COVID-19 pandemic, as well as the significant gap in the availability of professional psychological services," Xiao Sun, co-author of the paper, told Tech Xplore.

"This work builds on various research efforts, such as those by Fei-Fei Li et al on measuring depression severity from spoken language and facial expressions, those by Xiao Sun et al on multimodal attention networks for personality assessment, and the development of AI-based emotional support systems, like Google's LaMDA and OpenAI's ChatGPT."

The primary objective of this recent study was to create a cost-effective psychological support system that could perceive the emotions of users based on different inputs, producing personalized and insightful responses. This system is not designed to substitute professional help, but rather to alleviate stress and assist users in increasing their mental flexibility, a feature that has been associated with good mental health.

"EmoAda is a multimodal emotion interaction and psychological adaptation system designed to offer psychological support to individuals with limited access to mental health services," Sun explained. "It works by collecting real-time multimodal data (audio, video, and text) from users, extracting emotional features, and using a multimodal large language model to analyze these features for real-time emotion recognition, psychological profiling, and guidance strategy planning."

EmoAda, the platform created by Sun and his colleagues, can detect a user's emotions by analyzing various types of sensory data, including their voice, video footage of their face, and text. Based on these analyses, the system produces personalized emotional support dialogues, delivering them via text or via a digital avatar.

Based on a user's needs and the difficulties they mention, the platform might suggest various activities that might be beneficial. Some of these activities are facilitated by content available on the EmoAda platform, such as guided meditation practices and music for relaxation or stress relief.

"When tested with real users, EmoAda has been shown to provide natural and humanized psychological support," Sun said. "In these trials, we found that some users prefer conversing with AI because it can significantly reduce their anxieties about privacy breaches and social pressure. Engaging in dialogues with AI creates a safe, non-judgmental environment where users can express their feelings and concerns without fear of being judged or misunderstood. AI systems like EmoAda also offer round-the-clock support, free from time constraints, which is a significant advantage for users who need help at any given moment."

In initial test trials, the researchers found that one of the most appreciated aspects of EmoAda is its anonymity. In fact, users often mentioned that they felt comfortable with sharing private information that they would find difficult to discuss with other people face-to-face.

In the future, this new AI-based system could be deployed as a basic support service for people who cannot afford professional psychological care or are waiting to access available mental health services. In addition, EmoAda could serve as an inspiration for other research groups, paving way for the development of other AI-based mental health-related digital platforms.

"Our next studies will focus on addressing current system limitations, including optimizing the multimodal emotional interaction large language model to reduce misinformation generation, improve model interference performance, reduce costs, and integrate a psychological expert knowledge base to enhance system reliability and professionalism," Sun added.

More information: Tengteng Dong et al, EmoAda: A Multimodal Emotion Interaction and Psychological Adaptation System, MultiMedia Modeling (2024). DOI: 10.1007/978-3-031-53302-0_25

© 2024 Science X Network