An AI system that offers emotional support via chat
Homepage of the EmoAda Software System. On the homepage, the digital persona will provide guidance, and a user can access various function modules by clicking on the relevant icons or options or using voice commands. Credit: Dong et al

The rapid advancement of natural language processing (NLP) models and large language models (LLMs) has enabled the development of new use-specific conversational agents designed to answer specific types of queries. These range from AI agents that offer academic support to platforms offering general financial, legal or medical advice.

Researchers at Hefei University of Technology and Hefei Comprehensive National Science Center recently worked to develop an AI-based platform that can provide non-professional, but potentially helpful, psychological support. Their paper, presented at the International Conference on Multimedia Modeling held in Amsterdam from Jan. 29 to Feb. 2, introduces EmoAda, a conversational system trained to engage in emotional conversations, offering low-cost and basic psychological support.

"Our paper originated from a concern over the increasing prevalence of psychological disorders such as depression and anxiety, particularly following the COVID-19 pandemic, as well as the significant gap in the availability of professional psychological services," Xiao Sun, co-author of the paper, told Tech Xplore.

"This work builds on various research efforts, such as those by Fei-Fei Li et al on measuring depression severity from spoken language and facial expressions, those by Xiao Sun et al on multimodal attention networks for personality assessment, and the development of AI-based emotional support systems, like Google's LaMDA and OpenAI's ChatGPT."

The primary objective of this recent study was to create a cost-effective psychological support system that could perceive the emotions of users based on different inputs, producing personalized and insightful responses. This system is not designed to substitute professional help, but rather to alleviate stress and assist users in increasing their mental flexibility, a feature that has been associated with good mental health.

"EmoAda is a multimodal emotion interaction and psychological adaptation system designed to offer psychological support to individuals with limited access to ," Sun explained. "It works by collecting real-time multimodal data (audio, video, and text) from users, extracting emotional features, and using a multimodal large language model to analyze these features for emotion recognition, psychological profiling, and guidance strategy planning."

An AI system that offers emotional support via chat
The architecture of the Multimodal Emotion Interaction Large Language Model. The researchers used the open-source model baichuan13B-chat as the foundation, integrating deep feature vectors extracted from visual, text, and audio models through an MLP layer into baichuan13B-chat. They employed a Mixture-of-Modality Adaptation technique to achieve multimodal semantic alignment, enabling the LLM model to process multimodal information. The team also constructed a multimodal emotion fine-tuning dataset, including open-source PsyQA dataset and a team-collected dataset of psychological interview videos. Using HuggingGPT[6], they developed a multimodal fine-tuning instruction set to enhance the model's multimodal interaction capabilities. The researchers are creating a psychological knowledge graph to improve the model's accuracy in responding to psychological knowledge and reduce model hallucinations. By combining these various techniques, MEILLM can perform psychological assessments, conduct psychological interviews using psychological scales, and generate psychological assessment reports for users. MEILLM can also create comprehensive psychological profiles for users, including emotions, moods, and personality, and provides personalized psychological guidance based on the user's psychological profile. MEILLM offers users a more natural and humanized emotional support dialogue and a personalized psychological adaptation report. Credit: Dong et al

EmoAda, the platform created by Sun and his colleagues, can detect a user's emotions by analyzing various types of sensory data, including their voice, video footage of their face, and text. Based on these analyses, the system produces personalized emotional support dialogues, delivering them via text or via a digital avatar.

Based on a user's needs and the difficulties they mention, the platform might suggest various activities that might be beneficial. Some of these activities are facilitated by content available on the EmoAda platform, such as guided meditation practices and music for relaxation or stress relief.

"When tested with real users, EmoAda has been shown to provide natural and humanized psychological support," Sun said. "In these trials, we found that some users prefer conversing with AI because it can significantly reduce their anxieties about privacy breaches and social pressure. Engaging in dialogues with AI creates a safe, non-judgmental environment where users can express their feelings and concerns without fear of being judged or misunderstood. AI systems like EmoAda also offer round-the-clock support, free from time constraints, which is a significant advantage for users who need help at any given moment."

In initial test trials, the researchers found that one of the most appreciated aspects of EmoAda is its anonymity. In fact, users often mentioned that they felt comfortable with sharing private information that they would find difficult to discuss with other people face-to-face.

In the future, this new AI-based system could be deployed as a basic support service for people who cannot afford professional psychological care or are waiting to access available mental health services. In addition, EmoAda could serve as an inspiration for other research groups, paving way for the development of other AI-based mental health-related digital platforms.

"Our next studies will focus on addressing current system limitations, including optimizing the multimodal emotional interaction large language model to reduce misinformation generation, improve model interference performance, reduce costs, and integrate a psychological expert knowledge base to enhance system reliability and professionalism," Sun added.

More information: Tengteng Dong et al, EmoAda: A Multimodal Emotion Interaction and Psychological Adaptation System, MultiMedia Modeling (2024). DOI: 10.1007/978-3-031-53302-0_25

© 2024 Science X Network

Citation: An AI system that offers emotional support via chat (2024, March 1) retrieved 1 March 2024 from https://techxplore.com/news/2024-03-ai-emotional-chat.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.