Using computer-assisted neural networks, researchers at the Karlsruhe Institute of Technology (KIT) and the University of Duisburg-Essen have been able to accurately identify affective states from the body language of tennis players during games. For the first time, they trained a model based on artificial intelligence (AI) with data from actual games.

Their study, published in the journal Knowledge-Based Systems, demonstrates that AI can assess body language and emotions with accuracy similar to that of humans. However, it also points to ethical concerns.

For their study, "Recognizing affective states from the expressive behavior of tennis players using convolutional neural networks," sports sciences, software development and computer science researchers from KIT and the University of Duisburg-Essen developed a special AI model. They used pattern-recognition programs to analyze video of tennis players recorded during actual games.

Success rate of 68.9%

"Our model can identify affective states with an accuracy of up to 68.9%, which is comparable and sometimes even superior to assessments made by both human observers and earlier automated methods," said Professor Darko Jekauc of KIT's Institute of Sports and Sports Science.

An important and unique feature of the study is the project team's use of real-life scenes instead of simulated or contrived situations to train their AI system. The researchers recorded video sequences of 15 tennis players in a specific setting, focusing on the body language displayed when a point was won or lost.

The videos showed players with cues including lowered head, arms raised in exultation, hanging racket, or differences in walking speed; these cues could be used to identify the players' affective states.

After being fed with this data, the AI learned to associate body language signals with different affective reactions and to determine whether a point had been won (positive body language) or lost (negative body language). "Training in natural contexts is a significant advance for the identification of real emotional states, and it makes predictions possible in real scenarios," said Jekauc.

Humans and machines recognize negative emotions better than positive ones

Not only does the research show that AI algorithms may be able to surpass human observers in their ability to identify emotions in the future, it also revealed a further interesting aspect: Both humans and AI are better at recognizing negative emotions.

"The reason could be that negative emotions are easier to identify because they're expressed in more obvious ways," said Jekauc. "Psychological theories suggest that people are evolutionarily better adapted to perceive negative emotional expressions, for example, because defusing conflict situations quickly is essential to social cohesion."

Ethical aspects need clarification before use

The study envisions a number of sports applications for reliable emotion recognition, such as improving training methods, team dynamics and performance, and preventing burnout. Other fields, including health care, education, customer service and automotive safety, could also benefit from reliable early detection of emotional states.

"Although this technology offers the prospect of significant benefits, the potential risks associated with it also have to be taken into account, especially those relating to privacy and misuse of data," Jekauc said.

"Our study adhered strictly to existing ethical guidelines and data protection regulations. And with a view to future applications of such technology in practice, it will be essential to clarify ethical and legal issues ahead of time."

More information: Darko Jekauc et al, Recognizing affective states from the expressive behavior of tennis players using convolutional neural networks, Knowledge-Based Systems (2024). DOI: 10.1016/j.knosys.2024.111856