Engineers quicken the response time for robots to react to human conversation
Microphone setup on REEM-C. Credit: PLOS ONE (2024). DOI: 10.1371/journal.pone.0296452

Talking to a robot often feels stilted or delayed, thanks to computer software trying to keep up with the conversation. However, new research from the University of Waterloo has improved the ability for humans to communicate naturally with humanoid robots.

Waterloo researchers were able to solve how a humanoid robot can identify the direction where was coming from, re-orient itself to track that voice to create a more realistic conversation and speed up its to communicate back with its human counterpart.

"Creating more natural conversations with is an interesting challenge because speech is so key to our social interactions," said Ewen MacDonald, a professor in Waterloo's Department of Systems Design Engineering and a member of the research team.

Credit: University of Waterloo

The research team built a hearing system for a humanoid robot using two microphones where a human's ears would typically sit to help generate an estimate of the direction in which audio sounds were coming from. Sounds produced by humans and other acoustic sources are reflected by surfaces such as objects or walls.

A signal processing pipeline is needed to account for these reflections as they could incorrectly fool the humanoid robots on where the sound is coming from. For a robot to react as quickly as a human would sonically, the robot's computer needs to generate that estimated location extremely quickly.

Waterloo's research develops a framework that can optimize the robot's processing speed and characterize based on overall performance and latency. Testing on the humanoid robot used the framework with recordings in a variety of acoustic environments.

How to have a more natural conversation with a robot
Example of two simulated microphone signals. Visualization of the signals y1 (blue) and y2 (orange). Credit: PLOS ONE (2024). DOI: 10.1371/journal.pone.0296452

Pranav Barot, a graduate student in Systems Design who worked on the , said the team was motivated by the desire to test the capabilities of humanoid robots being able to listen and interact with humans in real-time. One major challenge of the research was testing how the robot reorients itself to hear humans in large, loud, or crowded spaces.

"The implications of this research are important in any scenario or environment where humanoid robots will work together with human beings, both in social robotic situations or where humans and robots are working together," said Barot.

The research is published in the journal PLOS ONE.

More information: Pranav Barot et al, Estimating speaker direction on a humanoid robot with binaural acoustic signals, PLOS ONE (2024). DOI: 10.1371/journal.pone.0296452

Citation: Engineers quicken the response time for robots to react to human conversation (2024, April 11) retrieved 11 April 2024 from https://techxplore.com/news/2024-04-quicken-response-robots-react-human.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.