Over the past four years, a research team at the Artificial and Mechanical Intelligence (AMI) lab at the Istituto Italiano di Tecnologia (IIT-Italian Institute of Technology) in Genova (Italy) has developed advanced avatar technologies, known as the iCub3 system, in continuous testing with real-world scenarios.

The system was utilized to enable a human operator to remotely visit locations 300 km away, entertain the public at events and television appearances, and compete in the international challenge ANA Avatar XPrize.

In their research paper, published in Science Robotics, the group highlights the challenges they faced, and the solutions implemented to develop the avatar system. Their approach highlights how crucial is for research to extend beyond laboratory confines and addresses the difficulties posed by the real-world variability, in order to develop a robust humanoid robotic platform that can soon become part of the economic and productive system.

The iCub3 system has already evolved into a new robot, the ergoCub robot, designed to maximize acceptability within work environments.

The research group at the AMI lab at the IIT is coordinated by the Italian researcher Daniele Pucci and comprises about 50 researchers. The iCub3 avatar system is mainly composed of the robot iCub3—which is one of the last versions of the iCub humanoid robot born in IIT 20 years ago—and the wearable technologies named iFeel that track human body motions.

The avatar system has been designed to facilitate the embodiment of humanoid robots by human operators encompassing aspects such as locomotion, manipulation, voice, and facial expressions with comprehensive sensory feedback including visual, auditory, haptic, weight, and touch modalities. It has been developed by IIT in collaboration with the Italian National Institute for Insurance against Accidents at Work (INAIL).

The IIT research group tested and tailored the iCub3 avatar system in three different real scenarios.

In November 2021, during the first test a human operator located in Genoa controlled the avatar at the Biennale di Venezia in Venice (approximately 300 kilometers away) having the possibility to visit remotely the Italian art exhibition. The challenges were about assuring stable remote communications between the two sites, within a limited testing timeframe.

Furthermore, the robot's movement and interaction with the surrounding environment had to be cautious and safe, given the delicate nature of the art exhibitions.

Pucci's group utilized the IIT wearable iFeel suit to track the operator's body motions, and the avatar system transferred these movements onto the iCub3 robot in Venice, allowing it to replicate the user's actions in Genova.

The operator exercised precise control over the robot's body parts, including the fingers and eyelids. Additionally, custom haptic devices were employed to allow the operator to feel a hug from the remote location. The transmission was streamed over a standard optical fiber internet connection, resulting in around a hundred milliseconds of delay.

In June 2022, the second test took place at the Italian digital innovation festival, We Make Future Show, presenting additional challenges.

The operator in Genova controlled a robot positioned at the festival venue in Rimini (approximately 300 km away). The robot's task involved receiving and transporting a payload from a person while navigating the theater stage before an audience of around 2,000 spectators.