Robotics systems have already been introduced in numerous real-world settings, including some industrial and manufacturing facilities. In these facilities, robots can assist human assembly line and warehouse workers, assembling some parts of products with high precision and then handing them to human agents tasked with performing additional actions.

In recent years, roboticists and computer scientists have been trying to develop increasingly advanced systems that could enhance these interactions between robots and humans in industrial settings. Some proposed solutions rely on so-called 'digital twin' systems, virtual models designed to accurately reproduce a physical object, such as specific products or components that are being manufactured.

Researchers at Nanjing University of Aeronautics and Astronautics in China recently introduced a new digital twin system that could improve the collaboration between human and robotic agents in manufacturing settings. This system, introduced in a paper published in Robotics and Computer-Integrated Manufacturing, can create a virtual map of real-world environments to plan and execute suitable robot behaviors as they cooperate with humans on a given task.

"In industrial settings, the current methods for constructing a human digital twin model rely on motion capture devices that require personnel to wear cumbersome equipment, which goes against the principle of flexible interaction advocated for HRC," Zequn Zhang, Yuchen Ji and their colleagues wrote in their paper.

"Furthermore, the current methods do not model humans and robots in a unified space, which is both unintuitive and inconvenient for perceiving and understanding the overall environment. To address these limitations, this paper proposes a digital twin system for HRC."

The digital twin system created by Zhang, Ji, and their colleagues creates a virtual replica of a scene in which a human and robot agent are collaborating. Subsequently, it plans effective collaborative strategies and executes them in a real-world environment.

Previously proposed digital twin systems that rely on data collected by motion capture sensors have been found to sometimes achieve unsatisfactory results in the presence of occlusions (i.e., when objects or agents of interest are outside of the sensors' field of view or are hidden behind obstacles). The researchers thus developed a human mesh recovery algorithm, a computational technique that can help to reconstruct occluded human bodies.

In addition, Zhang, Ji, and their colleagues introduced an uncertainty estimation technique in their system. This technique allows them to enhance the performance of the action recognition algorithm, a component of their system trained to recognize different human actions, by controlling the risk that this algorithm will make errors.

The researchers evaluated their new digital twin system in a series of experiments in laboratory settings, using a robot designed to be deployed in industrial settings. Their system was found to enhance the collaboration between this robot and a human agent in various tasks, including tasks that involved polishing, picking up, assembling, and placing down objects.

"Experimental results demonstrate the superiority of the proposed methods over baseline methods," Zhang, Ji, and their colleagues said. "Finally, the feasibility and effectiveness of the HRC system are validated through a case study involving component assembly."

The digital twin system developed by Zhang, Ji, and their colleagues could soon be implemented on other robots for industrial use and tested further in additional experiments, Ultimately, it could be introduced in real-world settings, to enhance the collaboration between robots and humans on various manufacturing and industrial tasks.

More information: Zequn Zhang et al, Enabling collaborative assembly between humans and robots using a digital twin system, Robotics and Computer-Integrated Manufacturing (2023). DOI: 10.1016/j.rcim.2023.102691.

© 2023 Science X Network