The key challenge in expanding human-robot collaboration is enabling robots to understand and adapt to human needs, fostering seamless interaction. This research develops an open-source framework integrated with a collaborative robotic arm to recognize, predict, and anticipate human actions in an assembly scenario. Human action classification was performed by comparing various approaches, including Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks, combined with YOLO-based object recognition. The study utilized a pneumatic cylinder as an example component for the assembly, along with the KUKA LBR iiwa 7 R800 robotic arm. The action recognition model, developed from scratch with a custom dataset, achieved 95% offline accuracy and over 90% online. The action prediction model, trained on real assembly sequences from human demonstrations, suggests the next robot action. This approach improves the flexibility and customization by allowing the assembly sequence to be learned directly observing the operator performing the task, without prior knowledge (except for the objects to detect). This advancement enhances cobots ability to recognize, predict, and anticipate human actions, improving intuitive and efficient collaboration in manufacturing.

Recognition and Anticipation of Human Actions in a Human-Robot Collaborative Assembly Scenario / Zoppi, G., Forlini, M., Palmieri, G., Neto, P.. - (2025), pp. 1229-1234. (34th IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2025 nld 2025) [10.1109/ro-man63969.2025.11217712].

Recognition and Anticipation of Human Actions in a Human-Robot Collaborative Assembly Scenario

Zoppi, Giorgio;Forlini, Matteo;Palmieri, Giacomo;
2025-01-01

Abstract

The key challenge in expanding human-robot collaboration is enabling robots to understand and adapt to human needs, fostering seamless interaction. This research develops an open-source framework integrated with a collaborative robotic arm to recognize, predict, and anticipate human actions in an assembly scenario. Human action classification was performed by comparing various approaches, including Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks, combined with YOLO-based object recognition. The study utilized a pneumatic cylinder as an example component for the assembly, along with the KUKA LBR iiwa 7 R800 robotic arm. The action recognition model, developed from scratch with a custom dataset, achieved 95% offline accuracy and over 90% online. The action prediction model, trained on real assembly sequences from human demonstrations, suggests the next robot action. This approach improves the flexibility and customization by allowing the assembly sequence to be learned directly observing the operator performing the task, without prior knowledge (except for the objects to detect). This advancement enhances cobots ability to recognize, predict, and anticipate human actions, improving intuitive and efficient collaboration in manufacturing.
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11566/367239
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact