Manufacturing is shifting from autonomous robots to collaborative robots (cobots) that work alongside humans. To foster effective human-robot collaboration, the robot must be aware of the operator's needs and be able to adapt accordingly. A fundamental step in this collaboration is the robot's ability to recognize and understand the operator's actions online. This study aims to develop a computer vision system to recognize and classify human actions in a collaborative assembly scenario. The goal has been to create a lightweight action recognition model, by investigating various models, including Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks, combined with YOLO-based object recognition. This work presents several model architectures designed to classify human actions based on different types of input data. Specifically, the models process hand landmark data from an RGB-D camera. Four actions commonly found in manufacturing environments are classified. A custom dataset was created from scratch to support this research. From the analysis of the different approaches, the best-performing model achieved an offline action recognition accuracy of 95%, and an online accuracy exceeding 90%. Comparing different methodologies and input data for human action recognition helps to highlight their strengths and limitations, enabling the identification of the most effective strategy for recognizing actions in human-robot collaboration. This approach contributes to significant advancements in developing more intuitive and efficient collaborative environments within manufacturing settings. Data, code, and results are available at: https://github.com/grgzpp/human-action-recognition-and-anticipation.
Recognizing Human Actions in Collaborative Human-Robot Assembly Manufacturing / Zoppi, G., Forlini, M., Palmieri, G., Neto, P.. - (2025). (30th IEEE International Conference on Emerging Technologies and Factory Automation, ETFA 2025 Porto, Portugal 9 - 12 September 2025) [10.1109/ETFA65518.2025.11205748].
Recognizing Human Actions in Collaborative Human-Robot Assembly Manufacturing
Zoppi G.Primo
;Forlini M.;Palmieri G.;
2025-01-01
Abstract
Manufacturing is shifting from autonomous robots to collaborative robots (cobots) that work alongside humans. To foster effective human-robot collaboration, the robot must be aware of the operator's needs and be able to adapt accordingly. A fundamental step in this collaboration is the robot's ability to recognize and understand the operator's actions online. This study aims to develop a computer vision system to recognize and classify human actions in a collaborative assembly scenario. The goal has been to create a lightweight action recognition model, by investigating various models, including Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks, combined with YOLO-based object recognition. This work presents several model architectures designed to classify human actions based on different types of input data. Specifically, the models process hand landmark data from an RGB-D camera. Four actions commonly found in manufacturing environments are classified. A custom dataset was created from scratch to support this research. From the analysis of the different approaches, the best-performing model achieved an offline action recognition accuracy of 95%, and an online accuracy exceeding 90%. Comparing different methodologies and input data for human action recognition helps to highlight their strengths and limitations, enabling the identification of the most effective strategy for recognizing actions in human-robot collaboration. This approach contributes to significant advancements in developing more intuitive and efficient collaborative environments within manufacturing settings. Data, code, and results are available at: https://github.com/grgzpp/human-action-recognition-and-anticipation.| File | Dimensione | Formato | |
|---|---|---|---|
|
Zoppi_Recognizing-Human-Actions-Collaborative_2025.pdf
Solo gestori archivio
Tipologia:
Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza d'uso:
Tutti i diritti riservati
Dimensione
1.42 MB
Formato
Adobe PDF
|
1.42 MB | Adobe PDF | Visualizza/Apri Richiedi una copia |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.


