Autonomous systems for monitoring and surveying are increasingly used in retail stores, since they improve the overall performance of the store and reduce the manpower cost. Moreover, an automated system improves the accuracy of collected data by avoiding human-related factors. This paper presents ROCKy, a mobile robot for data collection and surveying in a retail store that autonomously navigates and monitors store shelves based on real-time store heat maps; ROCKy is designed to automatically detect Shelf Out of Stock (SOOS) and Promotional Activities (PA) based on Deep Convolutional Neural Networks (DCNNs). The deep learning approach evaluates visual and textual content of an image simultaneously to classify and map SOOS and PA events during working hours. The proposed approach was applied and tested on several real scenarios, presenting a new public dataset with more than 14.000 annotated shelves images. Experimental results confirmed the effectiveness of the approach, showing high accuracy (up to 87%) in comparison with the existing state of the art SOOS and PA monitoring solutions, and a signification reduction of retail surveying time (45%).

Robotic retail surveying by deep learning visual and textual data / Paolanti, M.; Romeo, L.; Martini, M.; Mancini, A.; Frontoni, E.; Zingaretti, P.. - In: ROBOTICS AND AUTONOMOUS SYSTEMS. - ISSN 0921-8890. - 118:(2019), pp. 179-188. [10.1016/j.robot.2019.01.021]

Robotic retail surveying by deep learning visual and textual data

Paolanti M.;Romeo L.;Martini M.;Mancini A.;Frontoni E.;Zingaretti P.
2019-01-01

Abstract

Autonomous systems for monitoring and surveying are increasingly used in retail stores, since they improve the overall performance of the store and reduce the manpower cost. Moreover, an automated system improves the accuracy of collected data by avoiding human-related factors. This paper presents ROCKy, a mobile robot for data collection and surveying in a retail store that autonomously navigates and monitors store shelves based on real-time store heat maps; ROCKy is designed to automatically detect Shelf Out of Stock (SOOS) and Promotional Activities (PA) based on Deep Convolutional Neural Networks (DCNNs). The deep learning approach evaluates visual and textual content of an image simultaneously to classify and map SOOS and PA events during working hours. The proposed approach was applied and tested on several real scenarios, presenting a new public dataset with more than 14.000 annotated shelves images. Experimental results confirmed the effectiveness of the approach, showing high accuracy (up to 87%) in comparison with the existing state of the art SOOS and PA monitoring solutions, and a signification reduction of retail surveying time (45%).
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11566/271293
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 29
  • ???jsp.display-item.citation.isi??? 19
social impact