In the field of deep learning, large quantities of data are typically required to effectively train models. This challenge has given rise to techniques like zero -shot learning (ZSL), which trains models on a set of "seen"classes and evaluates them on a set of "unseen"classes. Although ZSL has shown considerable potential, particularly with the employment of generative methods, its generalizability to real -world scenarios remains uncertain. The hypothesis of this work is that the performance of ZSL models is systematically influenced by the chosen "splits"; in particular, the statistical properties of the classes and attributes used in training. In this paper, we test this hypothesis by introducing the concepts of generalizability and robustness in attribute -based ZSL and carry out a variety of experiments to stress -test ZSL models against different splits. Our aim is to lay the groundwork for future research on ZSL models' generalizability, robustness, and practical applications. We evaluate the accuracy of state-of-the-art models on benchmark datasets and identify consistent trends in generalizability and robustness. We analyze how these properties vary based on the dataset type, differentiating between coarse- and fine-grained datasets, and our findings indicate significant room for improvement in both generalizability and robustness. Furthermore, our results demonstrate the effectiveness of dimensionality reduction techniques in improving the performance of state-of-the-art models in fine-grained datasets.

Generalizability and robustness evaluation of attribute-based zero-shot learning / Rossi, L.; Fiorentino, M. C.; Mancini, A.; Paolanti, M.; Rosati, R.; Zingaretti, P.. - In: NEURAL NETWORKS. - ISSN 0893-6080. - 175:(2024). [10.1016/j.neunet.2024.106278]

Generalizability and robustness evaluation of attribute-based zero-shot learning

Rossi L.
;
Fiorentino M. C.;Mancini A.;Rosati R.;Zingaretti P.
2024-01-01

Abstract

In the field of deep learning, large quantities of data are typically required to effectively train models. This challenge has given rise to techniques like zero -shot learning (ZSL), which trains models on a set of "seen"classes and evaluates them on a set of "unseen"classes. Although ZSL has shown considerable potential, particularly with the employment of generative methods, its generalizability to real -world scenarios remains uncertain. The hypothesis of this work is that the performance of ZSL models is systematically influenced by the chosen "splits"; in particular, the statistical properties of the classes and attributes used in training. In this paper, we test this hypothesis by introducing the concepts of generalizability and robustness in attribute -based ZSL and carry out a variety of experiments to stress -test ZSL models against different splits. Our aim is to lay the groundwork for future research on ZSL models' generalizability, robustness, and practical applications. We evaluate the accuracy of state-of-the-art models on benchmark datasets and identify consistent trends in generalizability and robustness. We analyze how these properties vary based on the dataset type, differentiating between coarse- and fine-grained datasets, and our findings indicate significant room for improvement in both generalizability and robustness. Furthermore, our results demonstrate the effectiveness of dimensionality reduction techniques in improving the performance of state-of-the-art models in fine-grained datasets.
2024
File in questo prodotto:
File Dimensione Formato  
Rossi_Generalizability-robustness-evaluation-attribute-based_2024.pdf

accesso aperto

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza d'uso: Creative commons
Dimensione 1.87 MB
Formato Adobe PDF
1.87 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11566/336738
Citazioni
  • ???jsp.display-item.citation.pmc??? 0
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 2
social impact