Bias from the Wild Industry 4.0: Are We Really Classifying the Quality or Shotgun Series?

The traditional data quality control (QC) process was usually limited by the high time consuming and high resources demand, in addition to a limit in performance mainly due to the high intrinsic variability across different annotators. The application of Deep Learning (DL) strategies for solving the QC task open the realm of possibilities in order to overcome these challenges. However, not everything would be a bed of roses: the inability to detect bias from the collected data and the risk to reproduce bias in the outcome of DL model pose a remarkable and unresolved point in the Industrial 4.0 scenario. In this work, we propose a Deep Learning approach, specifically tailored for providing the aesthetic quality classification of shotguns based on the analysis of wood grains without running into an unwanted bias. The task as well as the collected dataset are the result of a collaboration with an industrial company. Although the proposed DL model based on VGG-16 and ordinal categorical cross-entropy loss has been proven to be reliable in solving the QC task, it is not immune to those who may be unwanted bias such as the typical characteristics of each shotgun series. This may lead to an overestimation of the DL performance, thus reflecting a more focus on the geometry than an evaluation of the wood grain. The proposed two-stage solution named Hierarchical Unbiased VGG-16 (HUVGG-16) is able to separate the shotgun series prediction (shotgun series task) from the quality class prediction (quality task). The higher performance (up to 0.95 of F1 score) by the proposed HUVGG-16 suggests how the proposed approach represents a solution for automatizing the overall QC procedure in a challenging industrial case scenario. Moreover, the saliency map results confirm how the proposed solution represents a proof of concept for detecting and mitigated unwanted bias by constraining the network to learn the characteristics that properly describe the quality of shotgun, rather than other confound characteristics (e.g. geometry).