Towards an Automatic Defect Detection on Historical Surfaces Based on Hyperspectral Imaging Data: a Hierarchical Classification Framework

Continuous and systematic monitoring of built heritage surfaces is essential for effective conservation, as it enables the early detection of defects and timely interventions, resulting not only in improved preservation outcomes but also in a more efficient management of resources. However, current inspection techniques are often time-consuming and strongly dependent on the professional expertise, especially when invasive quantitative analyses are not feasible and assessments rely primarily on qualitative observations. In this context, Hyperspectral (HS ) Imaging is gaining increasing relevance in the heritage field, as it allows physicochemical information to be associated with each image pixel of a surface. This study proposes a hierarchical and hybrid methodology that combines statistical analysis, Deep Learning (DL), and Machine Learning (ML) techniques applied to HS imaging data for the automatic classification of heritage surface alterations and defects. First, Independent Component Analysis (ICA) is employed as a statistical dimensionality reduction technique to enhance informative spectral patterns related to surface degradation. A selected ICA component is then used as input to a U-Net Convolutional Neural Network (CNN), which is trained to identify and segment macro-scale degraded areas on historical surfaces. Subsequently, a Random Forest (RF) classifier is applied to the original HS data, restricted to the regions detected by the U-Net, to discriminate specific degradation typologies. This multi-stage strategy enables the spatial identification of surface alterations and defects, and the workflow is further extended to the metric quantification of the detected areas by exploiting the Ground Sampling Distance (GSD). Although the classification is currently restricted to a single material and two specific types of degradation, and notwithstanding challenges posed by surface roughness and cast shadows, preliminary results are promising and suggest that the proposed approach has the potential to support conservation professionals in the rapid detection and quantification of built heritage surface alterations and defects.