Multispectral imaging have used extensively in many Digital humanities applications. The spectral signature captures in information regarding the light reflected from the imaged object. This reflectance can differ once different materials are imaged. In the following two studies, we unveiled illegible information from Multispectral and Hyperspectral images.
Learning to Cluster Multispectral Signatures from multispectral weakly annotated imaging
Ohr Dallal, Steve De Santiago Molina, Nachum Dershowitz, Israel Finkelstein, Shira Faigenbaum-Golovin
Multispectral (and hyperspectral) imaging across different spectral domains has emerged as a common non-invasive technique for identifying the material composition within various historical media. For instance, multispectral imaging can be employed to distinguish between carbon ink and clay, as they exhibit distinct reflectance properties, aiding in the identification of ink within ancient inscriptions. Unfortunately, the preservation of these documents, which were inscribed over two and a half millennia ago, is often poor due to post-depositional processes. Consequently, they frequently exhibit signs of effacement, blurring, and staining, with ink traces that are often barely visible. As a result, classifying the ink pixels within such inscriptions poses a significant challenge. In our study, we tackle the task of identifying multispectral signatures using weakly annotated data. The task is further complicated by sparse and partial labels, where only a fraction of image pixels is manually annotated by human experts. To address these challenges, we develop a Transformer-based deep neural networks (DNN) model, alongside methods for pre-processing and enriching the data representation via tailored visual augmentations, weakly-supervised and self-supervised multispectral learning. The Transformer-based DNN exploits the complex shape and spectral cues available in the data, which allow for better discrimination of ink and background and possibly even the reconstruction of ink invisible to the naked eye. These techniques are employed to cope with the sparsity of available data and improve the model's generalization performance. The outcome of the proposed methodology is an image that assigns a probability to each pixel, indicating its predicted likelihood of containing ink. We demonstrate the significance of the multispectral characteristics of the data and demonstrate the effectiveness of our approach on Iron Age Hebrew inscriptions from the Judahite desert, dated c.a. 600 BCE, with implications to image binarization.
In the figure below an ink probability map is presented for Lachish 3 ostraca, where red indicates a higher probability of ink based on the multispectral signature.
Enhancing Underdrawing Legibility with Hyperspectral Imaging Data from a 15th-Century Painting
Peaslee, W., Melchiorre Di Crescenzo, M., Daly, N., Daubechies, I., Faigenbaum-Golovin, S., Higgit, C., Sober, B.,
Infrared reflectance hyperspectral imaging is frequently used to unveil underdrawings, preliminary sketches concealed by layers of paint. However, the high dimensionality of hyperspectral data and the spatial heterogeneity of paintings challenge underdrawing visualization. In this paper, we aim to improve the visualization of underdrawings by embedding the hyperspectral cube into a single image combining information from various spectral bands. First, we consider the Principal Component Analysis (PCA) and Minimum Noise Fraction transform (MNF). Next, we introduce new spatial localization of both PCA and MNF to further enhance the underdrawings legibility. Finally, the proposed methods are applied to two sets of short-wave infrared reflectance hyperspectral data. The first one records a test panel, while the other records \textit{The Adoration of the Kings}, a fifteenth-century panel attributed to Sandro Botticelli and Filippino Lippi. We find the proposed methods may be beneficial for underdrawing enhancement and visualization, and that incorporating spatial information is valuable for larger and more spatially heterogeneous passages.
