PATCH: A deep learning method to assess heterogeneity of artistic practice in historical paintings

In the Renaissance and Early Modern period, paintings were largely produced by master painters who directed workshops of apprentices and others who often contributed to the piece. Discerning who created these masterworks and how they did so is a central question in technical art history and a nontrivial problem that machine learning can help solve by extending analysis to a microscopic scale. Analysis of workshop paintings presents a challenge, however, because information about the members of workshops and the processes by which artworks were created remains elusive. Thus, external examples are not available to train networks to recognize. Here, we present a novel machine learning approach we call pairwise assignment training for classifying heterogeneity (PATCH) that is capable of identifying individual artistic practice regimes with no external training data. We apply this method to two historical paintings by the Spanish Renaissance master, El Greco, and our findings regarding one of the works potentially challenge previous studies that assert that a considerable portion of the painting was completed by workshop members after El Greco’s death. PATCH outperforms statistical and unsupervised machine learning methods in this complex pairwise comparison problem lacking “ground truth” data, making it potentially useful across similar cases in the social and natural sciences, including image segmentation in remote sensing, urban development and design, and anomaly detection manufacturing contexts, among others.