Knowledge Graphs vs. Large Language Models: Competitors or Partners in Supporting Virtual Museums

Virtual museums are factual means for the dissemination and documentation of Cultural Heritage (CH) content. They are suitable environments for the semantic annotation of artifacts and automatic virtual guides. To this end, we identify and compare Traditional (ontology-based), Large Language Model (LLM)-extended, and LLM-pure methods for the semantic information strategies of digital CH. The traditional method is described through an application prototype, while the methods that involve LLM are tested experimentally. To investigate the integral tasks related to LLMs, our experiments include (i) semantic annotation using the CIDOC Conceptual Reference Model (CRM) and Knowledge Graph (KG) generation with LLMs for a painting sample, and (ii) painting ranking relying solely on LLMs using catalog descriptions as input. The experiments demonstrate the potential of these methods to enhance artwork interpretation, description, and refinement of the results. Based on the relevant literature on traditional semantic annotation and conducted experiments with LLMs, a combination of ontologies and LLMs may provide an optimal approach, as it offers the accuracy of structured knowledge while providing a tool that interprets these elements into natural language and vice versa. Relying solely on LLMs may be risky due to the lack of domain-specific knowledge in the training data of LLMs, whereas traditional methods demand expertise in a specific domain and are more time-consuming. Our approach shows potential in use cases such as guiding museum visitors to artifacts that match their interests, assisting museum curators with documentation, or helping CH researchers identify similarities in artifact collections.