Smart Culture Lens: An Application That Analyzes the Visual Elements of Ceramics

Abstract

The Smart Culture Lens developed in this study is an application developed by utilizing the visual element classification system of ceramic and AI technology. The user can analyze the visual elements of the ceramic photo taken with a smartphone and search for similar ceramics related to each visual element. For this development, as a first step, visual elements such as color, form, material, and pattern were defined as criteria for classifying ceramic appearance, and the visual element classification system of ceramics was organized. In the second step, 19,610 images of 7,346 ceramics were collected through museum visit photography and web, and a database was built by annotating these images with a visual element classification system. In the third step, representative object detection models, Faster R-CNN and Mask R-CNN were trained based on a ceramic classification system. Through those trained object detection models, visual elements and masks of the input image were recognized, and representative colors of the area were extracted using the k-means algorithm through the recognized masks. The performance of the trained object detection models (Average precision of Form / Material / Pattern 1st-level category = 0.87 / 0.89 / 0.72) shows that the amount of collected data and the established classification system are useful. Finally, by applying the above development results, a mobile application called 'Smart Culture Lens' was developed, and the usefulness of this application was confirmed through a user experience test. This study combines AI technology into cultural heritage so that people can intuitively explore artifacts from a new perspective, which differs from traditional artifact exploring methods. All the detailed processes of this development will be a guide to how to apply AI technology to the cultural heritage.