This repository contains the instrument to scraping data by crawlers and add info by AI module. For each of theese component the code is available in the respective parts:
The AI Module is the core of the framework for automatically process and classify works of art images with the aim of obtaining useful information about the depicted artwork. The framework consists of two stages. Firstly the artwork depicted in the image is classified among 5 classes: icons, frescoes, coins, manuscripts, and others.
After the image has been classified at the first stage, it goes to the stage classification models (specific to the type of artwork) to extract specific features (e.g., location, period, etc.).
In order to classify images, we use the neural network VGG16. VGG16 is a convolutional neural network for image classification task.
The following image datasets were collected and labelled as previously defined. For each artwork, the images were labelled according to a feature/field of each works of art (e.g., location, periods, material, etc.). Different classes belong to each field.
The following tables reports the data cardinality (number of images), for each class, used for the training and test of the models. The results are for both the phases defined.
The classification task is performed to classify the goods and the fields of each artworks. In each experiment:
- The dataset was split between training and test set: 80% and 20%, respectively. The data were taken to maintain class balance in both datasets.
- A model was trained from the pre-trained model on imagenet (fine tuning). The adopted architecture is vgg16, adaptive optimizer (Adam), learning rate 10e-5, mini-batch size 32, 50 training epochs.
Experiments were performed using 2 GPUs in parallel in the training phase. Only one GPU was used in the testing phase.
Hardware and Software specifications:
- 2 Nvidia GeForce RTX 2080 Ti
- CUDA Version: 11.2
- libcudnn.so.7
- Pytorch 1.8.1
The following Tables reports the results of all models.
