This project represents our contribution to the Data Challenges at Goethe University Frankfurt. The module is overseen by Dr. Karsten Tolle and Sebastian Gampe.
The challenge revolves around the classification of historical coins as part of the Corpus Nummorum Project: https://www.corpus-nummorum.eu/
This repository comprises three distinct approaches for coin classification. Each approach has been implemented using Jupyter notebooks. For in-depth information, please consult the specific directories and the main notebook.
The first approach assesses various pretrained PyTorch architectures and experiments with different augmentation pipelines.
The second approach applies OpenAI's CLIP model in a zero-shot manner, making it a truly versatile and adaptable strategy.
The third approach harnesses the descriptive information associated with the coins, implementing a straightforward fusion technique to combine visual and textual data modalities.
By exploring these distinct methodologies, we hope to provide a comprehensive analysis of coin classification, paving the way for future advancements in the field.
The final validation/test results can be found in a jupyter notebook in all results.
All trained models from the all result notebook can be found here: https://hessenbox-a10.rz.uni-frankfurt.de/getlink/fiWTBGKi44LLdUj141zK5T/
Incase installing requirments.txt does not work try to use this command for pytorch and cuda.
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
Also make sure that your GPU has cuda drivers installed that work with cuda 11.8
