Model compatiblity

[fabiannagel]

Hi! I noticed that you are neither relying on Stanford ColBERT, nor on RAGatouille in your dependencies. What models are compatible with pylate- only those which have also been created using your library?

[NohTow]

Hello,

Indeed, PyLate is a standalone library that use sentence-transformers (and thus transformers) for the modeling, so we do not use Stanford nor RAGatouille!

Right now, the models usable are the ones trained using the library, as well as ColBERT-v2 and ColBERT-small from AnswerAI.

For starter, I'll share a conversion script soon so people can translate their favorite models. Did you have any model in mind so I can help?
Also, I will work on being able to load the stanford-nlp weights on the fly and cast it to a PyLate model. It'll be better for cross-compatibility than to store both version on the same repository as done for colbert-small.

[fabiannagel]

Makes sense! But how come answerdotai/answerai-colbert-small-v1 is usable? I cannot see any reference to pylate on their model page but they do say it works with Stanford ColBERT or RAGatouille.

edit: I'm trying to get up and running in German which is unfortunately a bit difficult. I came across these models:

AdrienB134/ColBERTv1.0-german-mmarcoDE: Only ColBERTv1, but seems ready to go.
domci/ColBERTv2-mmarco-de-0.1: ColBERTv2, but not fully trained.
antoinelouis/colbert-xm: Multilingual ColBERTv2

[NohTow]

It comes from the fact that I know Ben and I worked with him to add the weights for PyLate on the repository!
However, this approach come at the cost of having to modify repositories, duplicate weights and actually created some issues.
Thus, I reworked the loading logic in #52 and you should now be able to load any existing stanford-nlp model!

I am still making some adjustments to handle more models (e.g, the very recent jina-colbert-v2) but it's already usable and should be even better very soon!

[fabiannagel]

So by "stanford-nlp model", you mean the reference implementation?

I guess this warning refers to the loading logic you mentioned. Is there any way to store the converted model on disk in order to save some initialization time?

sentence_transformers.SentenceTransformer          Load pretrained SentenceTransformer: antoinelouis/colbert-xm
WARNING pylate.models.colbert                      No sentence-transformers model found with name antoinelouis/colbert-xm. Creating a ColBERT model from base encoder.

Apart from that, kudos for this library. I got a small retriever running already :)

[NohTow]

Yes indeed, the models built using this repository (and also RAGatouille ones as it is using the lib as backend).

I am not 100% sure the model from Antoine Louis will be loaded correctly as it has been trained with its own codebase (and idk how much the modeling is compatible).

You can save the model locally by using model.save_pretrained(path). You can then load it by indicating the path. You can also upload it to the hub to be able to load it remotely!

My last PR (#54) allow to load the recent jina-colbert-v2 model which is an amazing multilingual ColBERT. You could use this one once the PR is merged.

[fabiannagel]

That's good to know. I haven't checked the results, but so far everything works without errors. jina-colbert-v2 looks perfect but unfortunately I cannot use it due to its license prohibiting commercial usage.

How much work would it be to make sure ColBERT-XM works properly with pylate? Can I help you with that somehow?
I looked at various libraries for ColBERT-based retrieval and so far pylate is the most mature in terms of stability & ease of use. So I'd love to keep it but I really rely on German or at least some multilingual capabilities.

[NohTow]

I honestly have no idea as I did not dig much into ColBERT-XM code.
I might have a look at some point but we have a lot of things planned and the support of ColBERT-XM might not be a priority right now.
Maybe @ant-louis will now better how painful it could be.

[ant-louis]

Hey, ColBERT-XM was trained with the original ColBERT library from stanford-nlp , so it should be compatible with PyLate natively too.

However, you need to manually activate the modular adapter layers for the language of interest and, if training, freeze the adapters and the embedding matrix. After a quick look to @NohTow's code, here is how I would do it:

from pylate import models

model = models.ColBERT(model_name_or_path="antoinelouis/colbert-xm")
language = "fr_XX" # Use a code from https://huggingface.co/facebook/xmod-base#languages

backbone = model[0].auto_model
if backbone.__class__.__name__.lower().startswith("xmod"):
    backbone.set_default_language(language)
    if do_train:
        backbone.freeze_embeddings_and_language_adapters()

I hope it helps!

[NohTow]

Thanks a lot for taking the time to have a look @ant-louis, really appreciate it!

[fabiannagel]

Thanks a lot @ant-louis. I took the liberty of moving the model over to Huggingface. It's configured for German with instructions for other languages: https://huggingface.co/fabiannagel/pylate-colbert-xm-german

Should be ready to go now for pylate.

[NohTow]

Very cool!
Did you assert the model is working correctly?

[fabiannagel]

I just manually inspected my retrieval results and they look fine, also in comparison with a BM25 baseline.

[fabiannagel]

Alright, I understand. I guess the following warning indicates that the conversion did not work properly...?

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

[NohTow]

Not really, it just means that the query/document prefixes have been added to the vocabulary.
It can mean that the prefixes are not correct or that the base model did not actually add them to the tokenizer.
In any case, it does not really affect the performance much.

Which model is it?

[fabiannagel]

Still antoinelouis/colbert-xm after loading from Huggingface & converting using model.save_pretrained(path).