What is the meaning of training?

[shizhediao]

Hi,
I am new to this amazing package. i am looking into this tutorial https://maartengr.github.io/BERTopic/getting_started/vectorizers/vectorizers.html.

It mentioned:

Before starting, it should be noted that you can pass the CountVectorizer before and after training your topic model

I was wondering what does the training mean? Is there any neural model who has parameters to be tuned inside? Or there is some hyper-parameters which would be tuned during the process?

Thanks!

[MaartenGr]

Training is the process of training the underlying models as mentioned in the procedure here. I would advise looking through that link to see each step of BERTopic's underlying procedure. All of those models (except for the fine-tuning lego block) are trained.

[shizhediao]

Thanks!
I read this great tutorial and have several questions. There are six blocks / steps, and let's take a look step by step:

Step 1 - Extract embeddings
embedding_model = SentenceTransformer("all-MiniLM-L6-v2")

Does the embedding model change according to my corpus? In my understanding, it will be fixed and the model weights will not be changed. Please correct me if I am wrong.

Step 2 - Reduce dimensionality
umap_model = UMAP(n_neighbors=15, n_components=5, min_dist=0.0, metric='cosine')

Step 3 - Cluster reduced embeddings
hdbscan_model = HDBSCAN(min_cluster_size=15, metric='euclidean', cluster_selection_method='eom', prediction_data=True)

For the clustering step, does the training mean adjusting the hyper-parameters according to my corpus inside the clustering algorithm?

Step 4 - Tokenize topics
vectorizer_model = CountVectorizer(stop_words="english")
This step is just a definition right? I thought this does not include the training.

Step 5 - Create topic representation
ctfidf_model = ClassTfidfTransformer()

I thought this step only does calculating the c-tf-idf, are there any other parameters? Or do you mean calculating the c-tf-idf could be deemed as training?

Step 6 - (Optional) Fine-tune topic representations with
a bertopic.representation model
representation_model = KeyBERTInspired()

Does that mean they KeyBERT model will be fixed and used to extract the topic words?

Thank you so much!

[MaartenGr]

Does the embedding model change according to my corpus? In my understanding, it will be fixed and the model weights will not be changed. Please correct me if I am wrong.

That's correct, the model will not chance.

For the clustering step, does the training mean adjusting the hyper-parameters according to my corpus inside the clustering algorithm?

Adjusting hyper-parameters is called hyperparameter-tuning. Training means that UMAP first learns to reduce the dimensionality of the embeddings based on the available data. Then, HDBSCAN learns which clusters are there as a result of it's underlying algorithm. Both models are trained and will have their own set of parameters as a result of this training process (not to be confused with hyperparameters).

I thought this step only does calculating the c-tf-idf, are there any other parameters? Or do you mean calculating the c-tf-idf could be deemed as training?

Yes, calculating c-TF-IDF is definitely deemed as training, the same way you train a LogisticRegression albeit more simple. Just because a procedure is more minimal than others, it does not mean it doesn't involve training.

Does that mean they KeyBERT model will be fixed and used to extract the topic words?

Yes, since there is nothing learned when running this model, merely extracted.

[shizhediao]

Hi @MaartenGr
Thank you so much for your explanation! It is quite clear. Thank you for your time!