Merge pull request #78 from Imageomics/fine-tune-svm

Add Fine-tune SVM example notebook
Imageomics · Jan 7, 2025 · 34016d0 · 34016d0
2 parents 7bdab5d + f746e09
commit 34016d0
Show file tree

Hide file tree

Showing 2 changed files with 995 additions and 0 deletions.
diff --git a/docs/python-tutorial.md b/docs/python-tutorial.md
@@ -136,6 +136,13 @@ small 7.165559509303421e-05
 [GradCamExperiment.ipynb](https://github.com/Imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb)  applies GradCAM AI explainability to BioCLIP. <a target="_blank" href="https://colab.research.google.com/github/Imageomics/pybioclip/blob/main/examples/GradCamExperiment.ipynb">
   <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
 
+### Fine-tune with SVM
+[FineTuneSVM.ipynb](https://github.com/Imageomics/pybioclip/blob/main/examples/FineTuneSVM.ipynb) Fine-tunes  BioCLIP by combining an [SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC) with BioCLIP image embeddings. <a target="_blank" href="https://colab.research.google.com/github/Imageomics/pybioclip/blob/main/examples/FineTuneSVM.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
+As can be seen from comparing the confusion matrices, fine-tuning an SVM classifier on BioCLIP image embeddings may yield better results than using BioCLIP in "zero-shot mode", i.e., predicting on a list of custom labels.
+
+This work is based on code from [biobench](https://github.com/samuelstevens/biobench). 
+
 ## PIL Images
 The predict() functions used in all the examples above allow passing a list of paths or a list of [PIL Images](https://pillow.readthedocs.io/en/stable/reference/Image.html).
 When a list of PIL images is passed the index of the image will be filled in for `file_name`. This is because PIL images may not have an associated file name.