poster.md

Demo: CLIP Research Poster

A demo of CLIP research paper using Lightning App

OpenAI

@OpenAI

--split--

Natural Language based Image Search

OpenAI introduced a neural network called CLIP which efficiently learns visual concepts from natural language supervision.

This app is a demo of Lightning Research Template app which allows authors to build an app to share their everything related to their work at a single place. Explore the tabs at the top of this app to view blog, paper, training logs and model demo.

You can fork this app and edit to customize according to your need.

Kudos to Soumik Rakshit and Manan Goel for their awesome repository clip-lightning

Thanks to Vivien for his inspiring application using CLIP Minimal user-friendly demo of OpenAI's CLIP for semantic image search.

CLIP pre-trains an image encoder and a text encoder to predict which images were paired with which texts in our dataset. We then use this behavior to turn CLIP into a zero-shot classifier. We convert all of a dataset's classes into captions such as "a photo of a dog" and predict the class of the caption CLIP estimates best pairs with a given image.

--split--

Lightning Apps

Lightning Apps can be built for any AI use case, including AI research, fault-tolerant production-ready pipelines, and everything in between.

!!! abstract "Key Features"

- **Easy to use-** Lightning apps follow the Lightning philosophy- easy to read, modular, intuitive, pythonic and highly composable interface that allows you to focus on what's important for you, and automate the rest.
- **Easy to scale**- Lightning provides a common experience locally and in the cloud. The Lightning.ai cloud platform abstracts the infrastructure, so you can run your apps at any scale. The modular and composable framework allows for simpler testing and debugging.
- **Leverage the power of the community-** Lightning.ai offers a variety of apps for any use case you can use as is or build upon. By following the best MLOps practices provided through the apps and documentation you can deploy state-of-the-art ML applications in days, not months.

graph LR
    A[local ML]
    A --> B{<b>Lightning Apps</b><br>Effortless GPU distributed compute}
    B -->|Frontend| C[Lightning Work 1]
    B -->|DB integration| D[Lightning Work 2]
    B -->|User auth| E[Lightning Work 3]

Loading

Available at : Lightning-AI/lightning-template-research-app/app.py

import lightning as L

paper = "https://arxiv.org/pdf/2103.00020.pdf"
blog = "https://openai.com/blog/clip/"
github = "https://github.com/soumik12345/clip-lightning/tree/AddModelCheckpoint"
wandb = "https://wandb.ai/manan-goel/clip-lightning-image_retrieval/runs/1cedtohj"

app = L.LightningApp(
    ResearchApp(
        resource_path="resources",
        paper=paper,
        blog=blog,
        training_log_url=wandb,
        github=github,
        notebook_path="resources/Interacting_with_CLIP.ipynb",
        launch_gradio=True,
    )
)

Citation

@article{YourName,
  title={Your Title},
  author={Your team},
  journal={Location},
  year={Year}
}