A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun
-
Updated
Feb 6, 2022 - Python
#
multimodality
Here are 140 public repositories matching this topic...
The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.
ai gcc multimodality vlm cradle computer-control lmm grounding ai-agent large-language-models llm generative-ai vision-language-model ai-agents-framework general-computer-control personoid foundation-agent
-
Updated
Nov 7, 2024 - Python
Collecting awesome papers of RAG for AIGC. We propose a taxonomy of RAG foundations, enhancements, and applications in paper "Retrieval-Augmented Generation for AI-Generated Content: A Survey".
-
Updated
Aug 20, 2024
A Comparative Framework for Multimodal Recommender Systems
collaborative-filtering matrix-factorization recommendation-system recommendation-engine recommender-system recommendation-algorithms multimodality multimodal-learning
-
Updated
Sep 14, 2024 - Python
An official implementation for "CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval"
search retrieval ranking clip multimodality multimodal-learning multimodal activitynet retrieval-model msvd msrvtt video-text-retrieval lsmdc didemo video-clip-retrieval
-
Updated
Apr 12, 2024 - Python
[TPAMI 2023] Multimodal Image Synthesis and Editing: The Generative AI Era
-
Updated
Nov 21, 2023 - TeX
Automated modeling and machine learning framework FEDOT
machine-learning automation genetic-programming hyperparameter-optimization evolutionary-algorithms multimodality automl automated-machine-learning parameter-tuning structural-learning fedot
-
Updated
Dec 17, 2024 - Python
✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models. The first work to correct hallucinations in MLLMs.
multimodality hallucination hallucinations large-language-models llm mllm multimodal-large-language-models
-
Updated
Jun 17, 2024 - Python
A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.
chatbot multimodality multimodal vision-language-model multimodal-large-language-models vision-language-learning qwen llama3
-
Updated
Nov 26, 2024 - Python
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest
-
Updated
Jun 11, 2024 - Python
A CLI tool/python module for generating images from text using guided diffusion and CLIP from OpenAI.
deep-learning artificial-intelligence openai image-generation multimodality text-to-image diffusion multimodal text-to-image-synthesis openai-clip
-
Updated
Feb 8, 2022 - Python
X-VLM: Multi-Grained Vision Language Pre-Training (ICML 2022)
-
Updated
Nov 25, 2022 - Python
A knowledge base construction engine for richly formatted data
-
Updated
Jun 23, 2021 - Python
LLM2CLIP makes SOTA pretrained CLIP model more SOTA ever.
-
Updated
Dec 13, 2024 - Python
Sequence-to-Sequence Framework in PyTorch
deep-learning cnn pytorch speech-recognition seq2seq neural-machine-translation nmt multimodality asr
-
Updated
Jan 5, 2023 - Jupyter Notebook
🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).
text-to-speech multimodality text-to-image text-to-audio text-to-video text-to-music multimodal-models aigc large-language-models llm text-to-3d multimodal-generation mllm text-to-sound large-vision-language-models multimodal-large-language-models lvlm
-
Updated
Nov 14, 2024 - HTML
This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"
machine-learning natural-language-processing deep-neural-networks computer-vision deep-learning evaluation question-answering stem multimodality multimodal-learning visual-question-answering multimodal multimodal-deep-learning foundation-models large-language-models llm llms large-multimodal-models
-
Updated
Dec 10, 2024 - Python
An open source implementation of "Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning", an all-new multi modal AI that uses just a decoder to generate both text and images
attention multimodality attention-is-all-you-need multimodal-learning multimodal imagegeneration dalle
-
Updated
Dec 15, 2023 - Python
DANCE: a deep learning library and benchmark platform for single-cell analysis
python data-science benchmark machine-learning bioinformatics deep-learning computational-biology dance single-cell multimodality single-cell-rna-seq graph-neural-networks spatial-transcriptomics single-cell-rna-sequencing
-
Updated
Dec 17, 2024 - Python
An official implementation for " UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation"
video localization caption alignment segmentation coin multimodality joint multimodal-sentiment-analysis pretrain pretraining msrvtt video-text-retrieval video-text video-language youcookii retrieval-task caption-task
-
Updated
Jul 25, 2024 - Python
Improve this page
Add a description, image, and links to the multimodality topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the multimodality topic, visit your repo's landing page and select "manage topics."