Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cut release #397

Merged
merged 2 commits into from
Nov 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 63 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@
[![Python Checks](https://github.com/brycedrennan/imaginAIry/actions/workflows/ci.yaml/badge.svg)](https://github.com/brycedrennan/imaginAIry/actions/workflows/ci.yaml)
[![Discord](https://flat.badgen.net/discord/members/FdD7ut3YjW)](https://discord.gg/FdD7ut3YjW)

AI imagined images. Pythonic generation of stable diffusion images.
AI imagined images. Pythonic generation of stable diffusion images **and videos** *!.

"just works" on Linux and macOS(M1) (and maybe windows?).
"just works" on Linux and macOS(M1) (and sometimes windows).


```bash
Expand All @@ -20,7 +20,47 @@ AI imagined images. Pythonic generation of stable diffusion images.
>> imagine "a scenic landscape" "a photo of a dog" "photo of a fruit bowl" "portrait photo of a freckled woman" "a bluejay"
# Make an animation showing the generation process
>> imagine --gif "a flower"
# Make an AI video
>> aimg videogen --start-image rocket.png
```
## Stable Video Diffusion
<p float="left">
<img src="assets/svd-rocket.gif" height="256">
<img src="assets/svd-athens.gif" height="256">
<img src="assets/svd-pearl-girl.gif" height="256">
<img src="assets/svd-starry-night.gif" height="256">
<img src="assets/svd-dog.gif" height="256">
<img src="assets/svd-xpbliss.gif" height="256">
</p>

### Rushed release of Stable Diffusion Video!
Works with Nvidia 3090 or 4090. Does not work on Mac or CPU. Other GPUs not tested.
```text
Usage: aimg videogen [OPTIONS]

AI generate a video from an image

Example:

aimg videogen --start-image assets/rocket-wide.png

Options:
--start-image TEXT Input path for image file.
--num-frames INTEGER Number of frames.
--num-steps INTEGER Number of steps.
--model TEXT Model to use. One of: svd, svd_xt, svd_image_decoder, svd_xt_image_decoder
--fps INTEGER FPS for the AI to target when generating video
--output-fps INTEGER FPS for the output video
--motion-amount INTEGER How much motion to generate. value between 0 and 255.
-r, --repeats INTEGER How many times to repeat the renders. [default: 1]
--cond-aug FLOAT Conditional augmentation.
--seed INTEGER Seed for random number generator.
--decoding_t INTEGER Number of frames decoded at a time.
--output_folder TEXT Output folder.
--help Show this message and exit.
```

### Images
<p float="left">
<img src="assets/000019_786355545_PLMS50_PS7.5_a_scenic_landscape.jpg" height="256">
<img src="assets/000032_337692011_PLMS40_PS7.5_a_photo_of_a_dog.jpg" height="256">
Expand All @@ -30,6 +70,19 @@ AI imagined images. Pythonic generation of stable diffusion images.
<img src="assets/009719_942389026_kdpmpp2m15_PS7.5_a_flower.gif" height="256">
</p>

### Whats New
**14.0.0**
- 🎉 video generation using [Stable Video Diffusion](https://github.com/Stability-AI/generative-models)
- 🎉 imaginairy is now backed by the [refiners library](https://github.com/finegrain-ai/refiners)
- This was a huge re-write which is why some features are not yet supported. On the plus side, refiners supports
cutting edge features (SDXL, image prompts, etc) which will be added to imaginairy soon.
- [self-attention guidance](https://github.com/SusungHong/Self-Attention-Guidance) which makes details of images more accurate
- support for pytorch 2.0
- deprecated: support for python 3.8
- deprecated: support for torch 1.13
- deprecated: support for sd 1.4, 2.0, 2.1
- broken: pix2pix editing, most samplers, some controlnets, and memory management

### Run API server and StableStudio web interface (alpha)
Generate images via API or web interface. Much smaller featureset compared to the command line tool.
```bash
Expand Down Expand Up @@ -493,15 +546,16 @@ A: The AI models are cached in `~/.cache/` (or `HUGGINGFACE_HUB_CACHE`). To dele
## ChangeLog

**14.0.0**
- 🎉 feature: uses refiners library for generation
- 🎉 feature: adds support for pytorch 2.0
- fix: stable diffusion 1.5 + inpainting working
- fix: self-attention guidance working. improves image generation quality
- fix: tile-mode working
- fix: self-attention guidance working. improves image generation quality
- 🎉 feature: video generation using [Stable Video Diffusion](https://github.com/Stability-AI/generative-models)
- 🎉 feature: imaginairy is now backed by the [refiners library](https://github.com/finegrain-ai/refiners)
- This was a huge re-write which is why some features are not yet supported. On the plus side, refiners supports
cutting edge features (SDXL, image prompts, etc) which will be added to imaginairy soon.
- feature: added support for [self-attention guidance](https://github.com/SusungHong/Self-Attention-Guidance) which makes details of images more accurate
- feature: adds support for pytorch 2.0
- deprecated: support for python 3.8
- deprecated: support for torch 1.13
- deprecated: support for sd 1.4, 2.0, 2.1
- broken: pix2pix editing, most samplers, most controlnets, and memory management
- broken: pix2pix editing, most samplers, some controlnets, and memory management

**13.2.1**
- fix: pydantic models for http server working now. Fixes #380
Expand Down
Binary file added assets/svd-athens.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/svd-dog.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/svd-pearl-girl.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/svd-rocket.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/svd-starry-night.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/svd-xpbliss.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 0 additions & 3 deletions imaginairy/cli/videogen.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@
@click.option(
"--decoding_t", default=1, type=int, help="Number of frames decoded at a time."
)
@click.option("--device", default=None, help="Device to use.")
@click.option("--output_folder", default=None, help="Output folder.")
def videogen_cmd(
start_image,
Expand All @@ -57,7 +56,6 @@ def videogen_cmd(
cond_aug,
seed,
decoding_t,
device,
output_folder,
):
"""
Expand Down Expand Up @@ -87,6 +85,5 @@ def videogen_cmd(
cond_aug=cond_aug,
seed=seed,
decoding_t=decoding_t,
device=device,
output_folder=output_folder,
)
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def get_git_revision_hash() -> str:
name="imaginAIry",
author="Bryce Drennan",
# author_email="b r y p y d o t io",
version="13.2.1",
version="14.0.0b1",
description="AI imagined images. Pythonic generation of images.",
long_description=readme,
long_description_content_type="text/markdown",
Expand Down
Loading