Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Allow patches applied to ModelPatcher to be implemented in a general form as functions. #3602

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ltdrdata
Copy link
Collaborator

@ltdrdata ltdrdata commented May 29, 2024

Allows this style model patching.

        new_model = model.clone()
        seed_offset = 0
        for key in model.model_keys:
            if key.find(target) < 0:
                continue

            def patch_function(p, weight, key):
                tp = weight.dtype
                gen = torch.manual_seed(tilt_seed + seed_offset)
                noise = torch.rand(size=weight.size(), dtype=tp, layout=weight.layout, generator=gen, device='cpu')
                noise = (noise.to(weight.device) - 0.5) * p[0] * 2

                gen = torch.manual_seed(mask_seed + seed_offset)
                mask_noise = torch.rand(size=weight.size(), dtype=tp, layout=weight.layout, generator=gen, device='cpu')
                noise *= ((mask_strength - mask_noise.to(weight.device)) * 1000000).clamp(0, 1)

                return weight + noise

            new_model.add_patches({
                key: ("patch_function", patch_function)
            }, strength_model=1.0, strength_patch=tilt_strength)
            seed_offset += 1

Eliminate the need for code patches like this:
https://github.com/icefairy64/comfyui-model-tilt/blob/main/model_patcher_add_tilt.patch

@ltdrdata ltdrdata changed the title Allow patches applied to ModelPatcher to be implemented in a general form as functions. feat: Allow patches applied to ModelPatcher to be implemented in a general form as functions. May 29, 2024
@shawnington
Copy link
Contributor

shawnington commented May 29, 2024

Oh man, calculate_weight could use a re-factor, beautiful beautiful spaghetti, but this is a good idea.

@mcmonkey4eva mcmonkey4eva added the Feature A new feature to add to ComfyUI. label Jun 28, 2024
@mcmonkey4eva mcmonkey4eva added the Good PR This PR looks good to go, it needs comfy's final review. label Jun 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature A new feature to add to ComfyUI. Good PR This PR looks good to go, it needs comfy's final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants