Question of Demo or Inference in the app.py

[HaoqianSong]

Hello, I have some doubts.

[HaoqianSong]

Can app.py only run on one GPU, not multiple GPUs? In addition, because the graphics card is too small, I modified the following two places to load the model for the CPU, and some problem occurred.
defload_model_from_config(config, ckpt, device, verbose=False):
global closure_device
closure_device = device
print(f"Loading model from {ckpt}")
pl_sd = torch.load(ckpt, map_location=device) #"cpu"
if "global_step" in pl_sd:
print(f"Global Step: {pl_sd['global_step']}")
sd = pl_sd["state_dict"]
model = instantiate_from_config(config.model)
m, u = model.load_state_dict(sd, strict=False)
if len(m) > 0 and verbose:
print("missing keys:")
print(m)
if len(u) > 0 and verbose:
print("unexpected keys:")
print(u)
model.to(device) #"cpu"
model.eval()
return model

[egeozguroglu]

Hi, the current Gradio demo script is for a single GPU, but it can easily be adapted to leverage parallelization across multiple smaller GPUs.

[Kris0823]

可演示脚本可以用多个GPU进行运行吗