demo/DemoDiffusion could not work: polygraphy.exception.exception.PolygraphyException: Could not deserialize engine.

[artetaout]

• configs
• 8 * NVIDIA A100-SXM4-80GB
• Driver Version: 470.161.03
• completely follow steps from https://github.com/NVIDIA/TensorRT/tree/release/8.6/demo/Diffusion
• the logs

/workspace/demo/Diffusion# python3 demo_txt2img.py "a beautiful photograph of Mt. Fuji during cherry blossom" --hf-token=$HF_TOKEN -v
[I] Initializing StableDiffusion txt2img demo using TensorRT
Downloading (…)tokenizer/vocab.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.06M/1.06M [00:00<00:00, 1.07MB/s]
Downloading (…)tokenizer/merges.txt: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 525k/525k [00:00<00:00, 700kB/s]
Downloading (…)cial_tokens_map.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 460/460 [00:00<00:00, 412kB/s]
Downloading (…)okenizer_config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 824/824 [00:00<00:00, 798kB/s]
Building TensorRT engine for onnx/vae.opt.onnx: engine/vae.plan
[W] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[I]     Configuring with profiles: [Profile().add('latent', min=(1, 4, 64, 64), opt=(1, 4, 64, 64), max=(16, 4, 64, 64))]
[I] Building engine with configuration:
    Flags                  | [FP16]
    Engine Capability      | EngineCapability.DEFAULT
    Memory Pools           | [WORKSPACE: 76738.19 MiB, TACTIC_DRAM: 81251.19 MiB]
    Tactic Sources         | []
    Profiling Verbosity    | ProfilingVerbosity.DETAILED
    Preview Features       | [DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805]
[W] kFASTER_DYNAMIC_SHAPES_0805 preview feature is disabled.
[W] Myelin graph with multiple dynamic values may have poor performance if they differ.
[W] TensorRT encountered issues when converting weights between types and that could affect accuracy.
[W] If this is not the desired behavior, please modify the weights or retrain with regularization to adjust the magnitude of the weights.
[W] Check verbose logs for the list of affected weights.
[W] - 5 weights are affected by this issue: Detected subnormal FP16 values.
[I] Finished engine building in 1056.613 seconds
[I] Saving engine to engine/vae.plan
Loading TensorRT engine: engine/clip.plan
[I] Loading bytes from engine/clip.plan
Loading TensorRT engine: engine/unet.plan
[I] Loading bytes from engine/unet.plan
[E] 1: [runtime.cpp::parsePlan::315] Error Code 1: Serialization (Serialization assertion plan->header.size == blobSize failed.)
[!] Could not deserialize engine. See log for details.
Traceback (most recent call last):
  File "demo_txt2img.py", line 76, in <module>
    demo.loadEngines(args.engine_dir, args.onnx_dir, args.onnx_opset,
  File "/workspace/demo/Diffusion/stable_diffusion_pipeline.py", line 306, in loadEngines
    engine.load()
  File "/workspace/demo/Diffusion/utilities.py", line 220, in load
    self.engine = engine_from_bytes(bytes_from_path(self.engine_path))
  File "<string>", line 3, in engine_from_bytes
  File "/usr/local/lib/python3.8/dist-packages/polygraphy/backend/base/loader.py", line 42, in __call__
    return self.call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/polygraphy/backend/trt/loader.py", line 568, in call_impl
    G_LOGGER.critical("Could not deserialize engine. See log for details.")
  File "/usr/local/lib/python3.8/dist-packages/polygraphy/logger/logger.py", line 597, in critical
    raise PolygraphyException(message) from None
polygraphy.exception.exception.PolygraphyException: Could not deserialize engine. See log for details.

[jettan]

It kind of looks like it built the clip and vae engines, but not the unet one... can you try to see that unet.plan has been built?

[zerollzeng]

[E] 1: [runtime.cpp::parsePlan::315] Error Code 1: Serialization (Serialization assertion plan->header.size == blobSize failed.)

Here is why it fails.

[zerollzeng]

I don't think I saw this error before, can you try load the engine with trtexec to see if it work?

[ttyio]

Closing since no activity for more than 3 weeks, pls reopen if you still have question, thanks!

[demuxin]

I had a similar problem and the reason was that the disk was full.

You can refer to it.

[artetaout]

sry for too late response, it's same as what you met
wish a good day!

I had a similar problem and the reason was that the disk was full.

You can refer to it.

Closing since no activity for more than 3 weeks, pls reopen if you still have question, thanks!