GenerativeModels

[rirase]

Hello everyone,

thank you for this interesting project. I have a question.

Currently, I am working on the reconstruction of the humerus from a CT. In several scans, the humerus is not full. I have worked on a statistical model that completes the missing part of the bone. i have the followin quiestion:

can I achieve the same using generative AI?
I have full humeurs scans that I can intentionally remove some images to simulate a missing part of the bone, and a AI model would estimate the missing part?

If yes, my question is:

I have would an input of a number of images (each image is 512 * 512 pixels), but the output of the model would be the same input plus the images generated by the AI? (For example, if I have a case of 200 images but cut it to 150images), the input of the model would be 150, but the ground truth would be 200. How can I manage this difference between input and output?

Can someone guide me on what model I could use?
Is there a tutorial in MONAI that could serve as a reference?

Thanks to all for your help

RIC

[marksgraham]

Hi,

You could pose this as an inpainting problem, where you train on scans of the full humerus and then do inference on partial images of the humerus with a mask outlining the part you want to extend.

Here is a tutorial showing you how to do inpainting.

[rirase]

Hello,
Thank you for your help. Yes, I am working in 3D ( it should be fine to convert everything to 3D ),
But the problem I'm having is that in the example model, the missing part is inside the image. In my problem, the missing part is outside the FOV (field of view) and it's in 3D.
So, I have images of size 512*512, which means if the missing part is lateral, I would need larger images than the input, and if it's along the z-axis, it means there are missing full images ( slices).
Could you help me by guiding me on how to progress with this problem?

[marksgraham]

we don't have any tutorials for doing outpainting but here is some info https://replicate.com/guides/stable-diffusion/outpainting