-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inference results didn't perform as expected #25
Comments
Sorry for the late response, it seems that you are trying to generate with multiple reference characters, which is a more challenging problem. In the supplementary of our paper, we provide an example of multi-character generation, but it is slightly less stable than single-character generation because we have not trained specifically for this situation. |
Hi Haoning,
I was trying inference.py, annd the results seem to be underperforming. I was wondering if you could help me check if I did everything correctly. I downloaded checkpoint_Story from https://huggingface.co/haoningwu/StoryGen/tree/main, and downloaded all requirements from environmenbt.yaml with some problems with packages that required c++ (libtorch_cuda_cu.so and xformers). Then I ran with these parameters:
![image](https://private-user-images.githubusercontent.com/77973600/335331407-b95948b1-2622-4541-85cd-57b4e0ea87a5.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQxMDM2MjgsIm5iZiI6MTcyNDEwMzMyOCwicGF0aCI6Ii83Nzk3MzYwMC8zMzUzMzE0MDctYjk1OTQ4YjEtMjYyMi00NTQxLTg1Y2QtNTdiNGUwZWE4N2E1LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDIxMzUyOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWE2NjM3OGJkMzYyZDU0NTQyNTE3MGZkOTgyNmQ0OWZhY2MyOWQwN2NmNzhkYTY1ZTY4YmU0ZWJiZWQ1YWVhYWEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.5Ybte-fMhZ70ERxmHkm43eLVPVpOn9lAlDo3SKAtnWY)
The results looked somewhat like this:
![image](https://private-user-images.githubusercontent.com/77973600/335333811-682b6795-6ca8-430e-9300-c2b992622fbb.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQxMDM2MjgsIm5iZiI6MTcyNDEwMzMyOCwicGF0aCI6Ii83Nzk3MzYwMC8zMzUzMzM4MTEtNjgyYjY3OTUtNmNhOC00MzBlLTkzMDAtYzJiOTkyNjIyZmJiLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDIxMzUyOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTkwODM5M2U5ZWUyNmVhZmY1MWY1ZTRkNDdlYjUxZTI2Y2FlZmNjMmM1NzVhMGUzNDYxYmMyNjU0N2IxNjVmYzImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.9YWGJZbHTxojMBD_0vmZxbFyTpKiME72I6dKINXzJg4)
The boy and the dog are all very different from the images I provided. Could you give more guide in terms of how to use inference.py, difference between "multi-image-condition" and "auto-regressive", and if I used the inputs correctly? Or these degradation are results from the c++ packages? I feel that's less likely though, since they appear to be packages for efficiency.
Thanks a lot!
The text was updated successfully, but these errors were encountered: