Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SAM with prior knowledge graph and customly set hyperparameters always returns "All solutions contain NaNs" assertion error. #134

Open
antsaukk opened this issue May 5, 2022 · 4 comments

Comments

@antsaukk
Copy link

antsaukk commented May 5, 2022

Hi,

I have done some hp optimization for SAM and it did yield an increase in performance. I wanted to go further and introduced prior knowledge graph (pkg later). However, when I am trying to run SAM with pkg and optimized hp, it always yields AssertionError: "All solutions contains NaNs" and AttributeError: "'Tensor' object has no attribute 'astype'". I have tried to experiment running SAM with different set of hp and pkg and find out that when hp "nruns=1" is set, the errors are arising. I have fed the pck as networkx.DiGraph and networkx.Graph objects -- in both cases it seems to yield the same assertion errors. What could be the reason for such behavior?

Best,
Anton

@diviyank
Copy link
Collaborator

Hello,
This issue often arises when the GAN does not converge, I think you may need to tweak the parameters of the GAN to make the optimization more smooth. what are the

Thing is, we are utilizing multiprocessing to optimize the compute-time so the issues raises only when n_jobs=1 or n_runs=1.

The Attribute error is concerning however, do you have the full error trace of that error ?

Thanks for the help,
Best,
Diviyan

@Christopher7622
Copy link

hello Diviyan:
When I was inputting parameters, I also encountered this type of problem. The following is my running code and specific error information.
dd60a73790425176bad0133d4b4fa5e
01cdde9b5bb27b19e6ba18a452b83a8
Looking forward to your reply very much!
Best
Christopher

@diviyank
Copy link
Collaborator

Hello @Christopher7622 , I think your problem arises with package versions, could you try using the docker image to execute the code?

Best,
Diviyan

@Christopher7622
Copy link

Hello @Christopher7622 , I think your problem arises with package versions, could you try using the docker image to execute the code?

Best, Diviyan

thanks a lot,An hour ago, I ran the program again according to the example provided in the official document. Although an error occurred, the program continued to run on the server. What is the reason for this?
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants