-
-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor Optimizer Config API #45
Labels
Comments
The reason why Config classes were originally set was because I expected all the optimizers to have a lot of "features" or options to tune in the future which makes the constructors really long. Imagine the complexity if each new optimizer has to mention 30+ arguments in their init signature. Sounds tough! |
This issue is related to #62 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently every optimizer comes with a
config
specifically for that optimizer that manages the hyperparameters for the optimizer.This is made because of the following reasons:
defaults
that are useful.This diverges from the PyTorch-esque way optimizers are where the classes need to be provided with full information on all the parameters.
This issue is to find a decent middle ground between the two -> Allow for immediate tiny changes in the Hyperparameters on the spot, with the object, while also allowing for inheritable parameters.
This adds a significant overhead and makes things a bit more complex, so it should ideally be abstracted out in the BaseConfig and BaseOptimizer, with all the utility functions that transform from one format to the other.
The text was updated successfully, but these errors were encountered: