Assert "valid" values for hyperparams in the Config

[bhavnicksm]

The configs do not have precise checks for the values that are passed to them, which allows for misunderstandings or illegal hyperparameter settings.

For example

config = AdamConfig(beta_1=-1) #this should not be possible

The duty of the config itself is to manage the class parameters well, store them in one location, handle inheritance properly without tedious calls, and do proper checks.

Data-classes in Python are great for handling the parameters, basic type checking, storing and exporting in different ways, but the "valid" value checking is not available behaviour by default.

Data-classes have the __post_init__ method which seems like a good candidate for ad-hoc checking at the moment, but the __post_init__ does not work properly with inheritance. That means, that when class B inherits class A, class B can completely override the __post_init__ defined by class A, leading to issues.

PyDantic is another option as well. Personally, the checking and the JSON serialization are great. But I want to avoid having more dependencies if possible.

Another thing is, I eventually plan to add PyYAML to the dependencies too, so it might become too heavy.

To do

• Check if PyDantic works well with Multiple Inheritance