-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ADD] Support for KFAC with type-2 Fisher #56
Conversation
Pull Request Test Coverage Report for Build 6802629667
💛 - Coveralls |
Conflicts: curvlinops/kfac.py test/test_kfac.py
I decided to start a section 'Internals' in the documentation which explains the tricky parts in math and also contains links to derivations. This might be useful for others who are interested in extending this library, or learning more about the details of each curvature approximation. I have worked on this topic for quite a bit, so it would be great to get feedback if it is comprehensible. You can take a look here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really like the refactor in terms of backpropagating the square root loss Hessian and the idea of having an internals section in the docs! I left a few comments/questions.
Resolves #49.
Note: I can't immediately see anymore why the implementation for
CrossEntropyLoss
works like this, will have to think about it, but should be correct.