Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADD] Option to treat weight and bias of a layer jointly in KFAC #57

Merged
merged 5 commits into from
Nov 8, 2023

Conversation

f-dangel
Copy link
Owner

@f-dangel f-dangel commented Nov 7, 2023

Eventually, this will resolve #53.

At the moment, I am struggling to get the tests running and would benefit from a second pair of eyes to look through the code. At the moment, the tests seem to execute for both separate_weight_and_bias = [False, True], but the matrix produced by KFAC is wrong for True.

I believe this is related to the order of parameters. But I think I am actually arranging them in the correct order 😂, so maybe I'm barking up the wrong tree. In any case, it would be great if you could skim the code and look for anything obvious I missed.

@f-dangel f-dangel requested a review from runame November 7, 2023 23:26
@f-dangel f-dangel marked this pull request as draft November 7, 2023 23:29
@f-dangel
Copy link
Owner Author

f-dangel commented Nov 8, 2023

Update: I found the problem and fixed everything. Ready for review.

@f-dangel f-dangel marked this pull request as ready for review November 8, 2023 02:31
@coveralls
Copy link

coveralls commented Nov 8, 2023

Pull Request Test Coverage Report for Build 6801539850

  • 22 of 22 (100.0%) changed or added relevant lines in 1 file are covered.
  • 40 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-4.8%) to 87.668%

Files with Coverage Reduction New Missed Lines %
curvlinops/examples/utils.py 6 41.67%
curvlinops/fisher.py 34 28.85%
Totals Coverage Status
Change from base Build 6790452156: -4.8%
Covered Lines: 718
Relevant Lines: 819

💛 - Coveralls

Copy link
Collaborator

@runame runame left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry was asleep :D But LGTM!

test/test_kfac.py Outdated Show resolved Hide resolved
@f-dangel f-dangel merged commit e842f31 into development Nov 8, 2023
13 checks passed
@f-dangel f-dangel deleted the joint-weight-and-bias branch January 13, 2024 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants