Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Flax FAQ - how to search, how to take the derivative w.r.t. a hidden layer, remat_scan vs scan(remat), recommended training libraries/metrics #3267

Merged
merged 2 commits into from
Oct 10, 2023

Conversation

8bitmp3
Copy link
Collaborator

@8bitmp3 8bitmp3 commented Aug 8, 2023

This PR adds a new Flax FAQ.

  • Item 1: How to search
  • Item 2: How to take a derivative w.r.t. a hidden layer with flax.linen.Module.perturb
  • Item 3: Flax Linen remat_scan vs. scan(remat(...))
  • Item 4: Recommended training loop libraries (CLU, etc)

@8bitmp3 8bitmp3 requested a review from chiamp August 8, 2023 14:36
@8bitmp3 8bitmp3 self-assigned this Aug 8, 2023
@8bitmp3 8bitmp3 changed the title Add Flax FAQ - how to search, how to take a derivative w.r.t. a hidden layer Add Flax FAQ - how to search, how to take a derivative w.r.t. a hidden layer, remat_scan vs scan(remat) Aug 8, 2023
@codecov-commenter
Copy link

codecov-commenter commented Aug 8, 2023

Codecov Report

Merging #3267 (7b1d8a6) into main (6635436) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #3267   +/-   ##
=======================================
  Coverage   83.60%   83.60%           
=======================================
  Files          56       56           
  Lines        6747     6747           
=======================================
  Hits         5641     5641           
  Misses       1106     1106           

@8bitmp3 8bitmp3 requested a review from cgarciae August 9, 2023 16:18
@8bitmp3 8bitmp3 changed the title Add Flax FAQ - how to search, how to take a derivative w.r.t. a hidden layer, remat_scan vs scan(remat) Add Flax FAQ - how to search, how to take the derivative w.r.t. a hidden layer, remat_scan vs scan(remat) Aug 14, 2023
docs/faq.rst Outdated Show resolved Hide resolved
docs/faq.rst Outdated Show resolved Hide resolved
docs/faq.rst Outdated Show resolved Hide resolved
docs/faq.rst Outdated Show resolved Hide resolved
docs/faq.rst Outdated Show resolved Hide resolved
docs/faq.rst Outdated Show resolved Hide resolved
@8bitmp3 8bitmp3 changed the title Add Flax FAQ - how to search, how to take the derivative w.r.t. a hidden layer, remat_scan vs scan(remat) Add Flax FAQ - how to search, how to take the derivative w.r.t. a hidden layer, remat_scan vs scan(remat), recommended training libraries/metrics Oct 9, 2023
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@8bitmp3 8bitmp3 reopened this Oct 9, 2023
@copybara-service copybara-service bot merged commit 30147a4 into google:main Oct 10, 2023
19 checks passed
@8bitmp3 8bitmp3 deleted the flax-faq branch October 10, 2023 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants