WIP: Add Support for Gradient Checkpointing#759

Draft

lenglaender wants to merge 4 commits intoadapter-hub:mainfrom lenglaender:fix/gradient_checkpointing

+141-6

Commits on Nov 7, 2024

Fix gradient checkpointing and write tests
lenglaender
committed

Commits on Nov 11, 2024

minor fix but doesn't resolve the remaining issues
lenglaender
committed

Commits on Nov 24, 2024

Only run adjust_tensors_for_parallel_ if bsz is different
calpt
committed

Commits on Nov 25, 2024

remove parallel grad checkpointing test
calpt
committed