Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
FIX: Adalora ranknum loaded on wrong device (#1852)
Locally, multiple AdaLoRA-related tests are failing. We did not catch this in the nightly run because the tests were missing the necessary pytest marker. The issue is related to the change in #1742, which made it possible to load different adapters on different devices. Although that PR itself was sound, the issue is that for AdaLoRA, one of its parameters, ranknum, was not listed in the other_param_names and was thus not moved to the correct device. This oversight is now fixed and the GPU tests are now passing locally for me. This PR also adds the missing pytest marker to the test class that was missing it, so that these errors should be caught by our nightly CI in the future.
- Loading branch information