[WIP] ENH Add support for Qwen2 #1906
Conversation
Add Qwen2 to default target modules, use tiny Qwen2 in tests.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| @@ -1419,6 +1419,9 @@ def _test_weighted_combination_of_adapters(self, model_id, config_cls, config_kw | |||
| if issubclass(config_cls, AdaLoraConfig): | |||
| # AdaLora does not support adding more than 1 adapter | |||
| return pytest.skip(f"Test not applicable for {config_cls}") | |||
| if model_id.endswith("qwen2"): | |||
There was a problem hiding this comment.
Is model_id a sufficiently reliable way to detect if the model is qwen2 type? Could we not look into its config and infer it from there?
There was a problem hiding this comment.
For prod code, I would agree, but this is just for the list of unit tested models, where we wouldn't introduce a different qwen2 architecture, as that would not help test coverage. Even if we did, the worst thing that could happen is that the test would fail. I prefer to have the code that skips the test collected in one place as early as possible in the function body, which is why I don't want to do this after initializing the config.
There was a problem hiding this comment.
Make sense. I will respect your judgement then.
There was a problem hiding this comment.
Thanks. Could you approve then?
Add Qwen2 to default target modules, use tiny Qwen2 in tests.
Some tests are still be failing, waiting for #1901.Done.