-
-
Notifications
You must be signed in to change notification settings - Fork 4.7k
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Loading state dict in a feature extraction network #2215
Comments
@ioangatop if you want classifier weights loaded into feature extraction wrapped models, you need to load weights as 'pretrained' so that they are loaded before the model is mutated. See related discussion, should work with >= 0.9 timm version https://github.com/hugginface/pytorch-image-models/discussions/1941 Although, example in that discussion should be a bit differentl, use the 'overlay' arg as in the train script Lines 463 to 468 in d4ef0b4
The overlay dict is merged with the models normal pretrained_cfg, the pretrained_cfg arg fully overrides it. Alternative to using the |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Describe the bug
Hi Ross! I'm facing a small issue with the features extractor, here are some details:
The function
create_model
supports the argument ofcheckpoint_path
which allows to load custom model weights. However, when we want to load a model as feature extractor, the model is wrapped around theFeatureGetterNet
class, and the loading fails as the keys do not much anymore; theFeatureGetterNet
stores the model underself.model
so in order to work, the state dict keys should have a prefixmodel.
, for exampleclass_token
->model.class_token
Additionally, one workaround is to do the loading of the model after the initialisation, but this also fails as some networks, like vision transformer, prune some layers and thus the state_dict has extra keys
To Reproduce
As always, thanks a lot 🙏
The text was updated successfully, but these errors were encountered: