Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add pretrained MAE weights, option to load checkpoints in ViT builder (…
…#479) Summary: For MAE fine-tuning, fine-tuning occurs just on the encoder (ViT). This change allows easy loading of MAE pretrained weights directly into our ViT class. Pull Request resolved: #479 Test Plan: ``` python -m pytest -v tests/models/* ... ========== 207 passed, 25 warnings in 424.67s (0:07:04) =========================== python -m pytest -v tests/modules/* ... ======================== 192 passed, 2 skipped, 22 warnings in 10.75s ========================== ``` Test instantiating ViT using MAE pretrained weights for each of the 3 checkpoints: <img width="1163" alt="Screenshot 2023-10-05 at 6 39 02 PM" src="https://github.com/facebookresearch/multimodal/assets/24319399/c159b2dd-0b04-4572-85b9-d3024eee9a53"> Reviewed By: kartikayk Differential Revision: D50015711 Pulled By: ebsmothers fbshipit-source-id: e09fd02560b31574427b9f66373f12e7fd663f06
- Loading branch information