-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
doubt about patch embeddings #1
Comments
transformer 文章里面是使用了这两个操作,你可以去看下原文中的模型结构。 Attention is all you need. |
但ViT的文章和代码好像没有用 |
这个我也不清楚了 说实话我没看过那篇文章跟代码... 你可以去掉试试 我觉得影响不大应该。 |
好的,多谢 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi~awsome repo.
But I wonder if it is necessary to implement GELU and LayerNorm after linear layer to get patch embedding. Neither the ViT paper and code applies these layers.
What I mentioned is in Line 269 and 270 in /SETR/transformer_model.py
The text was updated successfully, but these errors were encountered: