-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
求解释Saliency分数得到的tensor #26
Comments
不好意思忘记回了:这里[1,1,i,j]对应于从j到i的attention的重要性 (|attenion_prob_{i,j} * attenion_prob_{i,j}.grad|)(然后我当时为了处理方便就限制bsz为1了,第二个1是因为对所有head取了平均 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
您好!我参考您的代码,将应用于GPT2的Attentioner Manager应用到Llama上,然后得到了saliency分数,每一层都是[1,1,seq_len,seq_len],部分具体数值如下:
![image](https://private-user-images.githubusercontent.com/59468866/351317982-e62c830c-b2f3-42a8-a256-467b95e6f5e4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYwOTM5NTMsIm5iZiI6MTcyNjA5MzY1MywicGF0aCI6Ii81OTQ2ODg2Ni8zNTEzMTc5ODItZTYyYzgzMGMtYjJmMy00MmE4LWEyNTYtNDY3Yjk1ZTZmNWU0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTExVDIyMjczM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTllZDk0NTg2YjE1MTI1OTQyZjg1NWI2ZGYzZmZlMjE0ZWIzNGY4OThkZGVjN2ZkOGQ2MDcwMjViYTEwNDJkMjcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.X8HA4n7YXSzwuKgj5zR3llEJmxgyeRf0T0JAyLvpWbE)
我想知道这里每一层的saliency分数的具体含义?
我的代码如下:
The text was updated successfully, but these errors were encountered: