Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

channel_selection layer intraining process #77

Open
wendychina opened this issue Nov 12, 2021 · 2 comments
Open

channel_selection layer intraining process #77

wendychina opened this issue Nov 12, 2021 · 2 comments

Comments

@wendychina
Copy link

在进行DenseNet剪枝之后,需要继续训练进行finetune,我注意到channel层的index参数是默认配置为nn.Parameter,这种配置默认需要进行训练。在我的理解中,channel_selection层应该是不需要进行训练的,比较迷惑,请问是代码有问题还是我的理解有问题

@Eric-mingjie
Copy link
Owner

channel_selection 虽然为nn.Parameter,但是训练过程中不会update,因为这个parameter只是作为feature map tensor 的index来使用的,所以backpropogation的时候不会有gradient。

@JudasDie
Copy link

channel_selection 虽然为nn.Parameter,但是训练过程中不会update,因为这个parameter只是作为feature map tensor 的index来使用的,所以backpropogation的时候不会有gradient。

请问有没有遇到过channel selection layer加上去后训练速度变慢问题 (不是imagenet训练,是训练其他结构)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants