-
-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dimension of subsequent layers in Hypernetwork #169
Comments
This comment was marked as abuse.
This comment was marked as abuse.
Sorry for the very late reply. I'm not sure what you are referring to exactly, could you please point to a line or a section of code please? |
Lines 221-223 in class HyperLSTM state:
This chunk calls Line 120 in the initialisation function.
Thus, the first layer created by the code chunk is LSTMCell(hidden_size + input_size, hyper_size, layer_norm=True) Then the next layer is: LSTMCell(hidden_size + hidden_size, hyper_size, layer_norm=True) <-----I am confused about the 2*hidden_size dimension here. |
Hi, I was reading through your implementation of HyperLSTM and the associated paper. I got lost in the shaping of the layers after the first layer. Could you please explain why the input size is 2*main_lstm_hidden_size?
The text was updated successfully, but these errors were encountered: