You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After merging #63, the accumulation of H_K, H_C will be done with a tensor hook that gets installed onto the output of a layer. We also have a second hook which saves the input to a layer into a .inputs dictionary which acts like a global variable.
Proposal: Instead of saving the input to a layer into SINGD.layers, directly pass it to the tensor hook that computes H_K, H_C. This
Eliminates a 'global' variable (SINGD.inputs)
Reduces the number of hooks (and related boilerplate code like checking if .step matches the update frequency) from 2 to 1
The text was updated successfully, but these errors were encountered:
After merging #63, the accumulation of
H_K, H_C
will be done with a tensor hook that gets installed onto the output of a layer. We also have a second hook which saves the input to a layer into a.inputs
dictionary which acts like a global variable.Proposal: Instead of saving the input to a layer into
SINGD.layers
, directly pass it to the tensor hook that computesH_K, H_C
. ThisSINGD.inputs
).step
matches the update frequency) from 2 to 1The text was updated successfully, but these errors were encountered: