Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Makefile: parametrize vw args; also mention --multitask #6

Merged
merged 1 commit into from
Jun 14, 2023
Merged

Conversation

arielf
Copy link
Owner

@arielf arielf commented Jun 14, 2023

No description provided.

@arielf arielf merged commit 9074476 into main Jun 14, 2023
@arielf arielf deleted the mydev branch June 14, 2023 05:30
@arielf
Copy link
Owner Author

arielf commented Jun 14, 2023

For reference, this is the issue I opened in the official vw repo
VowpalWabbit/vowpal_wabbit#4614

My guess is that the leak happens through the full-connectivity of the features via the hidden layer.

Since the full connectivity is a done-deal (imposed at the start of run by the fact we want a fully-connected NN.)

It seems to me that the SGD update should somehow skip updates to weights that have nothing to do with the ones in the example. IOW the skips should be in run-time (rather than initialization time) and should update only those target feature-nodes that are present in the current example (and/or namespace).

Ideally, this skip vs non-skip (current default) should be controlled by a CLI switch.
--respect_namespaces --restricted_update or something like this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant