Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the SPA unit tests reliable #71

Open
jgosmann opened this issue Sep 15, 2017 · 0 comments
Open

Make the SPA unit tests reliable #71

jgosmann opened this issue Sep 15, 2017 · 0 comments

Comments

@jgosmann
Copy link
Collaborator

(original issue nengo/nengo#512)

Currently, a lot of the SPA unit tests break easily (e.g. working memory, thalamus) from changes which should improve the network. This defeats the purpose of the unit tests as they do not tell whether the code is working or not and one needs quite some background knowledge to evaluate whether they fail because something is not working or because the tests need an adjustment.

Each test should define somehow a bound on the permitted error and any improvement in accuracy should leave the tests passing. I assume figuring out how to determine that error in each case is the major difficulty.

It would be cool if those error values would be saved for every commit, so that we can track improvements and regressions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant