Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some performance-oriented testing #59

Open
vkottler opened this issue Apr 10, 2023 · 6 comments
Open

Add some performance-oriented testing #59

vkottler opened this issue Apr 10, 2023 · 6 comments

Comments

@vkottler
Copy link
Owner

It would be neat to take some of the connection interfaces, and the Protocol and PrimitiveArray implementations, and come up with some benchmark tests to keep track of how well they perform over time.

For an interesting end-to-end test, we could test the echo connections (in isolated ConnectionArbiter instances) for some specified amount of time and just see how many bytes were transmitted and received by both ends.

You could just send 1024-byte payloads or something.

It will also be interesting to compare performance on the different platforms.

@vkottler
Copy link
Owner Author

vkottler commented May 5, 2023

I think we need #70 for this

@vkottler
Copy link
Owner Author

I think this is kind of fruitless for Python. If it's slow, it needs to call into C/C++ code for certain things instead.

That said, we implemented #70 and #77 to assist with benchmarking.

We could create some benchmarking applications for fun.

@vkottler
Copy link
Owner Author

We could wire up Echo connections of each type for 5 seconds and check the stats or something..

@vkottler
Copy link
Owner Author

vkottler commented Sep 9, 2023

Can essentially do this with: https://github.com/vkottler/conntextual.

@vkottler
Copy link
Owner Author

I've been using this script to stress test performance: https://github.com/vkottler/runtimepy/blob/master/perf_test.sh.

We'll need a way to sample / collect data with the program running and then store it on disk somewhere for later analysis (stream telemetry protocol)

@vkottler
Copy link
Owner Author

I wonder if the server command with -l / --loopback would make building something like this easier (#156).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant