Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sightglass-next: extract native baseline benchmarking from webui_runner #96

Open
abrown opened this issue Mar 5, 2021 · 1 comment
Open
Assignees
Milestone

Comments

@abrown
Copy link
Collaborator

abrown commented Mar 5, 2021

One useful feature of the original sightglass code was the ability to run the benchmarks as native machine code in order to form a baseline for comparison. If we migrate this functionality from webui_runner to benchmarks-next (e.g.), we can then fully replace the old sightglass runner with the new one.

This involves some investigation to determine how to hook into the bench_start() and bench_end() calls with perf.

@abrown abrown added this to the MVP milestone Mar 5, 2021
@abrown abrown self-assigned this Mar 5, 2021
@abrown abrown assigned abrown and unassigned abrown Mar 5, 2021
@abrown
Copy link
Collaborator Author

abrown commented Apr 6, 2021

I've looked into this a bit more and it seems highly unlikely that we will be able to trigger perf to measure hardware events (e.g. instructions retired) using tracepoints. I was hoping that something like the following would work:

clang benchmark.c -O3 -g -I. -o benchmark
perf probe -x ./benchmark bench_start=_bench_start
perf probe -x ./benchmark bench_end=_bench_end
perf record -e cycles --filter "start probe_benchmark:bench_start, stop probe_benchmark:bench_end" ./benchmark

Unfortunately, the last call fails because perf does not measure hardware events using these types of triggers; the perf-record documentation points out that the --filter "option should follow an event selector (-e) which selects either tracepoint event(s) or a hardware trace PMU (e.g. Intel PT or CoreSight)".

Another option is to start measuring directly with perf_event_open inside the bench_start call, much like what happens in the recorder crate. We could modify the sightglass-next.h header to do this when compiling the benchmarks natively (i.e. not to Wasm) so that we could get at least some native measurements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@abrown and others