Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sycl-bench benchmarks #2047

Closed
wants to merge 28 commits into from
Closed
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
939b4d0
add syclbench.py
mateuszpn Sep 3, 2024
335faaa
unisa-hpc/sycl-bench benchmarks added
mateuszpn Sep 4, 2024
56cf9cf
Merge branch 'main' into add-sycl-bench
mateuszpn Sep 4, 2024
d5b553a
Merge branch 'oneapi-src:main' into add-sycl-bench
mateuszpn Sep 4, 2024
5f374f6
Benchamrk names simplified
mateuszpn Sep 5, 2024
c4854bf
Merge branch 'add-sycl-bench' of github.com:mateuszpn/unified-runtime…
mateuszpn Sep 5, 2024
0ab5039
Merge branch 'oneapi-src:main' into add-sycl-bench
mateuszpn Sep 6, 2024
535c723
update
mateuszpn Sep 10, 2024
7d751ba
update
mateuszpn Sep 10, 2024
6895f21
update
mateuszpn Sep 10, 2024
ff3a0f1
WIP
mateuszpn Sep 12, 2024
20d6859
Performance chart and geometrical mean
mateuszpn Sep 12, 2024
162681b
Merge branch 'oneapi-src:main' into add-sycl-bench
mateuszpn Sep 16, 2024
2af01f7
push benchmarks results
mateuszpn Sep 16, 2024
98f338b
Merge branch 'add-sycl-bench' of github.com:mateuszpn/unified-runtime…
mateuszpn Sep 16, 2024
b1acce4
WIP [skip ci]
mateuszpn Sep 17, 2024
1499fdd
WIP [skip ci]
mateuszpn Sep 17, 2024
8479e59
Merge branch 'oneapi-src:main' into add-sycl-bench
mateuszpn Sep 17, 2024
3b2f85e
png added
mateuszpn Sep 17, 2024
8df45d8
Merge branch 'add-sycl-bench' of github.com:mateuszpn/unified-runtime…
mateuszpn Sep 17, 2024
716f103
action added [skip ci]
mateuszpn Sep 17, 2024
50309a8
ascii bar chart added to benchmarks results [skip ci]
mateuszpn Sep 19, 2024
a2affaa
ascii bar fix [skip ci]
mateuszpn Sep 19, 2024
0358db2
bar chart update [skip ci]
mateuszpn Sep 20, 2024
90608c8
update [skip ci]
mateuszpn Sep 20, 2024
ebe4160
update [skip ci]
mateuszpn Sep 20, 2024
379e1bd
update [skip ci]
mateuszpn Sep 20, 2024
7af46ef
Merge branch 'oneapi-src:main' into add-sycl-bench
mateuszpn Sep 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/benchmarks_compute.yml
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,14 @@ jobs:
id: benchmarks
run: numactl -N 0 ${{ github.workspace }}/ur-repo/scripts/benchmarks/main.py ~/bench_workdir ${{github.workspace}}/sycl_build ${{github.workspace}}/ur-repo ${{ matrix.adapter.str_name }} ${{ inputs.bench_script_params }}

- name: Upload results
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem like a good idea from a security point of view. How is this git authenticated?
This should work similarly to how we publish documentation, see this existing workflow: https://github.com/oneapi-src/unified-runtime/blob/main/.github/workflows/docs.yml

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In final solution there are no changes in .yml files

run:
cd ${{github.workspace}}/ur-repo/
git switch bench-results
git add benchmarks_result.md benchmarks_chart.png
git commit -m results-$(date +%Y-%m-%d/%H_%M_%S)[skip ci]
gitr push

- name: Add comment to PR
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
if: ${{ always() && inputs.pr_no != 0 }}
Expand Down
10 changes: 6 additions & 4 deletions scripts/benchmarks/benches/compute.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,13 @@ def __init__(self, directory):
return

def setup(self):
if self.built:
build_path = create_build_path(self.directory, 'compute-benchmarks-build')
self.bins = os.path.join(build_path, 'bin')

if self.built or not options.rebuild:
Copy link
Contributor

@pbalcer pbalcer Sep 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?
self.built means that this was already built initialized by the previous invocation of setup. What options.rebuild is preventing is removing the build directory, but we still want to pull the changes, call cmake and see whether there aren't any updates.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, if it is crucial. The change made it possible to run the scripts on cluster nodes with no access to external servers...

return

repo_path = git_clone(self.directory, "compute-benchmarks-repo", "https://github.com/intel/compute-benchmarks.git", "08c41bb8bc1762ad53c6194df6d36bfcceff4aa2")
build_path = create_build_path(self.directory, 'compute-benchmarks-build')

configure_command = [
"cmake",
Expand All @@ -43,14 +45,14 @@ def setup(self):
run(f"cmake --build {build_path} -j", add_sycl=True)

self.built = True
self.bins = os.path.join(build_path, 'bin')

class ComputeBenchmark(Benchmark):
def __init__(self, bench, name, test):
self.bench = bench
self.bench_name = name
self.test = test
super().__init__(bench.directory)
self.setup()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed (after reverting the options.rebuild logic).


def bin_args(self) -> list[str]:
return []
Expand Down Expand Up @@ -78,7 +80,7 @@ def run(self, env_vars) -> Result:

result = self.run_bench(command, env_vars)
(label, mean) = self.parse_output(result)
return Result(label=label, value=mean, command=command, env=env_vars, stdout=result, lower_is_better=self.lower_is_better())
return [ Result(label=self.name(), value=mean, command=command, env=env_vars, stdout=result, lower_is_better=self.lower_is_better()) ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are you changing this into an array?
All the other benchmarks would have to change as well. You'd also need to change the return type of this function (and the base one as well).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single benchmark in sycl-bench may return a set of results, because in single run we in fact have a group of benchmarks. Functions in other classes are adjusted


def parse_output(self, output):
csv_file = io.StringIO(output)
Expand Down
Loading
Loading