Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor gemm #155

Merged
merged 7 commits into from
May 15, 2024
Merged

Refactor gemm #155

merged 7 commits into from
May 15, 2024

Conversation

willtebbutt
Copy link
Member

@willtebbutt willtebbutt commented May 15, 2024

This addresses #138 , and is going to provide a basis from which to explain #142 .

  • Reimplement BLAS at a marginally higher level, improve performance, and check type stability
  • Extend reimplementation to handle views into arrays

edit: I'm hoping that this will also improve our performance on some of the inter-AD benchmarks that we run, where I think the current implementation of gemm was a problem.

Copy link
Contributor

github-actions bot commented May 15, 2024

Performance Ratio:

┌────────────────────────────┬────────┬─────────┬─────────────┬─────────┐
│                      Label │  Tapir │  Zygote │ ReverseDiff │  Enzyme │
│                     String │ String │  String │      String │  String │
├────────────────────────────┼────────┼─────────┼─────────────┼─────────┤
│                        sum │   40.8 │   0.433 │        3.02 │   0.659 │
│                       _sum │   7.66 │   480.0 │        27.8 │   0.134 │
│                   kron_sum │   82.4 │    3.68 │       212.0 │    24.4 │
│              kron_view_sum │   94.7 │    11.3 │       243.0 │    8.47 │
│      naive_map_sin_cos_exp │   4.32 │ missing │        8.81 │    2.77 │
│            map_sin_cos_exp │   4.62 │    1.78 │        7.72 │    3.43 │
│      broadcast_sin_cos_exp │   4.46 │    2.66 │        1.67 │    2.85 │
│                 simple_mlp │   5.36 │    2.47 │        5.19 │    2.37 │
│                     gp_lml │   6.26 │    2.34 │     missing │ missing │
│ turing_broadcast_benchmark │   6.57 │ missing │        34.2 │ missing │
└────────────────────────────┴────────┴─────────┴─────────────┴─────────┘

@willtebbutt willtebbutt changed the title WIP: refactor gemm Refactor gemm May 15, 2024
@willtebbutt willtebbutt merged commit c906419 into main May 15, 2024
2 checks passed
@willtebbutt willtebbutt deleted the wct/refactor-gemm branch May 15, 2024 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant