Change default benchmark mode to upstream PyTorch

[anmyachev]

Current state (https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/10950264922 vs https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/10949253321):

• Softmax: triton geomean diff: 13%, xetla geomean diff: 8%, ratio geomean diff: 5%
  • The performance deteriorates most on small data. For example, for N=256, the average value of milliseconds, from which teraflops are calculated, changes as follows: 0.0055 vs 0.0116. The difference is about 5-6 microseconds. This is approximately the time spent on the host to run the kernel and cannot be avoided. An option may be to increase the data volume, which will reduce the impact of associated time losses.
• FA advanced: triton geomean diff: 2%, xetla geomean diff: 3%, ratio geomean diff: 2%. Correct numbers are: triton geomean diff: -2.3%, xetla geomean diff: -4%, ratio (triton/xetla) geomean diff: 1.7%
  • This change shouldn't be a problem, part of it is definitely fluctuations in measurements.
  • To check the correctness it seems we can use the cpu version: Use cpu version of torch sdpa until xpu version is fixed #2300
• GEMM advanced: triton geomean diff: 2%, xetla geomean diff: 2%, ratio geomean diff: 4%. Correct numbers are: triton geomean diff: -4.1%, xetla geomean diff: -2.7%, ratio (triton/xetla) geomean diff: -1.4%
• Float conversion microbenchmark: BF16 geomean diff: << 1%, FP16 geomean diff: << 1%

[anmyachev]

Let's move on. This change can be easily rolled back if necessary.

[whitneywhtsang]

@anmyachev Should the ratio diff be recalculated for FA as well? FA advanced: triton geomean diff: 2%, xetla geomean diff: 3%, ratio geomean diff: 2%.
Do you know why the degradation of Triton is much higher than XeTLA for Softmax?

[anmyachev]

Do you know why the degradation of Triton is much higher than XeTLA for Softmax?

Most likely due to different time spent on the host, in the case of Triton there is more Python code, which is slower than C++.

@whitneywhtsang since GEMM on Triton it became worse than planned, I suppose it is worth rolling back this change? Or is this an acceptable change?

UPD: about overhead in Triton: triton-lang/triton#3166

[etiotto]

Do you know why the degradation of Triton is much higher than XeTLA for Softmax?

Most likely due to different time spent on the host, in the case of Triton there is more Python code, which is slower than C++.

@whitneywhtsang since GEMM on Triton it became worse than planned, I suppose it is worth rolling back this change? Or is this an acceptable change?

UPD: about overhead in Triton: triton-lang/triton#3166

We should report the performance number with IPEX on by default because without IPEX the timing taken by upstream PyTorch is not precise (not just the kernel time). IMHO we should revert this change.