Several tests are available for the user to benchmark Pyccel against other common accelerators, notably pythran and numba. The same code is used for all tests, only the decorators change.
The dependencies can be installed using the command python3 -m pip install .
The code can be executed by running the script benchmarks/run_benchmarks.py
.
In order to test pyccel and/or pythran, configuration files must be provided. An example configuration for pythran is found in benchmarks/config.pythranrc
. This configuration is the default pythran configuration with the following additional flags:
-O3
-march=native
-mtune=native
-mavx
-ffast-math
Pyccel configurations valid for your machine can be generated using the following command (which may be adapted for c generation or other compiler languages, see the pyccel documentation):
pyccel --language=fortran --export-compile-info pyccel_fortran.json
This configuration can then be modified to include additional flags or use different compilers. The tests shown below add the following additional flags (which match the flags added to pythran):
-O3
-march=native
-mtune=native
-mavx
-ffast-math
Additional options can be used with this script to add further comparisons, change the output format, or change what is generated.
Run python3 benchmarks/run_benchmarks.py --help
for more details.
The results below are presented for the current state of the development branch of pyccel, as well as the most recent version of pyccel available on pypi.
A requirements.txt file providing the necessary packages to reproduce the tests run can be found in the version_specific_results
folder.
The environment can be reproduced using the following commands:
python3 -m venv my_virtual_environment
source my_virtual_environment/bin/activate
pip3 install -r requirements.txt
The tests used can be found in the benchmarks/tests directory.
A basic implementation of the Ackermann function which is one of the simplest and oldest examples of a total computable function that is not primitive recursive.
An algorithm for solving the shortest path problem. The code is adapted from examples written by J. Burkardt
An algorithm for solving the shortest path problem. The code is adapted from examples written by J. Burkardt
Solves an ordinary differential equation using Euler's method. The code is adapted from examples written by J. Burkardt
Solves an ordinary differential equation using the explicit midpoint method. The code is adapted from examples written by J. Burkardt
Solves an ordinary differential equation using the implicit midpoint method with a fixed number of iterations. The code is adapted from examples written by J. Burkardt
Solves an ordinary differential equation using a fourth order Runge-Kutta method. The code is adapted from examples written by J. Burkardt
Solves a 1D linear convection problem using Finite Differences methods. The code is adapted from examples written by L. A. Barba
Solves a 1D non-linear convection problem using Finite Differences methods. The code is adapted from examples written by L. A. Barba
Solves a 2D Poisson problem using Finite Differences methods. The code is adapted from examples written by L. A. Barba
Solves a 2D Laplace problem using Finite Differences methods. The code is adapted from examples written by L. A. Barba
Runs a molecular dynamics simulation. The code is adapted from examples written by J. Burkardt
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann | - | 2.28 | 2.10 | 0.33 | 1.33 | 1.33 | 1.49 | 1.48 |
Bellman Ford | - | 3.58 | 3.83 | 1.14 | 3.73 | 4.07 | 3.89 | 4.06 |
Dijkstra | - | 2.39 | 2.79 | 1.63 | 3.86 | 4.11 | 4.00 | 4.23 |
Euler | - | 2.65 | 3.03 | 2.02 | 3.65 | 3.96 | 3.80 | 4.01 |
Midpoint Explicit | - | 3.01 | 3.40 | 2.99 | 3.86 | 4.20 | 4.05 | 4.18 |
Midpoint Fixed | - | 3.50 | 3.98 | 3.22 | 4.05 | 4.42 | 4.08 | 4.24 |
RK4 | - | 3.70 | 4.17 | 3.74 | 4.41 | 4.62 | 4.47 | 4.75 |
FD - L Convection | - | 2.30 | 2.67 | 0.89 | 1.42 | 3.94 | 1.59 | 3.90 |
FD - NL Convection | - | 3.27 | 3.51 | 0.88 | 1.40 | 3.93 | 1.61 | 3.90 |
FD - Poisson | - | 3.36 | 3.73 | 1.33 | 1.53 | 4.04 | 2.86 | 4.02 |
FD - Laplace | - | 6.53 | 7.97 | 3.03 | 1.84 | 4.37 | 2.11 | 4.26 |
M-D | - | 6.39 | 6.68 | 4.06 | - | - | - | - |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann (ms) | 292.00 | 2.85 | 3.03 | 9.76 | 1.50 | 1.55 | 9.35 | 4.33 |
Bellman Ford (ms) | 1760.00 | 5.25 | 3.41 | 3.94 | 2.96 | 6.15 | - | 19.30 |
Dijkstra (ms) | 4900.00 | 25.60 | 16.10 | 20.40 | 17.90 | 31.60 | - | 22.70 |
Euler (ms) | 3810.00 | 24.70 | 25.40 | 37.10 | 15.90 | 143.00 | 14.00 | 127.00 |
Midpoint Explicit (ms) | 7800.00 | 52.70 | 51.20 | 78.50 | 23.60 | 280.00 | 16.90 | 250.00 |
Midpoint Fixed (ms) | 39400.00 | 252.00 | 92.90 | 376.00 | 75.10 | 1380.00 | 58.80 | 1210.00 |
RK4 (ms) | 19900.00 | 163.00 | 35.10 | 138.00 | 34.60 | 487.00 | 37.30 | 402.00 |
FD - L Convection (ms) | 2210.00 | 1.70 | 1.62 | 2.68 | 1.62 | 1.84 | - | 4.07 |
FD - NL Convection (ms) | 2690.00 | 1.82 | 1.68 | 2.83 | 1.85 | 2.19 | - | 4.29 |
FD - Poisson (ms) | 6260.00 | 2.96 | 5.44 | 7.26 | 2.76 | 3.74 | - | 4.93 |
FD - Laplace (ms) | 590.00 | 64.40 | 143.00 | 246.00 | 62.60 | 280.00 | - | 273.00 |
M-D (ms) | 14700.00 | 15.20 | 52.70 | 59.00 | - | - | - | - |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann | - | 1.82 | 1.93 | 0.34 | 1.29 | 1.26 | 1.37 | 1.34 |
Bellman Ford | - | 3.24 | 3.85 | 1.24 | 3.74 | 4.01 | 3.85 | 4.58 |
Dijkstra | - | 2.24 | 2.60 | 1.70 | 3.83 | 4.03 | 3.95 | 4.69 |
Euler | - | 2.47 | 2.92 | 2.04 | 3.63 | 4.00 | 3.87 | 4.56 |
Midpoint Explicit | - | 2.82 | 3.46 | 3.08 | 3.85 | 4.17 | 3.98 | 4.65 |
Midpoint Fixed | - | 3.30 | 3.92 | 3.38 | 3.99 | 4.38 | 4.03 | 4.77 |
RK4 | - | 3.46 | 4.09 | 3.79 | 4.35 | 4.59 | 4.46 | 5.11 |
FD - L Convection | - | 2.07 | 2.60 | 0.92 | 3.56 | 3.84 | 3.69 | 4.38 |
FD - NL Convection | - | 2.99 | 3.44 | 0.91 | 3.57 | 3.90 | 3.72 | 4.46 |
FD - Poisson | - | 3.10 | 3.62 | 1.35 | 3.68 | 3.99 | 4.91 | 4.48 |
FD - Laplace | - | 6.13 | 8.65 | 3.04 | 4.02 | 4.32 | 4.26 | 4.95 |
M-D | - | 5.98 | 7.55 | 4.03 | 4.39 | 4.49 | 4.57 | 5.35 |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann (ms) | 384.00 |
2.94 |
3.06 |
10.90 |
1.56 |
1.56 |
9.11 |
4.80 |
Bellman Ford (ms) | 2000.00 |
4.54 |
3.19 |
3.89 |
2.99 |
6.19 |
4.43 |
18.50 |
Dijkstra (ms) | 5050.00 |
23.80 |
16.90 |
20.50 |
19.70 |
31.10 |
25.40 |
23.10 |
Euler (ms) | 3990.00 |
29.10 |
25.80 |
39.30 |
15.40 |
147.00 |
14.30 |
129.00 |
Midpoint Explicit (ms) | 8130.00 |
52.70 |
52.10 |
82.50 |
23.50 |
287.00 |
16.20 |
256.00 |
Midpoint Fixed (ms) | 41800.00 |
254.00 |
92.70 |
399.00 |
76.70 |
1410.00 |
61.70 |
1250.00 |
RK4 (ms) | 21000.00 |
160.00 |
36.30 |
147.00 |
35.40 |
496.00 |
39.00 |
416.00 |
FD - L Convection (ms) | 2450.00 |
1.55 |
1.54 |
2.73 |
1.56 |
1.85 |
1.32 |
4.19 |
FD - NL Convection (ms) | 3020.00 |
1.80 |
1.72 |
2.82 |
1.77 |
2.20 |
1.54 |
4.20 |
FD - Poisson (ms) | 6450.00 |
3.01 |
5.72 |
7.23 |
2.81 |
3.86 |
2.72 |
5.72 |
FD - Laplace (ms) | 425.00 |
68.80 |
151.00 |
253.00 |
58.70 |
286.00 |
64.10 |
336.00 |
M-D (ms) | 15800.00 |
15.30 |
53.20 |
59.50 |
54.10 |
59.90 |
82.10 |
61.70 |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann | - | 1.80 | 1.93 | 0.33 | 1.30 | 1.25 | 1.36 | 1.34 |
Bellman Ford | - | 3.24 | 3.55 | 1.11 | 3.63 | 3.93 | 3.74 | 4.45 |
Dijkstra | - | 2.18 | 2.54 | 1.61 | 3.71 | 3.91 | 3.83 | 4.48 |
Euler | - | 2.45 | 2.88 | 2.06 | 3.61 | 3.90 | 3.70 | 4.41 |
Midpoint Explicit | - | 2.81 | 3.29 | 3.06 | 3.93 | 4.14 | 3.92 | 4.62 |
Midpoint Fixed | - | 3.19 | 3.73 | 3.29 | 3.87 | 4.20 | 3.99 | 4.76 |
RK4 | - | 3.46 | 4.06 | 3.82 | 4.33 | 4.55 | 4.42 | 5.10 |
FD - L Convection | - | 2.07 | 2.55 | 0.88 | 3.55 | 3.85 | 3.69 | 4.38 |
FD - NL Convection | - | 2.96 | 3.44 | 0.89 | 3.57 | 3.86 | 3.72 | 4.38 |
FD - Poisson | - | 3.07 | 3.63 | 1.36 | 3.69 | 4.00 | 4.94 | 4.50 |
FD - Laplace | - | 6.17 | 8.76 | 3.07 | 4.02 | 4.32 | 4.25 | 4.88 |
M-D | - | 6.02 | 7.59 | 4.09 | 4.37 | 4.49 | 4.53 | 5.35 |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann (ms) | 333.00 |
2.97 |
3.06 |
10.80 |
1.51 |
1.56 |
7.51 |
4.36 |
Bellman Ford (ms) | 1920.00 |
4.60 |
3.19 |
3.91 |
3.01 |
6.08 |
4.33 |
18.50 |
Dijkstra (ms) | 5150.00 |
24.90 |
16.40 |
19.70 |
18.10 |
30.00 |
24.10 |
22.40 |
Euler (ms) | 3880.00 |
25.70 |
26.60 |
37.50 |
15.00 |
144.00 |
13.80 |
128.00 |
Midpoint Explicit (ms) | 7930.00 |
53.10 |
52.00 |
76.50 |
23.50 |
284.00 |
16.00 |
256.00 |
Midpoint Fixed (ms) | 40300.00 |
253.00 |
92.80 |
368.00 |
75.70 |
1420.00 |
74.40 |
1240.00 |
RK4 (ms) | 20200.00 |
160.00 |
36.00 |
137.00 |
35.30 |
495.00 |
37.20 |
414.00 |
FD - L Convection (ms) | 2360.00 |
1.66 |
1.56 |
2.74 |
1.76 |
1.75 |
1.54 |
4.18 |
FD - NL Convection (ms) | 2940.00 |
1.58 |
1.74 |
2.83 |
2.07 |
2.02 |
1.53 |
4.19 |
FD - Poisson (ms) | 6610.00 |
3.18 |
5.83 |
7.25 |
2.83 |
3.87 |
2.70 |
5.71 |
FD - Laplace (ms) | 600.00 |
69.00 |
151.00 |
253.00 |
63.30 |
260.00 |
64.20 |
322.00 |
M-D (ms) | 15600.00 |
15.30 |
53.10 |
59.60 |
54.50 |
59.50 |
80.00 |
60.50 |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann | - | 2.19 | 2.09 | 0.33 | 1.26 | 1.21 | 1.32 | 1.30 |
Bellman Ford | - | 3.31 | 3.65 | 1.10 | 3.60 | 3.85 | 3.67 | 4.38 |
Dijkstra | - | 2.30 | 2.68 | 1.62 | 3.64 | 3.88 | 3.81 | 4.43 |
Euler | - | 2.57 | 3.03 | 2.06 | 3.55 | 3.87 | 3.66 | 4.36 |
Midpoint Explicit | - | 2.91 | 3.45 | 3.07 | 3.79 | 4.15 | 3.88 | 4.55 |
Midpoint Fixed | - | 3.28 | 3.86 | 3.29 | 3.84 | 4.16 | 3.94 | 4.64 |
RK4 | - | 3.58 | 4.18 | 3.85 | 4.27 | 4.55 | 4.34 | 5.04 |
FD - L Convection | - | 2.22 | 2.69 | 0.89 | 3.51 | 3.82 | 3.58 | 4.32 |
FD - NL Convection | - | 3.12 | 3.59 | 0.91 | 3.57 | 3.84 | 3.68 | 4.32 |
FD - Poisson | - | 3.21 | 3.75 | 1.36 | 3.65 | 3.94 | 4.87 | 4.44 |
FD - Laplace | - | 6.29 | 8.89 | 3.10 | 3.99 | 4.25 | 4.20 | 4.85 |
M-D | - | 6.14 | 7.72 | 4.12 | 4.31 | 4.41 | 4.46 | 5.29 |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann (ms) | 302.00 |
2.90 |
3.08 |
10.70 |
1.56 |
1.57 |
9.47 |
3.95 |
Bellman Ford (ms) | 1810.00 |
5.27 |
3.19 |
3.86 |
3.00 |
6.02 |
4.43 |
18.70 |
Dijkstra (ms) | 4920.00 |
24.70 |
16.60 |
19.10 |
18.20 |
30.40 |
23.20 |
22.00 |
Euler (ms) | 3890.00 |
25.40 |
25.50 |
37.80 |
15.30 |
148.00 |
13.70 |
129.00 |
Midpoint Explicit (ms) | 7860.00 |
54.70 |
51.70 |
80.40 |
23.70 |
284.00 |
16.20 |
255.00 |
Midpoint Fixed (ms) | 40000.00 |
254.00 |
92.40 |
383.00 |
75.60 |
1410.00 |
60.70 |
1250.00 |
RK4 (ms) | 20100.00 |
160.00 |
35.80 |
139.00 |
35.20 |
495.00 |
37.20 |
412.00 |
FD - L Convection (ms) | 2240.00 |
1.65 |
1.56 |
2.77 |
1.68 |
1.87 |
1.33 |
4.15 |
FD - NL Convection (ms) | 2740.00 |
1.79 |
1.73 |
2.94 |
2.03 |
2.18 |
1.54 |
4.15 |
FD - Poisson (ms) | 6410.00 |
2.99 |
5.67 |
7.35 |
2.83 |
3.86 |
2.71 |
5.73 |
FD - Laplace (ms) | 608.00 |
67.50 |
151.00 |
247.00 |
60.50 |
310.00 |
63.90 |
327.00 |
M-D (ms) | 15100.00 |
15.30 |
53.00 |
59.50 |
54.00 |
59.90 |
81.10 |
62.30 |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann | - | 2.20 | 2.12 | 0.29 | 1.36 | 1.27 | 1.42 | 1.39 |
Bellman Ford | - | 3.35 | 3.67 | 1.13 | 3.69 | 3.87 | 3.83 | 4.48 |
Dijkstra | - | 2.18 | 2.55 | 1.55 | 3.60 | 3.82 | 3.77 | 4.36 |
Euler | - | 2.44 | 2.90 | 1.95 | 3.49 | 3.81 | 3.60 | 4.28 |
Midpoint Explicit | - | 2.78 | 3.26 | 2.90 | 3.73 | 4.05 | 3.86 | 4.50 |
Midpoint Fixed | - | 3.12 | 3.69 | 3.11 | 3.78 | 4.10 | 3.90 | 4.58 |
RK4 | - | 3.40 | 4.01 | 3.61 | 4.18 | 4.44 | 4.25 | 4.96 |
FD - L Convection | - | 2.15 | 2.62 | 0.84 | 3.48 | 3.78 | 3.62 | 4.29 |
FD - NL Convection | - | 3.02 | 3.50 | 0.83 | 3.48 | 3.81 | 3.66 | 4.29 |
FD - Poisson | - | 3.07 | 3.56 | 1.27 | 3.61 | 3.91 | 4.82 | 4.40 |
FD - Laplace | - | 6.10 | 8.65 | 2.87 | 3.90 | 4.20 | 4.15 | 4.75 |
M-D | - | 6.13 | 7.47 | 3.98 | 4.22 | 4.36 | 4.38 | 5.21 |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann (ms) | 434.00 |
2.90 |
3.06 |
10.70 |
1.56 |
1.56 |
9.89 |
4.39 |
Bellman Ford (ms) | 1890.00 |
5.27 |
3.18 |
3.89 |
3.01 |
6.10 |
4.06 |
18.70 |
Dijkstra (ms) | 4870.00 |
23.50 |
15.90 |
18.90 |
17.70 |
30.10 |
23.40 |
21.70 |
Euler (ms) | 3850.00 |
25.40 |
25.40 |
37.30 |
15.30 |
144.00 |
13.70 |
128.00 |
Midpoint Explicit (ms) | 7930.00 |
52.60 |
52.90 |
76.90 |
23.00 |
285.00 |
15.90 |
253.00 |
Midpoint Fixed (ms) | 40200.00 |
257.00 |
92.60 |
367.00 |
75.10 |
1410.00 |
61.40 |
1240.00 |
RK4 (ms) | 19800.00 |
162.00 |
35.50 |
137.00 |
32.60 |
499.00 |
37.80 |
415.00 |
FD - L Convection (ms) | 2220.00 |
1.66 |
1.56 |
2.73 |
1.50 |
1.74 |
1.54 |
4.19 |
FD - NL Convection (ms) | 2750.00 |
1.82 |
1.70 |
2.84 |
1.97 |
2.08 |
1.53 |
4.20 |
FD - Poisson (ms) | 6190.00 |
3.06 |
5.65 |
7.23 |
2.79 |
3.83 |
2.67 |
5.70 |
FD - Laplace (ms) | 594.00 |
68.00 |
150.00 |
246.00 |
60.10 |
257.00 |
63.50 |
327.00 |
M-D (ms) | 14900.00 |
15.30 |
53.20 |
59.60 |
54.80 |
59.60 |
80.30 |
61.30 |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann | - | 1.95 | - | 0.30 | 1.39 | 1.32 | 1.42 | 1.40 |
Bellman Ford | - | 3.33 | - | 1.21 | 3.78 | 3.97 | 3.83 | 4.45 |
Dijkstra | - | 2.41 | - | 1.74 | 3.86 | 3.99 | 3.92 | 4.55 |
Euler | - | 2.54 | - | 2.03 | 3.63 | 3.94 | 3.80 | 4.43 |
Midpoint Explicit | - | 3.01 | - | 3.15 | 4.15 | 4.44 | 4.24 | 4.84 |
Midpoint Fixed | - | 3.38 | - | 3.28 | 4.02 | 4.47 | 4.13 | 4.85 |
RK4 | - | 3.76 | - | 3.85 | 4.59 | 5.13 | 4.85 | 5.52 |
FD - L Convection | - | 2.44 | - | 0.95 | 3.83 | 4.08 | 3.89 | 4.64 |
FD - NL Convection | - | 3.22 | - | 0.88 | 3.75 | 3.97 | 3.81 | 4.44 |
FD - Poisson | - | 3.35 | - | 1.37 | 3.81 | 4.16 | 5.15 | 4.70 |
FD - Laplace | - | 6.38 | - | 3.19 | 4.23 | 4.44 | 4.40 | 5.01 |
M-D | - | 6.47 | - | 4.09 | 4.62 | 4.61 | 4.78 | 5.46 |
Algorithm | python | pythran_gnu | pythran_intel | numba | pyccel_fortran_gnu | pyccel_c_gnu | pyccel_fortran_intel | pyccel_c_intel |
---|---|---|---|---|---|---|---|---|
Ackermann (ms) | 437.00 |
2.95 |
- | 10.80 |
1.56 |
1.61 |
10.60 |
4.37 |
Bellman Ford (ms) | 2530.00 |
4.59 |
- | 3.87 |
3.01 |
5.62 |
4.25 |
18.60 |
Dijkstra (ms) | 6850.00 |
26.80 |
- | 20.90 |
19.10 |
29.40 |
23.00 |
22.30 |
Euler (ms) | 4900.00 |
25.40 |
- | 37.80 |
15.50 |
144.00 |
13.80 |
128.00 |
Midpoint Explicit (ms) | 9820.00 |
52.60 |
- | 78.10 |
24.40 |
285.00 |
16.60 |
256.00 |
Midpoint Fixed (ms) | 48700.00 |
254.00 |
- | 387.00 |
75.40 |
1410.00 |
62.60 |
1260.00 |
RK4 (ms) | 24500.00 |
165.00 |
- | 147.00 |
36.40 |
498.00 |
38.90 |
417.00 |
FD - L Convection (ms) | 2950.00 |
1.63 |
- | 2.73 |
1.69 |
1.75 |
1.53 |
4.21 |
FD - NL Convection (ms) | 3640.00 |
1.81 |
- | 2.83 |
1.94 |
1.97 |
1.40 |
4.24 |
FD - Poisson (ms) | 8120.00 |
2.97 |
- | 7.25 |
2.83 |
3.82 |
2.68 |
5.75 |
FD - Laplace (ms) | 602.00 |
69.60 |
- | 248.00 |
62.60 |
257.00 |
64.20 |
329.00 |
M-D (ms) | 18900.00 |
15.20 |
- | 60.30 |
55.20 |
59.80 |
81.30 |
61.90 |