Skip to content

Commit

Permalink
Merge pull request #690 from LuxDL/ap/newdocs
Browse files Browse the repository at this point in the history 
Newer public functions
  • Loading branch information
@avik-pal
avik-pal authored Jun 8, 2024
2 parents 0913d81 + 5b005ee commit 9117660
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions docs/src/api/Accelerator_Support/LuxDeviceUtils.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,12 @@ reset_gpu_device!
supported_gpu_backends
default_device_rng
get_device
LuxDeviceUtils.loaded
LuxDeviceUtils.functional
```

## Multi-GPU Support

```@docs
LuxDeviceUtils.set_device!
```

1 comment on commit 9117660

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Results

Benchmark suite Current: 9117660 Previous: 0913d81 Ratio
Dense(2 => 2)/cpu/reverse/ReverseDiff (compiled)/(2, 128) 3631.75 ns 3722 ns 0.98
Dense(2 => 2)/cpu/reverse/Zygote/(2, 128) 7211.4 ns 7138.333333333333 ns 1.01
Dense(2 => 2)/cpu/reverse/Tracker/(2, 128) 20899 ns 21371 ns 0.98
Dense(2 => 2)/cpu/reverse/ReverseDiff/(2, 128) 9770.2 ns 9796.4 ns 1.00
Dense(2 => 2)/cpu/reverse/Flux/(2, 128) 8918.6 ns 9029 ns 0.99
Dense(2 => 2)/cpu/reverse/SimpleChains/(2, 128) 4458.25 ns 4483.5 ns 0.99
Dense(2 => 2)/cpu/reverse/Enzyme/(2, 128) 1163.2867132867134 ns 1162.527397260274 ns 1.00
Dense(2 => 2)/cpu/forward/NamedTuple/(2, 128) 1177.5859375 ns 1123.866883116883 ns 1.05
Dense(2 => 2)/cpu/forward/ComponentArray/(2, 128) 1181.5507246376812 ns 1185.589552238806 ns 1.00
Dense(2 => 2)/cpu/forward/Flux/(2, 128) 1812.48 ns 1781.85 ns 1.02
Dense(2 => 2)/cpu/forward/SimpleChains/(2, 128) 179.778859527121 ns 179.8589562764457 ns 1.00
Dense(20 => 20)/cpu/reverse/ReverseDiff (compiled)/(20, 128) 17182 ns 17272 ns 0.99
Dense(20 => 20)/cpu/reverse/Zygote/(20, 128) 16812 ns 17183 ns 0.98
Dense(20 => 20)/cpu/reverse/Tracker/(20, 128) 39183 ns 39975 ns 0.98
Dense(20 => 20)/cpu/reverse/ReverseDiff/(20, 128) 29334 ns 29385 ns 1.00
Dense(20 => 20)/cpu/reverse/Flux/(20, 128) 19908 ns 20368.5 ns 0.98
Dense(20 => 20)/cpu/reverse/SimpleChains/(20, 128) 17212 ns 17253 ns 1.00
Dense(20 => 20)/cpu/reverse/Enzyme/(20, 128) 4326.714285714285 ns 4352.428571428572 ns 0.99
Dense(20 => 20)/cpu/forward/NamedTuple/(20, 128) 3877.125 ns 3897.375 ns 0.99
Dense(20 => 20)/cpu/forward/ComponentArray/(20, 128) 3951.125 ns 3958.625 ns 1.00
Dense(20 => 20)/cpu/forward/Flux/(20, 128) 4891.857142857143 ns 4896.428571428572 ns 1.00
Dense(20 => 20)/cpu/forward/SimpleChains/(20, 128) 1655.1 ns 1661.1 ns 1.00
Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 3, 128) 43508166 ns 47444763 ns 0.92
Conv((3, 3), 3 => 3)/cpu/reverse/Zygote/(64, 64, 3, 128) 57524561 ns 58136701 ns 0.99
Conv((3, 3), 3 => 3)/cpu/reverse/Tracker/(64, 64, 3, 128) 88629241.5 ns 109888951 ns 0.81
Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff/(64, 64, 3, 128) 92599444.5 ns 107649518 ns 0.86
Conv((3, 3), 3 => 3)/cpu/reverse/Flux/(64, 64, 3, 128) 78309259 ns 84850803.5 ns 0.92
Conv((3, 3), 3 => 3)/cpu/reverse/SimpleChains/(64, 64, 3, 128) 12015849.5 ns 11703374 ns 1.03
Conv((3, 3), 3 => 3)/cpu/reverse/Enzyme/(64, 64, 3, 128) 17848430.5 ns 17810409 ns 1.00
Conv((3, 3), 3 => 3)/cpu/forward/NamedTuple/(64, 64, 3, 128) 6995839 ns 7073859 ns 0.99
Conv((3, 3), 3 => 3)/cpu/forward/ComponentArray/(64, 64, 3, 128) 6960013 ns 6986070 ns 1.00
Conv((3, 3), 3 => 3)/cpu/forward/Flux/(64, 64, 3, 128) 12283244 ns 18013332 ns 0.68
Conv((3, 3), 3 => 3)/cpu/forward/SimpleChains/(64, 64, 3, 128) 6382056 ns 6417450 ns 0.99
vgg16/cpu/reverse/Zygote/(32, 32, 3, 16) 743202742 ns 760294781 ns 0.98
vgg16/cpu/reverse/Zygote/(32, 32, 3, 64) 2535835067 ns 2547962999 ns 1.00
vgg16/cpu/reverse/Zygote/(32, 32, 3, 2) 131052099 ns 148653908.5 ns 0.88
vgg16/cpu/reverse/Tracker/(32, 32, 3, 16) 920350449.5 ns 975917312.5 ns 0.94
vgg16/cpu/reverse/Tracker/(32, 32, 3, 64) 3307447543 ns 3543598532 ns 0.93
vgg16/cpu/reverse/Tracker/(32, 32, 3, 2) 247142678.5 ns 258164941 ns 0.96
vgg16/cpu/reverse/Flux/(32, 32, 3, 16) 864160330 ns 841492108.5 ns 1.03
vgg16/cpu/reverse/Flux/(32, 32, 3, 64) 2794291278 ns 2995403559 ns 0.93
vgg16/cpu/reverse/Flux/(32, 32, 3, 2) 133204698.5 ns 152420409 ns 0.87
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 16) 171871796.5 ns 173734545 ns 0.99
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 64) 651737786.5 ns 652860171.5 ns 1.00
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 2) 45340561.5 ns 34689561 ns 1.31
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 16) 164662555.5 ns 164644931 ns 1.00
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 64) 642335324 ns 644016941 ns 1.00
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 2) 29842019 ns 30258337.5 ns 0.99
vgg16/cpu/forward/Flux/(32, 32, 3, 16) 194284264 ns 222517540.5 ns 0.87
vgg16/cpu/forward/Flux/(32, 32, 3, 64) 764771952 ns 850497083 ns 0.90
vgg16/cpu/forward/Flux/(32, 32, 3, 2) 35950578.5 ns 37426850.5 ns 0.96
Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 64, 128) 1265151448 ns 1336712734 ns 0.95
Conv((3, 3), 64 => 64)/cpu/reverse/Zygote/(64, 64, 64, 128) 1856838022.5 ns 1886391564.5 ns 0.98
Conv((3, 3), 64 => 64)/cpu/reverse/Tracker/(64, 64, 64, 128) 2470867127 ns 2439396346 ns 1.01
Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff/(64, 64, 64, 128) 2588619644 ns 2664856430 ns 0.97
Conv((3, 3), 64 => 64)/cpu/reverse/Flux/(64, 64, 64, 128) 1981078997 ns 1930823462.5 ns 1.03
Conv((3, 3), 64 => 64)/cpu/reverse/Enzyme/(64, 64, 64, 128) 565440373 ns 558264207 ns 1.01
Conv((3, 3), 64 => 64)/cpu/forward/NamedTuple/(64, 64, 64, 128) 318800113 ns 326524856 ns 0.98
Conv((3, 3), 64 => 64)/cpu/forward/ComponentArray/(64, 64, 64, 128) 316222183 ns 330220091 ns 0.96
Conv((3, 3), 64 => 64)/cpu/forward/Flux/(64, 64, 64, 128) 468362345 ns 487080425.5 ns 0.96
Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 1, 128) 11959272.5 ns 12010879 ns 1.00
Conv((3, 3), 1 => 1)/cpu/reverse/Zygote/(64, 64, 1, 128) 17759602 ns 18073698 ns 0.98
Conv((3, 3), 1 => 1)/cpu/reverse/Tracker/(64, 64, 1, 128) 19082919 ns 19247966 ns 0.99
Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff/(64, 64, 1, 128) 23814391.5 ns 24037250 ns 0.99
Conv((3, 3), 1 => 1)/cpu/reverse/Flux/(64, 64, 1, 128) 17785751 ns 18077227 ns 0.98
Conv((3, 3), 1 => 1)/cpu/reverse/SimpleChains/(64, 64, 1, 128) 1161806 ns 1171603 ns 0.99
Conv((3, 3), 1 => 1)/cpu/reverse/Enzyme/(64, 64, 1, 128) 5837965 ns 5852949.5 ns 1.00
Conv((3, 3), 1 => 1)/cpu/forward/NamedTuple/(64, 64, 1, 128) 2045774 ns 2061247 ns 0.99
Conv((3, 3), 1 => 1)/cpu/forward/ComponentArray/(64, 64, 1, 128) 2028016 ns 2058760 ns 0.99
Conv((3, 3), 1 => 1)/cpu/forward/Flux/(64, 64, 1, 128) 2065326 ns 2106164.5 ns 0.98
Conv((3, 3), 1 => 1)/cpu/forward/SimpleChains/(64, 64, 1, 128) 199352 ns 197542 ns 1.01
Dense(200 => 200)/cpu/reverse/ReverseDiff (compiled)/(200, 128) 292816 ns 293462 ns 1.00
Dense(200 => 200)/cpu/reverse/Zygote/(200, 128) 266748 ns 267553 ns 1.00
Dense(200 => 200)/cpu/reverse/Tracker/(200, 128) 368898 ns 367001 ns 1.01
Dense(200 => 200)/cpu/reverse/ReverseDiff/(200, 128) 409955 ns 406926 ns 1.01
Dense(200 => 200)/cpu/reverse/Flux/(200, 128) 274732 ns 274727 ns 1.00
Dense(200 => 200)/cpu/reverse/SimpleChains/(200, 128) 409513 ns 408018 ns 1.00
Dense(200 => 200)/cpu/reverse/Enzyme/(200, 128) 83326 ns 83087 ns 1.00
Dense(200 => 200)/cpu/forward/NamedTuple/(200, 128) 81883 ns 81092 ns 1.01
Dense(200 => 200)/cpu/forward/ComponentArray/(200, 128) 82364 ns 81925 ns 1.01
Dense(200 => 200)/cpu/forward/Flux/(200, 128) 86942 ns 87579.5 ns 0.99
Dense(200 => 200)/cpu/forward/SimpleChains/(200, 128) 104995 ns 104697 ns 1.00
Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 16, 128) 201833908 ns 202989353 ns 0.99
Conv((3, 3), 16 => 16)/cpu/reverse/Zygote/(64, 64, 16, 128) 325102058 ns 328775319 ns 0.99
Conv((3, 3), 16 => 16)/cpu/reverse/Tracker/(64, 64, 16, 128) 430782177.5 ns 440894408 ns 0.98
Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff/(64, 64, 16, 128) 476363075 ns 483293045 ns 0.99
Conv((3, 3), 16 => 16)/cpu/reverse/Flux/(64, 64, 16, 128) 414474300.5 ns 409330078.5 ns 1.01
Conv((3, 3), 16 => 16)/cpu/reverse/SimpleChains/(64, 64, 16, 128) 324481600.5 ns 327695827 ns 0.99
Conv((3, 3), 16 => 16)/cpu/reverse/Enzyme/(64, 64, 16, 128) 104043758 ns 101257055.5 ns 1.03
Conv((3, 3), 16 => 16)/cpu/forward/NamedTuple/(64, 64, 16, 128) 43612378 ns 44263361.5 ns 0.99
Conv((3, 3), 16 => 16)/cpu/forward/ComponentArray/(64, 64, 16, 128) 43522483 ns 44322824.5 ns 0.98
Conv((3, 3), 16 => 16)/cpu/forward/Flux/(64, 64, 16, 128) 68462141 ns 68874052.5 ns 0.99
Conv((3, 3), 16 => 16)/cpu/forward/SimpleChains/(64, 64, 16, 128) 28535633 ns 29030318 ns 0.98
Dense(2000 => 2000)/cpu/reverse/ReverseDiff (compiled)/(2000, 128) 19953296 ns 19024938 ns 1.05
Dense(2000 => 2000)/cpu/reverse/Zygote/(2000, 128) 19528479 ns 19572080.5 ns 1.00
Dense(2000 => 2000)/cpu/reverse/Tracker/(2000, 128) 23282437 ns 23527900 ns 0.99
Dense(2000 => 2000)/cpu/reverse/ReverseDiff/(2000, 128) 24072118 ns 24288344 ns 0.99
Dense(2000 => 2000)/cpu/reverse/Flux/(2000, 128) 19628468 ns 19781013 ns 0.99
Dense(2000 => 2000)/cpu/reverse/Enzyme/(2000, 128) 6519275 ns 6537819 ns 1.00
Dense(2000 => 2000)/cpu/forward/NamedTuple/(2000, 128) 6533531 ns 6573838 ns 0.99
Dense(2000 => 2000)/cpu/forward/ComponentArray/(2000, 128) 6476609 ns 6536567 ns 0.99
Dense(2000 => 2000)/cpu/forward/Flux/(2000, 128) 6506898 ns 6508655 ns 1.00

This comment was automatically generated by workflow using github-action-benchmark.

Please sign in to comment.