Skip to content

Commit

Permalink
fix: move replicate rrule to LuxCore
Browse files Browse the repository at this point in the history
  • Loading branch information
@avik-pal
avik-pal committed Jul 17, 2024
1 parent 2ec21e2 commit 8ab1300
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 2 deletions.
2 changes: 1 addition & 1 deletion Project.toml
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "Lux"
uuid = "b2108857-7c20-44ae-9111-449ecde12c47"
authors = ["Avik Pal <avikpal@mit.edu> and contributors"]
version = "0.5.61-DEV"
version = "0.5.61"

[deps]
ADTypes = "47edcb42-4c32-4615-8424-f2b9edc5f35b"
Expand Down
1 change: 0 additions & 1 deletion src/chainrules.jl
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
# Non Differentiable Functions
CRC.@non_differentiable replicate(::Any) # TODO: move to LuxCore.jl
CRC.@non_differentiable compute_adaptive_pooling_dims(::Any, ::Any)
CRC.@non_differentiable istraining(::Any)
CRC.@non_differentiable _get_norm_except_dims(::Any, ::Any)
Expand Down

3 comments on commit 8ab1300

@avik-pal
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator register

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/111193

Tip: Release Notes

Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.

@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.

Tagging

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v0.5.61 -m "<description of version>" 8ab13004dcbaae295b4fe67857812f4dd05c4caf
git push origin v0.5.61

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Results

Benchmark suite Current: 8ab1300 Previous: 29e466f Ratio
Dense(2 => 2)/cpu/reverse/ReverseDiff (compiled)/(2, 128) 3646.75 ns 3708.125 ns 0.98
Dense(2 => 2)/cpu/reverse/Zygote/(2, 128) 7145 ns 7202.583333333333 ns 0.99
Dense(2 => 2)/cpu/reverse/Tracker/(2, 128) 20819 ns 20769 ns 1.00
Dense(2 => 2)/cpu/reverse/ReverseDiff/(2, 128) 9744 ns 9814.2 ns 0.99
Dense(2 => 2)/cpu/reverse/Flux/(2, 128) 9009.25 ns 8940.8 ns 1.01
Dense(2 => 2)/cpu/reverse/SimpleChains/(2, 128) 4483.25 ns 4521 ns 0.99
Dense(2 => 2)/cpu/reverse/Enzyme/(2, 128) 1160.5785714285714 ns 1162.1666666666667 ns 1.00
Dense(2 => 2)/cpu/forward/NamedTuple/(2, 128) 1106.374193548387 ns 1108.0264900662253 ns 1.00
Dense(2 => 2)/cpu/forward/ComponentArray/(2, 128) 1183.5891472868218 ns 1193.234375 ns 0.99
Dense(2 => 2)/cpu/forward/Flux/(2, 128) 1781.5081967213114 ns 1782.7857142857142 ns 1.00
Dense(2 => 2)/cpu/forward/SimpleChains/(2, 128) 179.9675599435825 ns 179.94575799721835 ns 1.00
Dense(20 => 20)/cpu/reverse/ReverseDiff (compiled)/(20, 128) 17242 ns 17322 ns 1.00
Dense(20 => 20)/cpu/reverse/Zygote/(20, 128) 16742 ns 16781 ns 1.00
Dense(20 => 20)/cpu/reverse/Tracker/(20, 128) 36698 ns 37254.5 ns 0.99
Dense(20 => 20)/cpu/reverse/ReverseDiff/(20, 128) 28192 ns 29044 ns 0.97
Dense(20 => 20)/cpu/reverse/Flux/(20, 128) 19867 ns 21280 ns 0.93
Dense(20 => 20)/cpu/reverse/SimpleChains/(20, 128) 17171 ns 17282 ns 0.99
Dense(20 => 20)/cpu/reverse/Enzyme/(20, 128) 4296.571428571428 ns 4315.142857142857 ns 1.00
Dense(20 => 20)/cpu/forward/NamedTuple/(20, 128) 3812.125 ns 3829.625 ns 1.00
Dense(20 => 20)/cpu/forward/ComponentArray/(20, 128) 3921 ns 3934.875 ns 1.00
Dense(20 => 20)/cpu/forward/Flux/(20, 128) 4738.857142857143 ns 4817.571428571428 ns 0.98
Dense(20 => 20)/cpu/forward/SimpleChains/(20, 128) 1665.1 ns 1654.1 ns 1.01
Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 3, 128) 46655226 ns 41551661 ns 1.12
Conv((3, 3), 3 => 3)/cpu/reverse/Zygote/(64, 64, 3, 128) 57930597 ns 58251044.5 ns 0.99
Conv((3, 3), 3 => 3)/cpu/reverse/Tracker/(64, 64, 3, 128) 109924433 ns 79896817.5 ns 1.38
Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff/(64, 64, 3, 128) 107221408 ns 85832471 ns 1.25
Conv((3, 3), 3 => 3)/cpu/reverse/Flux/(64, 64, 3, 128) 110343774 ns 76140849 ns 1.45
Conv((3, 3), 3 => 3)/cpu/reverse/SimpleChains/(64, 64, 3, 128) 11774281 ns 11969223 ns 0.98
Conv((3, 3), 3 => 3)/cpu/reverse/Enzyme/(64, 64, 3, 128) 7010907.5 ns 7080191 ns 0.99
Conv((3, 3), 3 => 3)/cpu/forward/NamedTuple/(64, 64, 3, 128) 7140666 ns 7193590 ns 0.99
Conv((3, 3), 3 => 3)/cpu/forward/ComponentArray/(64, 64, 3, 128) 7074380 ns 7073414 ns 1.00
Conv((3, 3), 3 => 3)/cpu/forward/Flux/(64, 64, 3, 128) 18143119 ns 12338787.5 ns 1.47
Conv((3, 3), 3 => 3)/cpu/forward/SimpleChains/(64, 64, 3, 128) 6442721 ns 6431957 ns 1.00
vgg16/cpu/reverse/Zygote/(32, 32, 3, 16) 699313956 ns 703246914 ns 0.99
vgg16/cpu/reverse/Zygote/(32, 32, 3, 64) 2575198933 ns 2570700164 ns 1.00
vgg16/cpu/reverse/Zygote/(32, 32, 3, 2) 135608581.5 ns 146137798 ns 0.93
vgg16/cpu/reverse/Tracker/(32, 32, 3, 16) 847786248 ns 915110115.5 ns 0.93
vgg16/cpu/reverse/Tracker/(32, 32, 3, 64) 3184845339 ns 3187561892 ns 1.00
vgg16/cpu/reverse/Tracker/(32, 32, 3, 2) 194058890 ns 208837724.5 ns 0.93
vgg16/cpu/reverse/Flux/(32, 32, 3, 16) 792650971 ns 690645735.5 ns 1.15
vgg16/cpu/reverse/Flux/(32, 32, 3, 64) 2860326129 ns 2812418266 ns 1.02
vgg16/cpu/reverse/Flux/(32, 32, 3, 2) 147897037 ns 147830644.5 ns 1.00
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 16) 174522325 ns 174776833 ns 1.00
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 64) 653262254.5 ns 658430709 ns 0.99
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 2) 34510126 ns 34866148 ns 0.99
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 16) 165585424.5 ns 165524659 ns 1.00
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 64) 641859459.5 ns 641598171.5 ns 1.00
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 2) 30120730 ns 30417511 ns 0.99
vgg16/cpu/forward/Flux/(32, 32, 3, 16) 186849485 ns 188258028 ns 0.99
vgg16/cpu/forward/Flux/(32, 32, 3, 64) 718949033 ns 847204472 ns 0.85
vgg16/cpu/forward/Flux/(32, 32, 3, 2) 35778969 ns 38306001.5 ns 0.93
Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 64, 128) 1225826604 ns 1222361111.5 ns 1.00
Conv((3, 3), 64 => 64)/cpu/reverse/Zygote/(64, 64, 64, 128) 1863262254.5 ns 1876739603.5 ns 0.99
Conv((3, 3), 64 => 64)/cpu/reverse/Tracker/(64, 64, 64, 128) 2300837521 ns 2318950176 ns 0.99
Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff/(64, 64, 64, 128) 2537744257 ns 2586064020 ns 0.98
Conv((3, 3), 64 => 64)/cpu/reverse/Flux/(64, 64, 64, 128) 1849655323.5 ns 1910576873.5 ns 0.97
Conv((3, 3), 64 => 64)/cpu/reverse/Enzyme/(64, 64, 64, 128) 324944297 ns 328082151.5 ns 0.99
Conv((3, 3), 64 => 64)/cpu/forward/NamedTuple/(64, 64, 64, 128) 318880308 ns 326373060 ns 0.98
Conv((3, 3), 64 => 64)/cpu/forward/ComponentArray/(64, 64, 64, 128) 318129939 ns 318939236 ns 1.00
Conv((3, 3), 64 => 64)/cpu/forward/Flux/(64, 64, 64, 128) 457445781 ns 364450794.5 ns 1.26
Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 1, 128) 11728322.5 ns 12110116.5 ns 0.97
Conv((3, 3), 1 => 1)/cpu/reverse/Zygote/(64, 64, 1, 128) 17883530 ns 18065994.5 ns 0.99
Conv((3, 3), 1 => 1)/cpu/reverse/Tracker/(64, 64, 1, 128) 19112033.5 ns 19378161 ns 0.99
Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff/(64, 64, 1, 128) 23812224 ns 24003824 ns 0.99
Conv((3, 3), 1 => 1)/cpu/reverse/Flux/(64, 64, 1, 128) 17859788.5 ns 18052727 ns 0.99
Conv((3, 3), 1 => 1)/cpu/reverse/SimpleChains/(64, 64, 1, 128) 1150700.5 ns 1157534 ns 0.99
Conv((3, 3), 1 => 1)/cpu/reverse/Enzyme/(64, 64, 1, 128) 2114613.5 ns 2066505 ns 1.02
Conv((3, 3), 1 => 1)/cpu/forward/NamedTuple/(64, 64, 1, 128) 2129556 ns 2078777.5 ns 1.02
Conv((3, 3), 1 => 1)/cpu/forward/ComponentArray/(64, 64, 1, 128) 2081342 ns 2081573 ns 1.00
Conv((3, 3), 1 => 1)/cpu/forward/Flux/(64, 64, 1, 128) 2077645 ns 2073578 ns 1.00
Conv((3, 3), 1 => 1)/cpu/forward/SimpleChains/(64, 64, 1, 128) 201388.5 ns 199862.5 ns 1.01
Dense(200 => 200)/cpu/reverse/ReverseDiff (compiled)/(200, 128) 290378 ns 293267 ns 0.99
Dense(200 => 200)/cpu/reverse/Zygote/(200, 128) 264189.5 ns 265485 ns 1.00
Dense(200 => 200)/cpu/reverse/Tracker/(200, 128) 362176.5 ns 369329 ns 0.98
Dense(200 => 200)/cpu/reverse/ReverseDiff/(200, 128) 405332 ns 408983 ns 0.99
Dense(200 => 200)/cpu/reverse/Flux/(200, 128) 272856 ns 275033 ns 0.99
Dense(200 => 200)/cpu/reverse/SimpleChains/(200, 128) 407235 ns 414623.5 ns 0.98
Dense(200 => 200)/cpu/reverse/Enzyme/(200, 128) 83264 ns 83776 ns 0.99
Dense(200 => 200)/cpu/forward/NamedTuple/(200, 128) 80980 ns 81302 ns 1.00
Dense(200 => 200)/cpu/forward/ComponentArray/(200, 128) 81150 ns 81362 ns 1.00
Dense(200 => 200)/cpu/forward/Flux/(200, 128) 86385.5 ns 87232 ns 0.99
Dense(200 => 200)/cpu/forward/SimpleChains/(200, 128) 104644 ns 104334.5 ns 1.00
Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 16, 128) 197008139.5 ns 196580814.5 ns 1.00
Conv((3, 3), 16 => 16)/cpu/reverse/Zygote/(64, 64, 16, 128) 325175404 ns 325601095 ns 1.00
Conv((3, 3), 16 => 16)/cpu/reverse/Tracker/(64, 64, 16, 128) 390870470 ns 422107461.5 ns 0.93
Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff/(64, 64, 16, 128) 467902030.5 ns 460786759.5 ns 1.02
Conv((3, 3), 16 => 16)/cpu/reverse/Flux/(64, 64, 16, 128) 383951129 ns 376576919 ns 1.02
Conv((3, 3), 16 => 16)/cpu/reverse/SimpleChains/(64, 64, 16, 128) 346324394 ns 330132293 ns 1.05
Conv((3, 3), 16 => 16)/cpu/reverse/Enzyme/(64, 64, 16, 128) 44103971 ns 44296074 ns 1.00
Conv((3, 3), 16 => 16)/cpu/forward/NamedTuple/(64, 64, 16, 128) 44167839 ns 44477840 ns 0.99
Conv((3, 3), 16 => 16)/cpu/forward/ComponentArray/(64, 64, 16, 128) 43953644 ns 44123475 ns 1.00
Conv((3, 3), 16 => 16)/cpu/forward/Flux/(64, 64, 16, 128) 50064478 ns 57384877 ns 0.87
Conv((3, 3), 16 => 16)/cpu/forward/SimpleChains/(64, 64, 16, 128) 28998067 ns 28060856.5 ns 1.03
Dense(2000 => 2000)/cpu/reverse/ReverseDiff (compiled)/(2000, 128) 19020127 ns 18956391 ns 1.00
Dense(2000 => 2000)/cpu/reverse/Zygote/(2000, 128) 19628854.5 ns 19593936 ns 1.00
Dense(2000 => 2000)/cpu/reverse/Tracker/(2000, 128) 23296306 ns 23259183.5 ns 1.00
Dense(2000 => 2000)/cpu/reverse/ReverseDiff/(2000, 128) 24078356 ns 24171564 ns 1.00
Dense(2000 => 2000)/cpu/reverse/Flux/(2000, 128) 19825268.5 ns 19844960.5 ns 1.00
Dense(2000 => 2000)/cpu/reverse/Enzyme/(2000, 128) 6589612 ns 6539285 ns 1.01
Dense(2000 => 2000)/cpu/forward/NamedTuple/(2000, 128) 6625445 ns 6578089 ns 1.01
Dense(2000 => 2000)/cpu/forward/ComponentArray/(2000, 128) 6554008 ns 6592691 ns 0.99
Dense(2000 => 2000)/cpu/forward/Flux/(2000, 128) 6574531 ns 6544315 ns 1.00

This comment was automatically generated by workflow using github-action-benchmark.

Please sign in to comment.