diff --git a/.buildkite/pipeline.yml b/.buildkite/pipeline.yml index 519d0ecfb..617e97225 100644 --- a/.buildkite/pipeline.yml +++ b/.buildkite/pipeline.yml @@ -206,6 +206,7 @@ steps: cuda: "*" artifact_paths: - "tutorial_deps/*" + - "docs/build/**/*" env: DATADEPS_ALWAYS_ACCEPT: true JULIA_DEBUG: "Documenter" diff --git a/docs/src/index.md b/docs/src/index.md index ce2a2a339..a73264f4c 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -5,8 +5,8 @@ layout: home hero: name: LuxDL Docs - text: Elegant & Performant Deep Learning in JuliaLang - tagline: A Pure Julia Deep Learning Framework putting Correctness and Performance First + text: Elegant & Performant Scientific Machine Learning in JuliaLang + tagline: A Pure Julia Deep Learning Framework designed for Scientific Machine Learning actions: - theme: brand text: Tutorials diff --git a/docs/src/manual/distributed_utils.md b/docs/src/manual/distributed_utils.md index c0c68bb15..d1d98575c 100644 --- a/docs/src/manual/distributed_utils.md +++ b/docs/src/manual/distributed_utils.md @@ -10,16 +10,16 @@ DDP Training using `Lux.DistributedUtils` is a spiritual successor to ## Guide to Integrating DistributedUtils into your code -1. Initialize the respective backend with [`DistributedUtils.initialize`](@ref), by passing - in a backend type. It is important that you pass in the type, i.e. `NCCLBackend` and not - the object `NCCLBackend()`. +* Initialize the respective backend with [`DistributedUtils.initialize`](@ref), by passing + in a backend type. It is important that you pass in the type, i.e. `NCCLBackend` and not + the object `NCCLBackend()`. ```julia DistributedUtils.initialize(NCCLBackend) ``` -2. Obtain the backend via [`DistributedUtils.get_distributed_backend`](@ref) by passing in - the type of the backend (same note as last point applies here again). +* Obtain the backend via [`DistributedUtils.get_distributed_backend`](@ref) by passing in + the type of the backend (same note as last point applies here again). ```julia backend = DistributedUtils.get_distributed_backend(NCCLBackend) @@ -28,36 +28,36 @@ backend = DistributedUtils.get_distributed_backend(NCCLBackend) It is important that you use this function instead of directly constructing the backend, since there are certain internal states that need to be synchronized. -3. Next synchronize the parameters and states of the model. This is done by calling - [`DistributedUtils.synchronize!!`](@ref) with the backend and the respective input. +* Next synchronize the parameters and states of the model. This is done by calling + [`DistributedUtils.synchronize!!`](@ref) with the backend and the respective input. ```julia ps = DistributedUtils.synchronize!!(backend, ps) st = DistributedUtils.synchronize!!(backend, st) ``` -4. To split the data uniformly across the processes use - [`DistributedUtils.DistributedDataContainer`](@ref). Alternatively, one can manually - split the data. For the provided container to work - [`MLUtils.jl`](https://github.com/JuliaML/MLUtils.jl) must be installed and loaded. +* To split the data uniformly across the processes use + [`DistributedUtils.DistributedDataContainer`](@ref). Alternatively, one can manually + split the data. For the provided container to work + [`MLUtils.jl`](https://github.com/JuliaML/MLUtils.jl) must be installed and loaded. ```julia data = DistributedUtils.DistributedDataContainer(backend, data) ``` -5. Wrap the optimizer in [`DistributedUtils.DistributedOptimizer`](@ref) to ensure that the - optimizer is correctly synchronized across all processes before parameter updates. After - initializing the state of the optimizer, synchronize the state across all processes. +* Wrap the optimizer in [`DistributedUtils.DistributedOptimizer`](@ref) to ensure that the + optimizer is correctly synchronized across all processes before parameter updates. After + initializing the state of the optimizer, synchronize the state across all processes. ```julia opt = DistributedUtils.DistributedOptimizer(backend, opt) opt_state = Optimisers.setup(opt, ps) opt_state = DistributedUtils.synchronize!!(backend, opt_state) -``` + ``` -6. Finally change all logging and serialization code to trigger on - `local_rank(backend) == 0`. This ensures that only the master process logs and serializes - the model. +* Finally change all logging and serialization code to trigger on + `local_rank(backend) == 0`. This ensures that only the master process logs and serializes + the model. ## [GPU-Aware MPI](@id gpu-aware-mpi) @@ -108,4 +108,4 @@ And that's pretty much it! 1. Currently we don't run tests with CUDA or ROCM aware MPI, use those features at your own risk. We are working on adding tests for these features. 2. AMDGPU support is mostly experimental and causes deadlocks in certain situations, this is - being investigated. If you have a minimal reproducer for this, please open an issue. \ No newline at end of file + being investigated. If you have a minimal reproducer for this, please open an issue. diff --git a/docs/src/tutorials/index.md b/docs/src/tutorials/index.md index 1efdb8b2a..0092a7608 100644 --- a/docs/src/tutorials/index.md +++ b/docs/src/tutorials/index.md @@ -6,7 +6,7 @@ layout: page