Skip to content

Commit

Permalink
Update cudagraphs.md
Browse files Browse the repository at this point in the history
Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com>
  • Loading branch information
JimmyZhang12 authored Sep 17, 2024
1 parent e160751 commit 352a152
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions docs/source/performance/cudagraphs.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ As a demonstration we show that with a GPT3 20B model, cudagraphs can improve pe
### Usage
As of the NeMo 24.09 release, Cudagraphs is currently supported only for the pretraining of dense models.
To enable please add the following configs:

`model.enable_cuda_graph=True`

`model.use_te_rng_tracker=True`
Expand Down

0 comments on commit 352a152

Please sign in to comment.