Skip to content

Commit

Permalink
Bump 3rdparty/Megatron-LM from 99f23d2 to 2da43ef (NVIDIA#558)
Browse files Browse the repository at this point in the history
Bumps [3rdparty/Megatron-LM](https://github.com/NVIDIA/Megatron-LM) from
`99f23d2` to `2da43ef`.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/2da43ef4c1b9e76f03b7567360cf7390e877f1b6"><code>2da43ef</code></a>
Merge branch 'mmodal_eval_in_folder' into 'main'</li>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/e51a3ac1dcd366f51bcb0339ecca31790c3cfcd1"><code>e51a3ac</code></a>
ADLR/megatron-lm!2491 - Move mmodal evaluation code to its own
folder</li>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/d3c585e90ebd5937243c8d4c9d5d5cf9d61665d6"><code>d3c585e</code></a>
Merge branch 'jbarker/pp_unfreeze' into 'main'</li>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/1468ab01c079d5e14888dda97d1c99d2cb62afb2"><code>1468ab0</code></a>
ADLR/megatron-lm!2285 - Support --freeze-LM and --freeze-ViT with ranks
that ...</li>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/cf25d44037af4e9d5ea723918823de9b2416a30c"><code>cf25d44</code></a>
Merge branch 'boxin/nvlm_ckpt_release' into 'main'</li>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/1da9dad62b97917caacb1fd271abaed403581caa"><code>1da9dad</code></a>
ADLR/megatron-lm!2494 - Add model checkpoint links</li>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/25b1f33035ad55eeae6b9a4367f987f1fac804dd"><code>25b1f33</code></a>
Merge branch 'helenn-rope-fusion-mem-layout' into 'main'</li>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/7bb53792831d80007789ff5c60bc1798cbd34548"><code>7bb5379</code></a>
ADLR/megatron-lm!2469 - Correct strides for bshd layout and revert RoPE
tests...</li>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/b8420a1909980aa3b6750f75b2d7ab8b23338948"><code>b8420a1</code></a>
Merge branch 'group_topk' into 'main'</li>
<li><a
href="https://github.com/NVIDIA/Megatron-LM/commit/d0df563d8739e4dfe2b0e90ba190ac389f165157"><code>d0df563</code></a>
ADLR/megatron-lm!1934 - Support Device-Limited Routing and Sequence
Auxiliary...</li>
<li>Additional commits viewable in <a
href="https://github.com/NVIDIA/Megatron-LM/compare/99f23d2f111d12b73b1fbf386c60517101ff8abe...2da43ef4c1b9e76f03b7567360cf7390e877f1b6">compare
view</a></li>
</ul>
</details>
<br />

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Danny <dreidenbach@nvidia.com>
  • Loading branch information
dependabot[bot] authored and nvdreidenbach committed Jan 2, 2025
1 parent 9cfc25f commit b30c5cd
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion 3rdparty/Megatron-LM
Submodule Megatron-LM updated 87 files
+1 −1 .gitlab/stages/00.pre.yml
+1 −0 examples/gpt3/gpt_config.yaml
+55 −50 examples/inference/README.md
+5 −5 examples/inference/gpt/gpt_batch_inference.py
+3 −3 examples/inference/t5/simple_t5_batch_inference.py
+2 −2 examples/multimodal/README.md
+137 −66 examples/multimodal/dataset_helpers.py
+0 −0 examples/multimodal/evaluation/evaluate_ai2d.py
+0 −0 examples/multimodal/evaluation/evaluate_chartqa.py
+0 −0 examples/multimodal/evaluation/evaluate_coco.py
+0 −0 examples/multimodal/evaluation/evaluate_mathvista.py
+6 −0 examples/multimodal/evaluation/evaluate_mmmu.py
+0 −0 examples/multimodal/evaluation/evaluate_ocrbench.py
+0 −0 examples/multimodal/evaluation/evaluate_textvqa.py
+0 −0 examples/multimodal/evaluation/evaluate_vqav2.py
+0 −0 examples/multimodal/evaluation/evaluation_datasets.py
+9 −2 examples/multimodal/nvlm/README.md
+1 −1 examples/multimodal/nvlm/pretrain_qwen20_72b_internvit_6b.sh
+1 −1 examples/multimodal/nvlm/pretrain_yi_34b_internvit_6b.sh
+1 −1 examples/multimodal/nvlm/run_text_generation_qwen20_72b_internvit_6b.sh
+2 −2 examples/multimodal/nvlm/run_text_generation_yi_34b_internvit_6b.sh
+1 −1 examples/multimodal/nvlm/sft_34b_internvit.sh
+1 −1 examples/multimodal/nvlm/sft_qwen20_72b_internvit_6b.sh
+1 −6 examples/multimodal/pretrain_mistral_clip.sh
+1 −1 examples/multimodal/run_text_generation.py
+1 −6 examples/multimodal/sft_mistral_clip.sh
+4 −13 examples/multimodal/text_generation_mistral_clip.sh
+2 −2 examples/multimodal/train.py
+2 −1 megatron/core/dist_checkpointing/mapping.py
+0 −2 megatron/core/dist_checkpointing/serialization.py
+13 −14 megatron/core/dist_checkpointing/validation.py
+21 −6 megatron/core/distributed/distributed_data_parallel.py
+29 −18 megatron/core/extensions/transformer_engine.py
+4 −29 megatron/core/inference/common_inference_params.py
+15 −8 megatron/core/inference/engines/mcore_engine.py
+2 −2 megatron/core/inference/inference_request.py
+35 −0 megatron/core/inference/sampling_params.py
+3 −3 megatron/core/inference/scheduler.py
+4 −4 megatron/core/inference/text_generation_controllers/encoder_decoder_text_generation_controller.py
+3 −398 megatron/core/inference/text_generation_controllers/simple_text_generation_controller.py
+400 −0 megatron/core/inference/text_generation_controllers/text_generation_controller.py
+48 −24 megatron/core/models/bert/bert_layer_specs.py
+16 −13 megatron/core/models/common/embeddings/rope_utils.py
+5 −2 megatron/core/models/multimodal/llava_model.py
+49 −24 megatron/core/optimizer/__init__.py
+21 −9 megatron/core/optimizer/clip_grads.py
+193 −102 megatron/core/optimizer/distrib_optimizer.py
+143 −86 megatron/core/optimizer/optimizer.py
+65 −0 megatron/core/optimizer/optimizer_config.py
+10 −0 megatron/core/pipeline_parallel/schedules.py
+80 −76 megatron/core/rerun_state_machine.py
+670 −212 megatron/core/transformer/cuda_graphs.py
+4 −1 megatron/core/transformer/moe/README.md
+115 −4 megatron/core/transformer/moe/moe_utils.py
+55 −26 megatron/core/transformer/moe/router.py
+4 −1 megatron/core/transformer/transformer_block.py
+32 −6 megatron/core/transformer/transformer_config.py
+4 −5 megatron/inference/text_generation/forward_step.py
+39 −8 megatron/training/arguments.py
+27 −12 megatron/training/checkpointing.py
+37 −13 megatron/training/training.py
+49 −11 megatron/training/utils.py
+4 −10 pretrain_vlm.py
+1 −0 ...sts/test_cases/gpt/gpt3_345m_nightly_dgx_a100_1N8G_mcore_tp2_pp2_ep2_te_4experts2parallel/model_config.yaml
+0 −1 ...s/test_cases/gpt/gpt3_mr_mcore_te_tp1_pp2_resume_torch_dist_rope_embeddings_dgx_a100_1N8G/model_config.yaml
+0 −1 tests/functional_tests/test_cases/gpt/gpt3_mr_mcore_te_tp1_pp2_rope_embeddings_dgx_a100_1N8G/model_config.yaml
+1 −0 tests/functional_tests/test_cases/gpt/gpt3_nightly_mcore_te_tp2_pp1_modelopt_distill_resume/model_config.yaml
+1 −0 ...imodal-llava/multimodal_llava_mr_mcore_te_tp4_pp1_freeze_vit_freeze_lm_dgx_a100_1N8G/golden_values_dev.json
+1 −0 ...imodal-llava/multimodal_llava_mr_mcore_te_tp4_pp1_freeze_vit_freeze_lm_dgx_a100_1N8G/golden_values_lts.json
+57 −0 .../multimodal-llava/multimodal_llava_mr_mcore_te_tp4_pp1_freeze_vit_freeze_lm_dgx_a100_1N8G/model_config.yaml
+1 −0 ...ava/multimodal_llava_mr_mcore_te_tp4_pp1_freeze_vit_freeze_lm_dist_opt_dgx_a100_1N8G/golden_values_dev.json
+1 −0 ...ava/multimodal_llava_mr_mcore_te_tp4_pp1_freeze_vit_freeze_lm_dist_opt_dgx_a100_1N8G/golden_values_lts.json
+58 −0 ...al-llava/multimodal_llava_mr_mcore_te_tp4_pp1_freeze_vit_freeze_lm_dist_opt_dgx_a100_1N8G/model_config.yaml
+1 −0 ...ional_tests/test_cases/t5/t5_220m_nightly_dgx_a100_1N8G_mcore_te_tp1_pp1_vp1_resume_torch/model_config.yaml
+1 −0 tests/functional_tests/test_cases/t5/t5_220m_nightly_dgx_a100_1N8G_mcore_te_tp2_pp1_vp1/model_config.yaml
+1 −0 ..._tests/test_cases/t5/t5_220m_nightly_dgx_a100_1N8G_mcore_te_tp2_pp1_vp1_sequence_parallel/model_config.yaml
+2 −0 tests/test_utils/recipes/multimodal-llava.yaml
+68 −0 tests/unit_tests/dist_checkpointing/test_flattened_resharding.py
+33 −0 tests/unit_tests/dist_checkpointing/test_serialization.py
+6 −8 tests/unit_tests/inference/engines/test_mcore_engine.py
+3 −3 tests/unit_tests/inference/test_common_inference_params.py
+2 −2 tests/unit_tests/inference/test_scheduler.py
+2 −2 tests/unit_tests/inference/text_generation_controllers/test_encoder_decoder_text_generation_controller.py
+13 −13 tests/unit_tests/inference/text_generation_controllers/test_simple_text_generation_controller.py
+47 −0 tests/unit_tests/test_optimizer.py
+44 −0 tests/unit_tests/transformer/moe/test_aux_loss.py
+59 −0 tests/unit_tests/transformer/moe/test_routers.py

0 comments on commit b30c5cd

Please sign in to comment.