From 4cd08dc59239480516adec28c77e46de3b973abe Mon Sep 17 00:00:00 2001 From: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Date: Tue, 26 Aug 2025 10:33:40 +0300 Subject: [PATCH] model: Support nvidia/Llama-3_1-Nemotron-Ultra-253B-v1 (#9301) --- docs/supported_models/generative_models.md | 1 + test/srt/models/test_generation_models.py | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/docs/supported_models/generative_models.md b/docs/supported_models/generative_models.md index 3647e56e0..59fee1517 100644 --- a/docs/supported_models/generative_models.md +++ b/docs/supported_models/generative_models.md @@ -52,3 +52,4 @@ in the GitHub search bar. | **Granite 3.0, 3.1** (IBM) | `ibm-granite/granite-3.1-8b-instruct` | IBM's open dense foundation models optimized for reasoning, code, and business AI use cases. Integrated with Red Hat and watsonx systems. | | **Granite 3.0 MoE** (IBM) | `ibm-granite/granite-3.0-3b-a800m-instruct` | IBM’s Mixture-of-Experts models offering strong performance with cost-efficiency. MoE expert routing designed for enterprise deployment at scale. | | **Llama Nemotron Super** (v1, v1.5, NVIDIA) | `nvidia/Llama-3_3-Nemotron-Super-49B-v1`, `nvidia/Llama-3_3-Nemotron-Super-49B-v1_5` | The [NVIDIA Nemotron](https://www.nvidia.com/en-us/ai-data-science/foundation-models/nemotron/) family builds on the strongest open models in the ecosystem by enhancing them with greater accuracy, efficiency, and transparency using NVIDIA open synthetic datasets, advanced techniques, and tools. This enables the creation of practical, right-sized, and high-performing AI agents. | +| **Llama Nemotron Ultra** (v1, NVIDIA) | `nvidia/Llama-3_1-Nemotron-Ultra-253B-v1` | The [NVIDIA Nemotron](https://www.nvidia.com/en-us/ai-data-science/foundation-models/nemotron/) family builds on the strongest open models in the ecosystem by enhancing them with greater accuracy, efficiency, and transparency using NVIDIA open synthetic datasets, advanced techniques, and tools. This enables the creation of practical, right-sized, and high-performing AI agents. | diff --git a/test/srt/models/test_generation_models.py b/test/srt/models/test_generation_models.py index fa55de947..6d79d35aa 100644 --- a/test/srt/models/test_generation_models.py +++ b/test/srt/models/test_generation_models.py @@ -83,6 +83,12 @@ ALL_MODELS = [ trust_remote_code=True, skip_long_prompt=True, ), + ModelCase( + "nvidia/Llama-3_1-Nemotron-Ultra-253B-v1", + tp_size=8, + trust_remote_code=True, + skip_long_prompt=True, + ), ] TORCH_DTYPES = [torch.float16]