---
license: llama3.1
language:
- en
base_model:
- microsoft/phi-4
pipeline_tag: text-generation
library_name: transformers
tags:
- text-generation-inference
- llama3.1
- phi4
- LlamaForCausalLM
- Corpus
- trl
model-index:
- name: Megatron-Opus-14B-2.0
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: wis-k/instruction-following-eval
      split: train
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 66.94
      name: averaged accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: SaylorTwift/bbh
      split: test
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 54.7
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: lighteval/MATH-Hard
      split: test
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 27.79
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      split: train
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 14.54
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 10.52
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 46.34
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FMegatron-Opus-14B-2.0
      name: Open LLM Leaderboard
---

![xfghxxfdghfdgh.gif](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/dRGKgIqsV2h7W04Rp4Yld.gif)

# **Megatron-Opus-14B-2.0 [ Exp ]**

[Megatron-Opus-14B-2.0 ] Exp finetuned from Microsoft's Phi-4 is a state-of-the-art open model developed with a focus on responsible problem solving and advanced reasoning capabilities. Built upon a diverse blend of synthetic datasets, carefully filtered public domain websites, and high-quality academic books and Q&A datasets, Megatron-Opus-14B-2.0 ensures that small, capable models are trained with datasets of exceptional depth and precision.

Megatron-Opus-14B-2.0 adopts a robust safety post-training approach using open-source and in-house synthetic datasets. This involves a combination of SFT (Supervised Fine-Tuning) and iterative DPO (Direct Preference Optimization) techniques, ensuring helpful and harmless outputs across various safety categories.


# **Dataset Info**

Megatron-Opus-14B-2.0 is fine-tuned on a carefully curated synthetic dataset generated using an advanced pipeline optimized for Chain of Thought (CoT) reasoning and Responsible Problem Breakdown (RPB) methodologies. This ensures that the model excels at:

- **Logical reasoning**  
- **Step-by-step problem-solving**  
- **Breaking down complex tasks into manageable parts**  

The dataset also emphasizes responsible decision-making and fairness in generating solutions.

# **Run with Transformers**

```python
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("prithivMLmods/Megatron-Opus-14B-2.0")
model = AutoModelForCausalLM.from_pretrained(
    "prithivMLmods/Megatron-Opus-14B-2.0",
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

input_text = "Explain the concept of black holes."
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids, max_new_tokens=64)
print(tokenizer.decode(outputs[0]))
```

For chat-style interactions, use `tokenizer.apply_chat_template`:

```python
messages = [
    {"role": "user", "content": "Explain the concept of black holes."},
]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to("cuda")

outputs = model.generate(**input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))
```

# **Intended Use**

Megatron-Opus-14B-2.0 is tailored for a wide range of applications, especially those involving **advanced reasoning**, **multilingual capabilities**, and **responsible problem-solving**. Its primary use cases include:

1. **Responsible Problem Solving**  
   - Breaking down complex problems into logical, actionable steps.  
   - Offering ethical, well-rounded solutions in academic and professional contexts.  

2. **Advanced Reasoning Tasks**  
   - Excelling in mathematics, logic, and scientific reasoning.  
   - Providing detailed explanations and systematic answers.  

3. **Content Generation**  
   - Assisting in generating high-quality content for various domains, including creative writing and technical documentation.  
   - Supporting marketers, writers, and educators with detailed and well-structured outputs.  

4. **Educational Support**  
   - Acting as a virtual tutor for students by generating practice questions, answers, and detailed explanations.  
   - Helping educators design learning material that promotes critical thinking and step-by-step problem-solving.  

5. **Customer Support & Dialogue Systems**  
   - Enabling chatbots and virtual assistants to provide accurate, helpful, and responsible responses.  
   - Enhancing customer service with reasoning-driven automation.  

# **Limitations**

Despite its strengths, Megatron-Opus-14B-2.0 has some limitations that users should be aware of:

1. **Bias and Fairness**  
   - While great effort has been made to minimize biases, users should critically assess the model’s output in sensitive scenarios to avoid unintended bias.  

2. **Contextual Interpretation**  
   - The model may occasionally misinterpret highly nuanced prompts or ambiguous contexts, leading to suboptimal responses.  

3. **Knowledge Cutoff**  
   - Megatron-Opus-14B-2.0’s knowledge is static and based on the data available at the time of training. It does not include real-time updates or information on recent developments.  

4. **Safety and Harmlessness**  
   - Despite post-training safety alignment, inappropriate or harmful outputs may still occur. Continuous monitoring and human oversight are advised when using the model in critical contexts.  

5. **Computational Requirements**  
   - Deploying Megatron-Opus-14B-2.0 efficiently may require substantial computational resources, particularly for large-scale deployments or real-time applications.
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/prithivMLmods__Megatron-Opus-14B-2.0-details)!
Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=prithivMLmods%2FMegatron-Opus-14B-2.0&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!

|      Metric       |Value (%)|
|-------------------|--------:|
|**Average**        |    36.80|
|IFEval (0-Shot)    |    66.94|
|BBH (3-Shot)       |    54.70|
|MATH Lvl 5 (4-Shot)|    27.79|
|GPQA (0-shot)      |    14.54|
|MuSR (0-shot)      |    10.52|
|MMLU-PRO (5-shot)  |    46.34|

