63 lines
3.6 KiB
Markdown
63 lines
3.6 KiB
Markdown
---
|
|
library_name: transformers
|
|
license: llama3.1
|
|
language:
|
|
- en
|
|
base_model:
|
|
- deepseek-ai/DeepSeek-R1-Distill-Llama-8B
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- llama
|
|
- deepseek
|
|
---
|
|
# **Llama-8B-Distill-CoT**
|
|
|
|
Llama-8B-Distill-CoT is based on the *Llama [ KT ]* model, distilled by DeepSeek-R1-Distill-Llama-8B. It has been fine-tuned on the long chain-of-thought reasoning model and specialized datasets, focusing on chain-of-thought (CoT) reasoning for problem-solving. This model is optimized for tasks requiring logical reasoning, detailed explanations, and multi-step problem-solving, making it ideal for applications such as instruction-following, text generation, and complex reasoning tasks.
|
|
|
|
|
|
# **Use with transformers**
|
|
|
|
Starting with `transformers >= 4.43.0` onward, you can run conversational inference using the Transformers `pipeline` abstraction or by leveraging the Auto classes with the `generate()` function.
|
|
|
|
Make sure to update your transformers installation via `pip install --upgrade transformers`.
|
|
|
|
```python
|
|
import transformers
|
|
import torch
|
|
|
|
model_id = "prithivMLmods/Llama-8B-Distill-CoT"
|
|
|
|
pipeline = transformers.pipeline(
|
|
"text-generation",
|
|
model=model_id,
|
|
model_kwargs={"torch_dtype": torch.bfloat16},
|
|
device_map="auto",
|
|
)
|
|
|
|
messages = [
|
|
{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
|
|
{"role": "user", "content": "Who are you?"},
|
|
]
|
|
|
|
outputs = pipeline(
|
|
messages,
|
|
max_new_tokens=256,
|
|
)
|
|
print(outputs[0]["generated_text"][-1])
|
|
```
|
|
### **Intended Use:**
|
|
1. **Instruction-Following:** The model is designed to handle detailed instructions, making it ideal for virtual assistants, automation tools, and educational platforms.
|
|
2. **Problem-Solving:** Its fine-tuning on chain-of-thought (CoT) reasoning allows it to tackle multi-step problem-solving in domains such as mathematics, logic, and programming.
|
|
3. **Text Generation:** Capable of generating coherent and contextually relevant content, it is suitable for creative writing, documentation, and report generation.
|
|
4. **Education and Training:** Provides step-by-step explanations and logical reasoning, making it a useful tool for teaching and learning.
|
|
5. **Research and Analysis:** Supports researchers and professionals by generating detailed analyses and structured arguments for complex topics.
|
|
6. **Programming Assistance:** Helps in generating, debugging, and explaining code, as well as creating structured outputs like JSON or XML.
|
|
|
|
### **Limitations:**
|
|
1. **Resource Intensive:** Requires high computational resources to run efficiently, which may limit accessibility for small-scale deployments.
|
|
2. **Hallucination Risk:** May generate incorrect or misleading information, especially when handling ambiguous or poorly framed prompts.
|
|
3. **Domain-Specific Gaps:** While fine-tuned for reasoning, it may not perform well in specialized domains outside its training data.
|
|
4. **Bias in Training Data:** The model's responses can reflect biases present in the datasets it was trained on, potentially leading to biased or inappropriate outputs.
|
|
5. **Dependence on Input Quality:** Performance heavily depends on clear, structured inputs. Ambiguous or vague queries can result in suboptimal outputs.
|
|
6. **Limited Real-Time Context:** The model cannot access real-time information or updates beyond its training data, potentially affecting its relevance for time-sensitive queries.
|
|
7. **Scalability for Long-Context:** While capable of multi-step reasoning, its ability to handle extremely long or complex contexts may be limited compared to larger, more specialized models. |