110 lines
3.8 KiB
Markdown
110 lines
3.8 KiB
Markdown
---
|
||
license: apache-2.0
|
||
datasets:
|
||
- agentica-org/DeepScaleR-Preview-Dataset
|
||
base_model:
|
||
- Qwen/Qwen3-4B
|
||
language:
|
||
- en
|
||
pipeline_tag: text-generation
|
||
library_name: transformers
|
||
tags:
|
||
- moe
|
||
- text-generation-inference
|
||
- code
|
||
- deepscale
|
||
- math
|
||
---
|
||
|
||
|
||
|
||
# Segue-Qwen3\_DeepScaleR-Preview
|
||
|
||
> Segue-Qwen3\_DeepScaleR-Preview is an experimental fine-tuned variant of the Qwen3-4B model architecture. It is trained on the DeepScaleR-Preview dataset—comprising high-quality mathematical reasoning problems—to achieve exceptional performance in symbolic, mathematical, and logical tasks with lightweight computational requirements.
|
||
|
||
## Key Features
|
||
|
||
1. Precision Reasoning with DeepScaleR-Preview Dataset
|
||
Fine-tuned on approximately 40,000 curated math problem-answer pairs sourced from:
|
||
|
||
* AIME (1984–2023)
|
||
* AMC (pre-2023)
|
||
* Omni-MATH
|
||
This enables superior symbolic manipulation and step-by-step logical deduction.
|
||
|
||
2. Lightweight Code Understanding
|
||
Capable of interpreting and generating correct code in Python, C++, and other logic-intensive languages with an emphasis on problem-solving and structured thought.
|
||
|
||
3. Structured Output Formatting
|
||
Outputs are designed to be well-formatted in Markdown, JSON, LaTeX, or tables—ideal for technical documentation, math notebooks, and data workflows.
|
||
|
||
4. Instruction-Following Accuracy
|
||
Strong multi-step instruction adherence, particularly for STEM domains. Ensures continuity, factual correctness, and process transparency in reasoning chains.
|
||
|
||
5. Multilingual Capabilities
|
||
Supports over 20 languages for mathematical and logical reasoning, technical instruction translation, and cross-lingual academic support.
|
||
|
||
6. Efficient 4B Architecture
|
||
Built on the Qwen3-4B base model to balance performance and scalability. Runs efficiently on mid-range GPUs while delivering high-accuracy inference.
|
||
|
||
## Quickstart with Transformers
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
model_name = "prithivMLmods/Segue-Qwen3_DeepScaleR-Preview"
|
||
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_name,
|
||
torch_dtype="auto",
|
||
device_map="auto"
|
||
)
|
||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||
|
||
prompt = "Solve for x: 5(x - 2) = 3x + 4, showing all steps clearly."
|
||
|
||
messages = [
|
||
{"role": "system", "content": "You are a precise mathematical assistant trained on DeepScaleR-Preview dataset."},
|
||
{"role": "user", "content": prompt}
|
||
]
|
||
|
||
text = tokenizer.apply_chat_template(
|
||
messages,
|
||
tokenize=False,
|
||
add_generation_prompt=True
|
||
)
|
||
|
||
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
||
|
||
generated_ids = model.generate(
|
||
**model_inputs,
|
||
max_new_tokens=512
|
||
)
|
||
generated_ids = [
|
||
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
||
]
|
||
|
||
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
||
print(response)
|
||
```
|
||
|
||
## Intended Use
|
||
|
||
* Step-by-step mathematical problem solving
|
||
* Symbolic computation and logic derivation
|
||
* Code generation and correction in technical environments
|
||
* Automated LaTeX/Markdown/JSON generation for education and documentation
|
||
* Academic tutoring and educational assistants
|
||
* Multilingual reasoning and translation of structured content
|
||
|
||
## Limitations
|
||
|
||
* Less suitable for open-domain conversation or creative writing
|
||
* Smaller context window compared to large-scale LLMs
|
||
* May be sensitive to token formatting in edge-case symbolic prompts
|
||
* Could underperform on intentionally adversarial logic inputs
|
||
|
||
## References
|
||
|
||
1. Qwen2.5 Technical Report – [https://arxiv.org/pdf/2412.15115](https://arxiv.org/pdf/2412.15115)
|
||
2. YaRN: Context Window Extension for LLMs – [https://arxiv.org/pdf/2309.00071](https://arxiv.org/pdf/2309.00071) |