106 lines
3.3 KiB
Markdown
106 lines
3.3 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
- zh
|
||
|
|
tags:
|
||
|
|
- qwen3
|
||
|
|
- qwen
|
||
|
|
- causal-lm
|
||
|
|
- triton
|
||
|
|
- fine-tune
|
||
|
|
- safetensors
|
||
|
|
- bfloat16
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
base_model: Qwen/Qwen3-8B
|
||
|
|
---
|
||
|
|
> Note:This project represents independent research conducted on personal compute resources (rented from Modal.com) and is not associated with my employer or organization
|
||
|
|
# Qwen3-8B-Triton-Finetune
|
||
|
|
|
||
|
|
A fine-tuned version of **Qwen3-8B** (the latest generation of Qwen large language models by Alibaba Cloud), further trained using a **Triton-based** fine-tuning pipeline. This model retains the strong reasoning and instruction-following capabilities of Qwen3-8B while adding task-specific adaptations via custom Triton kernels.
|
||
|
|
|
||
|
|
## Model Details
|
||
|
|
|
||
|
|
- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
|
||
|
|
- **Architecture**: Qwen3ForCausalLM
|
||
|
|
- **Parameters**: 8.19B (BF16)
|
||
|
|
- **Hidden Size**: 4096
|
||
|
|
- **Intermediate Size**: 12288
|
||
|
|
- **Attention Heads**: 32 (8 KV heads, grouped-query attention)
|
||
|
|
- **Layers**: 36 (full attention, no sliding window)
|
||
|
|
- **Max Position Embeddings**: 40,960 tokens
|
||
|
|
- **Vocabulary Size**: 151,936
|
||
|
|
- **Attention Mechanism**: RoPE (Rotary Position Embeddings, theta=1,000,000)
|
||
|
|
- **Activation**: SiLU (SwigLU)
|
||
|
|
- **Precision**: bfloat16
|
||
|
|
- **Chat Template**: im_start/im_end format with tool calling support
|
||
|
|
|
||
|
|
## Capabilities
|
||
|
|
|
||
|
|
- **Text generation & completion**: General-purpose language understanding and generation
|
||
|
|
- **Instruction following**: Fine-tuned for chat and instructions
|
||
|
|
- **Multi-step tool calling**: Built-in tool/function calling support via the chat template
|
||
|
|
- **Extended context**: Supports up to ~40K tokens of context
|
||
|
|
- **Reasoning**: Supports think/reasoning blocks (`<think>...</think>`) in generation
|
||
|
|
|
||
|
|
## Files
|
||
|
|
|
||
|
|
| File | Description |
|
||
|
|
|------|-------------|
|
||
|
|
| `config.json` | Model architecture configuration |
|
||
|
|
| `generation_config.json` | Generation parameters |
|
||
|
|
| `model-*.safetensors` | Model weights (sharded across 4 files) |
|
||
|
|
| `model.safetensors.index.json` | Weight shard index |
|
||
|
|
| `tokenizer.json` | Tokenizer |
|
||
|
|
| `tokenizer_config.json` | Tokenizer configuration |
|
||
|
|
| `vocab.json` | Vocabulary |
|
||
|
|
| `merges.txt` | BPE merges |
|
||
|
|
| `added_tokens.json` | Special/added tokens |
|
||
|
|
| `special_tokens_map.json` | Special token mapping |
|
||
|
|
| `chat_template.jinja` | Chat template (Jinja) |
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
"edwixx/qwen3-8b-triton-finetune",
|
||
|
|
torch_dtype="bfloat16",
|
||
|
|
device_map="auto"
|
||
|
|
)
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("edwixx/qwen3-8b-triton-finetune")
|
||
|
|
|
||
|
|
messages = [
|
||
|
|
{"role": "system", "content": "You are a helpful assistant."},
|
||
|
|
{"role": "user", "content": "Explain what fine-tuning with Triton means."}
|
||
|
|
]
|
||
|
|
|
||
|
|
text = tokenizer.apply_chat_template(
|
||
|
|
messages,
|
||
|
|
tokenize=False,
|
||
|
|
add_generation_prompt=True
|
||
|
|
)
|
||
|
|
|
||
|
|
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||
|
|
outputs = model.generate(**inputs, max_new_tokens=512)
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
## License
|
||
|
|
|
||
|
|
Apache 2.0 (inherited from Qwen3-8B).
|
||
|
|
|
||
|
|
## Citation
|
||
|
|
|
||
|
|
```bibtex
|
||
|
|
@misc{edwixx-qwen3-8b-triton-finetune,
|
||
|
|
author = {Anurag Kanade},
|
||
|
|
title = {qwen3-8b-triton-finetune},
|
||
|
|
year = {2026},
|
||
|
|
publisher = {Hugging Face},
|
||
|
|
journal = {Hugging Face Hub},
|
||
|
|
howpublished = {\url{https://huggingface.co/edwixx/qwen3-8b-triton-finetune}}
|
||
|
|
}
|
||
|
|
```
|