44 lines
1.8 KiB
Markdown
44 lines
1.8 KiB
Markdown
---
|
|
license: mit
|
|
license_link: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B/blob/main/LICENSE
|
|
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
|
|
base_model_relation: finetune
|
|
finetuned_by: boatbomber
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- chat
|
|
- reasoning
|
|
- roblox
|
|
- luau
|
|
language:
|
|
- en
|
|
datasets:
|
|
- boatbomber/roblox-info-dump
|
|
- boatbomber/the-luau-stack
|
|
---
|
|
|
|
# R1-Distill-Qwen-14B-Roblox-Luau
|
|
|
|
A fine tune of [deepseek-ai/DeepSeek-R1-Distill-Qwen-14B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B) using [boatbomber/roblox-info-dump](https://huggingface.co/datasets/boatbomber/roblox-info-dump) and [boatbomber/the-luau-stack](https://huggingface.co/datasets/boatbomber/the-luau-stack) for Roblox domain knowledge.
|
|
|
|
Recommended inference settings:
|
|
|
|
| Parameter | Value | Notes |
|
|
| --------- | ----- | ----- |
|
|
| System Prompt | `You are an expert Roblox developer and Luau software engineer.` | Model was fine tuned with this prompt. |
|
|
| temperature | `0.5-0.7` | Underlying R1 Distill uses this. I've found best results with `0.55`. |
|
|
| top_p | `0.95` | Underlying R1 Distill uses this. |
|
|
|
|
Quantization done using [Unsloth](https://docs.unsloth.ai/).
|
|
|
|
Available quants:
|
|
|
|
| Quant | Size | Notes |
|
|
| ----- | ---- | ----- |
|
|
| F16 | 29.55GB | Retains 100% accuracy. Slow and memory hungry. |
|
|
| Q8_O | 15.70GB | High resource use, but generally acceptable. Use when accuracy is crucial. |
|
|
| Q6_K | 12.12GB | Uses Q6_K for all tensors. Good for high end GPUs. |
|
|
| Q5_K_M | 10.51GB | **Recommended.** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K |
|
|
| Q4_K_M | 8.99GB | **Recommended.** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K |
|
|
| Q3_K_M | 7.34GB | Uses Q4_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else Q3_K. Quality is noticeably degraded. |
|