初始化项目,由ModelHub XC社区提供模型
Model: build-small-hackathon/mind-of-tashi-micro-grpo-gguf Source: Original Platform
This commit is contained in:
37
.gitattributes
vendored
Normal file
37
.gitattributes
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
mind-of-tashi-micro-grpo-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
mind-of-tashi-micro-grpo-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
75
README.md
Normal file
75
README.md
Normal file
@@ -0,0 +1,75 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: build-small-hackathon/mind-of-tashi-micro-grpo
|
||||
tags:
|
||||
- gguf
|
||||
- llama-cpp
|
||||
- qwen3moe
|
||||
- reasoning
|
||||
- game
|
||||
- bilingual
|
||||
- grpo
|
||||
language:
|
||||
- en
|
||||
- hi
|
||||
- sa
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# The Mind of Tashi — micro student (GRPO, GGUF)
|
||||
|
||||
The GRPO-trained student exported to **GGUF** for llama.cpp. Drop-in
|
||||
replacement for the SFT GGUF in the playable
|
||||
[Space](https://huggingface.co/spaces/build-small-hackathon/mind-of-tashi)
|
||||
after an A/B (winning the game is not enough — the mind-scroll prose must
|
||||
hold up). Transformers source:
|
||||
[`…/mind-of-tashi-micro-grpo`](https://huggingface.co/build-small-hackathon/mind-of-tashi-micro-grpo).
|
||||
|
||||
> **Build status:** this GGUF is **built at push time** from the GRPO
|
||||
> checkpoint — it does not exist as a by-product of training. Use the exact
|
||||
> same recipe as the SFT GGUF.
|
||||
|
||||
## Files (after build)
|
||||
|
||||
| File | Approx size | Use |
|
||||
|---|---|---|
|
||||
| `mind-of-tashi-micro-grpo-Q4_K_M.gguf` | ~256 MB | deployed candidate |
|
||||
| `mind-of-tashi-micro-grpo-f16.gguf` | ~786 MB | zero-loss reference |
|
||||
|
||||
## Build recipe (no compiled binary needed)
|
||||
|
||||
1. Download the GRPO transformers checkpoint **with `chat_template.jinja`**
|
||||
(a missing template silently yields a garbage GGUF).
|
||||
2. `python convert_hf_to_gguf.py <ckpt> --outtype f16` → f16 GGUF.
|
||||
3. Quantise via the `llama-cpp-python` C binding:
|
||||
```python
|
||||
import ctypes, llama_cpp
|
||||
p = llama_cpp.llama_model_quantize_default_params()
|
||||
p.ftype = 15 # LLAMA_FTYPE_MOSTLY_Q4_K_M
|
||||
llama_cpp.llama_model_quantize(b"in-f16.gguf", b"out-Q4_K_M.gguf", ctypes.byref(p))
|
||||
```
|
||||
4. Grade via the format gate through `llama-cpp-python` (the real deploy path);
|
||||
ship Q4 if it clears ≥15/20 and stays within ~5 ladder points of f16.
|
||||
|
||||
### ⚠️ `norm_topk_prob` — required for llama.cpp
|
||||
|
||||
Inherited `norm_topk_prob=true` from SFT; llama.cpp's `qwen3moe` graph
|
||||
hardcodes `norm_w=true` and a mismatched checkpoint produces garbage on every
|
||||
llama.cpp runtime. (See the SFT GGUF card.)
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from llama_cpp import Llama
|
||||
|
||||
llm = Llama.from_pretrained(
|
||||
repo_id="build-small-hackathon/mind-of-tashi-micro-grpo-gguf",
|
||||
filename="mind-of-tashi-micro-grpo-Q4_K_M.gguf",
|
||||
n_ctx=4096, n_gpu_layers=0, logits_all=True,
|
||||
)
|
||||
```
|
||||
|
||||
## Part of the bundle
|
||||
|
||||
Game Space · self-play dataset · SFT model + GGUF · OpenEnv gym ·
|
||||
GRPO model + **GGUF (this)** — all under `build-small-hackathon/mind-of-tashi-*`.
|
||||
3
mind-of-tashi-micro-grpo-Q4_K_M.gguf
Normal file
3
mind-of-tashi-micro-grpo-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:26f6b1b6c48aecc0218e903680b334eb4b86a447f72330551d67c067c9a45a57
|
||||
size 255625344
|
||||
3
mind-of-tashi-micro-grpo-f16.gguf
Normal file
3
mind-of-tashi-micro-grpo-f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d19d66671e4e01c5adf9ccb5ca9b55822aeeff5eb8dba27507c493fe555992fa
|
||||
size 786273920
|
||||
Reference in New Issue
Block a user