commit 86c6307eefe8a735df4a5336ab7d08978fc295f3 Author: ModelHub XC Date: Sun Aug 30 08:03:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: ermiaazarkhalili/Qwen3-4B-SFT-Fable5-Glint-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..5b67603 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,38 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +qwen3-4b-sft-fable5-glint.q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text +qwen3-4b-sft-fable5-glint.q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text +qwen3-4b-sft-fable5-glint.q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..6719581 --- /dev/null +++ b/README.md @@ -0,0 +1,94 @@ +--- +base_model: + - unsloth/Qwen3-4B +library_name: gguf +pipeline_tag: text-generation +tags: + - gguf + - llama.cpp + - quantized + - unsloth + - lora + - trl + - sft +--- + +# Qwen3-4B-SFT-Fable5-Glint-GGUF + +GGUF quantizations of a LoRA fine-tune of [`unsloth/Qwen3-4B`](https://huggingface.co/unsloth/Qwen3-4B), supervised fine-tuned on `ermiaazarkhalili/Fable-5-Glint-Clean` (private). + +Quantized from [`ermiaazarkhalili/Qwen3-4B-SFT-Fable5-Glint`](https://huggingface.co/ermiaazarkhalili/Qwen3-4B-SFT-Fable5-Glint). See that repository for the full-precision weights. + +| | | +| --- | --- | +| **Base model** | [`unsloth/Qwen3-4B`](https://huggingface.co/unsloth/Qwen3-4B) | +| **Training data** | `ermiaazarkhalili/Fable-5-Glint-Clean` (private) | +| **Method** | LoRA supervised fine-tuning via [Unsloth](https://github.com/unslothai/unsloth) + [TRL](https://github.com/huggingface/trl) | + +## Available quantizations + +| File | Size | +| --- | --- | +| `qwen3-4b-sft-fable5-glint.q4_k_m.gguf` | 2.50 GB | +| `qwen3-4b-sft-fable5-glint.q5_k_m.gguf` | 2.89 GB | +| `qwen3-4b-sft-fable5-glint.q8_0.gguf` | 4.28 GB | + +## Usage + +### llama.cpp + +```bash +huggingface-cli download ermiaazarkhalili/Qwen3-4B-SFT-Fable5-Glint-GGUF qwen3-4b-sft-fable5-glint.q4_k_m.gguf --local-dir . +llama-cli -m qwen3-4b-sft-fable5-glint.q4_k_m.gguf -p "Explain gradient checkpointing in two sentences." -n 256 +``` + +### Ollama + +```bash +echo 'FROM ./qwen3-4b-sft-fable5-glint.q4_k_m.gguf' > Modelfile +ollama create qwen3-4b-sft-fable5-glint-gguf -f Modelfile +ollama run qwen3-4b-sft-fable5-glint-gguf +``` + +## Training configuration + +| Setting | Value | +| --- | --- | +| LoRA rank (r) | 16 | +| LoRA alpha | 16 | +| Learning rate | 0.0002 | +| Epochs | 3 | +| Effective batch size | 8 (2 x 4 grad accum) | +| Max sequence length | 4096 | +| Base precision | 4-bit (QLoRA) | +| Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` | + +## Observed training loss + +Measured from our SLURM logs for this configuration. These are training-loss +observations only — no downstream benchmark evaluation has been run on this +model, so they should not be read as a quality claim. + +| SLURM job | Steps | First loss | Final loss | +| --- | --- | --- | --- | +| unlabelled | 1,554 | 2.3999 | 0.9811 | + +## Limitations + +- No benchmark evaluation has been run on this checkpoint. The only reported + numbers are training-loss observations. +- Inherits the biases, knowledge cutoff and failure modes of the base model. +- Fine-tuned on a single instruction-following dataset; behaviour outside that + distribution is untested. +- LoRA adapters were merged into the base weights, so the merged model cannot + be detached from this fine-tune. + +## Reproducing + +Trained by `notebooks/fable_distillation_qwen3-4b_fable-glint_unsloth.ipynb`, executed non-interactively with +papermill on a SLURM H100 partition (Unsloth + TRL, LoRA). + +--- + +*Card generated from the training run's own configuration and logs by* +*`scripts/generate_hub_model_card.py`.* diff --git a/qwen3-4b-sft-fable5-glint.q4_k_m.gguf b/qwen3-4b-sft-fable5-glint.q4_k_m.gguf new file mode 100644 index 0000000..22c51eb --- /dev/null +++ b/qwen3-4b-sft-fable5-glint.q4_k_m.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3e0e5e2e78ae547721c7d36dcf1a03d3cdd450bd808d6e8f3727da3bcdc9630e +size 2497280640 diff --git a/qwen3-4b-sft-fable5-glint.q5_k_m.gguf b/qwen3-4b-sft-fable5-glint.q5_k_m.gguf new file mode 100644 index 0000000..5888ddb --- /dev/null +++ b/qwen3-4b-sft-fable5-glint.q5_k_m.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aa6b1ed3a7800ceb40cb0d2dc32795ea1c8880d957062b766d37826c37d9de57 +size 2889513600 diff --git a/qwen3-4b-sft-fable5-glint.q8_0.gguf b/qwen3-4b-sft-fable5-glint.q8_0.gguf new file mode 100644 index 0000000..3d7406b --- /dev/null +++ b/qwen3-4b-sft-fable5-glint.q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5e367f9c0650864e164c4cba89a5321609c38677b4ad3affe6aad673c7912cba +size 4280405120