初始化项目，由ModelHub XC社区提供模型

Model: perfectPresentation/rcrc-chat-v5-gemma-1b-cpt-sft Source: Original Platform
2026-08-09 04:57:16 +08:00
commit b55d3bb9fd
13 changed files with 416 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,44 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+gguf/rcrc-v5-gemma-1b-cpt-sft-F16.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/rcrc-v5-gemma-1b-cpt-sft-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/rcrc-v5-gemma-1b-cpt-sft-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+rcrc-v5-gemma-1b-cpt-sft-F16.gguf filter=lfs diff=lfs merge=lfs -text
+rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+rcrc-v5-gemma-1b-cpt-sft-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+rcrc-v5-gemma-1b-cpt-sft-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
--- a/17
+++ b/17
@@ -0,0 +1,17 @@
+FROM ./rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf
+
+PARAMETER temperature 0.5
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.15
+PARAMETER stop "<end_of_turn>"
+PARAMETER stop "<start_of_turn>"
+
+SYSTEM """أنت مساعد للهيئة الملكية لمدينة الرياض. تجيب على استفسارات المستخدمين عن خدمات وبرامج ومشاريع وأنظمة الهيئة بدقة وأدب."""
+
+TEMPLATE """<start_of_turn>user
+{{ if .System }}{{ .System }}
+
+{{ end }}{{ .Prompt }}<end_of_turn>
+<start_of_turn>model
+{{ .Response }}<end_of_turn>
+"""
--- a/README.md
+++ b/README.md
@@ -0,0 +1,181 @@
+---
+license: gemma
+base_model: perfectPresentation/rcrc-gemma-1b-cpt
+datasets:
+- perfectPresentation/rcrc-qa-v5
+language:
+- ar
+- en
+tags:
+- chat
+- rcrc
+- arabic
+- gemma
+- gemma3
+- closed-book
+library_name: transformers
+---
+
+# rcrc-chat-v5-gemma-1b-cpt-sft — Closed-Book RCRC Chatbot
+
+A 1B closed-book chatbot for the Royal Commission for Riyadh City (RCRC).
+Trained as **Path B** of the v5 comparison: continued pre-training on raw KB
+text, then chat SFT on Qwen3-235B-synthesized QA pairs.
+
+Closed-book here means the model answers from baked-in knowledge — there is
+no retrieval at inference.
+
+## Pipeline
+
+```
+google/gemma-3-1b-pt
+   ↓  CPT: 3 epochs on cleaned RCRC + Hanifa raw text
+perfectPresentation/rcrc-gemma-1b-cpt
+   ↓  SFT: 3 epochs on rcrc-qa-v5 (16,761 Qwen-synthesized QA pairs)
+perfectPresentation/rcrc-chat-v5-gemma-1b-cpt-sft   ← this repo
+```
+
+## Training data
+
+[`perfectPresentation/rcrc-qa-v5`](https://huggingface.co/datasets/perfectPresentation/rcrc-qa-v5):
+16,426 train + 335 validation single-turn (system, user, assistant) pairs
+synthesized by `Qwen/Qwen3-235B-A22B-Instruct-2507` from the cleaned RCRC
+website + Hanifa Urban Code chunks.
+
+## Training recipe
+
+| | |
+|---|---|
+| Base | `perfectPresentation/rcrc-gemma-1b-cpt` |
+| Epochs | 3 |
+| LR | 2e-5, cosine, 5% warmup |
+| Effective batch | 16 (per_device 4 × grad_accum 4) |
+| Max seq length | 1024, packing enabled |
+| Final eval loss | 0.71 |
+| Final eval token-accuracy | 83.5% |
+| Hardware | HF Jobs · 1× L4 (~1.6 h) |
+
+## Use
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+tok = AutoTokenizer.from_pretrained("perfectPresentation/rcrc-chat-v5-gemma-1b-cpt-sft")
+model = AutoModelForCausalLM.from_pretrained(
+    "perfectPresentation/rcrc-chat-v5-gemma-1b-cpt-sft",
+    dtype=torch.bfloat16,
+)
+
+messages = [
+    {"role": "system", "content":
+        "أنت مساعد للهيئة الملكية لمدينة الرياض. تجيب على استفسارات المستخدمين "
+        "عن خدمات وبرامج ومشاريع وأنظمة الهيئة بدقة وأدب."},
+    {"role": "user", "content": "ما هو الكود العمراني لوادي حنيفة؟"},
+]
+prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tok(prompt, return_tensors="pt").to(model.device)
+out = model.generate(
+    **inputs, max_new_tokens=400, do_sample=True, temperature=0.5, top_p=0.9,
+    repetition_penalty=1.15, no_repeat_ngram_size=6,
+)
+print(tok.decode(out[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))
+```
+
+## Honest evaluation: closed-book vs RAG
+
+On a 50-question internal eval set (RCRC + Hanifa + edge cases), this
+closed-book model was compared to a sibling RAG pipeline (the v3-rag
+checkpoint reading retrieved chunks from `rcrc-rag-index-v2`):
+
+| | accuracy | relevance | clarity | wins |
+|---|---:|---:|---:|---:|
+| RAG (v3-rag + index v2) | 3.92 | 4.08 | 4.42 | 34/50 |
+| **This model (closed-book)** | **2.64** | **3.36** | **3.56** | **13/50** |
+| Ties | | | | 3/50 |
+
+**Where closed-book is competitive**: dialect responses (Najdi/Hijazi),
+free-form opinion-style queries.
+
+**Where RAG dominates**: factual specifics from the Hanifa Urban Code,
+project details, organizational facts, numerical specs.
+
+For accuracy-sensitive deployments on the RCRC + Hanifa corpus, a RAG
+pipeline at the same parameter count outperforms this closed-book model
+by ~1.3 points on average. This model is provided for completeness of the
+v5 study and for offline / no-retrieval scenarios.
+
+## Limitations
+
+- 1B-scale closed-book recall is brittle on specifics (numbers,
+  exact procedure steps). Verify against rcrc.gov.sa.
+- Training data was Qwen-synthesized; some questions may carry the
+  synthesizer's biases.
+- No multi-turn conversational SFT — single-turn QA only.
+
+<!-- GGUF-BUILDS -->
+## GGUF builds (llama.cpp / Ollama)
+
+Quantized GGUF files live at the repo root.
+
+| File | Quant | Approx size |
+|---|---|---|
+| `rcrc-v5-gemma-1b-cpt-sft-F16.gguf` | F16 | ~2.0 GB |
+| `rcrc-v5-gemma-1b-cpt-sft-Q8_0.gguf` | Q8_0 | ~1.0 GB |
+| `rcrc-v5-gemma-1b-cpt-sft-Q5_K_M.gguf` | Q5_K_M | ~720 MB |
+| `rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf` | Q4_K_M | ~620 MB |
+
+### Ollama (one-liner)
+
+```bash
+ollama run huggingface.co/perfectPresentation/rcrc-chat-v5-gemma-1b-cpt-sft:Q4_K_M
+# or :Q8_0, :Q5_K_M, :F16
+```
+
+If you hit a host-redirect error (`hf.co` → `huggingface.co`), upgrade
+Ollama to a recent version, or use the `huggingface.co/...` URL above.
+
+### Manual Modelfile route
+
+```bash
+hf download perfectPresentation/rcrc-chat-v5-gemma-1b-cpt-sft rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf Modelfile --local-dir ./model
+cd model
+ollama create rcrc-v5-gemma-1b-cpt-sft -f Modelfile
+ollama run rcrc-v5-gemma-1b-cpt-sft
+```
+
+### llama.cpp
+
+```bash
+hf download perfectPresentation/rcrc-chat-v5-gemma-1b-cpt-sft rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf --local-dir .
+./llama-cli -m rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf -cnv
+```
+
+## GGUF builds (llama.cpp / Ollama)
+
+Quantized GGUF files live under `gguf/`. They are built directly from
+the safetensors above with `llama.cpp convert_hf_to_gguf.py` followed by
+`llama-quantize`.
+
+| File | Quant | Approx size |
+|---|---|---|
+| `gguf/rcrc-v5-gemma-1b-cpt-sft-F16.gguf` | F16 | ~2.0 GB |
+| `gguf/rcrc-v5-gemma-1b-cpt-sft-Q8_0.gguf` | Q8_0 | ~1.0 GB |
+| `gguf/rcrc-v5-gemma-1b-cpt-sft-Q5_K_M.gguf` | Q5_K_M | ~720 MB |
+| `gguf/rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf` | Q4_K_M | ~620 MB |
+
+### llama.cpp
+
+```bash
+hf download perfectPresentation/rcrc-chat-v5-gemma-1b-cpt-sft gguf/rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf --local-dir .
+./llama-cli -m gguf/rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf -cnv
+```
+
+### Ollama
+
+```bash
+hf download perfectPresentation/rcrc-chat-v5-gemma-1b-cpt-sft gguf/rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf gguf/Modelfile --local-dir ./model
+cd model/gguf
+ollama create rcrc-v5-gemma-1b-cpt-sft -f Modelfile
+ollama run rcrc-v5-gemma-1b-cpt-sft
+```
--- a/chat_template.jinja
+++ b/chat_template.jinja
@@ -0,0 +1,47 @@
+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}
--- a/config.json
+++ b/config.json
@@ -0,0 +1,72 @@
+{
+  "_sliding_window_pattern": 6,
+  "architectures": [
+    "Gemma3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_logit_softcapping": null,
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "dtype": "bfloat16",
+  "eos_token_id": 1,
+  "final_logit_softcapping": null,
+  "head_dim": 256,
+  "hidden_activation": "gelu_pytorch_tanh",
+  "hidden_size": 1152,
+  "initializer_range": 0.02,
+  "intermediate_size": 6912,
+  "layer_types": [
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "model_type": "gemma3_text",
+  "num_attention_heads": 4,
+  "num_hidden_layers": 26,
+  "num_key_value_heads": 1,
+  "pad_token_id": 0,
+  "query_pre_attn_scalar": 256,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "full_attention": {
+      "rope_theta": 1000000,
+      "rope_type": "default"
+    },
+    "sliding_attention": {
+      "rope_theta": 10000,
+      "rope_type": "default"
+    }
+  },
+  "sliding_window": 512,
+  "sliding_window_pattern": 6,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.7.0",
+  "use_bidirectional_attention": false,
+  "use_cache": false,
+  "vocab_size": 262144
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "5.7.0"
+}
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12908e2e398052d938641cb76620188c3ccc5a7e4ff9e72a2e5f3a39123e8930
+size 1999811208
--- a/rcrc-v5-gemma-1b-cpt-sft-F16.gguf
+++ b/rcrc-v5-gemma-1b-cpt-sft-F16.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8a1b69e3ee1af64b8b6434ceb2524513368b104f82a8f756a93ba56fb584fc1
+size 2006575072
--- a/rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf
+++ b/rcrc-v5-gemma-1b-cpt-sft-Q4_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9df5b73b2ae07b9971cae498ec62eceef59766a91f67dbe9889158ccbf93010d
+size 806059744
--- a/rcrc-v5-gemma-1b-cpt-sft-Q5_K_M.gguf
+++ b/rcrc-v5-gemma-1b-cpt-sft-Q5_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6af20c1ec35fe26280b87cff5f1cc559b1c7b59decc7de2d4cf0dd78b0c70873
+size 851347168
--- a/rcrc-v5-gemma-1b-cpt-sft-Q8_0.gguf
+++ b/rcrc-v5-gemma-1b-cpt-sft-Q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f7efa27263d7a9ce8f06a4aa487b0c0d7543c6868d995d87d4edf2114682884
+size 1069307872
--- a/tokenizer.json
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:daab2354f8a74e70d70b4d1f804939b68a8c9624dd06cb7858e52dd8970e9726
+size 33384567
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,24 @@
+{
+  "backend": "tokenizers",
+  "boi_token": "<start_of_image>",
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eoi_token": "<end_of_image>",
+  "eos_token": "<eos>",
+  "image_token": "<image_soft_token>",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "boi_token": "<start_of_image>",
+    "eoi_token": "<end_of_image>",
+    "image_token": "<image_soft_token>"
+  },
+  "pad_token": "<pad>",
+  "sp_model_kwargs": null,
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}