初始化项目，由ModelHub XC社区提供模型

Model: timothywong731/tim-360m-instruct
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,95 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+language:
+- en
+base_model: timothywong731/tim-360m-base
+datasets:
+- HuggingFaceTB/smol-smoltalk
+- HuggingFaceH4/ultrafeedback_binarized
+tags:
+- tim
+- qwen3
+- from-scratch
+- sft
+- dpo
+- chat
+model-index:
+- name: tim-360m-instruct
+  results:
+  - task: {type: text-generation, name: Text Generation}
+    dataset: {type: hellaswag, name: HellaSwag (0-shot)}
+    metrics: [{type: acc_norm, value: 0.395, name: acc_norm}]
+  - task: {type: text-generation, name: Text Generation}
+    dataset: {type: ai2_arc, name: ARC-Easy (0-shot)}
+    metrics: [{type: acc_norm, value: 0.501, name: acc_norm}]
+  - task: {type: text-generation, name: Text Generation}
+    dataset: {type: ai2_arc, name: ARC-Challenge (0-shot)}
+    metrics: [{type: acc_norm, value: 0.262, name: acc_norm}]
+  - task: {type: text-generation, name: Text Generation}
+    dataset: {type: piqa, name: PIQA (0-shot)}
+    metrics: [{type: acc_norm, value: 0.652, name: acc_norm}]
+  - task: {type: text-generation, name: Text Generation}
+    dataset: {type: winogrande, name: WinoGrande (0-shot)}
+    metrics: [{type: acc, value: 0.524, name: acc}]
+  - task: {type: text-generation, name: Text Generation}
+    dataset: {type: openbookqa, name: OpenBookQA (0-shot)}
+    metrics: [{type: acc_norm, value: 0.302, name: acc_norm}]
+  - task: {type: text-generation, name: Text Generation}
+    dataset: {type: social_i_qa, name: SocialIQA (0-shot)}
+    metrics: [{type: acc, value: 0.408, name: acc}]
+  - task: {type: text-generation, name: Text Generation}
+    dataset: {type: sciq, name: SciQ (0-shot)}
+    metrics: [{type: acc_norm, value: 0.714, name: acc_norm}]
+---
+
+# TIM-360M-instruct
+
+The **instruction-tuned** variant of [TIM-360M-base](https://huggingface.co/timothywong731/tim-360m-base) — a **361.8M-parameter** decoder-only transformer trained **from scratch** on 2× RTX 4090. Post-trained with **SFT → DPO**, no TRL.
+
+Exported as **`Qwen3ForCausalLM`** — loads with `transformers` and **no `trust_remote_code`**.
+
+## Chat format
+
+ChatML. Turns are wrapped `<|im_start|>{role}\n{content}<|im_end|>`; the assistant turn ends on `<|im_end|>` (token id 2), which is the model's generation stop token. Use the tokenizer's chat template — don't hand-format.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+tok = AutoTokenizer.from_pretrained("timothywong731/tim-360m-instruct")
+m = AutoModelForCausalLM.from_pretrained("timothywong731/tim-360m-instruct", torch_dtype=torch.bfloat16)
+
+msgs = [{"role": "user", "content": "Explain what a transformer is in two sentences."}]
+ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
+out = m.generate(ids, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9)
+print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
+```
+
+## Architecture
+
+Identical to the base model: SmolLM2-360M skeleton, pre-LN RMSNorm, RoPE (θ=100000), SwiGLU, **grouped-query attention** (15 query / 5 KV heads) with **per-head QK-norm**, tied embeddings, no biases. 32 layers, hidden 960, head_dim 64, FFN 2560, vocab 49,152. Context: `max_position_embeddings` 4096, trained at seq_len 2048.
+
+## Post-training
+
+1. **SFT** on [smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) — multi-turn instruction following, loss on assistant turns only.
+2. **DPO** on [ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) — a frozen reference policy, sigmoid preference loss (β=0.1, lr 5e-7), scored over the final assistant turn.
+
+## Evaluation
+
+lm-evaluation-harness (0.4.12), **0-shot**, `acc_norm` where applicable:
+
+| HellaSwag | ARC-Easy | ARC-Challenge | PIQA | WinoGrande | OpenBookQA | SciQ | SocialIQA | CommonsenseQA | WikiText (bpb ↓) |
+|---|---|---|---|---|---|---|---|---|---|
+| 0.395 | 0.501 | 0.262 | 0.652 | 0.524 | 0.302 | 0.714 | 0.408 | 0.203 | 1.130 |
+
+Post-training **preserved base capability** — most tasks move within noise of [the base model](https://huggingface.co/timothywong731/tim-360m-base), with a real gain on **SocialIQA (+3.0)** and small gains on HellaSwag / OpenBookQA. WikiText bits-per-byte rises (1.06 → 1.13), as expected: a chat-tuned model is no longer a pure next-token predictor of raw web text.
+
+## Limitations
+
+Small (360M) and lightly trained (~30B pretraining tokens, then light post-training). Instruction-following is **basic** — expect short, sometimes shallow answers, weak arithmetic and multi-step reasoning, and confident hallucination. English-only. **DPO is not safety alignment** — this model has no dedicated safety tuning; filter before exposing to end users.
+
+## Provenance & license
+
+**Apache-2.0.** Pretraining data (all named, public): FineWeb-Edu (ODC-BY), DCLM-Baseline (CC-BY-4.0), Stack-Edu, FineMath (ODC-BY). Post-training data: smol-smoltalk and ultrafeedback_binarized (see their dataset cards for licenses). No gated or scraped-private data.

config.json

@@ -0,0 +1,67 @@
+{
+  "architectures": [
+    "Qwen3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 960,
+  "initializer_range": 0.02,
+  "intermediate_size": 2560,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 4096,
+  "max_window_layers": 28,
+  "model_type": "qwen3",
+  "num_attention_heads": 15,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 5,
+  "pad_token_id": 3,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 100000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.13.1",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 49152
+}

generation_config.json

@@ -0,0 +1,5 @@
+{
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "pad_token_id": 3
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:846a0dd8f78f38ba81f4153f6220808b8bf2c9cd8a7d9f61e0ec59c6753be28d
+size 723690064

tokenizer_config.json

@@ -0,0 +1,524 @@
+{
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "model_max_length": 8192,
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|im_end|>",
+  "pad_token": "<|pad|>",
+  "unk_token": null,
+  "clean_up_tokenization_spaces": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|endoftext|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "1": {
+      "content": "<|im_start|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "2": {
+      "content": "<|im_end|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "3": {
+      "content": "<|pad|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "4": {
+      "content": "<|fim_prefix|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "5": {
+      "content": "<|fim_middle|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "6": {
+      "content": "<|fim_suffix|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "7": {
+      "content": "<|repo_name|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "8": {
+      "content": "<|reserved_0|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "9": {
+      "content": "<|reserved_1|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "10": {
+      "content": "<|reserved_2|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "11": {
+      "content": "<|reserved_3|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "12": {
+      "content": "<|reserved_4|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "13": {
+      "content": "<|reserved_5|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "14": {
+      "content": "<|reserved_6|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "15": {
+      "content": "<|reserved_7|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "16": {
+      "content": "<|reserved_8|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "17": {
+      "content": "<|reserved_9|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "18": {
+      "content": "<|reserved_10|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "19": {
+      "content": "<|reserved_11|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "20": {
+      "content": "<|reserved_12|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "21": {
+      "content": "<|reserved_13|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "22": {
+      "content": "<|reserved_14|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "23": {
+      "content": "<|reserved_15|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "24": {
+      "content": "<|reserved_16|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "25": {
+      "content": "<|reserved_17|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "26": {
+      "content": "<|reserved_18|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "27": {
+      "content": "<|reserved_19|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "28": {
+      "content": "<|reserved_20|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "29": {
+      "content": "<|reserved_21|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "30": {
+      "content": "<|reserved_22|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "31": {
+      "content": "<|reserved_23|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "32": {
+      "content": "<|reserved_24|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "33": {
+      "content": "<|reserved_25|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "34": {
+      "content": "<|reserved_26|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "35": {
+      "content": "<|reserved_27|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "36": {
+      "content": "<|reserved_28|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "37": {
+      "content": "<|reserved_29|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "38": {
+      "content": "<|reserved_30|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "39": {
+      "content": "<|reserved_31|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "40": {
+      "content": "<|reserved_32|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "41": {
+      "content": "<|reserved_33|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "42": {
+      "content": "<|reserved_34|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "43": {
+      "content": "<|reserved_35|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "44": {
+      "content": "<|reserved_36|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "45": {
+      "content": "<|reserved_37|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "46": {
+      "content": "<|reserved_38|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "47": {
+      "content": "<|reserved_39|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "48": {
+      "content": "<|reserved_40|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "49": {
+      "content": "<|reserved_41|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "50": {
+      "content": "<|reserved_42|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "51": {
+      "content": "<|reserved_43|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "52": {
+      "content": "<|reserved_44|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "53": {
+      "content": "<|reserved_45|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "54": {
+      "content": "<|reserved_46|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "55": {
+      "content": "<|reserved_47|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "56": {
+      "content": "<|reserved_48|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "57": {
+      "content": "<|reserved_49|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "58": {
+      "content": "<|reserved_50|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "59": {
+      "content": "<|reserved_51|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "60": {
+      "content": "<|reserved_52|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "61": {
+      "content": "<|reserved_53|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "62": {
+      "content": "<|reserved_54|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    },
+    "63": {
+      "content": "<|reserved_55|>",
+      "special": true,
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false
+    }
+  },
+  "chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{'<|im_start|>assistant\n'}}{% endif %}"
+}