初始化项目，由ModelHub XC社区提供模型

Model: User01110/supralabs-50M-testing
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,82 @@
+---
+base_model: SupraLabs/Supra-1.5-50M-Base-exp
+library_name: transformers
+tags:
+- sft
+- chatml
+- trl
+- python
+- math
+- instruction-tuned
+---
+
+# supralabs-50M-testing
+
+This is an experimental ChatML SFT run from `SupraLabs/Supra-1.5-50M-Base-exp`.
+
+## Training Setup
+
+| Field | Value |
+| --- | --- |
+| Base model | `SupraLabs/Supra-1.5-50M-Base-exp` |
+| Output repo | `User01110/supralabs-50M-testing` |
+| Sequence length | 1024 |
+| Max optimizer steps | 10,000 |
+| Per-device batch size | 128 |
+| Gradient accumulation | 4 |
+| Sample presentations per GPU | 5,120,000 |
+| Max token slots per GPU | 5,242,880,000 |
+| Learning rate | 2.00e-04 |
+| Warmup steps | 100 |
+| Weight decay | 0.05 |
+| Save/push cadence | every 1,000 optimizer steps plus final |
+| Loss mask | assistant response only |
+| Chat format | ChatML |
+| System prompt | `You are a helpful assistant.` |
+
+The stream reloops datasets as needed to reach the fixed step budget. `Cutecat6152/python-data-basic` is capped at three passes because it only has 100 rows.
+
+Unique one-pass source rows listed below: 3,667,971. First-cycle source presentations with the `python-data-basic` cap included: 3,668,171. The 20k-step training budget presents 5,120,000 examples per GPU, so larger sources are expected to reloop during training.
+
+## ChatML Compatibility
+
+The tokenizer is saved with:
+
+| Token | Purpose |
+| --- | --- |
+| `<|im_start|>` | ChatML message start |
+| `<|im_end|>` | ChatML message end |
+
+The uploaded tokenizer includes the ChatML template, so inference and future SFT should not require manually adding these tokens again.
+
+Example prompt:
+
+```python
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "Explain what a neural network is in simple terms."},
+]
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+```
+
+## Dataset Mix
+
+| Dataset | Config | Split | Rows | Schema | Mapping | Pass policy |
+| --- | --- | --- | ---: | --- | --- | --- |
+| nvidia/Nemotron-SFT-Instruction-Following-Chat-v2 | default | reasoning_off | 1,068,273 | messages[{role, content, reasoning_content}] | user/assistant message pairs; reasoning_off only | reloops as needed |
+| microsoft/orca-math-word-problems-200k | default | train | 200,035 | question, answer | user=question; assistant=answer | reloops as needed |
+| TIGER-Lab/MathInstruct | default | train | 262,039 | instruction, output | user=instruction; assistant=output | reloops as needed |
+| Programming-Language/codeagent-python | default | train | 296,837 | prompt, response | user=prompt; assistant=response | reloops as needed |
+| Cutecat6152/python-data-basic | default | train | 100 | id, instruction, response | user=instruction; assistant=response | max 3 passes, 300 presentations max |
+| flytech/python-codes-25k | default | train | 49,626 | instruction, input, output, text | user=instruction plus optional Input block; assistant=output | reloops as needed |
+| QuixiAI/open-instruct-uncensored | default | train | 1,756,115 | dataset, id, messages[{role, content}] | user/assistant message pairs | reloops as needed |
+| openai/gsm8k | main | train | 7,473 | question, answer | user=question; assistant=answer | reloops as needed |
+| openai/gsm8k | socratic | train | 7,473 | question, answer | user=question; assistant=answer | reloops as needed |
+| EleutherAI/arithmetic | 10 selected subsets | validation raw JSONL | 20,000 | context, completion | user=context with trailing Answer: stripped; assistant=completion | reloops as needed |
+
+## Notes
+
+- Dataset schemas and row counts were checked through Hugging Face Dataset Viewer metadata where available.
+- Nemotron is loaded from the direct `reasoning_off.jsonl` file to avoid mixing in reasoning-on schema fields.
+- EleutherAI arithmetic is loaded from raw JSONL files to avoid old dataset-script loading issues.
+- RoPE buffers and tokenizer/model load are verified during final export.

chat_template.jinja

@@ -0,0 +1 @@
+{% for message in messages %}{{ '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>\n' }}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}

config.json

@@ -0,0 +1,36 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 512,
+  "im_end_token_id": 32001,
+  "im_start_token_id": 32000,
+  "initializer_range": 0.02,
+  "intermediate_size": 1408,
+  "max_position_embeddings": 5120,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 8,
+  "num_hidden_layers": 12,
+  "num_key_value_heads": 4,
+  "pad_token_id": 1,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "factor": 1.0,
+    "rope_theta": 10000.0,
+    "rope_type": "linear",
+    "type": "linear"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.10.2",
+  "use_cache": false,
+  "vocab_size": 32002
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": [
+    2
+  ],
+  "pad_token_id": 1,
+  "transformers_version": "5.10.2"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c876ff8abf2cc4057a286435483f26dcf9c3789295aa7fc2dc6ff888073df10c
+size 207161232

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 1000000000,
+  "pad_token": "<pad>",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>"
+}