初始化项目，由ModelHub XC社区提供模型

Model: Shamima/babylm-2026-multilingual-v3-quality-filter
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,45 @@
+---
+license: mit
+language: [en, nl, zh]
+library_name: transformers
+pipeline_tag: text-generation
+tags: [babylm, babylm-2026, multilingual, llama, pretrained-from-scratch, quality-filter]
+---
+
+# BabyLM 2026 — MultiLingual track v3 (quality-filter, WSD, 5 epochs)
+
+Iteration v3 of our BabyLM 2026 MultiLingual submission. Same Llama-110M
+architecture, same joint BPE 32k tokenizer as v2, **with category filtering**:
+EN drops `padding-*`; NL drops `padding-opensubtitles` (= 63% of NL); ZH drops
+`subtitles` (= 66% of ZH = WenetSpeech). Hypothesis: the v1/v2 ceiling was
+data-quality driven, not schedule-driven.
+
+Filtered corpora (post-byte-premium reference tokens):
+- EN: 78.2 M (was 99 M)
+- NL: 38.8 M (was 105 M)
+- ZH: 49.7 M (was 147 M)
+- **Total: 166.7 M ref tokens available**
+
+## Training
+
+- Schedule: WSD (warmup 200 → constant 6e-4 → linear last 25% to 6e-5)
+- Total compute consumed: **500 M effective tokens** (5× v2's 100 M)
+- Per-language epochs: EN 2.13, NL 4.29, ZH 3.35 — within the ≤10 cap
+- 4× NVIDIA A10G, bf16, DDP, eff. batch 131 K tokens/step
+- 23,295 steps · 8.6 hours wallclock
+
+## Revisions
+
+`main` is `chck_400M` (the largest fast-eval checkpoint we saved).
+Available revisions: chck_1M, chck_2M, chck_3M, chck_4M, chck_5M, chck_6M, chck_7M, chck_8M, chck_9M, chck_10M, chck_20M, chck_30M, chck_40M, chck_50M, chck_60M, chck_70M, chck_80M, chck_90M, chck_100M, chck_200M, chck_300M, chck_400M.
+
+## How to evaluate
+
+```bash
+git clone https://github.com/babylm-org/babylm-eval
+cd babylm-eval/multilingual
+bash scripts/zeroshot_model.sh --model_name Shamima/babylm-2026-multilingual-v3-quality-filter
+```
+
+Companion repo (audit, scaffold, ablation configs, iteration log):
+https://github.com/silvererudite/bb-lm-challenge-sub

config.json

@@ -0,0 +1,30 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 2048,
+  "max_position_embeddings": 1024,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "num_key_value_heads": 12,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "use_cache": false,
+  "vocab_size": 32768
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 0,
+  "transformers_version": "4.49.0",
+  "use_cache": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:675d39f6424a0fe3650737b2896a3f975a8e4a5af0306349a824e2097aad4f7c
+size 440490952

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<bos>",
+  "eos_token": "<eos>",
+  "pad_token": "<pad>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "tokenizer_class": "PreTrainedTokenizer",
+  "unk_token": "<unk>"
+}