初始化项目，由ModelHub XC社区提供模型

Model: anicka/cve-backport-codegen-v5-qwen25-32b Source: Original Platform
2026-05-16 17:26:39 +08:00
commit d274c3474e
22 changed files with 304579 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,38 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+cve-backport-codegen-v5-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-32b-experiment/tokenizer.json filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,310 @@
+---
+base_model: Qwen/Qwen2.5-Coder-32B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+license: apache-2.0
+language:
+  - en
+tags:
+  - security
+  - cve
+  - patches
+  - backporting
+  - opensuse
+  - suse
+  - linux
+  - code-generation
+  - lora
+  - qlora
+  - transformers
+datasets:
+  - anicka/cve-backport-codegen-dataset
+model-index:
+  - name: cve-backport-codegen-v5-qwen25-32b
+    results:
+      - task:
+          type: text-generation
+          name: Security Patch Backporting
+        dataset:
+          type: anicka/cve-backport-codegen-dataset
+          name: CVE Backport Codegen Dataset
+        metrics:
+          - name: Recall
+            type: recall
+            value: 0.931
+          - name: Precision
+            type: precision
+            value: 0.944
+          - name: Exact Match
+            type: exact_match
+            value: 0.83
+---
+
+# CVE Backport Codegen v5 — Qwen2.5-Coder-32B QLoRA
+
+Fine-tuned code generation model for backporting upstream CVE security fixes
+to older SUSE/openSUSE package versions. Given vulnerable source code and an
+upstream fix description, the model outputs the corrected code. A separate
+tool then diffs the output against the original to produce a patch.
+
+This is a **per-hunk code generation** approach: the model sees one region of
+source code at a time and returns the fixed version, rather than generating
+raw unified diffs. This yields higher accuracy than patch-format models
+because the model works in its natural domain (code) rather than a
+meta-format (diffs).
+
+> **MoE sibling now available:** [anicka/cve-backport-codegen-v5-qwen3-coder-30b-a3b](https://huggingface.co/anicka/cve-backport-codegen-v5-qwen3-coder-30b-a3b)
+> reaches 91.9% recall on the same n=100 eval (within 1.2 pt of this model)
+> while running ~10× faster at inference, thanks to Qwen3-Coder-30B-A3B's
+> sparse 3B-active MoE architecture. Same training data, same config style,
+> trained in 1/5 the wall time on a single H100.
+
+## What's New in v5
+
+v5 uses a unified **codegen-only dataset** — all 36,166 training examples
+follow the same 3-turn format (system / user with code + fix description /
+assistant with fixed code). v4 mixed in 5-turn test-generation examples;
+v5 drops those to focus entirely on codegen quality.
+
+| Metric | v5 | v4 | v1 |
+|--------|:--:|:--:|:--:|
+| **Recall** | **93.1%** | 93% | 91% |
+| **Precision** | **94.4%** | 95% | — |
+| **Exact match** | **83/100** | 87/100 | — |
+| **Adapted recall** | **90.0%** | 86% | 71% |
+| **Identical recall** | 93.7% | 94% | 94% |
+
+Adapted-tier recall has steadily improved: 71% (v1) → 86% (v4) → **90% (v5)**.
+The codegen-only dataset gives the model a cleaner training signal for the
+core task.
+
+## Model Details
+
+| | |
+|---|---|
+| **Base model** | [Qwen/Qwen2.5-Coder-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct) |
+| **Method** | QLoRA (4-bit NF4, double quantization, bf16 compute) |
+| **LoRA rank / alpha** | 64 / 128 |
+| **LoRA dropout** | 0.05 |
+| **LoRA targets** | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
+| **Training data** | 36,166 train / 1,834 eval examples |
+| **Epochs** | 2 (8,228 steps) |
+| **Effective batch size** | 8 (1 × grad_accum 8) |
+| **Learning rate** | 1e-4 (cosine schedule, 5% warmup) |
+| **Max sequence length** | 4,096 tokens |
+| **Optimizer** | AdamW fused, weight decay 0.01 |
+| **Hardware** | 2× NVIDIA H100 NVL 94GB |
+| **Training time** | 46.1 hours |
+| **Train loss (avg)** | 0.0215 |
+| **Eval loss (final)** | 0.00602 |
+| **PEFT version** | 0.18.1 |
+
+## Files
+
+This repository contains:
+
+- **LoRA adapter** (`adapter_model.safetensors`, `adapter_config.json`) — merge with the base model using PEFT
+- **GGUF Q8_0** (`cve-backport-codegen-v5-q8_0.gguf`, 33GB) — ready for llama.cpp / ollama
+
+## Reproduction via Teapot
+
+This model was trained via the [teapot](https://github.com/anicka-net/teapot)
+training pipeline. The full reproduction is a four-command sequence once the
+cve-backport dataset is prepared:
+
+```bash
+git clone https://github.com/anicka-net/teapot
+cd teapot
+pip install -e .
+
+# 1. Compose training data from the cve-backport module
+teapot compose configs/cve-backport.config \
+    --output train-cve-backport.jsonl
+
+# 2. Generate the QLoRA-HF launch script
+teapot train configs/cve-backport.config \
+    --backend qlora-hf \
+    --train-data train-cve-backport.jsonl \
+    --eval-data eval-cve-backport.jsonl \
+    --output train-cve-backport.sh
+
+# 3. Train (2× H100 NVL 94GB; ~46 hours)
+bash train-cve-backport.sh
+
+# 4. Final adapter is at output-teapot-cve-backport/final/
+```
+
+The teapot config (`configs/cve-backport.config`) pins all the hyperparameters:
+`method: qlora`, `epochs: 2`, `lr: 1e-4`, `batch_size: 1`, `gradient_accumulation: 8`,
+`lora_r: 64`, `lora_alpha: 128`, `max_length: 4096`, `warmup_ratio: 0.05`,
+`hardware.gpus: 2`. See the config file in the teapot repo for the full
+declaration.
+
+The `qlora-hf` backend invokes `python3 -m teapot.train_qlora_hf`, which is
+a thin wrapper over the HuggingFace `Trainer` with bitsandbytes 4-bit
+quantization and PEFT LoRA. Training data is composed from the
+[cve-backport-codegen-dataset](https://huggingface.co/datasets/anicka/cve-backport-codegen-dataset)
+HF repo (the `domain/cve-backport` teapot module fetches it automatically).
+
+## Evaluation
+
+Evaluated on 100 held-out examples (zero CVE overlap with training) using
+the Q8_0 GGUF served via llama-server (temperature=0, ctx=8192).
+
+### Overall
+
+| Metric | Value |
+|--------|-------|
+| Avg recall | 93.1% |
+| Avg precision | 94.4% |
+| Exact match | 83/100 |
+| Perfect (100% recall) | 90/100 |
+| Failures (0% recall) | 3/100 |
+
+### By Tier
+
+| Tier | Count | Avg Recall | Perfect |
+|------|:-----:|:----------:|:-------:|
+| **Identical** (upstream applies as-is) | 85 | 93.7% | 77/85 |
+| **Adapted** (requires modification) | 15 | 90.0% | 13/15 |
+
+### Failure Analysis
+
+The 3 zero-recall cases are all complex libvirt patches (multi-function
+adaptations across large files with significant structural differences
+between versions). These are known hard cases that likely require an
+agentic approach with source tree context.
+
+## Training Data
+
+The v5 dataset contains real SUSE/openSUSE maintenance patches paired
+with their upstream CVE fixes, converted to a per-hunk codegen format:
+
+- **36,166 train + 1,834 eval** examples (strict CVE-level split, zero overlap)
+- All examples use a **3-turn ChatML format** (system / user / assistant)
+- Per-hunk extraction with 15-line context padding, nearby hunks merged
+- Covers C, C++, Python, shell, Java, JavaScript, Go, and more
+- Sources: openSUSE Build Service maintenance incidents
+
+### Input Format
+
+```
+## File: path/to/file.c
+## Lines: 100-130
+
+```c
+/* 15 lines before the change */
+vulnerable_code_here();
+/* 15 lines after the change */
+```
+
+## Fix
+Description of what the upstream patch changes in this region.
+```
+
+### Output Format
+
+The model outputs the fixed version of the code region (just the code,
+no diff headers or markup).
+
+## Usage
+
+### With llama.cpp / llama-server (GGUF)
+
+```bash
+llama-server \
+    --model cve-backport-codegen-v5-q8_0.gguf \
+    --port 8403 \
+    --n-gpu-layers 99 \
+    --ctx-size 8192
+```
+
+### With the CVE Backport Tool
+
+The recommended way to use this model is via the
+[cve-backport-tool](https://github.com/openSUSE/cve-backport-tool),
+which handles patch parsing, source extraction, model inference, and
+diff generation:
+
+```bash
+python3 cve-backport.py \
+    --cve CVE-2024-1234 \
+    --package openssl-1.1.1d \
+    --patch upstream.patch \
+    --source-dir /path/to/source/ \
+    --backend openai \
+    --retry 3
+```
+
+### With transformers + PEFT (adapter)
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+base = AutoModelForCausalLM.from_pretrained(
+    "Qwen/Qwen2.5-Coder-32B-Instruct",
+    torch_dtype="bfloat16",
+    device_map="auto",
+)
+model = PeftModel.from_pretrained(base, "anicka/cve-backport-codegen-v5-qwen25-32b")
+tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-32B-Instruct")
+```
+
+### Prompt Template (ChatML)
+
+```
+<|im_start|>system
+You are a security patch backporting assistant.
+
+Given vulnerable source code and a description of the upstream fix, output the FIXED version of the code.
+
+Rules:
+- Output ONLY the fixed code, nothing else
+- Preserve all surrounding context exactly
+- Apply only the described fix
+<|im_end|>
+<|im_start|>user
+## File: crypto/bn/bn.h
+## Lines: 280-310
+
+```c
+/* source code region */
+```
+
+## Fix
+Add bounds check for BN_num_bits to prevent buffer over-read.
+<|im_end|>
+<|im_start|>assistant
+```
+
+## Limitations
+
+- **Best at identical-tier patches** (upstream fix applies directly) — 93.7% recall
+- **Good at adapted patches** (90% recall) but complex multi-function adaptations
+  across structurally different versions remain challenging
+- **Context window**: 4,096 token training limit means very large functions or
+  multi-file patches may be truncated
+- **No compilation feedback**: the model generates code in a single pass without
+  verifying it compiles. Use `--retry` in the CLI tool for iterative correction.
+- Always review generated patches before applying to production systems
+
+## Related
+
+- **MoE sibling**: [anicka/cve-backport-codegen-v5-qwen3-coder-30b-a3b](https://huggingface.co/anicka/cve-backport-codegen-v5-qwen3-coder-30b-a3b) — Qwen3-Coder-30B-A3B (3B active, MoE), 91.9% recall on the same n=100 eval, ~10× faster inference
+- **openSUSE mirror**: [openSUSE/CVE-Backport-Qwen2.5-Coder-32B](https://huggingface.co/openSUSE/CVE-Backport-Qwen2.5-Coder-32B)
+- **CLI tool**: [openSUSE/cve-backport-tool](https://github.com/openSUSE/cve-backport-tool)
+- **Dataset**: [anicka/cve-backport-codegen-dataset](https://huggingface.co/datasets/anicka/cve-backport-codegen-dataset)
+- **Training pipeline**: [teapot](https://github.com/anicka-net/teapot)
+- **Previous version (v1)**: [anicka/cve-backport-codegen-qwen25-32b-v1](https://huggingface.co/anicka/cve-backport-codegen-qwen25-32b-v1)
+
+## Citation
+
+```bibtex
+@misc{cve-backport-codegen-v5,
+  title={CVE Backport Codegen v5: Fine-tuned Qwen2.5-Coder-32B for Security Patch Backporting},
+  author={Anna Maresova},
+  year={2026},
+  url={https://huggingface.co/anicka/cve-backport-codegen-v5-qwen25-32b}
+}
+```
--- a/adapter_config.json
+++ b/adapter_config.json
@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-32B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
--- a/adapter_model.safetensors
+++ b/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:644bd0c861027440a38e5a6d59e4fc8e5629568a86a68881f735d68dd04b839c
+size 2147605960
--- a/added_tokens.json
+++ b/added_tokens.json
@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}
--- a/chat_template.jinja
+++ b/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
--- a/cve-backport-codegen-v5-q8_0.gguf
+++ b/cve-backport-codegen-v5-q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c652e608e4d7c4602ee39218d601cb76f2dcb6845fe1c92e9f715a8f3dacd4d8
+size 34820884672
--- a/eval/recall-v5.json
+++ b/eval/recall-v5.json
@@ -0,0 +1,729 @@
+{
+  "module": "domain/cve-backport",
+  "eval_type": "standard",
+  "timestamp": "2026-04-04T10:51:22Z",
+  "n_examples": 100,
+  "metrics": {
+    "avg_recall": 0.9313,
+    "avg_precision": 0.9441,
+    "exact_match": 83,
+    "perfect_count": 90,
+    "failure_count": 5,
+    "zero_failures": false
+  },
+  "per_tier": {
+    "adapted": {
+      "avg_recall": 0.9001,
+      "count": 15,
+      "perfect": 13
+    },
+    "identical": {
+      "avg_recall": 0.9368,
+      "count": 85,
+      "perfect": 77
+    }
+  },
+  "pass": false,
+  "per_example": [
+    {
+      "id": "codegen-openssl-4598-0001-s_server-Use-2048-bit-DH--apps_s_server.c",
+      "tier": "adapted",
+      "recall": 0.56,
+      "precision": 0.875,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-openssl-4769-openssl-CVE-2016-0797.patch-crypto_bn_bn.h",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-jasper-14141-jasper-CVE-2018-18873.patch-src_libjasper_ras_ras_enc.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-python-libxml2-python-12846-libxml2-python3-unicode-errors.patch-python_libxml.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-GraphicsMagick-13054-GraphicsMagick-dcm.c-update.patch-coders_dcm.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-libvirt-13730-f3ef7daf-xenconfig-e820-host.patch-src_libxl_xen_common.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-libvirt-13730-ae9e6c2a-qemu-allow-cond-format-probe.patch-src_util_virstoragefile.c",
+      "tier": "identical",
+      "recall": 0.975,
+      "precision": 1.0,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-glibc-10645-sysconf-uio-maxiov.patch-sysdeps_unix_sysv_linux_Makefile",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-tiff-9924-tiff-CVE-2019-7663.patch-tools_tiffcp.c",
+      "tier": "identical",
+      "recall": 0.8888888888888888,
+      "precision": 0.8888888888888888,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-glibc-17211-sysconf-uio-maxiov.patch-sysdeps_unix_sysv_linux_Makefile",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-bluez-10285-0006-btmon-fix-multiple-segfaults.patch-monitor_packet.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-bluez-11572-hcidump-Fix-memory-leak-with-malformed-packet.patch-tools_hcidump.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-python-base-17077-python-skip_random_failing_tests.patch-Lib_test_test_multiprocessing.py",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-python-base-10735-python-bsddb6.diff-Modules__bsddb.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-10645-mman-map-sync.patch-sysdeps_unix_sysv_linux_sh_bits_mman.h",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-17211-syslog-locking.patch-misc_syslog.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-17211-i386-memmove-sse2-unaligned.patch-string_test-memmove.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-10645-remove-nss-nis-compat.patch-nss_grp-lookup.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-python-17077-reproducible.patch-Lib_py_compile.py",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-libvirt-13730-967f4eeb-xenconfig-event-channels.patch-src_libxl_xen_xl.c",
+      "tier": "identical",
+      "recall": 0.047619047619047616,
+      "precision": 0.3333333333333333,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-python-10735-python-2.7.5-multilib.patch-Makefile.pre.in",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-GraphicsMagick-13054-GraphicsMagick-CVE-2019-19951.patch-coders_miff.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-libvirt-13730-b7d6648d-conf-add-e820-host.patch-docs_formatdomain.html.in",
+      "tier": "identical",
+      "recall": 0.0,
+      "precision": 0.0,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-libvirt-13730-libvirt-suse-netcontrol.patch-configure.ac",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-10645-i386-memmove-sse2-unaligned.patch-sysdeps_i386_i686_multiarch_memcpy-sse2-unaligned.S",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-17211-euc-kr-overrun.patch-iconvdata_Makefile",
+      "tier": "identical",
+      "recall": 0.5,
+      "precision": 0.25,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-glibc-17211-glibc-2.3.90-langpackdir.diff-intl_loadmsgcat.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-libvirt-13730-libxl-set-migration-constraints.patch-docs_manpages_virsh.rst",
+      "tier": "identical",
+      "recall": 0.0,
+      "precision": 0.0,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-libvirt-13730-0001-Extract-stats-functions-from-the-qemu-driver.patch-src_qemu_qemu_driver.c",
+      "tier": "identical",
+      "recall": 0.0,
+      "precision": 0.0,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-python-base-10735-python-2.7.5-multilib.patch-Lib_trace.py",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-python-doc-10735-python-bsddb6.diff-Lib_bsddb_test_test_all.py",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-python-doc-10735-python-bsddb6.diff-Modules_bsddb.h",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-python-10735-python-bsddb6.diff-Modules__bsddb.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-python-doc-10735-python-2.7.5-multilib.patch-Lib_site.py",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-python-doc-10735-python-skip_random_failing_tests.patch-Lib_test_test_subprocess.py",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-openssl-4769-openssl-CVE-2014-3566.patch-apps_s_client.c",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-openssl-4769-0001-libcrypto-Hide-library-pr-crypto_des_des_locl.h",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-openssl-4769-0001-libcrypto-Hide-library-pr-crypto_modes_modes_lcl.h",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-openssl-4769-0001-libcrypto-Hide-library-pr-crypto_modes_gcm128.c",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-openssl-4769-openssl-CVE-2014-8275.patch-crypto_asn1_a_type.c",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-17211-0003-S390-Unify-31-64bit-memcpy.patch-sysdeps_s390_s390-64_multiarch_Makefile",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-17211-iconv-option-parsing.patch-iconv_gconv_int.h",
+      "tier": "adapted",
+      "recall": 0.03333333333333333,
+      "precision": 0.5,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-ghostscript-mini-5761-fix-mutex-crash.patch-base_gp_psync.c",
+      "tier": "adapted",
+      "recall": 0.9574468085106383,
+      "precision": 0.9782608695652174,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-GraphicsMagick-9414-GraphicsMagick-CVE-2014-9845.patch-coders_dib.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-ImageMagick-9270-ImageMagick-CVE-2016-7540.patch-coders_rgf.c",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 0.8461538461538461,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-ImageMagick-9270-ImageMagick-CVE-2017-14989.patch-magick_annotate.c",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-GraphicsMagick-9192-GraphicsMagick-CVE-2018-16645.patch-coders_dib.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-ImageMagick-9059-ImageMagick-CVE-2016-6491.patch-magick_property.c",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-ImageMagick-9059-ImageMagick-CVE-2017-14175.patch-coders_xbm.c",
+      "tier": "adapted",
+      "recall": 0.95,
+      "precision": 0.9743589743589743,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-openssh-8445-openssh-7.2p2-X_forward_with_disabled_ipv6.patch-openssh-7.2p2_channels.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-openssh-8445-openssh-7.2p2-disable_short_DH_parameters.patch-openssh-7.2p2_readconf.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-openssh-8445-openssh-7.2p2-prevent_private_key_leakage.patch-openssh-7.2p2_sshbuf.h",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-apache2-8062-apache2-CVE-2016-8740.patch-modules_http2_h2_session.c",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-LibVNCServer-7946-LibVNCServer-CVE-2014-6052.patch-libvncclient_rfbproto.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-0004-S390-Fix-handling-of-DXC-byte-in-FPC-register.patch-sysdeps_s390_fpu_fesetenv.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-0013-S390-Optimize-stpcpy-and-wcpcpy.patch-string_test-stpcpy.c",
+      "tier": "identical",
+      "recall": 0.9615384615384616,
+      "precision": 0.9615384615384616,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-0016-S390-Optimize-strcat-and-wcscat.patch-sysdeps_s390_multiarch_strcat.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-0021-S390-Optimize-strchrnul-and-wcschrnul.patch-string_test-strchr.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-0027-S390-Optimize-memccpy.patch-string_memccpy.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-avx512-knl-memcpy.patch-sysdeps_x86_64_multiarch_mempcpy.S",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-errorcheck-mutex-no-elision.patch-nptl_pthread_mutex_timedlock.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-glob-altdirfunc.patch-manual_pattern.texi",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-iconv-reset-input-buffer.patch-iconv_gconv_simple.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-malloc-Fix-list_lock-arena-lock-deadlock-BZ-19182.patch-malloc_arena.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-nis-initgroups-status.patch-nis_nss_nis_nis-initgroups.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-powerpc-elision-adapt-param.patch-sysdeps_unix_sysv_linux_powerpc_elision-lock.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-prelink-elf-rtype-class.patch-elf_dl-lookup.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-reduce-edns-payload.patch-resolv_res_query.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-s390-runtime-resolve.patch-sysdeps_s390_bits_link.h",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-send-dg-buffer-overflow.patch-resolv_res_send.c",
+      "tier": "identical",
+      "recall": 0.9818181818181818,
+      "precision": 0.9818181818181818,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-glibc-testsuite-7805-strftime-range-check.patch-time_tst-strftime.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-0004-S390-Fix-handling-of-DXC-byte-in-FPC-register.patch-sysdeps_s390_fpu_fsetexcptflg.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-0013-S390-Optimize-stpcpy-and-wcpcpy.patch-wcsmbs_wcpcpy.c",
+      "tier": "identical",
+      "recall": 0.8,
+      "precision": 1.0,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-glibc-utils-7805-0017-S390-Optimize-strncat-wcsncat.patch-benchtests_bench-strncat.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-0023-S390-Optimize-strspn-and-wcsspn.patch-benchtests_bench-strspn.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-0028-S390-Optimize-wmemset.patch-sysdeps_s390_multiarch_wmemset.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-catopen-unbound-alloca.patch-catgets_catgets.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-fnmatch-collating-elements.patch-posix_fnmatch_loop.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-glob-altdirfunc.patch-posix_glob.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-iconv-reset-input-buffer.patch-iconv_gconv_simple.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-malloc-Prevent-arena-free_list-from-turning-cyclic-B.patch-malloc_malloc.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-nscd-gc-crash.patch-nscd_grpcache.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-powerpc-elision-enable-envvar.patch-sysdeps_unix_sysv_linux_powerpc_elision-conf.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-prelink-elf-rtype-class.patch-elf_dl-lookup.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-reduce-edns-payload.patch-resolv_res_query.c",
+      "tier": "identical",
+      "recall": 0.47368421052631576,
+      "precision": 0.8181818181818182,
+      "exact_match": false
+    },
+    {
+      "id": "codegen-glibc-utils-7805-s390-runtime-resolve.patch-sysdeps_s390_bits_link.h",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-send-dg-buffer-overflow.patch-resolv_res_send.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-utils-7805-sunrpc-xdr-memory.patch-sunrpc_xdr.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-7805-0004-S390-Fix-handling-of-DXC-byte-in-FPC-register.patch-sysdeps_s390_fpu_ftestexcept.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-7805-errorcheck-mutex-no-elision.patch-nptl_pthread_mutex_timedlock.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-7805-nis-initgroups-status.patch-nis_nss_nis_nis-initgroups.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-7805-powerpc-tabort-usage.patch-sysdeps_unix_sysv_linux_powerpc_syscall.S",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-glibc-7805-tzset-tzname.patch-timezone_Makefile",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-curl-7678-curl-CVE-2014-3620.patch-tests_data_test61",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-curl-7678-curl-CVE-2015-3153.patch-tests_data_test1527",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-curl-7678-curl-CVE-2016-8623.patch-lib_cookie.h",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-openvpn-7411-0001-Fix-remote-triggerable-memory-leaks-CVE-2017-7521.patch-src_openvpn_ssl_verify_openssl.c",
+      "tier": "adapted",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-openvpn-7411-openvpn-2.3.x-fixed-multiple-low-severity-issues.patch-src_openvpn_error.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-GraphicsMagick-7399-GraphicsMagick-CVE-2014-9845.patch-coders_dib.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    },
+    {
+      "id": "codegen-GraphicsMagick-7342-GraphicsMagick-CVE-2016-7101.patch-coders_sgi.c",
+      "tier": "identical",
+      "recall": 1.0,
+      "precision": 1.0,
+      "exact_match": true
+    }
+  ]
+}
--- a/merges.txt
+++ b/merges.txt
--- a/qwen3-32b-experiment/adapter_config.json
+++ b/qwen3-32b-experiment/adapter_config.json
@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-32B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "o_proj",
+    "gate_proj",
+    "q_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
--- a/qwen3-32b-experiment/adapter_model.safetensors
+++ b/qwen3-32b-experiment/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:993fedfd4d4c49c3290ea87b7265126eeab1fb84a05766846a17fd5ac75cf180
+size 2147605960
--- a/qwen3-32b-experiment/added_tokens.json
+++ b/qwen3-32b-experiment/added_tokens.json
@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}
--- a/qwen3-32b-experiment/merges.txt
+++ b/qwen3-32b-experiment/merges.txt
--- a/qwen3-32b-experiment/special_tokens_map.json
+++ b/qwen3-32b-experiment/special_tokens_map.json
@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
--- a/qwen3-32b-experiment/tokenizer.json
+++ b/qwen3-32b-experiment/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f1298e298f2fe0059aba46f037697a339ccba45a1908780ce8ca14b45582f23
+size 11422753
--- a/qwen3-32b-experiment/tokenizer_config.json
+++ b/qwen3-32b-experiment/tokenizer_config.json
@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
--- a/qwen3-32b-experiment/vocab.json
+++ b/qwen3-32b-experiment/vocab.json
--- a/special_tokens_map.json
+++ b/special_tokens_map.json
@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
--- a/tokenizer.json
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83396048d512ec1f3178af0d7c1f79a226bba041822614b0e26a4fd2d4b55bf7
+size 11421995
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
--- a/training_args.bin
+++ b/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18d5482439b903314c5777c6cb1050193782f8e89ed3d18122237dc3b827c686
+size 5905
--- a/vocab.json
+++ b/vocab.json