Praneshrajan15/DataForge-0.5B-SFT
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: Praneshrajan15/DataForge-0.5B-SFT
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-30 04:43:16 +08:00
commit 36502d45fa
10 changed files with 27670 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

121
README.md Normal file
View File

@@ -0,0 +1,121 @@
---
license: apache-2.0
base_model: Qwen/Qwen2.5-0.5B-Instruct
library_name: transformers
tags:
- dataforge
- data-quality
- supervised-fine-tuning
- qlora
- kaggle
datasets:
- Praneshrajan15/dataforge-sft-trajectories
metrics:
- f1
model-index:
- name: DataForge-0.5B-SFT
results:
- task:
type: text-generation
name: DataForge repair planning
dataset:
name: DataForge SFT trajectories
type: Praneshrajan15/dataforge-sft-trajectories
metrics:
- type: macro_f1
name: Held-out macro F1
value: null
---
# DataForge-0.5B-SFT
DataForge-0.5B-SFT is a supervised-fine-tuned warmup checkpoint for tabular
data-quality repair experiments. The current training path uses chunk-level
DataForge expert trajectories whose exact repairs are derived from audited
dirty/clean CSV diffs. The earlier `v0-smoke` release only proved the
Kaggle-to-Hugging-Face pipeline and should not be read as a performance claim.
## Intended Use
- Research on tabular data-quality agents and repair planning.
- Offline evaluation on DataForge-Bench-style Hospital, Flights, and Beers
tasks.
- Warm-starting later DataForge RL experiments.
This checkpoint is not intended for autonomous production data modification,
medical decision support, regulated data governance, or unsupervised repair of
private datasets.
## Training Data
- Dataset repo: `Praneshrajan15/dataforge-sft-trajectories`.
- Dataset repo SHA used for this run: `94e2dd556d4f1260c5123d93ca6bf4f9da9b160a`.
- Training examples: `1958` chunk-level `expert_v4.jsonl`
records.
- Data sources: Raha benchmark Hospital, Flights, and Beers datasets via the
BigDaMa/raha repository.
- Primary label source: `oracle_from_clean_diff` dirty/clean CSV diffs.
- Legacy teacher lineage: Groq-hosted `clean-diff-v1` ReAct smoke records may
remain for auditability, but exact repairs are not teacher-discovered labels.
- Flights schedule and actual-time repairs are supervised from dirty/clean
labels; they are not inferred from incomplete prompt context.
- Split safety: held-out rows are reserved before chunking and excluded from SFT
target rows, context rows, normalization candidates, fixes, and messages.
- Hard negatives: clean train chunks are retained as `finish` examples with
empty repairs so the model is penalized for unnecessary edits.
The trajectory JSONL includes state, tool calls, diagnosis text, proposed fixes,
teacher/oracle metadata, benchmark metrics, split metadata, and source
provenance for auditability.
## Training Procedure
- Base model: `Qwen/Qwen2.5-0.5B-Instruct`.
- Method: 4-bit QLoRA warmup, then LoRA merge into fp16 merged weights.
- Compute target: Kaggle or Hugging Face remote GPU only; no laptop model
training or full evaluation.
- Kaggle hours used: `1.147`.
- Epochs: 2.
- Batch size: 1 per device with gradient accumulation of 16.
- Learning rate: 2e-5.
## Evaluation
Evaluation is reported on held-out DataForge-Bench-style tasks sampled after the
training trajectory seeds. The release status generated by the notebook is
`quality_improved_verified`. Only `quality_improved_verified` should be treated as a
quality milestone. `diagnostic_complete_no_gain` means the run is authentic and
published, but not promoted.
| Model | Held-out macro F1 |
| --- | ---: |
| `Qwen/Qwen2.5-0.5B-Instruct` | `0.0` |
| `DataForge-0.5B-SFT` | `0.0077` |
Release gates:
- Parse success: `1.0`.
- Schema-case errors: `0`.
- Quality milestone: `True`.
These numbers are produced by the publishing notebook and should not be edited
manually. Re-run the notebook to regenerate them. Detailed per-dataset metrics
are stored in `training_metrics.json` under `base_eval` and `sft_eval`.
Bounded per-task failure evidence is stored in `eval_diagnostics.json`.
## Limitations
- The checkpoint is a Week 9 warmup model, not the final DataForge model family.
- It has only seen small chunk-level ReAct traces and may fail on larger schemas,
unseen domains, adversarial dirty values, or tasks requiring multi-step
database access.
- Legacy teacher traces can contain teacher errors; the primary current labels
come from exact dirty/clean diffs.
- The model should be used behind DataForge's safety, verifier, and transaction
layers before any real data changes.
## License
Weights are published as `apache-2.0` after verifying the base model
metadata for `Qwen/Qwen2.5-0.5B-Instruct`. Users must also comply with the source dataset
licenses/terms and the teacher model terms that governed trajectory generation.

54
chat_template.jinja Normal file
View File

@@ -0,0 +1,54 @@
{%- if tools %}
{{- '<|im_start|>system\n' }}
{%- if messages[0]['role'] == 'system' %}
{{- messages[0]['content'] }}
{%- else %}
{{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
{%- endif %}
{{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
{%- if messages[0]['role'] == 'system' %}
{{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
{%- else %}
{{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- for message in messages %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{{- '<|im_start|>' + message.role }}
{%- if message.content %}
{{- '\n' + message.content }}
{%- endif %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.function is defined %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '\n<tool_call>\n{"name": "' }}
{{- tool_call.name }}
{{- '", "arguments": ' }}
{{- tool_call.arguments | tojson }}
{{- '}\n</tool_call>' }}
{%- endfor %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- message.content }}
{{- '\n</tool_response>' }}
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- endif %}

57
config.json Normal file
View File

@@ -0,0 +1,57 @@
{
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"dtype": "float16",
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 896,
"initializer_range": 0.02,
"intermediate_size": 4864,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"max_position_embeddings": 32768,
"max_window_layers": 21,
"model_type": "qwen2",
"num_attention_heads": 14,
"num_hidden_layers": 24,
"num_key_value_heads": 2,
"pad_token_id": null,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000.0,
"rope_type": "default"
},
"sliding_window": null,
"tie_word_embeddings": true,
"transformers_version": "5.7.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}

27106
eval_diagnostics.json Normal file
View File

File diff suppressed because it is too large Load Diff

14
generation_config.json Normal file
View File

@@ -0,0 +1,14 @@
{
"bos_token_id": 151643,
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"repetition_penalty": 1.1,
"temperature": 0.7,
"top_k": 20,
"top_p": 0.8,
"transformers_version": "5.7.0"
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:78c22203b4f04770a0f3f416a6ab12976649765da995dea02502af5b187fc24c
size 988097536

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
size 11421892

30
tokenizer_config.json Normal file
View File

@@ -0,0 +1,30 @@
{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"extra_special_tokens": [
"<|im_start|>",
"<|im_end|>",
"<|object_ref_start|>",
"<|object_ref_end|>",
"<|box_start|>",
"<|box_end|>",
"<|quad_start|>",
"<|quad_end|>",
"<|vision_start|>",
"<|vision_end|>",
"<|vision_pad|>",
"<|image_pad|>",
"<|video_pad|>"
],
"is_local": false,
"local_files_only": false,
"model_max_length": 131072,
"pad_token": "<|endoftext|>",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null
}

246
training_metrics.json Normal file
View File

@@ -0,0 +1,246 @@
{
"model_name": "DataForge-0.5B-SFT",
"model_license": "apache-2.0",
"base_model": "Qwen/Qwen2.5-0.5B-Instruct",
"teacher_model": "clean-diff-v1",
"dataset_repo": "Praneshrajan15/dataforge-sft-trajectories",
"dataset_sha": "94e2dd556d4f1260c5123d93ca6bf4f9da9b160a",
"dataset_records": 2058,
"training_examples": 1958,
"heldout_examples": 100,
"trajectory_filename": "expert_v4.jsonl",
"promotion_slice": "deterministic_normalization",
"config_sha256": "5a6d8ea6531da6d1cb2f594e22b7e63ecf190c1db271af5ad143ca0319271ee8",
"trajectory_sha256": "11032d48db0b5b5cf1c7b3faa7041f82bdc3115543ec51b8518c682309c8686a",
"split_manifest_sha256": "1fd0853f7ca6d0b39dcfcb6edd9a31791eaecebf05712786ea1fda6809d04e2a",
"kaggle_hours": 1.147,
"base_f1": 0.0,
"sft_f1": 0.0077,
"base_eval": {
"macro_f1": 0.0053,
"mean_f1": 0.0054,
"dataset_f1": {
"beers": 0.0061,
"flights": 0.0,
"hospital": 0.0099
},
"parse_success_rate": 1.0,
"schema_case_error_count": 113,
"failure_taxonomy": {
"missed_repair": 553,
"overrepair": 158,
"schema_case_error": 113,
"wrong_cell": 40,
"wrong_value": 60
},
"promotion_slice": "deterministic_normalization",
"slice_scores": {
"deterministic_normalization": {
"tasks": 20,
"macro_f1": 0.0,
"mean_f1": 0.0,
"parse_success_rate": 1.0,
"schema_case_error_count": 29,
"finish_rate": 0.0,
"false_repair_rate": 1.0,
"false_repair_count": 73,
"dataset_f1": {
"beers": 0.0
}
},
"external_reference_required": {
"tasks": 66,
"macro_f1": 0.0093,
"mean_f1": 0.0081,
"parse_success_rate": 1.0,
"schema_case_error_count": 70,
"finish_rate": 0.0,
"false_repair_rate": 0.9882,
"false_repair_count": 252,
"dataset_f1": {
"beers": 0.0154,
"flights": 0.0,
"hospital": 0.0125
}
},
"not_inferable_from_prompt": {
"tasks": 14,
"macro_f1": 0.0,
"mean_f1": 0.0,
"parse_success_rate": 1.0,
"schema_case_error_count": 14,
"finish_rate": 0.0,
"false_repair_rate": 1.0,
"false_repair_count": 46,
"dataset_f1": {
"flights": 0.0,
"hospital": 0.0
}
}
}
},
"sft_eval": {
"macro_f1": 0.0106,
"mean_f1": 0.0106,
"dataset_f1": {
"beers": 0.0107,
"flights": 0.0126,
"hospital": 0.0084
},
"parse_success_rate": 0.99,
"schema_case_error_count": 16,
"failure_taxonomy": {
"missed_repair": 519,
"overrepair": 249,
"schema_case_error": 16,
"truncated_json": 1,
"wrong_cell": 3,
"wrong_value": 92
},
"promotion_slice": "deterministic_normalization",
"slice_scores": {
"deterministic_normalization": {
"tasks": 20,
"macro_f1": 0.0077,
"mean_f1": 0.0077,
"parse_success_rate": 1.0,
"schema_case_error_count": 0,
"finish_rate": 0.05,
"false_repair_rate": 0.9867,
"false_repair_count": 74,
"dataset_f1": {
"beers": 0.0077
}
},
"external_reference_required": {
"tasks": 66,
"macro_f1": 0.014,
"mean_f1": 0.0137,
"parse_success_rate": 0.9848,
"schema_case_error_count": 8,
"finish_rate": 0.0758,
"false_repair_rate": 0.9831,
"false_repair_count": 233,
"dataset_f1": {
"beers": 0.0154,
"flights": 0.016,
"hospital": 0.0106
}
},
"not_inferable_from_prompt": {
"tasks": 14,
"macro_f1": 0.0,
"mean_f1": 0.0,
"parse_success_rate": 1.0,
"schema_case_error_count": 8,
"finish_rate": 0.0,
"false_repair_rate": 1.0,
"false_repair_count": 53,
"dataset_f1": {
"flights": 0.0,
"hospital": 0.0
}
}
}
},
"slice_scores": {
"base": {
"deterministic_normalization": {
"tasks": 20,
"macro_f1": 0.0,
"mean_f1": 0.0,
"parse_success_rate": 1.0,
"schema_case_error_count": 29,
"finish_rate": 0.0,
"false_repair_rate": 1.0,
"false_repair_count": 73,
"dataset_f1": {
"beers": 0.0
}
},
"external_reference_required": {
"tasks": 66,
"macro_f1": 0.0093,
"mean_f1": 0.0081,
"parse_success_rate": 1.0,
"schema_case_error_count": 70,
"finish_rate": 0.0,
"false_repair_rate": 0.9882,
"false_repair_count": 252,
"dataset_f1": {
"beers": 0.0154,
"flights": 0.0,
"hospital": 0.0125
}
},
"not_inferable_from_prompt": {
"tasks": 14,
"macro_f1": 0.0,
"mean_f1": 0.0,
"parse_success_rate": 1.0,
"schema_case_error_count": 14,
"finish_rate": 0.0,
"false_repair_rate": 1.0,
"false_repair_count": 46,
"dataset_f1": {
"flights": 0.0,
"hospital": 0.0
}
}
},
"sft": {
"deterministic_normalization": {
"tasks": 20,
"macro_f1": 0.0077,
"mean_f1": 0.0077,
"parse_success_rate": 1.0,
"schema_case_error_count": 0,
"finish_rate": 0.05,
"false_repair_rate": 0.9867,
"false_repair_count": 74,
"dataset_f1": {
"beers": 0.0077
}
},
"external_reference_required": {
"tasks": 66,
"macro_f1": 0.014,
"mean_f1": 0.0137,
"parse_success_rate": 0.9848,
"schema_case_error_count": 8,
"finish_rate": 0.0758,
"false_repair_rate": 0.9831,
"false_repair_count": 233,
"dataset_f1": {
"beers": 0.0154,
"flights": 0.016,
"hospital": 0.0106
}
},
"not_inferable_from_prompt": {
"tasks": 14,
"macro_f1": 0.0,
"mean_f1": 0.0,
"parse_success_rate": 1.0,
"schema_case_error_count": 8,
"finish_rate": 0.0,
"false_repair_rate": 1.0,
"false_repair_count": 53,
"dataset_f1": {
"flights": 0.0,
"hospital": 0.0
}
}
}
},
"evaluation_chunk_width": 4,
"evaluation_max_new_tokens": 1024,
"parse_success_rate": 1.0,
"schema_case_error_count": 0,
"quality_gate_failures": [],
"prompt_contract_drift": false,
"heldout_leakage_detected": false,
"quality_milestone": true,
"release_status": "quality_improved_verified",
"repo_id": "Praneshrajan15/DataForge-0.5B-SFT"
}