初始化项目,由ModelHub XC社区提供模型

Model: Vilyam888/Broken_Code_Generation.1.0
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-16 06:33:19 +08:00
commit b0af1ebc67
22 changed files with 1377 additions and 0 deletions

View File

@@ -0,0 +1,35 @@
ОТЧЁТ ОБ ОЦЕНКЕ МОДЕЛИ Broken_Code_Generation.1.0
Репозиторий: Vilyam888/Broken_Code_Generation.1.0
Базовая модель: Qwen/Qwen2.5-Coder-3B-Instruct
Дата оценки: 2026-06-11
Выборка: test.json, N = 100 (hold-out test)
Генерация: temperature = 0.2, max_new_tokens = 1200
1. Perplexity (validation, checkpoint-501):
• train_loss_final: 0.1867
• eval_loss_final: 0.2523
• eval_mean_token_accuracy: 0.9323
• perplexity_validation: 1.29
• num_train_epochs: 3
• global_step: 501
По эпохам:
epoch 1.0: PPL=1.3247, eval_loss=0.2812, acc=0.9243
epoch 2.0: PPL=1.2856, eval_loss=0.2512, acc=0.9317
epoch 3.0: PPL=1.2869, eval_loss=0.2523, acc=0.9323
2. JSON validity (QLoRA vs baseline):
• valid_json_rate: 0.94 (baseline 0.78, Δ 0.16)
• required_fields_rate: 0.92 (baseline 0.74, Δ 0.18)
• difficulty_match_rate: 0.96 (baseline 0.85, Δ 0.11)
• topic_tag_key_match_rate: 0.97 (baseline 0.83, Δ 0.14)
3. BLEU / ROUGE (QLoRA vs baseline):
• bleu4_corpus: 0.68 (baseline 0.52, Δ 0.16)
• rouge1_f1: 0.73 (baseline 0.57, Δ 0.16)
• rouge2_f1: 0.58 (baseline 0.41, Δ 0.17)
• rougeL_f1: 0.71 (baseline 0.54, Δ 0.17)
4. Code metrics (поле broken_code):
• broken_code_syntax_valid_rate: 0.91
• codebleu_broken_code: 0.47