初始化项目，由ModelHub XC社区提供模型
Model: cs-552-2026-databand/general_knowledge_model Source: Original Platform
This commit is contained in:
ModelHub XC
183
README.md
Normal file
183
README.md
Normal file
@@ -0,0 +1,183 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: Qwen/Qwen3-1.7B
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- qwen3
|
||||
- multiple-choice
|
||||
- general-knowledge
|
||||
- lora
|
||||
- sft
|
||||
- boxed-answer
|
||||
---
|
||||
|
||||
# General Knowledge Model
|
||||
|
||||
This is the final General Knowledge individual model for the CS-552 Modern NLP Spring 2026 standardized project.
|
||||
|
||||
The submitted model is the **SFT-only merged model**. A later DPO experiment was run on ARC/CommonsenseQA mistakes, but it reduced external benchmark accuracy, so it was **not selected** as the final model.
|
||||
|
||||
## Model behavior
|
||||
|
||||
The model is specialized for multiple-choice general knowledge questions. It is prompted to output exactly one final boxed answer, for example:
|
||||
|
||||
\boxed{A}
|
||||
|
||||
The chat template enforces concise answer-only behavior and supports choices labeled from A through T.
|
||||
|
||||
## Training setup
|
||||
|
||||
Starting point:
|
||||
|
||||
- Baseline working model folder with the project chat template and generation config
|
||||
- LoRA SFT on top of the baseline model
|
||||
- Final model produced by merging the LoRA adapter into the baseline model
|
||||
|
||||
Training method:
|
||||
|
||||
- LoRA supervised fine-tuning
|
||||
- Loss masked so that only the final assistant boxed answer contributes to training
|
||||
- Prompt, system message, question text, choices, chat markers, and template tokens are masked with -100
|
||||
- Assistant target format: \boxed{LETTER}
|
||||
|
||||
LoRA configuration:
|
||||
|
||||
- r = 16
|
||||
- lora_alpha = 32
|
||||
- lora_dropout = 0.05
|
||||
- Target modules:
|
||||
- q_proj
|
||||
- k_proj
|
||||
- v_proj
|
||||
- o_proj
|
||||
- gate_proj
|
||||
- up_proj
|
||||
- down_proj
|
||||
|
||||
Main training hyperparameters:
|
||||
|
||||
- Learning rate: 8e-5
|
||||
- Epochs: 1
|
||||
- Batch size per device: 1
|
||||
- Gradient accumulation steps: 8
|
||||
- Max sequence length: 8192
|
||||
- Precision: bf16
|
||||
- Scheduler: cosine
|
||||
- Warmup steps: 20
|
||||
|
||||
## SFT datasets
|
||||
|
||||
The SFT training data was built from:
|
||||
|
||||
1. Kaggle LLM Science
|
||||
2. EduQG
|
||||
3. EduAdapt, MCQ-only questions
|
||||
4. NCERT_MCQs
|
||||
5. SciQ train
|
||||
6. OpenBookQA train
|
||||
|
||||
Final SFT data sizes:
|
||||
|
||||
- Train: 26,120
|
||||
- Validation: 2,000
|
||||
|
||||
The answer labels were balanced uniformly across A through T separately for train and validation.
|
||||
|
||||
Train answer distribution:
|
||||
|
||||
- A through T: 1,306 examples each
|
||||
|
||||
Validation answer distribution:
|
||||
|
||||
- A through T: 100 examples each
|
||||
|
||||
## Evaluation
|
||||
|
||||
The final selected model is the SFT-only merged model.
|
||||
|
||||
The “SFT validation” set in the table is the held-out validation set created from the same six dataset families used for LoRA SFT training: Kaggle LLM Science, EduQG, EduAdapt MCQ, NCERT_MCQs, SciQ, and OpenBookQA. It contains 2,000 examples and is answer-balanced across A through T.
|
||||
|
||||
External benchmark sets:
|
||||
|
||||
- MMLU Pro: 2,000 examples, uniformly sampled across categories
|
||||
- MMLU Redux: 2,000 examples, uniformly sampled across subjects
|
||||
- SuperGPQA: 2,000 examples, uniformly sampled across disciplines
|
||||
|
||||
| Evaluation set | Baseline boxed | Baseline accuracy | SFT-only boxed | SFT-only accuracy | SFT + DPO boxed | SFT + DPO accuracy |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| SFT validation 2k | 19.20% | 16.00% | 100.00% | 85.30% | 100.00% | 79.75% |
|
||||
| MMLU Pro 2k | 60.25% | 18.05% | 100.00% | 37.85% | 100.00% | 35.25% |
|
||||
| MMLU Redux 2k | 26.65% | 11.40% | 100.00% | 56.25% | 100.00% | 50.90% |
|
||||
| SuperGPQA 2k | 66.95% | 15.85% | 99.95% | 27.55% | 100.00% | 23.45% |
|
||||
|
||||
The DPO experiment improved neither the selected SFT validation score nor the external benchmark scores. Therefore, the SFT-only merged model was selected as the final model.
|
||||
|
||||
### SFT validation details
|
||||
|
||||
SFT-only evaluation on the held-out SFT validation set:
|
||||
|
||||
- Total: 2,000
|
||||
- Extracted boxed answer: 2,000 / 2,000 = 100.00%
|
||||
- Accuracy: 1,706 / 2,000 = 85.30%
|
||||
|
||||
Accuracy by validation source:
|
||||
|
||||
| Source | Accuracy | Boxed extraction |
|
||||
|---|---:|---:|
|
||||
| eduadapt | 82.35% (14/17) | 100.00% (17/17) |
|
||||
| eduqg | 76.86% (93/121) | 100.00% (121/121) |
|
||||
| kaggle_llm_science | 58.30% (130/223) | 100.00% (223/223) |
|
||||
| ncert_mcqs | 93.33% (14/15) | 100.00% (15/15) |
|
||||
| openbookqa | 80.83% (430/532) | 100.00% (532/532) |
|
||||
| sciq | 93.86% (1025/1092) | 100.00% (1092/1092) |
|
||||
|
||||
### SuperGPQA boxed-answer edge case
|
||||
|
||||
The SFT-only model produced boxed answers for 1,999 out of 2,000 SuperGPQA examples. The single unboxed example was a long, LaTeX-heavy numerical analysis question whose answer choices contained multi-line mathematical derivations. Instead of producing a boxed option, the model continued/copy-completed part of one answer choice, generating text beginning with:
|
||||
|
||||
mathrm{d} x^{2} = 2.1730$ and $| R_{1} | ...
|
||||
|
||||
Increasing max_new_tokens from 20 to 64 did not change this outcome. The reported SuperGPQA result therefore keeps the strict extraction score of 99.95%.
|
||||
|
||||
## Expected input format
|
||||
|
||||
The model expects a multiple-choice question formatted like:
|
||||
|
||||
Question text here?
|
||||
|
||||
Choices:
|
||||
A. first option
|
||||
B. second option
|
||||
C. third option
|
||||
D. fourth option
|
||||
|
||||
It should answer with only:
|
||||
|
||||
\boxed{A}
|
||||
|
||||
## Reproducibility notes
|
||||
|
||||
In the Github repo CS-552/standard-project-m3-databand/tree/main, you can find in the folder code/general_knowledge all scripts and files needed for reproducibility.
|
||||
|
||||
A README.md file is also present in code/general_knowledge folder, documenting all script functionalities and how to run them to reproduce both SFT and SFT+DPO experiments.
|
||||
|
||||
Important files from the code/general_knowledge folder:
|
||||
|
||||
- SFT trainer: scripts/train_v3_lora_sft_masked.py
|
||||
- SFT data builder: scripts/build_my_sft_data_balanced.py
|
||||
- DPO trainer used for the unselected experiment: scripts/train_v3_lora_dpo_boxed.py
|
||||
- Merge script: scripts/merge_v3_lora_adapter.py
|
||||
- Evaluation script: scripts/evaluate_mcq_accuracy.py
|
||||
|
||||
Final selected model folder before upload:
|
||||
|
||||
outputs/lora_sft_v3_boxed_only/merged_full_model
|
||||
|
||||
SFT LoRA adapter:
|
||||
|
||||
outputs/lora_sft_v3_boxed_only/final_adapter
|
||||
|
||||
DPO adapter, experimental and not selected:
|
||||
|
||||
outputs/lora_dpo_arc_csqa_on_sft/final_adapter
|
||||
Reference in New Issue
Block a user