105 lines
3.4 KiB
Markdown
105 lines
3.4 KiB
Markdown
---
|
||
library_name: transformers
|
||
license: apache-2.0
|
||
base_model: Qwen/Qwen3-1.7B
|
||
pipeline_tag: text-generation
|
||
model_name: general_knowledge_model
|
||
tags:
|
||
- general-knowledge
|
||
- multiple-choice
|
||
- reasoning
|
||
- rejection-sampling
|
||
- rft
|
||
- lora
|
||
- cs-552
|
||
---
|
||
|
||
# Model Card for `general_knowledge_model`
|
||
|
||
Post-trained version of [`Qwen/Qwen3-1.7B`](https://huggingface.co/Qwen/Qwen3-1.7B)
|
||
for the **General Knowledge** benchmark of EPFL **CS-552 — Modern NLP** (Spring 2026),
|
||
team CentraleSupéchec.
|
||
|
||
The task is **closed-book multiple-choice QA** (2–20 options). The model reasons
|
||
inside a `<think> ... </think>` block and ends its reply with the answer wrapped in
|
||
`\boxed{LETTER}`, which is parsed for `pass@1` scoring.
|
||
|
||
## Training
|
||
|
||
The model is trained with **Rejection Fine-Tuning (RFT)** — STaR-style
|
||
self-distillation — with an **answer-only loss**:
|
||
|
||
1. Sample `n=8` completions (`T=0.7`) from the base model over a ~4.7k-question
|
||
pool of **GPQA** and **MMLU-Pro** (excluding Math/CS).
|
||
2. Keep the 722 questions the base fails at `pass@1` but solves under repeated
|
||
sampling, producing self-generated correct reasoning traces.
|
||
3. Fine-tune a **LoRA** adapter (`r=16`, `α=32`) with the cross-entropy loss
|
||
**masked to the `\boxed{}` answer span only** — the `<think>` reasoning
|
||
conditions the forward pass but receives no gradient. This preserves the
|
||
model's pretrained reasoning while sharpening answer commitment and output
|
||
formatting.
|
||
|
||
The chat template (baked into the tokenizer) enforces a strict `\boxed{LETTER}`
|
||
output and a 16,384-token reasoning budget.
|
||
|
||
## Quick start
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
repo = "cs-552-2026-centralesupechec/general_knowledge_model"
|
||
tok = AutoTokenizer.from_pretrained(repo)
|
||
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="cuda")
|
||
|
||
question = (
|
||
"Which of the following is the capital of Australia?\n\n"
|
||
"Choices:\nA. Sydney\nB. Melbourne\nC. Canberra\nD. Perth"
|
||
)
|
||
inputs = tok.apply_chat_template(
|
||
[{"role": "user", "content": question}],
|
||
add_generation_prompt=True, return_tensors="pt",
|
||
).to(model.device)
|
||
|
||
out = model.generate(inputs, max_new_tokens=16384, temperature=0.6, top_p=0.95, top_k=20)
|
||
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
|
||
# ... reasoning ... \boxed{C}
|
||
```
|
||
|
||
For vLLM, mirror the CI: apply the model's chat template, `seed=42`,
|
||
`max_new_tokens=16384`, `temperature=0.6`, `top_p=0.95`, `top_k=20`.
|
||
|
||
## Generation config
|
||
|
||
`max_new_tokens: 16384` · `temperature: 0.6` · `top_p: 0.95` · `top_k: 20` ·
|
||
`do_sample: true`. The 16k budget is essential: it removes the format failures
|
||
that occur when reasoning is truncated before the boxed answer.
|
||
|
||
## Evaluation
|
||
|
||
`pass@1` on held-out sets disjoint from training (n=4, 16k tokens):
|
||
|
||
| Set | pass@1 |
|
||
|---|---|
|
||
| 650-question MMLU sweep (26 subjects) | ~0.74 |
|
||
| Internal 100-question expert set | ~0.59 |
|
||
|
||
See the project report and code for the full comparison against the base model,
|
||
full-trace SFT, and GRPO.
|
||
|
||
## Framework versions
|
||
|
||
- Transformers 5.7.0
|
||
- PyTorch 2.10.0+cu128
|
||
- TRL 0.12, PEFT 0.13
|
||
|
||
## Citation
|
||
|
||
```bibtex
|
||
@inproceedings{zelikman2022star,
|
||
title = {{STaR}: Bootstrapping Reasoning With Reasoning},
|
||
author = {Zelikman, Eric and Wu, Yuhuai and Mu, Jesse and Goodman, Noah D.},
|
||
booktitle = {Advances in Neural Information Processing Systems},
|
||
year = {2022}
|
||
}
|
||
```
|