98 lines
3.6 KiB
Markdown
98 lines
3.6 KiB
Markdown
|
---
|
|||
|
|
license: mit
|
|||
|
|
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
|
|||
|
|
tags:
|
|||
|
|
- iol-ai-2026
|
|||
|
|
- reasoning
|
|||
|
|
- awq
|
|||
|
|
- rag
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# DeepSeek-R1-Distill-Qwen-7B-AWQ book-RAG submission
|
|||
|
|
|
|||
|
|
This is a training-free IOL-AI submission using a 4-bit AWQ conversion of
|
|||
|
|
`deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`. It retrieves only from the public,
|
|||
|
|
string-only extraction of Vlad A. Neacșu's *Linguistics Olympiad: Training
|
|||
|
|
guide*. Private curated training data are not present or required at runtime.
|
|||
|
|
|
|||
|
|
The checkpoint uses `Qwen2ForCausalLM`, so it is compatible with the challenge's
|
|||
|
|
Transformers 4.44.1 and AutoAWQ 0.2.7 runtime. The included weights are from
|
|||
|
|
`casperhansen/deepseek-r1-distill-qwen-7b-awq`, revision
|
|||
|
|
`305e6f12907dc78ae61a1f0bb7a19faa2b25e8a3`, which is an AWQ conversion of the
|
|||
|
|
official DeepSeek model.
|
|||
|
|
|
|||
|
|
## Challenge execution
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python script.py
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The script reads `/tmp/data/test.csv` and writes `submission.csv`. Its columns
|
|||
|
|
are `id,pred,explanation`; `pred` is a JSON-encoded list of answer strings.
|
|||
|
|
Explanation generation is disabled by default, leaving that optional column
|
|||
|
|
blank. It can be restored with:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python script.py --explanations on
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
For local data:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
python script.py --input path/to/test.csv --output submission.csv
|
|||
|
|
python script.py --self-test
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
After the model loads successfully, the output file is initialized and then
|
|||
|
|
atomically rewritten after every completed row. If evaluation reaches its time
|
|||
|
|
limit, predictions already completed remain in a valid submission file.
|
|||
|
|
|
|||
|
|
## DeepSeek reasoning and generation
|
|||
|
|
|
|||
|
|
DeepSeek's solver instructions, retrieved context, and current problem are
|
|||
|
|
placed in one user message rather than a separate system message. The assistant
|
|||
|
|
prompt is prefixed with `<think>` to engage the distilled reasoning behavior.
|
|||
|
|
Reasoning stops when `</think>` appears or at its configured cap. The runtime
|
|||
|
|
then starts a separate `FINAL ANSWERS:` stage, guaranteeing that reasoning
|
|||
|
|
cannot consume the answer budget. Only that answer block is serialized into
|
|||
|
|
`pred`.
|
|||
|
|
|
|||
|
|
Defaults follow DeepSeek's recommended sampling values while retaining a bounded
|
|||
|
|
challenge-time output:
|
|||
|
|
|
|||
|
|
- `temperature=0.6`
|
|||
|
|
- `top_p=0.95`
|
|||
|
|
- sampling enabled
|
|||
|
|
- `max_reasoning_tokens=4096`
|
|||
|
|
- `max_answer_tokens=1024`
|
|||
|
|
- one answer-only retry with `answer_retry_tokens=512`
|
|||
|
|
|
|||
|
|
The model context is capped at 32,768 tokens. The script loads AWQ directly in
|
|||
|
|
FP16, enables the KV cache, and applies an inference-only last-token `lm_head`
|
|||
|
|
hook to avoid materializing full-prompt FP32 vocabulary logits on the T4.
|
|||
|
|
|
|||
|
|
## Book-only retrieval
|
|||
|
|
|
|||
|
|
The default prompt includes one general book method and two worked book
|
|||
|
|
examples. Retrieval combines dependency-free BM25, character 3–5-gram TF-IDF,
|
|||
|
|
task-family inference, and book metadata boosts. It uses only the current
|
|||
|
|
problem's `context + query`; answers and private curated datasets are never
|
|||
|
|
indexed.
|
|||
|
|
|
|||
|
|
Settings are in `rag_resources/config.json`. Environment overrides are available
|
|||
|
|
for `IOL_TOP_METHODS`, `IOL_TOP_EXAMPLES`, `IOL_CHAR_TFIDF_WEIGHT`,
|
|||
|
|
`IOL_RAG_MAX_CHARS`, `IOL_MAX_REASONING_TOKENS`,
|
|||
|
|
`IOL_MAX_ANSWER_TOKENS`, `IOL_ANSWER_RETRY_TOKENS`,
|
|||
|
|
`IOL_EXPLANATION_MAX_NEW_TOKENS`,
|
|||
|
|
`IOL_ENABLE_EXPLANATIONS`, and `IOL_SEED`.
|
|||
|
|
|
|||
|
|
## Sources and licenses
|
|||
|
|
|
|||
|
|
- [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B)
|
|||
|
|
- [AWQ conversion](https://huggingface.co/casperhansen/deepseek-r1-distill-qwen-7b-awq)
|
|||
|
|
- [IOL-AI 2026 challenge](https://iolai.org/)
|
|||
|
|
- [Language Science Press book source](https://github.com/langsci/420)
|
|||
|
|
|
|||
|
|
The model and conversion are marked MIT. The book-derived resources retain
|
|||
|
|
their own license and attribution in `rag_resources/ATTRIBUTION.md`.
|