DRIFT-8B-Material/README.md

license, base_model, library_name, pipeline_tag, tags, language, arxiv, github, blog

license	base_model	library_name	pipeline_tag	tags	language	arxiv	github	blog
other	Qwen/Qwen3-8B	transformers	text-generation	qwen3 causal-lm transformers	zh en	2606.30345	https://github.com/LianjiaTech/drift	https://lianjiatech.github.io/drift/blog/

Drift-8B-Material

This repository contains a merged HuggingFace checkpoint fine-tuned based on Qwen/Qwen3-8B.

Quick Links

• GitHub Repository: LianjiaTech/drift
• Technical Blog: DRIFT Blog
• Academic Paper: arXiv:2606.30345

Model Summary

• Base model: Qwen/Qwen3-8B
• Architecture: Qwen3ForCausalLM
• Precision: bfloat16
• Context length (config): max_position_embeddings = 40960
• Weights format: sharded safetensors (4 shards)

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Yiwei6534/Drift-8B-Material"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What can you help me with?"},
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Generation Defaults

The bundled generation_config.json uses temperature=0.6, top_k=20, top_p=0.95. Adjust based on your deployment.

Integrity Files

• FILE_MANIFEST.json: list of distributed files and their byte sizes.
• SHA256SUMS.txt: SHA256 checksums for all distributed files (verify with sha256sum -c SHA256SUMS.txt).

Limitations

• The model may hallucinate tool calls or produce invalid arguments.
• Output quality depends on the serving template and tool schema formatting.
• Safety, bias, and domain-specific failure modes are not fully documented here.

Citation

If you find DRIFT or this model helpful in your research, please cite:

@article{luo2026drift,
  title={DRIFT: Difficulty Routing Self-DIstillation with Rhythm-Gated Exploration and Success BuFfer Training},
  author={Luo, Haisen and Liu, Yiwei and Wang, Haoning and Liu, Dan and Yin, Junxi and Wang, Haotian and Zhang, Lei and Tian, Xiaoyu and Chen, Shuaiting and Song, Yuansheng and others},
  journal={arXiv preprint arXiv:2606.30345},
  year={2026}
}

License

This repository uses license: other as a placeholder. Replace it with the correct license for the base model, your fine-tuning data, and your distribution terms before publishing.