license, language, base_model, pipeline_tag, library_name, tags, model-index

license	language	base_model	pipeline_tag	library_name	tags	model-index
apache-2.0	en	deepseek-ai/DeepSeek-R1-Distill-Qwen-7B	text-generation	transformers	text-generation-inference	name results QwQ-R1-Distill-7B-CoT task dataset metrics source type name text-generation Text Generation name type split args IFEval (0-Shot) wis-k/instruction-following-eval train num_few_shot 0 type value name inst_level_strict_acc and prompt_level_strict_acc 35.0 averaged accuracy url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-R1-Distill-7B-CoT Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args BBH (3-Shot) SaylorTwift/bbh test num_few_shot 3 type value name acc_norm 20.95 normalized accuracy url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-R1-Distill-7B-CoT Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args MATH Lvl 5 (4-Shot) lighteval/MATH-Hard test num_few_shot 4 type value name exact_match 27.19 exact match url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-R1-Distill-7B-CoT Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args GPQA (0-shot) Idavidrein/gpqa train num_few_shot 0 type value name acc_norm 5.82 acc_norm url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-R1-Distill-7B-CoT Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args MuSR (0-shot) TAUR-Lab/MuSR num_few_shot 0 type value name acc_norm 4.5 acc_norm url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-R1-Distill-7B-CoT Open LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type config split args MMLU-PRO (5-shot) TIGER-Lab/MMLU-Pro main test num_few_shot 5 type value name acc 20.05 accuracy url name https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=prithivMLmods%2FQwQ-R1-Distill-7B-CoT Open LLM Leaderboard

QwQ-R1-Distill-7B-CoT

QwQ-R1-Distill-7B-CoT is based on the Qwen [ KT ] model, which was distilled by DeepSeek-R1-Distill-Qwen-7B. It has been fine-tuned on the long chain-of-thought reasoning model and specialized datasets, focusing on chain-of-thought (CoT) reasoning for problem-solving. This model is optimized for tasks requiring logical reasoning, detailed explanations, and multi-step problem-solving, making it ideal for applications such as instruction-following, text generation, and complex reasoning tasks.

Quickstart with Transformers

Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "prithivMLmods/QwQ-R1-Distill-7B-CoT"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

Intended Use:

1. Instruction-Following: The model excels in understanding and executing detailed instructions, making it ideal for automation systems, virtual assistants, and educational tools.
2. Text Generation: It can produce coherent, logically structured, and contextually relevant text for use in content creation, summarization, and report writing.
3. Complex Reasoning Tasks: With its fine-tuning for chain-of-thought reasoning, the model is well-suited for multi-step problem-solving, logical deduction, and question-answering tasks.
4. Research and Development: It can support researchers and developers in exploring advancements in logical reasoning and fine-tuning methodologies.
5. Educational Applications: The model can assist in teaching logical reasoning and problem-solving by generating step-by-step solutions.

Limitations:

1. Domain-Specific Knowledge: While fine-tuned on reasoning datasets, the model may lack deep expertise in highly specialized or technical domains.
2. Hallucination: Like many large language models, it can generate incorrect or fabricated information, especially when reasoning beyond its training data.
3. Bias in Training Data: The model's outputs may reflect biases present in the datasets it was fine-tuned on, which could limit its objectivity in certain contexts.
4. Performance on Non-Reasoning Tasks: The model is optimized for chain-of-thought reasoning and may underperform on tasks that require simpler, less structured responses.
5. Resource-Intensive: Running the model efficiently requires significant computational resources, which may limit accessibility for smaller-scale deployments.
6. Dependence on Input Quality: The model’s performance heavily depends on the clarity and quality of the input provided. Ambiguous or poorly structured prompts may yield suboptimal results.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here! Summarized results can be found here!

Metric	Value (%)
Average	18.92
IFEval (0-Shot)	35.00
BBH (3-Shot)	20.95
MATH Lvl 5 (4-Shot)	27.19
GPQA (0-shot)	5.82
MuSR (0-shot)	4.50
MMLU-PRO (5-shot)	20.05