Qwen-3-0.6B-Claude-4.7-Opus-Distilled/README.md

---
base_model: Qwen/Qwen3-0.6B
# base_model: Unsloth/Qwen3-0.6B-Unsloth-bnb-4bit - Variant that I used for fine-tuning (4-bit BNB quant by Unsloth)
tags:
- text-generation-inference
- transformers
- unsloth
- qwen3
- claude
- reasoning
- 4.7-opus
- opus-4.7
- claude-4.7-opus
- claude-opus-4.7
- distilled
- claude-opus
license: apache-2.0
language:
- en
datasets:
- lordx64/reasoning-distill-opus-4-7-max-sft
---

# Qwen 3 0.6B (Claude 4.7 Opus Distilled)

What happens if you take the reasoning of Claude 4.7 Opus and put it in Qwen 3 0.6B? You'll get Qwen 3 0.6B (Claude 4.7 Opus Distilled)!

Fine-tuned from [Qwen 3 0.6B](https://www.huggingface.co/Qwen/Qwen3-0.6B) (with Unsloth), this model is designed for tackling hard problems on any device!

# Features
* **Adaptive Reasoning:** Passes the Strawberry test and tackles quite hard problems, using a lot of reasoning tokens and comparable with larger models!
* **Tiny Size:** Beats any other large model on speed and memory usage. No other model is able to compete with this!

# Issues
* Don't use it for accurate facts, like the Minecraft tests I tried hallucinated wrong answers!

## Example
```python
from unsloth import FastLanguageModel
from transformers import TextStreamer
import torch

# 1. Configuration
max_seq_length = 8192 # Your reasoning trace might be long
dtype = None          # None for auto detection (FP16 for T4, BF16 for Ampere)
load_in_4bit = False  # You mentioned using 16-bit for better logic/SVG stability

# 2. Load the model and tokenizer from HF
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "MihaiPopa-1/Qwen-3-0.6B-Claude-4.7-Opus-Distilied",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    # device_map = "auto", # Unsloth handles this, but you can keep it if needed
)

# 3. Switch to Inference Mode
FastLanguageModel.for_inference(model)

# 2. Define the Streamer
# skip_prompt=True ensures you don't print the input prompt twice
text_streamer = TextStreamer(tokenizer, skip_prompt=True)

# 3. The High-Reasoning Prompt
prompt = """<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
Give 2-digit numbers which the product of each digit equals 24.<|im_end|>
<|im_start|>assistant"""

inputs = tokenizer([prompt], return_tensors = "pt").to("cuda")

# 4. Generate with the streamer
_ = model.generate(
    **inputs, 
    streamer = text_streamer, # This line enables the real-time display
    max_new_tokens = 8192,
    temperature = 0.6,
    top_p = 0.95,
    top_k = 20,
    repetition_penalty = 1.15,
    use_cache = True
)
```

gave this:
```
<think>
Okay, so I need to find two-digit numbers where the product of their digits is equal to 24. Let me think about how to approach this.

First, let's recall that a two-digit number has tens and units place. So if the number is AB (where A is the tens digit and B is the units), then the product of its digits would be A*B = 24. My goal is to find all such pairs (A,B) where A and B are single digits (from 0-9) and their product is 24.

Let me start by listing possible factors of 24. The prime factorization of 24 is 2^3 * 3. Therefore, any combination of digits whose product is 24 must include these primes in some way. Since we're dealing with two digits, maybe I can list all possible combinations of two digits from 1 to 9 (since leading zeros aren't allowed for two-digit numbers) that multiply to 24.

Let me try different possibilities:

Starting with 1 as one of the digits. If A=1, then B needs to be 24. But since B is a single digit, there's no solution here because 24 isn't a valid digit. Similarly, if B=1, then A should be 24, again invalid. So 1 cannot be part of either digit.

Next, considering 2 as one of the digits. If A=2, then B=12. Again, 12 isn't a single digit. If B=2, then A=12, also not valid. So 2 doesn't work unless both digits are larger than 2.

What about 3? If A=3, then B=8. Both 3 and 8 are single digits. That works! So 3*8=24. So the number could be 38 or 83.

Wait, but what about other combinations?

How about 4? If A=4, then B=6. Both 4 and 6 are single digits. Product is 24. Number: 46 or 64.

Similarly, 5? Then B would have to be 24/5 = 4.8, which isn't an integer. Not valid.

6? Then B=24/6=4. Valid. So 64 or 46.

7? 24 divided by 7 is approximately 3.428... Not integer.

8? 24 divided by 8 is 3. Which gives us 83 or 38.

Same with 9? 24 divided by 9 is roughly 2.666..., not integer.

So compiling all valid ones:

From 3 and 8: 38, 83

From 4 and 6: 46, 64

From 6 and 4: same as above, already covered.

Also, check if there are more possibilities. For example, 2 and 12 (invalid), 3 and 8, etc.

Is there another pair? Let me see. What about 12 and something else? No, they have to be single digits. How about 24 and something? No, only two digits.

Therefore, the two-digit numbers are 38, 83; 46, 64.

But wait, let me verify once more. Let's take 38: 3*8=24 ✔️. 83: 8*3=24 ✔️. 46: 4*6=24 ✔️. 64: 6*4=24 ✔️. Yes, those are correct.

Are there any others? Let me think. Suppose someone might consider 24 itself as a two-digit number, but it's already considered as 24, but the problem says "two-digit numbers", so 24 is acceptable. However, does 24 count? Wait, the question says "product of each digit". For 24, the digits are 2 and 4, product is 8 ≠ 24. So 24 is excluded. Hence, our previous list is correct.

Another thought: Could there be numbers like 12, 24, etc., but as mentioned earlier, only when digits are higher. So yes, seems like 38, 46, 64, 83 are the solutions.

I don't think there are more possibilities. All other combinations either result in non-integer digits or exceed the maximum single-digit value.
</think>

To solve the problem of finding **two-digit numbers** where the **product of each digit equals 24**, follow these steps:

---

### Step-by-step Analysis:

We are looking for two-digit numbers $ \text{AB} $, i.e., numbers between 10 and 99 inclusive, such that:

$$
A \times B = 24
$$

Where:
- $ A $ is the tens digit,
- $ B $ is the units digit,
- Both $ A $ and $ B $ are integers between 0 and 9 (inclusive).

---

### Finding Valid Combinations:

Start by factoring 24 into two integers between 1 and 9:

- $ 3 \times 8 = 24 $
- $ 4 \times 6 = 24 $

These give us the following valid two-digit numbers:

- $ 38 $ → $ 3 \times 8 = 24 $
- $ 83 $ → $ 8 \times 3 = 24 $
- $ 46 $ → $ 4 \times 6 = 24 $
- $ 64 $ → $ 6 \times 4 = 24 $

All four combinations satisfy the condition.

---

### Final Answer:

The two-digit numbers are:

$$
\boxed{38}, \quad \boxed{83}, \quad \boxed{46}, \quad \boxed{64}
$$<|im_end|>
```

# Usage
Code is by Gemini 3 Flash (then some little modifications by myself):
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# 1. Load from your Hugging Face Repo
model_id = "MihaiPopa-1/Qwen-3-0.6B-Claude-4.7-Opus-Distilled"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float32, # Standard for CPU
    device_map="cpu"           # Forces CPU usage
)

# 2. Generate
prompt = "<|im_start|>system\nYou are a helpful AI assistant.<|im_end|>\n<|im_start|>user\nGive 2-digit numbers which the product of each digit equals 24<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cpu")

with torch.no_grad():
    outputs = model.generate(
    **inputs,
    max_new_tokens = 8192,
    temperature = 0.6,
    top_p = 0.95,
    top_k = 20,
    repetition_penalty = 1.15,
    use_cache = True
)
    
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

# Data Used
Greetings to [Opus 4.7 Max SFT](https://www.huggingface.co/datasets/lordx64/reasoning-distill-opus-4-7-max-sft) for the amazing dataset!

---

# Uploaded finetuned model

- **Developed by:** MihaiPopa-1
- **License:** apache-2.0
- **Finetuned from model :** unsloth/qwen3-0.6b-unsloth-bnb-4bit

This qwen3 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)