micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5

Files

ModelHub XC ef53f4cc43 初始化项目，由ModelHub XC社区提供模型

Model: micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5
Source: Original Platform

2026-09-11 07:44:14 +08:00

2.9 KiB

Raw Permalink Blame History

license, base_model, tags, pipeline_tag, language

license

base_model

CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5

CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5 is a fine-tuned coding assistant built on top of CodeMate-Qwen-1.5B-8K.

The model was further trained on converted Claude Fable 5 coding traces to improve:

Code generation
Code explanation
Debugging
Multi-turn coding conversations
Software engineering reasoning

Model Details

Base Model: micymike/codemate-qwen-1.5B-8k
Architecture: Qwen2 Causal LM
Training Method: LoRA fine-tuning merged into full weights
Precision: BF16
Configured Context Length: 32,768 tokens

Context Configuration

This model has been configured for a 32K context window using YaRN RoPE scaling.

from transformers import AutoConfig

config = AutoConfig.from_pretrained(
    "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5"
)

print(config.max_position_embeddings)
print(config.rope_scaling)

Current configuration:

{
    "rope_type": "yarn",
    "factor": 4.0,
    "original_max_position_embeddings": 8192,
    "rope_theta": 1000000.0
}

Note: Long-context performance beyond the original context length should be evaluated carefully for specific workloads.

Dataset

The model was trained on converted Claude Fable 5 coding traces formatted into OpenAI-style conversations.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {
        "role": "system",
        "content": "You are CodeMate, an expert programming assistant."
    },
    {
        "role": "user",
        "content": "Write a Python function to compute edit distance."
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

Experimental research model.
Long-context capabilities require further evaluation.
May generate incorrect or insecure code.

Acknowledgements

Built upon:

Qwen2
Transformers
PEFT
Hugging Face
llama.cpp
Claude Fable traces

Disclaimer

This project is an independent research effort and is not affiliated with or endorsed by Anthropic, Claude, Alibaba, or Qwen.

Author

Built by micymike 🇰🇪

2.9 KiB Raw Permalink Blame History