初始化项目，由ModelHub XC社区提供模型

Model: micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5 Source: Original Platform
2026-09-11 07:44:14 +08:00
commit ef53f4cc43
8 changed files with 328 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,141 @@
+---
+license: apache-2.0
+base_model: micymike/codemate-qwen-1.5B-8k
+tags:
+- code
+- coding
+- qwen
+- qwen2
+- transformers
+- text-generation
+- distillation
+- 32k-context
+- software-engineering
+- chat
+pipeline_tag: text-generation
+language:
+- en
+---
+
+# CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5
+
+CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5 is a fine-tuned coding assistant built on top of **CodeMate-Qwen-1.5B-8K**.
+
+The model was further trained on converted Claude Fable 5 coding traces to improve:
+
+- Code generation
+- Code explanation
+- Debugging
+- Multi-turn coding conversations
+- Software engineering reasoning
+
+## Model Details
+
+- **Base Model:** `micymike/codemate-qwen-1.5B-8k`
+- **Architecture:** Qwen2 Causal LM
+- **Training Method:** LoRA fine-tuning merged into full weights
+- **Precision:** BF16
+- **Configured Context Length:** 32,768 tokens
+
+## Context Configuration
+
+This model has been configured for a 32K context window using YaRN RoPE scaling.
+
+```python
+from transformers import AutoConfig
+
+config = AutoConfig.from_pretrained(
+    "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5"
+)
+
+print(config.max_position_embeddings)
+print(config.rope_scaling)
+```
+
+Current configuration:
+
+```python
+{
+    "rope_type": "yarn",
+    "factor": 4.0,
+    "original_max_position_embeddings": 8192,
+    "rope_theta": 1000000.0
+}
+```
+
+Note: Long-context performance beyond the original context length should be evaluated carefully for specific workloads.
+
+## Dataset
+
+The model was trained on converted Claude Fable 5 coding traces formatted into OpenAI-style conversations.
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+model_id = "micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+
+messages = [
+    {
+        "role": "system",
+        "content": "You are CodeMate, an expert programming assistant."
+    },
+    {
+        "role": "user",
+        "content": "Write a Python function to compute edit distance."
+    }
+]
+
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    temperature=0.7,
+    top_p=0.9,
+    do_sample=True
+)
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Limitations
+
+* Experimental research model.
+* Long-context capabilities require further evaluation.
+* May generate incorrect or insecure code.
+
+## Acknowledgements
+
+Built upon:
+
+* Qwen2
+* Transformers
+* PEFT
+* Hugging Face
+* llama.cpp
+* Claude Fable traces
+
+## Disclaimer
+
+This project is an independent research effort and is not affiliated with or endorsed by Anthropic, Claude, Alibaba, or Qwen.
+
+## Author
+
+Built by **micymike** 🇰🇪