初始化项目，由ModelHub XC社区提供模型
Model: pedrodev2026/microcoder-1.5b Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
37
DATASET_CREDITS.md
Normal file
37
DATASET_CREDITS.md
Normal file
@@ -0,0 +1,37 @@
|
||||
# Credits
|
||||
|
||||
This dataset is a combination of three existing datasets, pre-processed with **deduplication** and **token limit of 1024 tokens per example**.
|
||||
|
||||
## Included Datasets
|
||||
|
||||
1. **[CyberNative/Code_Vulnerability_Security_DPO](https://huggingface.co/datasets/CyberNative/Code_Vulnerability_Security_DPO)**
|
||||
- Creator: CyberNative
|
||||
- License: Apache 2.0
|
||||
- Description: Code dataset focused on security vulnerabilities.
|
||||
|
||||
2. **[Madras1/minimax-m2.5-code-distilled-14k](https://huggingface.co/datasets/Madras1/minimax-m2.5-code-distilled-14k)**
|
||||
- Creator: Madras1
|
||||
- License: Apache 2.0
|
||||
- Description: Distilled code dataset emphasizing coding patterns and representations.
|
||||
|
||||
3. **[pedrodev2026/pedro-open-distil-dataset](https://huggingface.co/datasets/pedrodev2026/pedro-open-distil-dataset)**
|
||||
- Creator: pedrodev2026
|
||||
- License: BSD 3-Clause
|
||||
- Description: Custom distilled code dataset created and maintained by pedrodev2026.
|
||||
|
||||
## Preprocessing
|
||||
|
||||
The combined dataset was prepared by:
|
||||
|
||||
- **Deduplicating** all examples to remove redundancy.
|
||||
- Limiting examples to **1024 tokens each**.
|
||||
|
||||
## License
|
||||
|
||||
The final combined dataset is licensed under **BSD 3-Clause**.
|
||||
Users must still respect the original licenses of the included datasets when redistributing or using the original unmodified datasets.
|
||||
|
||||
- Original licenses:
|
||||
- **[CyberNative/Code_Vulnerability_Security_DPO](https://huggingface.co/datasets/CyberNative/Code_Vulnerability_Security_DPO)**: Apache 2.0
|
||||
- **[Madras1/minimax-m2.5-code-distilled-14k](https://huggingface.co/datasets/Madras1/minimax-m2.5-code-distilled-14k)**: Apache 2.0
|
||||
- **[pedrodev2026/pedro-open-distil-dataset](https://huggingface.co/datasets/pedrodev2026/pedro-open-distil-dataset)**: BSD 3-Clause
|
||||
11
LICENSE
Normal file
11
LICENSE
Normal file
@@ -0,0 +1,11 @@
|
||||
Copyright 2026 pedrodev2026
|
||||
|
||||
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
||||
|
||||
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
|
||||
|
||||
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
||||
|
||||
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
74
MODEL_CREDITS.md
Normal file
74
MODEL_CREDITS.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# Model Credits - Microcoder-1.5B
|
||||
|
||||
## Base Model
|
||||
|
||||
This fine-tuned model is built upon **Qwen 2.5 Coder 1.5B Instruct**, created and maintained by [Alibaba Cloud](https://www.alibabacloud.com/).
|
||||
|
||||
### Original Model Information
|
||||
|
||||
- **Model Name**: Qwen 2.5 Coder 1.5B Instruct
|
||||
- **Creator**: Alibaba Cloud
|
||||
- **Repository**: [Qwen Hugging Face](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct)
|
||||
- **License**: Apache 2.0
|
||||
|
||||
The Qwen 2.5 Coder series represents a significant advancement in code generation models, optimized for programming tasks and instruction following.
|
||||
|
||||
## Model Redistribution
|
||||
|
||||
We acknowledge **Unsloth** for their role in redistributing and optimizing the base model, making it more accessible to the community.
|
||||
|
||||
- **Organization**: Unsloth
|
||||
- **Website**: [Unsloth.ai](https://unsloth.ai)
|
||||
|
||||
## Fine-Tuned Model (Microcoder-1.5B)
|
||||
|
||||
- **License**: BSD-3-Clause
|
||||
- **Status**: This fine-tuned version incorporates specialized training and optimizations
|
||||
|
||||
## License Summary
|
||||
|
||||
| Component | License |
|
||||
|-----------|---------|
|
||||
| Base Model (Qwen 2.5 Coder 1.5B) | Apache 2.0 |
|
||||
| Fine-tuned Model (Microcoder-1.5B) | BSD-3-Clause |
|
||||
|
||||
## Dataset Credits
|
||||
|
||||
For detailed information about the datasets used in the fine-tuning process, please refer to [`DATASET_CREDITS.md`](./DATASET_CREDITS.md).
|
||||
|
||||
## Attribution
|
||||
|
||||
When using Microcoder-1.5B, please provide appropriate attribution to:
|
||||
|
||||
1. **Alibaba Cloud** - for the original Qwen 2.5 Coder model
|
||||
2. **Unsloth** - for model redistribution and optimization
|
||||
3. **Microcoder Contributors** - for the fine-tuning and improvements
|
||||
|
||||
## Citation
|
||||
|
||||
If you use this model in your research or projects, please consider citing:
|
||||
|
||||
```bibtex
|
||||
@misc{microcoder2026,
|
||||
title={Microcoder-1.5B: A Fine-tuned Code Generation Model},
|
||||
author={[pedrodev2026]},
|
||||
year={2026},
|
||||
url={[https://huggingface.co/pedrodev2026/microcoder-1.5b]}
|
||||
}
|
||||
```
|
||||
|
||||
And also cite the original Qwen model:
|
||||
|
||||
```bibtex
|
||||
@article{hui2024qwen2,
|
||||
title={Qwen2.5-Coder Technical Report},
|
||||
author={Hui, Binyuan and Yang, Jian and Cui, Zeyu and Yang, Jiaxi and Liu, Dayiheng and Zhang, Lei and Liu, Tianyu and Zhang, Jiajun and Yu, Bowen and Dang, Kai and others},
|
||||
journal={arXiv preprint arXiv:2409.12186},
|
||||
year={2024}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2026
|
||||
**Model Version**: 1.5B
|
||||
97
README.md
Normal file
97
README.md
Normal file
@@ -0,0 +1,97 @@
|
||||
---
|
||||
license: bsd-3-clause
|
||||
datasets:
|
||||
- pedrodev2026/microcoder-dataset-1024-tokens
|
||||
base_model:
|
||||
- unsloth/Qwen2.5-Coder-1.5B-Instruct
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- coder
|
||||
- code
|
||||
- microcoder
|
||||
---
|
||||
# Microcoder 1.5B
|
||||
|
||||
**Microcoder 1.5B** is a code-focused language model fine-tuned from [Qwen 2.5 Coder 1.5B Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) using LoRA (Low-Rank Adaptation) on curated code datasets. It is designed for code generation, completion, and instruction-following tasks in a lightweight, efficient package.
|
||||
|
||||
---
|
||||
|
||||
## Model Details
|
||||
|
||||
| Property | Value |
|
||||
|------------------|--------------------------------------------|
|
||||
| **Base Model** | Qwen 2.5 Coder 1.5B Instruct |
|
||||
| **Fine-tuning** | LoRA |
|
||||
| **Parameters** | ~1.5B |
|
||||
| **License** | BSD 3-Clause |
|
||||
| **Language** | English (primary), multilingual code |
|
||||
| **Task** | Code generation, completion, instruction following |
|
||||
|
||||
---
|
||||
|
||||
## Benchmarks
|
||||
|
||||
| Benchmark | Metric | Score |
|
||||
|--------------------|----------|--------------|
|
||||
| HumanEval | pass@1 | **59.15%** |
|
||||
| MBPP+ | pass@1 | **52.91%** |
|
||||
> HumanEval and MBPP+ results were obtained using the model in **GGUF format** with **Q5_K_M quantization**. Results may vary slightly with other formats or quantization levels.
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
> **Important:** You must use `apply_chat_template` when formatting inputs. Passing raw text directly to the tokenizer will produce incorrect results.
|
||||
|
||||
```python
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
|
||||
model_id = "pedrodev2026/microcoder-1.5b"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_id)
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Write a Python function that returns the nth Fibonacci number."
|
||||
}
|
||||
]
|
||||
|
||||
input_text = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True
|
||||
)
|
||||
|
||||
inputs = tokenizer(input_text, return_tensors="pt")
|
||||
outputs = model.generate(**inputs, max_new_tokens=256)
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Training Details
|
||||
|
||||
Microcoder 1.5B was fine-tuned using LoRA on top of Qwen 2.5 Coder 1.5B Instruct. The training focused on code-heavy datasets covering multiple programming languages and problem-solving scenarios, aiming to improve instruction-following and code correctness at a small model scale.
|
||||
|
||||
---
|
||||
|
||||
## Credits
|
||||
|
||||
- **Model credits** — see [`MODEL_CREDITS.md`](./MODEL_CREDITS.md)
|
||||
- **Dataset credits** — see [`DATASET_CREDITS.md`](./DATASET_CREDITS.md)
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
The Microcoder 1.5B model weights and associated code in this repository are released under the **BSD 3-Clause License**. See [`LICENSE`](./LICENSE) for details.
|
||||
|
||||
Note that the base model (Qwen 2.5 Coder 1.5B Instruct) and the datasets used for fine-tuning are subject to their own respective licenses, as detailed in the credit files above.
|
||||
|
||||
---
|
||||
|
||||
## Notice
|
||||
|
||||
The documentation files in this repository (including `README.md`, `MODEL_CREDITS.md`, `DATASET_CREDITS.md`, and other `.md` files) were generated with the assistance of an AI language model.
|
||||
54
chat_template.jinja
Normal file
54
chat_template.jinja
Normal file
@@ -0,0 +1,54 @@
|
||||
{%- if tools %}
|
||||
{{- '<|im_start|>system\n' }}
|
||||
{%- if messages[0]['role'] == 'system' %}
|
||||
{{- messages[0]['content'] }}
|
||||
{%- else %}
|
||||
{{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
|
||||
{%- endif %}
|
||||
{{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||
{%- for tool in tools %}
|
||||
{{- "\n" }}
|
||||
{{- tool | tojson }}
|
||||
{%- endfor %}
|
||||
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||
{%- else %}
|
||||
{%- if messages[0]['role'] == 'system' %}
|
||||
{{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- for message in messages %}
|
||||
{%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
|
||||
{%- elif message.role == "assistant" %}
|
||||
{{- '<|im_start|>' + message.role }}
|
||||
{%- if message.content %}
|
||||
{{- '\n' + message.content }}
|
||||
{%- endif %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if tool_call.function is defined %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '\n<tool_call>\n{"name": "' }}
|
||||
{{- tool_call.name }}
|
||||
{{- '", "arguments": ' }}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{{- '}\n</tool_call>' }}
|
||||
{%- endfor %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif message.role == "tool" %}
|
||||
{%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
|
||||
{{- '<|im_start|>user' }}
|
||||
{%- endif %}
|
||||
{{- '\n<tool_response>\n' }}
|
||||
{{- message.content }}
|
||||
{{- '\n</tool_response>' }}
|
||||
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|im_start|>assistant\n' }}
|
||||
{%- endif %}
|
||||
62
config.json
Normal file
62
config.json
Normal file
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Qwen2ForCausalLM"
|
||||
],
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"dtype": "bfloat16",
|
||||
"eos_token_id": 151645,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 1536,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 8960,
|
||||
"layer_types": [
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention"
|
||||
],
|
||||
"max_position_embeddings": 32768,
|
||||
"max_window_layers": 21,
|
||||
"model_type": "qwen2",
|
||||
"num_attention_heads": 12,
|
||||
"num_hidden_layers": 28,
|
||||
"num_key_value_heads": 2,
|
||||
"pad_token_id": 151665,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_parameters": {
|
||||
"rope_theta": 1000000.0,
|
||||
"rope_type": "default"
|
||||
},
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "5.3.0",
|
||||
"unsloth_fixed": true,
|
||||
"use_cache": true,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936
|
||||
}
|
||||
15
generation_config.json
Normal file
15
generation_config.json
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"bos_token_id": 151643,
|
||||
"do_sample": true,
|
||||
"eos_token_id": [
|
||||
151645,
|
||||
151643
|
||||
],
|
||||
"max_length": 32768,
|
||||
"pad_token_id": 151665,
|
||||
"repetition_penalty": 1.1,
|
||||
"temperature": 0.7,
|
||||
"top_k": 20,
|
||||
"top_p": 0.8,
|
||||
"transformers_version": "5.3.0"
|
||||
}
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:061d181166df0b8232bfdf8624669f4b8676bb1711de5da78db0c0407abe4516
|
||||
size 3087467144
|
||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
|
||||
size 11422356
|
||||
16
tokenizer_config.json
Normal file
16
tokenizer_config.json
Normal file
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"add_prefix_space": false,
|
||||
"backend": "tokenizers",
|
||||
"bos_token": null,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": [],
|
||||
"is_local": true,
|
||||
"model_max_length": 32768,
|
||||
"pad_token": "<|PAD_TOKEN|>",
|
||||
"padding_side": "right",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null
|
||||
}
|
||||
Reference in New Issue
Block a user