初始化项目,由ModelHub XC社区提供模型

Model: qingy2024/GRMR-V3-Q1.7B
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-04 06:24:18 +08:00
commit 0af27ded00
12 changed files with 152053 additions and 0 deletions

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

223
README.md Normal file
View File

@@ -0,0 +1,223 @@
---
base_model: unsloth/Qwen3-1.7B-Base
tags:
- text-generation-inference
- transformers
- unsloth
- qwen
- trl
- sft
license: apache-2.0
language:
- en
---
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
</head>
<div class="container"><h1>GRMR-V3-Q1.7B</h1><p>GRMR-V3-Q1.7B is a fine-tuned version of <a href="https://huggingface.co/unsloth/Qwen3-1.7B-Base">unsloth/Qwen3-1.7B-Base</a> specifically optimized for grammar correction tasks.</p><div class="important-note"><p><strong>IMPORTANT:</strong> Please ensure you are using the following sampler settings for optimal results:</p><pre><code>temperature = 0.7
frequency_penalty = 0.0
presence_penalty = 0.0
min_p = 0.01
top_p = 0.95
top_k = 40</code></pre></div><h2>Model description</h2><p>GRMR-V3-Q1.7B is a grammar correction model built on Qwen3 1.7B base model. It has been fine-tuned on a large dataset of grammar correction examples to help improve text quality by fixing grammatical errors, punctuation, spelling, and other language issues.</p><p>The model uses a specialized chat template that structures inputs as "text" and outputs as "corrected" to maintain a clear distinction between original and corrected content.</p><h3>Model Details</h3><div class="model-meta"><p><strong>Developed by:</strong> qingy2024</p><p><strong>Finetuned from model:</strong> <a href="https://huggingface.co/unsloth/Qwen3-1.7B-Base">unsloth/Qwen3-1.7B-Base</a></p><p><strong>Model type:</strong> Qwen3</p><p><strong>Language(s):</strong> English</p><p><strong>License:</strong> apache-2.0</p><p><strong>Training Dataset:</strong> <a href="https://huggingface.co/datasets/qingy2024/grmr-v4-60k">qingy2024/grmr-v4-60k</a></p></div><hr><h2>Manual Testing Examples</h2><p>Here are a few examples of grammar corrections this model can handle:</p><table><thead><tr><th>Original Text</th><th>Corrected Text</th></tr></thead><tbody><tr><td>i dont know weather to bring a umbrella today</td><td>I don't know whether to bring an umbrella today.</td></tr><tr><td>she go to the store yesterday</td><td>She went to the store yesterday.</td></tr></tr><tr><td>they is going to be late for the meeting</td><td>They are going to be late for the meeting.</td></tr><tr><td>the cat laying on the floor all day</td><td>The cat is laying on the floor all day.</td></tr></tbody></table><hr><h2>Training procedure</h2><p>The model was fine-tuned using full parameter fine-tuning (not LoRA) on the GRMR-V4-60K dataset. The training was optimized using the Unsloth framework for efficient training of LLMs.</p><h3>Training hyperparameters</h3><ul><li><strong>Batch size:</strong> 8</li><li><strong>Gradient accumulation steps:</strong> 2</li><li><strong>Learning rate:</strong> 5e-5</li><li><strong>Epochs:</strong> 1</li><li><strong>Optimizer:</strong> AdamW (8-bit)</li><li><strong>Weight decay:</strong> 0.01</li><li><strong>LR scheduler:</strong> Cosine</li><li><strong>Warmup steps:</strong> 180</li><li><strong>Max sequence length:</strong> 16,384</li><li><strong>Training precision:</strong> Mixed precision (BF16 where available, FP16 otherwise)</li></ul><h2>Intended uses & limitations</h2><p>This model is designed for grammar correction tasks. It can be used to:</p><ul><li>Fix grammatical errors in written text</li><li>Correct punctuation</li><li>Address spelling mistakes</li><li>Improve sentence structure and clarity</li></ul><h3>Limitations</h3><ul><li>The model may struggle with highly technical or domain-specific content</li><li>It may not fully understand context-dependent grammar rules in all cases</li><li>Performance may vary for non-standard English or text with multiple errors</li></ul><h2>How to use</h2><p>Projects based on Hugging Face transformers should be able to run this model easily.</p><p>For pure <code>transformers</code> code, you can refer here:</p><pre><code class="language-python">from transformers import AutoModelForCausalLM, AutoTokenizer
# Load model and tokenizer
model_name = "qingy2024/GRMR-V3-Q1.7B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Text with grammar errors to correct
text_to_correct = "i am going to the store tommorow and buy some thing for dinner"
# Format as messages
messages = [
{"role": "user", "content": text_to_correct}
]
# Apply the custom chat template
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
inputs["input_ids"],
max_new_tokens=512,
temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7
do_sample=True
)
# Decode and print the corrected text
corrected_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(corrected_text)</code></pre><h3>Using with the Hugging Face pipeline</h3><pre><code class="language-python">from transformers import pipeline
pipe = pipeline(
"text-generation",
model="qingy2024/GRMR-V3-Q1.7B",
torch_dtype="auto",
device_map="auto"
)
messages = [
{"role": "user", "content": "i dont know weather to bring a umbrella today"}
]
result = pipe(
messages,
max_new_tokens=100,
temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7
do_sample=True,
return_full_text=False
)[0]["generated_text"]
print(result)</code></pre><p><em>Note: The Python examples above use <code>temperature=0.1</code> for reproducibility in quick tests. For optimal grammar correction quality, please use the recommended sampler settings, especially <code>temperature=0.7</code>.</em></p><h2>Custom Chat Template</h2><p class="chat-template-info">The model uses a custom chat template with special formatting for grammar correction:</p><ul><li>User inputs are formatted with <code><|text_start|></code> and <code><|text_end|></code> tags</li><li>Model outputs are formatted with <code><|corrected_start|></code> and <code><|corrected_end|></code> tags</li></ul><p>The complete chat template is:</p><pre><code class="language-jinja">{%- for message in messages %}
{%- if message.role == "user" %}
{{- '<|text_start|>\n' + message.content + '<|text_end|>\n' }}
{%- elif message.role == "assistant" %}
{{- '<|corrected_start|>\n' + message.content + '<|corrected_end|>\n' }}
{%- else %}
{{- raise('Unknown role: ' + message.role) }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|corrected_start|>\n' }}
{%- endif %}</code></pre><h2>Training Dataset</h2><p>The model was fine-tuned on the <a href="https://huggingface.co/datasets/qingy2024/grmr-v4-60k">qingy2024/grmr-v4-60k</a> dataset, which contains 60,000 examples of original text and their grammatically corrected versions.</p><h2>Bias, Risks, and Limitations</h2><ul><li>The model may reflect biases present in the training data</li><li>It may not perform equally well across different writing styles or domains</li><li>The model might occasionally introduce errors or change the meaning of text</li><li>It focuses on grammatical correctness rather than stylistic improvements</li></ul><h2>Citations</h2><pre><code>@misc{qwen3technicalreport,
title={Qwen3 Technical Report},
author={Qwen Team},
year={2025},
eprint={2505.09388},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.09388},
}</code></pre><h2>Contact</h2><p>For questions or issues related to the model, please reach out via Hugging Face or by creating an issue in the repository.</p></div>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";
line-height: 1.6;
margin: 0;
padding: 0;
background-color: #f8f9fa;
color: #333;
}
.container {
max-width: 1200px;
margin: 10px auto;
padding: 25px;
background-color: #ffffff;
border-radius: 8px;
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.08);
}
h1, h2, h3 {
color: #0056b3; /* Primary Blue */
margin-top: 1.5em;
margin-bottom: 0.7em;
}
h1 {
text-align: center;
font-size: 2.2em;
border-bottom: 2px solid #e0e0e0;
padding-bottom: 0.5em;
margin-top: 0;
}
h2 {
font-size: 1.8em;
border-bottom: 1px solid #e9ecef;
padding-bottom: 0.3em;
}
h3 {
font-size: 1.4em;
color: #007bff; /* Lighter Blue for sub-headings */
}
p, li {
font-size: 1em;
color: #555;
}
a {
color: #007bff;
text-decoration: none;
}
a:hover {
text-decoration: underline;
color: #0056b3;
}
.important-note {
background-color: #e7f3ff; /* Light blue background */
border-left: 5px solid #007bff; /* Blue accent border */
margin: 20px 0px;
border-radius: 5px;
}
.important-note strong {
color: #0056b3;
font-weight: 600;
}
.important-note {
background-color: #d0e8ff;
padding: 0.05em 1.0em;
border-radius: 3px;
font-size: 0.9em;
}
code {
padding: 0.1em 0.4em;
border-radius: 3px;
font-size: 0.9em;
}
table {
width: 100%;
border-collapse: collapse;
margin: 20px 0;
box-shadow: 0 2px 4px rgba(0,0,0,0.05);
}
th, td {
border: 1px solid #dee2e6;
padding: 10px 12px;
text-align: left;
vertical-align: top;
}
th {
background-color: #e9ecef; /* Light gray for headers */
font-weight: 600;
color: #212529;
}
td:first-child {
/* font-style: italic; */
color: #444;
}
pre {
background-color: #f1f3f5;
padding: 15px;
border-radius: 5px;
overflow-x: auto;
border: 1px solid #ced4da;
font-size: 0.9em;
}
code {
font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;
background-color: #e9ecef;
padding: 0.2em 0.4em;
border-radius: 3px;
font-size: 0.9em;
}
pre code {
background-color: transparent;
padding: 0;
border-radius: 0;
font-size: 1em;
}
ul {
padding-left: 20px;
}
li {
margin-bottom: 0.5em;
}
hr {
border: none;
border-top: 1px solid #e0e0e0;
margin: 30px 0;
}
.model-meta {
background-color: #f8f9fa;
padding: 15px;
border-radius: 5px;
margin-bottom: 20px;
border: 1px solid #e9ecef;
}
.model-meta p { margin-bottom: 0.5em; }
.model-meta strong { color: #333; }
/* Specific styling for chat template explanation */
.chat-template-info span {
font-weight: bold;
color: #0056b3;
}
</style></html>

32
added_tokens.json Normal file
View File

@@ -0,0 +1,32 @@
{
"</think>": 151668,
"</tool_call>": 151658,
"</tool_response>": 151666,
"<think>": 151667,
"<tool_call>": 151657,
"<tool_response>": 151665,
"<|box_end|>": 151649,
"<|box_start|>": 151648,
"<|corrected_end|>": 151672,
"<|corrected_start|>": 151671,
"<|endoftext|>": 151643,
"<|file_sep|>": 151664,
"<|fim_middle|>": 151660,
"<|fim_pad|>": 151662,
"<|fim_prefix|>": 151659,
"<|fim_suffix|>": 151661,
"<|im_end|>": 151645,
"<|im_start|>": 151644,
"<|image_pad|>": 151655,
"<|object_ref_end|>": 151647,
"<|object_ref_start|>": 151646,
"<|quad_end|>": 151651,
"<|quad_start|>": 151650,
"<|repo_name|>": 151663,
"<|text_end|>": 151670,
"<|text_start|>": 151669,
"<|video_pad|>": 151656,
"<|vision_end|>": 151653,
"<|vision_pad|>": 151654,
"<|vision_start|>": 151652
}

17
chat_template.jinja Normal file
View File

@@ -0,0 +1,17 @@
{%- for message in messages %}
{%- if message.role == "user" %}
{{- '<|text_start|>
' + message.content + '<|text_end|>
' }}
{%- elif message.role == "assistant" %}
{{- '<|corrected_start|>
' + message.content + '<|corrected_end|>
' }}
{%- else %}
{{- raise('Unknown role: ' + message.role) }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|corrected_start|>
' }}
{%- endif %}

33
config.json Normal file
View File

@@ -0,0 +1,33 @@
{
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"chat_template": "{%- for message in messages %}\n{%- if message.role == \"user\" %}\n{{- '<|text_start|>\n' + message.content + '<|text_end|>\n' }}\n{%- elif message.role == \"assistant\" %}\n{{- '<|corrected_start|>\n' + message.content + '<|corrected_end|>\n' }}\n{%- else %}\n{# Raise an error for unsupported roles, as per requirement to remove system message stuff #}\n{{- raise('Unknown role: ' + message.role) }}\n{%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n{{- '<|corrected_start|>\n' }}\n{%- endif %}",
"eos_token_id": 151643,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 2048,
"initializer_range": 0.02,
"intermediate_size": 6144,
"max_position_embeddings": 32768,
"max_window_layers": 28,
"model_type": "qwen3",
"num_attention_heads": 16,
"num_hidden_layers": 28,
"num_key_value_heads": 8,
"pad_token_id": 151654,
"rms_norm_eps": 1e-06,
"rope_scaling": null,
"rope_theta": 1000000,
"sliding_window": null,
"tie_word_embeddings": true,
"torch_dtype": "bfloat16",
"transformers_version": "4.52.4",
"unsloth_fixed": true,
"unsloth_version": "2025.5.9",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151673
}

8
generation_config.json Normal file
View File

@@ -0,0 +1,8 @@
{
"bos_token_id": 151643,
"eos_token_id": 151643,
"max_length": 32768,
"max_new_tokens": 2048,
"pad_token_id": 151654,
"transformers_version": "4.52.4"
}

151388
merges.txt Normal file

File diff suppressed because it is too large Load Diff

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:96d143345340f933c9506b1a1debc421bb2d3b09cc905712ef5fc7948a9f948d
size 3440108360

46
special_tokens_map.json Normal file
View File

@@ -0,0 +1,46 @@
{
"additional_special_tokens": [
{
"content": "<|text_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
{
"content": "<|text_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
{
"content": "<|corrected_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
{
"content": "<|corrected_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
],
"eos_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<|vision_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:40fd5b853d0598451f720d22bf96c9c4cd4b5c6413cd74d4f5e3494ee990f38d
size 11423523

263
tokenizer_config.json Normal file
View File

@@ -0,0 +1,263 @@
{
"add_bos_token": false,
"add_prefix_space": false,
"added_tokens_decoder": {
"151643": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151644": {
"content": "<|im_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151645": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151646": {
"content": "<|object_ref_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151647": {
"content": "<|object_ref_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151648": {
"content": "<|box_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151649": {
"content": "<|box_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151650": {
"content": "<|quad_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151651": {
"content": "<|quad_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151652": {
"content": "<|vision_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151653": {
"content": "<|vision_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151654": {
"content": "<|vision_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151655": {
"content": "<|image_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151656": {
"content": "<|video_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151657": {
"content": "<tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151658": {
"content": "</tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151659": {
"content": "<|fim_prefix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151660": {
"content": "<|fim_middle|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151661": {
"content": "<|fim_suffix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151662": {
"content": "<|fim_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151663": {
"content": "<|repo_name|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151664": {
"content": "<|file_sep|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151665": {
"content": "<tool_response>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151666": {
"content": "</tool_response>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151667": {
"content": "<think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151668": {
"content": "</think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151669": {
"content": "<|text_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151670": {
"content": "<|text_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151671": {
"content": "<|corrected_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151672": {
"content": "<|corrected_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"additional_special_tokens": [
"<|text_start|>",
"<|text_end|>",
"<|corrected_start|>",
"<|corrected_end|>"
],
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|endoftext|>",
"errors": "replace",
"extra_special_tokens": {},
"model_max_length": 32768,
"pad_token": "<|vision_pad|>",
"padding_side": "right",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null
}

1
vocab.json Normal file

File diff suppressed because one or more lines are too long