初始化项目,由ModelHub XC社区提供模型
Model: g023/qwen3-tiny-v2-finetuned Source: Original Platform
This commit is contained in:
1
.gitattributes
vendored
Normal file
1
.gitattributes
vendored
Normal file
@@ -0,0 +1 @@
|
|||||||
|
*.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
78
Modelfile
Normal file
78
Modelfile
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
FROM ./Qwen3-g023-tiny-v2-FT-Q8_0.gguf
|
||||||
|
TEMPLATE """
|
||||||
|
{{- $lastUserIdx := -1 -}}
|
||||||
|
{{- range $idx, $msg := .Messages -}}
|
||||||
|
{{- if eq $msg.Role "user" }}{{ $lastUserIdx = $idx }}{{ end -}}
|
||||||
|
{{- end }}
|
||||||
|
{{- if or .System .Tools }}<|im_start|>system
|
||||||
|
{{ if .System }}
|
||||||
|
{{ .System }}
|
||||||
|
{{- end }}
|
||||||
|
{{- if .Tools }}
|
||||||
|
|
||||||
|
# Tools
|
||||||
|
|
||||||
|
You may call one or more functions to assist with the user query.
|
||||||
|
|
||||||
|
You are provided with function signatures within <tools></tools> XML tags:
|
||||||
|
<tools>
|
||||||
|
{{- range .Tools }}
|
||||||
|
{"type": "function", "function": {{ .Function }}}
|
||||||
|
{{- end }}
|
||||||
|
</tools>
|
||||||
|
|
||||||
|
For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
|
||||||
|
<tool_call>
|
||||||
|
{"name": <function-name>, "arguments": <args-json-object>}
|
||||||
|
</tool_call>
|
||||||
|
{{- end -}}
|
||||||
|
<|im_end|>
|
||||||
|
{{ end }}
|
||||||
|
{{- range $i, $_ := .Messages }}
|
||||||
|
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
|
||||||
|
{{- if eq .Role "user" }}<|im_start|>user
|
||||||
|
{{ .Content }}
|
||||||
|
{{- if and $.IsThinkSet (eq $i $lastUserIdx) }}
|
||||||
|
{{- if $.Think -}}
|
||||||
|
{{- " "}}/think
|
||||||
|
{{- else -}}
|
||||||
|
{{- " "}}/no_think
|
||||||
|
{{- end -}}
|
||||||
|
{{- end }}<|im_end|>
|
||||||
|
{{ else if eq .Role "assistant" }}<|im_start|>assistant
|
||||||
|
{{ if (and $.IsThinkSet (and .Thinking (or $last (gt $i $lastUserIdx)))) -}}
|
||||||
|
<think>{{ .Thinking }}</think>
|
||||||
|
{{ end -}}
|
||||||
|
{{ if .Content }}{{ .Content }}
|
||||||
|
{{- else if .ToolCalls }}<tool_call>
|
||||||
|
{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
|
||||||
|
{{ end }}</tool_call>
|
||||||
|
{{- end }}{{ if not $last }}<|im_end|>
|
||||||
|
{{ end }}
|
||||||
|
{{- else if eq .Role "tool" }}<|im_start|>user
|
||||||
|
<tool_response>
|
||||||
|
{{ .Content }}
|
||||||
|
</tool_response><|im_end|>
|
||||||
|
{{ end }}
|
||||||
|
{{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
|
||||||
|
{{ if and $.IsThinkSet (not $.Think) -}}
|
||||||
|
<think>
|
||||||
|
|
||||||
|
</think>
|
||||||
|
|
||||||
|
{{ end -}}
|
||||||
|
{{ end }}
|
||||||
|
{{- end }}
|
||||||
|
"""
|
||||||
|
PARAMETER num_ctx 40000
|
||||||
|
PARAMETER repeat_last_n 16384
|
||||||
|
PARAMETER stop <|im_start|>
|
||||||
|
PARAMETER stop <|im_end|>
|
||||||
|
PARAMETER temperature 0.65
|
||||||
|
PARAMETER top_p 0.9
|
||||||
|
PARAMETER top_k 20
|
||||||
|
PARAMETER min_p 0.0
|
||||||
|
PARAMETER repeat_penalty 1.05
|
||||||
|
PARAMETER presence_penalty 0.1
|
||||||
|
PARAMETER frequency_penalty 0.1
|
||||||
|
SYSTEM "You are a helpful assistant."
|
||||||
3
Qwen3-g023-tiny-v2-FT-Q8_0.gguf
Normal file
3
Qwen3-g023-tiny-v2-FT-Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f0ed606874dbfaf528ea1bef31edc70d63f4ba91707162871edb3690ea2ed0b9
|
||||||
|
size 1941413632
|
||||||
68
README.md
Normal file
68
README.md
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
---
|
||||||
|
license: apache-2.0
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
base_model: g023/qwen3-tiny-v2
|
||||||
|
tags:
|
||||||
|
- qwen3
|
||||||
|
- gguf
|
||||||
|
- q8_0
|
||||||
|
- finetuned
|
||||||
|
- grpo
|
||||||
|
- lora-merged
|
||||||
|
- text-generation
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
library_name: llama.cpp
|
||||||
|
quantized_by: g023
|
||||||
|
---
|
||||||
|
|
||||||
|
# Qwen3-g023-tiny-v2-FT-Q8_0 - GRPO Finetuned Q8_0 GGUF Export
|
||||||
|
|
||||||
|
https://huggingface.co/g023/qwen3-tiny-v2-finetuned/
|
||||||
|
|
||||||
|
Q8_0 GGUF export of a GRPO finetuned Qwen3 model to achieve improved reasoning and reduced repetition.
|
||||||
|
Original SRC Model: https://huggingface.co/g023/qwen3-tiny-v2
|
||||||
|
|
||||||
|
*THIS IS A WIP (WORK IN PROGRESS)*
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
- `Qwen3-g023-tiny-v2-FT-Q8_0.gguf`: Q8_0 GGUF model (~1.81 GB)
|
||||||
|
- `Modelfile`: Ollama template + tested default sampling settings
|
||||||
|
- `params_best.json`: Best sampled parameters from automated sweep
|
||||||
|
- `sweep_results.json`: Full sweep results and per-test outcomes
|
||||||
|
|
||||||
|
## Tested Best Parameters (Default in Modelfile)
|
||||||
|
|
||||||
|
- `temperature`: 0.65
|
||||||
|
- `top_p`: 0.9
|
||||||
|
- `top_k`: 20
|
||||||
|
- `min_p`: 0.0
|
||||||
|
- `repeat_penalty`: 1.05
|
||||||
|
- `presence_penalty`: 0.1
|
||||||
|
- `frequency_penalty`: 0.1
|
||||||
|
- `num_ctx`: 40000
|
||||||
|
|
||||||
|
## Usage (Ollama)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ollama create qwen3-g023-tiny-v2-FT-Q8_0 -f Modelfile
|
||||||
|
ollama run qwen3-g023-tiny-v2-FT-Q8_0
|
||||||
|
|
||||||
|
# thinking on
|
||||||
|
ollama run qwen3-g023-tiny-v2-FT-Q8_0 --think "Explain why the sky is blue"
|
||||||
|
|
||||||
|
# thinking off
|
||||||
|
ollama run qwen3-g023-tiny-v2-FT-Q8_0 --think=false "Explain why the sky is blue"
|
||||||
|
```
|
||||||
|
|
||||||
|
### or pull from huggingface directly to ollama:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ollama run hf.co/g023/qwen3-tiny-v2-finetuned:Q8_0
|
||||||
|
```
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Template is the Qwen3-compatible template with think/no_think handling.
|
||||||
|
- If you want stricter non-thinking behavior, compare alternatives in `sweep_results.json`.
|
||||||
10
params_best.json
Normal file
10
params_best.json
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
{
|
||||||
|
"label": "balanced_01",
|
||||||
|
"temperature": 0.65,
|
||||||
|
"top_p": 0.9,
|
||||||
|
"top_k": 20,
|
||||||
|
"min_p": 0.0,
|
||||||
|
"repeat_penalty": 1.05,
|
||||||
|
"presence_penalty": 0.1,
|
||||||
|
"frequency_penalty": 0.1
|
||||||
|
}
|
||||||
1272
sweep_results.json
Normal file
1272
sweep_results.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user