初始化项目,由ModelHub XC社区提供模型
Model: g023/qwen3-tiny-v2-finetuned Source: Original Platform
This commit is contained in:
1
.gitattributes
vendored
Normal file
1
.gitattributes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
*.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
78
Modelfile
Normal file
78
Modelfile
Normal file
@@ -0,0 +1,78 @@
|
||||
FROM ./Qwen3-g023-tiny-v2-FT-Q8_0.gguf
|
||||
TEMPLATE """
|
||||
{{- $lastUserIdx := -1 -}}
|
||||
{{- range $idx, $msg := .Messages -}}
|
||||
{{- if eq $msg.Role "user" }}{{ $lastUserIdx = $idx }}{{ end -}}
|
||||
{{- end }}
|
||||
{{- if or .System .Tools }}<|im_start|>system
|
||||
{{ if .System }}
|
||||
{{ .System }}
|
||||
{{- end }}
|
||||
{{- if .Tools }}
|
||||
|
||||
# Tools
|
||||
|
||||
You may call one or more functions to assist with the user query.
|
||||
|
||||
You are provided with function signatures within <tools></tools> XML tags:
|
||||
<tools>
|
||||
{{- range .Tools }}
|
||||
{"type": "function", "function": {{ .Function }}}
|
||||
{{- end }}
|
||||
</tools>
|
||||
|
||||
For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
|
||||
<tool_call>
|
||||
{"name": <function-name>, "arguments": <args-json-object>}
|
||||
</tool_call>
|
||||
{{- end -}}
|
||||
<|im_end|>
|
||||
{{ end }}
|
||||
{{- range $i, $_ := .Messages }}
|
||||
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
|
||||
{{- if eq .Role "user" }}<|im_start|>user
|
||||
{{ .Content }}
|
||||
{{- if and $.IsThinkSet (eq $i $lastUserIdx) }}
|
||||
{{- if $.Think -}}
|
||||
{{- " "}}/think
|
||||
{{- else -}}
|
||||
{{- " "}}/no_think
|
||||
{{- end -}}
|
||||
{{- end }}<|im_end|>
|
||||
{{ else if eq .Role "assistant" }}<|im_start|>assistant
|
||||
{{ if (and $.IsThinkSet (and .Thinking (or $last (gt $i $lastUserIdx)))) -}}
|
||||
<think>{{ .Thinking }}</think>
|
||||
{{ end -}}
|
||||
{{ if .Content }}{{ .Content }}
|
||||
{{- else if .ToolCalls }}<tool_call>
|
||||
{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
|
||||
{{ end }}</tool_call>
|
||||
{{- end }}{{ if not $last }}<|im_end|>
|
||||
{{ end }}
|
||||
{{- else if eq .Role "tool" }}<|im_start|>user
|
||||
<tool_response>
|
||||
{{ .Content }}
|
||||
</tool_response><|im_end|>
|
||||
{{ end }}
|
||||
{{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
|
||||
{{ if and $.IsThinkSet (not $.Think) -}}
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
{{ end -}}
|
||||
{{ end }}
|
||||
{{- end }}
|
||||
"""
|
||||
PARAMETER num_ctx 40000
|
||||
PARAMETER repeat_last_n 16384
|
||||
PARAMETER stop <|im_start|>
|
||||
PARAMETER stop <|im_end|>
|
||||
PARAMETER temperature 0.65
|
||||
PARAMETER top_p 0.9
|
||||
PARAMETER top_k 20
|
||||
PARAMETER min_p 0.0
|
||||
PARAMETER repeat_penalty 1.05
|
||||
PARAMETER presence_penalty 0.1
|
||||
PARAMETER frequency_penalty 0.1
|
||||
SYSTEM "You are a helpful assistant."
|
||||
3
Qwen3-g023-tiny-v2-FT-Q8_0.gguf
Normal file
3
Qwen3-g023-tiny-v2-FT-Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f0ed606874dbfaf528ea1bef31edc70d63f4ba91707162871edb3690ea2ed0b9
|
||||
size 1941413632
|
||||
68
README.md
Normal file
68
README.md
Normal file
@@ -0,0 +1,68 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
base_model: g023/qwen3-tiny-v2
|
||||
tags:
|
||||
- qwen3
|
||||
- gguf
|
||||
- q8_0
|
||||
- finetuned
|
||||
- grpo
|
||||
- lora-merged
|
||||
- text-generation
|
||||
pipeline_tag: text-generation
|
||||
library_name: llama.cpp
|
||||
quantized_by: g023
|
||||
---
|
||||
|
||||
# Qwen3-g023-tiny-v2-FT-Q8_0 - GRPO Finetuned Q8_0 GGUF Export
|
||||
|
||||
https://huggingface.co/g023/qwen3-tiny-v2-finetuned/
|
||||
|
||||
Q8_0 GGUF export of a GRPO finetuned Qwen3 model to achieve improved reasoning and reduced repetition.
|
||||
Original SRC Model: https://huggingface.co/g023/qwen3-tiny-v2
|
||||
|
||||
*THIS IS A WIP (WORK IN PROGRESS)*
|
||||
|
||||
## Files
|
||||
|
||||
- `Qwen3-g023-tiny-v2-FT-Q8_0.gguf`: Q8_0 GGUF model (~1.81 GB)
|
||||
- `Modelfile`: Ollama template + tested default sampling settings
|
||||
- `params_best.json`: Best sampled parameters from automated sweep
|
||||
- `sweep_results.json`: Full sweep results and per-test outcomes
|
||||
|
||||
## Tested Best Parameters (Default in Modelfile)
|
||||
|
||||
- `temperature`: 0.65
|
||||
- `top_p`: 0.9
|
||||
- `top_k`: 20
|
||||
- `min_p`: 0.0
|
||||
- `repeat_penalty`: 1.05
|
||||
- `presence_penalty`: 0.1
|
||||
- `frequency_penalty`: 0.1
|
||||
- `num_ctx`: 40000
|
||||
|
||||
## Usage (Ollama)
|
||||
|
||||
```bash
|
||||
ollama create qwen3-g023-tiny-v2-FT-Q8_0 -f Modelfile
|
||||
ollama run qwen3-g023-tiny-v2-FT-Q8_0
|
||||
|
||||
# thinking on
|
||||
ollama run qwen3-g023-tiny-v2-FT-Q8_0 --think "Explain why the sky is blue"
|
||||
|
||||
# thinking off
|
||||
ollama run qwen3-g023-tiny-v2-FT-Q8_0 --think=false "Explain why the sky is blue"
|
||||
```
|
||||
|
||||
### or pull from huggingface directly to ollama:
|
||||
|
||||
```bash
|
||||
ollama run hf.co/g023/qwen3-tiny-v2-finetuned:Q8_0
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Template is the Qwen3-compatible template with think/no_think handling.
|
||||
- If you want stricter non-thinking behavior, compare alternatives in `sweep_results.json`.
|
||||
10
params_best.json
Normal file
10
params_best.json
Normal file
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"label": "balanced_01",
|
||||
"temperature": 0.65,
|
||||
"top_p": 0.9,
|
||||
"top_k": 20,
|
||||
"min_p": 0.0,
|
||||
"repeat_penalty": 1.05,
|
||||
"presence_penalty": 0.1,
|
||||
"frequency_penalty": 0.1
|
||||
}
|
||||
1272
sweep_results.json
Normal file
1272
sweep_results.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user