144 lines
4.4 KiB
Markdown
144 lines
4.4 KiB
Markdown
---
|
||
language:
|
||
- zh
|
||
- en
|
||
|
||
license: apache-2.0
|
||
|
||
base_model: Qwen/Qwen3-8B
|
||
library_name: transformers
|
||
|
||
pipeline_tag: text-generation
|
||
|
||
tags:
|
||
- qwen
|
||
- lora
|
||
- sft
|
||
- chinese
|
||
- conversational
|
||
- style-transfer
|
||
|
||
datasets:
|
||
- hzb29/Zhoulifeng-Streaming-Dataset
|
||
---
|
||
|
||
# Qwen3-8B-FengGe-SFT
|
||
|
||
## 📌 Overview
|
||
|
||
**Qwen3-8B-FengGe-SFT** is a LoRA fine-tuned conversational model based on **Qwen/Qwen3-8B**, trained to mimic the distinctive speaking style (“峰哥”) using the **Zhoulifeng Streaming Dataset**.
|
||
|
||
📚 Training data: https://huggingface.co/datasets/hzb29/Zhoulifeng-Streaming-Dataset
|
||
|
||
The model is designed for informal, conversational Chinese dialogue with a strong personality style, emphasizing long-form, free-flowing responses.
|
||
|
||
---
|
||
|
||
## 🧠 Base Model
|
||
|
||
- **Model**: `Qwen/Qwen3-8B`
|
||
- **Fine-tuning method**: LoRA
|
||
|
||
---
|
||
|
||
## 📉 Training Loss
|
||
|
||
|
||
|
||
---
|
||
|
||
## 🚀 Inference
|
||
|
||
### Load with Hugging Face
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
model_name = "boqiny/Qwen3-8B-FengGe-SFT"
|
||
|
||
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
|
||
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
|
||
|
||
prompt = "峰哥我似乎总是爱上不会喜欢我的女生，有的干脆不搭理，这是命运的安排还是性压抑"
|
||
|
||
inputs = tokenizer(prompt, return_tensors="pt")
|
||
outputs = model.generate(
|
||
**inputs,
|
||
max_new_tokens=256,
|
||
repetition_penalty=1.3
|
||
)
|
||
|
||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
```
|
||
|
||
> ⚠️ Note: `repetition_penalty=1.3` is important to reduce looping/repetitive outputs.
|
||
|
||
---
|
||
|
||
## 💬 Example Generations
|
||
|
||
### Example 1
|
||
|
||
**User:**
|
||
峰哥我似乎总是爱上不会喜欢我的女生，有的干脆不搭理，这是命运的安排还是性压抑
|
||
|
||
**Assistant:**
|
||
你都没有接触过的对象你怎么就知道人家讨厌你也喜欢不上你自己呀？就是因为不知道原因才能让大脑胡思乱想了。反正你要是有机会真正见面试试才知道具体的结果到底是怎么回事。有时候男生在一起后啊聊聊天拉着手散步牵手搂腰抱大腿跳舞唱歌喝酒吃饭逛街逛商场看电影发微信刷抖音视频直播打游戏健身锻炼运动登山爬山露营蹦迪广场街边闲游夜店酒吧桑拿按摩足疗KTV歌厅酒店民宿农家乐烧烤园游泳馆温泉池水乐园动物园游乐场公园花园球场足球场篮球场羽毛球场网球场地自行车赛道马球赛射箭靶标射击场馆搏斗格挡竞技角拳击摔跤空战对抗表演魔幻魔术杂技特艺等都能体验一次两口子在一起时快乐时光越长越好，在一块的时间越多就越不容易分开离开彼此。相反如果你一直单飞孤军奋战独自一人形单影只孤独终老一辈子也没谈成恋爱怎么办呢？还能干点别的事挺开心的，找一群男家人组团出去玩一圈也很不错。
|
||
|
||
---
|
||
|
||
### Example 2
|
||
|
||
**User:**
|
||
峰哥，你是怎么做到解答世间万物最后得出这是好事的结论，能教教我吗
|
||
|
||
**Assistant:**
|
||
那就是你的价值观就是这样子而已。有些人可能认为这样不好那样不行，但我觉得好的地方太多了。只要能够满足人类的基本需求就可以，没必要追求一些高精尖的东西，没有必要做那种超越时代的事儿也不必要成为名人富豪科学家啥的，只要是安安稳稳的生活就好。所以说我的观点也是基于我自己生活经验之上形成的这么个看法。没有什么特别的原因只是因为我更喜欢这样的生活方式罢了，就这么简单。
|
||
|
||
---
|
||
|
||
## 📦 Training with LLaMA-Factory
|
||
|
||
This model was trained using **LLaMA-Factory** with the following configuration:
|
||
|
||
```yaml
|
||
### model
|
||
model_name_or_path: Qwen/Qwen3-8B
|
||
trust_remote_code: true
|
||
|
||
### method
|
||
stage: sft
|
||
do_train: true
|
||
finetuning_type: lora
|
||
lora_rank: 32
|
||
lora_target: all
|
||
|
||
### dataset
|
||
dataset: zhoulifeng_streaming
|
||
template: qwen3_nothink
|
||
cutoff_len: 2048
|
||
|
||
### train
|
||
per_device_train_batch_size: 1
|
||
gradient_accumulation_steps: 8
|
||
learning_rate: 5.0e-5
|
||
num_train_epochs: 2.0
|
||
lr_scheduler_type: cosine
|
||
warmup_ratio: 0.1
|
||
bf16: true
|
||
```
|
||
|
||
---
|
||
|
||
## ⚠️ Limitations
|
||
|
||
- Strong stylistic bias toward “峰哥” conversational tone
|
||
- May generate long, repetitive enumerations without proper decoding settings
|
||
- Not suitable for factual QA or safety-critical applications
|
||
|
||
---
|
||
|
||
## 📜 License
|
||
|
||
Please follow the license of the base model:
|
||
👉 Qwen/Qwen3-8B |