初始化项目，由ModelHub XC社区提供模型

Model: blackhao0426/pref-extractor-qwen3-0.6b-full-sft Source: Original Platform
2026-04-23 23:35:05 +08:00
commit 69e312bc01
12 changed files with 152006 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,115 @@
+---
+base_model: Qwen/Qwen3-0.6B
+datasets:
+- blackhao0426/user-preference-564k
+language:
+- en
+license: apache-2.0
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- preference-extraction
+- personalization
+- user-modeling
+---
+
+# Preference Extractor (Qwen3-0.6B Full SFT)
+
+[![arXiv](https://img.shields.io/badge/arXiv-2603.20939-b31b1b.svg)](https://arxiv.org/abs/2603.20939)
+[![Code](https://img.shields.io/badge/GitHub-VARS-black?logo=github)](https://github.com/YurenHao0426/VARS)
+[![Dataset](https://img.shields.io/badge/🤗_Dataset-user--preference--564k-blue)](https://huggingface.co/datasets/blackhao0426/user-preference-564k)
+
+A lightweight preference extraction model finetuned from [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B). Given a dialogue window, it outputs structured JSON preference tuples describing user style preferences.
+
+This model was introduced in the paper [User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction](https://arxiv.org/abs/2603.20939). It is a core component of **VARS** (Vector-Adapted Retrieval Scoring), a framework for personalized LLM assistants.
+
+## Task
+
+Given recent conversation turns between a user and an AI assistant, extract user preferences as structured condition-action rules in JSON format:
+
+```json
+{
+  "preferences": [
+    {"condition": "when explaining math", "action": "show step-by-step derivation", "confidence": 0.9},
+    {"condition": "general", "action": "respond in Chinese", "confidence": 0.85}
+  ]
+}
+```
+
+Each preference is a `(condition, action, confidence)` tuple:
+- **condition**: when this preference applies (e.g., "when writing code", "general", "when debugging")
+- **action**: the preferred behavior (e.g., "use Python with type hints", "show step-by-step derivation")
+- **confidence**: extraction confidence score (0.0–1.0)
+
+## Performance
+
+Evaluated on a held-out test set:
+
+| Metric | Value |
+|--------|-------|
+| JSON validity | 99.7% |
+| Recall | 97.5% |
+| Precision | 37.7% |
+| Eval loss | 0.1611 |
+
+The model intentionally favors **high recall** over precision: it over-extracts candidate preferences, and the downstream reranker and user vector in the VARS pipeline filter irrelevant ones.
+
+## Usage
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "blackhao0426/pref-extractor-qwen3-0.6b-full-sft"
+tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="bfloat16", device_map="auto", trust_remote_code=True)
+
+# system_prompt should describe the extraction task
+system_prompt = "You are a preference extractor. Given the dialogue window, extract preferences into JSON format."
+
+conversation = """User: Can you explain the quicksort algorithm? I prefer step-by-step breakdowns with Python code.
+Assistant: Sure! Here's quicksort step by step...
+User: Great, but can you add type hints to the code?"""
+
+messages = [
+    {"role": "system", "content": system_prompt},
+    {"role": "user", "content": conversation},
+]
+
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+with torch.no_grad():
+    outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
+
+# Decode only the newly generated tokens
+result = tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)
+# Parse JSON from result to get preference list
+```
+
+## Training
+
+| Parameter | Value |
+|-----------|-------|
+| Base model | Qwen/Qwen3-0.6B |
+| Training data | [blackhao0426/user-preference-564k](https://huggingface.co/datasets/blackhao0426/user-preference-564k) (564K examples) |
+| Learning rate | 2e-05 |
+| Batch size | 128 (32 per device x 4 GPUs) |
+| Epochs | 1 |
+| LR scheduler | Cosine with 5% warmup |
+| Optimizer | AdamW (fused) |
+| Framework | LLaMA-Factory |
+
+## Citation
+
+```bibtex
+@misc{hao2026userpreferencemodelingconversational,
+      title={User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction},
+      author={Yuren Hao and Shuhaib Mehri and ChengXiang Zhai and Dilek Hakkani-Tür},
+      year={2026},
+      eprint={2603.20939},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2603.20939},
+}
+```