初始化项目，由ModelHub XC社区提供模型

Model: praveenkrovvidi/rl-cas-trl-agent
Source: Original Platform

README.md

@@ -0,0 +1,95 @@
+---
+license: apache-2.0
+language:
+  - en
+base_model: Qwen/Qwen2.5-3B-Instruct
+pipeline_tag: text-generation
+tags:
+  - reinforcement-learning
+  - grpo
+  - customer-service
+  - tool-use
+  - sft
+  - qlora
+---
+
+# RL-CAS TRL Agent
+
+Fine-tuned **Qwen2.5-3B-Instruct** that selects enterprise tools to resolve customer service queries. Trained via **SFT + GRPO** on expert trajectories from an RL-trained SAC agent.
+
+> **Project:** [huggingface.co/spaces/praveenkrovvidi/rl-cas](https://huggingface.co/spaces/praveenkrovvidi/rl-cas)
+
+## Training Pipeline
+
+1. **SAC-Discrete** (Round 1) -- MLP [256,256] trained for 3,000 episodes on a 14-signal dense reward environment with 13 actions (7 API tools + 5 terminal + 1 meta)
+2. **SFT** (Round 2a) -- QLoRA fine-tuning on SAC expert trajectories (reward >= 5.0), 3 epochs, lr=2e-4
+3. **GRPO** (Round 2b) -- RL with live environment rewards, 482 steps, lr=5e-5, 4 completions/prompt
+
+## Model Details
+
+| | |
+|---|---|
+| **Base model** | Qwen/Qwen2.5-3B-Instruct |
+| **Architecture** | Qwen2ForCausalLM, 36 layers, 2048 hidden, GQA (16 heads / 2 KV) |
+| **Parameters** | 3B (fp16, 5.8 GB) |
+| **Context length** | 32,768 tokens |
+| **Fine-tuning** | QLoRA (rank=16, alpha=32, 4-bit NF4) merged to fp16 |
+| **Hardware** | NVIDIA A10G (24 GB VRAM) |
+
+## Input / Output
+
+**Input:** System prompt + user query describing a customer service issue.
+
+**Output:** JSON with chain-of-thought reasoning and action ID.
+
+```json
+{"reasoning": "Customer needs order status. I should look up their order first.", "action_id": 0}
+```
+
+### Action Space
+
+| ID | Action | Type |
+|----|--------|------|
+| 0 | `invoke_order_service` | tool |
+| 1 | `invoke_refund_service` | tool |
+| 2 | `invoke_customer_service` | tool |
+| 3 | `invoke_auth_service` | tool |
+| 4 | `invoke_logistics_service` | tool |
+| 5 | `invoke_inventory_service` | tool |
+| 6 | `invoke_knowledge_base` | tool |
+| 7 | `resolve_query` | terminal |
+| 8 | `request_customer_information` | terminal |
+| 9 | `route_to_billing_department` | terminal |
+| 10 | `route_to_technical_support` | terminal |
+| 11 | `escalate_to_human_agent` | terminal |
+| 12 | `consult_llm` | meta |
+
+## Benchmark (100 Queries, 8 Categories)
+
+| Agent | Avg Reward | Resolution | Avg Steps |
+|-------|-----------|-----------|----------|
+| GPT-4o-mini (zero-shot) | 19.43 | 100% | 2.5 |
+| SAC (MLP policy) | 19.36 | 100% | 3.0 |
+| Gemini-2.5-flash | 16.46 | 100% | 2.2 |
+| **This model (TRL)** | **15.58** | **97%** | **3.1** |
+
+## Intended Use
+
+Designed for the RL-CAS customer service environment. Selects tools from a fixed action space -- does **not** generate free-form customer responses.
+
+## Limitations
+
+- 97% resolution rate (struggles with `auth` and `refund` multi-tool sequences)
+- Requires the RL-CAS mock backend for tool execution
+- Trained on synthetic customer service data, not real customer interactions
+
+## Citation
+
+```bibtex
+@misc{rlcas2026,
+  title={RL-CAS: Reinforcement Learning for Customer Service Agent},
+  author={Praveen Krovvidi},
+  year={2026},
+  url={https://huggingface.co/praveenkrovvidi/rl-cas-trl-agent}
+}
+```