初始化项目，由ModelHub XC社区提供模型

Model: Jinyang23/Seed-AlfWorld-3B Source: Original Platform
2026-07-22 18:57:12 +08:00
commit 1587fb3f79
14 changed files with 152472 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,199 @@
+---
+license: other
+license_name: qwen-research
+license_link: https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE
+language:
+- en
+metrics:
+- accuracy
+base_model:
+- Qwen/Qwen2.5-3B-Instruct
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- agent
+- reinforcement-learning
+- long-horizon
+- embodied-ai
+- self-distillation
+- on-policy-distillation
+- alfworld
+- seed
+- qwen2.5
+---
+
+# SEED-ALFWorld-3B
+
+## Overview
+
+Built with Qwen2.5-3B-Instruct, SEED-ALFWorld-3B is trained for text-based
+embodied interaction in ALFWorld using **SEED** (Self-Evolving On-Policy
+Distillation). Training unfolds in two stages:
+
+- **Hindsight-skill SFT:** the model learns to analyze a completed trajectory
+  and summarize it as a concise, episode-level skill.
+- **Self-evolving on-policy distillation:** in each reinforcement-learning
+  iteration, the current policy both acts in the environment and analyzes the
+  trajectories it has just collected. The generated skills are used to re-score
+  the same sampled actions, providing dense token-level supervision alongside
+  GRPO. Once updated, the improved policy takes on both roles in the next
+  iteration, allowing its behavior and hindsight supervision to evolve
+  together.
+
+At inference time, the model acts from the standard interaction history alone,
+with no analyzer, skill bank, retrieval module, or additional skill prompt.
+
+## Key Features
+
+- **Self-evolving supervision:** policy updates improve both action selection
+  and the analyzer used to generate the next round of hindsight skills.
+- **On-policy skill generation:** supervision follows the states, actions, and
+  failure modes encountered by the current policy.
+- **Dense token-level credit:** skill-induced log-probability shifts provide
+  decision-specific guidance beyond terminal rewards.
+- **No inference overhead:** privileged hindsight skills are used only during
+  training and are internalized by the released policy.
+## Performance
+
+The following results are success rates (%) on the ALFWorld seen test
+split. `Avg.` is the unweighted macro-average over the six task families. An
+asterisk indicates that the method also receives a skill prompt at evaluation;
+SEED does not use skills at evaluation.
+
+| Method | Pick | Look | Clean | Heat | Cool | Pick2 | Avg. |
+|---|---:|---:|---:|---:|---:|---:|---:|
+| Vanilla | 44.4 | 11.1 | 6.2 | 15.4 | 28.6 | 12.5 | 21.9 |
+| Skill-Prompt* | 51.7 | 66.7 | 48.4 | 0.0 | 4.3 | 10.0 | 28.9 |
+| OPSD | 48.8 | 41.7 | 16.7 | 0.0 | 15.8 | 16.7 | 28.1 |
+| GRPO | 91.2 | 62.5 | 96.2 | 61.9 | 65.0 | 47.4 | 75.0 |
+| Skill-GRPO | 88.9 | 71.4 | 58.8 | 70.6 | 40.7 | 29.2 | 60.2 |
+| Skill-GRPO* | 94.3 | 57.1 | 100.0 | 66.7 | 73.1 | 57.1 | 80.5 |
+| GRPO+OPSD | 100.0 | 82.4 | 85.7 | 75.0 | 70.0 | 60.0 | 81.2 |
+| Skill-SD | 88.2 | 50.0 | 96.2 | 52.4 | 65.0 | 57.9 | 73.4 |
+| RLSD | 87.9 | 75.0 | 90.9 | 75.0 | 73.1 | 68.4 | 79.7 |
+| SDAR | 97.1 | 62.5 | 100.0 | 61.9 | 75.0 | 84.2 | 84.4 |
+| **SEED (ours)** | **100.0** | **100.0** | **100.0** | **100.0** | **70.6** | **80.0** | **91.8** |
+
+On the ALFWorld unseen split, the same 3B SEED checkpoint reaches an
+86.2 macro-average, compared with 70.9 for the matched GRPO baseline.
+
+| Method | Pick | Look | Clean | Heat | Cool | Pick2 | Avg. |
+|---|---:|---:|---:|---:|---:|---:|---:|
+| ReAct | 17.4 | 6.7 | 8.8 | 7.4 | 9.1 | 0.0 | 8.2 |
+| GRPO | 73.9 | 60.0 | 82.4 | 59.3 | 72.7 | 76.9 | 70.9 |
+| **SEED (ours)** | **90.4** | **78.3** | **79.5** | **94.3** | **86.2** | **88.2** | **86.2** |
+
+## Quickstart
+
+Install the runtime dependencies:
+
+```bash
+pip install "transformers>=4.37.0" accelerate torch
+```
+
+Load the checkpoint and generate one ALFWorld action:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Jinyang23/Seed-AlfWorld-3B"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype="auto",
+    device_map="auto",
+)
+
+
+def build_alfworld_prompt(
+    task_description,
+    step_count,
+    history_length,
+    action_history,
+    current_step,
+    current_observation,
+    admissible_actions,
+):
+    actions = ", ".join(admissible_actions)
+    return f"""You are an expert agent operating in the ALFRED Embodied Environment. Your task is
+to: {task_description}
+Prior to this step, you have already taken {step_count} step(s). Below are the most recent
+{history_length} observations and the corresponding actions you took: {action_history}
+You are now at step {current_step} and your current observation is: {current_observation}
+Your admissible actions of the current situation are: [{actions}].
+
+Now it's your turn to take an action.
+
+You should first reason step-by-step about the current situation. This reasoning process
+MUST be enclosed within <think> </think> tags.
+Once you've finished your reasoning, you should choose an admissible action for current
+step and present it within <action> </action> tags."""
+
+
+prompt = build_alfworld_prompt(
+    task_description="put a candle in toilet.",
+    step_count=0,
+    history_length=0,
+    action_history="None.",
+    current_step=1,
+    current_observation="You are in the middle of a room.",
+    admissible_actions=["look", "go to shelf 1", "go to toilet 1"],
+)
+
+text = tokenizer.apply_chat_template(
+    [{"role": "user", "content": prompt}],
+    tokenize=False,
+    add_generation_prompt=True,
+)
+inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    do_sample=False,
+)
+new_tokens = outputs[:, inputs.input_ids.shape[1]:]
+response = tokenizer.batch_decode(new_tokens, skip_special_tokens=True)[0]
+print(response)
+```
+
+In a complete evaluation loop, replace the example values with the current
+ALFWorld task, recent observation-action history, observation, and admissible
+actions at every step. Only execute the action emitted inside `<action>` tags.
+
+The model can also be served with vLLM:
+
+```bash
+vllm serve Jinyang23/Seed-AlfWorld-3B \
+  --served-model-name Seed-AlfWorld-3B \
+  --dtype bfloat16
+```
+
+## Limitations
+
+- The model is specialized for ALFWorld-style text interaction and may not
+  transfer reliably to unrelated environments or prompting formats.
+
+## License
+
+This checkpoint is a derivative of Qwen2.5-3B-Instruct and is distributed under
+the [Qwen Research License Agreement](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE).
+
+## Citation
+
+Citation metadata will be added after the public paper release.
+
+## Acknowledgements
+
+The SEED implementation builds on
+[veRL](https://github.com/volcengine/verl),
+[verl-agent](https://github.com/langfengQ/verl-agent), and
+[SDAR](https://github.com/ZJU-REAL/SDAR).
+
+## Links
+
+- Model: https://huggingface.co/Jinyang23/Seed-AlfWorld-3B
+- Base model: https://huggingface.co/Qwen/Qwen2.5-3B-Instruct
+- Paper: https://arxiv.org/pdf/2607.14777
+- Code: https://github.com/jinyangwu/SEED