初始化项目，由ModelHub XC社区提供模型

Model: AI71ai/agrillm-Qwen3-30B-A3B Source: Original Platform
2026-07-22 03:40:13 +08:00
commit acd024b5c3
26 changed files with 171114 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
--- a/AgriLLM_Logo.png
+++ b/AgriLLM_Logo.png
--- a/README.md
+++ b/README.md
@@ -0,0 +1,368 @@
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- Qwen/Qwen3-30B-A3B-Instruct-2507
+tags:
+- climate
+- agriculture
+- rag
+library_name: transformers
+datasets:
+- AI71ai/agrillm-train-146k
+---
+![AgriLLM](./AgriLLM_Logo.png)
+# Model Card for AI71ai/agrillm-Qwen3-30B-A3B
+
+**AI71ai/agrillm-Qwen3-30B-A3B** is an agriculture-focused Foundation Large Language Model (LLM) developed by ai71 with support from leading institutions in the agricultural sector. It is fine-tuned using LoRA on top of [Qwen/Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507).
+
+This model is developed as part of the AgriLLM initiative, a multi-stakeholder collaboration involving the International Affairs Office at the UAE Presidential Court, the [Gates Foundation](https://www.gatesfoundation.org), [CGIAR](https://www.cgiar.org), [Embrapa](https://www.embrapa.br), [ECHO](https://www.echocommunity.org), [FAO](https://www.fao.org), [IFAD](https://www.ifad.org/), the [World Bank](https://www.worldbank.org/), [Digital Green](https://digitalgreen.org/), and other leading organizations in the agriculture domain.
+
+The model is fine-tuned on agriculture-specific Q&A pairs to strengthen its ability to understand agricultural contexts, provide accurate agronomic guidance, and generate reliable, expert-aligned responses, while still preserving the broad capabilities of the underlying Qwen3-A3B-30B model.
+
+It is primarily designed for use within Retrieval-Augmented Generation (RAG) systems - where it can leverage external agricultural knowledge bases for accuracy and contextual grounding - rather than as a standalone model.
+
+
+## 🚜 What is AgriLLM?
+
+AgriLLM is an initiative to provide the global agriculture community with open, foundation AI building blocks that support wider AI adoption and help close the information gap faced by smallholder farmers and agricultural professionals worldwide.
+
+As part of the initiative, four open-source public goods will be released:
+1. A set of fine-tuned LLMs specialized for agriculture
+2. The supervised training dataset of agriculture-focused Q&A pairs used for fine-tuning, enabling anyone to train their own models
+3. An agriculture evaluation benchmark (datasets and metrics) providing a common standard to assess and compare model performance
+4. A corpus of agricultural documents for building Retrieval-Augmented Generation (RAG) pipelines
+
+The initiative’s philosophy is to empower the community with practical, high-quality AI resources - allowing researchers, developers, and institutions to create their own downstream agricultural AI applications built on top of these open building blocks, including the fine-tuned AgriLLMs.
+
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+<details open>
+
+```python
+# Load model directly
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("AI71ai/agrillm-Qwen3-30B-A3B")
+model = AutoModelForCausalLM.from_pretrained("AI71ai/agrillm-Qwen3-30B-A3B")
+messages = [
+    {"role": "user", "content": "How to grow maize in Kenya?"},
+]
+inputs = tokenizer.apply_chat_template(
+	messages,
+	add_generation_prompt=True,
+	tokenize=True,
+	return_dict=True,
+	return_tensors="pt",
+).to(model.device)
+
+outputs = model.generate(**inputs, max_new_tokens=40)
+print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))
+```
+</details>
+</br>
+
+#  Table of Contents
+
+- [Model Card for AI71ai/agrillm-Qwen3-30B-A3B](#model-card-for-ai71aiagrillm-qwen3-30b-a3b)
+- [Model Details](#model-details)  
+  - [Model Description](#model-description)  
+    - [🔧 Base Model](#-base-model)  
+    - [🎯 Objective of Finetuning](#-objective-of-finetuning)  
+    - [🌾 Model Capabilities](#-model-capabilities)  
+- [Uses](#uses)  
+  - [Farmers](#farmers)  
+  - [Field Extension Agents](#field-extension-agents)  
+  - [Academics & Researchers](#academics--researchers)  
+  - [Policymakers & Project Managers](#policymakers--project-managers)  
+  - [⚠️ Out-of-Scope Use](#️-out-of-scope-use)  
+- [Bias, Risks, and Limitations](#bias-risks-and-limitations)
+- [Training Details](#training-details)  
+  - [Training Data](#training-data)  
+  - [Training Procedure](#training-procedure)
+- [Evaluation](#evaluation)  
+  - [Testing Data, Factors & Metrics](#testing-data-factors--metrics)  
+    - [Testing Data](#testing-data)  
+    - [Factors](#factors)  
+    - [Metrics](#metrics)  
+  - [Results](#results)
+- [Environmental Impact](#environmental-impact)
+- [Acknowledgements & Data Sources](#acknowledgements--data-sources)
+- [Citation](#citation)
+- [Model Card Contact](#model-card-contact)
+- [How to Get Started with the Model](#how-to-get-started-with-the-model)
+
+
+# Model Details
+
+## Model Description
+
+### 🔧 Base Model
+
+- Base: `Qwen/Qwen3-30B-A3B-Instruct-2507`
+- Finetuning Method: LoRA (parameter-efficient finetuning)
+- Architecture: 30B transformer (Qwen3 family)
+
+### 🎯 Objective of Finetuning
+- **Strengthen agricultural reasoning**: Improve understanding of agronomic concepts, and the ability to reason about agricultural scenarios
+- **Generate expert-aligned and domain-relevant outputs**: Produce advisory responses that reflect best practices, validated knowledge, and use terminology appropriate to the agricultural context
+- **Ensure reliability and safety: Reduce hallucinations**, maintain factual accuracy, and preserve groundedness in agriculture-related queries
+- **Leverage retrieved context effectively**: Enhance the model’s ability to interpret retrieved information and identify the most relevant agricultural content in RAG applications
+Preserve general capabilities: Maintain the broad reasoning and generative abilities of the Qwen3-A3B-30B base model
+
+
+### 🌾 Model Capabilities
+
+`AI71ai/agrillm-Qwen3-30B-A3B` provides foundational AI capabilities that can be applied across the agricultural domain, including:
+- **Question Answering**: Responds accurately to agricultural queries based on provided or retrieved information
+- **Summarization**: Condenses technical agricultural documents, research papers, and policy briefs into concise summaries
+- **Advisory Generation**: Produces structured guidance or recommendations based on domain knowledge
+- **Reasoning**: Supports scenario analysis, domain-specific reasoning, and decision support
+- **Context Evaluation**: Assesses retrieved content for relevance when generating outputs (optimized for RAG pipelines)
+
+
+## Uses
+
+`AI71ai/agrillm-Qwen3-30B-A3B` is primarily intended as a **foundation model building block**. It is designed to better understand agricultural contexts and perform effectively when connected to internal knowledge bases or used in Retrieval-Augmented Generation (RAG) pipelines. This model is **not a standalone solution** with universal answers; instead, it provides specialized capabilities that downstream applications can leverage to deliver accurate, grounded, and context-aware outputs in agriculture.
+The model can assist multiple personas across the agricultural ecosystem. Example capabilities include:
+### Farmers
+- Answer questions on crop production (sowing, irrigation, harvesting)
+- Provide pest and disease management guidance based on symptoms
+- Recommend fertilizers and nutrient applications
+- Advise on livestock care
+
+### Field Extension Agents
+- Generate advisory responses for farmers
+- Support diagnostic workflows and on-field problem-solving
+- Prepare step-by-step field instructions and protocols
+- Interpret technical manuals, guidelines, and extension materials
+
+### Academics & Researchers
+- Summarize agricultural literature and research papers
+- Explain research methodologies and concepts
+- Analyze and interpret policy briefs and technical reports
+- Support domain-specific reasoning and scenario modeling
+
+### Policymakers & Project Managers
+- Assist in agricultural program assessments and evaluations
+- Support impact analysis and data-driven recommendations
+- Generate evidence-based policy or project briefs
+- Provide reasoning grounded in agricultural principles and best practices
+…
+
+## ⚠️ Out-of-Scope Use
+
+This model is not a **universal source of answers** and is **not intended as a standalone solution**. It is designed to support agricultural workflows but **cannot replace expert knowledge**. All outputs should be verified, especially in high-stakes contexts.
+
+**Moreover**, the model is **not intended for**:
+- Medical or veterinary diagnosis
+- Producing legally binding recommendations or official documents
+- High-risk decision-making without expert supervision
+- Replacing certified agronomists, extension agents, or researchers
+- Providing real-time field measurements or monitoring (e.g., soil moisture, weather, or crop sensor data)
+- Making financial, legal, or regulatory decisions in agricultural projects
+
+
+# Bias, Risks, and Limitations
+
+Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
+
+# Training Details
+
+## Training Data
+
+The model was fine-tuned on 200k high-quality examples (146k was domain specific), combining:
+
+1. **Human expert-generated Q&A pairs** - Written or approved by agricultural domain specialists (e.g., agronomists, researchers, extension agents, etc.)
+2. **Q&A pairs extracted from real-world interactions** - Extracted from forums, email threads, SMS-based extension services, and other practical agricultural communications.
+3. **Synthetic Q&A pairs**: Generated and curated through controlled extraction from agricultural documents using LLMs with carefully designed prompts to prevent any hallucination
+4. **Domain-specific tasks**:
+    - Summarization of agronomy texts
+    - Reading comprehension of agricultural guidelines
+    - Soil, crop, and livestock reasoning tasks
+    - Policy, research, and project-management reasoning
+5. **General ability tasks**:
+    - Included to prevent catastrophic forgetting
+    - Maintains strong general reasoning, math, and language skills
+
+No private or personal data is included. All partner datasets were anonymized and ethically prepared.
+
+
+## Training Procedure
+
+This model was created by performing LoRA (parameter-efficient) finetuning on top of the instruct-foundation model: [Qwen/Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507)
+
+```
+Hardware: 8xH100 GPU
+Training Time: ~38 hours
+Model Checkpoint: 400 steps 
+Batch Size: 8
+gradient_accumulation_steps: 4
+lora_rank: 128
+lora_alpha: 256
+target_modules: 
+    - q_proj
+    - k_proj
+    - v_proj
+    - o_proj
+    - gate_proj
+    - up_proj
+    - down_proj
+    - lm_head
+    - gate
+    - shared_expert.gate_proj
+    - shared_expert.up_proj
+    - shared_expert.down_proj
+    - embed_tokens
+```
+ 
+# Evaluation
+
+The first evaluation was performed covering:
+- Basic generation quality
+- Domain consistency
+- Manual SME review of sample outputs
+- A full benchmark suite (field Q&A accuracy, agronomy reasoning, livestock QA, safety evaluation) will be added in a future update.
+
+| Model                            | Answer Correctness | Factual Correctness (Recall) | Factual Correctness (Precision) |
+| -------------------------------- | ------------------ | ---------------------------- | ------------------------------- |
+| GPT-4o                           | 0.4046921554       | 0.4960951189                 | 0.2404005006                    |
+| GPT-4o with RAG                  | 0.4888914377       | 0.630975                     | 0.3546307885                    |
+| Llama-3.3-70B                    | 0.3832698282       | 0.4624625                    | 0.2320625                       |
+| Llama-3.3-70B Finetuned          | 0.4739898234       | 0.3576146789                 | 0.3594901961                    |
+| Llama-3.3-70B Finetuned with RAG | 0.5569369224       | 0.4872040302                 | 0.5322138365                    |
+| Qwen3-30B-A3B                    | 0.3773581101       | 0.4801293661                 | 0.1803589744                    |
+| Qwen3-30B-A3B Finetuned          | 0.4877470022       | 0.3549163449                 | 0.3906931964                    |
+| Qwen3-30B-A3B Finetuned with RAG | 0.5645609396       | 0.4957125                    | 0.5091625                       |
+| Falcon3-10B Finetuned            | 0.4516784406       | 0.3506540881                 | 0.3499623588                    |
+| Falcon3-10B Finetuned with RAG   | 0.5453206181       | 0.459025                     | 0.5092                          |
+
+## Testing Data, Factors & Metrics
+
+### Testing Data
+
+We evaluated the model on the dataset available at [agrillm-qa-eval-800](https://huggingface.co/datasets/AI71ai/agrillm-qa-eval-800), which contains 800 Q&A pairs, covering multiple agricultural topics, crops, geographies and tasks.
+
+### Factors
+The creation of the evaluation set considered multiple dimensions:
+
+- **User personas**: 4 primary personas spanning farmers, extension agents, researchers, and policymakers
+- **Topics and domains**: Multiple crops, produce types, and subdomains of agricultural knowledge
+- **Answer types**: Single-turn and multi-turn Q&A, as well as domain ‘tasks’ (summarization, classification, etc.)
+
+
+### Metrics
+
+The evaluation pipeline leverages [RAGAS](https://docs.ragas.io/en/stable/) with LLM-as-a-judge using GPT-4o, where the model’s responses are automatically assessed against reference answers for correctness, completeness, and language quality.
+
+#### Example (Agriculture – Rice Irrigation):
+
+Ground-truth facts:
+- Rice needs standing water during early growth.
+- Drip irrigation is rarely used for rice.
+- Water requirement is highest during tillering stage.
+
+LLM Response:
+> - Rice generally grows best with standing water.
+> - Restrictive licenses are good
+
+when we grade the response we find
+
+> ✅ Rice generally grows best with standing water.
+> ❌ Restrictive licenses are good
+
+`factual_correctness_precision`: Out of 2 stated facts, 1 is correct (standing water) → Precision = 1/2.
+
+`factual_correctness_recall`: Out of 3 ground-truth facts, the model mentioned only 1 → Recall = 1/3.
+
+## Results 
+
+Check our leaderboard for more information: https://huggingface.co/spaces/AI71ai/agrillm-leaderboard
+
+# Environmental Impact 
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+### Training Hardware
+- 8× NVIDIA H100 GPUs
+- 150-core CPU
+- 500 GB storage
+- Single-node configuration in a UAE data center
+- 38 hours training time
+
+### Estimated Energy Consumption
+- Estimated IT power (GPUs + CPU + system): approx. 7.1 kW
+- Data-center PUE (Power Usage Effectiveness): 1.4
+- Estimated total facility power:
+  7.1 kW × 1.4 = **9.94 kW**
+- Total energy consumed:
+  9.94 kW × 38 hours = **377.7 kWh**
+
+### Estimated Carbon Emissions
+- UAE grid emission factor: 0.40 kg CO₂e per kWh
+- Total emissions:
+  377.7 kWh × 0.40 = **151.1 kg CO₂e**
+
+### Summary
+- **Total training energy:** approx. 377.7 kWh  
+- **Total training emissions:** approx. 151.1 kg CO₂e
+
+### Assumptions and Methodology
+- GPU power based on NVIDIA H100 SXM TDP of ~700 W per GPU.
+- CPU + system power estimated at ~1.5 kW under load.
+- Full GPU utilization assumed during training.
+- Data-center overhead modeled using PUE = 1.4.
+- UAE grid intensity assumed at 0.40 kg CO₂e/kWh.
+- Only operational electricity use is included; hardware manufacturing and networking emissions are excluded.
+- Calculations use:
+  - Energy = Power × Time
+  - Emissions = Energy × Emission Factor
+
+# Acknowledgements & Data Sources
+We gratefully acknowledge the contributions of our partners and collaborators who made this work possible:
+
+- The International Affairs Office of the UAE Presidential Court
+- [Gates Foundation](https://www.gatesfoundation.org/)
+- [CGIAR](https://www.cgiar.org/)– Consultative Group on International Agricultural Research
+- [Embrapa](https://www.embrapa.br/) – Empresa Brasileira de Pesquisa Agropecuária
+- [ECHO](https://echonet.org/)
+- [FAO](https://www.fao.org/) – Food and Agriculture Organization of the United Nations
+- [IFAD](https://www.ifad.org) – International Fund for Agricultural Development
+- [The World Bank](https://www.worldbank.org)
+- [Digital Green](https://digitalgreen.org)
+- [KIADPAI](https://kiaai.ae/) – Khalifa International Award for Date Palm and Agricultural Innovation
+- [KALRO](https://kalro.org/) – Kenya Agricultural and Livestock Research Organization
+- [Extension Foundation](https://extension.org/)
+
+Special thanks to all partners for their invaluable support, including:
+
+**Data preparation**: Curating agricultural documents and Q&A pairs, with manual verification by domain experts
+**Expert guidance**: Supporting the verification of synthetic Q&A pairs generated for model fine-tuning
+**AI assistant design**: Providing expertise on designing downstream AI applications to test the models
+**Model testing**: Manually evaluating model outputs to ensure quality and relevance
+**Field engagement**: Collaborating with end-users in agricultural settings to support adoption, and collect current needs and feedback
+
+
+All datasets used were anonymized and ethically prepared, and no private or personal data was included
+
+# Citation
+
+If you find this model useful, please cite us:
+
+```
+@misc{agrillm-Qwen3-30B-A3B,
+      title={agrillm-Qwen3-30B-A3B}, 
+      author={Mamoun Alaoui, Ojas Agarwal, Zafar Shadman, Derek Thomas},
+      year={2025},
+}
+```
+
+# Model Card Contact
+- Mamoun Alaoui [mamoun.alaoui@ai71.ai](mailto:mamoun.alaoui@ai71.ai)
+- Ojas Agarwal [ojas.agarwal@ai71.ai](mailto:ojas.agarwal@ai71.ai)
+- Zafar Shadman [zafar.shadman@ai71.ai](mailto:zafar.shadman@ai71.ai)
+- Derek Thomas [derek.thomas@ai71.ai](mailto:derek.thomas@ai71.ai)
--- a/added_tokens.json
+++ b/added_tokens.json
@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}
--- a/chat_template.jinja
+++ b/chat_template.jinja
@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
--- a/config.json
+++ b/config.json
@@ -0,0 +1,38 @@
+{
+  "architectures": [
+    "Qwen3MoeForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "decoder_sparse_step": 1,
+  "dtype": "float16",
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "max_position_embeddings": 262144,
+  "max_window_layers": 48,
+  "mlp_only_layers": [],
+  "model_type": "qwen3_moe",
+  "moe_intermediate_size": 768,
+  "norm_topk_prob": true,
+  "num_attention_heads": 32,
+  "num_experts": 128,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 48,
+  "num_key_value_heads": 4,
+  "output_router_logits": false,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 10000000,
+  "router_aux_loss_coef": 0.001,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.1",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "4.57.1"
+}
--- a/merges.txt
+++ b/merges.txt
--- a/model-00001-of-00013.safetensors
+++ b/model-00001-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e19ea030f0cec9e4bea59c06bbf55224d49e1ab1b185f4835698629aaa73945
+size 4997183592
--- a/model-00002-of-00013.safetensors
+++ b/model-00002-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbe0fd61d308ca17306c61c4d6d02e563e509e3a20cc85642d618637f7b351ca
+size 4997740032
--- a/model-00003-of-00013.safetensors
+++ b/model-00003-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d38613165b43a8abbcde4e9cdb3b7c13c326c000a6e38903373055a3f77fc517
+size 4997740632
--- a/model-00004-of-00013.safetensors
+++ b/model-00004-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b37c09c7ef55c33797802ddeb4bb7097213657c828780bf4f8b5fc3b92d11b3
+size 4997741608
--- a/model-00005-of-00013.safetensors
+++ b/model-00005-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a28fe68813df582e49132be8bad391ac8e15cb0aa6ac884e39eda0ee5cb783d
+size 4997741608
--- a/model-00006-of-00013.safetensors
+++ b/model-00006-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f156acd5ea9e064047342552398ef1cf5756774f4c7fecc6a22d6ee2b2f4012
+size 4997741608
--- a/model-00007-of-00013.safetensors
+++ b/model-00007-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8abbef394c0d84382b88d831c12fb890e55de881f4ea30bb63770b290fbf874
+size 4997741608
--- a/model-00008-of-00013.safetensors
+++ b/model-00008-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f9d3a80fb366144f0743024acd6e2452873f23faffc919d54f948e484c90d99
+size 4997741608
--- a/model-00009-of-00013.safetensors
+++ b/model-00009-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbaab486a9690aac95296de239c2595429f73441489d2379a683a78937e6a766
+size 4997741608
--- a/model-00010-of-00013.safetensors
+++ b/model-00010-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48ba9c5d9ac18b06de958ac01dee43fe6d6222ca5dbcc44ac93e6ebc6b1d15a3
+size 4997741608
--- a/model-00011-of-00013.safetensors
+++ b/model-00011-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c03b5b407fafad380d037e2c640491b8892c6fce30324cdb06a95c769adb5d4
+size 4997741608
--- a/model-00012-of-00013.safetensors
+++ b/model-00012-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37066c2de538c799046567602d5f1d1ad00d78ee0dc7e45473847555bed3b247
+size 4997741608
--- a/model-00013-of-00013.safetensors
+++ b/model-00013-of-00013.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44221dcdf600059d41387ea2b4d404814426866709d636b4202a6c18fe0ddae0
+size 1094220136
--- a/model.safetensors.index.json
+++ b/model.safetensors.index.json
--- a/special_tokens_map.json
+++ b/special_tokens_map.json
@@ -0,0 +1,25 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}
--- a/tokenizer.json
+++ b/tokenizer.json
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
--- a/vocab.json
+++ b/vocab.json