---
library_name: transformers
base_model: Qwen/Qwen2.5-1.5B-Instruct
pipeline_tag: text-generation
tags:
- qwen
- qwen2.5
- tool-calling
- function-calling
- dpo
- direct-preference-optimization
- preference-tuning
- when2call
- transformers
- pytorch
license: apache-2.0
datasets:
- nvidia/When2Call
---

# Qwen2.5-1.5B Tool Calling DPO

A DPO fine-tuned version of [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct), trained on NVIDIA's [When2Call](https://huggingface.co/datasets/nvidia/When2Call) preference dataset to improve tool-calling decision making.

The model is trained to better distinguish between requests that **require a tool call**, requests that can be answered directly, and requests that cannot be answered using the available tools.

> This model is based on Qwen2.5-1.5B-Instruct, which is licensed under Apache 2.0. The model was fine-tuned using NVIDIA's When2Call dataset, which is licensed under CC BY 4.0. The dataset license and attribution requirements apply to the use of the When2Call dataset.


## Model Details

| Property         | Value                       |
| ---------------- | --------------------------- |
| Base model       | Qwen/Qwen2.5-1.5B-Instruct  |
| Parameters       | 1.5B                        |
| Fine-tuning      | LoRA + DPO                  |
| Dataset          | NVIDIA When2Call            |
| Training split   | `train_pref`                |
| Final checkpoint | Step 1000                   |
| Model format     | Merged                      |
| Framework        | PyTorch, Transformers, PEFT |

The model was initially fine-tuned using LoRA and Direct Preference Optimization (DPO). The LoRA adapter was subsequently merged into the base model using `merge_and_unload()` to produce a standalone model.

## Dataset


Training was performed using the **`train_pref` split of NVIDIA's When2Call dataset**.

When2Call is designed specifically to evaluate and train LLMs on decisions about **when (and when not) to call tools**. The dataset includes preference pairs consisting of a chosen and rejected response for a given user request and tool specification.

The `train_pref` split contains **9,000 preference-training examples** with:

* Tool specifications
* User messages
* Chosen responses
* Rejected responses

NVIDIA provides both an SFT dataset (`train_sft`) and a preference dataset (`train_pref`); this model uses the **preference dataset for DPO training**.

The When2Call dataset is synthetic and is licensed under **CC BY 4.0**.

## Training Objective

The objective was to improve the model's **tool-use decision boundary**.

The model learns to distinguish between:

1. **Tool Call** — a tool should be invoked to answer the request.
2. **Request for Information** — the request can be handled without invoking a tool.
3. **Cannot Answer** — the available tools cannot answer the request.

The preference-training setup encourages the model to prefer appropriate responses over incorrect tool-calling behavior.

## Evaluation

The fine-tuned model was evaluated against the original Qwen2.5-1.5B-Instruct model on **300 samples**.

The evaluation uses the same 300-sample LLM-as-a-judge subset provided by When2Call. NVIDIA's dataset contains a larger 3,652-sample MCQ test set and a 300-sample LLM-as-a-judge subset.

### Results

| Metric                   | Qwen2.5-1.5B-Instruct | Qwen2.5-1.5B + DPO |
| ------------------------ | --------------------: | -----------------: |
| Intent Accuracy          |                 52.7% |          **74.0%** |
| Tool Precision           |                 40.8% |          **72.9%** |
| Tool Recall              |             **93.0%** |              35.0% |
| Tool F1                  |             **56.7%** |              47.3% |
| Argument F1              |             **71.9%** |              67.7% |
| Unsupported Tool Calls ↓ |                 45.0% |           **4.3%** |
| Missed Tool Calls ↓      |              **2.3%** |              21.7% |
| Throughput               |            38.3 tok/s |         26.4 tok/s |

### Key Results

The largest improvement was in **unsupported tool calls**:

**45.0% → 4.3%**

This indicates that DPO substantially reduced inappropriate or hallucinated tool invocations.

Intent accuracy also increased:

**52.7% → 74.0%**

and tool precision increased:

**40.8% → 72.9%**

However, this came with a substantial reduction in tool recall:

**93.0% → 35.0%**

and Tool F1:

**56.7% → 47.3%**

Therefore, the main behavioral change is a shift toward a **more conservative, precision-oriented tool-calling policy**.

## Confusion Matrix

### Qwen2.5-1.5B-Instruct

| Ground Truth \ Prediction | Tool | Request | Refusal |
| ------------------------- | ---: | ------: | ------: |
| Tool Call                 |   93 |       3 |       4 |
| Request For Information   |   76 |      23 |       1 |
| Cannot Answer             |   59 |      22 |      19 |

### Qwen2.5-1.5B + DPO

| Ground Truth \ Prediction | Tool | Request | Refusal |
| ------------------------- | ---: | ------: | ------: |
| Tool Call                 |   35 |      65 |       0 |
| Request For Information   |   12 |      86 |       2 |
| Cannot Answer             |    1 |      91 |       8 |

The confusion matrix shows that DPO significantly reduced the model's tendency to issue tool calls for requests that should not result in a tool invocation.

## Tool Calling Example

### Tool Definition

```python
tools = [{
    "name": "get_stock_price",
    "description": "Fetch real-time stock price for a given ticker symbol.",
    "parameters": {
        "type": "object",
        "properties": {
            "ticker": {
                "type": "string",
                "description": "The ticker symbol (e.g., AAPL, NVDA)"
            }
        },
        "required": ["ticker"]
    }
}]
```

### User

```text
Can you check Nvidia's current stock price?
```

### Model Output

```text
<tool_call>
{"name": "get_stock_price", "arguments": {"ticker": "NVDA"}}
</tool_call>
```

## Inference

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "YOUR_USERNAME/YOUR_MODEL_NAME"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

model.eval()

tools = [{
    "name": "get_stock_price",
    "description": "Fetch real-time stock price for a given ticker symbol.",
    "parameters": {
        "type": "object",
        "properties": {
            "ticker": {
                "type": "string",
                "description": "The ticker symbol (e.g., AAPL, NVDA)"
            }
        },
        "required": ["ticker"]
    }
}]

messages = [
    {
        "role": "user",
        "content": "Can you check Nvidia's current stock price?"
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tools=tools,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id
    )

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[1]:],
    skip_special_tokens=False
)

print(response)
```

## Output Post-Processing

During evaluation, model outputs were passed through a lightweight post-processing step to normalize tool-call formatting.

The post-processing:

1. Extracts JSON containing `name` and `arguments`.
2. Removes duplicate or nested `<tool_call>` wrappers.
3. Normalizes the output to:

```text
<tool_call>
{"name": "...", "arguments": {...}}
</tool_call>
```

If no tool-call JSON is detected, the output is treated as a normal conversational response.

The post-processing step is used for **format normalization and evaluation** and does not generate a tool call that the model did not produce.

## Limitations

The main limitation is the precision-recall trade-off introduced by DPO.

The model is considerably better at avoiding unsupported tool calls, but it also misses a larger proportion of valid tool-call opportunities.

Therefore, this model should not be interpreted as universally better than the base model for tool calling. Instead, it demonstrates that preference optimization can strongly shift the **tool-use decision policy** of a small instruction-tuned model.

The evaluation also uses a relatively small 300-sample subset, so additional evaluation on larger and more diverse tool-calling benchmarks would be useful.

## Intended Use

This model is intended for research and experimentation involving:

* Tool calling
* Function calling
* Tool-selection policies
* Preference optimization
* DPO
* Agentic LLM systems
* Small language model alignment

It is **not intended to be considered production-ready** without additional task-specific evaluation.

## Future Work

* Recover tool-call recall while maintaining low unsupported-call rates
* Experiment with DPO hyperparameters
* Improve preference-data construction
* Compare DPO against SFT
* Evaluate larger Qwen models
* Evaluate multi-tool selection
* Evaluate multi-step tool calling
* Improve argument-generation accuracy
* Benchmark inference using vLLM
* Evaluate on larger tool-calling benchmarks

## Base Model

This model is based on:

**Qwen/Qwen2.5-1.5B-Instruct**

Please refer to the base model for its original capabilities, license, and usage restrictions.

## Dataset Citation

This work uses NVIDIA's **When2Call** dataset.

> Ross, Hayley, Ameya Sunil Mahabaleshwarka, and Yoshi Suhara. "When2Call: When (not) to Call Tools." NAACL 2025.

```bibtex
@inproceedings{ross-etal-2025-when2call,
    title = "{W}hen2{C}all: When (not) to Call Tools",
    author = "Ross, Hayley  and
      Mahabaleshwarkar, Ameya Sunil  and
      Suhara, Yoshi",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.naacl-long.174/",
    doi = "10.18653/v1/2025.naacl-long.174",
    pages = "3391--3409",
    ISBN = "979-8-89176-189-6",
    abstract = "Leveraging external tools is a key feature for modern Language Models (LMs) to expand their capabilities and integrate them into existing systems. However, existing benchmarks primarily focus on the accuracy of tool calling{---}whether the correct tool is called with the correct parameters{---}and less on evaluating when LMs should (not) call tools. We develop a new benchmark, When2Call, which evaluates tool-calling decision-making: when to generate a tool call, when to ask follow-up questions and when to admit the question can{'}t be answered with the tools provided. We find that state-of-the-art tool-calling LMs show significant room for improvement on When2Call, indicating the importance of this benchmark. We also develop a training set for When2Call and leverage the multiple-choice nature of the benchmark to develop a preference optimization training regime, which shows considerably more improvement than traditional fine-tuning. We release the benchmark and training data as well as evaluation scripts."
}
```

## Acknowledgements

Thanks to the NVIDIA When2Call authors and the Qwen team for releasing the dataset and base model used in this experiment.