Tiny-Agent-a-0.5B/README.md

---
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen2.5-Coder-0.5B-Instruct
pipeline_tag: text-generation
library_name: transformers
tags:
- code
- chat
- qwen
- qwen-coder
- agent
---


# Tiny-Agent-α

## Introduction

***Tiny-Agent-α*** is an extension of Dria-Agent-a, trained on top of the [Qwen2.5-Coder](https://huggingface.co/collections/Qwen/qwen25-coder-66eaa22e6f99801bf65b0c2f) series to be used in edge devices. These models are carefully fine tuned with quantization aware training to minimize performance degradation after quantization. 

Tiny-Agent-α employs ***Pythonic function calling***, which is LLMs using blocks of Python code to interact with provided tools and output actions. This method was inspired by many previous work, including but not limited to [DynaSaur](https://arxiv.org/pdf/2411.01747), [RLEF](https://arxiv.org/pdf/2410.02089), [ADAS](https://arxiv.org/pdf/2408.08435) and [CAMEL](https://arxiv.org/pdf/2303.17760). This way of function calling has a few advantages over traditional JSON-based function calling methods:

1. **One-shot Parallel Multiple Function Calls:** The model can can utilise many synchronous processes in one chat turn to arrive to a solution, which would require other function calling models multiple turns of conversation.
2. **Free-form Reasoning and Actions:** The model provides reasoning traces freely in natural language and the actions in between \`\`\`python \`\`\` blocks, as it already tends to do without special prompting or tuning. This tries to mitigate the possible performance loss caused by imposing specific formats on LLM outputs discussed in [Let Me Speak Freely?](https://arxiv.org/pdf/2408.02442)
3. **On-the-fly Complex Solution Generation:** The solution provided by the model is essentially a Python program with the exclusion of some "risky" builtins like `exec`, `eval` and `compile` (see full list in **Quickstart** below). This enables the model to implement custom complex logic with conditionals and synchronous pipelines (using the output of one function in the next function's arguments) which would not be possible with the current JSON-based function calling methods (as far as we know).

## Quickstart

You can use tiny-agents easily with the dria_agent package:

````bash
pip install dria_agent
````

This package handles models, tools, code execution, and backend (supports mlx, ollama, and transformers).

### Usage 

Decorate functions with @tool to expose them to the agent.

````python
from dria_agent import tool

@tool
def check_availability(day: str, start_time: str, end_time: str) -> bool:
    """
    Checks if a given time slot is available.

    :param day: The date in "YYYY-MM-DD" format.
    :param start_time: The start time of the desired slot (HH:MM format, 24-hour).
    :param end_time: The end time of the desired slot (HH:MM format, 24-hour).
    :return: True if the slot is available, otherwise False.
    """
    # Mock implementation
    if start_time == "12:00" and end_time == "13:00":
        return False
    return True
````

Create an agent with custom tools:

```python
from dria_agent import ToolCallingAgent

agent = ToolCallingAgent(
    tools=[check_availability]
)
```

Use agent.run(query) to execute tasks with tools.

```python
execution = agent.run("Check my calendar for tomorrow noon", print_results=True)
```

Where output is
````
let me help you check your availability for a 1-hour meditation session       
starting at noon tomorrow.                                                    
                                                                                
Step-by-step reasoning:                                                       
 1. We need to check availability for a specific time slot (noon)              
 2. The duration is 1 hour, so we'll use the same start and end times          
 3. Since it's tomorrow, we should format the date as "YYYY-MM-DD"             
 4. Use the check_availability() function with these parameters                
                                                                                
Here's the code to check your availability:                                   
                                                                                
```python                                                                     
tomorrow = (datetime.now() + timedelta(days=1)).strftime("%Y-%m-%d")          
start_time = "12:00"  # Noon in 24-hour format                                
end_time = "13:00"   # One hour after noon                                    
                                                                                
availability = check_availability(tomorrow, start_time, end_time)             
```                                                                           
                                                                                
The code will:                                                                
- Calculate tomorrow's date using datetime and timedelta                      
- Set the time slot to noon (12:00) for 1 hour duration                       
- Check if this time slot is available using the check_availability function  
                                                                                
The availability variable will contain True if you're available, or False if  
not. 
````

If using dria_agent, system prompts and tooling work out of the box—no extra setup needed. 

You can use Tiny-Agent-1.5B directly with transformers:

````python
import json
from typing import Any, Dict, List
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model_name = "driaforall/Tiny-Agent-a-3B"
model = AutoModelForCausalLM.from_pretrained(
    model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# System prompt for optimal performance
SYSTEM_PROMPT = """
You are an expert AI assistant that specializes in providing Python code to solve the task/problem at hand provided by the user.

You can use Python code freely, including the following available functions:

<|functions_schema|>
{{functions_schema}}
<|end_functions_schema|>

The following dangerous builtins are restricted for security:
- exec
- eval
- execfile
- compile
- importlib
- input
- exit

Think step by step and provide your reasoning, outside of function calls.
You can write Python code and use the available functions. Provide all your Python code in a SINGLE markdown code block.

DO NOT use print() statements AT ALL. Avoid mutating variables whenever possible.
""".strip()

# Example function schema (define the functions available to the model)
FUNCTIONS_SCHEMA = """
def check_availability(day: str, start_time: str, end_time: str) -> bool:
    """
    Checks if a given time slot is available.

    :param day: The date in "YYYY-MM-DD" format.
    :param start_time: The start time of the desired slot (HH:MM format, 24-hour).
    :param end_time: The end time of the desired slot (HH:MM format, 24-hour).
    :return: True if the slot is available, otherwise False.
    """
    pass
"""

# Format system prompt
system_prompt = SYSTEM_PROMPT.replace("{{functions_schema}}", FUNCTIONS_SCHEMA)

# Example user query
USER_QUERY = "Check if I'm available for an hour long meditation at tomorrow noon."

# Format messages for the model
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": USER_QUERY},
]

# Prepare input for the model
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# Generate response
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)

# Extract new generated tokens
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

# Decode response
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)
````

## Evaluation & Performance

We evaluate the model on the **Dria-Pythonic-Agent-Benchmark ([DPAB](https://github.com/firstbatchxyz/function-calling-eval)):** The benchmark we curated with a synthetic data generation +model-based validation + filtering and manual selection to evaluate LLMs on their Pythonic function calling ability, spanning multiple scenarios and tasks. See [blog](https://huggingface.co/blog/andthattoo/dpab-a) for more information.

Below are the DPAB results: 

Current benchmark results for various models **(strict)**:

| Model Name                      | Pythonic | JSON |
|---------------------------------|----------|------|
| **Closed Models**               |          |      |
| Claude 3.5 Sonnet              | 87       | 45   |
| gpt-4o-2024-11-20              | 60       | 30   |
| **Open Models**                 |          |      |
| **> 100B Parameters**           |          |      |
| DeepSeek V3 (685B)             | 63       | 33   |
| MiniMax-01                     | 62       | 40   |
| Llama-3.1-405B-Instruct        | 60       | 38   |
| **> 30B Parameters**            |          |      |
| Qwen-2.5-Coder-32b-Instruct    | 68       | 32   |
| Qwen-2.5-72b-instruct          | 65       | 39   |
| Llama-3.3-70b-Instruct         | 59       | 40   |
| QwQ-32b-Preview                | 47       | 21   |
| **< 20B Parameters**           |          |      |
| Phi-4 (14B)                    | 55       | 35   |
| Qwen2.5-Coder-7B-Instruct      | 44       | 39   |
| Qwen-2.5-7B-Instruct           | 47       | 34   |
| **Tiny-Agent-a-3B**               | **72**       | 34   |
| Qwen2.5-Coder-3B-Instruct      | 26       | 37   |
| **Tiny-Agent-a-1.5B**               | **73**       | 30   |

#### Citation

```
@misc{Dria-Agent-a,
      url={https://huggingface.co/blog/andthattoo/dria-agent-a},
      title={Dria-Agent-a},
      author={"andthattoo", "Atakan Tekparmak"}
}
```