ModelHub XC da1eaad014 初始化项目,由ModelHub XC社区提供模型
Model: jgalego/amalia-desenrasca-9b
Source: Original Platform
2026-11-14 15:00:50 +08:00

base_model, datasets, language, library_name, license, model_name, pipeline_tag, tags
base_model datasets language library_name license model_name pipeline_tag tags
amalia-llm/AMALIA-9B-0626-SFT
jgalego/function-calling-pt-pt
pt transformers apache-2.0 amalia-desenrasca-9b text-generation
tool-calling
function-calling
agent
pt-PT
unsloth

Desenrasca logo: an azulejo tile whose pattern is made of wrenches

amalia-desenrasca-9b

amalia-llm/AMALIA-9B-0626-SFT fine-tuned to call tools, with prompts in European Portuguese (pt-PT). Desenrascanço is the Portuguese art of improvising your way out of a mess.

🚀 Usage

The tools go in the system message. The model answers with <tool_call>{...}</tool_call>, or in plain text when no tool fits. Send the tool result back with the tool role.

import json

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "jgalego/amalia-desenrasca-9b"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")

tools = [
    {
        "name": "get_weather",
        "description": "Estado do tempo atual numa cidade",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string", "description": "Nome da cidade"}},
            "required": ["city"],
        },
    }
]
system = "Tens acesso \u00e0s ferramentas abaixo. Para usar uma, responde apenas com \u003ctool_call\u003e{\"name\": \"nome\", \"arguments\": {\"argumento\": \"valor\"}}\u003c/tool_call\u003e. Se nenhuma ferramenta servir, diz com educa\u00e7\u00e3o que n\u00e3o podes ajudar."
system += "\n<tools>\n" + "\n".join(json.dumps(tool, ensure_ascii=False) for tool in tools) + "\n</tools>"
messages = [
    {"role": "system", "content": system},
    {"role": "user", "content": "Está a chover no Porto?"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True))
# Expected: <tool_call>{"name": "get_weather", "arguments": {"city": "Porto"}}</tool_call>

🏋️ Training

  • Why: AMALIA's chat template has no tool support, and there is no public pt-PT tool-calling data. The tools go in the system message as plain text, and the template renders the tool role as is.
  • Data: 2,700 conversations from jgalego/function-calling-pt-pt: 1,000 xLAM, 700 ToolACE, 400 Hermes and 600 When2Call refusals. Another 300 generated conversations use the demo's tools for parallel calls, relative dates, percentages, currencies and unsupported forecasts. User turns and final answers are pt-PT; tool schemas, calls and results stay in English.
  • Pairs: Each assistant turn is one prompt/completion pair. Calls may be parallel. Only When2Call refusals are used because its follow-up questions often ask for information the user already supplied.
  • Method: LoRA (r=16, alpha=32, all attention and MLP projections) on the bf16 model with Unsloth. 2 epochs, learning rate 2e-4 with a cosine schedule, effective batch size 16. The adapter is merged into the base model.

📊 Evaluation

BFCL pt-PT uses the BFCL v3 single-turn questions translated into pt-PT, with the original functions and answer keys. Scores are shares of questions answered correctly.

Metric AMALIA v1 Desenrasca
BFCL pt-PT 0.589 0.554 0.775
BFCL English 0.592 0.526 0.762
pt-PT answerable 0.636 0.544 0.743
pt-PT irrelevance 0.484 0.571 0.844
pt-PT parallel 0.528 0.090 0.802

The second evaluation uses 200 held-out Glaive-derived conversations with pt-PT user turns: 96 structured call references and 104 without a structured call.

Metric AMALIA v1 Desenrasca
Valid call 0.990 0.938 1.000
Right function name 0.990 0.938 1.000
Exact arguments 0.635 0.875 0.844
No call when the reference made none 0.423 0.808 0.654

The last row is reference agreement, not a measure of whether no tool fits. A manual audit of its 104 rows found:

Reference behavior Rows Desenrasca made no call
Required details missing 41 29
No available tool fits 36 36
Generic conversation 2 2
A tool fits despite the no-call reference 23 1
City names supplied to coordinate-only tools 2 0

On the 41 missing-detail rows, 23 outputs explicitly request more information. Twelve call a tool instead; eleven use arbitrary defaults or placeholders and one infers Apple's AAPL ticker.

⚠️ Limitations

  • Missing arguments: The model sometimes invents required values instead of asking the user. This happened on 11 of 41 audited missing-detail requests.
  • Coordinates: It inferred city-center coordinates in two distance questions whose tools accepted coordinates only.
  • Translated data: The pt-PT data is machine-translated by AMALIA itself, and nobody has reviewed it.
  • English arguments: Values like city names follow the English source data, so expect "New York" rather than "Nova Iorque".
S
Description
Model synced from source: jgalego/amalia-desenrasca-9b
Readme
33 KiB
0 Stars 1 Watchers 0 Forks
Languages
Jinja 100%