初始化项目,由ModelHub XC社区提供模型

Model: praiselab-picuslab/Llama-3.2-1B-Instruct-Medicina-Generale
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-08 14:06:18 +08:00
commit a9e2b43800
9 changed files with 2412 additions and 0 deletions

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

136
README.md Normal file
View File

@@ -0,0 +1,136 @@
---
license: cc-by-nc-4.0
language:
- it
tags:
- llama
- llama-3
- meta
- medical-qa
- italian
- biomedical
- question-answering
- fine-tuning
- unsloth
- bnb
- 4bit
- imb
- Medicina-Generale
datasets:
- praiselab-picuslab/IMB
base_model:
- unsloth/Llama-3.2-1B-Instruct-unsloth-bnb-4bit
---
# 🧠 Llama-3.2-1B-Instruct — IMB Medicina Generale Fine-Tuned Model
This model is a fine-tuned version of [`unsloth/Llama-3.2-1B-Instruct-unsloth-bnb-4bit`](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-unsloth-bnb-4bit), optimized for **Italian medical question answering**, with a specific focus on **Medicina Generale**.
The fine-tuning was performed using a **subset of the IMB (Italian Medical Benchmark) dataset**, specifically:
- Medicina Generale category only
- ~10,000 training samples
The training was performed using the **Unsloth** library with LoRA fine-tuning, and the adapter weights were later merged into the base model to provide a standalone checkpoint.
This model relies on data from the IMB dataset. **If you use this model in research or applications, you must cite the IMB paper (see Citation section below).**
---
## 📚 Training Dataset — IMB (Italian Medical Benchmark)
IMB is an Italian benchmark for medical question answering, designed to evaluate and improve LLM performance in clinical-domain Italian language understanding and reasoning.
The full dataset includes:
- **IMB-QA**: 782,644 doctor-patient conversations collected from Italian online medical forums
- **IMB-MCQA**: 25,862 multiple-choice questions derived from Italian medical specialization exams
⚠️ **Important:**
This model was trained **only on the Medicina Generale subset (~10,000 samples)** of IMB, not on the full dataset.
Dataset repository:
👉 https://github.com/PRAISELab-PicusLab/IMB
---
## 🧪 Usage Example
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("praiselab-picuslab/Llama-3.2-1B-Instruct-Medicina Generale")
tokenizer = AutoTokenizer.from_pretrained("praiselab-picuslab/Llama-3.2-1B-Instruct-Medicina Generale")
prompt = "[Example question in Italian about Medicina Generale]"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
---
## ⚠️ Usage Restrictions
* Allowed use: **Non-commercial research only**
* Redistribution: Not allowed without explicit authorization
* Mandatory citation: The IMB dataset paper must be cited in any publication or derived work
---
## 📄 Citation
If you use this model, the IMB dataset, or derived outputs in research, please cite:
```bibtex
@inproceedings{DBLP:conf/clic-it/RomanoRBPM25,
author = {Antonio Romano and
Giuseppe Riccio and
Mariano Barone and
Marco Postiglione and
Vincenzo Moscato},
editor = {Cristina Bosco and
Elisabetta Jezek and
Marco Polignano and
Manuela Sanguinetti},
title = {{IMB:} An Italian Medical Benchmark for Question Answering},
booktitle = {Proceedings of the Eleventh Italian Conference on Computational Linguistics
(CLiC-it 2025), Cagliari, Italy, September 24-26, 2025},
series = {{CEUR} Workshop Proceedings},
volume = {4112},
publisher = {CEUR-WS.org},
year = {2025},
url = {https://ceur-ws.org/Vol-4112/92_main_long.pdf}
}
```
---
## 🏗 Training Details
* Base model: `unsloth/Llama-3.2-1B-Instruct-unsloth-bnb-4bit`
* Fine-tuning method: LoRA (Unsloth)
* Quantization: 4-bit (BitsAndBytes)
* Adapter merging: Yes (Full merged model)
* Language: Italian
* Domain: Medical — Medicina Generale
* Training size: ~10,000 samples
---
## 📜 License
This work is licensed under a
[Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License][cc-by-nc-nd].
[![CC BY-NC-ND 4.0][cc-by-nc-nd-image]][cc-by-nc-nd]
[cc-by-nc-nd]: http://creativecommons.org/licenses/by-nc-nd/4.0/
[cc-by-nc-nd-image]: https://licensebuttons.net/l/by-nc-nd/4.0/88x31.png
[cc-by-nc-nd-shield]: https://img.shields.io/badge/License-CC%20BY--NC--ND%204.0-lightgrey.svg
---
## 🤝 Acknowledgements
👨‍💻 This project was developed by Mariano Barone, Roberta Di Marino, Francesco Di Serio, Giovanni Dioguardi, Marco Postiglione, Antonio Romano, Giuseppe Riccio, and Vincenzo Moscato at University of Naples, Federico II

93
chat_template.jinja Normal file
View File

@@ -0,0 +1,93 @@
{{- bos_token }}
{%- if custom_tools is defined %}
{%- set tools = custom_tools %}
{%- endif %}
{%- if not tools_in_user_message is defined %}
{%- set tools_in_user_message = true %}
{%- endif %}
{%- if not date_string is defined %}
{%- if strftime_now is defined %}
{%- set date_string = strftime_now("%d %b %Y") %}
{%- else %}
{%- set date_string = "26 Jul 2024" %}
{%- endif %}
{%- endif %}
{%- if not tools is defined %}
{%- set tools = none %}
{%- endif %}
{#- This block extracts the system message, so we can slot it into the right place. #}
{%- if messages[0]['role'] == 'system' %}
{%- set system_message = messages[0]['content']|trim %}
{%- set messages = messages[1:] %}
{%- else %}
{%- set system_message = "" %}
{%- endif %}
{#- System message #}
{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
{%- if tools is not none %}
{{- "Environment: ipython\n" }}
{%- endif %}
{{- "Cutting Knowledge Date: December 2023\n" }}
{{- "Today Date: " + date_string + "\n\n" }}
{%- if tools is not none and not tools_in_user_message %}
{{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
{{- "Do not use variables.\n\n" }}
{%- for t in tools %}
{{- t | tojson(indent=4) }}
{{- "\n\n" }}
{%- endfor %}
{%- endif %}
{{- system_message }}
{{- "<|eot_id|>" }}
{#- Custom tools are passed in a user message with some extra guidance #}
{%- if tools_in_user_message and not tools is none %}
{#- Extract the first user message so we can plug it in here #}
{%- if messages | length != 0 %}
{%- set first_user_message = messages[0]['content']|trim %}
{%- set messages = messages[1:] %}
{%- else %}
{{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
{%- endif %}
{{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
{{- "Given the following functions, please respond with a JSON for a function call " }}
{{- "with its proper arguments that best answers the given prompt.\n\n" }}
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
{{- "Do not use variables.\n\n" }}
{%- for t in tools %}
{{- t | tojson(indent=4) }}
{{- "\n\n" }}
{%- endfor %}
{{- first_user_message + "<|eot_id|>"}}
{%- endif %}
{%- for message in messages %}
{%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
{{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
{%- elif 'tool_calls' in message %}
{%- if not message.tool_calls|length == 1 %}
{{- raise_exception("This model only supports single tool-calls at once!") }}
{%- endif %}
{%- set tool_call = message.tool_calls[0].function %}
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
{{- '{"name": "' + tool_call.name + '", ' }}
{{- '"parameters": ' }}
{{- tool_call.arguments | tojson }}
{{- "}" }}
{{- "<|eot_id|>" }}
{%- elif message.role == "tool" or message.role == "ipython" %}
{{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
{%- if message.content is mapping or message.content is iterable %}
{{- message.content | tojson }}
{%- else %}
{{- message.content }}
{%- endif %}
{{- "<|eot_id|>" }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
{%- endif %}

38
config.json Normal file
View File

@@ -0,0 +1,38 @@
{
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 128000,
"dtype": "float16",
"eos_token_id": 128009,
"head_dim": 64,
"hidden_act": "silu",
"hidden_size": 2048,
"initializer_range": 0.02,
"intermediate_size": 8192,
"max_position_embeddings": 131072,
"mlp_bias": false,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 16,
"num_key_value_heads": 8,
"pad_token_id": 128004,
"pretraining_tp": 1,
"rms_norm_eps": 1e-05,
"rope_scaling": {
"factor": 32.0,
"high_freq_factor": 4.0,
"low_freq_factor": 1.0,
"original_max_position_embeddings": 8192,
"rope_type": "llama3"
},
"rope_theta": 500000.0,
"tie_word_embeddings": true,
"transformers_version": "4.57.1",
"unsloth_fixed": true,
"unsloth_version": "2026.5.2",
"use_cache": true,
"vocab_size": 128256
}

14
generation_config.json Normal file
View File

@@ -0,0 +1,14 @@
{
"bos_token_id": 128000,
"do_sample": true,
"eos_token_id": [
128001,
128008,
128009
],
"max_length": 131072,
"pad_token_id": 128004,
"temperature": 0.6,
"top_p": 0.9,
"transformers_version": "4.57.1"
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e89c465d6fe781b7d8883c194fad68a4a9949877d42b838f5a5cb6b1635cf1a1
size 2471645464

23
special_tokens_map.json Normal file
View File

@@ -0,0 +1,23 @@
{
"bos_token": {
"content": "<|begin_of_text|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<|eot_id|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<|finetune_right_pad_id|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

BIN
tokenizer.json (Stored with Git LFS) Normal file

Binary file not shown.

2066
tokenizer_config.json Normal file

File diff suppressed because it is too large Load Diff