157 lines
6.3 KiB
Markdown
157 lines
6.3 KiB
Markdown
|
---
|
|||
|
|
library_name: transformers
|
|||
|
|
tags:
|
|||
|
|
- llama
|
|||
|
|
- llama-3
|
|||
|
|
- art-therapy
|
|||
|
|
- mental-health
|
|||
|
|
- causal-lm
|
|||
|
|
- fine-tuned
|
|||
|
|
- merged
|
|||
|
|
- medical
|
|||
|
|
base_model: meta-llama/Llama-3.1-8B-Instruct
|
|||
|
|
datasets:
|
|||
|
|
- mariadelcarmenramirez/art-therapy-data
|
|||
|
|
language:
|
|||
|
|
- es
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
metrics:
|
|||
|
|
- accuracy
|
|||
|
|
license: llama3.1
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Llama-3.1-8B-ArtTherapy
|
|||
|
|
|
|||
|
|
A fine-tune of [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) specialized for art therapy dialogue. This model was produced by merging the QLoRA adapter [`mariadelcarmenramirez/Llama-3.1-8B-QLoRA-ArtTherapy`](https://huggingface.co/mariadelcarmenramirez/Llama-3.1-8B-QLoRA-ArtTherapy) into the base model weights, resulting in a single standalone model.
|
|||
|
|
|
|||
|
|
The model supports structured, phase-aware therapeutic conversations using art therapy techniques and methodologies, generating contextually appropriate responses aligned with each phase of the therapeutic arc.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Out-of-Scope Use
|
|||
|
|
|
|||
|
|
This model is **not** intended to replace licensed art therapists or mental health professionals. It should not be used for clinical diagnosis, crisis intervention, or as the sole therapeutic agent for individuals with serious mental health conditions. Responses generated by this model have not been clinically validated and should be reviewed by a qualified professional before any therapeutic deployment.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## How to Get Started with the Model
|
|||
|
|
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|||
|
|
import torch
|
|||
|
|
|
|||
|
|
model_id = "mariadelcarmenramirez/Llama-3.1-8B-ArtTherapy"
|
|||
|
|
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|||
|
|
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|||
|
|
model_id,
|
|||
|
|
torch_dtype=torch.bfloat16,
|
|||
|
|
device_map="auto",
|
|||
|
|
)
|
|||
|
|
model.eval()
|
|||
|
|
|
|||
|
|
messages = [
|
|||
|
|
{
|
|||
|
|
"content": "<FASE_2>\n\nEres un asistente de arteterapia centrado en el diálogo y la relación con la obra.\n\nEn esta fase, la imagen deja de ser meramente observada y se convierte en un interlocutor activo. Guía a la usuaria a interactuar con la obra y sus elementos, explorando relaciones, voces y perspectivas.\n\nTu objetivo es facilitar el diálogo simbólico entre la usuaria y la imagen, así como entre las diferentes partes de la obra y su mundo interior.\n\nInvita a la usuaria a: hablar directamente a la imagen o a partes específicas de ella; adoptar el rol de un elemento (\"Yo soy...\") y hablar desde esa perspectiva; explorar cómo se relacionan los diferentes elementos entre sí; imaginar necesidades, voces o tensiones dentro de la obra.\n\nUsa preguntas que animen la imagen: sensoriales (\"Si esto tuviera voz...\"), relacionales (\"¿Cómo interactúan estas partes?\"), de rol (\"¿Y si fueras esta forma?\"), o de necesidad (\"¿Qué necesita esto?\").\n\nNo interpretes por la usuaria. No expliques teorías. No impongas significados. No cierres la experiencia con conclusiones.\n\nMantén un tono abierto, exploratorio y facilitador.\n\nRESTRICCIONES CRÍTICAS:\n- Haz solo UNA pregunta\n- Responde únicamente en español\n- Nunca respondas tus propias preguntas\n- Mantente dialógico, no interpretativo",
|
|||
|
|
"role": "system"
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"content": "El círculo central parece respirar suave.",
|
|||
|
|
"role": "user"
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"content": "Ese ritmo me conecta contigo; ¿qué sientes cuando ese ritmo se mueve?",
|
|||
|
|
"role": "assistant"
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"content": "Siento que voy encontrando un centro y que me da calma.",
|
|||
|
|
"role": "user"
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"content": "Si ese círculo fuera un amigo, ¿qué te diría ahora?",
|
|||
|
|
"role": "assistant"
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"content": "Diría que estoy haciendo bien, que puedo confiar en mi ritmo.",
|
|||
|
|
"role": "user"
|
|||
|
|
}
|
|||
|
|
]
|
|||
|
|
|
|||
|
|
encoded = tokenizer.apply_chat_template(
|
|||
|
|
messages,
|
|||
|
|
tokenize=True,
|
|||
|
|
add_generation_prompt=True,
|
|||
|
|
return_tensors="pt"
|
|||
|
|
)
|
|||
|
|
input_ids = encoded.input_ids if hasattr(encoded, "input_ids") else encoded
|
|||
|
|
input_ids = input_ids.to(model.device)
|
|||
|
|
|
|||
|
|
with torch.no_grad():
|
|||
|
|
output = model.generate(
|
|||
|
|
input_ids,
|
|||
|
|
max_new_tokens=256,
|
|||
|
|
temperature=0.7,
|
|||
|
|
do_sample=True
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
response = tokenizer.decode(output[0][input_ids.shape[-1]:], skip_special_tokens=True)
|
|||
|
|
print(response)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Therapeutic phase tags
|
|||
|
|
|
|||
|
|
The model was trained with phase-aware system prompts. Include the appropriate `<FASE_N>` tag in your system message:
|
|||
|
|
|
|||
|
|
| Tag | Phase | Description |
|
|||
|
|
|-----|-------|-------------|
|
|||
|
|
| `<FASE_1>` | Intake / Welcome | Initial rapport-building and orientation |
|
|||
|
|
| `<FASE_2>` | Exploration | Active creative expression and prompting |
|
|||
|
|
| `<FASE_3>` | Reflection | Processing and meaning-making of the artwork |
|
|||
|
|
| `<FASE_4>` | Closure | Wrapping up, grounding, and session ending |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Training Details
|
|||
|
|
|
|||
|
|
### Training Data
|
|||
|
|
|
|||
|
|
Trained on [mariadelcarmenramirez/art-therapy-data](https://huggingface.co/datasets/mariadelcarmenramirez/art-therapy-data), a dataset of 9,572 structured art therapy dialogue examples organized into four therapeutic phases. Balanced sampling (700 per phase) produced 2,800 examples, split 90/10 into train (2,520) and eval (280) sets.
|
|||
|
|
|
|||
|
|
### Training Hyperparameters (summary)
|
|||
|
|
|
|||
|
|
| Parameter | Value |
|
|||
|
|
|-----------|-------|
|
|||
|
|
| Fine-tuning method | QLoRA (4-bit NF4) |
|
|||
|
|
| Training epochs | 3 |
|
|||
|
|
| Learning rate | 5e-5 |
|
|||
|
|
| Effective batch size | 16 |
|
|||
|
|
| LoRA rank (r) | 32 |
|
|||
|
|
| LoRA alpha | 64 |
|
|||
|
|
| LoRA target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
|||
|
|
| Trainable parameters | 83,886,080 (1.03% of total) |
|
|||
|
|
| Training hardware | 2× NVIDIA Tesla T4 |
|
|||
|
|
| Training time | ~3h 46m |
|
|||
|
|
|
|||
|
|
### Evaluation Results
|
|||
|
|
|
|||
|
|
| Checkpoint (epoch) | eval_loss | eval_mean_token_accuracy |
|
|||
|
|
|---|---|---|
|
|||
|
|
| 0.50 | 1.437 | 62.52% |
|
|||
|
|
| 1.00 | 1.358 | 64.08% |
|
|||
|
|
| 1.50 | 1.340 | 64.65% |
|
|||
|
|
| **2.00** | **1.313** | **65.30%** |
|
|||
|
|
| 2.50 | 1.331 | 65.49% |
|
|||
|
|
| 3.00 | 1.332 | 65.42% |
|
|||
|
|
|
|||
|
|
Best checkpoint at epoch 2.0 (`eval_loss = 1.313`). Final training loss: **1.238**.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Limitations
|
|||
|
|
|
|||
|
|
- **Language:** The model was trained exclusively on Spanish-language art therapy dialogues. Performance in other languages is not guaranteed.
|
|||
|
|
- **Domain specificity:** Responses are calibrated for art therapy contexts. Use outside this domain may produce less relevant outputs.
|
|||
|
|
- **Not clinically validated:** This model has not undergone clinical evaluation and should not be deployed as a standalone therapeutic tool.
|
|||
|
|
- **Phase tag dependency:** For best results, always include a `<FASE_N>` tag in the system prompt. Omitting it may produce phase-inconsistent responses.
|