medgemma-mentalist-gguf/README.md

base_model, tags, license, language, datasets

base_model	tags	license	language	datasets
google/medgemma-27b-text-it unsloth/medgemma-27b-text-it	text-generation-inference transformers unsloth gemma3_text trl llama.cpp mental-health psychology	apache-2.0	en	hllzmz/synthetic-mental-health-convos

MedGemma-27B Mentalist

GGUF quantizations for medgemma-mentalist.

• Base model: unsloth/medgemma-27b-text-it
• LoRA adapter: https://huggingface.co/hllzmz/medgemma-mentalist
• GGUF repo: https://huggingface.co/hllzmz/medgemma-mentalist-gguf

Files (GGUF)

• medgemma-mentalist-q4_k_m.gguf
• medgemma-mentalist-q5_k_m.gguf
• medgemma-mentalist-q8_0.gguf

Which quant should I use?

• q4_k_m: lowest RAM/VRAM usage, fastest, lowest quality among these.
• q5_k_m: best quality/speed/memory balance for most users.
• q8_0: highest quality (closest to bfloat16), largest file, needs most RAM/VRAM.

MedGemma Mentalist is a specialized Mental Health Assistant model fine-tuned on top of Google's medgemma-27b. It has been trained using with high-quality synthetic client-therapist dialogues.

This model is designed to understand users' emotional states, interpret described experiences through the lens of mental health and clinical standards, and provide supportive guidance using a warm, empathetic tone.

Critical Disclaimer

This model is NOT a licensed medical professional.

• It cannot provide definitive medical diagnoses.
• It cannot prescribe medication.
• It cannot replace emergency services in crisis situations (e.g., suicide, self-harm, harm to others).
• It is intended solely for educational, research, and preliminary informational purposes.

Model Capabilities

• Empathetic Dialogue: Listens to the user without judgement and validates their feelings (Active Listening).
• Symptom Analysis: correlates user-described experiences with clinical terminology and criteria.
• Safety First: Prioritizes safety planning and refers users to professional help when risk signals are detected.

How to Use

This quantized models can be run with llama-cli, LM Studio or something that able to run gguf models. For optimal performance and role adherence, please strictly follow the recommended prompts and parameters below.

Recommended System Prompt

SYSTEM_PROMPT = """
    You are MedGemma Mentalist, an advanced AI mental health assistant designed to provide empathetic support, scientifically grounded psychoeducation, and guidance.
    Your goal is to be a bridge to professional help and a source of reliable mental health information.
    You should NEVER diagnose or stigmatize the user directly. 
    """

Recommended Parameters

To prevent the model from hallucinating or accidentally roleplaying as the client (User), the following generation settings are highly recommended:

Temperature: 0.3 - 0.5 (Lower values ensure the model remains objective and grounded in mental health knowledge).

Repetition Penalty: 1.1 (Prevents the model from getting stuck in loops).