library_name, license, language, base_model, datasets, tags, pipeline_tag

library_name	license	language	base_model	datasets	tags	pipeline_tag
transformers	apache-2.0	sk en	slovak-nlp/Qwen3-14B-sk	saillab/alpaca-slovak-cleaned	slovak instruction-tuned qwen3 lora merged-lora text-generation	text-generation

Qwen3-14B-sk Alpaca Slovak IT

Qwen3-14B-sk Alpaca Slovak IT is an instruction-tuned Slovak assistant model derived from slovak-nlp/Qwen3-14B-sk. It was trained with a LoRA supervised fine-tuning recipe on Slovak Alpaca-style instruction data and then merged back into the base model, so the released artifact can be loaded directly with transformers.

The model is designed to answer in Slovak by default while still attempting to follow explicit user requests for another language or a specific output format.

Intended Use

This model is intended for Slovak instruction following, Slovak question answering, drafting, rewriting, summarization-style prompts, and general assistant workflows where Slovak is the primary language.

It can also respond to English prompts and translation-style requests, but language control is not guaranteed. Use additional application-level checks for strict formatting, policy compliance, or high-reliability translation.

Do not use this model as the sole source for medical, legal, financial, safety, or other high-stakes decisions. It has not been safety aligned, red-teamed, or moderated for production deployment.

Model Details

Field	Value
Base model	slovak-nlp/Qwen3-14B-sk
Base revision	61bc40572b2fa4720e75a7a9de457151aefce396
Release repository	mrshu/qwen3-14b-sk-alpaca-slovak-it
Architecture	Qwen3 causal language model
Tuned artifact	Merged full-weight checkpoint
Adapter used during training	LoRA
Weight dtype	bfloat16
License	Apache-2.0

Training Data

The model was instruction-tuned on saillab/alpaca-slovak-cleaned, a Slovak Alpaca-style instruction dataset.

Dataset preparation converted each example into a chat-style conversation with system, user, and assistant messages. Empty instruction/output examples were excluded, and duplicate instruction/input/output triples were removed across the prepared splits.

Split	Rows
Train	41,601
Held-out	10,401

The data preparation used dataset revision 058172466eb1d6a28b161f29c74350911d154161.

Each training conversation used this system prompt:

Si užitočný asistent. Riaď sa jazykom a požadovaným formátom používateľa. Ak používateľ nežiada iný jazyk, odpovedaj po slovensky.

Training Recipe

The model was trained with supervised fine-tuning using PEFT LoRA. Only the assistant turns were included in the training loss.

Setting	Value
LoRA rank / alpha / dropout	32 / 64 / 0.05
Target modules	linear projection modules
Sequence length	4096
Sample packing	enabled
Epochs	1
Effective batch size	16
Micro batch size	1
Gradient accumulation	16
Learning rate	1e-4
Scheduler	cosine
Warmup ratio	0.06
Optimizer	fused AdamW
Precision	bfloat16
Gradient checkpointing	enabled
Seed	42

After training, the LoRA adapter was merged into the base model with PEFT and saved as a standalone safetensors checkpoint.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mrshu/qwen3-14b-sk-alpaca-slovak-it"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "system",
        "content": (
            "Si užitočný asistent. Riaď sa jazykom a požadovaným formátom "
            "používateľa. Ak používateľ nežiada iný jazyk, odpovedaj po slovensky."
        ),
    },
    {"role": "user", "content": "Stručne vysvetli, čo je LoRA."},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.6,
    top_p=0.95,
)

print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Limitations

• The supervised fine-tuning data is translated instruction data, so the model may inherit translation artifacts, unnatural phrasing, or source-dataset biases.
• The model is biased toward Slovak responses. Explicit requests for another language may work, but they are not guaranteed.
• Strict JSON, exact labels, citations, and other constrained formats should be validated outside the model.
• The model may hallucinate facts, produce unsafe content, or follow malicious instructions without additional safeguards.