81 lines
2.9 KiB
Markdown
81 lines
2.9 KiB
Markdown
---
|
|
language:
|
|
- uz
|
|
- en
|
|
license: cc-by-nc-4.0
|
|
datasets:
|
|
- yakhyo/uz-wiki
|
|
- tahrirchi/uz-books-v2
|
|
- tahrirchi/uz-crawl
|
|
- saillab/alpaca_uzbek_taco
|
|
- behbudiy/alpaca-cleaned-uz
|
|
- UAzimov/uzbek-instruct-llm
|
|
- CohereLabs/aya_collection_language_split
|
|
- med-alex/qa_mt_ru_to_uzn
|
|
- med-alex/qa_mt_tr_to_uzn
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
base_model: inspirebek/qwen3-4b-uzbek-v2
|
|
tags:
|
|
- uzbek
|
|
- qwen3
|
|
- quantized
|
|
- 4-bit
|
|
- awq
|
|
---
|
|
|
|
# qwen3-4b-uzbek-v2-awq
|
|
|
|
awq 4-bit activation-aware quant (~3.4 gb) of [`inspirebek/qwen3-4b-uzbek-v2`](https://huggingface.co/inspirebek/qwen3-4b-uzbek-v2). fast gpu inference via vllm / tgi / transformers.
|
|
|
|
## usage
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
tok = AutoTokenizer.from_pretrained("inspirebek/qwen3-4b-uzbek-v2-awq")
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
"inspirebek/qwen3-4b-uzbek-v2-awq",
|
|
device_map="auto",
|
|
)
|
|
```
|
|
|
|
with vllm:
|
|
|
|
```bash
|
|
vllm serve inspirebek/qwen3-4b-uzbek-v2-awq --quantization awq --dtype float16
|
|
```
|
|
|
|
## quantization
|
|
|
|
- method: awq (`autoawq` 0.2.9, gemm version)
|
|
- `w_bit=4, q_group_size=128, zero_point=True`
|
|
- calibration: 128 uzbek samples (2048 tokens each) from `fluency.jsonl`
|
|
|
|
## datasets
|
|
|
|
**stage a — fluency (continued pretraining):**
|
|
|
|
- [`yakhyo/uz-wiki`](https://huggingface.co/datasets/yakhyo/uz-wiki) · MIT
|
|
- [`tahrirchi/uz-books-v2`](https://huggingface.co/datasets/tahrirchi/uz-books-v2) · MIT
|
|
- [`tahrirchi/uz-crawl`](https://huggingface.co/datasets/tahrirchi/uz-crawl) · Apache-2.0
|
|
|
|
**stage b — instruct (sft):**
|
|
|
|
- [`saillab/alpaca_uzbek_taco`](https://huggingface.co/datasets/saillab/alpaca_uzbek_taco) · CC-BY-NC-4.0
|
|
- [`behbudiy/alpaca-cleaned-uz`](https://huggingface.co/datasets/behbudiy/alpaca-cleaned-uz) · CC-BY-4.0
|
|
- [`UAzimov/uzbek-instruct-llm`](https://huggingface.co/datasets/UAzimov/uzbek-instruct-llm) · Apache-2.0
|
|
- [`CohereLabs/aya_collection_language_split`](https://huggingface.co/datasets/CohereLabs/aya_collection_language_split) · Apache-2.0
|
|
- [`med-alex/qa_mt_ru_to_uzn`](https://huggingface.co/datasets/med-alex/qa_mt_ru_to_uzn) · unspecified
|
|
- [`med-alex/qa_mt_tr_to_uzn`](https://huggingface.co/datasets/med-alex/qa_mt_tr_to_uzn) · unspecified
|
|
|
|
> ⚠️ licensing note: `saillab/alpaca_uzbek_taco` is cc-by-nc-4.0, which restricts commercial use of derivative models. downstream users who need a fully permissive license should retrain without that subset.
|
|
|
|
## sibling formats
|
|
|
|
- [`inspirebek/qwen3-4b-uzbek-v2`](https://huggingface.co/inspirebek/qwen3-4b-uzbek-v2)
|
|
- [`inspirebek/qwen3-4b-uzbek-v2-lora`](https://huggingface.co/inspirebek/qwen3-4b-uzbek-v2-lora)
|
|
- [`inspirebek/qwen3-4b-uzbek-v2-bnb-4bit`](https://huggingface.co/inspirebek/qwen3-4b-uzbek-v2-bnb-4bit)
|
|
- [`inspirebek/qwen3-4b-uzbek-v2-awq`](https://huggingface.co/inspirebek/qwen3-4b-uzbek-v2-awq)
|
|
- [`inspirebek/qwen3-4b-uzbek-v2-GGUF`](https://huggingface.co/inspirebek/qwen3-4b-uzbek-v2-GGUF)
|