OmniTranslate-1.0-GGUF/README.md

---
base_model: MihaiPopa-1/OmniTranslate-1.0
datasets:
- MihaiPopa-1/OmniSurgical-1.0
language:
- abk
- abq
- abs
- acm
- adh
- adi
- ady
- aeb
- afr
- agx
- aii
- aim
- ain
- ajz
- akb
- aln
- als
- alt
- amh
- anp
- aoz
- apc
- apt
- arb
- arg
- arq
- ars
- ary
- arz
- asm
- ast
- atb
- ava
- awa
- ayp
- ayr
- azb
- azj
- bak
- bam
- ban
- bar
- bas
- bbc
- bbk
- bcl
- bdq
- bel
- ben
- bew
- bho
- bhp
- bis
- biu
- bjn
- bod
- bos
- brh
- brx
- bts
- btx
- bug
- bul
- bwi
- bxr
- cat
- cbk
- ccp
- ceb
- ces
- cfm
- cha
- che
- chr
- chu
- chv
- cjs
- ckb
- ckt
- cmn
- cnh
- cnw
- cos
- crh
- crj
- crk
- crl
- crs
- csb
- csw
- csy
- ctd
- cym
- czt
- dak
- dan
- dar
- deu
- dik
- diu
- div
- dje
- dks
- dln
- dng
- dnw
- doi
- dru
- dsb
- dtp
- dty
- dzo
- ekk
- ell
- enl
- enm
- epo
- ess
- eus
- eve
- ewo
- ext
- fao
- fas
- ffm
- fij
- fil
- fin
- fit
- fkv
- fmu
- fra
- fro
- frp
- fry
- fuf
- fur
- fuv
- gag
- gaz
- gcf
- gla
- gle
- glg
- glk
- glv
- gmh
- gnb
- goh
- gom
- gos
- grc
- gsw
- gug
- guj
- guz
- hac
- hae
- hak
- hat
- hau
- haw
- hbo
- heb
- her
- hif
- hil
- hin
- hmr
- hne
- hns
- hrv
- hrx
- hsb
- hun
- hwc
- hye
- hyw
- iba
- ibg
- ibo
- ife
- ike
- ikt
- ilo
- ina
- ind
- inh
- isl
- ita
- ivv
- jav
- jpn
- jun
- kaa
- kab
- kac
- kak
- kal
- kam
- kan
- kas
- kat
- kaz
- kbd
- kca
- kdh
- kdr
- kea
- kei
- kgp
- kha
- khk
- khm
- kik
- kin
- kir
- kiu
- kjb
- kjh
- kmr
- knc
- koi
- kor
- kos
- kpv
- krj
- krl
- kru
- ksh
- ksw
- ktj
- ktz
- kua
- kum
- kwn
- kyu
- kzj
- lad
- lao
- lat
- lbe
- ldn
- lew
- lez
- lfn
- lim
- lin
- lis
- lit
- lki
- lld
- lmk
- lnd
- lrc
- ltg
- ltz
- lud
- lug
- luo
- lus
- lvs
- lwg
- lzh
- mag
- mah
- mai
- mak
- mal
- mar
- mas
- mbf
- mdf
- mer
- mfe
- mfg
- mfy
- mhi
- mhr
- mhy
- min
- mip
- mjw
- mkd
- mlt
- mni
- mnk
- mns
- mnw
- moh
- mph
- mqy
- mri
- mrj
- mrw
- mtg
- mui
- mup
- mus
- mvp
- mwf
- mwl
- mww
- mya
- myv
- myx
- mzh
- nah
- nan
- nap
- naq
- nbu
- nde
- ndo
- nds
- new
- nio
- njn
- njo
- nld
- nmf
- nmz
- nno
- nob
- nog
- non
- npi
- npo
- nrf
- nri
- nrm
- nse
- nus
- nya
- nyn
- nzm
- obo
- oci
- ojb
- olo
- orv
- ory
- oss
- ota
- oto
- otw
- pam
- pan
- pap
- pbt
- pcd
- pck
- pcm
- pfl
- plt
- pmq
- pmx
- pnb
- pnt
- pol
- por
- pov
- ppk
- pps
- prg
- pui
- pxm
- quc
- qul
- qup
- qus
- quz
- raw
- rcf
- rel
- rhg
- ria
- rjs
- rmc
- rml
- rmn
- rmy
- rnl
- roh
- ron
- rtm
- rue
- run
- rus
- sah
- san
- sat
- sck
- scn
- sda
- sdc
- sdh
- ses
- sgc
- sgh
- sid
- sin
- sju
- skr
- slk
- slv
- sma
- sme
- smj
- smn
- smo
- sms
- smt
- sna
- snd
- som
- sot
- spa
- srd
- srp
- ssw
- sun
- swe
- swg
- swh
- syc
- syl
- szl
- tab
- tam
- taq
- tat
- tcy
- tcz
- tel
- tet
- tgk
- tha
- thl
- tig
- tir
- tkl
- tkr
- tlh
- tly
- tok
- ton
- tpi
- tpw
- trc
- trp
- trs
- ttj
- tuk
- tur
- tuv
- twx
- tyv
- tzl
- tzm
- udm
- uig
- ukr
- urd
- uzn
- uzs
- vap
- vie
- vot
- vro
- war
- way
- wba
- wbm
- wes
- whk
- wlx
- wol
- wsg
- wwa
- xal
- xho
- xmm
- xmv
- xog
- yaz
- ydd
- yor
- yrk
- yrl
- yua
- yue
- zea
- zgh
- zom
- zsm
- zul
library_name: transformers
license: apache-2.0
mradermacher:
  readme_rev: 1
quantized_by: mradermacher
tags:
- text-generation-inference
- translation
- transformers
- unsloth
- qwen3
- omnitranslate
---
## About

<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type:  -->
<!-- ### tags:  -->
<!-- ### quants:  x-f16 Q4_K_S Q2_K Q8_0 Q6_K Q3_K_M Q3_K_S Q3_K_L Q4_K_M Q5_K_S Q5_K_M IQ4_XS -->
<!-- ### quants_skip:  -->
<!-- ### skip_mmproj:  -->
static quants of https://huggingface.co/MihaiPopa-1/OmniTranslate-1.0

<!-- provided-files -->

***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#OmniTranslate-1.0-GGUF).***

weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion.
## Usage

If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.

## Provided Quants

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q2_K.gguf) | Q2_K | 0.4 |  |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q3_K_S.gguf) | Q3_K_S | 0.4 |  |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q3_K_M.gguf) | Q3_K_M | 0.4 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q3_K_L.gguf) | Q3_K_L | 0.5 |  |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.IQ4_XS.gguf) | IQ4_XS | 0.5 |  |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q4_K_S.gguf) | Q4_K_S | 0.5 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q4_K_M.gguf) | Q4_K_M | 0.5 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q5_K_S.gguf) | Q5_K_S | 0.5 |  |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q5_K_M.gguf) | Q5_K_M | 0.5 |  |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q6_K.gguf) | Q6_K | 0.6 | very good quality |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.Q8_0.gguf) | Q8_0 | 0.7 | fast, best quality |
| [GGUF](https://huggingface.co/mradermacher/OmniTranslate-1.0-GGUF/resolve/main/OmniTranslate-1.0.f16.gguf) | f16 | 1.3 | 16 bpw, overkill |

Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):

![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)

And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

## FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.

## Thanks

I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time.

<!-- end -->