Violet_Twilight-v0.2-GGUF-IQ-Imatrix/README.md

base_model, license, inference, tags

base_model	license	inference	tags
Epiculous/Violet_Twilight-v0.2	apache-2.0	false	mistral nemo roleplay sillytavern gguf

Model name:
Violet_Twilight-v0.2

Description:
"Now for something a bit different, Violet_Twilight-v0.2! This model is a SLERP merge of Azure_Dusk-v0.2 and Crimson_Dawn-v0.2!"
– by Author.

Use the ChatML prompt format.

Important

[Added]
ARM quants:
"Q4_0_4_4", "Q4_0_4_8", "Q4_0_8_8"

Tip

Presets:
You can use ChatML presets within SillyTavern and adjust from there.
Alternatively, check out Virt-io's ChatML v1.9 presets here, make sure you read the repository page for how to use them properly.
The author also provides links to custom sampler presets in the model page here.

Note

Original model page:
https://huggingface.co/Epiculous/Violet_Twilight-v0.2

Quantized using llama.cpp-b3829:

1. Base⇢ Convert-GGUF(FP16)⇢ Generate-Imatrix-Data(FP16)
2. Base⇢ Convert-GGUF(BF16)⇢ Use-Imatrix-Data(FP16)⇢ Quantize-GGUF(Imatrix-Quants)