license, language, base_model, pipeline_tag, tags

license	language	base_model	pipeline_tag	tags
apache-2.0	en	DarwinAnim8or/Trouper-12B	text-generation	character_roleplay creative_writing roleplay llama-cpp

Trouper-12B GGUF by DarwinAnim8or

From the original model card:

A character roleplay model trained on the custom "Actors" dataset, fine-tuned from Mistral-Nemo-Base-12B. This model was made to expand on the things I learned from TinyRP, and to overcome certain limitations I found from it; also on an entirely new dataset made just for this model.

This model writes more naturally, less like "AI"; even more so than the 24B model I'm also releasing. I suppose this is because the 12B model saw less synthethic data, and is thus less likely to use phrases typical in AI writing & prose.

Using an importance matrix actually increases KLD (lowers quality) for Q8_0 and Q6_K, so I didn't use one for those quants. For anything smaller the importance matrix provides better quality with no inference performance penalty.

The importance matrix was computed using the Bartowski's calibration_datav3.txt dataset, which has been proven to produce good results for all kinds of tasks.

The KLD metric was measured against the wikitext-2 validation split with 4096 tokens of context.

Quant	BPW	Size (GiB)	KLD99	iMatrix
BF16	16.00	22.81	N/A	N/A
Q8_0	8.50	12.12	0.005840
Q6_K	6.56	9.36	0.023252
Q5_K_M	5.70	8.12	0.044965	X
Q5_K_S	5.56	7.93	0.053320	X
Q4_K_M	4.88	6.96	0.130179	X
Q4_K_S	4.65	6.62	0.156863	X
Q3_K_L	4.28	6.10	0.391981	X
Q3_K_M	3.97	5.66	0.453535	X
Q3_K_S	3.61	5.15	1.682329	X
Q2_K	3.12	4.45	2.297512	X
Q2_K_S	2.93	4.18	2.207151	X
IQ4_NL	4.63	6.60	0.165374	X
IQ4_XS	4.40	6.27	0.171350	X
IQ3_M	3.73	5.32	0.513056	X
IQ3_S	3.63	5.17	0.544215	X
IQ3_XS	3.46	4.93	0.733339	X
IQ3_XXS	3.23	4.60	1.081350	X
IQ2_M	2.89	4.12	1.818141	X
IQ2_S	2.70	3.85	2.526351	X
IQ2_XS	2.55	3.64	2.840284	X
IQ2_XXS	2.34	3.34	3.772680	X
IQ1_M	2.10	2.99	5.374491	X
IQ1_S	1.95	2.79	6.467200	X