LFM-2.5-1.2b-Instruct-roleplay-tuned is my attempt at making a roleplay model that anybody can run, no matter how bad the hardware is, it was tuned on 5M~ tokens of high quality roleplay data with the aim to make it better at roleplay.
Strengths and challenges:
Advantages
Better creative writing, less slop, a little bit of repetitiveness remains due to size, fixed by samplers
Better formatting, knows how to use asterisks really well
Has a certain "Soul" to it, was personally pretty fun to test
Runs literally anywhere, almost anywhere
Insane inference speed
Disadvantages
Context not that good
Undestanding of complex concepts not that good
Sensitive to quantization, Q8_0 or BF16 is recommended
My first tune, altough I tried my best
Because its a 1.2b dense model, its not the smartest, and may sometimes use wrong pronouns, if it was performed on any model of this size that is not LFM2.5 1.2b instruct, the results would probably be worse.
Why I made this tune?
Because I was very bored, and I wanted something that can work anywhere on virtually any hardware, my own hardware is not the best, but was enough for making this tune, I think there are a lot to improve on with this model, and I will probably work on its mistakes just when I get the time, it can still sometimes glitch a bit, especially in lower quants, but I'm still personally satisfied with the result.
Quants(Speaking from personal experience with this specific model):
BF16- Recommended, highest quality, least logical mistakes.
Q8_0- Recommended, high quality, makes slightly more mistakes but nonetheless near lossless.
Q6_K- Recommended if Q8_0 is too much, degradation begins, not exactly notable here, but you will notice minor detail loss.
Q5_K_M- Recommended if hardware is really, REALLY bad, degradation becomes noticeable.
Q4_K_M- Not recommended for most use cases, degradation is clearly noticeable.
Quants can be found in the repository, along with safetensors.
Wow
I didnt expect this finetune to become that popular, sincere thanks to anyone reading this! V2 is already cooked up and ready for release, I lately realised that 5M~ tokens of data may have been too little for this model, I already tested the v2 and seen real improvements in the way it writes.
