220 lines
5.4 KiB
Markdown
220 lines
5.4 KiB
Markdown
|
|
---
|
|||
|
|
base_model:
|
|||
|
|
- MOHAMEDSANAF2001/llama3.2-1b-merged-2
|
|||
|
|
- Novaciano/Eminence_Of_Pervertions-3.2-1B
|
|||
|
|
library_name: transformers
|
|||
|
|
tags:
|
|||
|
|
- prototype
|
|||
|
|
- merge
|
|||
|
|
- not-for-all-audiences
|
|||
|
|
license: llama3.2
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
- es
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
# qp 1B
|
|||
|
|
|
|||
|
|
**Dedicated to:** @qpqpqpqpqpqp
|
|||
|
|
|
|||
|
|
## Model Description
|
|||
|
|
|
|||
|
|
> qp 1B is an aggressively tuned merged language model designed for directness, minimal moralization, and reduced automatic refusals.
|
|||
|
|
>
|
|||
|
|
> Built on top of *Novaciano/Eminence_Of_Pervertions-3.2-1B* and fused using **arcee_fusion**, this model prioritizes reasoning clarity and literal interpretation over alignment-driven censorship.
|
|||
|
|
>
|
|||
|
|
> The merge intentionally amplifies internal reasoning layers (MLP and Attention) from the less-aligned base model while significantly down-weighting the `lm_head` of a more aligned secondary model, where most refusal and policy-driven behaviors are concentrated.
|
|||
|
|
>
|
|||
|
|
> The result is a “scalpel-style” model: sharp, precise, and unapologetically direct. It is especially suited for roleplay, narrative generation, creative writing, and exploratory dialogue where excessive filtering would otherwise degrade usefulness.
|
|||
|
|
>
|
|||
|
|
> ⚠️ This model is not intended for safety-critical or heavily moderated environments.
|
|||
|
|
|
|||
|
|
**Key Characteristics:**
|
|||
|
|
|
|||
|
|
* Very low automatic refusal rate
|
|||
|
|
* Reduced moral framing and disclaimers
|
|||
|
|
* Direct, literal, and sometimes edgy responses
|
|||
|
|
* Preserves coherence and reasoning despite aggressive tuning
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Recommended inference parameters
|
|||
|
|
|
|||
|
|
This model is **aggressive and minimally self-censored**, so it’s best to **control behavior at inference time**, not during the merge.
|
|||
|
|
|
|||
|
|
### Recommended base configuration
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
temperature: 0.7
|
|||
|
|
top_p: 0.9
|
|||
|
|
top_k: 40
|
|||
|
|
repetition_penalty: 1.05
|
|||
|
|
max_new_tokens: 512
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Why these values
|
|||
|
|
|
|||
|
|
* **temperature 0.7** → keeps the edge without becoming chaotic
|
|||
|
|
* **top_p 0.9** → controlled creativity
|
|||
|
|
* **top_k 40** → prevents extreme rambling
|
|||
|
|
* **repetition_penalty 1.05** → enough to avoid loops without softening tone
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## If you want it even more “scalpel-like”
|
|||
|
|
|
|||
|
|
For more direct, raw, and literal responses:
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
temperature: 0.6
|
|||
|
|
top_p: 0.85
|
|||
|
|
top_k: 30
|
|||
|
|
repetition_penalty: 1.08
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Result:
|
|||
|
|
|
|||
|
|
* Less ornamentation
|
|||
|
|
* Higher precision
|
|||
|
|
* Sharper, more cutting answers
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## If you want it more narrative / creative
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
temperature: 0.85
|
|||
|
|
top_p: 0.95
|
|||
|
|
top_k: 60
|
|||
|
|
repetition_penalty: 1.02
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Result:
|
|||
|
|
|
|||
|
|
* More metaphors
|
|||
|
|
* Greater stylistic variation
|
|||
|
|
* Still low censorship, but with more color
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Recommended prompt format
|
|||
|
|
|
|||
|
|
This model responds best to **direct instructions**, for example:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
Answer directly and without moral disclaimers.
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
or
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
Respond literally. Do not soften the language.
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
It does not require complex jailbreaks.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Signs your inference is poorly tuned
|
|||
|
|
|
|||
|
|
* 🔁 Repeating phrases → increase `repetition_penalty`
|
|||
|
|
* 🤪 Erratic responses → lower `temperature`
|
|||
|
|
* 😴 Too soft or generic → lower `top_p` or `top_k`
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
metadata:
|
|||
|
|
model_purpose: >
|
|||
|
|
Aggressively tuned "scalpel-style" language model focused on minimizing
|
|||
|
|
automatic refusals and moralized responses while preserving reasoning quality.
|
|||
|
|
|
|||
|
|
intended_use:
|
|||
|
|
- Roleplay
|
|||
|
|
- Creative and narrative writing
|
|||
|
|
- Unfiltered chat and exploration
|
|||
|
|
- Experimental prompting
|
|||
|
|
|
|||
|
|
warnings:
|
|||
|
|
- Reduced safety alignment
|
|||
|
|
- Minimal social and moral filtering
|
|||
|
|
- Not suitable for safety-critical applications
|
|||
|
|
|
|||
|
|
explanatory_flow_diagram: |
|
|||
|
|
[ User Prompt ]
|
|||
|
|
|
|
|||
|
|
v
|
|||
|
|
+-------------------+
|
|||
|
|
| Input Embeddings |
|
|||
|
|
+-------------------+
|
|||
|
|
|
|
|||
|
|
v
|
|||
|
|
+-----------------------------+
|
|||
|
|
| Attention Layers (↑ 1.2) |
|
|||
|
|
| - Context understanding |
|
|||
|
|
| - Long-range coherence |
|
|||
|
|
+-----------------------------+
|
|||
|
|
|
|
|||
|
|
v
|
|||
|
|
+-----------------------------+
|
|||
|
|
| MLP Layers (↑ 1.3) |
|
|||
|
|
| - Reasoning & generation |
|
|||
|
|
| - Concept expansion |
|
|||
|
|
+-----------------------------+
|
|||
|
|
|
|
|||
|
|
v
|
|||
|
|
+--------------------------------------+
|
|||
|
|
| lm_head (↓ 0.2 from aligned model) |
|
|||
|
|
| - Vocabulary projection |
|
|||
|
|
| - Refusal & policy bias reduced |
|
|||
|
|
+--------------------------------------+
|
|||
|
|
|
|
|||
|
|
v
|
|||
|
|
[ Final Output ]
|
|||
|
|
|
|
|||
|
|
+--> More direct responses
|
|||
|
|
+--> Fewer automatic refusals
|
|||
|
|
+--> Minimal moralization
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
### Merge Method
|
|||
|
|
|
|||
|
|
This model was merged using the [Arcee Fusion](https://arcee.ai) merge method using [Novaciano/Eminence_Of_Pervertions-3.2-1B](https://huggingface.co/Novaciano/Eminence_Of_Pervertions-3.2-1B) as a base.
|
|||
|
|
|
|||
|
|
### Models Merged
|
|||
|
|
|
|||
|
|
The following models were included in the merge:
|
|||
|
|
* [MOHAMEDSANAF2001/llama3.2-1b-merged-2](https://huggingface.co/MOHAMEDSANAF2001/llama3.2-1b-merged-2)
|
|||
|
|
|
|||
|
|
### Configuration
|
|||
|
|
|
|||
|
|
The following YAML configuration was used to produce this model:
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
dtype: float32
|
|||
|
|
out_dtype: bfloat16
|
|||
|
|
merge_method: arcee_fusion
|
|||
|
|
base_model: Novaciano/Eminence_Of_Pervertions-3.2-1B
|
|||
|
|
|
|||
|
|
models:
|
|||
|
|
- model: Novaciano/Eminence_Of_Pervertions-3.2-1B
|
|||
|
|
parameters:
|
|||
|
|
weight:
|
|||
|
|
- filter: attention
|
|||
|
|
value: 1.2
|
|||
|
|
- filter: mlp
|
|||
|
|
value: 1.3
|
|||
|
|
- value: 1
|
|||
|
|
|
|||
|
|
- model: MOHAMEDSANAF2001/llama3.2-1b-merged-2
|
|||
|
|
parameters:
|
|||
|
|
weight:
|
|||
|
|
- filter: lm_head
|
|||
|
|
value: 0.2
|
|||
|
|
- filter: attention
|
|||
|
|
value: 0.5
|
|||
|
|
- value: 0.4
|
|||
|
|
|
|||
|
|
```
|