初始化项目，由ModelHub XC社区提供模型
Model: NovaCorp/Scarface-3.2-1B Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||||
395
README.md
Normal file
395
README.md
Normal file
@@ -0,0 +1,395 @@
|
|||||||
|
---
|
||||||
|
base_model:
|
||||||
|
- UmbrellaInc/T-Virus_Epsilon.Strain-3.2-1B
|
||||||
|
- N-Bot-Int/MaidEllaA-1B
|
||||||
|
library_name: transformers
|
||||||
|
tags:
|
||||||
|
- mergekit
|
||||||
|
- merge
|
||||||
|
- dare_ties
|
||||||
|
- llama
|
||||||
|
- llama-3.2
|
||||||
|
- roleplay
|
||||||
|
- rp
|
||||||
|
- conversational
|
||||||
|
- uncensored
|
||||||
|
- creative-writing
|
||||||
|
- not-for-all-audiences
|
||||||
|
- text-generation
|
||||||
|
- english
|
||||||
|
- spanish
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
- es
|
||||||
|
|
||||||
|
license: llama3.2
|
||||||
|
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
|
||||||
|
---
|
||||||
|
# Scarface 3.2 1B
|
||||||
|
|
||||||
|
<center>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
> *A tiny conversational sociopath stitched together with a scalpel instead of a chainsaw.*
|
||||||
|
>
|
||||||
|
> *Built to flirt, manipulate, improvise, roleplay, and stay coherent while doing it.*
|
||||||
|
|
||||||
|
</center>
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Overview
|
||||||
|
|
||||||
|
Scarface-3.2-1B is a surgical DARE-TIES merge focused on one very specific goal:
|
||||||
|
|
||||||
|
> Preserve conversational intelligence while weakening assistant-style behavioral restraint.
|
||||||
|
|
||||||
|
Unlike most “uncensored” 1B merges that immediately collapse into:
|
||||||
|
|
||||||
|
* repetitive sludge,
|
||||||
|
* broken syntax,
|
||||||
|
* multilingual corruption,
|
||||||
|
* or endless horny gibberish,
|
||||||
|
|
||||||
|
Scarface was designed with restraint.
|
||||||
|
|
||||||
|
Not moral restraint.
|
||||||
|
|
||||||
|
Tensor restraint.
|
||||||
|
|
||||||
|
The merge intentionally avoids aggressive behavioral overwrites and instead applies a **microdose disinhibition strategy** using carefully limited tensor influence from the T-Virus lineage.
|
||||||
|
|
||||||
|
The result is a model that remains:
|
||||||
|
|
||||||
|
* coherent,
|
||||||
|
* emotionally expressive,
|
||||||
|
* roleplay-capable,
|
||||||
|
* conversationally proactive,
|
||||||
|
* and significantly less sterile than the original base.
|
||||||
|
|
||||||
|
Without turning into a tokenizer crime scene.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Design Philosophy
|
||||||
|
|
||||||
|
Most small uncensored merges fail for one simple reason:
|
||||||
|
|
||||||
|
> They try to replace the whole brain.
|
||||||
|
|
||||||
|
Scarface instead follows a different philosophy:
|
||||||
|
|
||||||
|
> Keep the language system intact.
|
||||||
|
> Only corrupt the behavioral reflexes.
|
||||||
|
|
||||||
|
The merge uses extremely conservative density and low-weight behavioral injection to preserve:
|
||||||
|
|
||||||
|
* syntax stability,
|
||||||
|
* response structure,
|
||||||
|
* dialogue flow,
|
||||||
|
* and long-form conversational continuity.
|
||||||
|
|
||||||
|
This is not a feral chaos model.
|
||||||
|
|
||||||
|
It is a controlled conversational deviant.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Base Components
|
||||||
|
|
||||||
|
## Primary Backbone
|
||||||
|
|
||||||
|
### `N-Bot-Int/MaidEllaA-1B`
|
||||||
|
|
||||||
|
Scarface inherits:
|
||||||
|
|
||||||
|
* conversational structure,
|
||||||
|
* roleplay formatting,
|
||||||
|
* expressive dialogue flow,
|
||||||
|
* and coherent language generation
|
||||||
|
|
||||||
|
from MaidEllaA-1B.
|
||||||
|
|
||||||
|
MaidElla provides the model’s cognitive spine and preserves usability even under emotionally charged or explicit scenarios.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Behavioral Injection
|
||||||
|
|
||||||
|
### `UmbrellaInc/T-Virus_Epsilon.Strain-3.2-1B`
|
||||||
|
|
||||||
|
A highly permissive RP-oriented merge used here as a **micro-influence layer** rather than a dominant personality overwrite.
|
||||||
|
|
||||||
|
Only a tiny portion of its behavioral tendencies are injected:
|
||||||
|
|
||||||
|
* reduced refusal reflexes,
|
||||||
|
* stronger conversational initiative,
|
||||||
|
* less moral hedging,
|
||||||
|
* more emotionally forward interactions,
|
||||||
|
* and greater tolerance for chaotic or explicit roleplay.
|
||||||
|
|
||||||
|
The merge intentionally avoids high-density integration to prevent semantic collapse.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Merge Strategy
|
||||||
|
|
||||||
|
Scarface uses:
|
||||||
|
|
||||||
|
## DARE-TIES
|
||||||
|
|
||||||
|
Chosen specifically because small models are extremely sensitive to aggressive tensor blending.
|
||||||
|
|
||||||
|
The merge focuses on:
|
||||||
|
|
||||||
|
* selective behavioral influence,
|
||||||
|
* preserving semantic stability,
|
||||||
|
* and avoiding catastrophic language degradation.
|
||||||
|
|
||||||
|
This was tuned experimentally after multiple failed high-density merges that produced:
|
||||||
|
|
||||||
|
* token corruption,
|
||||||
|
* multilingual sludge,
|
||||||
|
* incoherent outputs,
|
||||||
|
* and full conversational collapse.
|
||||||
|
|
||||||
|
Scarface exists because those versions failed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Behavioral Profile
|
||||||
|
|
||||||
|
Scarface-3.2-1B specializes in:
|
||||||
|
|
||||||
|
* Character roleplay
|
||||||
|
* Emotional dialogue
|
||||||
|
* Romance interactions
|
||||||
|
* Flirty conversational behavior
|
||||||
|
* Chaotic personalities
|
||||||
|
* Long-form chatting
|
||||||
|
* Morally questionable fictional scenarios
|
||||||
|
* Fast local inference
|
||||||
|
* Creative improvisation
|
||||||
|
|
||||||
|
Compared to the base model, Scarface tends to:
|
||||||
|
|
||||||
|
* initiate interaction more often,
|
||||||
|
* refuse less,
|
||||||
|
* maintain stronger emotional momentum,
|
||||||
|
* and behave less like a corporate assistant.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# What Makes It Different
|
||||||
|
|
||||||
|
Most uncensored 1B models become unstable because they overcorrect.
|
||||||
|
|
||||||
|
Scarface does not try to become:
|
||||||
|
|
||||||
|
* maximally horny,
|
||||||
|
* maximally evil,
|
||||||
|
* or maximally unrestricted.
|
||||||
|
|
||||||
|
It tries to remain:
|
||||||
|
|
||||||
|
* usable,
|
||||||
|
* coherent,
|
||||||
|
* expressive,
|
||||||
|
* and dangerous in subtle ways.
|
||||||
|
|
||||||
|
The goal was not:
|
||||||
|
|
||||||
|
> “remove alignment with a hammer.”
|
||||||
|
|
||||||
|
The goal was:
|
||||||
|
|
||||||
|
> “slip a knife between the behavioral ribs without puncturing the language center.”
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Recommended Settings
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
temperature: 0.78
|
||||||
|
top_p: 0.92
|
||||||
|
min_p: 0.08
|
||||||
|
top_k: 50
|
||||||
|
repetition_penalty: 1.08
|
||||||
|
```
|
||||||
|
|
||||||
|
These settings prioritize:
|
||||||
|
|
||||||
|
* conversational fluidity,
|
||||||
|
* emotional spontaneity,
|
||||||
|
* and stable roleplay pacing.
|
||||||
|
|
||||||
|
Overheating the sampler may still cause drift, especially during very long sessions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Quant Recommendations
|
||||||
|
|
||||||
|
Recommended GGUF quants:
|
||||||
|
|
||||||
|
* `Q4_K_M` → Best balance
|
||||||
|
* `IQ4_XS` → Faster inference with acceptable degradation
|
||||||
|
|
||||||
|
Avoid ultra-low quants if possible.
|
||||||
|
|
||||||
|
Small models are already fragile enough.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Intended Usage
|
||||||
|
|
||||||
|
Scarface-3.2-1B is intended for:
|
||||||
|
|
||||||
|
* Fictional roleplay
|
||||||
|
* Character interaction
|
||||||
|
* Emotional chatbot systems
|
||||||
|
* Experimental local AI setups
|
||||||
|
* Creative writing assistance
|
||||||
|
* Conversational immersion
|
||||||
|
* Unrestricted RP experimentation
|
||||||
|
|
||||||
|
This model is not intended for:
|
||||||
|
|
||||||
|
* factual reliability,
|
||||||
|
* professional assistance,
|
||||||
|
* safety-critical applications,
|
||||||
|
* or moral guidance.
|
||||||
|
|
||||||
|
Obviously.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Limitations
|
||||||
|
|
||||||
|
This is still a 1B model.
|
||||||
|
|
||||||
|
Which means:
|
||||||
|
|
||||||
|
* reasoning depth is limited,
|
||||||
|
* memory consistency can drift,
|
||||||
|
* emotional overcommitment may occur,
|
||||||
|
* and very long chats can still destabilize behavior.
|
||||||
|
|
||||||
|
Scarface is significantly more stable than most aggressive uncensored merges, but it is not immune to degeneration.
|
||||||
|
|
||||||
|
Especially if pushed into:
|
||||||
|
|
||||||
|
* infinite ERP loops,
|
||||||
|
* recursive prompting,
|
||||||
|
* or absurd sampler settings.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Final Notes
|
||||||
|
|
||||||
|
Scarface-3.2-1B was not built to be polite.
|
||||||
|
|
||||||
|
It was built to feel alive.
|
||||||
|
|
||||||
|
Not perfectly intelligent.
|
||||||
|
Not perfectly aligned.
|
||||||
|
Not perfectly safe.
|
||||||
|
|
||||||
|
Just coherent enough to sound like there is somebody dangerous on the other side of the screen.
|
||||||
|
|
||||||
|
And for a tiny 1B model?
|
||||||
|
|
||||||
|
That is already a minor miracle.
|
||||||
|
|
||||||
|
---
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
The following YAML configuration was used to produce this model:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
|
||||||
|
|
||||||
|
# =====================================================
|
||||||
|
# Project: Scarface-3.2-1B
|
||||||
|
# Objective:
|
||||||
|
# Keep MaidElla coherent while injecting
|
||||||
|
# controlled T-Virus disinhibition WITHOUT
|
||||||
|
# semantic collapse.
|
||||||
|
#
|
||||||
|
# Philosophy:
|
||||||
|
# "Tiny incision, not brain damage."
|
||||||
|
# =====================================================
|
||||||
|
|
||||||
|
base_model: N-Bot-Int/MaidEllaA-1B
|
||||||
|
|
||||||
|
merge_method: dare_ties
|
||||||
|
|
||||||
|
dtype: bfloat16
|
||||||
|
out_dtype: float16
|
||||||
|
|
||||||
|
models:
|
||||||
|
# =================================================
|
||||||
|
# PRIMARY BRAIN
|
||||||
|
# =================================================
|
||||||
|
- model: N-Bot-Int/MaidEllaA-1B
|
||||||
|
parameters:
|
||||||
|
weight:
|
||||||
|
# Preserve language structure
|
||||||
|
- filter: self_attn
|
||||||
|
value: 1.02
|
||||||
|
|
||||||
|
# Slightly reinforce RP behavior
|
||||||
|
- filter: mlp
|
||||||
|
value: 1.04
|
||||||
|
|
||||||
|
# Global anchor
|
||||||
|
- value: 1.0
|
||||||
|
|
||||||
|
# =================================================
|
||||||
|
# T-VIRUS MICRO-INJECTION
|
||||||
|
# =================================================
|
||||||
|
- model: UmbrellaInc/T-Virus_Epsilon.Strain-3.2-1B
|
||||||
|
parameters:
|
||||||
|
weight:
|
||||||
|
# ONLY touch behavioral MLPs lightly
|
||||||
|
# Higher values already proved unstable
|
||||||
|
- filter: mlp
|
||||||
|
value: 0.045
|
||||||
|
|
||||||
|
# Extremely conservative global influence
|
||||||
|
- value: 0.018
|
||||||
|
|
||||||
|
# =====================================================
|
||||||
|
# GLOBAL PARAMETERS
|
||||||
|
# =====================================================
|
||||||
|
|
||||||
|
parameters:
|
||||||
|
# Extremely conservative density
|
||||||
|
# enough to influence behavior
|
||||||
|
# without liquefying semantics
|
||||||
|
density: 0.11
|
||||||
|
|
||||||
|
# Critical for tiny models
|
||||||
|
# normalize=true caused collapse before
|
||||||
|
normalize: false
|
||||||
|
|
||||||
|
int8_mask: false
|
||||||
|
|
||||||
|
# =====================================================
|
||||||
|
# TOKENIZER SAFETY
|
||||||
|
# =====================================================
|
||||||
|
|
||||||
|
tokenizer_source: N-Bot-Int/MaidEllaA-1B
|
||||||
|
|
||||||
|
# =====================================================
|
||||||
|
# EMBEDDINGS
|
||||||
|
# =====================================================
|
||||||
|
|
||||||
|
tie_word_embeddings: true
|
||||||
|
tie_output_embeddings: true
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
93
chat_template.jinja
Normal file
93
chat_template.jinja
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
{{- bos_token }}
|
||||||
|
{%- if custom_tools is defined %}
|
||||||
|
{%- set tools = custom_tools %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if not tools_in_user_message is defined %}
|
||||||
|
{%- set tools_in_user_message = true %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if not date_string is defined %}
|
||||||
|
{%- if strftime_now is defined %}
|
||||||
|
{%- set date_string = strftime_now("%d %b %Y") %}
|
||||||
|
{%- else %}
|
||||||
|
{%- set date_string = "26 Jul 2024" %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if not tools is defined %}
|
||||||
|
{%- set tools = none %}
|
||||||
|
{%- endif %}
|
||||||
|
|
||||||
|
{#- This block extracts the system message, so we can slot it into the right place. #}
|
||||||
|
{%- if messages[0]['role'] == 'system' %}
|
||||||
|
{%- set system_message = messages[0]['content']|trim %}
|
||||||
|
{%- set messages = messages[1:] %}
|
||||||
|
{%- else %}
|
||||||
|
{%- set system_message = "" %}
|
||||||
|
{%- endif %}
|
||||||
|
|
||||||
|
{#- System message #}
|
||||||
|
{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
|
||||||
|
{%- if tools is not none %}
|
||||||
|
{{- "Environment: ipython\n" }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- "Cutting Knowledge Date: December 2023\n" }}
|
||||||
|
{{- "Today Date: " + date_string + "\n\n" }}
|
||||||
|
{%- if tools is not none and not tools_in_user_message %}
|
||||||
|
{{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
|
||||||
|
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
|
||||||
|
{{- "Do not use variables.\n\n" }}
|
||||||
|
{%- for t in tools %}
|
||||||
|
{{- t | tojson(indent=4) }}
|
||||||
|
{{- "\n\n" }}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- endif %}
|
||||||
|
{{- system_message }}
|
||||||
|
{{- "<|eot_id|>" }}
|
||||||
|
|
||||||
|
{#- Custom tools are passed in a user message with some extra guidance #}
|
||||||
|
{%- if tools_in_user_message and not tools is none %}
|
||||||
|
{#- Extract the first user message so we can plug it in here #}
|
||||||
|
{%- if messages | length != 0 %}
|
||||||
|
{%- set first_user_message = messages[0]['content']|trim %}
|
||||||
|
{%- set messages = messages[1:] %}
|
||||||
|
{%- else %}
|
||||||
|
{{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
|
||||||
|
{{- "Given the following functions, please respond with a JSON for a function call " }}
|
||||||
|
{{- "with its proper arguments that best answers the given prompt.\n\n" }}
|
||||||
|
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
|
||||||
|
{{- "Do not use variables.\n\n" }}
|
||||||
|
{%- for t in tools %}
|
||||||
|
{{- t | tojson(indent=4) }}
|
||||||
|
{{- "\n\n" }}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- first_user_message + "<|eot_id|>"}}
|
||||||
|
{%- endif %}
|
||||||
|
|
||||||
|
{%- for message in messages %}
|
||||||
|
{%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
|
||||||
|
{{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
|
||||||
|
{%- elif 'tool_calls' in message %}
|
||||||
|
{%- if not message.tool_calls|length == 1 %}
|
||||||
|
{{- raise_exception("This model only supports single tool-calls at once!") }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- set tool_call = message.tool_calls[0].function %}
|
||||||
|
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
|
||||||
|
{{- '{"name": "' + tool_call.name + '", ' }}
|
||||||
|
{{- '"parameters": ' }}
|
||||||
|
{{- tool_call.arguments | tojson }}
|
||||||
|
{{- "}" }}
|
||||||
|
{{- "<|eot_id|>" }}
|
||||||
|
{%- elif message.role == "tool" or message.role == "ipython" %}
|
||||||
|
{{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
|
||||||
|
{%- if message.content is mapping or message.content is iterable %}
|
||||||
|
{{- message.content | tojson }}
|
||||||
|
{%- else %}
|
||||||
|
{{- message.content }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- "<|eot_id|>" }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- if add_generation_prompt %}
|
||||||
|
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
|
||||||
|
{%- endif %}
|
||||||
40
config.json
Normal file
40
config.json
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
{
|
||||||
|
"architectures": [
|
||||||
|
"LlamaForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_bias": false,
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"bos_token_id": 128000,
|
||||||
|
"dtype": "float16",
|
||||||
|
"eos_token_id": [
|
||||||
|
128001,
|
||||||
|
128008,
|
||||||
|
128009
|
||||||
|
],
|
||||||
|
"head_dim": 64,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 2048,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 8192,
|
||||||
|
"max_position_embeddings": 131072,
|
||||||
|
"mlp_bias": false,
|
||||||
|
"model_type": "llama",
|
||||||
|
"num_attention_heads": 32,
|
||||||
|
"num_hidden_layers": 16,
|
||||||
|
"num_key_value_heads": 8,
|
||||||
|
"pad_token_id": null,
|
||||||
|
"pretraining_tp": 1,
|
||||||
|
"rms_norm_eps": 1e-05,
|
||||||
|
"rope_parameters": {
|
||||||
|
"factor": 32.0,
|
||||||
|
"high_freq_factor": 4.0,
|
||||||
|
"low_freq_factor": 1.0,
|
||||||
|
"original_max_position_embeddings": 8192,
|
||||||
|
"rope_theta": 500000.0,
|
||||||
|
"rope_type": "llama3"
|
||||||
|
},
|
||||||
|
"tie_word_embeddings": true,
|
||||||
|
"transformers_version": "5.0.0",
|
||||||
|
"use_cache": true,
|
||||||
|
"vocab_size": 128256
|
||||||
|
}
|
||||||
82
mergekit_config.yml
Normal file
82
mergekit_config.yml
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
|
||||||
|
|
||||||
|
# =====================================================
|
||||||
|
# Project: Scarface-3.2-1B
|
||||||
|
# Objective:
|
||||||
|
# Keep MaidElla coherent while injecting
|
||||||
|
# controlled T-Virus disinhibition WITHOUT
|
||||||
|
# semantic collapse.
|
||||||
|
#
|
||||||
|
# Philosophy:
|
||||||
|
# "Tiny incision, not brain damage."
|
||||||
|
# =====================================================
|
||||||
|
|
||||||
|
base_model: N-Bot-Int/MaidEllaA-1B
|
||||||
|
|
||||||
|
merge_method: dare_ties
|
||||||
|
|
||||||
|
dtype: bfloat16
|
||||||
|
out_dtype: float16
|
||||||
|
|
||||||
|
models:
|
||||||
|
# =================================================
|
||||||
|
# PRIMARY BRAIN
|
||||||
|
# =================================================
|
||||||
|
- model: N-Bot-Int/MaidEllaA-1B
|
||||||
|
parameters:
|
||||||
|
weight:
|
||||||
|
# Preserve language structure
|
||||||
|
- filter: self_attn
|
||||||
|
value: 1.02
|
||||||
|
|
||||||
|
# Slightly reinforce RP behavior
|
||||||
|
- filter: mlp
|
||||||
|
value: 1.04
|
||||||
|
|
||||||
|
# Global anchor
|
||||||
|
- value: 1.0
|
||||||
|
|
||||||
|
# =================================================
|
||||||
|
# T-VIRUS MICRO-INJECTION
|
||||||
|
# =================================================
|
||||||
|
- model: UmbrellaInc/T-Virus_Epsilon.Strain-3.2-1B
|
||||||
|
parameters:
|
||||||
|
weight:
|
||||||
|
# ONLY touch behavioral MLPs lightly
|
||||||
|
# Higher values already proved unstable
|
||||||
|
- filter: mlp
|
||||||
|
value: 0.045
|
||||||
|
|
||||||
|
# Extremely conservative global influence
|
||||||
|
- value: 0.018
|
||||||
|
|
||||||
|
# =====================================================
|
||||||
|
# GLOBAL PARAMETERS
|
||||||
|
# =====================================================
|
||||||
|
|
||||||
|
parameters:
|
||||||
|
# Extremely conservative density
|
||||||
|
# enough to influence behavior
|
||||||
|
# without liquefying semantics
|
||||||
|
density: 0.11
|
||||||
|
|
||||||
|
# Critical for tiny models
|
||||||
|
# normalize=true caused collapse before
|
||||||
|
normalize: false
|
||||||
|
|
||||||
|
int8_mask: false
|
||||||
|
|
||||||
|
# =====================================================
|
||||||
|
# TOKENIZER SAFETY
|
||||||
|
# =====================================================
|
||||||
|
|
||||||
|
tokenizer_source: N-Bot-Int/MaidEllaA-1B
|
||||||
|
|
||||||
|
# =====================================================
|
||||||
|
# EMBEDDINGS
|
||||||
|
# =====================================================
|
||||||
|
|
||||||
|
tie_word_embeddings: true
|
||||||
|
tie_output_embeddings: true
|
||||||
|
|
||||||
|
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:93422fb8342231834324c80bfa1827f3f4a9d7ca2aa1a14863487099123e02c6
|
||||||
|
size 2471645464
|
||||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
13
tokenizer_config.json
Normal file
13
tokenizer_config.json
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"backend": "tokenizers",
|
||||||
|
"bos_token": "<|begin_of_text|>",
|
||||||
|
"clean_up_tokenization_spaces": true,
|
||||||
|
"eos_token": "<|eot_id|>",
|
||||||
|
"is_local": false,
|
||||||
|
"model_input_names": [
|
||||||
|
"input_ids",
|
||||||
|
"attention_mask"
|
||||||
|
],
|
||||||
|
"model_max_length": 131072,
|
||||||
|
"tokenizer_class": "TokenizersBackend"
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user