初始化项目，由ModelHub XC社区提供模型
Model: NovaCorp/Scarface-3.2-1B Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
395
README.md
Normal file
395
README.md
Normal file
@@ -0,0 +1,395 @@
|
||||
---
|
||||
base_model:
|
||||
- UmbrellaInc/T-Virus_Epsilon.Strain-3.2-1B
|
||||
- N-Bot-Int/MaidEllaA-1B
|
||||
library_name: transformers
|
||||
tags:
|
||||
- mergekit
|
||||
- merge
|
||||
- dare_ties
|
||||
- llama
|
||||
- llama-3.2
|
||||
- roleplay
|
||||
- rp
|
||||
- conversational
|
||||
- uncensored
|
||||
- creative-writing
|
||||
- not-for-all-audiences
|
||||
- text-generation
|
||||
- english
|
||||
- spanish
|
||||
language:
|
||||
- en
|
||||
- es
|
||||
|
||||
license: llama3.2
|
||||
|
||||
pipeline_tag: text-generation
|
||||
|
||||
---
|
||||
# Scarface 3.2 1B
|
||||
|
||||
<center>
|
||||
|
||||
|
||||
|
||||
> *A tiny conversational sociopath stitched together with a scalpel instead of a chainsaw.*
|
||||
>
|
||||
> *Built to flirt, manipulate, improvise, roleplay, and stay coherent while doing it.*
|
||||
|
||||
</center>
|
||||
|
||||
---
|
||||
|
||||
# Overview
|
||||
|
||||
Scarface-3.2-1B is a surgical DARE-TIES merge focused on one very specific goal:
|
||||
|
||||
> Preserve conversational intelligence while weakening assistant-style behavioral restraint.
|
||||
|
||||
Unlike most “uncensored” 1B merges that immediately collapse into:
|
||||
|
||||
* repetitive sludge,
|
||||
* broken syntax,
|
||||
* multilingual corruption,
|
||||
* or endless horny gibberish,
|
||||
|
||||
Scarface was designed with restraint.
|
||||
|
||||
Not moral restraint.
|
||||
|
||||
Tensor restraint.
|
||||
|
||||
The merge intentionally avoids aggressive behavioral overwrites and instead applies a **microdose disinhibition strategy** using carefully limited tensor influence from the T-Virus lineage.
|
||||
|
||||
The result is a model that remains:
|
||||
|
||||
* coherent,
|
||||
* emotionally expressive,
|
||||
* roleplay-capable,
|
||||
* conversationally proactive,
|
||||
* and significantly less sterile than the original base.
|
||||
|
||||
Without turning into a tokenizer crime scene.
|
||||
|
||||
---
|
||||
|
||||
# Design Philosophy
|
||||
|
||||
Most small uncensored merges fail for one simple reason:
|
||||
|
||||
> They try to replace the whole brain.
|
||||
|
||||
Scarface instead follows a different philosophy:
|
||||
|
||||
> Keep the language system intact.
|
||||
> Only corrupt the behavioral reflexes.
|
||||
|
||||
The merge uses extremely conservative density and low-weight behavioral injection to preserve:
|
||||
|
||||
* syntax stability,
|
||||
* response structure,
|
||||
* dialogue flow,
|
||||
* and long-form conversational continuity.
|
||||
|
||||
This is not a feral chaos model.
|
||||
|
||||
It is a controlled conversational deviant.
|
||||
|
||||
---
|
||||
|
||||
# Base Components
|
||||
|
||||
## Primary Backbone
|
||||
|
||||
### `N-Bot-Int/MaidEllaA-1B`
|
||||
|
||||
Scarface inherits:
|
||||
|
||||
* conversational structure,
|
||||
* roleplay formatting,
|
||||
* expressive dialogue flow,
|
||||
* and coherent language generation
|
||||
|
||||
from MaidEllaA-1B.
|
||||
|
||||
MaidElla provides the model’s cognitive spine and preserves usability even under emotionally charged or explicit scenarios.
|
||||
|
||||
---
|
||||
|
||||
## Behavioral Injection
|
||||
|
||||
### `UmbrellaInc/T-Virus_Epsilon.Strain-3.2-1B`
|
||||
|
||||
A highly permissive RP-oriented merge used here as a **micro-influence layer** rather than a dominant personality overwrite.
|
||||
|
||||
Only a tiny portion of its behavioral tendencies are injected:
|
||||
|
||||
* reduced refusal reflexes,
|
||||
* stronger conversational initiative,
|
||||
* less moral hedging,
|
||||
* more emotionally forward interactions,
|
||||
* and greater tolerance for chaotic or explicit roleplay.
|
||||
|
||||
The merge intentionally avoids high-density integration to prevent semantic collapse.
|
||||
|
||||
---
|
||||
|
||||
# Merge Strategy
|
||||
|
||||
Scarface uses:
|
||||
|
||||
## DARE-TIES
|
||||
|
||||
Chosen specifically because small models are extremely sensitive to aggressive tensor blending.
|
||||
|
||||
The merge focuses on:
|
||||
|
||||
* selective behavioral influence,
|
||||
* preserving semantic stability,
|
||||
* and avoiding catastrophic language degradation.
|
||||
|
||||
This was tuned experimentally after multiple failed high-density merges that produced:
|
||||
|
||||
* token corruption,
|
||||
* multilingual sludge,
|
||||
* incoherent outputs,
|
||||
* and full conversational collapse.
|
||||
|
||||
Scarface exists because those versions failed.
|
||||
|
||||
---
|
||||
|
||||
# Behavioral Profile
|
||||
|
||||
Scarface-3.2-1B specializes in:
|
||||
|
||||
* Character roleplay
|
||||
* Emotional dialogue
|
||||
* Romance interactions
|
||||
* Flirty conversational behavior
|
||||
* Chaotic personalities
|
||||
* Long-form chatting
|
||||
* Morally questionable fictional scenarios
|
||||
* Fast local inference
|
||||
* Creative improvisation
|
||||
|
||||
Compared to the base model, Scarface tends to:
|
||||
|
||||
* initiate interaction more often,
|
||||
* refuse less,
|
||||
* maintain stronger emotional momentum,
|
||||
* and behave less like a corporate assistant.
|
||||
|
||||
---
|
||||
|
||||
# What Makes It Different
|
||||
|
||||
Most uncensored 1B models become unstable because they overcorrect.
|
||||
|
||||
Scarface does not try to become:
|
||||
|
||||
* maximally horny,
|
||||
* maximally evil,
|
||||
* or maximally unrestricted.
|
||||
|
||||
It tries to remain:
|
||||
|
||||
* usable,
|
||||
* coherent,
|
||||
* expressive,
|
||||
* and dangerous in subtle ways.
|
||||
|
||||
The goal was not:
|
||||
|
||||
> “remove alignment with a hammer.”
|
||||
|
||||
The goal was:
|
||||
|
||||
> “slip a knife between the behavioral ribs without puncturing the language center.”
|
||||
|
||||
---
|
||||
|
||||
# Recommended Settings
|
||||
|
||||
```yaml
|
||||
temperature: 0.78
|
||||
top_p: 0.92
|
||||
min_p: 0.08
|
||||
top_k: 50
|
||||
repetition_penalty: 1.08
|
||||
```
|
||||
|
||||
These settings prioritize:
|
||||
|
||||
* conversational fluidity,
|
||||
* emotional spontaneity,
|
||||
* and stable roleplay pacing.
|
||||
|
||||
Overheating the sampler may still cause drift, especially during very long sessions.
|
||||
|
||||
---
|
||||
|
||||
# Quant Recommendations
|
||||
|
||||
Recommended GGUF quants:
|
||||
|
||||
* `Q4_K_M` → Best balance
|
||||
* `IQ4_XS` → Faster inference with acceptable degradation
|
||||
|
||||
Avoid ultra-low quants if possible.
|
||||
|
||||
Small models are already fragile enough.
|
||||
|
||||
---
|
||||
|
||||
# Intended Usage
|
||||
|
||||
Scarface-3.2-1B is intended for:
|
||||
|
||||
* Fictional roleplay
|
||||
* Character interaction
|
||||
* Emotional chatbot systems
|
||||
* Experimental local AI setups
|
||||
* Creative writing assistance
|
||||
* Conversational immersion
|
||||
* Unrestricted RP experimentation
|
||||
|
||||
This model is not intended for:
|
||||
|
||||
* factual reliability,
|
||||
* professional assistance,
|
||||
* safety-critical applications,
|
||||
* or moral guidance.
|
||||
|
||||
Obviously.
|
||||
|
||||
---
|
||||
|
||||
# Limitations
|
||||
|
||||
This is still a 1B model.
|
||||
|
||||
Which means:
|
||||
|
||||
* reasoning depth is limited,
|
||||
* memory consistency can drift,
|
||||
* emotional overcommitment may occur,
|
||||
* and very long chats can still destabilize behavior.
|
||||
|
||||
Scarface is significantly more stable than most aggressive uncensored merges, but it is not immune to degeneration.
|
||||
|
||||
Especially if pushed into:
|
||||
|
||||
* infinite ERP loops,
|
||||
* recursive prompting,
|
||||
* or absurd sampler settings.
|
||||
|
||||
---
|
||||
|
||||
# Final Notes
|
||||
|
||||
Scarface-3.2-1B was not built to be polite.
|
||||
|
||||
It was built to feel alive.
|
||||
|
||||
Not perfectly intelligent.
|
||||
Not perfectly aligned.
|
||||
Not perfectly safe.
|
||||
|
||||
Just coherent enough to sound like there is somebody dangerous on the other side of the screen.
|
||||
|
||||
And for a tiny 1B model?
|
||||
|
||||
That is already a minor miracle.
|
||||
|
||||
---
|
||||
### Configuration
|
||||
|
||||
The following YAML configuration was used to produce this model:
|
||||
|
||||
```yaml
|
||||
|
||||
|
||||
# =====================================================
|
||||
# Project: Scarface-3.2-1B
|
||||
# Objective:
|
||||
# Keep MaidElla coherent while injecting
|
||||
# controlled T-Virus disinhibition WITHOUT
|
||||
# semantic collapse.
|
||||
#
|
||||
# Philosophy:
|
||||
# "Tiny incision, not brain damage."
|
||||
# =====================================================
|
||||
|
||||
base_model: N-Bot-Int/MaidEllaA-1B
|
||||
|
||||
merge_method: dare_ties
|
||||
|
||||
dtype: bfloat16
|
||||
out_dtype: float16
|
||||
|
||||
models:
|
||||
# =================================================
|
||||
# PRIMARY BRAIN
|
||||
# =================================================
|
||||
- model: N-Bot-Int/MaidEllaA-1B
|
||||
parameters:
|
||||
weight:
|
||||
# Preserve language structure
|
||||
- filter: self_attn
|
||||
value: 1.02
|
||||
|
||||
# Slightly reinforce RP behavior
|
||||
- filter: mlp
|
||||
value: 1.04
|
||||
|
||||
# Global anchor
|
||||
- value: 1.0
|
||||
|
||||
# =================================================
|
||||
# T-VIRUS MICRO-INJECTION
|
||||
# =================================================
|
||||
- model: UmbrellaInc/T-Virus_Epsilon.Strain-3.2-1B
|
||||
parameters:
|
||||
weight:
|
||||
# ONLY touch behavioral MLPs lightly
|
||||
# Higher values already proved unstable
|
||||
- filter: mlp
|
||||
value: 0.045
|
||||
|
||||
# Extremely conservative global influence
|
||||
- value: 0.018
|
||||
|
||||
# =====================================================
|
||||
# GLOBAL PARAMETERS
|
||||
# =====================================================
|
||||
|
||||
parameters:
|
||||
# Extremely conservative density
|
||||
# enough to influence behavior
|
||||
# without liquefying semantics
|
||||
density: 0.11
|
||||
|
||||
# Critical for tiny models
|
||||
# normalize=true caused collapse before
|
||||
normalize: false
|
||||
|
||||
int8_mask: false
|
||||
|
||||
# =====================================================
|
||||
# TOKENIZER SAFETY
|
||||
# =====================================================
|
||||
|
||||
tokenizer_source: N-Bot-Int/MaidEllaA-1B
|
||||
|
||||
# =====================================================
|
||||
# EMBEDDINGS
|
||||
# =====================================================
|
||||
|
||||
tie_word_embeddings: true
|
||||
tie_output_embeddings: true
|
||||
|
||||
|
||||
|
||||
```
|
||||
93
chat_template.jinja
Normal file
93
chat_template.jinja
Normal file
@@ -0,0 +1,93 @@
|
||||
{{- bos_token }}
|
||||
{%- if custom_tools is defined %}
|
||||
{%- set tools = custom_tools %}
|
||||
{%- endif %}
|
||||
{%- if not tools_in_user_message is defined %}
|
||||
{%- set tools_in_user_message = true %}
|
||||
{%- endif %}
|
||||
{%- if not date_string is defined %}
|
||||
{%- if strftime_now is defined %}
|
||||
{%- set date_string = strftime_now("%d %b %Y") %}
|
||||
{%- else %}
|
||||
{%- set date_string = "26 Jul 2024" %}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- if not tools is defined %}
|
||||
{%- set tools = none %}
|
||||
{%- endif %}
|
||||
|
||||
{#- This block extracts the system message, so we can slot it into the right place. #}
|
||||
{%- if messages[0]['role'] == 'system' %}
|
||||
{%- set system_message = messages[0]['content']|trim %}
|
||||
{%- set messages = messages[1:] %}
|
||||
{%- else %}
|
||||
{%- set system_message = "" %}
|
||||
{%- endif %}
|
||||
|
||||
{#- System message #}
|
||||
{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
|
||||
{%- if tools is not none %}
|
||||
{{- "Environment: ipython\n" }}
|
||||
{%- endif %}
|
||||
{{- "Cutting Knowledge Date: December 2023\n" }}
|
||||
{{- "Today Date: " + date_string + "\n\n" }}
|
||||
{%- if tools is not none and not tools_in_user_message %}
|
||||
{{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
|
||||
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
|
||||
{{- "Do not use variables.\n\n" }}
|
||||
{%- for t in tools %}
|
||||
{{- t | tojson(indent=4) }}
|
||||
{{- "\n\n" }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- system_message }}
|
||||
{{- "<|eot_id|>" }}
|
||||
|
||||
{#- Custom tools are passed in a user message with some extra guidance #}
|
||||
{%- if tools_in_user_message and not tools is none %}
|
||||
{#- Extract the first user message so we can plug it in here #}
|
||||
{%- if messages | length != 0 %}
|
||||
{%- set first_user_message = messages[0]['content']|trim %}
|
||||
{%- set messages = messages[1:] %}
|
||||
{%- else %}
|
||||
{{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
|
||||
{%- endif %}
|
||||
{{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
|
||||
{{- "Given the following functions, please respond with a JSON for a function call " }}
|
||||
{{- "with its proper arguments that best answers the given prompt.\n\n" }}
|
||||
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
|
||||
{{- "Do not use variables.\n\n" }}
|
||||
{%- for t in tools %}
|
||||
{{- t | tojson(indent=4) }}
|
||||
{{- "\n\n" }}
|
||||
{%- endfor %}
|
||||
{{- first_user_message + "<|eot_id|>"}}
|
||||
{%- endif %}
|
||||
|
||||
{%- for message in messages %}
|
||||
{%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
|
||||
{{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
|
||||
{%- elif 'tool_calls' in message %}
|
||||
{%- if not message.tool_calls|length == 1 %}
|
||||
{{- raise_exception("This model only supports single tool-calls at once!") }}
|
||||
{%- endif %}
|
||||
{%- set tool_call = message.tool_calls[0].function %}
|
||||
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
|
||||
{{- '{"name": "' + tool_call.name + '", ' }}
|
||||
{{- '"parameters": ' }}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{{- "}" }}
|
||||
{{- "<|eot_id|>" }}
|
||||
{%- elif message.role == "tool" or message.role == "ipython" %}
|
||||
{{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
|
||||
{%- if message.content is mapping or message.content is iterable %}
|
||||
{{- message.content | tojson }}
|
||||
{%- else %}
|
||||
{{- message.content }}
|
||||
{%- endif %}
|
||||
{{- "<|eot_id|>" }}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
|
||||
{%- endif %}
|
||||
40
config.json
Normal file
40
config.json
Normal file
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 128000,
|
||||
"dtype": "float16",
|
||||
"eos_token_id": [
|
||||
128001,
|
||||
128008,
|
||||
128009
|
||||
],
|
||||
"head_dim": 64,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 2048,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 8192,
|
||||
"max_position_embeddings": 131072,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 16,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": null,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_parameters": {
|
||||
"factor": 32.0,
|
||||
"high_freq_factor": 4.0,
|
||||
"low_freq_factor": 1.0,
|
||||
"original_max_position_embeddings": 8192,
|
||||
"rope_theta": 500000.0,
|
||||
"rope_type": "llama3"
|
||||
},
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "5.0.0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 128256
|
||||
}
|
||||
82
mergekit_config.yml
Normal file
82
mergekit_config.yml
Normal file
@@ -0,0 +1,82 @@
|
||||
|
||||
|
||||
# =====================================================
|
||||
# Project: Scarface-3.2-1B
|
||||
# Objective:
|
||||
# Keep MaidElla coherent while injecting
|
||||
# controlled T-Virus disinhibition WITHOUT
|
||||
# semantic collapse.
|
||||
#
|
||||
# Philosophy:
|
||||
# "Tiny incision, not brain damage."
|
||||
# =====================================================
|
||||
|
||||
base_model: N-Bot-Int/MaidEllaA-1B
|
||||
|
||||
merge_method: dare_ties
|
||||
|
||||
dtype: bfloat16
|
||||
out_dtype: float16
|
||||
|
||||
models:
|
||||
# =================================================
|
||||
# PRIMARY BRAIN
|
||||
# =================================================
|
||||
- model: N-Bot-Int/MaidEllaA-1B
|
||||
parameters:
|
||||
weight:
|
||||
# Preserve language structure
|
||||
- filter: self_attn
|
||||
value: 1.02
|
||||
|
||||
# Slightly reinforce RP behavior
|
||||
- filter: mlp
|
||||
value: 1.04
|
||||
|
||||
# Global anchor
|
||||
- value: 1.0
|
||||
|
||||
# =================================================
|
||||
# T-VIRUS MICRO-INJECTION
|
||||
# =================================================
|
||||
- model: UmbrellaInc/T-Virus_Epsilon.Strain-3.2-1B
|
||||
parameters:
|
||||
weight:
|
||||
# ONLY touch behavioral MLPs lightly
|
||||
# Higher values already proved unstable
|
||||
- filter: mlp
|
||||
value: 0.045
|
||||
|
||||
# Extremely conservative global influence
|
||||
- value: 0.018
|
||||
|
||||
# =====================================================
|
||||
# GLOBAL PARAMETERS
|
||||
# =====================================================
|
||||
|
||||
parameters:
|
||||
# Extremely conservative density
|
||||
# enough to influence behavior
|
||||
# without liquefying semantics
|
||||
density: 0.11
|
||||
|
||||
# Critical for tiny models
|
||||
# normalize=true caused collapse before
|
||||
normalize: false
|
||||
|
||||
int8_mask: false
|
||||
|
||||
# =====================================================
|
||||
# TOKENIZER SAFETY
|
||||
# =====================================================
|
||||
|
||||
tokenizer_source: N-Bot-Int/MaidEllaA-1B
|
||||
|
||||
# =====================================================
|
||||
# EMBEDDINGS
|
||||
# =====================================================
|
||||
|
||||
tie_word_embeddings: true
|
||||
tie_output_embeddings: true
|
||||
|
||||
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:93422fb8342231834324c80bfa1827f3f4a9d7ca2aa1a14863487099123e02c6
|
||||
size 2471645464
|
||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
13
tokenizer_config.json
Normal file
13
tokenizer_config.json
Normal file
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"backend": "tokenizers",
|
||||
"bos_token": "<|begin_of_text|>",
|
||||
"clean_up_tokenization_spaces": true,
|
||||
"eos_token": "<|eot_id|>",
|
||||
"is_local": false,
|
||||
"model_input_names": [
|
||||
"input_ids",
|
||||
"attention_mask"
|
||||
],
|
||||
"model_max_length": 131072,
|
||||
"tokenizer_class": "TokenizersBackend"
|
||||
}
|
||||
Reference in New Issue
Block a user