kyutai/helium-1-preview-2b
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: kyutai/helium-1-preview-2b
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-27 17:08:13 +08:00
commit 54390d8b99
8 changed files with 194209 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

202
README.md Normal file
View File

@@ -0,0 +1,202 @@
---
library_name: transformers
license: cc-by-4.0
language:
- en
- fr
- de
- it
- pt
- es
pipeline_tag: text-generation
---
# helium-1-preview-2b
<!-- Provide a quick summary of what the model is/does. -->
<img src="https://huggingface.co/kyutai/moshi-1-2b/resolve/main/helium_sticker.png" width="400">
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
Helium-1 preview is a lightweight language model with 2B parameters, targeting edge and mobile devices.
It supports the following languages: English, French, German, Italian, Portuguese, Spanish.
⚠️ Helium-1 Preview is a base model, which was not fine-tuned to follow instructions or human preferences.
For most downstream use cases, the model should be aligned with supervised fine-tuning, RLHF or related methods.
- **Developed by:** Kyutai
- **Model type:** Large Language Model
- **Language(s) (NLP):** English, French, German, Italian, Portuguese, Spanish
- **License:** CC-BY 4.0
<!-- ### Model Sources [optional]
Provide the basic links for the model.
- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed] -->
## Uses
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
### Direct Use
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
The intended use of the Helium model is research and development of natural language processing systems, including but not limited to language generation and understanding.
The model can be used in English, French, German, Italian, Portuguese and Spanish.
For most downstream use cases, the model should be aligned with supervised fine-tuning, RLHF or related methods.
### Out-of-Scope Use
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
The model should not be used in other languages than the ones on which it was trained.
The model is not intended to be used for any malicious or illegal activities of any kind.
The model was not fine-tuned to follow instructions, and thus should not be used as such.
## Bias, Risks, and Limitations
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
Helium-1 preview is a base language model, which was not aligned to human preferences.
As such, the model can generate incorrect, biased, harmful or generally unhelpful content.
Thus, the model should not be used for downstream applications without further alignment, evaluations and mitigations of risks.
<!-- Thus, it should not be used without further evaluations of risks and mitigations. -->
## How to Get Started with the Model
Use the code below to get started with the model.
```python
import torch
from transformers import pipeline
model_id = "kyutai/helium-1-preview-2b"
pipe = pipeline(
"text-generation",
model=model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
text = pipe("Hello, today is a great day to")
```
## Training Details
### Training Data
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
Helium-1 preview was trained on a mix of data including: Wikipedia, Stack Exchange, open-access scientific articles (from peS2o) and Common Crawl.
<!--#### Training Hyperparameters
- **Training regime:** [More Information Needed] -->
<!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
## Evaluation
<!-- This section describes the evaluation protocols and provides the results. -->
#### Testing Data
<!-- This should link to a Dataset Card if possible. -->
The model was evaluated on MMLU, TriviaQA, NaturalQuestions, ARC Easy & Challenge, Open Book QA, Common Sense QA,
Physical Interaction QA, Social Interaction QA, HellaSwag, WinoGrande, Multilingual Knowledge QA, FLORES 200.
#### Metrics
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
We report accuracy on MMLU, ARC, OBQA, CSQA, PIQA, SIQA, HellaSwag, WinoGrande.
We report exact match on TriviaQA, NQ and MKQA.
We report BLEU on FLORES.
#### English Results
| Benchmark | Helium-1 Preview | HF SmolLM2 (1.7B) | Gemma-2 (2.6B) | Llama-3.2 (3B) | Qwen2.5 (1.5B) |
|--------------|:------:|:------:|:------:|:------:|:------:|
| | | | | | |
| MMLU | 51.2 | 50.4 | 53.1 | 56.6 | 61.0 |
| NQ | 17.3 | 15.1 | 17.7 | 22.0 | 13.1 |
| TQA | 47.9 | 45.4 | 49.9 | 53.6 | 35.9 |
| ARC E | 80.9 | 81.8 | 81.1 | 84.6 | 89.7 |
| ARC C | 62.7 | 64.7 | 66.0 | 69.0 | 77.2 |
| OBQA | 63.8 | 61.4 | 64.6 | 68.4 | 73.8 |
| CSQA | 65.6 | 59.0 | 64.4 | 65.4 | 72.4 |
| PIQA | 77.4 | 77.7 | 79.8 | 78.9 | 76.0 |
| SIQA | 64.4 | 57.5 | 61.9 | 63.8 | 68.7 |
| HS | 69.7 | 73.2 | 74.7 | 76.9 | 67.5 |
| WG | 66.5 | 65.6 | 71.2 | 72.0 | 64.8 |
| | | | | | |
| Average | 60.7 | 59.3 | 62.2 | 64.7 | 63.6 |
#### Multilingual Results
| Language | Benchmark | Helium-1 Preview | HF SmolLM2 (1.7B) | Gemma-2 (2.6B) | Llama-3.2 (3B) | Qwen2.5 (1.5B) |
|-----|--------------|:------:|:------:|:------:|:------:|:------:|
| | | | | | | |
| German | MMLU | 45.6 | 35.3 | 45.0 | 47.5 | 49.5 |
| | ARC C | 56.7 | 38.4 | 54.7 | 58.3 | 60.2 |
| | HS | 53.5 | 33.9 | 53.4 | 53.7 | 42.8 |
| | MKQA | 16.1 | 7.1 | 18.9 | 20.2 | 10.4 |
| | FLORES | 33.9 | 12.2 | 30.7 | 28.2 | 20.8 |
| Spanish | MMLU | 46.5 | 38.9 | 46.2 | 49.6 | 52.8 |
| | ARC C | 58.3 | 43.2 | 58.8 | 60.0 | 68.1 |
| | HS | 58.6 | 40.8 | 60.5 | 61.1 | 51.4 |
| | MKQA | 16.0 | 7.9 | 18.5 | 20.6 | 10.6 |
| | FLORES | 25.7 | 15.0 | 25.7 | 23.7 | 20.4 |
| French | MMLU | 46.0 | 37.7 | 45.7 | 48.8 | 51.9 |
| | ARC C | 57.9 | 40.6 | 57.5 | 60.1 | 67.4 |
| | HS | 59.0 | 41.1 | 60.4 | 59.6 | 51.2 |
| | MKQA | 16.8 | 8.4 | 18.4 | 19.6 | 9.7 |
| | FLORES | 44.3 | 20.0 | 43.3 | 39.3 | 31.2 |
| Italian | MMLU | 46.1 | 36.3 | 45.6 | 48.8 | 50.5 |
| | ARC C | 57.4 | 39.1 | 53.9 | 60.1 | 64.6 |
| | HS | 55.2 | 37.7 | 56.2 | 56.8 | 46.8 |
| | MKQA | 15.3 | 6.3 | 18.0 | 19.0 | 9.9 |
| | FLORES | 25.8 | 10.4 | 25.2 | 23.8 | 16.4 |
| Portuguese | MMLU | 46.2 | 37.7 | 45.6 | 49.2 | 53.0 |
| | ARC C | 56.8 | 40.6 | 57.0 | 62.1 | 66.6 |
| | HS | 57.3 | 41.0 | 58.7 | 59.1 | 50.9 |
| | MKQA | 14.7 | 6.6 | 16.9 | 19.1 | 9.2 |
| | FLORES | 43.0 | 20.0 | 43.6 | 40.5 | 33.0 |
| | | | | | | |
| | Average | 42.1 | 27.8 | 42.3 | 43.6 | 40.0
## Technical Specifications
### Model Architecture and Objective
| Hyperparameter | Value |
|--------------|:------:|
| Layers | 24 |
| Heads | 20 |
| Model dimension | 2560 |
| MLP dimension | 7040 |
| Context size | 4096 |
| Theta RoPE | 100,000 |
#### Hardware
The model was trained on 128 NVIDIA H100 Tensor Core GPUs.
#### Software
The model was trained using Jax.
## Citation
Blog post: https://kyutai.org/2025/01/13/helium.html

28
config.json Normal file
View File

@@ -0,0 +1,28 @@
{
"architectures": [
"HeliumForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 1,
"eos_token_id": 2,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 2560,
"initializer_range": 0.02,
"intermediate_size": 7040,
"max_position_embeddings": 4096,
"mlp_bias": false,
"model_type": "helium",
"num_attention_heads": 20,
"num_hidden_layers": 24,
"num_key_value_heads": 20,
"pretraining_tp": 1,
"rms_norm_eps": 1e-08,
"rope_theta": 100000.0,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.45.0.dev0",
"use_cache": true,
"vocab_size": 48000
}

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6f3424334c6989857d8f0bac20e029f8ef6239bab3bc900ecdf5d26d786dd415
size 4345313008

1
special_tokens_map.json Normal file
View File

@@ -0,0 +1 @@
{}

193082
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

857
tokenizer_config.json Normal file
View File

@@ -0,0 +1,857 @@
{
"added_tokens_decoder": {
"1": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": true
},
"2": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": true
},
"3": {
"content": "<pad>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": true
},
"4": {
"content": "<|im_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"5": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"6": {
"content": "<|im_sp_00|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"7": {
"content": "<|im_sp_01|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"8": {
"content": "<|im_sp_02|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"9": {
"content": "<|im_sp_03|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"10": {
"content": "<|im_sp_04|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"11": {
"content": "<|im_sp_05|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"12": {
"content": "<|im_sp_06|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"13": {
"content": "<|im_sp_07|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"14": {
"content": "<|im_sp_08|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"15": {
"content": "<|im_sp_09|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"16": {
"content": "<|im_sp_10|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"17": {
"content": "<|im_sp_11|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"18": {
"content": "<|im_sp_12|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"19": {
"content": "<|im_sp_13|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"20": {
"content": "<|im_sp_14|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"21": {
"content": "<|im_sp_15|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"22": {
"content": "<|im_sp_16|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"23": {
"content": "<|im_sp_17|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"24": {
"content": "<|im_sp_18|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"25": {
"content": "<|im_sp_19|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"26": {
"content": "<|im_sp_20|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"27": {
"content": "<|im_sp_21|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"28": {
"content": "<|im_sp_22|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"29": {
"content": "<|im_sp_23|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"30": {
"content": "<|im_sp_24|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"31": {
"content": "<|im_sp_25|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"32": {
"content": "<|im_sp_26|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"33": {
"content": "<|im_sp_27|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"34": {
"content": "<|im_sp_28|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"35": {
"content": "<|im_sp_29|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"36": {
"content": "<|im_sp_30|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"37": {
"content": "<|im_sp_31|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"38": {
"content": "<|im_sp_32|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"39": {
"content": "<|im_sp_33|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"40": {
"content": "<|im_sp_34|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"41": {
"content": "<|im_sp_35|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"42": {
"content": "<|im_sp_36|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"43": {
"content": "<|im_sp_37|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"44": {
"content": "<|im_sp_38|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"45": {
"content": "<|im_sp_39|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"46": {
"content": "<|im_sp_40|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"47": {
"content": "<|im_sp_41|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"48": {
"content": "<|im_sp_42|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"49": {
"content": "<|im_sp_43|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"50": {
"content": "<|im_sp_44|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"51": {
"content": "<|im_sp_45|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"52": {
"content": "<|im_sp_46|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"53": {
"content": "<|im_sp_47|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"54": {
"content": "<|im_sp_48|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"55": {
"content": "<|im_sp_49|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"56": {
"content": "<|im_sp_50|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"57": {
"content": "<|im_sp_51|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"58": {
"content": "<|im_sp_52|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"59": {
"content": "<|im_sp_53|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"60": {
"content": "<|im_sp_54|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"61": {
"content": "<|im_sp_55|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"62": {
"content": "<|im_sp_56|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"63": {
"content": "<|im_sp_57|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"64": {
"content": "<|im_sp_58|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"65": {
"content": "<|im_sp_59|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"66": {
"content": "<|im_sp_60|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"67": {
"content": "<|im_sp_61|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"68": {
"content": "<|im_sp_62|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"69": {
"content": "<|im_sp_63|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"70": {
"content": "<|im_sp_64|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"71": {
"content": "<|im_sp_65|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"72": {
"content": "<|im_sp_66|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"73": {
"content": "<|im_sp_67|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"74": {
"content": "<|im_sp_68|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"75": {
"content": "<|im_sp_69|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"76": {
"content": "<|im_sp_70|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"77": {
"content": "<|im_sp_71|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"78": {
"content": "<|im_sp_72|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"79": {
"content": "<|im_sp_73|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"80": {
"content": "<|im_sp_74|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"81": {
"content": "<|im_sp_75|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"82": {
"content": "<|im_sp_76|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"83": {
"content": "<|im_sp_77|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"84": {
"content": "<|im_sp_78|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"85": {
"content": "<|im_sp_79|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"86": {
"content": "<|im_sp_80|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"87": {
"content": "<|im_sp_81|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"88": {
"content": "<|im_sp_82|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"89": {
"content": "<|im_sp_83|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"90": {
"content": "<|im_sp_84|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"91": {
"content": "<|im_sp_85|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"92": {
"content": "<|im_sp_86|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"93": {
"content": "<|im_sp_87|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"94": {
"content": "<|im_sp_88|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"95": {
"content": "<|im_sp_89|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"96": {
"content": "<|im_sp_90|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"97": {
"content": "<|im_sp_91|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"98": {
"content": "<|im_sp_92|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"99": {
"content": "<|im_sp_93|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"100": {
"content": "<|im_sp_94|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"101": {
"content": "<|im_sp_95|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"102": {
"content": "<|im_sp_96|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"103": {
"content": "<|im_sp_97|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"104": {
"content": "<|im_sp_98|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"105": {
"content": "<|im_sp_99|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": true,
"special": false
},
"116": {
"content": "\n",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
}
},
"clean_up_tokenization_spaces": false,
"extra_special_tokens": {},
"model_max_length": 1000000000000000019884624838656,
"tokenizer_class": "PreTrainedTokenizerFast",
"model_input_names": ["input_ids", "attention_mask"]
}