gvadhul/byte-llama-2layer
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: gvadhul/byte-llama-2layer
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-10-29 05:04:16 +08:00
commit 58778019c4
7 changed files with 603 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

199
README.md Normal file
View File

@@ -0,0 +1,199 @@
---
library_name: transformers
tags: []
---
# Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- **Developed by:** [More Information Needed]
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** [More Information Needed]
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** [More Information Needed]
### Model Sources [optional]
<!-- Provide the basic links for the model. -->
- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]
## Uses
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
### Direct Use
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
[More Information Needed]
### Downstream Use [optional]
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
[More Information Needed]
### Out-of-Scope Use
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
[More Information Needed]
## Bias, Risks, and Limitations
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
[More Information Needed]
### Recommendations
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
## How to Get Started with the Model
Use the code below to get started with the model.
[More Information Needed]
## Training Details
### Training Data
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
[More Information Needed]
### Training Procedure
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
#### Preprocessing [optional]
[More Information Needed]
#### Training Hyperparameters
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
#### Speeds, Sizes, Times [optional]
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
[More Information Needed]
## Evaluation
<!-- This section describes the evaluation protocols and provides the results. -->
### Testing Data, Factors & Metrics
#### Testing Data
<!-- This should link to a Dataset Card if possible. -->
[More Information Needed]
#### Factors
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
[More Information Needed]
#### Metrics
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
[More Information Needed]
### Results
[More Information Needed]
#### Summary
## Model Examination [optional]
<!-- Relevant interpretability work for the model goes here -->
[More Information Needed]
## Environmental Impact
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
- **Hardware Type:** [More Information Needed]
- **Hours used:** [More Information Needed]
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]
## Technical Specifications [optional]
### Model Architecture and Objective
[More Information Needed]
### Compute Infrastructure
[More Information Needed]
#### Hardware
[More Information Needed]
#### Software
[More Information Needed]
## Citation [optional]
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
**BibTeX:**
[More Information Needed]
**APA:**
[More Information Needed]
## Glossary [optional]
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
[More Information Needed]
## More Information [optional]
[More Information Needed]
## Model Card Authors [optional]
[More Information Needed]
## Model Card Contact
[More Information Needed]

32
config.json Normal file
View File

@@ -0,0 +1,32 @@
{
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 2,
"dtype": "float32",
"eos_token_id": 3,
"head_dim": 64,
"hidden_act": "silu",
"hidden_size": 256,
"initializer_range": 0.02,
"intermediate_size": 1024,
"max_position_embeddings": 2048,
"mlp_bias": false,
"model_type": "llama",
"num_attention_heads": 4,
"num_hidden_layers": 2,
"num_key_value_heads": 4,
"pad_token_id": 0,
"pretraining_tp": 1,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 10000.0,
"rope_type": "default"
},
"tie_word_embeddings": false,
"transformers_version": "5.12.1",
"use_cache": true,
"vocab_size": 256
}

10
generation_config.json Normal file
View File

@@ -0,0 +1,10 @@
{
"_from_model_config": true,
"bos_token_id": 2,
"eos_token_id": 3,
"output_attentions": false,
"output_hidden_states": false,
"pad_token_id": 0,
"transformers_version": "5.12.1",
"use_cache": true
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eadb941712bfa968cd1fd81ca78789a728e3096680342d14e035bd93350d1814
size 8920232

314
tokenizer.json Normal file
View File

@@ -0,0 +1,314 @@
{
"version": "1.0",
"truncation": null,
"padding": null,
"added_tokens": [],
"normalizer": null,
"pre_tokenizer": {
"type": "ByteLevel",
"add_prefix_space": false,
"trim_offsets": true,
"use_regex": false
},
"post_processor": {
"type": "TemplateProcessing",
"single": [
{
"Sequence": {
"id": "A",
"type_id": 0
}
}
],
"pair": [
{
"Sequence": {
"id": "A",
"type_id": 0
}
},
{
"Sequence": {
"id": "B",
"type_id": 1
}
}
],
"special_tokens": {}
},
"decoder": {
"type": "ByteLevel",
"add_prefix_space": true,
"trim_offsets": true,
"use_regex": true
},
"model": {
"type": "BPE",
"dropout": null,
"unk_token": null,
"continuing_subword_prefix": null,
"end_of_word_suffix": null,
"fuse_unk": false,
"byte_fallback": false,
"ignore_merges": false,
"vocab": {
"Ā": 0,
"ā": 1,
"Ă": 2,
"ă": 3,
"Ą": 4,
"ą": 5,
"Ć": 6,
"ć": 7,
"Ĉ": 8,
"ĉ": 9,
"Ċ": 10,
"ċ": 11,
"Č": 12,
"č": 13,
"Ď": 14,
"ď": 15,
"Đ": 16,
"đ": 17,
"Ē": 18,
"ē": 19,
"Ĕ": 20,
"ĕ": 21,
"Ė": 22,
"ė": 23,
"Ę": 24,
"ę": 25,
"Ě": 26,
"ě": 27,
"Ĝ": 28,
"ĝ": 29,
"Ğ": 30,
"ğ": 31,
"Ġ": 32,
"!": 33,
"\"": 34,
"#": 35,
"$": 36,
"%": 37,
"&": 38,
"'": 39,
"(": 40,
")": 41,
"*": 42,
"+": 43,
",": 44,
"-": 45,
".": 46,
"/": 47,
"0": 48,
"1": 49,
"2": 50,
"3": 51,
"4": 52,
"5": 53,
"6": 54,
"7": 55,
"8": 56,
"9": 57,
":": 58,
";": 59,
"<": 60,
"=": 61,
">": 62,
"?": 63,
"@": 64,
"A": 65,
"B": 66,
"C": 67,
"D": 68,
"E": 69,
"F": 70,
"G": 71,
"H": 72,
"I": 73,
"J": 74,
"K": 75,
"L": 76,
"M": 77,
"N": 78,
"O": 79,
"P": 80,
"Q": 81,
"R": 82,
"S": 83,
"T": 84,
"U": 85,
"V": 86,
"W": 87,
"X": 88,
"Y": 89,
"Z": 90,
"[": 91,
"\\": 92,
"]": 93,
"^": 94,
"_": 95,
"`": 96,
"a": 97,
"b": 98,
"c": 99,
"d": 100,
"e": 101,
"f": 102,
"g": 103,
"h": 104,
"i": 105,
"j": 106,
"k": 107,
"l": 108,
"m": 109,
"n": 110,
"o": 111,
"p": 112,
"q": 113,
"r": 114,
"s": 115,
"t": 116,
"u": 117,
"v": 118,
"w": 119,
"x": 120,
"y": 121,
"z": 122,
"{": 123,
"|": 124,
"}": 125,
"~": 126,
"ġ": 127,
"Ģ": 128,
"ģ": 129,
"Ĥ": 130,
"ĥ": 131,
"Ħ": 132,
"ħ": 133,
"Ĩ": 134,
"ĩ": 135,
"Ī": 136,
"ī": 137,
"Ĭ": 138,
"ĭ": 139,
"Į": 140,
"į": 141,
"İ": 142,
"ı": 143,
"IJ": 144,
"ij": 145,
"Ĵ": 146,
"ĵ": 147,
"Ķ": 148,
"ķ": 149,
"ĸ": 150,
"Ĺ": 151,
"ĺ": 152,
"Ļ": 153,
"ļ": 154,
"Ľ": 155,
"ľ": 156,
"Ŀ": 157,
"ŀ": 158,
"Ł": 159,
"ł": 160,
"¡": 161,
"¢": 162,
"£": 163,
"¤": 164,
"¥": 165,
"¦": 166,
"§": 167,
"¨": 168,
"©": 169,
"ª": 170,
"«": 171,
"¬": 172,
"Ń": 173,
"®": 174,
"¯": 175,
"°": 176,
"±": 177,
"²": 178,
"³": 179,
"´": 180,
"µ": 181,
"¶": 182,
"·": 183,
"¸": 184,
"¹": 185,
"º": 186,
"»": 187,
"¼": 188,
"½": 189,
"¾": 190,
"¿": 191,
"À": 192,
"Á": 193,
"Â": 194,
"Ã": 195,
"Ä": 196,
"Å": 197,
"Æ": 198,
"Ç": 199,
"È": 200,
"É": 201,
"Ê": 202,
"Ë": 203,
"Ì": 204,
"Í": 205,
"Î": 206,
"Ï": 207,
"Ð": 208,
"Ñ": 209,
"Ò": 210,
"Ó": 211,
"Ô": 212,
"Õ": 213,
"Ö": 214,
"×": 215,
"Ø": 216,
"Ù": 217,
"Ú": 218,
"Û": 219,
"Ü": 220,
"Ý": 221,
"Þ": 222,
"ß": 223,
"à": 224,
"á": 225,
"â": 226,
"ã": 227,
"ä": 228,
"å": 229,
"æ": 230,
"ç": 231,
"è": 232,
"é": 233,
"ê": 234,
"ë": 235,
"ì": 236,
"í": 237,
"î": 238,
"ï": 239,
"ð": 240,
"ñ": 241,
"ò": 242,
"ó": 243,
"ô": 244,
"õ": 245,
"ö": 246,
"÷": 247,
"ø": 248,
"ù": 249,
"ú": 250,
"û": 251,
"ü": 252,
"ý": 253,
"þ": 254,
"ÿ": 255
},
"merges": []
}
}

10
tokenizer_config.json Normal file
View File

@@ -0,0 +1,10 @@
{
"backend": "tokenizers",
"model_input_names": [
"input_ids",
"attention_mask"
],
"model_max_length": 1000000000000000019884624838656,
"pad_token": "Ā",
"tokenizer_class": "TokenizersBackend"
}