初始化项目,由ModelHub XC社区提供模型
Model: opencerebral/Boris-1.3-75M-Instruct Source: Original Platform
This commit is contained in:
38
.gitattributes
vendored
Normal file
38
.gitattributes
vendored
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Boris-75M.png filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Boris-1.3-75M.png filter=lfs diff=lfs merge=lfs -text
|
||||||
|
benchmarks.png filter=lfs diff=lfs merge=lfs -text
|
||||||
3
Boris-1.3-75M.png
Normal file
3
Boris-1.3-75M.png
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:c7c872c8f0f655c2e60c1f807ef125ff6913063431c5eede951c1b017c219da3
|
||||||
|
size 783476
|
||||||
88
README.md
Normal file
88
README.md
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
---
|
||||||
|
license: apache-2.0
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
library_name: transformers
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
datasets:
|
||||||
|
- HuggingFaceFW/fineweb-edu
|
||||||
|
- mlfoundations/dclm-baseline-1.0-parquet
|
||||||
|
- HuggingFaceTB/smol-smoltalk
|
||||||
|
tags:
|
||||||
|
- boris
|
||||||
|
- nmai
|
||||||
|
- gpt2
|
||||||
|
- 75M
|
||||||
|
- instruct
|
||||||
|
base_model:
|
||||||
|
- KSP-NMAI/Boris-1.3-75M
|
||||||
|
---
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
# Boris-1.3-75M-Instruct
|
||||||
|
|
||||||
|
Boris-1.3-75M-Instruct is a 75 million-parameter language model created by New
|
||||||
|
Millennium Artificial Intelligence (NMAI).
|
||||||
|
|
||||||
|
This is an **instruction-tuned model**. It follows instructions and holds a
|
||||||
|
conversation. For the base (pretrained-only) version, see
|
||||||
|
[KSP-NMAI/Boris-1.3-75M](https://huggingface.co/KSP-NMAI/Boris-1.3-75M).
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```python
|
||||||
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||||
|
|
||||||
|
tok = AutoTokenizer.from_pretrained("KSP-NMAI/Boris-1.3-75M-Instruct")
|
||||||
|
model = AutoModelForCausalLM.from_pretrained("KSP-NMAI/Boris-1.3-75M-Instruct")
|
||||||
|
|
||||||
|
messages = [{"role": "user", "content": "What's a good way to start learning C?"}]
|
||||||
|
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
||||||
|
out = model.generate(ids, max_new_tokens=120, do_sample=True, top_p=0.95)
|
||||||
|
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
|
||||||
|
```
|
||||||
|
|
||||||
|
## Details
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| Architecture | GPT-2 (pre-LN, learned positional embeddings, tied embeddings) |
|
||||||
|
| Layers / heads / d_model | 12 / 9 / 576 |
|
||||||
|
| Context length | 1024 |
|
||||||
|
| Vocab | 50304 (GPT-NeoX-20B BPE, padded) |
|
||||||
|
| Tokenizer | `EleutherAI/gpt-neox-20b` |
|
||||||
|
| Precision | trained in bf16 autocast with fp32 master weights |
|
||||||
|
|
||||||
|
## Fine-tuning
|
||||||
|
|
||||||
|
Fine-tuned on [smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) and [OpenAssistant](https://huggingface.co/datasets/OpenAssistant/oasst1) for *279,815* examples over *8:19:30* on one RTX 3060.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| Final loss | *1.4162* |
|
||||||
|
| Final grad norm | *1.185* |
|
||||||
|
| Final learning rate | *1.00e-05* |
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
A model of this size will produce text that is frequently inaccurate,
|
||||||
|
inconsistent, or offensive. It has received no alignment or safety tuning and
|
||||||
|
should not be used for factual reference or deployed without supervision.
|
||||||
|
|
||||||
|
## Copyright & License
|
||||||
|
|
||||||
|
*Copyright 2026 Joseph Jones*
|
||||||
|
|
||||||
|
This project and all associated files (the "Work") are licensed under the Apache
|
||||||
|
License, Version 2.0 (the "License"); you may not use this project except in
|
||||||
|
compliance with the License. You may obtain a copy of the License at:
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software distributed
|
||||||
|
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
|
||||||
|
CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||||
|
specific language governing permissions and limitations under the License.
|
||||||
3
benchmarks.png
Normal file
3
benchmarks.png
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:7cb9e4300acca79c53167b65c655b180ec77f2b3a3ed8744350d3c71582141cc
|
||||||
|
size 186997
|
||||||
35
config.json
Normal file
35
config.json
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
{
|
||||||
|
"activation_function": "gelu",
|
||||||
|
"add_cross_attention": false,
|
||||||
|
"architectures": [
|
||||||
|
"GPT2LMHeadModel"
|
||||||
|
],
|
||||||
|
"attn_pdrop": 0.0,
|
||||||
|
"bos_token_id": 0,
|
||||||
|
"dtype": "float32",
|
||||||
|
"embd_pdrop": 0.0,
|
||||||
|
"eos_token_id": 0,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"layer_norm_epsilon": 1e-05,
|
||||||
|
"model_type": "gpt2",
|
||||||
|
"n_ctx": 1024,
|
||||||
|
"n_embd": 576,
|
||||||
|
"n_head": 9,
|
||||||
|
"n_inner": null,
|
||||||
|
"n_layer": 12,
|
||||||
|
"n_positions": 1024,
|
||||||
|
"pad_token_id": null,
|
||||||
|
"reorder_and_upcast_attn": false,
|
||||||
|
"resid_pdrop": 0.0,
|
||||||
|
"scale_attn_by_inverse_layer_idx": false,
|
||||||
|
"scale_attn_weights": true,
|
||||||
|
"summary_activation": null,
|
||||||
|
"summary_first_dropout": 0.1,
|
||||||
|
"summary_proj_to_labels": true,
|
||||||
|
"summary_type": "cls_index",
|
||||||
|
"summary_use_proj": true,
|
||||||
|
"tie_word_embeddings": true,
|
||||||
|
"transformers_version": "5.13.1",
|
||||||
|
"use_cache": true,
|
||||||
|
"vocab_size": 50304
|
||||||
|
}
|
||||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
{
|
||||||
|
"_from_model_config": true,
|
||||||
|
"bos_token_id": 0,
|
||||||
|
"eos_token_id": 0,
|
||||||
|
"output_attentions": false,
|
||||||
|
"output_hidden_states": false,
|
||||||
|
"transformers_version": "5.13.1",
|
||||||
|
"use_cache": true
|
||||||
|
}
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:6fee1ba947d5603c7d9297dc69830d7f1db9b8afa4c2bffbbe7060d9f4b73ed5
|
||||||
|
size 309741640
|
||||||
250557
tokenizer.json
Normal file
250557
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
15
tokenizer_config.json
Normal file
15
tokenizer_config.json
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"add_prefix_space": false,
|
||||||
|
"backend": "tokenizers",
|
||||||
|
"bos_token": "<|endoftext|>",
|
||||||
|
"eos_token": "<|endoftext|>",
|
||||||
|
"errors": "replace",
|
||||||
|
"is_local": false,
|
||||||
|
"local_files_only": false,
|
||||||
|
"model_max_length": 1000000000000000019884624838656,
|
||||||
|
"pad_token": "<|padding|>",
|
||||||
|
"tokenizer_class": "GPTNeoXTokenizer",
|
||||||
|
"trim_offsets": true,
|
||||||
|
"unk_token": "<|endoftext|>",
|
||||||
|
"chat_template": "{%- set ns = namespace(sys='') -%}{%- for message in messages -%}{%- if message['role'] == 'system' -%}{%- set ns.sys = (message['content'] | trim) -%}{%- elif message['role'] == 'user' -%}{%- set instr = (message['content'] | trim) -%}{%- if ns.sys -%}{%- set instr = ns.sys + '\n\n' + instr -%}{%- set ns.sys = '' -%}{%- endif -%}{{- 'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n' + instr + '\n\n### Response:\n' -}}{%- elif message['role'] == 'assistant' -%}{{- ' ' + (message['content'] | trim) + '<|endoftext|>' -}}{%- endif -%}{%- endfor -%}"
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user