初始化项目,由ModelHub XC社区提供模型

Model: opencerebral/Boris-1.3-75M-Instruct
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-15 02:08:17 +08:00
commit 58ebe0a0a9
9 changed files with 250751 additions and 0 deletions

38
.gitattributes vendored Normal file
View File

@@ -0,0 +1,38 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Boris-75M.png filter=lfs diff=lfs merge=lfs -text
Boris-1.3-75M.png filter=lfs diff=lfs merge=lfs -text
benchmarks.png filter=lfs diff=lfs merge=lfs -text

3
Boris-1.3-75M.png Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c7c872c8f0f655c2e60c1f807ef125ff6913063431c5eede951c1b017c219da3
size 783476

88
README.md Normal file
View File

@@ -0,0 +1,88 @@
---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
datasets:
- HuggingFaceFW/fineweb-edu
- mlfoundations/dclm-baseline-1.0-parquet
- HuggingFaceTB/smol-smoltalk
tags:
- boris
- nmai
- gpt2
- 75M
- instruct
base_model:
- KSP-NMAI/Boris-1.3-75M
---
![Boris](Boris-1.3-75M.png)
# Boris-1.3-75M-Instruct
Boris-1.3-75M-Instruct is a 75 million-parameter language model created by New
Millennium Artificial Intelligence (NMAI).
This is an **instruction-tuned model**. It follows instructions and holds a
conversation. For the base (pretrained-only) version, see
[KSP-NMAI/Boris-1.3-75M](https://huggingface.co/KSP-NMAI/Boris-1.3-75M).
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("KSP-NMAI/Boris-1.3-75M-Instruct")
model = AutoModelForCausalLM.from_pretrained("KSP-NMAI/Boris-1.3-75M-Instruct")
messages = [{"role": "user", "content": "What's a good way to start learning C?"}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=120, do_sample=True, top_p=0.95)
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
```
## Details
| | |
|---|---|
| Architecture | GPT-2 (pre-LN, learned positional embeddings, tied embeddings) |
| Layers / heads / d_model | 12 / 9 / 576 |
| Context length | 1024 |
| Vocab | 50304 (GPT-NeoX-20B BPE, padded) |
| Tokenizer | `EleutherAI/gpt-neox-20b` |
| Precision | trained in bf16 autocast with fp32 master weights |
## Fine-tuning
Fine-tuned on [smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) and [OpenAssistant](https://huggingface.co/datasets/OpenAssistant/oasst1) for *279,815* examples over *8:19:30* on one RTX 3060.
| | |
|---|---|
| Final loss | *1.4162* |
| Final grad norm | *1.185* |
| Final learning rate | *1.00e-05* |
![Benchmarks](benchmarks.png)
## Limitations
A model of this size will produce text that is frequently inaccurate,
inconsistent, or offensive. It has received no alignment or safety tuning and
should not be used for factual reference or deployed without supervision.
## Copyright & License
*Copyright 2026 Joseph Jones*
This project and all associated files (the "Work") are licensed under the Apache
License, Version 2.0 (the "License"); you may not use this project except in
compliance with the License. You may obtain a copy of the License at:
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.

3
benchmarks.png Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7cb9e4300acca79c53167b65c655b180ec77f2b3a3ed8744350d3c71582141cc
size 186997

35
config.json Normal file
View File

@@ -0,0 +1,35 @@
{
"activation_function": "gelu",
"add_cross_attention": false,
"architectures": [
"GPT2LMHeadModel"
],
"attn_pdrop": 0.0,
"bos_token_id": 0,
"dtype": "float32",
"embd_pdrop": 0.0,
"eos_token_id": 0,
"initializer_range": 0.02,
"layer_norm_epsilon": 1e-05,
"model_type": "gpt2",
"n_ctx": 1024,
"n_embd": 576,
"n_head": 9,
"n_inner": null,
"n_layer": 12,
"n_positions": 1024,
"pad_token_id": null,
"reorder_and_upcast_attn": false,
"resid_pdrop": 0.0,
"scale_attn_by_inverse_layer_idx": false,
"scale_attn_weights": true,
"summary_activation": null,
"summary_first_dropout": 0.1,
"summary_proj_to_labels": true,
"summary_type": "cls_index",
"summary_use_proj": true,
"tie_word_embeddings": true,
"transformers_version": "5.13.1",
"use_cache": true,
"vocab_size": 50304
}

9
generation_config.json Normal file
View File

@@ -0,0 +1,9 @@
{
"_from_model_config": true,
"bos_token_id": 0,
"eos_token_id": 0,
"output_attentions": false,
"output_hidden_states": false,
"transformers_version": "5.13.1",
"use_cache": true
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6fee1ba947d5603c7d9297dc69830d7f1db9b8afa4c2bffbbe7060d9f4b73ed5
size 309741640

250557
tokenizer.json Normal file

File diff suppressed because it is too large Load Diff

15
tokenizer_config.json Normal file
View File

@@ -0,0 +1,15 @@
{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": "<|endoftext|>",
"eos_token": "<|endoftext|>",
"errors": "replace",
"is_local": false,
"local_files_only": false,
"model_max_length": 1000000000000000019884624838656,
"pad_token": "<|padding|>",
"tokenizer_class": "GPTNeoXTokenizer",
"trim_offsets": true,
"unk_token": "<|endoftext|>",
"chat_template": "{%- set ns = namespace(sys='') -%}{%- for message in messages -%}{%- if message['role'] == 'system' -%}{%- set ns.sys = (message['content'] | trim) -%}{%- elif message['role'] == 'user' -%}{%- set instr = (message['content'] | trim) -%}{%- if ns.sys -%}{%- set instr = ns.sys + '\n\n' + instr -%}{%- set ns.sys = '' -%}{%- endif -%}{{- 'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n' + instr + '\n\n### Response:\n' -}}{%- elif message['role'] == 'assistant' -%}{{- ' ' + (message['content'] | trim) + '<|endoftext|>' -}}{%- endif -%}{%- endfor -%}"
}