初始化项目,由ModelHub XC社区提供模型
Model: opencerebral/Boris-1.3-75M-Instruct Source: Original Platform
This commit is contained in:
38
.gitattributes
vendored
Normal file
38
.gitattributes
vendored
Normal file
@@ -0,0 +1,38 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
Boris-75M.png filter=lfs diff=lfs merge=lfs -text
|
||||
Boris-1.3-75M.png filter=lfs diff=lfs merge=lfs -text
|
||||
benchmarks.png filter=lfs diff=lfs merge=lfs -text
|
||||
3
Boris-1.3-75M.png
Normal file
3
Boris-1.3-75M.png
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c7c872c8f0f655c2e60c1f807ef125ff6913063431c5eede951c1b017c219da3
|
||||
size 783476
|
||||
88
README.md
Normal file
88
README.md
Normal file
@@ -0,0 +1,88 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
datasets:
|
||||
- HuggingFaceFW/fineweb-edu
|
||||
- mlfoundations/dclm-baseline-1.0-parquet
|
||||
- HuggingFaceTB/smol-smoltalk
|
||||
tags:
|
||||
- boris
|
||||
- nmai
|
||||
- gpt2
|
||||
- 75M
|
||||
- instruct
|
||||
base_model:
|
||||
- KSP-NMAI/Boris-1.3-75M
|
||||
---
|
||||
|
||||

|
||||
|
||||
# Boris-1.3-75M-Instruct
|
||||
|
||||
Boris-1.3-75M-Instruct is a 75 million-parameter language model created by New
|
||||
Millennium Artificial Intelligence (NMAI).
|
||||
|
||||
This is an **instruction-tuned model**. It follows instructions and holds a
|
||||
conversation. For the base (pretrained-only) version, see
|
||||
[KSP-NMAI/Boris-1.3-75M](https://huggingface.co/KSP-NMAI/Boris-1.3-75M).
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
tok = AutoTokenizer.from_pretrained("KSP-NMAI/Boris-1.3-75M-Instruct")
|
||||
model = AutoModelForCausalLM.from_pretrained("KSP-NMAI/Boris-1.3-75M-Instruct")
|
||||
|
||||
messages = [{"role": "user", "content": "What's a good way to start learning C?"}]
|
||||
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
||||
out = model.generate(ids, max_new_tokens=120, do_sample=True, top_p=0.95)
|
||||
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Details
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Architecture | GPT-2 (pre-LN, learned positional embeddings, tied embeddings) |
|
||||
| Layers / heads / d_model | 12 / 9 / 576 |
|
||||
| Context length | 1024 |
|
||||
| Vocab | 50304 (GPT-NeoX-20B BPE, padded) |
|
||||
| Tokenizer | `EleutherAI/gpt-neox-20b` |
|
||||
| Precision | trained in bf16 autocast with fp32 master weights |
|
||||
|
||||
## Fine-tuning
|
||||
|
||||
Fine-tuned on [smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) and [OpenAssistant](https://huggingface.co/datasets/OpenAssistant/oasst1) for *279,815* examples over *8:19:30* on one RTX 3060.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Final loss | *1.4162* |
|
||||
| Final grad norm | *1.185* |
|
||||
| Final learning rate | *1.00e-05* |
|
||||
|
||||

|
||||
|
||||
## Limitations
|
||||
|
||||
A model of this size will produce text that is frequently inaccurate,
|
||||
inconsistent, or offensive. It has received no alignment or safety tuning and
|
||||
should not be used for factual reference or deployed without supervision.
|
||||
|
||||
## Copyright & License
|
||||
|
||||
*Copyright 2026 Joseph Jones*
|
||||
|
||||
This project and all associated files (the "Work") are licensed under the Apache
|
||||
License, Version 2.0 (the "License"); you may not use this project except in
|
||||
compliance with the License. You may obtain a copy of the License at:
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software distributed
|
||||
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
|
||||
CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations under the License.
|
||||
3
benchmarks.png
Normal file
3
benchmarks.png
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7cb9e4300acca79c53167b65c655b180ec77f2b3a3ed8744350d3c71582141cc
|
||||
size 186997
|
||||
35
config.json
Normal file
35
config.json
Normal file
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"activation_function": "gelu",
|
||||
"add_cross_attention": false,
|
||||
"architectures": [
|
||||
"GPT2LMHeadModel"
|
||||
],
|
||||
"attn_pdrop": 0.0,
|
||||
"bos_token_id": 0,
|
||||
"dtype": "float32",
|
||||
"embd_pdrop": 0.0,
|
||||
"eos_token_id": 0,
|
||||
"initializer_range": 0.02,
|
||||
"layer_norm_epsilon": 1e-05,
|
||||
"model_type": "gpt2",
|
||||
"n_ctx": 1024,
|
||||
"n_embd": 576,
|
||||
"n_head": 9,
|
||||
"n_inner": null,
|
||||
"n_layer": 12,
|
||||
"n_positions": 1024,
|
||||
"pad_token_id": null,
|
||||
"reorder_and_upcast_attn": false,
|
||||
"resid_pdrop": 0.0,
|
||||
"scale_attn_by_inverse_layer_idx": false,
|
||||
"scale_attn_weights": true,
|
||||
"summary_activation": null,
|
||||
"summary_first_dropout": 0.1,
|
||||
"summary_proj_to_labels": true,
|
||||
"summary_type": "cls_index",
|
||||
"summary_use_proj": true,
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "5.13.1",
|
||||
"use_cache": true,
|
||||
"vocab_size": 50304
|
||||
}
|
||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 0,
|
||||
"eos_token_id": 0,
|
||||
"output_attentions": false,
|
||||
"output_hidden_states": false,
|
||||
"transformers_version": "5.13.1",
|
||||
"use_cache": true
|
||||
}
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6fee1ba947d5603c7d9297dc69830d7f1db9b8afa4c2bffbbe7060d9f4b73ed5
|
||||
size 309741640
|
||||
250557
tokenizer.json
Normal file
250557
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
15
tokenizer_config.json
Normal file
15
tokenizer_config.json
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"add_prefix_space": false,
|
||||
"backend": "tokenizers",
|
||||
"bos_token": "<|endoftext|>",
|
||||
"eos_token": "<|endoftext|>",
|
||||
"errors": "replace",
|
||||
"is_local": false,
|
||||
"local_files_only": false,
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"pad_token": "<|padding|>",
|
||||
"tokenizer_class": "GPTNeoXTokenizer",
|
||||
"trim_offsets": true,
|
||||
"unk_token": "<|endoftext|>",
|
||||
"chat_template": "{%- set ns = namespace(sys='') -%}{%- for message in messages -%}{%- if message['role'] == 'system' -%}{%- set ns.sys = (message['content'] | trim) -%}{%- elif message['role'] == 'user' -%}{%- set instr = (message['content'] | trim) -%}{%- if ns.sys -%}{%- set instr = ns.sys + '\n\n' + instr -%}{%- set ns.sys = '' -%}{%- endif -%}{{- 'Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n' + instr + '\n\n### Response:\n' -}}{%- elif message['role'] == 'assistant' -%}{{- ' ' + (message['content'] | trim) + '<|endoftext|>' -}}{%- endif -%}{%- endfor -%}"
|
||||
}
|
||||
Reference in New Issue
Block a user