RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-04 06:06:15 +08:00
commit b6fd5ce008
24 changed files with 274 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q3_K.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q4_K.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q5_K.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
OpenBezoar-HH-RLHF-SFT.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

3
OpenBezoar-HH-RLHF-SFT.IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f3c863f966eb75841deaf6a87a0184c79d19ac1f0dbad0a6a47c915836c311b6
size 2056852736

3
OpenBezoar-HH-RLHF-SFT.IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4cef0fd1ea745d304462f25364dee27dd76c67221a58fe970e4168951360ec6b
size 1979924736

3
OpenBezoar-HH-RLHF-SFT.IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8336e294b5555de56c5dc0b2cb2d57bfcd98f54e18d839ba065c41b9e418d326
size 1979924736

3
OpenBezoar-HH-RLHF-SFT.IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8321ad0f77b31bd7716b12641fbaa163d93db5199041b853fec803c4da672dd7
size 1995476736

3
OpenBezoar-HH-RLHF-SFT.IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e34c9e94ab39317a4ac0c9b530ba20927b54e05a694c8bf22c2e259f95fd8469
size 1995476736

3
OpenBezoar-HH-RLHF-SFT.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e25620160fb021ac138ffd11f9c1b308c9a68155204487c316faf0b74e4fad7c
size 1979924736

3
OpenBezoar-HH-RLHF-SFT.Q3_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dd02d1ca918a50b162a0f78b7303d9594b698bf7e18fd6c82b6b52def07d89ea
size 2139348736

3
OpenBezoar-HH-RLHF-SFT.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:973c3e8b7f8dcd75234442192cc2a0ef520b9b762ec993f3aa6d6f688f57bf57
size 2214548736

3
OpenBezoar-HH-RLHF-SFT.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dd02d1ca918a50b162a0f78b7303d9594b698bf7e18fd6c82b6b52def07d89ea
size 2139348736

3
OpenBezoar-HH-RLHF-SFT.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e1057b062ecba7c5f10182143bd2df727678c98f6517da9f7a7a96f06c9ba31d
size 1979924736

3
OpenBezoar-HH-RLHF-SFT.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0c258f8d015e46d60c9324654af2a0c1948d40a9d10b23182d42f01a3b727dd4
size 1979924736

3
OpenBezoar-HH-RLHF-SFT.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:acd8c8ababe4a484db5b764448cb560f39e372addcb9588b6811bdac1f4d91e9
size 2187668736

3
OpenBezoar-HH-RLHF-SFT.Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cec00121bac10a42f642ade884d8bfbc3618322c173fe7c7b03f076c229c1316
size 2580116736

3
OpenBezoar-HH-RLHF-SFT.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cec00121bac10a42f642ade884d8bfbc3618322c173fe7c7b03f076c229c1316
size 2580116736

3
OpenBezoar-HH-RLHF-SFT.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:435c22c0ba55f8ecaeb88262e5dae1f592d032a18aa596d2ea6f27c6b81f449c
size 2403156736

3
OpenBezoar-HH-RLHF-SFT.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:841238f89639e844a7e58e2b82b14f759af57e5fcacbdf8b32606044916a4d7e
size 2395412736

3
OpenBezoar-HH-RLHF-SFT.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c3b865ac7aeef09e011ce31b676d9afeba5fc13ae4a9ab3b3e32033e779b4902
size 2603156736

3
OpenBezoar-HH-RLHF-SFT.Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0a8de077d5c354ec0b63f24bafda992431662a46fba2fc5c67fe450b755304fb
size 2757076736

3
OpenBezoar-HH-RLHF-SFT.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0a8de077d5c354ec0b63f24bafda992431662a46fba2fc5c67fe450b755304fb
size 2757076736

3
OpenBezoar-HH-RLHF-SFT.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f8a2e040b1643a9a30bc1da3f5857bffa1c53820d80861633de3071a42be350b
size 2603156736

3
OpenBezoar-HH-RLHF-SFT.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6a044f876b4cd1340889d2f072f91df70ebd15fe4c2c2c2abf97e9049a929942
size 3641876736

3
OpenBezoar-HH-RLHF-SFT.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ae44472f792e6e0384f1e8aa78e166b64fa83f2d0c1858950a9b5d81e82cec91
size 3641876736

151
README.md Normal file
View File

@@ -0,0 +1,151 @@
Quantization made by Richard Erkhov.
[Github](https://github.com/RichardErkhov)
[Discord](https://discord.gg/pvy7H8DZMG)
[Request more models](https://github.com/RichardErkhov/quant_request)
OpenBezoar-HH-RLHF-SFT - GGUF
- Model creator: https://huggingface.co/SurgeGlobal/
- Original model: https://huggingface.co/SurgeGlobal/OpenBezoar-HH-RLHF-SFT/
| Name | Quant method | Size |
| ---- | ---- | ---- |
| [OpenBezoar-HH-RLHF-SFT.Q2_K.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q2_K.gguf) | Q2_K | 1.84GB |
| [OpenBezoar-HH-RLHF-SFT.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.IQ3_XS.gguf) | IQ3_XS | 1.84GB |
| [OpenBezoar-HH-RLHF-SFT.IQ3_S.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.IQ3_S.gguf) | IQ3_S | 1.84GB |
| [OpenBezoar-HH-RLHF-SFT.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q3_K_S.gguf) | Q3_K_S | 1.84GB |
| [OpenBezoar-HH-RLHF-SFT.IQ3_M.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.IQ3_M.gguf) | IQ3_M | 1.92GB |
| [OpenBezoar-HH-RLHF-SFT.Q3_K.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q3_K.gguf) | Q3_K | 1.99GB |
| [OpenBezoar-HH-RLHF-SFT.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q3_K_M.gguf) | Q3_K_M | 1.99GB |
| [OpenBezoar-HH-RLHF-SFT.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q3_K_L.gguf) | Q3_K_L | 2.06GB |
| [OpenBezoar-HH-RLHF-SFT.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.IQ4_XS.gguf) | IQ4_XS | 1.86GB |
| [OpenBezoar-HH-RLHF-SFT.Q4_0.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q4_0.gguf) | Q4_0 | 1.84GB |
| [OpenBezoar-HH-RLHF-SFT.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.IQ4_NL.gguf) | IQ4_NL | 1.86GB |
| [OpenBezoar-HH-RLHF-SFT.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q4_K_S.gguf) | Q4_K_S | 2.24GB |
| [OpenBezoar-HH-RLHF-SFT.Q4_K.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q4_K.gguf) | Q4_K | 2.4GB |
| [OpenBezoar-HH-RLHF-SFT.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q4_K_M.gguf) | Q4_K_M | 2.4GB |
| [OpenBezoar-HH-RLHF-SFT.Q4_1.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q4_1.gguf) | Q4_1 | 2.04GB |
| [OpenBezoar-HH-RLHF-SFT.Q5_0.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q5_0.gguf) | Q5_0 | 2.23GB |
| [OpenBezoar-HH-RLHF-SFT.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q5_K_S.gguf) | Q5_K_S | 2.42GB |
| [OpenBezoar-HH-RLHF-SFT.Q5_K.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q5_K.gguf) | Q5_K | 2.57GB |
| [OpenBezoar-HH-RLHF-SFT.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q5_K_M.gguf) | Q5_K_M | 2.57GB |
| [OpenBezoar-HH-RLHF-SFT.Q5_1.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q5_1.gguf) | Q5_1 | 2.42GB |
| [OpenBezoar-HH-RLHF-SFT.Q6_K.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q6_K.gguf) | Q6_K | 3.39GB |
| [OpenBezoar-HH-RLHF-SFT.Q8_0.gguf](https://huggingface.co/RichardErkhov/SurgeGlobal_-_OpenBezoar-HH-RLHF-SFT-gguf/blob/main/OpenBezoar-HH-RLHF-SFT.Q8_0.gguf) | Q8_0 | 3.39GB |
Original model description:
---
license: cc-by-nc-4.0
datasets:
- Anthropic/hh-rlhf
language:
- en
pipeline_tag: text-generation
tags:
- text-generation-inference
---
# OpenBezoar-HH-RLHF-SFT
The OpenBezoar-HH-RLHF-SFT is an LLM that has been further instruction fine tuned version of [OpenBezoar-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-SFT) model on a subset of [Anthropic's HH-RLHF Dataset](https://huggingface.co/datasets/Anthropic/hh-rlhf).
## Model Details
- Base Model: [OpenBezoar-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-SFT)
- Dataset used for SFT: First 100K examples of the [HH-RLHF](https://huggingface.co/datasets/Anthropic/hh-rlhf) dataset
- Epochs: 1
### Model Description
OpenBezoar-HH-RLHF-SFT is an LLM that is built upon the OpenLLaMA 3B v2 architecture. Primary purpose of performing SFT on [OpenBezoar-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-SFT) is to minimize the distribution shift before applying Direct Preference Optimization (DPO) for human preferences alignment. For more information please refer to our paper.
### Model Sources
- **Repository:** [Bitbucket Project](https://bitbucket.org/paladinanalytics/workspace/projects/OP)
- **Paper :** [Pre-Print](https://arxiv.org/abs/2404.12195)
## Instruction Format
We follow a modified version of the Alpaca prompt template as shown below. It is important to utilize this template in order to obtain best responses for instruction related tasks.
```
### System:
Below is an instruction that describes a task, optionally paired with an input that provides further context following that instruction. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Response:
```
Notice that **no** end-of-sentence (eos) token is being appended.
*Note: The system prompt shown in the following figure is the one that the model has been trained on most of the time. However, you may attempt to use any other system prompt that is available in the [Orca](https://arxiv.org/abs/2306.02707) scheme.*
## Usage
```python
from peft import PeftConfig, PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig, AutoModelForSeq2SeqLM
checkpoint = "SurgeGlobal/OpenBezoar-HH-RLHF-SFT"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(
checkpoint,
load_in_4bit=True, # optionally for low resource environments
device_map="auto"
)
prompt = """### System:
Below is an instruction that describes a task, optionally paired with an input that provides further context following that instruction. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Response:""".format(
instruction="What is the world state in the year 1597."
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=True)
print(tokenizer.decode(outputs[0]))
```
## Evaluations
Refer to our self-reported evaluations in our paper (Section 4).
## Limitations
- The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.
- This model is not aligned to human preferences and therefore it may generate harmful and uncensored content.
- Caution is urged against relying on this model for production or adjacent use-cases.
## Citation
If you find our work useful, please cite our paper as follows:
```
@misc{surge2024openbezoar,
title={OpenBezoar: Small, Cost-Effective and Open Models Trained on Mixes of Instruction Data},
author={Chandeepa Dissanayake and Lahiru Lowe and Sachith Gunasekara and Yasiru Ratnayake},
year={2024},
eprint={2404.12195},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
## Model Authors
Chandeepa Dissanayake, Lahiru Lowe, Sachith Gunasekara, and Yasiru Ratnayake