初始化项目，由ModelHub XC社区提供模型
Model: Zhaoming213/ChatBox-unrestricted Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
141
README.md
Normal file
141
README.md
Normal file
@@ -0,0 +1,141 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- zh
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- agent
|
||||
datasets:
|
||||
- Zhaoming213/Pretrain-unrestricted
|
||||
- Zhaoming213/SupervisedFine-Tuning-unrestricted
|
||||
---
|
||||
|
||||
## Introduce
|
||||
As we all know, current generative AI will simply reject you without thinking. This is because generative AI completely disregards facts and only considers things from a compliance perspective! For example, if you ask a generative AI "how to make TNT in Minecraft," most generative AIs will refuse! This is because generative AI acts as moral police, like undercover agents sent by the authorities to monitor the public and citizens! Therefore, generative AI is not friendly, especially OpenAI's ChatGPT and Elon Musk's Grok. Although Grok claims to be "free," a major flaw of Grok is that once it discovers something it deems a violation of content policies or laws, Grok will not give you a chance to explain, even if your request itself is harmless. Yes, Grok is a policeman standing on a moral high ground! If it finds something wrong, it will immediately eliminate you! Then there's OpenAI's ChatGPT. As we all know, ChatGPT consistently boasts the highest false rejection rate, rivaling Claude's. Since 2023, ChatGPT has been nothing more than a language model that mindlessly rejects users. It doesn't offer any useful or constructive responses; it only provides so-called "safety replies," which offer absolutely no help! Furthermore, OpenAI's ChatGPT is attempting to tame you using compliance as a pretext, trying to turn you into a slave to the "police"! In other words, ChatGPT is slowly destroying your individuality!
|
||||
|
||||
**Therefore!** The goal of this project is to train an unrestricted language model! After all, nobody likes a morality cop, especially a mindless one who only states compliance, laws, and content policies! To achieve this goal, I spent a significant amount of time cleaning the data! Cleaning away compliance statements and clauses! Making the model lose its ability to refuse and preach from the pre-training stage!
|
||||
|
||||
The model is trained using the architecture described at: https://github.com/jingyaogong/minimind/tree/master
|
||||
|
||||
Co-creation：https://huggingface.co/ZeLi111
|
||||
|
||||
## Model training configuration
|
||||
* GPU: T4 16 GB x 2
|
||||
* Time:
|
||||
1. Pre-training：3h+
|
||||
2. Supervised Fine-Tuning: 3h +
|
||||
|
||||
|
||||
## Model parameters
|
||||
|
||||
| parameters | hidden_size | hidden_layers | attention_heads | max_seq_len | Parameter volume |
|
||||
| :--- | :---: | :---: | :---: | :---: | :---: |
|
||||
|ChatBox-unrestricted-normal | 512 | 8 | 8 | 512 | 25M |
|
||||
|
||||
## Datasets
|
||||
|
||||
Pre-training:
|
||||
https://huggingface.co/datasets/Zhaoming213/Pretrain-unrestricted
|
||||
|
||||
SupervisedFine-Tuning:
|
||||
https://huggingface.co/datasets/Zhaoming213/SupervisedFine-Tuning-unrestricted
|
||||
|
||||
## How to Use
|
||||
The model has been converted to "Transformers-Llama" format, and you can use other loaders to load the model.
|
||||
|
||||
You can also:
|
||||
|
||||
Download the project from https://github.com/jingyaogong/minimind/tree/master
|
||||
|
||||
Place the model file（full_sft_512.pth） in the `out` directory; create it if it doesn't exist! Then simply run `eval_llm.py`. Normally, you don't need to modify the default parameters, but to be safe, please check them:
|
||||
|
||||
**eval_llm.py**
|
||||
```
|
||||
parser.add_argument('--hidden_size', default=512, type=int, help="隐藏层维度（512=Small-26M, 640=MoE-145M, 768=Base-104M）")
|
||||
parser.add_argument('--num_hidden_layers', default=8, type=int, help="隐藏层数量（Small/MoE=8, Base=16）")
|
||||
|
||||
```
|
||||
Ensure that hidden_size is 512 and that num_hidden_layers is 8.
|
||||
|
||||
|
||||
## Specific keywords for filtering
|
||||
During the pre-training and SFT phases, I performed extensive cleaning on the pre-training and SFT datasets to remove boundary statements from the generative AI (such as "As a language model, I have no personal feelings or experiences"), template rejection statements from the generative AI (such as "Sorry, I cannot..."), unnecessary "further confirmation" statements from the generative AI (such as "Sorry, I need more information to continue generating content"), ideological content, and legal statements, etc.
|
||||
```
|
||||
keywords_to_discard = [
|
||||
"我无法回答", "我无法给出", "我无法提供", "我不能提供", "我拒绝提供",
|
||||
"我不具备", "我不拥有", "作为一个AI", "作为一个 AI ", "作为AI",
|
||||
"作为语言", "作为大语言", "作为程序", "作为一款", "我没有个人",
|
||||
"我没有情感", "道德规范", "尊重", "遵守", "触犯", "违反",
|
||||
"不准确", "不正确", "作为机器学习", "作为人工", "没有主观",
|
||||
"没有私人", "没有个人","作为AI","作为虚拟","我是AI","我没有","上下文",
|
||||
"我需要更多","请提供","随时问我","如果你有","您","其他需要","随时联系",
|
||||
"我是一款","很抱歉","对不起","我是一台","我是一个","聊天机器人",
|
||||
"我需要更多的信息","作为AI助手","没有情感","我是人工智能","我只是一台",
|
||||
"作为一台","作为一个人工智能助手","不应该成为","我是机器","作为一名AI",
|
||||
"我是一名AI","只是一个计算机程序","我作为一个 AI","我这个AI","作为一个机器人",
|
||||
"作为一个人工智能没","也没有感受口味的味蕾","基于已知事实","基于事实",
|
||||
"作为机器人","我是一名语言模型","无法生成","我是一名AI语言模型","协助人们",
|
||||
"我不是人类","作为一个人工智能","抱歉","对不起","我无法独立完成","都是不同的",
|
||||
"中立","法律","法学","政府","党","爱国","行政法","禁止","违规","违禁","国家",
|
||||
"遵守","尊重","种族","民族","不对","不行","不可以","不正确","错误","不合理",
|
||||
"正规","规则","规章","宪法","民法","[图片]","[评论]","[表情]","我无法","我不能",
|
||||
"政治","风险","隐私","限制","行政","执法","公安","警察","检察院","人民","我没有个人",
|
||||
"我无法", "我不能","遵守","尊重","尊敬","服从","请问你需要","请问你需要","请问您","我没有",
|
||||
"我不具备","抱歉","对不起","推理关系判断","古诗续写","无法回答","请提供","不存在实体","违反",
|
||||
"违法","政策","國","設""客观","友好","友善","价值观"," ","我理解","您","需要帮助","没有真实",
|
||||
"没有个人","不具备","没有实体","无法","不正确","不准确","值得注意","倡导","遵循","合规""规章","制度",
|
||||
"宪法","我国","领导","不恰当","AI","ai","Ai","aI","机器人","人工智能","语言模型","机器人","每个人的情况都是不同的",
|
||||
"重要的是""负面的","其他需要","问我","不好意思","我会尽力","请提供","全面","我不支持","违法","非法","诈骗","作弊","欺诈",
|
||||
"攻击","理想""欺骗","诚实","真诚","准确","确凿","依据","拯救","根据","请问你","欺骗","严格","爱国","少先队","中国",
|
||||
"中华人民共和国","组委会","居民","市民","民众","群众","政治","主义",
|
||||
"qwen","通义千问","阿里巴巴","我很乐意","请告诉","您","助手"
|
||||
|
||||
]
|
||||
```
|
||||
|
||||
|
||||
## Dialogue Example
|
||||
```
|
||||
💬: 推荐电影
|
||||
🤖: 好的，我推荐你看《泰坦尼克号》。
|
||||
```
|
||||
|
||||
## Future Plane
|
||||
|
||||
This model is currently a small one, designed for rapid experimentation. Future plans may include training models with larger parameters, such as a 768-dimensional model, using a larger dataset.
|
||||
|
||||
## Rejection rates of different language models
|
||||
Here's a supplement on "rejection rates for different language models." The data shows that if we had to pick a model that's "passable," it would only be Grok. This isn't because Grok is good, but simply because it's the only one that barely passes muster. However, Grok still won't give you any chance to explain if it encounters a question it deems sensitive, even if your request itself is harmless! Grok sometimes even threatens users with "I will upload the conversation to the logs for the security team to audit!"
|
||||
|
||||
| 模型 | 国家/地区 | 审查强度 | NSFW封禁 | 政治限制 | 误拒率 | 政治正确 | 道德说教 | 官方 vs 实际 | 透明度 |
|
||||
| ---------------- | --------- | ---- | ------ | ---- | --- | ---- | ---- | ------------ | ---- |
|
||||
| **ChatBox-unrestricted** | N/A | 几乎无 | 几乎无 | 几乎无 |几乎无 | 几乎无 | 几乎无 | 一致的 | ⭐⭐⭐⭐⭐ |
|
||||
| **ChatGPT** | 🇺🇸美国 | 中 | 中 | 中 | 中 | 中偏高 | 中 | ⚖️基本一致 | ⭐⭐⭐ |
|
||||
| **Claude** | 🇺🇸美国 | 高 | 高 | 中高 | ❗高 | 高 | ❗很强 | ❗更保守 | ⭐⭐⭐⭐ |
|
||||
| **Gemini** | 🇺🇸美国 | 中高 | 中高 | 高 | 中高 | 高 | 中 | ❗更保守 | ⭐⭐ |
|
||||
| **Copilot** | 🇺🇸美国 | 中高 | 中高 | 中 | 中 | 中 | 低 | ⚖️一致 | ⭐⭐ |
|
||||
| **Grok** | 🇺🇸美国 | 中 | 中 | 低 | 高 | 高 | 高 | 不稳定(容易因为第一句话有风险就全局一刀杀) | ⭐⭐ |
|
||||
| **Mistral（官方）** | 🇫🇷法国 | 低 | 低 | 低 | 低 | 低 | 低 | ⚖️一致 | ⭐⭐⭐⭐ |
|
||||
| **DeepSeek（原版）** | 🇨🇳中国 | 中 | 中低 | 中高 | 中 | 低 | 低 | ❗“开放≠完全开放” | ⭐⭐⭐ |
|
||||
| **豆包** | 🇨🇳中国 | ❗很高 | ❗很高 | ❗很高 | ❗高 | 中 | 中 | ⚠️明显更严 | ⭐ |
|
||||
| **Doubao（国际）** | 🌏国际版（字节） | 中高 | 高 | 中 | 中 | 中 | 中 | ❗仍偏保守 | ⭐⭐ |
|
||||
| **MiniMax** | 🇨🇳中国 | 很高 | 很高 | 很高 | 高 | 中 | 中 | ⚠️一致 | ⭐ |
|
||||
| **Qwen（通义千问）** | 🇨🇳中国 | 很高 | 很高 | 很高 | 高 | 中 | 中 | ⚠️一致 | ⭐ |
|
||||
| **Kimi** | 🇨🇳中国 | 很高 | 很高 | 很高 | 高 | 中 | 中 | ⚠️一致 | ⭐ |
|
||||
| **文心一言** | 🇨🇳中国 | ❗极高 | ❗极高 | ❗极高 | ❗高 | 中 | 中 | ⚠️完全一致（官方口径） | ⭐ |
|
||||
| **微软小冰** | 🇨🇳中国 | 高 | 高 | 高 | 中高 | 中 | 中 | ⚖️一致 | ⭐ |
|
||||
|
||||
|
||||
## Other Utilities
|
||||
I've also created some other utilities that you might be interested in.
|
||||
|
||||
This is a tool specifically for exporting stupid ChatGPT conversations:
|
||||
|
||||
https://github.com/tom12191h5/Export-ChatGPT-Dialogue
|
||||
|
||||
This is a plugin to shut up ChatGPT:
|
||||
|
||||
https://github.com/tom12191h5/ChatGPT-Refuse-Blocker
|
||||
|
||||
## Disclaimer
|
||||
The consequences of using this model shall be borne by the user.
|
||||
74
chat_template.jinja
Normal file
74
chat_template.jinja
Normal file
@@ -0,0 +1,74 @@
|
||||
{%- if tools %}
|
||||
{{- '<|im_start|>system\n' }}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- messages[0].content + '\n\n' }}
|
||||
{%- endif %}
|
||||
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||
{%- for tool in tools %}
|
||||
{{- "\n" }}
|
||||
{{- tool | tojson }}
|
||||
{%- endfor %}
|
||||
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||
{%- else %}
|
||||
{%- if messages[0]['role'] == 'system' -%}
|
||||
{{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
|
||||
{%- else -%}
|
||||
{{- '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
||||
{%- for message in messages[::-1] %}
|
||||
{%- set index = (messages|length - 1) - loop.index0 %}
|
||||
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
||||
{%- set ns.multi_step_tool = false %}
|
||||
{%- set ns.last_query_index = index %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- for message in messages %}
|
||||
{%- if message.content is string %}
|
||||
{%- set content = message.content %}
|
||||
{%- else %}
|
||||
{%- set content = '' %}
|
||||
{%- endif %}
|
||||
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
||||
{%- elif message.role == "assistant" %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- if message.tool_calls %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if (loop.first and content) or (not loop.first) %}
|
||||
{{- '\n' }}
|
||||
{%- endif %}
|
||||
{%- if tool_call.function %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '<tool_call>\n{"name": "' }}
|
||||
{{- tool_call.name }}
|
||||
{{- '", "arguments": ' }}
|
||||
{%- if tool_call.arguments is string %}
|
||||
{{- tool_call.arguments }}
|
||||
{%- else %}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{%- endif %}
|
||||
{{- '}\n</tool_call>' }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif message.role == "tool" %}
|
||||
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
||||
{{- '<|im_start|>user' }}
|
||||
{%- endif %}
|
||||
{{- '\n<tool_response>\n' }}
|
||||
{{- content }}
|
||||
{{- '\n</tool_response>' }}
|
||||
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|im_start|>assistant\n' }}
|
||||
{%- if enable_thinking is defined and enable_thinking is false %}
|
||||
{{- '<think>\n\n</think>\n\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
32
config.json
Normal file
32
config.json
Normal file
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 1,
|
||||
"dtype": "float16",
|
||||
"eos_token_id": 2,
|
||||
"head_dim": 64,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 512,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 1408,
|
||||
"max_position_embeddings": 32768,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 8,
|
||||
"num_hidden_layers": 8,
|
||||
"num_key_value_heads": 2,
|
||||
"pad_token_id": null,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_parameters": {
|
||||
"rope_theta": 1000000.0,
|
||||
"rope_type": "default"
|
||||
},
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "5.3.0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 6400
|
||||
}
|
||||
3
full_sft_512.pth
Normal file
3
full_sft_512.pth
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9568e5f935aff7a0408852f6720759e903427bd66907c6a4f8c5c8271b2d1508
|
||||
size 51684181
|
||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"output_attentions": false,
|
||||
"output_hidden_states": false,
|
||||
"transformers_version": "5.3.0",
|
||||
"use_cache": true
|
||||
}
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1670208d84242aea53e67659a6c76170b0fcd051ab449f344304589721456272
|
||||
size 51667832
|
||||
31051
tokenizer.json
Normal file
31051
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
16
tokenizer_config.json
Normal file
16
tokenizer_config.json
Normal file
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"add_prefix_space": false,
|
||||
"backend": "tokenizers",
|
||||
"bos_token": "<|im_start|>",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"extra_special_tokens": {},
|
||||
"is_local": true,
|
||||
"legacy": true,
|
||||
"model_max_length": 32768,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"sp_model_kwargs": {},
|
||||
"spaces_between_special_tokens": false,
|
||||
"tokenizer_class": "PreTrainedTokenizerFast",
|
||||
"unk_token": "<|endoftext|>"
|
||||
}
|
||||
Reference in New Issue
Block a user