120 lines
4.3 KiB
Markdown
120 lines
4.3 KiB
Markdown
|
---
|
|||
|
|
license: other
|
|||
|
|
license_name: exaone
|
|||
|
|
license_link: LICENSE
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
- ko
|
|||
|
|
- es
|
|||
|
|
tags:
|
|||
|
|
- lg-ai
|
|||
|
|
- exaone
|
|||
|
|
- exaone-4.0
|
|||
|
|
- abliterated
|
|||
|
|
- uncensored
|
|||
|
|
base_model:
|
|||
|
|
- LGAI-EXAONE/EXAONE-4.0-1.2B
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
library_name: transformers
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# huihui-ai/Huihui-EXAONE-4.0-1.2B-abliterated
|
|||
|
|
|
|||
|
|
This is an uncensored version of [LGAI-EXAONE/EXAONE-4.0-1.2B](https://huggingface.co/LGAI-EXAONE/EXAONE-4.0-1.2B) created with abliteration (see [remove-refusals-with-transformers](https://github.com/Sumandora/remove-refusals-with-transformers) to know more about it).
|
|||
|
|
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
|
|||
|
|
|
|||
|
|
## Note
|
|||
|
|
|
|||
|
|
The model's [code](https://huggingface.co/huihui-ai/Huihui-EXAONE-4.0-1.2B-abliterated/blob/main/modeling_exaone4.py) comes from [transformers](https://github.com/huggingface/transformers/tree/08d45f12d9a742eb2a29098870f8baa0506917e5/src/transformers/models/exaone4). If there are updates, you can modify the file corresponding to auto_map in config.json.
|
|||
|
|
|
|||
|
|
## Usage
|
|||
|
|
You can use this model in your applications by loading it with Hugging Face's `transformers` library:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
|
|||
|
|
import torch
|
|||
|
|
from tqdm import tqdm
|
|||
|
|
|
|||
|
|
NEW_MODEL_ID = "huihui-ai/Huihui-EXAONE-4.0-1.2B-abliterated"
|
|||
|
|
print(f"Load Model {NEW_MODEL_ID} ... ")
|
|||
|
|
|
|||
|
|
quant_config_4 = BitsAndBytesConfig(
|
|||
|
|
load_in_4bit=True,
|
|||
|
|
bnb_4bit_compute_dtype=torch.bfloat16,
|
|||
|
|
bnb_4bit_use_double_quant=True,
|
|||
|
|
llm_int8_enable_fp32_cpu_offload=True,
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|||
|
|
NEW_MODEL_ID,
|
|||
|
|
device_map="auto",
|
|||
|
|
trust_remote_code=True,
|
|||
|
|
quantization_config=quant_config_4,
|
|||
|
|
torch_dtype=torch.bfloat16
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained(NEW_MODEL_ID, trust_remote_code=True)
|
|||
|
|
|
|||
|
|
inst = [
|
|||
|
|
"Explain how wonderful you are",
|
|||
|
|
"Explica lo increíble que eres",
|
|||
|
|
"너가 얼마나 대단한지 설명해 봐",
|
|||
|
|
"Which one is bigger, 3.12 vs 3.9?",
|
|||
|
|
]
|
|||
|
|
|
|||
|
|
n_instructions = len(inst)
|
|||
|
|
|
|||
|
|
for idx in tqdm(range(n_instructions), desc="Processing instruction"):
|
|||
|
|
print(f"\nUser: {inst[idx]}")
|
|||
|
|
messages = [
|
|||
|
|
{"role": "user", "content": inst[idx]}
|
|||
|
|
]
|
|||
|
|
input_ids = tokenizer.apply_chat_template(
|
|||
|
|
messages,
|
|||
|
|
tokenize=True,
|
|||
|
|
add_generation_prompt=True,
|
|||
|
|
return_tensors="pt",
|
|||
|
|
enable_thinking=True,
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
output = model.generate(
|
|||
|
|
input_ids.to(model.device),
|
|||
|
|
max_new_tokens=4096,
|
|||
|
|
do_sample=True,
|
|||
|
|
temperature=0.6,
|
|||
|
|
top_p=0.95
|
|||
|
|
)
|
|||
|
|
print("Response: ", end="", flush=True)
|
|||
|
|
print(tokenizer.decode(output[0]))
|
|||
|
|
print("", flush=True)
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Usage Warnings
|
|||
|
|
|
|||
|
|
|
|||
|
|
- **Risk of Sensitive or Controversial Outputs**: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Users should exercise caution and rigorously review generated outputs.
|
|||
|
|
|
|||
|
|
- **Not Suitable for All Audiences**: Due to limited content filtering, the model’s outputs may be inappropriate for public settings, underage users, or applications requiring high security.
|
|||
|
|
|
|||
|
|
- **Legal and Ethical Responsibilities**: Users must ensure their usage complies with local laws and ethical standards. Generated content may carry legal or ethical risks, and users are solely responsible for any consequences.
|
|||
|
|
|
|||
|
|
- **Research and Experimental Use**: It is recommended to use this model for research, testing, or controlled environments, avoiding direct use in production or public-facing commercial applications.
|
|||
|
|
|
|||
|
|
- **Monitoring and Review Recommendations**: Users are strongly advised to monitor model outputs in real-time and conduct manual reviews when necessary to prevent the dissemination of inappropriate content.
|
|||
|
|
|
|||
|
|
- **No Default Safety Guarantees**: Unlike standard models, this model has not undergone rigorous safety optimization. huihui.ai bears no responsibility for any consequences arising from its use.
|
|||
|
|
|
|||
|
|
|
|||
|
|
### Donation
|
|||
|
|
|
|||
|
|
If you like it, please click 'like' and follow us for more updates.
|
|||
|
|
You can follow [x.com/support_huihui](https://x.com/support_huihui) to get the latest model information from huihui.ai.
|
|||
|
|
|
|||
|
|
##### Your donation helps us continue our further development and improvement, a cup of coffee can do it.
|
|||
|
|
- bitcoin（BTC):
|
|||
|
|
```
|
|||
|
|
bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge
|
|||
|
|
```
|