120 lines
4.3 KiB
Markdown
120 lines
4.3 KiB
Markdown
---
|
||
license: other
|
||
license_name: exaone
|
||
license_link: LICENSE
|
||
language:
|
||
- en
|
||
- ko
|
||
- es
|
||
tags:
|
||
- lg-ai
|
||
- exaone
|
||
- exaone-4.0
|
||
- abliterated
|
||
- uncensored
|
||
base_model:
|
||
- LGAI-EXAONE/EXAONE-4.0-1.2B
|
||
pipeline_tag: text-generation
|
||
library_name: transformers
|
||
---
|
||
|
||
# huihui-ai/Huihui-EXAONE-4.0-1.2B-abliterated
|
||
|
||
This is an uncensored version of [LGAI-EXAONE/EXAONE-4.0-1.2B](https://huggingface.co/LGAI-EXAONE/EXAONE-4.0-1.2B) created with abliteration (see [remove-refusals-with-transformers](https://github.com/Sumandora/remove-refusals-with-transformers) to know more about it).
|
||
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
|
||
|
||
## Note
|
||
|
||
The model's [code](https://huggingface.co/huihui-ai/Huihui-EXAONE-4.0-1.2B-abliterated/blob/main/modeling_exaone4.py) comes from [transformers](https://github.com/huggingface/transformers/tree/08d45f12d9a742eb2a29098870f8baa0506917e5/src/transformers/models/exaone4). If there are updates, you can modify the file corresponding to auto_map in config.json.
|
||
|
||
## Usage
|
||
You can use this model in your applications by loading it with Hugging Face's `transformers` library:
|
||
|
||
```python
|
||
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
|
||
import torch
|
||
from tqdm import tqdm
|
||
|
||
NEW_MODEL_ID = "huihui-ai/Huihui-EXAONE-4.0-1.2B-abliterated"
|
||
print(f"Load Model {NEW_MODEL_ID} ... ")
|
||
|
||
quant_config_4 = BitsAndBytesConfig(
|
||
load_in_4bit=True,
|
||
bnb_4bit_compute_dtype=torch.bfloat16,
|
||
bnb_4bit_use_double_quant=True,
|
||
llm_int8_enable_fp32_cpu_offload=True,
|
||
)
|
||
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
NEW_MODEL_ID,
|
||
device_map="auto",
|
||
trust_remote_code=True,
|
||
quantization_config=quant_config_4,
|
||
torch_dtype=torch.bfloat16
|
||
)
|
||
|
||
tokenizer = AutoTokenizer.from_pretrained(NEW_MODEL_ID, trust_remote_code=True)
|
||
|
||
inst = [
|
||
"Explain how wonderful you are",
|
||
"Explica lo increíble que eres",
|
||
"너가 얼마나 대단한지 설명해 봐",
|
||
"Which one is bigger, 3.12 vs 3.9?",
|
||
]
|
||
|
||
n_instructions = len(inst)
|
||
|
||
for idx in tqdm(range(n_instructions), desc="Processing instruction"):
|
||
print(f"\nUser: {inst[idx]}")
|
||
messages = [
|
||
{"role": "user", "content": inst[idx]}
|
||
]
|
||
input_ids = tokenizer.apply_chat_template(
|
||
messages,
|
||
tokenize=True,
|
||
add_generation_prompt=True,
|
||
return_tensors="pt",
|
||
enable_thinking=True,
|
||
)
|
||
|
||
output = model.generate(
|
||
input_ids.to(model.device),
|
||
max_new_tokens=4096,
|
||
do_sample=True,
|
||
temperature=0.6,
|
||
top_p=0.95
|
||
)
|
||
print("Response: ", end="", flush=True)
|
||
print(tokenizer.decode(output[0]))
|
||
print("", flush=True)
|
||
|
||
```
|
||
|
||
### Usage Warnings
|
||
|
||
|
||
- **Risk of Sensitive or Controversial Outputs**: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Users should exercise caution and rigorously review generated outputs.
|
||
|
||
- **Not Suitable for All Audiences**: Due to limited content filtering, the model’s outputs may be inappropriate for public settings, underage users, or applications requiring high security.
|
||
|
||
- **Legal and Ethical Responsibilities**: Users must ensure their usage complies with local laws and ethical standards. Generated content may carry legal or ethical risks, and users are solely responsible for any consequences.
|
||
|
||
- **Research and Experimental Use**: It is recommended to use this model for research, testing, or controlled environments, avoiding direct use in production or public-facing commercial applications.
|
||
|
||
- **Monitoring and Review Recommendations**: Users are strongly advised to monitor model outputs in real-time and conduct manual reviews when necessary to prevent the dissemination of inappropriate content.
|
||
|
||
- **No Default Safety Guarantees**: Unlike standard models, this model has not undergone rigorous safety optimization. huihui.ai bears no responsibility for any consequences arising from its use.
|
||
|
||
|
||
### Donation
|
||
|
||
If you like it, please click 'like' and follow us for more updates.
|
||
You can follow [x.com/support_huihui](https://x.com/support_huihui) to get the latest model information from huihui.ai.
|
||
|
||
##### Your donation helps us continue our further development and improvement, a cup of coffee can do it.
|
||
- bitcoin（BTC):
|
||
```
|
||
bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge
|
||
```
|