初始化项目，由ModelHub XC社区提供模型
Model: afrideva/phine-2-v0-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
42
.gitattributes
vendored
Normal file
42
.gitattributes
vendored
Normal file
@@ -0,0 +1,42 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
phine-2-v0.fp16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
phine-2-v0.q2_k.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
phine-2-v0.q3_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
phine-2-v0.q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
phine-2-v0.q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
phine-2-v0.q6_k.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
phine-2-v0.q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
193
README.md
Normal file
193
README.md
Normal file
@@ -0,0 +1,193 @@
|
||||
---
|
||||
base_model: freecs/phine-2-v0
|
||||
datasets:
|
||||
- vicgalle/alpaca-gpt4
|
||||
inference: false
|
||||
license: unknown
|
||||
model_creator: freecs
|
||||
model_name: phine-2-v0
|
||||
pipeline_tag: text-generation
|
||||
quantized_by: afrideva
|
||||
tags:
|
||||
- gguf
|
||||
- ggml
|
||||
- quantized
|
||||
- q2_k
|
||||
- q3_k_m
|
||||
- q4_k_m
|
||||
- q5_k_m
|
||||
- q6_k
|
||||
- q8_0
|
||||
---
|
||||
# freecs/phine-2-v0-GGUF
|
||||
|
||||
Quantized GGUF model files for [phine-2-v0](https://huggingface.co/freecs/phine-2-v0) from [freecs](https://huggingface.co/freecs)
|
||||
|
||||
|
||||
| Name | Quant method | Size |
|
||||
| ---- | ---- | ---- |
|
||||
| [phine-2-v0.fp16.gguf](https://huggingface.co/afrideva/phine-2-v0-GGUF/resolve/main/phine-2-v0.fp16.gguf) | fp16 | 5.56 GB |
|
||||
| [phine-2-v0.q2_k.gguf](https://huggingface.co/afrideva/phine-2-v0-GGUF/resolve/main/phine-2-v0.q2_k.gguf) | q2_k | 1.17 GB |
|
||||
| [phine-2-v0.q3_k_m.gguf](https://huggingface.co/afrideva/phine-2-v0-GGUF/resolve/main/phine-2-v0.q3_k_m.gguf) | q3_k_m | 1.48 GB |
|
||||
| [phine-2-v0.q4_k_m.gguf](https://huggingface.co/afrideva/phine-2-v0-GGUF/resolve/main/phine-2-v0.q4_k_m.gguf) | q4_k_m | 1.79 GB |
|
||||
| [phine-2-v0.q5_k_m.gguf](https://huggingface.co/afrideva/phine-2-v0-GGUF/resolve/main/phine-2-v0.q5_k_m.gguf) | q5_k_m | 2.07 GB |
|
||||
| [phine-2-v0.q6_k.gguf](https://huggingface.co/afrideva/phine-2-v0-GGUF/resolve/main/phine-2-v0.q6_k.gguf) | q6_k | 2.29 GB |
|
||||
| [phine-2-v0.q8_0.gguf](https://huggingface.co/afrideva/phine-2-v0-GGUF/resolve/main/phine-2-v0.q8_0.gguf) | q8_0 | 2.96 GB |
|
||||
|
||||
|
||||
|
||||
## Original Model Card:
|
||||
---
|
||||
# Model Card: Phine-2-v0
|
||||
|
||||
## Overview
|
||||
|
||||
- **Model Name:** Phine-2
|
||||
- **Base Model:** Phi-2 (Microsoft model)
|
||||
- **Created By:** [GR](https://twitter.com/gr_username)
|
||||
- **Donations Link:** [Click Me](https://www.buymeacoffee.com/gr.0)
|
||||
|
||||
## Code Usage
|
||||
|
||||
To try Phine, use the following Python code snippet:
|
||||
|
||||
```python
|
||||
#######################
|
||||
'''
|
||||
Name: Phine Inference
|
||||
License: MIT
|
||||
'''
|
||||
#######################
|
||||
|
||||
|
||||
##### Dependencies
|
||||
|
||||
""" IMPORTANT: Uncomment the following line if you are in a Colab/Notebook environment """
|
||||
|
||||
#!pip install gradio einops accelerate bitsandbytes transformers
|
||||
|
||||
#####
|
||||
|
||||
import gradio as gr
|
||||
import transformers
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
import torch
|
||||
import random
|
||||
import re
|
||||
|
||||
def cut_text_after_last_token(text, token):
|
||||
|
||||
last_occurrence = text.rfind(token)
|
||||
|
||||
if last_occurrence != -1:
|
||||
result = text[last_occurrence + len(token):].strip()
|
||||
return result
|
||||
else:
|
||||
return None
|
||||
|
||||
|
||||
class _SentinelTokenStoppingCriteria(transformers.StoppingCriteria):
|
||||
|
||||
def __init__(self, sentinel_token_ids: torch.LongTensor,
|
||||
starting_idx: int):
|
||||
transformers.StoppingCriteria.__init__(self)
|
||||
self.sentinel_token_ids = sentinel_token_ids
|
||||
self.starting_idx = starting_idx
|
||||
|
||||
def __call__(self, input_ids: torch.LongTensor,
|
||||
_scores: torch.FloatTensor) -> bool:
|
||||
for sample in input_ids:
|
||||
trimmed_sample = sample[self.starting_idx:]
|
||||
|
||||
if trimmed_sample.shape[-1] < self.sentinel_token_ids.shape[-1]:
|
||||
continue
|
||||
|
||||
for window in trimmed_sample.unfold(
|
||||
0, self.sentinel_token_ids.shape[-1], 1):
|
||||
if torch.all(torch.eq(self.sentinel_token_ids, window)):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
model_path = 'freecs/phine-2-v0'
|
||||
|
||||
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True, load_in_4bit=False, torch_dtype=torch.float16).to(device) #remove .to() if load_in_4/8bit = True
|
||||
|
||||
sys_message = "You are an AI assistant named Phine developed by FreeCS.org. You are polite and smart." #System Message
|
||||
|
||||
def phine(message, history, temperature, top_p, top_k, repetition_penalty):
|
||||
|
||||
|
||||
|
||||
n = 0
|
||||
context = ""
|
||||
if history and len(history) > 0:
|
||||
|
||||
for x in history:
|
||||
for h in x:
|
||||
if n%2 == 0:
|
||||
context+=f"""\n<|prompt|>{h}\n"""
|
||||
else:
|
||||
context+=f"""<|response|>{h}"""
|
||||
n+=1
|
||||
else:
|
||||
|
||||
context = ""
|
||||
|
||||
|
||||
|
||||
|
||||
prompt = f"""\n<|system|>{sys_message}"""+context+"\n<|prompt|>"+message+"<|endoftext|>\n<|response|>"
|
||||
tokenized = tokenizer(prompt, return_tensors="pt").to(device)
|
||||
|
||||
|
||||
stopping_criteria_list = transformers.StoppingCriteriaList([
|
||||
_SentinelTokenStoppingCriteria(
|
||||
sentinel_token_ids=tokenizer(
|
||||
"<|endoftext|>",
|
||||
add_special_tokens=False,
|
||||
return_tensors="pt",
|
||||
).input_ids.to(device),
|
||||
starting_idx=tokenized.input_ids.shape[-1])
|
||||
])
|
||||
|
||||
|
||||
token = model.generate(**tokenized,
|
||||
stopping_criteria=stopping_criteria_list,
|
||||
do_sample=True,
|
||||
max_length=2048, temperature=temperature, top_p=top_p, top_k = top_k, repetition_penalty = repetition_penalty
|
||||
)
|
||||
|
||||
completion = tokenizer.decode(token[0], skip_special_tokens=False)
|
||||
token = "<|response|>"
|
||||
res = cut_text_after_last_token(completion, token)
|
||||
return res.replace('<|endoftext|>', '')
|
||||
demo = gr.ChatInterface(phine,
|
||||
additional_inputs=[
|
||||
gr.Slider(0.1, 2.0, label="temperature", value=0.5),
|
||||
gr.Slider(0.1, 2.0, label="Top P", value=0.9),
|
||||
gr.Slider(1, 500, label="Top K", value=50),
|
||||
gr.Slider(0.1, 2.0, label="Repetition Penalty", value=1.15)
|
||||
]
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
demo.queue().launch(share=True, debug=True) #If debug=True causes problems you can set it to False
|
||||
```
|
||||
|
||||
## Contact
|
||||
|
||||
For inquiries, collaboration opportunities, or additional information, reach out to me on Twitter: [gr](https://twitter.com/gr_username).
|
||||
|
||||
## Disclaimer
|
||||
|
||||
As of now, I have not applied Reinforcement Learning from Human Feedback (RLHF). Due to this, the model may generate unexpected or potentially unethical outputs.
|
||||
|
||||
---
|
||||
3
phine-2-v0.fp16.gguf
Normal file
3
phine-2-v0.fp16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5ca815195f3ba0e6478b3cb8203c184afc5d822ee3a10f751fe3074967738005
|
||||
size 5563088672
|
||||
3
phine-2-v0.q2_k.gguf
Normal file
3
phine-2-v0.q2_k.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:58d9baacd93e645fb119137f339b81ed1e73fa27b46892250ccd3aae17b1f61a
|
||||
size 1173610336
|
||||
3
phine-2-v0.q3_k_m.gguf
Normal file
3
phine-2-v0.q3_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1b5252145007598b0cdcb93c7a5a0093728f1720bd09ebf06bdb149a16fae070
|
||||
size 1480195936
|
||||
3
phine-2-v0.q4_k_m.gguf
Normal file
3
phine-2-v0.q4_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6f6627312c0b5c701d81c2044daef55704c8ee001a811917edbbafd3fcdbe40f
|
||||
size 1789239136
|
||||
3
phine-2-v0.q5_k_m.gguf
Normal file
3
phine-2-v0.q5_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bf69e54234a22f9ca78c829b5be07ba13dd18330b98bfc56f835690b6827dcef
|
||||
size 2072682336
|
||||
3
phine-2-v0.q6_k.gguf
Normal file
3
phine-2-v0.q6_k.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:fe45ef9911ae39fbabecc218fb6032736ff73ca4363f46478e1dc9105cbcf544
|
||||
size 2285059936
|
||||
3
phine-2-v0.q8_0.gguf
Normal file
3
phine-2-v0.q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5b41f23e5eb5b9616ffdc118e26e386e5312d778d4d8b971429e13b185a90e77
|
||||
size 2958032736
|
||||
Reference in New Issue
Block a user