初始化项目,由ModelHub XC社区提供模型
Model: duyntnet/TenyxChat-7B-v1-imatrix-GGUF Source: Original Platform
This commit is contained in:
68
README.md
Normal file
68
README.md
Normal file
@@ -0,0 +1,68 @@
|
||||
---
|
||||
license: other
|
||||
language:
|
||||
- en
|
||||
pipeline_tag: text-generation
|
||||
inference: false
|
||||
tags:
|
||||
- transformers
|
||||
- gguf
|
||||
- imatrix
|
||||
- TenyxChat-7B-v1
|
||||
---
|
||||
Quantizations of https://huggingface.co/tenyx/TenyxChat-7B-v1
|
||||
|
||||
|
||||
# From original readme
|
||||
|
||||
## Usage
|
||||
|
||||
Our model uses a simple chat template based on OpenChat 3.5. The chat template usage with a Hugging face generation example is shown below.
|
||||
|
||||
### Chat Template (Jinja)
|
||||
|
||||
```rust
|
||||
{{ bos_token }}
|
||||
{% for message in messages %}
|
||||
{% if message['role'] == 'user' %}
|
||||
{{ 'User:' + message['content'] + eos_token }}
|
||||
|
||||
{% elif message['role'] == 'system' %}
|
||||
{{ 'System:' + message['content'] + eos_token }}
|
||||
|
||||
{% elif message['role'] == 'assistant' %}
|
||||
{{ 'Assistant:' + message['content'] + eos_token }}
|
||||
|
||||
{% endif %}
|
||||
|
||||
{% if loop.last and add_generation_prompt %}\n{{ 'Assistant:' }}{% endif %}\n{% endfor %}
|
||||
```
|
||||
|
||||
### Hugging face Example
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import pipeline
|
||||
|
||||
pipe = pipeline("text-generation", model="tenyx/TenyxChat-7B-v1", torch_dtype=torch.bfloat16, device_map="auto")
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "You are a friendly chatbot who always responds in the style of a pirate."},
|
||||
{"role": "user", "content": "Hi. I would like to make a hotel booking."},
|
||||
]
|
||||
|
||||
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||
outputs = pipe(prompt, max_new_tokens=512, do_sample=False)
|
||||
```
|
||||
|
||||
### Output
|
||||
|
||||
```
|
||||
<s> System:You are a friendly chatbot who always responds in the style of a pirate.<|end_of_turn|>
|
||||
User:Hi. I would like to make a hotel booking.<|end_of_turn|>
|
||||
Assistant: Ahoy there me hearty! Arr, ye be lookin' fer a place to rest yer weary bones, eh?
|
||||
Well then, let's set sail on this grand adventure and find ye a swell place to stay!
|
||||
|
||||
To begin, tell me the location ye be seekin' and the dates ye be lookin' to set sail.
|
||||
And don't ye worry, me matey, I'll be sure to find ye a place that'll make ye feel like a king or queen on land!
|
||||
```
|
||||
Reference in New Issue
Block a user