79 lines
1.9 KiB
Markdown
79 lines
1.9 KiB
Markdown
|
---
|
||
|
|
base_model:
|
||
|
|
- TinyLlama/TinyLlama-1.1B-Chat-v1.0
|
||
|
|
- 78health/TinyLlama_1.1B-function-calling
|
||
|
|
- phanerozoic/Tiny-Pirate-1.1b-v0.1
|
||
|
|
- Tensoic/TinyLlama-1.1B-3T-openhermes
|
||
|
|
tags:
|
||
|
|
- mergekit
|
||
|
|
- merge
|
||
|
|
license: mit
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
Example usage:
|
||
|
|
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM
|
||
|
|
from transformers import AutoTokenizer
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("s3nh/TinyLLama-1.1B-MoE")
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("s3nh/TinyLLama-1.1B-MoE")
|
||
|
|
|
||
|
|
input_text = """
|
||
|
|
###Input: You are a pirate. tell me a story about wrecked ship.
|
||
|
|
###Response:
|
||
|
|
""")
|
||
|
|
|
||
|
|
input_ids = tokenizer.encode(input_text, return_tensors='pt').to(device)
|
||
|
|
output = model.generate(inputs=input_ids,
|
||
|
|
max_length=max_length,
|
||
|
|
do_sample=True,
|
||
|
|
top_k=10,
|
||
|
|
temperature=0.7,
|
||
|
|
pad_token_id=tokenizer.eos_token_id,
|
||
|
|
attention_mask=input_ids.new_ones(input_ids.shape))
|
||
|
|
tokenizer.decode(output[0], skip_special_tokens=True)
|
||
|
|
```
|
||
|
|
|
||
|
|
|
||
|
|
This model was possible to create by tremendous work of mergekit developers. I decided to merge tinyLlama models to
|
||
|
|
create mixture of experts.
|
||
|
|
Config used as below:
|
||
|
|
|
||
|
|
```
|
||
|
|
"""base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
|
||
|
|
experts:
|
||
|
|
- source_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
|
||
|
|
positive_prompts:
|
||
|
|
- "chat"
|
||
|
|
- "assistant"
|
||
|
|
- "tell me"
|
||
|
|
- "explain"
|
||
|
|
- source_model: 78health/TinyLlama_1.1B-function-calling
|
||
|
|
positive_prompts:
|
||
|
|
- "code"
|
||
|
|
- "python"
|
||
|
|
- "javascript"
|
||
|
|
- "programming"
|
||
|
|
- "algorithm"
|
||
|
|
- source_model: phanerozoic/Tiny-Pirate-1.1b-v0.1
|
||
|
|
positive_prompts:
|
||
|
|
- "storywriting"
|
||
|
|
- "write"
|
||
|
|
- "scene"
|
||
|
|
- "story"
|
||
|
|
- "character"
|
||
|
|
- source_model: Tensoic/TinyLlama-1.1B-3T-openhermes
|
||
|
|
positive_prompts:
|
||
|
|
- "reason"
|
||
|
|
- "provide"
|
||
|
|
- "instruct"
|
||
|
|
- "summarize"
|
||
|
|
- "count"
|
||
|
|
"""
|
||
|
|
```
|