38 lines
912 B
Markdown
38 lines
912 B
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
datasets:
|
||
|
|
- qikp/sillychat
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
base_model:
|
||
|
|
- cerebras/Cerebras-GPT-111M
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
library_name: transformers
|
||
|
|
new_version: qikp/hummingbird-2.7-110m
|
||
|
|
---
|
||
|
|
|
||
|
|
# Hummingbird
|
||
|
|
|
||
|
|
🎉 You are looking at Hummingbird 2.6, which uses different and slightly better datasets!
|
||
|
|
|
||
|
|
Hummingbird is a Cerebras-GPT derivative trained to be conversational.
|
||
|
|
|
||
|
|
## Training
|
||
|
|
|
||
|
|
The model was trained using 500 steps, and 4 batch size.
|
||
|
|
|
||
|
|
### Datasets
|
||
|
|
|
||
|
|
The training corpus is made up of a collated [qikp/sillychat](https://huggingface.co/datasets/qikp/sillychat) to be multi-turn.
|
||
|
|
|
||
|
|
### Chat template
|
||
|
|
|
||
|
|
The Zephyr chat template was used.
|
||
|
|
|
||
|
|
## Limitations
|
||
|
|
|
||
|
|
The model frequently outputs incorrect information, confirmation with a larger, mature model is advised.
|
||
|
|
|
||
|
|
## Benchmark
|
||
|
|
|
||
|
|
This model was benchmarked and compared using embeddings. See the results [here](https://codeberg.org/qikp/benchmarks).
|