初始化项目，由ModelHub XC社区提供模型
Model: rtc2022/kirk-tung Source: Original Platform
This commit is contained in:
ModelHub XC
37
.gitattributes
vendored
Normal file
37
.gitattributes
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
quirky.png filter=lfs diff=lfs merge=lfs -text
|
||||
kirkcr.png filter=lfs diff=lfs merge=lfs -text
|
||||
108
README.md
Normal file
108
README.md
Normal file
@@ -0,0 +1,108 @@
|
||||
---
|
||||
license: cc-by-sa-4.0
|
||||
datasets:
|
||||
- rtc2022/corpus-kirkensis
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
tags:
|
||||
- weird
|
||||
- tiny
|
||||
- llama
|
||||
---
|
||||
|
||||
# Kirk Tung
|
||||
|
||||
|
||||
|
||||
Kirk Tung is a tiny language model trained on the [Corpus Kirkensis](https://huggingface.co/datasets/rtc2022/corpus-kirkensis), meant to produce
|
||||
output which is entertainingly silly as an experiment in using the Llama architecture.
|
||||
|
||||
Tung has a Libre and Open dataset, meaning that the content it was trained on is nearly 100% under a free license, with few exceptions.
|
||||
|
||||
## Technical details
|
||||
|
||||
Kirk Tung was trained on 1.1B tokens over the course of 4 hours on a single A100 GPU.
|
||||
|
||||
| Name | Value |
|
||||
| -------------- | ----- |
|
||||
| Architecture | Llama |
|
||||
| Context Length | 1024 |
|
||||
| Vocab size | 32606 |
|
||||
| RoPE theta | 10000 |
|
||||
|
||||
```
|
||||
num_attention_heads = 8
|
||||
num_key_value_heads = 2
|
||||
num_hidden_layers = 10
|
||||
hidden_size = 512
|
||||
intermediate_size = 1728
|
||||
|
||||
tie_word_embeddings = True
|
||||
```
|
||||
|
||||
### Training hyperparameters
|
||||
|
||||
| Name | Value |
|
||||
| --------------------------- | ------ |
|
||||
| Per-device batch size | 12 |
|
||||
| Learning rate | 5e-4 |
|
||||
| Learning scheduler | Cosine |
|
||||
| Weight decay | 0.1 |
|
||||
| Epochs | 1 |
|
||||
| Gradient accumulation steps | 8 |
|
||||
|
||||
## Example completions
|
||||
|
||||
**Prompt:** "Once upon a time,"
|
||||
|
||||
> Once upon a time, the two brothers would be able to use their powers to defeat the first."
|
||||
>
|
||||
> Rosa was a very wise man. He had a love for a woman and a friend. He was so strong, he felt very happy that he had to be.
|
||||
>
|
||||
> "You never thought you could be like you. I am a woman, a man who never wanted to go to the house, and never wanted to go to the house that night."
|
||||
>
|
||||
> The two brothers sat in silence. The three brothers had a good laugh. "Birds, I can see you. Look for me."
|
||||
>
|
||||
> The brothers tried to raise their hands. "I won't," they say. "You don't have to be a mother. I will be a mother. You don't have to be a mother," she says.
|
||||
>
|
||||
> The brothers moved into their respective houses. The brothers were very happy. They were always happy. They always thought of it.
|
||||
>
|
||||
> The brothers were very happy. The brothers knew that they would never have to go back home. They did not want to be friends. They never wanted to be alone again. The brothers were always happy. But they were always happy. They had to be happy. They had no friendship. They would
|
||||
|
||||
**Prompt:** "Artificial intelligence is"
|
||||
|
||||
> Artificial Intelligence is the source of these data.
|
||||
>
|
||||
> Witness the following:
|
||||
>
|
||||
> "1. The United Nations has been stricter in internal affairs, and the agenda has been growing; the people have become more and more bankrupt, the world a power under international law. The foreign powers have become many times more bankrupt, and the foreign powers have become much more constraining of all their financial resources. It has allowed for still more operations than any other nation in the world. The U.S. Government has found its way into international agreements, financial concessions and a continuous flow of railways locomotives into the borders. This solution has been seen as possible two long colonization attempts."https://www.theguardian.com/world/2014/sep/10/go-topics-for-the-existing-response-event-at-the-united-from-the-loyal-response-president-on/
|
||||
>
|
||||
> 2. The First International Commission for the Investigation of National Socialist Crimes
|
||||
>
|
||||
> In October 1945, before the Great Purge, Germany invaded Poland from the East (not to be confused with the Red Army), the Central Powers, and the Baltic states. (Silesia was not a neutral
|
||||
|
||||
## How to use
|
||||
|
||||
The easiest way to use Kirk Tung programatically is with a pipeline in HF
|
||||
transformers.
|
||||
|
||||
```
|
||||
$ pip install transformers torch
|
||||
```
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import pipeline
|
||||
|
||||
device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
|
||||
pipe = pipeline(
|
||||
"text-generation", model="rtc2022/kirk-tung", device=device
|
||||
)
|
||||
```
|
||||
|
||||
## Support the creator
|
||||
|
||||
Support the creation of more models like this one by directly supporting the main creator:
|
||||
|
||||
[Support Lopes on Ko-Fi](https://ko-fi.com/hvilaverde)
|
||||
34
config.json
Normal file
34
config.json
Normal file
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 1,
|
||||
"dtype": "float32",
|
||||
"eos_token_id": 2,
|
||||
"head_dim": 64,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 512,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 1728,
|
||||
"max_position_embeddings": 1024,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 8,
|
||||
"num_hidden_layers": 11,
|
||||
"num_key_value_heads": 2,
|
||||
"pad_token_id": 5,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_parameters": {
|
||||
"factor": 2.0,
|
||||
"rope_theta": 10000,
|
||||
"rope_type": "dynamic",
|
||||
"type": "dynamic"
|
||||
},
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "5.0.0",
|
||||
"use_cache": false,
|
||||
"vocab_size": 32606
|
||||
}
|
||||
12
generation_config.json
Normal file
12
generation_config.json
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": [
|
||||
2
|
||||
],
|
||||
"output_attentions": false,
|
||||
"output_hidden_states": false,
|
||||
"pad_token_id": 5,
|
||||
"transformers_version": "5.0.0",
|
||||
"use_cache": true
|
||||
}
|
||||
3
kirkcr.png
Normal file
3
kirkcr.png
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:012c9a616cf3f87715d68da5f55189f36d997e7796abb67d6a234729e4dd062d
|
||||
size 1167574
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ea9998e6e08a922090ace4a83686bb34d3c59490c9c914141ada7201416bf68a
|
||||
size 212456360
|
||||
3
quirky.png
Normal file
3
quirky.png
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:db24b285978ee78f7523a241e586fc4a88f6bee1aa26cb429e6e2f600cfe6471
|
||||
size 13155168
|
||||
162244
tokenizer.json
Normal file
162244
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
9
tokenizer_config.json
Normal file
9
tokenizer_config.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"backend": "tokenizers",
|
||||
"bos_token": "<s>",
|
||||
"eos_token": "</s>",
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"pad_token": "<pad>",
|
||||
"tokenizer_class": "TokenizersBackend",
|
||||
"unk_token": "<unk>"
|
||||
}
|
||||
Reference in New Issue
Block a user