初始化项目,由ModelHub XC社区提供模型
Model: kamoo-ai/kamoo-one-135m-instruct Source: Original Platform
This commit is contained in:
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
77
README.md
Normal file
77
README.md
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
---
|
||||||
|
language:
|
||||||
|
- nl
|
||||||
|
license: apache-2.0
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
library_name: transformers
|
||||||
|
base_model: kamoo-ai/kamoo-one-135m
|
||||||
|
tags:
|
||||||
|
- dutch
|
||||||
|
- nederlands
|
||||||
|
- sovereign-ai
|
||||||
|
- small-language-model
|
||||||
|
- instruct
|
||||||
|
- chat
|
||||||
|
---
|
||||||
|
|
||||||
|
# kamoo-one-135m-instruct
|
||||||
|
|
||||||
|
**De instruct-variant van [kamoo-one-135m](https://huggingface.co/kamoo-ai/kamoo-one-135m): volgt Nederlandse instructies, antwoordt in ChatML en stopt netjes na zijn antwoord.**
|
||||||
|
|
||||||
|
Volledige fine-tune (assistant-only loss) van het basismodel op 22.636
|
||||||
|
Nederlandse instructievoorbeelden over negen taaktypes: instructies volgen,
|
||||||
|
kort vraag-antwoord, herschrijven naar B1, samenvatten, meerbeurten-chat,
|
||||||
|
JSON-extractie, gegrond vraag-antwoord, weigeren bij
|
||||||
|
onzekerheid/privacy, en classificeren/routeren. De tokenizer is uitgebreid
|
||||||
|
met ChatML-tokens (vocab 32.832); `<|im_end|>` is het stopteken.
|
||||||
|
|
||||||
|
## Gebruik
|
||||||
|
|
||||||
|
```bash
|
||||||
|
vllm serve kamoo-ai/kamoo-one-135m-instruct --max-model-len 1024
|
||||||
|
```
|
||||||
|
|
||||||
|
```python
|
||||||
|
from transformers import pipeline
|
||||||
|
pipe = pipeline("text-generation", model="kamoo-ai/kamoo-one-135m-instruct")
|
||||||
|
out = pipe([
|
||||||
|
{"role": "system", "content": "Je bent Kamoo, een behulpzame Nederlandstalige AI-assistent. Antwoord helder, feitelijk en beknopt."},
|
||||||
|
{"role": "user", "content": "Herschrijf naar eenvoudig Nederlands: De aanvraag wordt buiten behandeling gesteld."},
|
||||||
|
], max_new_tokens=80)
|
||||||
|
```
|
||||||
|
|
||||||
|
Geef altijd een systeemprompt mee die de taak stuurt; zonder systeemprompt
|
||||||
|
kan het model in de verkeerde taakmodus schieten.
|
||||||
|
|
||||||
|
## Trainingsdetails
|
||||||
|
|
||||||
|
- Basis: kamoo-one-135m (v0.8-corpus, 7,3 mld Nederlandse tokens; zie de
|
||||||
|
model card van het basismodel voor de volledige datastamboom).
|
||||||
|
- SFT: 2 epochs over 2,9M tokens, assistant-only loss, lr 5e-5.
|
||||||
|
- SFT-data: kamoo-sft v0.3-everyday, 22.636 synthetische, getemplate
|
||||||
|
Nederlandse voorbeelden (geen klant- of persoonsdata).
|
||||||
|
|
||||||
|
## Grenzen, eerlijk
|
||||||
|
|
||||||
|
Alles wat voor het basismodel geldt (geen betrouwbare parametrische feiten,
|
||||||
|
alleen Nederlands, 1.024 context), plus specifiek voor deze versie:
|
||||||
|
|
||||||
|
- **Vorm is sterker dan trouw.** Het model houdt zich goed aan formats
|
||||||
|
(JSON-schema's, B1-register, beknopt antwoorden) maar kan bij extractie
|
||||||
|
waarden invullen die niet letterlijk in de invoer staan. Controleer
|
||||||
|
extractie-output tegen de bron; gebruik het niet ongecontroleerd voor
|
||||||
|
gegevensinvoer.
|
||||||
|
- De SFT-data is volledig synthetisch/getemplated (v0.3); een versie met
|
||||||
|
meer gegronde en diverse voorbeelden is in ontwikkeling.
|
||||||
|
|
||||||
|
## English summary
|
||||||
|
|
||||||
|
Instruction-tuned variant of the from-scratch Dutch 135M model
|
||||||
|
kamoo-one-135m. ChatML format, stops correctly, strong at format-following
|
||||||
|
(JSON, register rewriting, summarizing); extraction faithfulness is a known
|
||||||
|
limitation of the v0.3 templated SFT data. Apache-2.0. Built by
|
||||||
|
[kamoo](https://kamoo.nl).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*kamoo · [kamoo.nl](https://kamoo.nl) · [platform.kamoo.nl](https://platform.kamoo.nl)*
|
||||||
23
config.json
Normal file
23
config.json
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"architectures": [
|
||||||
|
"LlamaForCausalLM"
|
||||||
|
],
|
||||||
|
"model_type": "llama",
|
||||||
|
"vocab_size": 32832,
|
||||||
|
"hidden_size": 768,
|
||||||
|
"intermediate_size": 2048,
|
||||||
|
"num_hidden_layers": 16,
|
||||||
|
"num_attention_heads": 12,
|
||||||
|
"num_key_value_heads": 4,
|
||||||
|
"head_dim": 64,
|
||||||
|
"max_position_embeddings": 1024,
|
||||||
|
"rms_norm_eps": 1e-05,
|
||||||
|
"rope_theta": 10000.0,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"tie_word_embeddings": true,
|
||||||
|
"attention_bias": false,
|
||||||
|
"mlp_bias": false,
|
||||||
|
"bos_token_id": 0,
|
||||||
|
"eos_token_id": 32769,
|
||||||
|
"torch_dtype": "bfloat16"
|
||||||
|
}
|
||||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
{
|
||||||
|
"bos_token_id": 0,
|
||||||
|
"eos_token_id": 32769,
|
||||||
|
"do_sample": true,
|
||||||
|
"temperature": 0.7,
|
||||||
|
"top_p": 0.95,
|
||||||
|
"repetition_penalty": 1.1,
|
||||||
|
"max_new_tokens": 256
|
||||||
|
}
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:327001120c18e8d46c7071886e53b4ea735b5a228f5f97dbec520db49fe2dc74
|
||||||
|
size 251823568
|
||||||
163432
tokenizer.json
Normal file
163432
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
9
tokenizer_config.json
Normal file
9
tokenizer_config.json
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
{
|
||||||
|
"tokenizer_class": "PreTrainedTokenizerFast",
|
||||||
|
"bos_token": "<|endoftext|>",
|
||||||
|
"eos_token": "<|im_end|>",
|
||||||
|
"unk_token": "<|endoftext|>",
|
||||||
|
"model_max_length": 1024,
|
||||||
|
"clean_up_tokenization_spaces": false,
|
||||||
|
"chat_template": "{% for message in messages %}{{ '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n' }}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}"
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user