初始化项目,由ModelHub XC社区提供模型
Model: kamoo-ai/kamoo-one-135m-instruct Source: Original Platform
This commit is contained in:
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
77
README.md
Normal file
77
README.md
Normal file
@@ -0,0 +1,77 @@
|
||||
---
|
||||
language:
|
||||
- nl
|
||||
license: apache-2.0
|
||||
pipeline_tag: text-generation
|
||||
library_name: transformers
|
||||
base_model: kamoo-ai/kamoo-one-135m
|
||||
tags:
|
||||
- dutch
|
||||
- nederlands
|
||||
- sovereign-ai
|
||||
- small-language-model
|
||||
- instruct
|
||||
- chat
|
||||
---
|
||||
|
||||
# kamoo-one-135m-instruct
|
||||
|
||||
**De instruct-variant van [kamoo-one-135m](https://huggingface.co/kamoo-ai/kamoo-one-135m): volgt Nederlandse instructies, antwoordt in ChatML en stopt netjes na zijn antwoord.**
|
||||
|
||||
Volledige fine-tune (assistant-only loss) van het basismodel op 22.636
|
||||
Nederlandse instructievoorbeelden over negen taaktypes: instructies volgen,
|
||||
kort vraag-antwoord, herschrijven naar B1, samenvatten, meerbeurten-chat,
|
||||
JSON-extractie, gegrond vraag-antwoord, weigeren bij
|
||||
onzekerheid/privacy, en classificeren/routeren. De tokenizer is uitgebreid
|
||||
met ChatML-tokens (vocab 32.832); `<|im_end|>` is het stopteken.
|
||||
|
||||
## Gebruik
|
||||
|
||||
```bash
|
||||
vllm serve kamoo-ai/kamoo-one-135m-instruct --max-model-len 1024
|
||||
```
|
||||
|
||||
```python
|
||||
from transformers import pipeline
|
||||
pipe = pipeline("text-generation", model="kamoo-ai/kamoo-one-135m-instruct")
|
||||
out = pipe([
|
||||
{"role": "system", "content": "Je bent Kamoo, een behulpzame Nederlandstalige AI-assistent. Antwoord helder, feitelijk en beknopt."},
|
||||
{"role": "user", "content": "Herschrijf naar eenvoudig Nederlands: De aanvraag wordt buiten behandeling gesteld."},
|
||||
], max_new_tokens=80)
|
||||
```
|
||||
|
||||
Geef altijd een systeemprompt mee die de taak stuurt; zonder systeemprompt
|
||||
kan het model in de verkeerde taakmodus schieten.
|
||||
|
||||
## Trainingsdetails
|
||||
|
||||
- Basis: kamoo-one-135m (v0.8-corpus, 7,3 mld Nederlandse tokens; zie de
|
||||
model card van het basismodel voor de volledige datastamboom).
|
||||
- SFT: 2 epochs over 2,9M tokens, assistant-only loss, lr 5e-5.
|
||||
- SFT-data: kamoo-sft v0.3-everyday, 22.636 synthetische, getemplate
|
||||
Nederlandse voorbeelden (geen klant- of persoonsdata).
|
||||
|
||||
## Grenzen, eerlijk
|
||||
|
||||
Alles wat voor het basismodel geldt (geen betrouwbare parametrische feiten,
|
||||
alleen Nederlands, 1.024 context), plus specifiek voor deze versie:
|
||||
|
||||
- **Vorm is sterker dan trouw.** Het model houdt zich goed aan formats
|
||||
(JSON-schema's, B1-register, beknopt antwoorden) maar kan bij extractie
|
||||
waarden invullen die niet letterlijk in de invoer staan. Controleer
|
||||
extractie-output tegen de bron; gebruik het niet ongecontroleerd voor
|
||||
gegevensinvoer.
|
||||
- De SFT-data is volledig synthetisch/getemplated (v0.3); een versie met
|
||||
meer gegronde en diverse voorbeelden is in ontwikkeling.
|
||||
|
||||
## English summary
|
||||
|
||||
Instruction-tuned variant of the from-scratch Dutch 135M model
|
||||
kamoo-one-135m. ChatML format, stops correctly, strong at format-following
|
||||
(JSON, register rewriting, summarizing); extraction faithfulness is a known
|
||||
limitation of the v0.3 templated SFT data. Apache-2.0. Built by
|
||||
[kamoo](https://kamoo.nl).
|
||||
|
||||
---
|
||||
|
||||
*kamoo · [kamoo.nl](https://kamoo.nl) · [platform.kamoo.nl](https://platform.kamoo.nl)*
|
||||
23
config.json
Normal file
23
config.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"model_type": "llama",
|
||||
"vocab_size": 32832,
|
||||
"hidden_size": 768,
|
||||
"intermediate_size": 2048,
|
||||
"num_hidden_layers": 16,
|
||||
"num_attention_heads": 12,
|
||||
"num_key_value_heads": 4,
|
||||
"head_dim": 64,
|
||||
"max_position_embeddings": 1024,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_theta": 10000.0,
|
||||
"hidden_act": "silu",
|
||||
"tie_word_embeddings": true,
|
||||
"attention_bias": false,
|
||||
"mlp_bias": false,
|
||||
"bos_token_id": 0,
|
||||
"eos_token_id": 32769,
|
||||
"torch_dtype": "bfloat16"
|
||||
}
|
||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"bos_token_id": 0,
|
||||
"eos_token_id": 32769,
|
||||
"do_sample": true,
|
||||
"temperature": 0.7,
|
||||
"top_p": 0.95,
|
||||
"repetition_penalty": 1.1,
|
||||
"max_new_tokens": 256
|
||||
}
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:327001120c18e8d46c7071886e53b4ea735b5a228f5f97dbec520db49fe2dc74
|
||||
size 251823568
|
||||
163432
tokenizer.json
Normal file
163432
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
9
tokenizer_config.json
Normal file
9
tokenizer_config.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"tokenizer_class": "PreTrainedTokenizerFast",
|
||||
"bos_token": "<|endoftext|>",
|
||||
"eos_token": "<|im_end|>",
|
||||
"unk_token": "<|endoftext|>",
|
||||
"model_max_length": 1024,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"chat_template": "{% for message in messages %}{{ '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n' }}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}"
|
||||
}
|
||||
Reference in New Issue
Block a user