41 lines
869 B
Markdown
41 lines
869 B
Markdown
|
---
|
||
|
|
language:
|
||
|
|
- fr
|
||
|
|
license: apache-2.0
|
||
|
|
tags:
|
||
|
|
- llama
|
||
|
|
- french
|
||
|
|
- calcyon
|
||
|
|
- technical
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# cALCYON-1B
|
||
|
|
|
||
|
|
Modele de langage francophone technique (1.152B parametres).
|
||
|
|
|
||
|
|
## Architecture
|
||
|
|
- LLaMA-style decoder, 22 couches, GQA 16/8 heads, SwiGLU
|
||
|
|
- Tokenizer SentencePiece BPE 32k optimise francais
|
||
|
|
- Contexte 8192 tokens, RoPE theta=500000
|
||
|
|
|
||
|
|
## Domaines
|
||
|
|
Aeronautique, electricite/electrotechnique, hydraulique/hydroelectrique,
|
||
|
|
developpement Python/C++, hardware, FPV, LiDAR
|
||
|
|
|
||
|
|
## Prompt format (ChatML)
|
||
|
|
```
|
||
|
|
<|system|>
|
||
|
|
Tu es cALCYON.<|end|>
|
||
|
|
<|user|>
|
||
|
|
Question<|end|>
|
||
|
|
<|assistant|>
|
||
|
|
```
|
||
|
|
|
||
|
|
## Entrainement
|
||
|
|
SFT LoRA r=32 sur Wikipedia-FR + OpenHermes-FR + donnees techniques.
|
||
|
|
GPU : NVIDIA RTX 5080 16GB.
|
||
|
|
|
||
|
|
## Usage LM Studio / Ollama
|
||
|
|
Telecharger `calcyon-1b-q4_k_m.gguf` (709 MB) et charger dans LM Studio ou Ollama.
|