OvercastLab/Quark-50m
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: OvercastLab/Quark-50m
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-05 08:06:17 +08:00
commit c886f7fbd6
14 changed files with 490232 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
.ipynb_checkpoints/config-checkpoint.json Normal file
View File

@@ -0,0 +1,14 @@
{
"vocab_size": 49152,
"d_model": 384,
"n_heads": 6,
"n_kv_heads": 2,
"n_layers": 24,
"d_ff": 1024,
"head_dim": 64,
"max_seq_len": 2048,
"rope_theta": 10000.0,
"rms_eps": 1e-05,
"qkv_bias": true,
"dropout": 0.0
}