初始化项目,由ModelHub XC社区提供模型
Model: diverWayne/mikky-64m Source: Original Platform
This commit is contained in:
108
README.md
Normal file
108
README.md
Normal file
@@ -0,0 +1,108 @@
|
||||
---
|
||||
language:
|
||||
- zh
|
||||
- en
|
||||
license: apache-2.0
|
||||
pipeline_tag: text-generation
|
||||
library_name: transformers
|
||||
tags:
|
||||
- text-generation
|
||||
- gguf
|
||||
- safetensors
|
||||
- minimind
|
||||
- llama-cpp
|
||||
- qwen3-compatible
|
||||
base_model: jingyaogong/minimind
|
||||
datasets:
|
||||
- jingyaogong/minimind_dataset
|
||||
---
|
||||
|
||||
# mikky-64m
|
||||
|
||||
**mikky-64m** is a 63,912,192-parameter small language model named **mikky**.
|
||||
It was trained by **HUANG JUNZHE 黄俊哲** with the `minimind-scratch` codebase, based on the MiniMind project/data format.
|
||||
|
||||
This release is intended as a compact learning and experimentation checkpoint for local inference, model-format conversion, and small-model alignment workflows.
|
||||
|
||||
## Training Line
|
||||
|
||||
The released checkpoint uses the completed alignment path:
|
||||
|
||||
`pretrain -> SFT -> mikky LoRA identity SFT -> DPO`
|
||||
|
||||
GRPO was only run as a probe and is **not** used as the final release checkpoint.
|
||||
PPO was skipped because the local reward signal was not strong enough to justify another RL stage.
|
||||
|
||||
## Identity
|
||||
|
||||
The model identity/persona is:
|
||||
|
||||
- Name: **mikky**
|
||||
- Trainer: **HUANG JUNZHE 黄俊哲**
|
||||
- Origin: a small-parameter model trained from this MiniMind-based scratch project
|
||||
|
||||
## Files
|
||||
|
||||
- `mikky-64m.pth`: native `minimind_scratch` state dict, BF16 tensors.
|
||||
- `model.safetensors`: Qwen3-compatible Hugging Face tensor names, BF16 tensors.
|
||||
- `mikky-64m-bf16.gguf`: llama.cpp GGUF export, BF16, not quantized.
|
||||
- `tokenizer.json`, `tokenizer_config.json`: MiniMind tokenizer files.
|
||||
- `config.json`, `generation_config.json`: Qwen3-compatible metadata used for conversion and loading.
|
||||
|
||||
The final source checkpoint was `checkpoints/dpo_768_resume.pth`.
|
||||
|
||||
## Prompt Format
|
||||
|
||||
The training code uses MiniMind chat markers:
|
||||
|
||||
```text
|
||||
<|im_start|>user
|
||||
你的问题<|im_end|>
|
||||
<|im_start|>assistant
|
||||
```
|
||||
|
||||
## Native Usage
|
||||
|
||||
Use the project code for native scratch inference:
|
||||
|
||||
```bash
|
||||
python -m minimind_scratch.cli chat \
|
||||
--weight out/hf/mikky-64m/mikky-64m.pth \
|
||||
--prompt "请用一句话介绍你自己"
|
||||
```
|
||||
|
||||
## llama.cpp / GGUF
|
||||
|
||||
The GGUF file is BF16 and intentionally not quantized:
|
||||
|
||||
```bash
|
||||
llama-cli -m mikky-64m-bf16.gguf \
|
||||
-p "<|im_start|>user\n请用一句话介绍你自己<|im_end|>\n<|im_start|>assistant\n" \
|
||||
-n 128
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
The GGUF export maps the scratch model to a Qwen3-compatible tensor layout because the model uses RMSNorm, SwiGLU MLP, grouped-query attention, RoPE, and q/k normalization.
|
||||
The GGUF structure and metadata were verified locally. Always verify generation quality in your target runtime before treating the GGUF file as production-ready.
|
||||
|
||||
## Limitations
|
||||
|
||||
- This is a very small model; expect limited reasoning, math, factual recall, and safety behavior.
|
||||
- It is not suitable for high-stakes medical, legal, financial, or safety-critical use.
|
||||
- GRPO/PPO are not part of the final release checkpoint.
|
||||
|
||||
## Dataset And License
|
||||
|
||||
This model was trained with the MiniMind small-data recipe from
|
||||
[`jingyaogong/minimind_dataset`](https://huggingface.co/datasets/jingyaogong/minimind_dataset).
|
||||
For this release, the dataset reference follows the MiniMind small dataset license: **Apache-2.0**.
|
||||
|
||||
Main data files used by this run:
|
||||
|
||||
- `pretrain_t2t_mini.jsonl`: pretraining data.
|
||||
- `sft_t2t_mini.jsonl`: supervised fine-tuning data.
|
||||
- `dpo.jsonl`: preference data for DPO.
|
||||
- `lora_identity_mikky.jsonl`: project-authored identity/persona data for mikky.
|
||||
|
||||
The model card, exported native checkpoint, Safetensors checkpoint, and GGUF artifact are released under **Apache-2.0**.
|
||||
Reference in New Issue
Block a user