初始化项目,由ModelHub XC社区提供模型
Model: voidful/smol-360m-ft Source: Original Platform
This commit is contained in:
19
README.md
Normal file
19
README.md
Normal file
@@ -0,0 +1,19 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
datasets:
|
||||
- voidful/mixed_inst_multi_chat
|
||||
base_model:
|
||||
- HuggingFaceTB/SmolLM2-360M
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
|
||||
Model Summary
|
||||
|
||||
SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device.
|
||||
|
||||
SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 360M model was trained on 4 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own curated datasets. We then applied Direct Preference Optimization (DPO) using UltraFeedback.
|
||||
|
||||
The instruct model additionally supports tasks such as text rewriting, summarization and function calling thanks to datasets developed by Argilla such as Synth-APIGen-v0.1.
|
||||
|
||||
For more details refer to: https://github.com/huggingface/smollm. You will find pre-training, post-training, evaluation and local inference code.
|
||||
Reference in New Issue
Block a user