初始化项目，由ModelHub XC社区提供模型
Model: cygnisai/Cygnis-Alpha-1.7B-v0.1 Source: Original Platform
This commit is contained in:
ModelHub XC
136
README.md
Normal file
136
README.md
Normal file
@@ -0,0 +1,136 @@
|
||||
---
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
pipeline_tag: text-generation
|
||||
base_model: HuggingFaceTB/SmolLM2-1.7B
|
||||
tags:
|
||||
- finetuned
|
||||
- sft
|
||||
- smollm2
|
||||
- sovereign-ai
|
||||
- safetensors
|
||||
- onnx
|
||||
- transformers.js
|
||||
---
|
||||
|
||||
|
||||
# Cygnis Alpha 1
|
||||
|
||||
<div align="center" style="background:#06090f; border-radius:14px; border:1px solid #0f1e30; overflow:hidden; margin-bottom:20px;">
|
||||
<img src="https://huggingface.co/cygnisai/Cygnis-Alpha-1.7B-v0.1/resolve/main/Cygnis-Alpha-1.7B-v1.png" width="100%" style="display:block;">
|
||||
</div>
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Model Summary](#model-summary)
|
||||
2. [Evaluation](#evaluation)
|
||||
3. [Examples](#examples)
|
||||
4. [Limitations](#limitations)
|
||||
5. [Training](#training)
|
||||
6. [License](#license)
|
||||
7. [Citation](#citation)
|
||||
|
||||
## Model Summary
|
||||
|
||||
**Cygnis Alpha v1** is a compact, high-performance language model based on the **SmolLM2-1.7B** architecture. It is designed to be lightweight enough to run on-device while maintaining high reasoning and instruction-following capabilities.
|
||||
|
||||
The 1.7B variant represents a significant leap over previous generations, trained on **11 trillion tokens** using a high-quality mix of FineWeb-Edu, DCLM, and specialized coding/math datasets. This model has been refined using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to ensure logical consistency and helpfulness.
|
||||
|
||||
### How to use
|
||||
|
||||
#### Transformers
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
checkpoint = "cygnisai/Cygnis-Alpha-1.7B-v0.1"
|
||||
device = "cuda" # for GPU usage or "cpu" for CPU usage
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
|
||||
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
|
||||
|
||||
messages = [{"role": "user", "content": "What is the core philosophy of sovereign AI?"}]
|
||||
input_text = tokenizer.apply_chat_template(messages, tokenize=False)
|
||||
inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
|
||||
outputs = model.generate(inputs, max_new_tokens=100, temperature=0.2, top_p=0.9, do_sample=True)
|
||||
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
#### Transformers.js
|
||||
```javascript
|
||||
import { pipeline } from "@huggingface/transformers";
|
||||
|
||||
const generator = await pipeline(
|
||||
"text-generation",
|
||||
"cygnisai/Cygnis-Alpha-1.7B-v0.1",
|
||||
);
|
||||
|
||||
const messages = [
|
||||
{ role: "system", content: "You are Cygnis, a sovereign AI assistant." },
|
||||
{ role: "user", content: "Hello!" },
|
||||
];
|
||||
|
||||
const output = await generator(messages, { max_new_tokens: 128 });
|
||||
console.log(output[0].generated_text.at(-1).content);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Evaluation
|
||||
|
||||
Cygnis Alpha v1 inherits the state-of-the-art benchmarks of the SmolLM2-1.7B-Instruct core. Evaluations are zero-shot using [lighteval](https://github.com/huggingface/lighteval).
|
||||
|
||||
### Instruction Model Benchmarks
|
||||
|
||||
| Metric | Cygnis Alpha v1 (1.7B) | Llama-1B-Instruct | Qwen2.5-1.5B-Instruct |
|
||||
|:------------------------------|:---------------------:|:-----------------:|:----------------------:|
|
||||
| **IFEval** (Avg prompt/inst) | **56.7** | 53.5 | 47.4 |
|
||||
| **MT-Bench** | 6.13 | 5.48 | **6.52** |
|
||||
| **HellaSwag** | **66.1** | 56.1 | 60.9 |
|
||||
| **ARC (Average)** | **51.7** | 41.6 | 46.2 |
|
||||
| **GSM8K (5-shot)** | **48.2** | 26.8 | 42.8 |
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Text Rewriting
|
||||
**System Prompt:** "You are an AI writing assistant. Your task is to rewrite the user's email to make it more professional and approachable while maintaining its main points and key message."
|
||||
|
||||
### Function Calling
|
||||
Cygnis Alpha v1 supports tool-use and function calling. It scores **27%** on the BFCL Leaderboard, allowing it to interface with APIs to fetch real-time data or perform calculations.
|
||||
|
||||
---
|
||||
|
||||
## Limitations
|
||||
|
||||
Cygnis Alpha v1 Instruct primarily understands and generates content in English. While powerful for its size, it may produce factually inaccurate or logically inconsistent content for highly specialized tasks. Users should verify important information and use it as an assistive tool.
|
||||
|
||||
## Training
|
||||
|
||||
### Model Specifications
|
||||
- **Architecture:** Transformer decoder
|
||||
- **Pretraining tokens:** 11T
|
||||
- **Precision:** bfloat16
|
||||
- **Hardware:** 256 H100 GPUs (original base training)
|
||||
|
||||
### Software
|
||||
- **Training Framework:** [nanotron](https://github.com/huggingface/nanotron/tree/main)
|
||||
- **Alignment:** [alignment-handbook](https://github.com/huggingface/alignment-handbook/)
|
||||
|
||||
## License
|
||||
|
||||
This model is licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).
|
||||
|
||||
## Citation
|
||||
```bibtex
|
||||
@misc{cygnis_alpha_1.7b_v0.1,
|
||||
title={SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model},
|
||||
author={Simonc-44},
|
||||
year={2026},
|
||||
eprint={2502.02737},
|
||||
archivePrefix={arXiv},
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user