初始化项目,由ModelHub XC社区提供模型
Model: claytonwang/rlcr-qwen3-8b Source: Original Platform
This commit is contained in:
99
README.md
Normal file
99
README.md
Normal file
@@ -0,0 +1,99 @@
|
||||
---
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- qwen3
|
||||
- causal-lm
|
||||
- safetensors
|
||||
- bf16
|
||||
---
|
||||
|
||||
# Qwen3 Causal Language Model (BF16)
|
||||
|
||||
This repository contains a decoder-only causal language model using the Qwen3
|
||||
architecture. The weights are stored in sharded Safetensors format using BF16
|
||||
precision.
|
||||
|
||||
## Model details
|
||||
|
||||
| Item | Value |
|
||||
| --- | --- |
|
||||
| Architecture | `Qwen3ForCausalLM` |
|
||||
| Model type | `qwen3` |
|
||||
| Parameters | 8,190,735,360 |
|
||||
| Weight precision | BF16 |
|
||||
| Hidden size | 4096 |
|
||||
| Layers | 36 |
|
||||
| Attention heads | 32 |
|
||||
| Key/value heads | 8 |
|
||||
| Vocabulary size | 151,936 |
|
||||
| Configured context length | 40,960 tokens |
|
||||
| Weight format | 7 sharded Safetensors files (about 5 GB each) |
|
||||
|
||||
The tokenizer configuration currently declares a maximum length of 131,072,
|
||||
while the model configuration declares 40,960 positions. Until the longer
|
||||
context length is independently validated, use 40,960 tokens as the supported
|
||||
maximum.
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_id = "YOUR_USERNAME/YOUR_MODEL_NAME"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id,
|
||||
torch_dtype=torch.bfloat16,
|
||||
device_map="auto",
|
||||
attn_implementation="sdpa",
|
||||
)
|
||||
|
||||
messages = [{"role": "user", "content": "你好,请介绍一下你自己。"}]
|
||||
text = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
)
|
||||
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||||
|
||||
with torch.inference_mode():
|
||||
output = model.generate(**inputs, max_new_tokens=256)
|
||||
|
||||
answer = output[0, inputs.input_ids.shape[1]:]
|
||||
print(tokenizer.decode(answer, skip_special_tokens=True))
|
||||
```
|
||||
|
||||
BF16 weights require roughly 16 GB just for the model parameters, plus
|
||||
additional memory for inference state and the KV cache. Use a machine with
|
||||
sufficient RAM or accelerator memory. The example selects SDPA so that
|
||||
FlashAttention 2 is not required; installations with FlashAttention 2 may
|
||||
choose `attn_implementation="flash_attention_2"` instead.
|
||||
|
||||
## Training and provenance
|
||||
|
||||
The files supplied with this model do not identify the following information:
|
||||
|
||||
- Base model or checkpoint
|
||||
- Training or fine-tuning method
|
||||
- Training datasets
|
||||
- Evaluation results
|
||||
- Model author or organization
|
||||
|
||||
The uploader should add these details before publishing so users can assess the
|
||||
model's capabilities, limitations, and provenance.
|
||||
|
||||
## License
|
||||
|
||||
No license information was included with the supplied model files. The uploader
|
||||
must verify the licenses of the base model, training data, and resulting weights
|
||||
and add the correct license before making the repository public. Absence of a
|
||||
license does not grant permission to use, modify, or redistribute the model.
|
||||
|
||||
## Limitations
|
||||
|
||||
No benchmark, safety, bias, or long-context evaluation results were supplied.
|
||||
Do not rely on this model for high-impact decisions without appropriate testing
|
||||
and human review.
|
||||
Reference in New Issue
Block a user