初始化项目，由ModelHub XC社区提供模型

Model: LL-Square/LLSquare-7B-Instruct Source: Original Platform
2026-07-21 19:02:15 +08:00
commit 4cd50bbd49
14 changed files with 455615 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,102 @@
+---
+license: apache-2.0
+license_link: https://huggingface.co/LL-Square/LLSquare-7B-Instruct
+language:
+- en
+pipeline_tag: text-generation
+base_model: LL-Square/LLSquare-7B-Instruct-7B
+tags:
+- chat
+library_name: transformers
+---
+
+# LLSquare-7B-Instruct
+<a href="https://llsquare.vercel.app" target="_blank" style="margin: 2px;">
+    <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20LLSquare%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
+</a>
+
+## Introduction
+
+LLSquare-7B-Instruct is the base model of LL-Square large language models. For LLSquare-7B-Instruct, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. LLSquare-7B-Instruct brings the following improvements upon LL-Square2:
+
+- Significantly **more knowledge** and has greatly improved capabilities in **coding** and **mathematics**, thanks to our specialized expert models in these domains.
+- Significant improvements in **instruction following**, **generating long texts** (over 8K tokens), **understanding structured data** (e.g, tables), and **generating structured outputs** especially JSON. **More resilient to the diversity of system prompts**, enhancing role-play implementation and condition-setting for chatbots.
+- **Long-context Support** up to 128K tokens and can generate up to 8K tokens.
+- **Multilingual support** for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more. 
+
+**This repo contains the instruction-tuned 7B LLSquare-7B-Instruct model**, which has the following features:
+- Type: Causal Language Models
+- Training Stage: Pretraining & Post-training
+- Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
+- Number of Parameters: 7.61B
+- Number of Paramaters (Non-Embedding): 6.53B
+- Number of Layers: 28
+- Number of Attention Heads (GQA): 28 for Q and 4 for KV
+- Context Length: Full 131,072 tokens and generation 8192 tokens
+  - Please refer to [this section](#processing-long-texts) for detailed instructions on how to deploy LLSquare-7B-Instruct for handling long texts.
+
+
+## Requirements
+
+The code of LLSquare-7B-Instruct has been in the latest Hugging face `transformers` and we advise you to use the latest version of `transformers`.
+
+With `transformers<4.37.0`, you will encounter the following error:
+```
+KeyError: 'LL-Square2'
+```
+
+## Quickstart
+
+Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "LL-Square/LLSquare-7B-Instruct-7B-Instruct"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = "Give me a short introduction to large language model."
+messages = [
+    {"role": "system", "content": "You are LL-Square, created by Alibaba Cloud. You are a helpful assistant."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=512
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+```
+
+### Processing Long Texts
+
+The current `config.json` is set for context length up to 32,768 tokens.
+To handle extensive inputs exceeding 32,768 tokens, we utilize [YaRN](https://arxiv.org/abs/2309.00071), a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts.
+
+For supported frameworks, you could add the following to `config.json` to enable YaRN:
+```json
+{
+  ...,
+  "rope_scaling": {
+    "factor": 4.0,
+    "original_max_position_embeddings": 32768,
+    "type": "yarn"
+  }
+}
+```