初始化项目，由ModelHub XC社区提供模型

Model: InfiX-ai/InfiR-1B-Instruct
Source: Original Platform

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,226 @@
+---
+license: llama3.2
+language:
+- en
+base_model:
+- meta-llama/Llama-3.2-1B
+pipeline_tag: text-generation
+---
+# Model Card for InfiR-1B-Instruct
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+InfR aims to advance AI systems by improving reasoning, reducing adoption barriers, and addressing privacy concerns through smaller model sizes.
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** InfiX
+- **Language(s) (NLP):** English
+- **Continual pretrained from model:** [[meta-llama/Llama-3.2-1B]](https://huggingface.co/meta-llama/Llama-3.2-1B)
+
+### Model Sources
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [[github]](https://github.com/InfiXAI/InfiR)
+- **Paper [optional]:** [[Arxiv]](https://arxiv.org/abs/2502.11573)
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+- **Performance gaps** remain vs. 70 B+ models on very hard reasoning (e.g., OlympiadBench).  
+- **Safety & bias**: inherits Llama-3.2 tokenizer & pre-training distribution; may reflect web biases.  
+- **Knowledge cut-off**: mid-2023.  
+- **Evaluation** has focused on English benchmarks; multilingual robustness not verified.
+
+
+## How to Get Started with the Model
+
+### Installation
+
+First, install the required dependencies:
+
+```bash
+pip install torch transformers
+```
+
+For optimal performance, we recommend using PyTorch 2.0+ and CUDA 11.8+.
+
+### Basic Usage
+
+Here's a simple example to get started with InfiR-1B-Instruct:
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+# Define messages in chat format
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "A new program had 60 downloads in the first month. The number of downloads in the second month was three times as many as the downloads in the first month, but then reduced by 30% in the third month. How many downloads did the program have total over the three months? Think step by step."},
+]
+
+# Load model and tokenizer
+tokenizer = AutoTokenizer.from_pretrained("InfiX-ai/InfiR-1B-Instruct")
+model = AutoModelForCausalLM.from_pretrained("InfiX-ai/InfiR-1B-Instruct")
+
+# Apply chat template and generate
+raw_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(raw_prompt, return_tensors="pt")
+outputs = model.generate(inputs["input_ids"], max_new_tokens=2048)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+### Advanced Usage Examples
+
+#### 1. Mathematical Reasoning
+
+```python
+# Mathematical problem solving with chat format
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "If a rectangle has a length of 8 units and a width of 6 units, what is its area and perimeter? Solve this step by step."},
+]
+
+raw_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(raw_prompt, return_tensors="pt")
+outputs = model.generate(
+    inputs["input_ids"], 
+    max_new_tokens=512,
+    temperature=0.1,
+    do_sample=True
+)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+#### 2. Code Generation
+
+```python
+# Code generation example with chat format
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "Write a Python function to calculate the factorial of a number."},
+]
+
+raw_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(raw_prompt, return_tensors="pt")
+outputs = model.generate(
+    inputs["input_ids"],
+    max_new_tokens=256,
+    temperature=0.2,
+    do_sample=True
+)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+#### 3. Chain-of-Thought Reasoning
+
+```python
+# Chain-of-thought reasoning with chat format
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "A train travels 120 km in 2 hours. What is its speed in km/h? Let's approach this step by step."},
+]
+
+raw_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(raw_prompt, return_tensors="pt")
+outputs = model.generate(
+    inputs["input_ids"],
+    max_new_tokens=300,
+    temperature=0.3,
+    do_sample=True
+)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+| Stage | Tokens | Composition |
+|-------|--------|-------------|
+| Pre-training | 900 B | 52 % code, 48 % high-quality web (math, science, encyclopedic) |
+| Annealing | 40 B | extra math & code + synthetic samples |
+| SFT | ~4 M | Infinity-Instruct, Orca-AgentInstruct-1M, NuminaMath, ScaleQuest (filtered) |
+
+Data cleaning: heuristic filters, MinHash de-duplication, 10-gram benchmark decontamination, reward-model rejection sampling.
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+| Hyper-parameter | Value |
+|-----------------|-------|
+| Precision | bf16 mixed |
+| Optimizer | AdamW |
+| LR (pre-train) | 1.4 e-3, cosine → 0 |
+| LR (SFT) | 2 e-5, cosine w/ 10 % warm-up |
+| Batch size | 2048 (pre-train), 128 (SFT) |
+| Sequence len | 4096 |
+| Epochs | 1 (pre-train), 1 (anneal), 4 (SFT) |
+| GPUs | 64 × H800, 5760 GPU-hours total |
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Benchmarks & Results
+
+| Benchmark | InfiR-1B-Instruct | Llama-3.2-1B-Instruct | Qwen-2.5-1.5B-Instruct |
+|-----------|-------------------|------------------------|-------------------------|
+| MMLU | 50.22 | 46.27 | 61.78 |
+| GSM8K | 70.9 | 47.9 | 74.3 |
+| MATH | 46.4 | 30.0 | 53.4 |
+| HumanEval | 58.54 | 39.63 | 51.83 |
+| MBPP | 56.03 | 49.03 | 56.81 |
+
+
+## Technical Specifications
+
+### Model Architecture and Objective
+
+- Base: Llama-3.2-1B (32 layers, 32 heads, RoPE, GQA, 2 k ctx → 4 k extended)  
+
+## Citation
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+```bibtex
+@misc{xie2025infir,
+  title={InfiR: Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning},
+  author={Xie, Congkai and Cai, Shuo and Wang, Wenjun and others},
+  year={2025},
+  eprint={2502.11573},
+  archivePrefix={arXiv},
+  primaryClass={cs.CL}
+}
+```
+
+**APA:**
+
+Xie, C., Cai, S., Wang, W., et al. (2025). *InfiR: Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning*. arXiv:2502.11573.
+
+---
+
+## Glossary
+
+- **SLM**: Small Language Model (<2 B parameters)  
+- **CoT**: Chain-of-Thought prompting or training  
+- **REC**: Renewable Energy Certificate  
+- **PUE**: Power Usage Effectiveness (ratio of total facility power to IT power)
+
+---

config.json

@@ -0,0 +1,36 @@
+{
+  "_name_or_path": "/s2/caishuo/models/20250214_mix/checkpoint",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": 128001,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 16,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 32.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": true,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.46.1",
+  "use_cache": false,
+  "vocab_size": 128256
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "eos_token_id": 128001,
+  "transformers_version": "4.46.1"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb02adfd465b35c9e6f0cf5b7dab416db319210347f891fd92899458e0511109
+size 2996982344

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>",
+  "sep_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}