QuantFactory/Llama-3.1-Swallow-8B-v0.1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: QuantFactory/Llama-3.1-Swallow-8B-v0.1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-22 12:56:12 +08:00
commit 20f98904ea
17 changed files with 308 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

49
.gitattributes vendored Normal file
View File

@@ -0,0 +1,49 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-v0.1.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Llama-3.1-Swallow-8B-v0.1.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:159174b451c1a21d5e889fed69715e89fc1032ce4b820fb022b29c4d95724ae2
size 3179131264

3
Llama-3.1-Swallow-8B-v0.1.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2bd4b0f9e068f3454aa747abc5b896b4872450f4c550955092303dc613653448
size 4321956224

3
Llama-3.1-Swallow-8B-v0.1.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:991e48b771c6ee65bb8fd0bd8fbbeb1410e4477ae099fbc36a448fb484826ade
size 4018917760

3
Llama-3.1-Swallow-8B-v0.1.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3e7c6afa7eb4fe94deaa5cefe51e03673cd2cd03b9294f741e7c210874691395
size 3664499072

3
Llama-3.1-Swallow-8B-v0.1.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:23cfbe82fe7b479d94e62e14096339121842287206583d9e9ae3dacca3d15c77
size 4661211520

3
Llama-3.1-Swallow-8B-v0.1.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:04fed98bc784dd56d2a0ee3705924e2e5cc09b420f861b5c1b63dcb8cb622b33
size 5130252672

3
Llama-3.1-Swallow-8B-v0.1.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:be789df823de05010605a54228def7487ccac5825de2e84c62762eb4d3992ee4
size 4920734080

3
Llama-3.1-Swallow-8B-v0.1.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cb0e1591cc1a6d88406f5a24734a37a36c98e37e664878a0e1d767e003c360bf
size 4692668800

3
Llama-3.1-Swallow-8B-v0.1.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:47309b46ac11c10aaa23bed7414bd6e3fd7bb43b81745faf494f87f6f38928ba
size 5599293824

3
Llama-3.1-Swallow-8B-v0.1.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:db2f6748dac7c82e0e8b72e306e3f5d9c10a4bcb15d5ce2608585a9e583b93d5
size 6068334976

3
Llama-3.1-Swallow-8B-v0.1.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dfcd79e647c79d0c06ec2e6089966b1171672d3b6e7deb8cc8d485a596e25cfc
size 5732987264

3
Llama-3.1-Swallow-8B-v0.1.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aa1abb0347a08a260569cc89f0c7372d4ef48d25a9dadbf3dd0fd9dc5ca02cb8
size 5599293824

3
Llama-3.1-Swallow-8B-v0.1.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8bcccd9e7d32c455a099d934eb7b2c269a253b98754340ccdc7b1e2e5f316c4c
size 6596006272

3
Llama-3.1-Swallow-8B-v0.1.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:568fad6aef019e71e55193dd1767452256d23745eb92f2c36bf7e1e71469d011
size 8540770688

216
README.md Normal file
View File

@@ -0,0 +1,216 @@
---
language:
- en
- ja
library_name: transformers
pipeline_tag: text-generation
license: llama3.1
model_type: llama
---
[![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory)
# QuantFactory/Llama-3.1-Swallow-8B-v0.1-GGUF
This is quantized version of [tokyotech-llm/Llama-3.1-Swallow-8B-v0.1](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-v0.1) created using llama.cpp
# Original Model Card
# Llama 3.1 Swallow - Built with Llama
Llama 3.1 Swallow is a series of large language models (8B, 70B) that were built by continual pre-training on the [Meta Llama 3.1](https://huggingface.co/collections/meta-llama/llama-31-669fc079a0c406a149a5738f) models.
Llama 3.1 Swallow enhanced the Japanese language capabilities of the original Llama 3.1 while retaining the English language capabilities.
We use approximately 200 billion tokens that were sampled from a large Japanese web corpus (Swallow Corpus Version 2), Japanese and English Wikipedia articles, and mathematical and
coding contents, etc for continual pre-training.
The instruction-tuned models (Instruct) were built by supervised fine-tuning (SFT) on the synthetic data specially built for Japanese (see the Training Datasets section for details).
See the Swallow Model Index section to find other model variants.
# Release History
- **October 08, 2024**: Released [Llama-3.1-Swallow-8B-v0.1](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-v0.1), [Llama-3.1-Swallow-8B-Instruct-v0.1](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.1), [Llama-3.1-Swallow-70B-v0.1](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-70B-v0.1), and [Llama-3.1-Swallow-70B-Instruct-v0.1](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-70B-Instruct-v0.1).
## Swallow Model Index
|Model|Llama-3.1-Swallow|Llama-3.1-Swallow-Instruct|
|---|---|---|
|8B| [Link](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-v0.1) | [Link](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.1) |
|70B| [Link](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-70B-v0.1) | [Link](https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-70B-Instruct-v0.1) |
![logo](./logo.png)
The website [https://swallow-llm.github.io/](https://swallow-llm.github.io/) provides large language models developed by the Swallow team.
## Model Details
* **Model type**: Please refer to [Llama 3.1 MODEL_CARD](https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md) for details on the model architecture.
* **Language(s)**: Japanese English
* **Library**: [Megatron-LM](https://github.com/NVIDIA/Megatron-LM)
* **Tokenizer**: Please refer to [Llama 3.1 blog](https://ai.meta.com/blog/meta-llama-3-1) for details on the tokenizer.
* **Contact**: swallow[at]nlp.c.titech.ac.jp
## Model Performance
### Japanese tasks
|Model|JCom.|JEMHopQA|NIILC|JSQuAD|XL-Sum|MGSM|WMT20-en-ja|WMT20-ja-en|JMMLU|JHumanEval|Ja Avg|
|---|---|---|---|---|---|---|---|---|---|---|---|
| |4-shot|4-shot|4-shot|4-shot|1-shot|4-shot|4-shot|4-shot|5-shot|0-shot| |
| |EM acc|Char-F1|Char-F1|Char-F1|ROUGE-2|EM acc|BLEU|BLEU|EM acc|pass@1| |
| Qwen2-7B | 0.8776 | 0.4627 | 0.3766 | 0.8984 | 0.1716 | 0.5480 | 0.2080 | 0.1949 | 0.5871 | **0.4183** | 0.4743 |
| Qwen2.5-7B | **0.9240** | 0.4581 | 0.4259 | **0.9071** | **0.2162** | **0.6200** | 0.2295 | 0.1989 | **0.6337** | 0.2665 | 0.4880 |
| Sarashina2-7B | 0.7417 | 0.5089 | **0.6353** | 0.8683 | 0.1420 | 0.0800 | 0.2727 | 0.2015 | 0.3835 | 0.0000 | 0.3834 |
| Llama 3 8B | 0.8356 | 0.4454 | 0.4002 | 0.8881 | 0.1757 | 0.3320 | 0.2199 | 0.2087 | 0.4558 | 0.3311 | 0.4292 |
| Llama 3.1 8B | 0.8436 | 0.4461 | 0.4050 | 0.8962 | 0.1794 | 0.3560 | 0.2209 | 0.2077 | 0.4767 | 0.3274 | 0.4359 |
| Llama 3 Youko 8B | 0.8660 | 0.4902 | 0.5155 | 0.8947 | 0.2127 | 0.2840 | 0.2740 | 0.2180 | 0.4493 | 0.2183 | 0.4423 |
| Llama 3 Swallow 8B | 0.8945 | 0.4848 | 0.5640 | 0.8947 | 0.1981 | 0.4240 | 0.2758 | 0.2223 | 0.4699 | 0.2890 | 0.4717 |
| Llama 3.1 Swallow 8B | 0.9124 | **0.5092** | 0.6011 | 0.8991 | 0.2020 | 0.4600 | **0.2909** | **0.2313** | 0.5182 | 0.2811 | **0.4905** |
### English tasks
|Model|OpenBookQA|TriviaQA|HellaSWAG|SQuAD2.0|XWINO|MMLU|GSM8K|BBH|HumanEval|En Avg|
|---|---|---|---|---|---|---|---|---|---|---|
| |4-shot|4-shot|4-shot|4-shot|4-shot|5-shot|4-shot|3-shot|0-shot| |
| |Acc|EM acc|Acc|EM acc|Acc|Acc|EM acc|CoT EM Acc|pass@1| |
| Qwen2-7B | 0.3740 | 0.6105 | 0.6006 | 0.3623 | 0.8916 | 0.7045 | 0.7748 | 0.5325 | **0.4622** | 0.5903 |
| Qwen2.5-7B | **0.3940** | 0.6011 | 0.5999 | **0.3743** | 0.8890 | **0.7424** | **0.8324** | 0.5620 | 0.4213 | **0.6018** |
| Sarashina2-7B | 0.3420 | 0.4784 | 0.5327 | 0.2911 | 0.8903 | 0.4267 | 0.1008 | 0.3746 | 0.0000 | 0.3818 |
| Llama 3 8B | 0.3760 | **0.7109** | **0.6124** | 0.3356 | 0.9032 | 0.6509 | 0.4936 | **0.6211** | 0.3793 | 0.5648 |
| Llama 3.1 8B | 0.3780 | 0.7017 | 0.6094 | 0.3330 | **0.9045** | 0.6525 | 0.5057 | 0.6176 | 0.3695 | 0.5636 |
| Llama 3 Youko 8B | 0.3500 | 0.6252 | 0.5885 | 0.3247 | 0.8959 | 0.5993 | 0.3571 | 0.5704 | 0.2793 | 0.5100 |
| Llama 3 Swallow 8B | 0.3520 | 0.6563 | 0.5901 | 0.3507 | 0.9006 | 0.6152 | 0.4875 | 0.5936 | 0.3323 | 0.5420 |
| Llama 3.1 Swallow 8B | 0.3800 | 0.6711 | 0.6057 | 0.3468 | 0.9032 | 0.6237 | 0.5110 | 0.6153 | 0.3622 | 0.5577 |
## Evaluation Benchmarks
### Japanese evaluation benchmarks
We used llm-jp-eval(v1.3.0), JP Language Model Evaluation Harness(commit #9b42d41) and Code Generation LM Evaluation Harness(commit #0261c52). The details are as follows:
- Multiple-choice question answering (JCommonsenseQA [Kurihara et al., 2022])
- Open-ended question answering (JEMHopQA [Ishii et al., 2024])
- Open-ended question answering (NIILC [関根, 2003])
- Machine reading comprehension (JSQuAD [Kurihara et al., 2022])
- Automatic summarization (XL-Sum [Hasan et al., 2021])
- Machine translation (WMT2020 ja-en [Barrault et al., 2020])
- Machine translation (WMT2020 en-ja [Barrault et al., 2020])
- Mathematical reasoning (MGSM [Shi et al., 2023])
- Academic exams (JMMLU [尹ら, 2024])
- Code generation (JHumanEval [佐藤ら, 2024])
### English evaluation benchmarks
We used the Language Model Evaluation Harness(v.0.4.2) and Code Generation LM Evaluation Harness(commit #0261c52). The details are as follows:
- Multiple-choice question answering (OpenBookQA [Mihaylov et al., 2018])
- Open-ended question answering (TriviaQA [Joshi et al., 2017])
- Machine reading comprehension (SQuAD2 [Rajpurkar et al., 2018])
- Commonsense reasoning (XWINO [Tikhonov and Ryabinin, 2021])
- Natural language inference (HellaSwag [Zellers et al., 2019])
- Mathematical reasoning (GSM8K [Cobbe et al., 2021])
- Reasoning (BBH (BIG-Bench-Hard) [Suzgun et al., 2023])
- Academic exams (MMLU [Hendrycks et al., 2021])
- Code generation (HumanEval [Chen et al., 2021])
## Training Datasets
### Continual Pre-Training
The following datasets were used for continual pre-training.
- [Cosmopedia](https://huggingface.co/datasets/HuggingFaceTB/cosmopedia)
- [Dclm-baseline-1.0](https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0)
- [English Wikipedia](https://dumps.wikimedia.org/other/cirrussearch)
- [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch)
- [Laboro ParaCorpus](https://github.com/laboroai/Laboro-ParaCorpus)
- [Swallow Corpus Version 2](https://arxiv.org/abs/2404.17733)
- [The-stack-v2](https://huggingface.co/datasets/bigcode/the-stack-v2-train-smol-ids)
## Risks and Limitations
The models released here are still in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.
## Acknowledgements
We thank Meta Research for releasing Llama 3.1 under a generous open license.
We received various supports including:
+ AIST project: "Research and Development of Foundation Models for Generative AI in the Physical Domain"
+ NEDO project: "Development of Artificial Intelligence Application Technology to Support Judgment in Design Risk Assessment Work Based on the Perspective of Skilled Persons" (JPNP18002) of "Development of Integration Technology as the Core of Next Generation Artificial Intelligence and Robotics"
+ MEXT project: "Formation of R&D center to ensure transparency and reliability of generative AI models"
+ AIST program: [Large Generative AI Development Support Program](https://abci.ai/en/link/lfm_support_program.html)
## License
[META LLAMA 3.1 COMMUNITY LICENSE](https://www.llama.com/llama3_1/license/)
## Authors
Here are the team members:
- From [Tokyo Institute of Technology Okazaki Laboratory](https://www.nlp.c.titech.ac.jp/index.en.html), the following members:
- [Naoaki Okazaki](https://www.chokkan.org/index.ja.html)
- [Sakae Mizuki](https://s-mizuki-nlp.github.io/)
- [Youmi Ma](https://www.nlp.c.titech.ac.jp/member/youmi.en.html)
- [Koki Maeda](https://sites.google.com/view/silviase)
- [Kakeru Hattori](https://aya-se.vercel.app/)
- [Masanari Ohi](https://sites.google.com/view/masanariohi)
- [Taihei Shiotani](https://github.com/inatoihs)
- [Koshiro Saito](https://sites.google.com/view/koshiro-saito)
- From [Tokyo Institute of Technology YOKOTA Laboratory](https://www.rio.gsic.titech.ac.jp/en/index.html), the following members:
- [Rio Yokota](https://twitter.com/rioyokota)
- [Kazuki Fujii](https://twitter.com/okoge_kaz)
- [Taishi Nakamura](https://twitter.com/Setuna7777_2)
- [Takumi Okamoto](https://www.linkedin.com/in/takumi-okamoto)
- [Ishida Shigeki](https://www.wantedly.com/id/reborn27)
- From [Artificial Intelligence Research Center, AIST, Japan](https://www.airc.aist.go.jp/en/teams/), the following members:
- [Hiroya Takamura](https://sites.google.com/view/hjtakamura)
## How to cite
If you find our work helpful, please feel free to cite these papers.
```
@inproceedings{Fujii:COLM2024,
title={Continual Pre-Training for Cross-Lingual LLM Adaptation:
Enhancing Japanese Language Capabilities},
author={Kazuki Fujii and Taishi Nakamura and Mengsay Loem and Hiroki
Iida and Masanari Ohi and Kakeru Hattori and Hirai Shota and Sakae
Mizuki and Rio Yokota and Naoaki Okazaki},
booktitle="Proceedings of the First Conference on Language Modeling",
series={COLM},
pages="(to appear)",
year="2024",
month=oct,
address={University of Pennsylvania, USA},
}
@inproceedings{Okazaki:COLM2024,
title={Building a Large Japanese Web Corpus for Large Language Models},
author={Naoaki Okazaki and Kakeru Hattori and Hirai Shota and Hiroki
Iida and Masanari Ohi and Kazuki Fujii and Taishi Nakamura and Mengsay
Loem and Rio Yokota and Sakae Mizuki},
booktitle="Proceedings of the First Conference on Language Modeling",
series={COLM},
pages="(to appear)",
year="2024",
month=oct,
address={University of Pennsylvania, USA},
}
```
### References
```tex
@misc{dubey2024llama3herdmodels,
title={The Llama 3 Herd of Models},
author={Abhimanyu Dubey and Abhinav Jauhri and Abhinav Pandey and Abhishek Kadian and Ahmad Al-Dahle and Aiesha Letman and Akhil Mathur and Alan Schelten and Amy Yang and Angela Fan et al.},
year={2024},
eprint={2407.21783},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2407.21783},
}
```

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}