初始化项目，由ModelHub XC社区提供模型

Model: shisa-ai/shisa-v2.1c-lfm2-350m Source: Original Platform
2026-06-25 09:07:13 +08:00
commit 25faa7fd04
11 changed files with 4323 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,51 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+ 
+ 
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*.tfevents* filter=lfs diff=lfs merge=lfs -text
+*.db* filter=lfs diff=lfs merge=lfs -text
+*.ark* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
+ 
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.gguf* filter=lfs diff=lfs merge=lfs -text
+*.ggml filter=lfs diff=lfs merge=lfs -text
+*.llamafile* filter=lfs diff=lfs merge=lfs -text
+*.pt2 filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+
+model.safetensors filter=lfs diff=lfs merge=lfs -text
+training_args.bin filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,57 @@
+---
+library_name: transformers
+model_name: outputs/shisa-chotto-lfm2-350m-dpo2
+tags:
+- generated_from_trainer
+- dpo
+- trl
+licence: license
+language:
+- en
+- ja
+base_model:
+- LiquidAI/LFM2-350M-ENJP-MT
+---
+
+# Model Card for shisa-ai/shisa-v2.1c-lfm2-350m
+
+SOTA Japanese Shaberi Benchmarks @ <0.5B and <1B!
+
+This was made just for fun on a Saturday for the Liquid AI Hackathon, but this is an early preview of our upcoming V2.1 models...
+
+For full code and related evals see: 
+- https://github.com/lhl/liquid-ai-hackathon-tokyo/tree/main
+
+Presentation here:
+- https://docs.google.com/presentation/d/1wZmfGE3bDloHNXjvYQBfGJ5jH-d1ksX9LGQn_aVfHbI/edit?usp=sharing
+
+| Model                                             | Average  | ELYZA 100  | JA-MT | Rakuda | Tengu  |
+|---------------------------------------------------|----------|------------|-------|--------|--------|
+| google/gemma-3-4b-it                              | 6.44     | 7.34       | 6.78  | 5.68   | 5.97   |
+| 045-llama3.2-1b-v2new-dpo405b                     | 5.40     | 5.44       | 5.22  | 6.35   | 4.61   |
+| 037-rakuten-2.0-mini-instruct-1.5b-v2new-dpo405b  | 5.10     | 5.42       | 4.60  | 5.68   | 4.70   |
+| augmxnt/shisa-gamma-7b-v1                         | 4.80     | 5.86       | 4.07  | 4.55   | 4.72   |
+| **shisa-ai/shisa-v2.1c-lfm2-350m**                | **4.51** | **4.30**   | **4.75** | **5.03** | **3.95** |
+| meta-llama/Llama-3.2-3B-Instruct                  | 4.49     | 5.62       | 4.50  | 3.43   | 4.43   |
+| Qwen/Qwen3-0.6B                                   | 4.14     | 5.16       | 4.00  | 3.18   | 4.23   |
+| augmxnt/shisa-7b-v1                               | 3.95     | 4.36       | 3.75  | 3.88   | 3.83   |
+| [shisa-ai/shisa-v2.1c-lfm2-350m-sft3-tlonly](https://huggingface.co/shisa-ai/shisa-v2.1c-lfm2-350m-sft3-tlonly) | 3.87     | 3.78       | 3.70  | 4.50   | 3.51   |
+| LiquidAI/LFM2-350M                                | 3.76     | 3.92       | 4.07  | 3.55   | 3.51   |
+| meta-llama/Llama-3.2-1B-Instruct                  | 2.97     | 3.82       | 2.82  | 2.45   | 2.79   |
+| google/gemma3-270m-it                             | 2.53     | 3.42       | 2.33  | 2.10   | 2.28   |
+| [LiquidAI/LFM2-350M-ENJP-MT](https://huggingface.co/LiquidAI/LFM2-350M-ENJP-MT)                        | 1.69     | 2.98       | 1.37  | 1.00   | 1.42   |
+| tiiuae/Falcon-H1-0.5B-Instruct                    | 1.30	   | 2.32	    | 1.47	| 1.00	 | 0.41   |
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/augmxnt/lfm2/runs/lvcw785j) 
+
+
+### Framework versions
+
+- TRL: 0.23.0
+- Transformers: 4.56.1
+- Pytorch: 2.10.0.dev20251008+cu130
+- Datasets: 4.2.0
+- Tokenizers: 0.22.1
+
+### Compute
+This model was trained on an 8xMI300X node on the [AMD Developer Cloud](https://www.amd.com/en/developer/resources/cloud-access/amd-developer-cloud.html) with compute generously sponsored by AMD.
--- a/chat_template.jinja
+++ b/chat_template.jinja
@@ -0,0 +1,37 @@
+{{- bos_token -}}
+{%- set system_prompt = "" -%}
+{%- set ns = namespace(system_prompt="") -%}
+{%- if messages[0]["role"] == "system" -%}
+	{%- set ns.system_prompt = messages[0]["content"] -%}
+	{%- set messages = messages[1:] -%}
+{%- endif -%}
+{%- if tools -%}
+	{%- set ns.system_prompt = ns.system_prompt + ("\n" if ns.system_prompt else "") + "List of tools: <|tool_list_start|>[" -%}
+	{%- for tool in tools -%}
+		{%- if tool is not string -%}
+            {%- set tool = tool | tojson -%}
+		{%- endif -%}
+		{%- set ns.system_prompt = ns.system_prompt + tool -%}
+        {%- if not loop.last -%}
+            {%- set ns.system_prompt = ns.system_prompt + ", " -%}
+        {%- endif -%}
+	{%- endfor -%}
+	{%- set ns.system_prompt = ns.system_prompt + "]<|tool_list_end|>" -%}
+{%- endif -%}
+{%- if ns.system_prompt -%}
+	{{- "<|im_start|>system\n" + ns.system_prompt + "<|im_end|>\n" -}}
+{%- endif -%}
+{%- for message in messages -%}
+	{{- "<|im_start|>" + message["role"] + "\n" -}}
+	{%- set content = message["content"] -%}
+	{%- if content is not string -%}
+		{%- set content = content | tojson -%}
+	{%- endif -%}
+	{%- if message["role"] == "tool" -%}
+		{%- set content = "<|tool_response_start|>" + content + "<|tool_response_end|>" -%}
+	{%- endif -%}
+	{{- content + "<|im_end|>\n" -}}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+	{{- "<|im_start|>assistant\n" -}}
+{%- endif -%}
--- a/config.json
+++ b/config.json
@@ -0,0 +1,57 @@
+{
+  "architectures": [
+    "Lfm2ForCausalLM"
+  ],
+  "block_auto_adjust_ff_dim": true,
+  "block_dim": 1024,
+  "block_ff_dim": 6656,
+  "block_ffn_dim_multiplier": 1.0,
+  "block_mlp_init_scale": 1.0,
+  "block_multiple_of": 256,
+  "block_norm_eps": 1e-05,
+  "block_out_init_scale": 1.0,
+  "block_use_swiglu": true,
+  "block_use_xavier_init": true,
+  "bos_token_id": 1,
+  "conv_L_cache": 3,
+  "conv_bias": false,
+  "conv_dim": 1024,
+  "conv_dim_out": 1024,
+  "conv_use_xavier_init": true,
+  "dtype": "bfloat16",
+  "eos_token_id": 7,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 6656,
+  "layer_types": [
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "full_attention",
+    "conv",
+    "full_attention",
+    "conv",
+    "full_attention",
+    "conv"
+  ],
+  "max_position_embeddings": 128000,
+  "model_type": "lfm2",
+  "norm_eps": 1e-05,
+  "num_attention_heads": 16,
+  "num_heads": 16,
+  "num_hidden_layers": 16,
+  "num_key_value_heads": 8,
+  "pad_token_id": 0,
+  "rope_theta": 1000000.0,
+  "transformers_version": "4.56.1",
+  "use_cache": false,
+  "use_pos_enc": true,
+  "vocab_size": 65536
+}
--- a/configuration.json
+++ b/configuration.json
@@ -0,0 +1 @@
+{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": [
+    7
+  ],
+  "pad_token_id": 0,
+  "transformers_version": "4.56.1"
+}
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff7f2cf167a214f38f6c48c3174951dcc3d8ebf04f2728551dbc735e25238dc8
+size 708984464
--- a/special_tokens_map.json
+++ b/special_tokens_map.json
@@ -0,0 +1,23 @@
+{
+  "bos_token": {
+    "content": "<|startoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
--- a/tokenizer.json
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98cff83b4f6d7e9d8929bebc62b07e92cf1b3f99c80d16bafe8b84a75448f40b
+size 4732426
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
--- a/training_args.bin
+++ b/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aab001766e60566c45e5452ed3bf2b83f6f3da2bec99aa81c033f06b12f68798
+size 10193
				`@@ -0,0 +1 @@`
				`{"framework": "pytorch", "task": "text-generation", "allow_remote": true}`