初始化项目，由ModelHub XC社区提供模型

Model: farbodtavakkoli/OTel-LLM-8.3B-IT Source: Original Platform
2026-05-17 19:52:29 +08:00
commit 724dbe2148
16 changed files with 2921 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,100 @@
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- EssentialAI/rnj-1-instruct
+tags:
+- telecom
+- telecommunications
+- gsma
+- fine-tuned
+pipeline_tag: text-generation
+---
+
+# OTel-LLM-8.3B-IT
+
+**OTel-LLM-8.3B-IT** is a telecom-specialized language model fine-tuned on telecommunications domain data. It is part of the [OTel Family of Models](https://huggingface.co/collections/farbodtavakkoli/otel-llm), an open-source initiative to build industry-standard AI models for the global telecommunications sector.
+
+## Model Details
+
+| Attribute | Value |
+|-----------|-------|
+| **Base Model** | [EssentialAI/rnj-1-instruct](https://huggingface.co/EssentialAI/rnj-1-instruct) |
+| **Parameters** | 8.3B |
+| **Training Method** | Full parameter fine-tuning |
+| **Language** | English |
+| **License** | Apache 2.0 |
+
+## Training Data
+
+The model was trained on telecom-focused data curated by 100+ domain experts. Each source class was contributed by a specific institutional partner:
+
+| Source | Contributor |
+|---|---|
+| arXiv telecom papers, 3GPP standards, telecom Wikipedia, telecom Common Crawl | Yale University |
+| GSMA Permanent Reference Documents, Discover portal | GSMA |
+| IETF RFC series | NetoAI |
+| Industry whitepapers | Khalifa University |
+| O-RAN specifications (working groups 1, 2, 4, 5, 6, 7, 8, 9, 10) | University of Leeds |
+| O-RAN documents across working groups | The University of Texas at Dallas |
+
+Released datasets: [OTel-LLM](https://huggingface.co/datasets/farbodtavakkoli/OTel-LLM), [OTel-Embedding](https://huggingface.co/datasets/farbodtavakkoli/OTel-Embedding), [OTel-Reranker](https://huggingface.co/datasets/farbodtavakkoli/OTel-Reranker), [OTel-Safety](https://huggingface.co/datasets/farbodtavakkoli/OTel-Safety).
+
+## Intended Use
+ 
+The OTel model family is designed to power end-to-end Retrieval-Augmented Generation (RAG) pipelines for telecommunications. The three model types serve complementary roles:
+ 
+1. **Embedding** — Retrieve relevant chunks from telecom specifications, standards, and documentation.
+2. **Reranker** — Re-score and prioritize the retrieved chunks for relevance.
+3. **LLM** — Generate accurate responses grounded in the retrieved context.
+ 
+Users can deploy the full pipeline or use individual models independently based on their needs.
+ 
+**Note:** The LLMs include abstention training — if the model does not receive sufficient context, it will decline to answer rather than hallucinate. This means the models are optimized for context-grounded generation, not open-ended question answering.
+
+## Related Models
+
+### Language Models
+- [OTel LLM Collection](https://huggingface.co/collections/farbodtavakkoli/otel-llm)
+
+### Embedding Models
+- [OTel Embedding Collection](https://huggingface.co/collections/farbodtavakkoli/otel-embedding)
+
+### Reranker Models
+- [OTel Reranker Collection](https://huggingface.co/collections/farbodtavakkoli/otel-reranker)
+
+## Related Datasets
+
+- [OTel-Embedding](https://huggingface.co/datasets/farbodtavakkoli/OTel-Embedding)
+- [OTel-Safety](https://huggingface.co/datasets/farbodtavakkoli/OTel-Safety)
+- [OTel-LLM](https://huggingface.co/datasets/farbodtavakkoli/OTel-LLM)
+- [OTel-Reranker](https://huggingface.co/datasets/farbodtavakkoli/OTel-Reranker)
+
+## Training Infrastructure
+
+- **Framework**: ScalarLM (GPU-agnostic)
+- **Compute**: AMD and NVIDIA GPUs.
+
+## Project Resources
+
+- **Project page:** https://huggingface.co/farbodtavakkoli
+- **Code:** https://github.com/farbodtavakkoli/OTel
+- **Media coverage list:** https://github.com/farbodtavakkoli/OTel/blob/main/docs/media_coverage.md
+
+## Citation
+
+
+```bibtex
+@misc{otel_models_2026,
+  title  = {OTel: Open Telco AI Datasets, Benchmarks, and Models},
+  author = {Tavakkoli, Farbod and others},
+  year   = {2026},
+  note   = {Open Telco (OTel) model release},
+  url    = {https://huggingface.co/farbodtavakkoli}
+}
+```
+
+## Contact
+
+If you have any technical questions, please feel free to reach out to farbod.tavakkoli@att.com or farbodtavakoli@gmail.com
--- a/chat_template.jinja
+++ b/chat_template.jinja
@@ -0,0 +1,128 @@
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) -%}
+{%- set emit = namespace(started=false) -%}
+
+{# ---------- Build base system message (always emitted) ---------- #}
+{%- set base_system = 'You are rnj-1, a foundation model trained by Essential AI.\n' -%}
+
+{# ---------- Default system prompt if user system is absent ---------- #}
+{%- set default_system = 'You are a helpful assistant.' -%}
+
+{# Detect whether the first message is a user-provided system message #}
+{%- set has_user_system = (messages
+    and messages[0].role == 'system'
+    and (messages[0].content is string)) -%}
+
+{# The system instruction that should apply (user system wins; else default) #}
+{%- set effective_system = (has_user_system and messages[0].content) or default_system -%}
+
+
+{# ---------- Optional tools preface as a synthetic system message ---------- #}
+{%- if tools %}
+  {%- set sys_preamble -%}
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{%- for tool in tools %}
+{{ "\n" ~ (tool | tojson) }}
+{% endfor %}
+</tools>
+
+For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
+<tool_call>
+{"name": <function-name>, "arguments": <args-json-object>}
+</tool_call>
+  {%- endset -%}
+
+  {# Always include effective_system; user system prevails over default #}
+  {%- set sys_content = effective_system ~ "\n\n" ~ sys_preamble -%}
+
+  {%- set content = '<|start_header_id|>system<|end_header_id|>\n'
+      ~ base_system ~ '\n' ~ sys_content ~ '<|eot_id|>' -%}
+  {%- if not emit.started -%}{%- set content = bos_token ~ content -%}{%- set emit.started = true -%}{%- endif -%}
+  {{- content -}}
+{%- else %}
+  {# No tools: always emit base_system + effective_system #}
+  {%- set content = '<|start_header_id|>system<|end_header_id|>\n'
+      ~ base_system ~ '\n' ~ effective_system ~ '<|eot_id|>' -%}
+  {%- if not emit.started -%}{%- set content = bos_token ~ content -%}{%- set emit.started = true -%}{%- endif -%}
+  {{- content -}}
+{%- endif -%}
+
+{# ---------- Locate last user query for multi-step tool behavior ---------- #}
+{%- for message in messages[::-1] %}
+  {%- set index = (messages|length - 1) - loop.index0 -%}
+  {%- if ns.multi_step_tool
+        and message.role == "user"
+        and message.content is string
+        and not (message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) -%}
+    {%- set ns.multi_step_tool = false -%}
+    {%- set ns.last_query_index = index -%}
+  {%- endif -%}
+{%- endfor -%}
+
+{# ---------- Walk all messages and emit in Llama-3 format ---------- #}
+{%- for message in messages %}
+  {%- if message.content is string -%}
+    {%- set content = message.content -%}
+  {%- else -%}
+    {%- set content = '' -%}
+  {%- endif -%}
+
+  {# Skip the FIRST system message if it existed, since we already embedded it in effective_system #}
+  {%- if loop.first and message.role == "system" -%}
+    {# no-op #}
+
+  {%- elif (message.role == "user") or (message.role == "system") -%}
+    {%- set block = '<|start_header_id|>' ~ message.role ~ '<|end_header_id|>\n' ~ content ~ '<|eot_id|>' -%}
+    {%- if not emit.started -%}{%- set block = bos_token ~ block -%}{%- set emit.started = true -%}{%- endif -%}
+    {{- block -}}
+
+  {%- elif message.role == "assistant" -%}
+    {%- set body = content -%}
+    {%- set header = '<|start_header_id|>assistant<|end_header_id|>\n' -%}
+    {%- if not emit.started -%}{{ bos_token }}{%- set emit.started = true -%}{%- endif -%}
+    {{- header -}}
+    {% generation %}
+    {{- body -}}
+    {%- if message.tool_calls -%}
+      {%- for tool_call in message.tool_calls -%}
+        {%- if tool_call.function -%}{%- set tc = tool_call.function -%}{%- else -%}{%- set tc = tool_call -%}{%- endif -%}
+        {%- set args_json = (tc.arguments if (tc.arguments is string) else (tc.arguments | tojson)) -%}
+        {%- if loop.first -%}
+          {{- '<tool_call>\n{"name": "' ~ tc.name ~ '", "arguments": ' ~ args_json ~ '}\n</tool_call>' -}}
+        {%- else -%}
+          {{- '\n<tool_call>\n{"name": "' ~ tc.name ~ '", "arguments": ' ~ args_json ~ '}\n</tool_call>' -}}
+        {%- endif -%}
+      {%- endfor -%}
+    {%- endif -%}
+    {{- '<|eot_id|>' -}}{%- endgeneration -%}
+
+  {%- elif message.role == "tool" -%}
+    {%- set open_user = (loop.first or (loop.index0 > 0 and messages[loop.index0 - 1].role != "tool")) -%}
+    {%- set close_user = (loop.last or (loop.index0 < messages|length - 1 and messages[loop.index0 + 1].role != "tool")) -%}
+
+    {%- if open_user -%}
+      {%- set header = '<|start_header_id|>user<|end_header_id|>\n' -%}
+      {%- if not emit.started -%}{%- set header = bos_token ~ header -%}{%- set emit.started = true -%}{%- endif -%}
+      {{- header -}}
+    {%- endif -%}
+    {%- if open_user -%}
+      {{- '<tool_response>\n' -}}
+    {%- else -%}
+      {{- '\n<tool_response>\n' -}}
+    {%- endif -%}
+    {{- content -}}
+    {{- '\n</tool_response>' -}}
+
+    {%- if close_user -%}
+      {{- '<|eot_id|>' -}}
+    {%- endif -%}
+  {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+  {{- '<|start_header_id|>assistant<|end_header_id|>\n' -}}
+{%- endif -%}
--- a/config.json
+++ b/config.json
@@ -0,0 +1,113 @@
+{
+  "_sliding_window_pattern": 1,
+  "architectures": [
+    "Gemma3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_logit_softcapping": null,
+  "bos_token_id": 128000,
+  "cache_implementation": "hybrid",
+  "dtype": "bfloat16",
+  "eos_token_id": 128001,
+  "final_logit_softcapping": 30.0,
+  "head_dim": 128,
+  "hidden_act": "gelu_pytorch_tanh",
+  "hidden_activation": "gelu_pytorch_tanh",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 16384,
+  "layer_type": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "model_type": "gemma3_text",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 128001,
+  "query_pre_attn_scalar": 128,
+  "rms_norm_eps": 1e-06,
+  "rope_local_base_freq": 10000,
+  "rope_scaling": {
+    "attn_factor": 1.0,
+    "beta_fast": 64.0,
+    "beta_slow": 1.0,
+    "extrapolation_factor": 1.0,
+    "factor": 4.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "yarn"
+  },
+  "rope_theta": 10000,
+  "sliding_window": 32768,
+  "sliding_window_pattern": 1,
+  "transformers_version": "4.57.6",
+  "use_bidirectional_attention": false,
+  "use_cache": false,
+  "vocab_size": 128256
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,13 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "cache_implementation": "hybrid",
+  "do_sample": true,
+  "eos_token_id": [
+    128001,
+    128009
+  ],
+  "pad_token_id": 128001,
+  "temperature": 0.2,
+  "transformers_version": "4.57.6"
+}
--- a/pytorch_model-00001-of-00007.bin
+++ b/pytorch_model-00001-of-00007.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c0dc081ad930e2fb1e49fa294844f7c16590fd0a09bcebde067392dd99c4d1e
+size 4752154631
--- a/pytorch_model-00002-of-00007.bin
+++ b/pytorch_model-00002-of-00007.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e168f3b1d8c924ce316a00ee8b7687a51a381befe00a382bb2ce9a23662be879
+size 4865405263
--- a/pytorch_model-00003-of-00007.bin
+++ b/pytorch_model-00003-of-00007.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d583f417cfe657403baccb8153e6463167eb397ac8f35809a7d8faf5b1267a34
+size 4865405327
--- a/pytorch_model-00004-of-00007.bin
+++ b/pytorch_model-00004-of-00007.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e61eafef04e786617311beeb85ccf98962dd1b23ce14dd4497104a26ed44b7a5
+size 4865405327
--- a/pytorch_model-00005-of-00007.bin
+++ b/pytorch_model-00005-of-00007.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2003bb0a58a4ffaf60b36832c66fa00e6c29025e2a6f7befe834e87a075fc0e0
+size 4865405327
--- a/pytorch_model-00006-of-00007.bin
+++ b/pytorch_model-00006-of-00007.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b055239848cd4c4d1005a6da75f9d00220ae085dcd920e4133fb5f199a5b73da
+size 4865405327
--- a/pytorch_model-00007-of-00007.bin
+++ b/pytorch_model-00007-of-00007.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:711b35fa5a94fa07a478feed4933c0897196e1dd7c112d4422729fd8f55d8e08
+size 4162971383
--- a/pytorch_model.bin.index.json
+++ b/pytorch_model.bin.index.json
@@ -0,0 +1,426 @@
+{
+  "metadata": {
+    "total_size": 33242005504
+  },
+  "weight_map": {
+    "lm_head.weight": "pytorch_model-00001-of-00007.bin",
+    "model.embed_tokens.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.0.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.0.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.0.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.0.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.0.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.1.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.1.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.1.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.1.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.1.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.10.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.10.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.10.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.10.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.10.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.10.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.10.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.11.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.11.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.11.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.11.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.11.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.11.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.11.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.12.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.12.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.12.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.12.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.12.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.12.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.12.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.13.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.13.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.13.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.13.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.13.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.13.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.13.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.14.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.14.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.14.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.14.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.14.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.14.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.14.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.15.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.15.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.15.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.15.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.15.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.15.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.15.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.16.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.16.mlp.down_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.16.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.16.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.16.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.16.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.16.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.17.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.17.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.17.mlp.up_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.17.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.17.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.17.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.17.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00004-of-00007.bin",
+    "model.layers.18.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.18.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.18.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.18.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.18.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.18.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.18.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.19.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.19.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.19.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.19.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.19.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.19.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.19.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.2.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.2.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.2.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.2.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.2.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.2.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00007.bin",
+    "model.layers.20.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.20.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.20.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.20.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.20.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.20.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.20.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.21.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.21.mlp.down_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.21.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.21.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.21.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.21.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.21.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.22.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.22.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.22.mlp.up_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.22.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.22.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.22.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.22.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00005-of-00007.bin",
+    "model.layers.23.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.23.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.23.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.23.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.23.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.23.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.23.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.24.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.24.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.24.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.24.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.24.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.24.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.24.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.25.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.25.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.25.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.25.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.25.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.25.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.25.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.26.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.26.mlp.down_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.26.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.26.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.26.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.26.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.26.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.27.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.27.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.27.mlp.up_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.27.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.27.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.27.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.27.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00006-of-00007.bin",
+    "model.layers.28.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.mlp.up_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.mlp.up_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.3.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.3.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.3.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.3.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.3.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.3.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.3.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.30.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.mlp.up_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.mlp.down_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.mlp.up_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.4.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.4.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.4.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.4.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.4.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.4.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.4.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.5.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.5.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.5.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.5.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.5.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.5.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.5.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.6.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.6.mlp.down_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.6.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.6.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.6.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.6.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.6.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.7.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.7.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.7.mlp.up_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.7.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.7.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.7.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.7.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00002-of-00007.bin",
+    "model.layers.8.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.8.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.8.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.8.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.8.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.8.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.8.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.9.input_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.9.mlp.down_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.9.mlp.up_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.9.post_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.9.pre_feedforward_layernorm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.9.self_attn.k_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.9.self_attn.q_norm.weight": "pytorch_model-00007-of-00007.bin",
+    "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00003-of-00007.bin",
+    "model.norm.weight": "pytorch_model-00007-of-00007.bin"
+  }
+}
--- a/special_tokens_map.json
+++ b/special_tokens_map.json
@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|end_of_text|>"
+}
--- a/tokenizer.json
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fc0ffc87911804ac536e98844d0e69361a64682031c46845d8604785ab8a5ae
+size 17209894
--- a/tokenizer_config.json
+++ b/tokenizer_config.json