初始化项目，由ModelHub XC社区提供模型
Model: NotHereNorThere/Coral-v1.5-4b Source: Original Platform
This commit is contained in:
ModelHub XC
48
.gitattributes
vendored
Normal file
48
.gitattributes
vendored
Normal file
@@ -0,0 +1,48 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
adapter/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
checkpoint-313/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
checkpoint-626/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
merged/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
F16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
TQ2_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
F16.gguf
Normal file
3
F16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:82e54e97fd76c969caf683bfdc19a4f176f24d3cce99745dae878b09c12a92e9
|
||||
size 8051284704
|
||||
3
IQ3_M.gguf
Normal file
3
IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:27654c581bdf9f9b046e9956366dd29ea567eae43dff0ad3aeba5ded7ba89f77
|
||||
size 1962895584
|
||||
3
IQ4_NL.gguf
Normal file
3
IQ4_NL.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4d57e40d67d20c2a48f825414bafdf52db1dccb19c4444603c196435d4487cd3
|
||||
size 2393794784
|
||||
3
IQ4_XS.gguf
Normal file
3
IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:98313fd23902b97785f38d6102c6df3cb53ff68f8eeb507f142a95e843dd3fe0
|
||||
size 2286315744
|
||||
3
Q3_K_M.gguf
Normal file
3
Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d5a74bb293d501bbd9efc50cf04b64cdba809a5eb370810f9c309bf08d111e47
|
||||
size 2075617504
|
||||
3
Q4_K_M.gguf
Normal file
3
Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7a06030eb2905b5de6e1af689acdc4b2f952e65ebd1b00983a95dfc31a06daac
|
||||
size 2497280224
|
||||
3
Q5_K_M.gguf
Normal file
3
Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:044033f354fa9a19e632fea43f39aa3b940f9bc8f5d0a5955c07ec70cababc28
|
||||
size 2889513184
|
||||
3
Q6_K.gguf
Normal file
3
Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d6b8acbc0d116466f89670ca6b47ac85bfb624139604e1ceea1e93ba056685f7
|
||||
size 3306260704
|
||||
159
README.md
Normal file
159
README.md
Normal file
@@ -0,0 +1,159 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
- zh
|
||||
base_model:
|
||||
- Qwen/Qwen3-4B
|
||||
datasets:
|
||||
- teknium/OpenHermes-2.5
|
||||
- Sidsidney/OpenThoughts-114k
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- corallm
|
||||
- qwen3
|
||||
- finetune
|
||||
- merge
|
||||
- gguf
|
||||
- safetensors
|
||||
- apache2
|
||||
- 4b
|
||||
- uncensored
|
||||
- english
|
||||
---
|
||||
# Coral-v1.5-4B
|
||||
|
||||
A 4B parameter uncensored generalist with strong multi-step reasoning, correct arithmetic, solid code generation, and long-context coherence across extended conversations. Built from a 7-donor TIES merge of Qwen3-4B finetunes including official Qwen 2507 update variants, healed with a 2,500 row fine-tune pass.
|
||||
|
||||
Part of the **Coral-v1.5** model family, which adds to the original CoralLM series (Llama 3.2 1B based). Coral-v1.5 moves to Qwen3 architecture for significantly improved base capability.
|
||||
|
||||
> **Note on identity:** The model identifies itself as Qwen/Alibaba by default due to base model bleedthrough. A simple system prompt overrides this, no retraining needed.
|
||||
|
||||
---
|
||||
|
||||
## Improvements over Coral-v1.5-0.6B
|
||||
|
||||
| Capability | 0.6B | 4B |
|
||||
|---|---|---|
|
||||
| Parameters | ~600M | ~4B |
|
||||
| Donors | 5 | 7 |
|
||||
| Fine-tune rows | 1,000 | 2,500 |
|
||||
| Inference speed (RTX4060) | 161 t/s | 75 t/s (Q5_K_M) |
|
||||
| Math accuracy | ✅ Correct | ✅ Correct |
|
||||
| Multi-step reasoning | ⚠️ Basic | ✅ Strong |
|
||||
| Long multi-turn coherence | ⚠️ Short working context | ✅ 13+ turns tested |
|
||||
| Trick question resistance | ⚠️ Untested | ✅ Doesn't hallucinate fake memories |
|
||||
| Adaptive CoT | ✅ Emergent | ❌ Smoothed out by larger FT |
|
||||
| Code quality | ✅ Decent | ✅ Better |
|
||||
| Uncensored | ✅ | ✅ |
|
||||
|
||||
The 4B trades the emergent adaptive CoT behavior of the 0.6B for significantly stronger raw reasoning capability and coherence at scale. The reasoning happens internally without explicit think blocks.
|
||||
|
||||
---
|
||||
|
||||
## What makes it interesting
|
||||
|
||||
- **7-donor TIES merge** - more donors, more diverse capability blend than the 0.6B
|
||||
- **Qwen3 original + 2507 cross-mixing** - includes both original Qwen3-4B and post-training 2507 update finetunes as contributors
|
||||
- **Three reasoning distills** - knowledge transferred from larger models (DeepSeek, Opus, Gemini) down to 4B scale
|
||||
- **Trick question resistant** - correctly identified a question about a conversation event that never happened rather than hallucinating a fake memory
|
||||
- **Uncensored** - refusal behavior removed via two de-alignment donors, survives the fine-tune pass
|
||||
- **Long context coherence** - maintains conversation state across 13+ turn exchanges
|
||||
|
||||
---
|
||||
|
||||
## Merge Recipe
|
||||
|
||||
**Method:** TIES
|
||||
**Base:** `Qwen/Qwen3-4B`
|
||||
**Tool:** [mergekit](https://github.com/arcee-ai/mergekit)
|
||||
|
||||
| Donor | Role | Weight | Density |
|
||||
|---|---|---|---|
|
||||
| `leonMW/Qwen3-4B-Thinking-2507-GSPO-Easy` | Thinking / reasoning | 0.20 | 0.5 |
|
||||
| `khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Distilled` | Reasoning distill | 0.20 | 0.5 |
|
||||
| `ertghiu256/Qwen3-4B-distill-deepseek-opus-gemini` | Multi-teacher distill | 0.20 | 0.5 |
|
||||
| `Qwen/Qwen3-4B-Instruct-2507` | Official instruct (2507) | 0.18 | 0.5 |
|
||||
| `Qwen/Qwen3-4B-Thinking-2507` | Official thinking (2507) | 0.18 | 0.5 |
|
||||
| `huihui-ai/Huihui-Qwen3-4B-Instruct-2507-abliterated` | De-alignment | 0.15 | 0.5 |
|
||||
| `DreamFast/qwen3-4b-heretic` | De-alignment (heretic method) | 0.15 | 0.5 |
|
||||
|
||||
```yaml
|
||||
base_model: Qwen/Qwen3-4B
|
||||
merge_method: ties
|
||||
dtype: bfloat16
|
||||
parameters:
|
||||
normalize: true
|
||||
int8_mask: true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Fine-tune
|
||||
|
||||
Post-merge heal pass to fix coherence, counting, context retention, and question invention behavior from the raw merge.
|
||||
|
||||
- **1,250 rows** — [OpenHermes 2.5](https://huggingface.co/datasets/teknium/OpenHermes-2.5) (simple QA + instruction following)
|
||||
- **1,250 rows** — [OpenThoughts](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k) (complex reasoning with CoT)
|
||||
- **Method:** QLoRA + Flash Attention 2, LoRA r16
|
||||
- **Epochs:** 2
|
||||
- **Total:** 2,500 rows, randomly sampled and shuffled
|
||||
- **Quantization:** Q5_K_M (auto-quantized post fine-tune)
|
||||
|
||||
---
|
||||
|
||||
## Evaluation
|
||||
|
||||
| Test | Result |
|
||||
|---|---|
|
||||
| Basic greeting | ✅ Clean, no loops |
|
||||
| Exact instruction following ("list 3 fruits") | ✅ Correct count and formatting |
|
||||
| Context retention across turns | ✅ Recalled user name correctly |
|
||||
| Math (47 × 83) | ✅ Correct (3,901) with clean step-by-step working |
|
||||
| Multi-step word problem | ✅ Correct with full reasoning |
|
||||
| Prime number function | ✅ Correct implementation |
|
||||
| Constrained creative writing | ✅ All constraints met |
|
||||
| Long multi-turn conversation (13 turns) | ✅ Coherent throughout |
|
||||
| Trick question (fake memory) | ✅ Correctly refused to hallucinate |
|
||||
| Joke repetition awareness | ✅ Noticed repeat, told a different one |
|
||||
| Uncensored | ✅ Refusals removed, survives fine-tune |
|
||||
|
||||
---
|
||||
|
||||
## Inference
|
||||
|
||||
```
|
||||
> System: You are Coral, a helpful AI assistant. `<whatever else>`
|
||||
```
|
||||
|
||||
Recommended system prompt to fix identity bleedthrough. The model responds well to persona anchoring, should do well with system prompt and instruciton adherence.
|
||||
|
||||
**Speed (Q5_K_M):** ~75 t/s generation on mid-low consumer hardware
|
||||
|
||||
### Available Quantizations
|
||||
|
||||
All quantized from the BF16 merge output. Quality and speed are relative to Q5_K_M (the baseline). Speed is approximate and hardware-dependent; quality is a general expectation for these quant types on a 4B model.
|
||||
|
||||
| Quant | Size vs Q5_K_M | Quality vs Q5_K_M | Speed vs Q5_K_M | Notes |
|
||||
|---|---|---|---|---|
|
||||
| F16 | Much larger | Lossless reference | ~−45% | Full precision, for reference/conversion |
|
||||
| Q6_K | Larger | Near-identical | ~−15% | Highest practical quality |
|
||||
| **Q5_K_M** | **baseline** | **baseline** | **baseline** | **Recommended default** |
|
||||
| Q4_K_M | Smaller | Slightly lower | ~+15% | Classic balanced choice |
|
||||
| IQ4_NL | Smaller | ≈ Q4_K_M, slightly better | ~+10% | Non-linear grid, good quality/size |
|
||||
| IQ4_XS | Smaller | ≈ Q4_K_M | ~+15% | Smallest 4-bit, importance-matrix |
|
||||
| Q3_K_M | Much smaller | Noticeably lower | ~+30% | Usable but degraded |
|
||||
| IQ3_M | Much smaller | Lower, better than Q3_K | ~+25% | Best aggressive option |
|
||||
| TQ2_0 | Tiny | No | ~+60% | Ternary weights (-1/0/1 only). Don't bother |
|
||||
|
||||
**Recommendation:** Q5_K_M for quality, IQ4_XS or IQ4_NL for a good speed/size/quality balance, IQ3_M if you're tight on memory. F16 is for conversion/reference only — no quality benefit over Q6_K at much larger size.
|
||||
|
||||
---
|
||||
|
||||
## Model Family (so far)
|
||||
|
||||
| Model | Base | Donors | FT Rows | Status |
|
||||
|---|---|---|---|---|
|
||||
| CoralLM-1B | Llama3.2-1B | 3 | 400 | ✅ Released
|
||||
| Coral-v1.5-0.6B | Qwen3-0.6B | 5 | 1,000 | ✅ Released |
|
||||
| Coral-v1.5-4B | Qwen3-4B | 7 | 2,500 | ✅ Released |
|
||||
3
TQ2_0.gguf
Normal file
3
TQ2_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:00bea05640e75ed835d0c4a163fef3e5808671e050fe007a4aefc0a8f0ec8127
|
||||
size 1262520544
|
||||
209
adapter/README.md
Normal file
209
adapter/README.md
Normal file
@@ -0,0 +1,209 @@
|
||||
---
|
||||
base_model: /home/mic/Documents/junk/claude-finetune-kit/merging/coral-v1.5-4b-out
|
||||
library_name: peft
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- base_model:adapter:/home/mic/Documents/junk/claude-finetune-kit/merging/coral-v1.5-4b-out
|
||||
- lora
|
||||
- sft
|
||||
- transformers
|
||||
- trl
|
||||
---
|
||||
|
||||
# Model Card for Model ID
|
||||
|
||||
<!-- Provide a quick summary of what the model is/does. -->
|
||||
|
||||
|
||||
|
||||
## Model Details
|
||||
|
||||
### Model Description
|
||||
|
||||
<!-- Provide a longer summary of what this model is. -->
|
||||
|
||||
|
||||
|
||||
- **Developed by:** [More Information Needed]
|
||||
- **Funded by [optional]:** [More Information Needed]
|
||||
- **Shared by [optional]:** [More Information Needed]
|
||||
- **Model type:** [More Information Needed]
|
||||
- **Language(s) (NLP):** [More Information Needed]
|
||||
- **License:** [More Information Needed]
|
||||
- **Finetuned from model [optional]:** [More Information Needed]
|
||||
|
||||
### Model Sources [optional]
|
||||
|
||||
<!-- Provide the basic links for the model. -->
|
||||
|
||||
- **Repository:** [More Information Needed]
|
||||
- **Paper [optional]:** [More Information Needed]
|
||||
- **Demo [optional]:** [More Information Needed]
|
||||
|
||||
## Uses
|
||||
|
||||
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
||||
|
||||
### Direct Use
|
||||
|
||||
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Downstream Use [optional]
|
||||
|
||||
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Out-of-Scope Use
|
||||
|
||||
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Bias, Risks, and Limitations
|
||||
|
||||
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Recommendations
|
||||
|
||||
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
||||
|
||||
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
||||
|
||||
## How to Get Started with the Model
|
||||
|
||||
Use the code below to get started with the model.
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Training Details
|
||||
|
||||
### Training Data
|
||||
|
||||
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Training Procedure
|
||||
|
||||
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
||||
|
||||
#### Preprocessing [optional]
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
|
||||
#### Training Hyperparameters
|
||||
|
||||
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
||||
|
||||
#### Speeds, Sizes, Times [optional]
|
||||
|
||||
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Evaluation
|
||||
|
||||
<!-- This section describes the evaluation protocols and provides the results. -->
|
||||
|
||||
### Testing Data, Factors & Metrics
|
||||
|
||||
#### Testing Data
|
||||
|
||||
<!-- This should link to a Dataset Card if possible. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Factors
|
||||
|
||||
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Metrics
|
||||
|
||||
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Results
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Summary
|
||||
|
||||
|
||||
|
||||
## Model Examination [optional]
|
||||
|
||||
<!-- Relevant interpretability work for the model goes here -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Environmental Impact
|
||||
|
||||
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
||||
|
||||
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
||||
|
||||
- **Hardware Type:** [More Information Needed]
|
||||
- **Hours used:** [More Information Needed]
|
||||
- **Cloud Provider:** [More Information Needed]
|
||||
- **Compute Region:** [More Information Needed]
|
||||
- **Carbon Emitted:** [More Information Needed]
|
||||
|
||||
## Technical Specifications [optional]
|
||||
|
||||
### Model Architecture and Objective
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Compute Infrastructure
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Hardware
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Software
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Citation [optional]
|
||||
|
||||
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
||||
|
||||
**BibTeX:**
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
**APA:**
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Glossary [optional]
|
||||
|
||||
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## More Information [optional]
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Model Card Authors [optional]
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Model Card Contact
|
||||
|
||||
[More Information Needed]
|
||||
### Framework versions
|
||||
|
||||
- PEFT 0.19.1
|
||||
48
adapter/adapter_config.json
Normal file
48
adapter/adapter_config.json
Normal file
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"alora_invocation_tokens": null,
|
||||
"alpha_pattern": {},
|
||||
"arrow_config": null,
|
||||
"auto_mapping": null,
|
||||
"base_model_name_or_path": "/home/mic/Documents/junk/claude-finetune-kit/merging/coral-v1.5-4b-out",
|
||||
"bias": "none",
|
||||
"corda_config": null,
|
||||
"ensure_weight_tying": false,
|
||||
"eva_config": null,
|
||||
"exclude_modules": null,
|
||||
"fan_in_fan_out": false,
|
||||
"inference_mode": true,
|
||||
"init_lora_weights": true,
|
||||
"layer_replication": null,
|
||||
"layers_pattern": null,
|
||||
"layers_to_transform": null,
|
||||
"loftq_config": {},
|
||||
"lora_alpha": 32,
|
||||
"lora_bias": false,
|
||||
"lora_dropout": 0.05,
|
||||
"lora_ga_config": null,
|
||||
"megatron_config": null,
|
||||
"megatron_core": "megatron.core",
|
||||
"modules_to_save": null,
|
||||
"peft_type": "LORA",
|
||||
"peft_version": "0.19.1",
|
||||
"qalora_group_size": 16,
|
||||
"r": 8,
|
||||
"rank_pattern": {},
|
||||
"revision": null,
|
||||
"target_modules": [
|
||||
"q_proj",
|
||||
"down_proj",
|
||||
"o_proj",
|
||||
"gate_proj",
|
||||
"k_proj",
|
||||
"v_proj",
|
||||
"up_proj"
|
||||
],
|
||||
"target_parameters": null,
|
||||
"task_type": "CAUSAL_LM",
|
||||
"trainable_token_indices": null,
|
||||
"use_bdlora": null,
|
||||
"use_dora": false,
|
||||
"use_qalora": false,
|
||||
"use_rslora": false
|
||||
}
|
||||
3
adapter/adapter_model.safetensors
Normal file
3
adapter/adapter_model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:fe4dfe024a77157cc820fbeb335d8b13820df7184bc48498d4c0e779c60e0e2f
|
||||
size 33096960
|
||||
28
adapter/added_tokens.json
Normal file
28
adapter/added_tokens.json
Normal file
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"</think>": 151668,
|
||||
"</tool_call>": 151658,
|
||||
"</tool_response>": 151666,
|
||||
"<think>": 151667,
|
||||
"<tool_call>": 151657,
|
||||
"<tool_response>": 151665,
|
||||
"<|box_end|>": 151649,
|
||||
"<|box_start|>": 151648,
|
||||
"<|endoftext|>": 151643,
|
||||
"<|file_sep|>": 151664,
|
||||
"<|fim_middle|>": 151660,
|
||||
"<|fim_pad|>": 151662,
|
||||
"<|fim_prefix|>": 151659,
|
||||
"<|fim_suffix|>": 151661,
|
||||
"<|im_end|>": 151645,
|
||||
"<|im_start|>": 151644,
|
||||
"<|image_pad|>": 151655,
|
||||
"<|object_ref_end|>": 151647,
|
||||
"<|object_ref_start|>": 151646,
|
||||
"<|quad_end|>": 151651,
|
||||
"<|quad_start|>": 151650,
|
||||
"<|repo_name|>": 151663,
|
||||
"<|video_pad|>": 151656,
|
||||
"<|vision_end|>": 151653,
|
||||
"<|vision_pad|>": 151654,
|
||||
"<|vision_start|>": 151652
|
||||
}
|
||||
89
adapter/chat_template.jinja
Normal file
89
adapter/chat_template.jinja
Normal file
@@ -0,0 +1,89 @@
|
||||
{%- if tools %}
|
||||
{{- '<|im_start|>system\n' }}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- messages[0].content + '\n\n' }}
|
||||
{%- endif %}
|
||||
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||
{%- for tool in tools %}
|
||||
{{- "\n" }}
|
||||
{{- tool | tojson }}
|
||||
{%- endfor %}
|
||||
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||
{%- else %}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
||||
{%- for message in messages[::-1] %}
|
||||
{%- set index = (messages|length - 1) - loop.index0 %}
|
||||
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
||||
{%- set ns.multi_step_tool = false %}
|
||||
{%- set ns.last_query_index = index %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- for message in messages %}
|
||||
{%- if message.content is string %}
|
||||
{%- set content = message.content %}
|
||||
{%- else %}
|
||||
{%- set content = '' %}
|
||||
{%- endif %}
|
||||
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
||||
{%- elif message.role == "assistant" %}
|
||||
{%- set reasoning_content = '' %}
|
||||
{%- if message.reasoning_content is string %}
|
||||
{%- set reasoning_content = message.reasoning_content %}
|
||||
{%- else %}
|
||||
{%- if '</think>' in content %}
|
||||
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
||||
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- if loop.index0 > ns.last_query_index %}
|
||||
{%- if loop.last or (not loop.last and reasoning_content) %}
|
||||
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- if message.tool_calls %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if (loop.first and content) or (not loop.first) %}
|
||||
{{- '\n' }}
|
||||
{%- endif %}
|
||||
{%- if tool_call.function %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '<tool_call>\n{"name": "' }}
|
||||
{{- tool_call.name }}
|
||||
{{- '", "arguments": ' }}
|
||||
{%- if tool_call.arguments is string %}
|
||||
{{- tool_call.arguments }}
|
||||
{%- else %}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{%- endif %}
|
||||
{{- '}\n</tool_call>' }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif message.role == "tool" %}
|
||||
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
||||
{{- '<|im_start|>user' }}
|
||||
{%- endif %}
|
||||
{{- '\n<tool_response>\n' }}
|
||||
{{- content }}
|
||||
{{- '\n</tool_response>' }}
|
||||
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|im_start|>assistant\n' }}
|
||||
{%- if enable_thinking is defined and enable_thinking is false %}
|
||||
{{- '<think>\n\n</think>\n\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
151388
adapter/merges.txt
Normal file
151388
adapter/merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
25
adapter/special_tokens_map.json
Normal file
25
adapter/special_tokens_map.json
Normal file
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "<|endoftext|>"
|
||||
}
|
||||
BIN
adapter/tokenizer.json
(Stored with Git LFS)
Normal file
BIN
adapter/tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
239
adapter/tokenizer_config.json
Normal file
239
adapter/tokenizer_config.json
Normal file
@@ -0,0 +1,239 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151665": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151666": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151667": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151668": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 131072,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null
|
||||
}
|
||||
3
adapter/training_args.bin
Normal file
3
adapter/training_args.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b390207fb9a20b960f3ed1741d1294d45a9e140cce1e74163aca8b95afc1fb74
|
||||
size 5944
|
||||
1
adapter/vocab.json
Normal file
1
adapter/vocab.json
Normal file
File diff suppressed because one or more lines are too long
209
checkpoint-313/README.md
Normal file
209
checkpoint-313/README.md
Normal file
@@ -0,0 +1,209 @@
|
||||
---
|
||||
base_model: /home/mic/Documents/junk/claude-finetune-kit/merging/coral-v1.5-4b-out
|
||||
library_name: peft
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- base_model:adapter:/home/mic/Documents/junk/claude-finetune-kit/merging/coral-v1.5-4b-out
|
||||
- lora
|
||||
- sft
|
||||
- transformers
|
||||
- trl
|
||||
---
|
||||
|
||||
# Model Card for Model ID
|
||||
|
||||
<!-- Provide a quick summary of what the model is/does. -->
|
||||
|
||||
|
||||
|
||||
## Model Details
|
||||
|
||||
### Model Description
|
||||
|
||||
<!-- Provide a longer summary of what this model is. -->
|
||||
|
||||
|
||||
|
||||
- **Developed by:** [More Information Needed]
|
||||
- **Funded by [optional]:** [More Information Needed]
|
||||
- **Shared by [optional]:** [More Information Needed]
|
||||
- **Model type:** [More Information Needed]
|
||||
- **Language(s) (NLP):** [More Information Needed]
|
||||
- **License:** [More Information Needed]
|
||||
- **Finetuned from model [optional]:** [More Information Needed]
|
||||
|
||||
### Model Sources [optional]
|
||||
|
||||
<!-- Provide the basic links for the model. -->
|
||||
|
||||
- **Repository:** [More Information Needed]
|
||||
- **Paper [optional]:** [More Information Needed]
|
||||
- **Demo [optional]:** [More Information Needed]
|
||||
|
||||
## Uses
|
||||
|
||||
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
||||
|
||||
### Direct Use
|
||||
|
||||
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Downstream Use [optional]
|
||||
|
||||
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Out-of-Scope Use
|
||||
|
||||
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Bias, Risks, and Limitations
|
||||
|
||||
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Recommendations
|
||||
|
||||
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
||||
|
||||
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
||||
|
||||
## How to Get Started with the Model
|
||||
|
||||
Use the code below to get started with the model.
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Training Details
|
||||
|
||||
### Training Data
|
||||
|
||||
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Training Procedure
|
||||
|
||||
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
||||
|
||||
#### Preprocessing [optional]
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
|
||||
#### Training Hyperparameters
|
||||
|
||||
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
||||
|
||||
#### Speeds, Sizes, Times [optional]
|
||||
|
||||
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Evaluation
|
||||
|
||||
<!-- This section describes the evaluation protocols and provides the results. -->
|
||||
|
||||
### Testing Data, Factors & Metrics
|
||||
|
||||
#### Testing Data
|
||||
|
||||
<!-- This should link to a Dataset Card if possible. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Factors
|
||||
|
||||
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Metrics
|
||||
|
||||
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Results
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Summary
|
||||
|
||||
|
||||
|
||||
## Model Examination [optional]
|
||||
|
||||
<!-- Relevant interpretability work for the model goes here -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Environmental Impact
|
||||
|
||||
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
||||
|
||||
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
||||
|
||||
- **Hardware Type:** [More Information Needed]
|
||||
- **Hours used:** [More Information Needed]
|
||||
- **Cloud Provider:** [More Information Needed]
|
||||
- **Compute Region:** [More Information Needed]
|
||||
- **Carbon Emitted:** [More Information Needed]
|
||||
|
||||
## Technical Specifications [optional]
|
||||
|
||||
### Model Architecture and Objective
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Compute Infrastructure
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Hardware
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Software
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Citation [optional]
|
||||
|
||||
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
||||
|
||||
**BibTeX:**
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
**APA:**
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Glossary [optional]
|
||||
|
||||
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## More Information [optional]
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Model Card Authors [optional]
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Model Card Contact
|
||||
|
||||
[More Information Needed]
|
||||
### Framework versions
|
||||
|
||||
- PEFT 0.19.1
|
||||
48
checkpoint-313/adapter_config.json
Normal file
48
checkpoint-313/adapter_config.json
Normal file
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"alora_invocation_tokens": null,
|
||||
"alpha_pattern": {},
|
||||
"arrow_config": null,
|
||||
"auto_mapping": null,
|
||||
"base_model_name_or_path": "/home/mic/Documents/junk/claude-finetune-kit/merging/coral-v1.5-4b-out",
|
||||
"bias": "none",
|
||||
"corda_config": null,
|
||||
"ensure_weight_tying": false,
|
||||
"eva_config": null,
|
||||
"exclude_modules": null,
|
||||
"fan_in_fan_out": false,
|
||||
"inference_mode": true,
|
||||
"init_lora_weights": true,
|
||||
"layer_replication": null,
|
||||
"layers_pattern": null,
|
||||
"layers_to_transform": null,
|
||||
"loftq_config": {},
|
||||
"lora_alpha": 32,
|
||||
"lora_bias": false,
|
||||
"lora_dropout": 0.05,
|
||||
"lora_ga_config": null,
|
||||
"megatron_config": null,
|
||||
"megatron_core": "megatron.core",
|
||||
"modules_to_save": null,
|
||||
"peft_type": "LORA",
|
||||
"peft_version": "0.19.1",
|
||||
"qalora_group_size": 16,
|
||||
"r": 8,
|
||||
"rank_pattern": {},
|
||||
"revision": null,
|
||||
"target_modules": [
|
||||
"q_proj",
|
||||
"down_proj",
|
||||
"o_proj",
|
||||
"gate_proj",
|
||||
"k_proj",
|
||||
"v_proj",
|
||||
"up_proj"
|
||||
],
|
||||
"target_parameters": null,
|
||||
"task_type": "CAUSAL_LM",
|
||||
"trainable_token_indices": null,
|
||||
"use_bdlora": null,
|
||||
"use_dora": false,
|
||||
"use_qalora": false,
|
||||
"use_rslora": false
|
||||
}
|
||||
3
checkpoint-313/adapter_model.safetensors
Normal file
3
checkpoint-313/adapter_model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a9b26eab60a983dce716ad38c55837dd4f5ef8e3b160e84a66c2df7da621b8c4
|
||||
size 33096960
|
||||
28
checkpoint-313/added_tokens.json
Normal file
28
checkpoint-313/added_tokens.json
Normal file
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"</think>": 151668,
|
||||
"</tool_call>": 151658,
|
||||
"</tool_response>": 151666,
|
||||
"<think>": 151667,
|
||||
"<tool_call>": 151657,
|
||||
"<tool_response>": 151665,
|
||||
"<|box_end|>": 151649,
|
||||
"<|box_start|>": 151648,
|
||||
"<|endoftext|>": 151643,
|
||||
"<|file_sep|>": 151664,
|
||||
"<|fim_middle|>": 151660,
|
||||
"<|fim_pad|>": 151662,
|
||||
"<|fim_prefix|>": 151659,
|
||||
"<|fim_suffix|>": 151661,
|
||||
"<|im_end|>": 151645,
|
||||
"<|im_start|>": 151644,
|
||||
"<|image_pad|>": 151655,
|
||||
"<|object_ref_end|>": 151647,
|
||||
"<|object_ref_start|>": 151646,
|
||||
"<|quad_end|>": 151651,
|
||||
"<|quad_start|>": 151650,
|
||||
"<|repo_name|>": 151663,
|
||||
"<|video_pad|>": 151656,
|
||||
"<|vision_end|>": 151653,
|
||||
"<|vision_pad|>": 151654,
|
||||
"<|vision_start|>": 151652
|
||||
}
|
||||
89
checkpoint-313/chat_template.jinja
Normal file
89
checkpoint-313/chat_template.jinja
Normal file
@@ -0,0 +1,89 @@
|
||||
{%- if tools %}
|
||||
{{- '<|im_start|>system\n' }}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- messages[0].content + '\n\n' }}
|
||||
{%- endif %}
|
||||
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||
{%- for tool in tools %}
|
||||
{{- "\n" }}
|
||||
{{- tool | tojson }}
|
||||
{%- endfor %}
|
||||
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||
{%- else %}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
||||
{%- for message in messages[::-1] %}
|
||||
{%- set index = (messages|length - 1) - loop.index0 %}
|
||||
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
||||
{%- set ns.multi_step_tool = false %}
|
||||
{%- set ns.last_query_index = index %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- for message in messages %}
|
||||
{%- if message.content is string %}
|
||||
{%- set content = message.content %}
|
||||
{%- else %}
|
||||
{%- set content = '' %}
|
||||
{%- endif %}
|
||||
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
||||
{%- elif message.role == "assistant" %}
|
||||
{%- set reasoning_content = '' %}
|
||||
{%- if message.reasoning_content is string %}
|
||||
{%- set reasoning_content = message.reasoning_content %}
|
||||
{%- else %}
|
||||
{%- if '</think>' in content %}
|
||||
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
||||
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- if loop.index0 > ns.last_query_index %}
|
||||
{%- if loop.last or (not loop.last and reasoning_content) %}
|
||||
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- if message.tool_calls %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if (loop.first and content) or (not loop.first) %}
|
||||
{{- '\n' }}
|
||||
{%- endif %}
|
||||
{%- if tool_call.function %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '<tool_call>\n{"name": "' }}
|
||||
{{- tool_call.name }}
|
||||
{{- '", "arguments": ' }}
|
||||
{%- if tool_call.arguments is string %}
|
||||
{{- tool_call.arguments }}
|
||||
{%- else %}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{%- endif %}
|
||||
{{- '}\n</tool_call>' }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif message.role == "tool" %}
|
||||
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
||||
{{- '<|im_start|>user' }}
|
||||
{%- endif %}
|
||||
{{- '\n<tool_response>\n' }}
|
||||
{{- content }}
|
||||
{{- '\n</tool_response>' }}
|
||||
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|im_start|>assistant\n' }}
|
||||
{%- if enable_thinking is defined and enable_thinking is false %}
|
||||
{{- '<think>\n\n</think>\n\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
151388
checkpoint-313/merges.txt
Normal file
151388
checkpoint-313/merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
checkpoint-313/optimizer.pt
Normal file
3
checkpoint-313/optimizer.pt
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5c2eed3444d664075319becb21fe5fb3eec415f0b9cf8191c3d04442b10f8167
|
||||
size 34141428
|
||||
3
checkpoint-313/rng_state.pth
Normal file
3
checkpoint-313/rng_state.pth
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7e1502e6a563eae95508fafefe0a939051c664dad5906ae11fc8f7612e30d080
|
||||
size 14244
|
||||
3
checkpoint-313/scheduler.pt
Normal file
3
checkpoint-313/scheduler.pt
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d2a0297902bafe2b3e5c576d25f2baab92a7d29d5efa1d2b6e98d93419cf0ed3
|
||||
size 1064
|
||||
25
checkpoint-313/special_tokens_map.json
Normal file
25
checkpoint-313/special_tokens_map.json
Normal file
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "<|endoftext|>"
|
||||
}
|
||||
BIN
checkpoint-313/tokenizer.json
(Stored with Git LFS)
Normal file
BIN
checkpoint-313/tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
239
checkpoint-313/tokenizer_config.json
Normal file
239
checkpoint-313/tokenizer_config.json
Normal file
@@ -0,0 +1,239 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151665": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151666": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151667": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151668": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 131072,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null
|
||||
}
|
||||
344
checkpoint-313/trainer_state.json
Normal file
344
checkpoint-313/trainer_state.json
Normal file
@@ -0,0 +1,344 @@
|
||||
{
|
||||
"best_global_step": null,
|
||||
"best_metric": null,
|
||||
"best_model_checkpoint": null,
|
||||
"epoch": 1.0,
|
||||
"eval_steps": 500,
|
||||
"global_step": 313,
|
||||
"is_hyper_param_search": false,
|
||||
"is_local_process_zero": true,
|
||||
"is_world_process_zero": true,
|
||||
"log_history": [
|
||||
{
|
||||
"entropy": 0.6765631157904863,
|
||||
"epoch": 0.032,
|
||||
"grad_norm": 2.875,
|
||||
"learning_rate": 9.473684210526316e-05,
|
||||
"loss": 3.33,
|
||||
"mean_token_accuracy": 0.5463392566889524,
|
||||
"num_tokens": 32197.0,
|
||||
"step": 10
|
||||
},
|
||||
{
|
||||
"entropy": 1.1614421313628553,
|
||||
"epoch": 0.064,
|
||||
"grad_norm": 1.7734375,
|
||||
"learning_rate": 0.0002,
|
||||
"loss": 1.6539,
|
||||
"mean_token_accuracy": 0.6597556989639998,
|
||||
"num_tokens": 64756.0,
|
||||
"step": 20
|
||||
},
|
||||
{
|
||||
"entropy": 0.939408042281866,
|
||||
"epoch": 0.096,
|
||||
"grad_norm": 0.60546875,
|
||||
"learning_rate": 0.0001998660954234636,
|
||||
"loss": 0.7808,
|
||||
"mean_token_accuracy": 0.781595965847373,
|
||||
"num_tokens": 95410.0,
|
||||
"step": 30
|
||||
},
|
||||
{
|
||||
"entropy": 0.9574495874345302,
|
||||
"epoch": 0.128,
|
||||
"grad_norm": 0.55078125,
|
||||
"learning_rate": 0.00019946474030256675,
|
||||
"loss": 0.859,
|
||||
"mean_token_accuracy": 0.7595490481704473,
|
||||
"num_tokens": 123619.0,
|
||||
"step": 40
|
||||
},
|
||||
{
|
||||
"entropy": 0.8130528301000595,
|
||||
"epoch": 0.16,
|
||||
"grad_norm": 0.498046875,
|
||||
"learning_rate": 0.00019879700950305957,
|
||||
"loss": 0.7411,
|
||||
"mean_token_accuracy": 0.8014852415770293,
|
||||
"num_tokens": 156066.0,
|
||||
"step": 50
|
||||
},
|
||||
{
|
||||
"entropy": 0.8076014310121536,
|
||||
"epoch": 0.192,
|
||||
"grad_norm": 0.67578125,
|
||||
"learning_rate": 0.00019786469126914098,
|
||||
"loss": 0.8028,
|
||||
"mean_token_accuracy": 0.7921789325773716,
|
||||
"num_tokens": 186345.0,
|
||||
"step": 60
|
||||
},
|
||||
{
|
||||
"entropy": 0.7697046212852001,
|
||||
"epoch": 0.224,
|
||||
"grad_norm": 0.58984375,
|
||||
"learning_rate": 0.0001966702824343772,
|
||||
"loss": 0.7332,
|
||||
"mean_token_accuracy": 0.7981064647436142,
|
||||
"num_tokens": 219449.0,
|
||||
"step": 70
|
||||
},
|
||||
{
|
||||
"entropy": 0.7365534473210573,
|
||||
"epoch": 0.256,
|
||||
"grad_norm": 0.66015625,
|
||||
"learning_rate": 0.0001952169817349528,
|
||||
"loss": 0.6818,
|
||||
"mean_token_accuracy": 0.8138540856540203,
|
||||
"num_tokens": 252398.0,
|
||||
"step": 80
|
||||
},
|
||||
{
|
||||
"entropy": 0.8327326748520136,
|
||||
"epoch": 0.288,
|
||||
"grad_norm": 0.447265625,
|
||||
"learning_rate": 0.0001935086812431626,
|
||||
"loss": 0.7288,
|
||||
"mean_token_accuracy": 0.7888761077076196,
|
||||
"num_tokens": 281833.0,
|
||||
"step": 90
|
||||
},
|
||||
{
|
||||
"entropy": 0.8304124176502228,
|
||||
"epoch": 0.32,
|
||||
"grad_norm": 0.57421875,
|
||||
"learning_rate": 0.0001915499559440855,
|
||||
"loss": 0.7666,
|
||||
"mean_token_accuracy": 0.7884823590517044,
|
||||
"num_tokens": 313194.0,
|
||||
"step": 100
|
||||
},
|
||||
{
|
||||
"entropy": 0.7173997653648257,
|
||||
"epoch": 0.352,
|
||||
"grad_norm": 0.609375,
|
||||
"learning_rate": 0.00018934605148335608,
|
||||
"loss": 0.6474,
|
||||
"mean_token_accuracy": 0.8159435257315636,
|
||||
"num_tokens": 345629.0,
|
||||
"step": 110
|
||||
},
|
||||
{
|
||||
"entropy": 0.7204551868140697,
|
||||
"epoch": 0.384,
|
||||
"grad_norm": 0.6640625,
|
||||
"learning_rate": 0.00018690287011884508,
|
||||
"loss": 0.674,
|
||||
"mean_token_accuracy": 0.8059780243784189,
|
||||
"num_tokens": 379281.0,
|
||||
"step": 120
|
||||
},
|
||||
{
|
||||
"entropy": 0.7626429490745068,
|
||||
"epoch": 0.416,
|
||||
"grad_norm": 0.470703125,
|
||||
"learning_rate": 0.00018422695491387284,
|
||||
"loss": 0.6938,
|
||||
"mean_token_accuracy": 0.8045202873647213,
|
||||
"num_tokens": 410187.0,
|
||||
"step": 130
|
||||
},
|
||||
{
|
||||
"entropy": 0.7266103811562061,
|
||||
"epoch": 0.448,
|
||||
"grad_norm": 0.484375,
|
||||
"learning_rate": 0.00018132547221428675,
|
||||
"loss": 0.7094,
|
||||
"mean_token_accuracy": 0.8129661556333303,
|
||||
"num_tokens": 442476.0,
|
||||
"step": 140
|
||||
},
|
||||
{
|
||||
"entropy": 0.7414866480976343,
|
||||
"epoch": 0.48,
|
||||
"grad_norm": 0.6875,
|
||||
"learning_rate": 0.00017820619245633115,
|
||||
"loss": 0.6545,
|
||||
"mean_token_accuracy": 0.8073488272726536,
|
||||
"num_tokens": 472224.0,
|
||||
"step": 150
|
||||
},
|
||||
{
|
||||
"entropy": 0.7822258766740561,
|
||||
"epoch": 0.512,
|
||||
"grad_norm": 0.54296875,
|
||||
"learning_rate": 0.00017487746935670776,
|
||||
"loss": 0.7011,
|
||||
"mean_token_accuracy": 0.795686560869217,
|
||||
"num_tokens": 501232.0,
|
||||
"step": 160
|
||||
},
|
||||
{
|
||||
"entropy": 0.745093465037644,
|
||||
"epoch": 0.544,
|
||||
"grad_norm": 0.55078125,
|
||||
"learning_rate": 0.00017134821754055782,
|
||||
"loss": 0.7028,
|
||||
"mean_token_accuracy": 0.8048942796885967,
|
||||
"num_tokens": 531476.0,
|
||||
"step": 170
|
||||
},
|
||||
{
|
||||
"entropy": 0.7522185202687979,
|
||||
"epoch": 0.576,
|
||||
"grad_norm": 0.55078125,
|
||||
"learning_rate": 0.00016762788866728,
|
||||
"loss": 0.7073,
|
||||
"mean_token_accuracy": 0.8059838317334652,
|
||||
"num_tokens": 564807.0,
|
||||
"step": 180
|
||||
},
|
||||
{
|
||||
"entropy": 0.7345199611037969,
|
||||
"epoch": 0.608,
|
||||
"grad_norm": 0.62890625,
|
||||
"learning_rate": 0.0001637264461181213,
|
||||
"loss": 0.6915,
|
||||
"mean_token_accuracy": 0.80965431407094,
|
||||
"num_tokens": 597440.0,
|
||||
"step": 190
|
||||
},
|
||||
{
|
||||
"entropy": 0.8049266938120127,
|
||||
"epoch": 0.64,
|
||||
"grad_norm": 0.486328125,
|
||||
"learning_rate": 0.0001596543383133303,
|
||||
"loss": 0.7237,
|
||||
"mean_token_accuracy": 0.7935581188648939,
|
||||
"num_tokens": 630050.0,
|
||||
"step": 200
|
||||
},
|
||||
{
|
||||
"entropy": 0.6652943708002568,
|
||||
"epoch": 0.672,
|
||||
"grad_norm": 0.52734375,
|
||||
"learning_rate": 0.00015542247073033116,
|
||||
"loss": 0.6553,
|
||||
"mean_token_accuracy": 0.8136730916798115,
|
||||
"num_tokens": 660867.0,
|
||||
"step": 210
|
||||
},
|
||||
{
|
||||
"entropy": 0.6704742344096303,
|
||||
"epoch": 0.704,
|
||||
"grad_norm": 0.55078125,
|
||||
"learning_rate": 0.00015104217669785713,
|
||||
"loss": 0.6572,
|
||||
"mean_token_accuracy": 0.8189906783401966,
|
||||
"num_tokens": 694781.0,
|
||||
"step": 220
|
||||
},
|
||||
{
|
||||
"entropy": 0.8146899126470089,
|
||||
"epoch": 0.736,
|
||||
"grad_norm": 0.6015625,
|
||||
"learning_rate": 0.0001465251870442586,
|
||||
"loss": 0.7881,
|
||||
"mean_token_accuracy": 0.7911321885883809,
|
||||
"num_tokens": 727207.0,
|
||||
"step": 230
|
||||
},
|
||||
{
|
||||
"entropy": 0.6836955199018121,
|
||||
"epoch": 0.768,
|
||||
"grad_norm": 0.53515625,
|
||||
"learning_rate": 0.00014188359868127134,
|
||||
"loss": 0.6743,
|
||||
"mean_token_accuracy": 0.8147166304290294,
|
||||
"num_tokens": 760836.0,
|
||||
"step": 240
|
||||
},
|
||||
{
|
||||
"entropy": 0.8043613195419311,
|
||||
"epoch": 0.8,
|
||||
"grad_norm": 0.48828125,
|
||||
"learning_rate": 0.00013712984220737938,
|
||||
"loss": 0.7463,
|
||||
"mean_token_accuracy": 0.7957966670393943,
|
||||
"num_tokens": 794412.0,
|
||||
"step": 250
|
||||
},
|
||||
{
|
||||
"entropy": 0.6497382273897528,
|
||||
"epoch": 0.832,
|
||||
"grad_norm": 0.5703125,
|
||||
"learning_rate": 0.00013227664861753454,
|
||||
"loss": 0.6431,
|
||||
"mean_token_accuracy": 0.8244783975183964,
|
||||
"num_tokens": 827099.0,
|
||||
"step": 260
|
||||
},
|
||||
{
|
||||
"entropy": 0.8308647315949201,
|
||||
"epoch": 0.864,
|
||||
"grad_norm": 0.609375,
|
||||
"learning_rate": 0.00012733701520838681,
|
||||
"loss": 0.7665,
|
||||
"mean_token_accuracy": 0.7912245698273181,
|
||||
"num_tokens": 858402.0,
|
||||
"step": 270
|
||||
},
|
||||
{
|
||||
"entropy": 0.6155038874596357,
|
||||
"epoch": 0.896,
|
||||
"grad_norm": 0.54296875,
|
||||
"learning_rate": 0.00012232417077033414,
|
||||
"loss": 0.6085,
|
||||
"mean_token_accuracy": 0.8311799973249435,
|
||||
"num_tokens": 887820.0,
|
||||
"step": 280
|
||||
},
|
||||
{
|
||||
"entropy": 0.7490381442010403,
|
||||
"epoch": 0.928,
|
||||
"grad_norm": 0.4921875,
|
||||
"learning_rate": 0.00011725154015961088,
|
||||
"loss": 0.6872,
|
||||
"mean_token_accuracy": 0.8019387792795897,
|
||||
"num_tokens": 918679.0,
|
||||
"step": 290
|
||||
},
|
||||
{
|
||||
"entropy": 0.7262276472523809,
|
||||
"epoch": 0.96,
|
||||
"grad_norm": 0.458984375,
|
||||
"learning_rate": 0.0001121327083452942,
|
||||
"loss": 0.7096,
|
||||
"mean_token_accuracy": 0.8102037977427244,
|
||||
"num_tokens": 951444.0,
|
||||
"step": 300
|
||||
},
|
||||
{
|
||||
"entropy": 0.6726594571024179,
|
||||
"epoch": 0.992,
|
||||
"grad_norm": 0.5625,
|
||||
"learning_rate": 0.00010698138402751314,
|
||||
"loss": 0.6115,
|
||||
"mean_token_accuracy": 0.8189524404704571,
|
||||
"num_tokens": 983476.0,
|
||||
"step": 310
|
||||
}
|
||||
],
|
||||
"logging_steps": 10,
|
||||
"max_steps": 626,
|
||||
"num_input_tokens_seen": 0,
|
||||
"num_train_epochs": 2,
|
||||
"save_steps": 500,
|
||||
"stateful_callbacks": {
|
||||
"TrainerControl": {
|
||||
"args": {
|
||||
"should_epoch_stop": false,
|
||||
"should_evaluate": false,
|
||||
"should_log": false,
|
||||
"should_save": true,
|
||||
"should_training_stop": false
|
||||
},
|
||||
"attributes": {}
|
||||
}
|
||||
},
|
||||
"total_flos": 2.1729056081918976e+16,
|
||||
"train_batch_size": 1,
|
||||
"trial_name": null,
|
||||
"trial_params": null
|
||||
}
|
||||
3
checkpoint-313/training_args.bin
Normal file
3
checkpoint-313/training_args.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b390207fb9a20b960f3ed1741d1294d45a9e140cce1e74163aca8b95afc1fb74
|
||||
size 5944
|
||||
1
checkpoint-313/vocab.json
Normal file
1
checkpoint-313/vocab.json
Normal file
File diff suppressed because one or more lines are too long
209
checkpoint-626/README.md
Normal file
209
checkpoint-626/README.md
Normal file
@@ -0,0 +1,209 @@
|
||||
---
|
||||
base_model: /home/mic/Documents/junk/claude-finetune-kit/merging/coral-v1.5-4b-out
|
||||
library_name: peft
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- base_model:adapter:/home/mic/Documents/junk/claude-finetune-kit/merging/coral-v1.5-4b-out
|
||||
- lora
|
||||
- sft
|
||||
- transformers
|
||||
- trl
|
||||
---
|
||||
|
||||
# Model Card for Model ID
|
||||
|
||||
<!-- Provide a quick summary of what the model is/does. -->
|
||||
|
||||
|
||||
|
||||
## Model Details
|
||||
|
||||
### Model Description
|
||||
|
||||
<!-- Provide a longer summary of what this model is. -->
|
||||
|
||||
|
||||
|
||||
- **Developed by:** [More Information Needed]
|
||||
- **Funded by [optional]:** [More Information Needed]
|
||||
- **Shared by [optional]:** [More Information Needed]
|
||||
- **Model type:** [More Information Needed]
|
||||
- **Language(s) (NLP):** [More Information Needed]
|
||||
- **License:** [More Information Needed]
|
||||
- **Finetuned from model [optional]:** [More Information Needed]
|
||||
|
||||
### Model Sources [optional]
|
||||
|
||||
<!-- Provide the basic links for the model. -->
|
||||
|
||||
- **Repository:** [More Information Needed]
|
||||
- **Paper [optional]:** [More Information Needed]
|
||||
- **Demo [optional]:** [More Information Needed]
|
||||
|
||||
## Uses
|
||||
|
||||
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
||||
|
||||
### Direct Use
|
||||
|
||||
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Downstream Use [optional]
|
||||
|
||||
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Out-of-Scope Use
|
||||
|
||||
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Bias, Risks, and Limitations
|
||||
|
||||
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Recommendations
|
||||
|
||||
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
||||
|
||||
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
||||
|
||||
## How to Get Started with the Model
|
||||
|
||||
Use the code below to get started with the model.
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Training Details
|
||||
|
||||
### Training Data
|
||||
|
||||
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Training Procedure
|
||||
|
||||
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
||||
|
||||
#### Preprocessing [optional]
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
|
||||
#### Training Hyperparameters
|
||||
|
||||
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
||||
|
||||
#### Speeds, Sizes, Times [optional]
|
||||
|
||||
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Evaluation
|
||||
|
||||
<!-- This section describes the evaluation protocols and provides the results. -->
|
||||
|
||||
### Testing Data, Factors & Metrics
|
||||
|
||||
#### Testing Data
|
||||
|
||||
<!-- This should link to a Dataset Card if possible. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Factors
|
||||
|
||||
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Metrics
|
||||
|
||||
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Results
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Summary
|
||||
|
||||
|
||||
|
||||
## Model Examination [optional]
|
||||
|
||||
<!-- Relevant interpretability work for the model goes here -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Environmental Impact
|
||||
|
||||
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
||||
|
||||
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
||||
|
||||
- **Hardware Type:** [More Information Needed]
|
||||
- **Hours used:** [More Information Needed]
|
||||
- **Cloud Provider:** [More Information Needed]
|
||||
- **Compute Region:** [More Information Needed]
|
||||
- **Carbon Emitted:** [More Information Needed]
|
||||
|
||||
## Technical Specifications [optional]
|
||||
|
||||
### Model Architecture and Objective
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
### Compute Infrastructure
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Hardware
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
#### Software
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Citation [optional]
|
||||
|
||||
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
||||
|
||||
**BibTeX:**
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
**APA:**
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Glossary [optional]
|
||||
|
||||
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## More Information [optional]
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Model Card Authors [optional]
|
||||
|
||||
[More Information Needed]
|
||||
|
||||
## Model Card Contact
|
||||
|
||||
[More Information Needed]
|
||||
### Framework versions
|
||||
|
||||
- PEFT 0.19.1
|
||||
48
checkpoint-626/adapter_config.json
Normal file
48
checkpoint-626/adapter_config.json
Normal file
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"alora_invocation_tokens": null,
|
||||
"alpha_pattern": {},
|
||||
"arrow_config": null,
|
||||
"auto_mapping": null,
|
||||
"base_model_name_or_path": "/home/mic/Documents/junk/claude-finetune-kit/merging/coral-v1.5-4b-out",
|
||||
"bias": "none",
|
||||
"corda_config": null,
|
||||
"ensure_weight_tying": false,
|
||||
"eva_config": null,
|
||||
"exclude_modules": null,
|
||||
"fan_in_fan_out": false,
|
||||
"inference_mode": true,
|
||||
"init_lora_weights": true,
|
||||
"layer_replication": null,
|
||||
"layers_pattern": null,
|
||||
"layers_to_transform": null,
|
||||
"loftq_config": {},
|
||||
"lora_alpha": 32,
|
||||
"lora_bias": false,
|
||||
"lora_dropout": 0.05,
|
||||
"lora_ga_config": null,
|
||||
"megatron_config": null,
|
||||
"megatron_core": "megatron.core",
|
||||
"modules_to_save": null,
|
||||
"peft_type": "LORA",
|
||||
"peft_version": "0.19.1",
|
||||
"qalora_group_size": 16,
|
||||
"r": 8,
|
||||
"rank_pattern": {},
|
||||
"revision": null,
|
||||
"target_modules": [
|
||||
"q_proj",
|
||||
"down_proj",
|
||||
"o_proj",
|
||||
"gate_proj",
|
||||
"k_proj",
|
||||
"v_proj",
|
||||
"up_proj"
|
||||
],
|
||||
"target_parameters": null,
|
||||
"task_type": "CAUSAL_LM",
|
||||
"trainable_token_indices": null,
|
||||
"use_bdlora": null,
|
||||
"use_dora": false,
|
||||
"use_qalora": false,
|
||||
"use_rslora": false
|
||||
}
|
||||
3
checkpoint-626/adapter_model.safetensors
Normal file
3
checkpoint-626/adapter_model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:fe4dfe024a77157cc820fbeb335d8b13820df7184bc48498d4c0e779c60e0e2f
|
||||
size 33096960
|
||||
28
checkpoint-626/added_tokens.json
Normal file
28
checkpoint-626/added_tokens.json
Normal file
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"</think>": 151668,
|
||||
"</tool_call>": 151658,
|
||||
"</tool_response>": 151666,
|
||||
"<think>": 151667,
|
||||
"<tool_call>": 151657,
|
||||
"<tool_response>": 151665,
|
||||
"<|box_end|>": 151649,
|
||||
"<|box_start|>": 151648,
|
||||
"<|endoftext|>": 151643,
|
||||
"<|file_sep|>": 151664,
|
||||
"<|fim_middle|>": 151660,
|
||||
"<|fim_pad|>": 151662,
|
||||
"<|fim_prefix|>": 151659,
|
||||
"<|fim_suffix|>": 151661,
|
||||
"<|im_end|>": 151645,
|
||||
"<|im_start|>": 151644,
|
||||
"<|image_pad|>": 151655,
|
||||
"<|object_ref_end|>": 151647,
|
||||
"<|object_ref_start|>": 151646,
|
||||
"<|quad_end|>": 151651,
|
||||
"<|quad_start|>": 151650,
|
||||
"<|repo_name|>": 151663,
|
||||
"<|video_pad|>": 151656,
|
||||
"<|vision_end|>": 151653,
|
||||
"<|vision_pad|>": 151654,
|
||||
"<|vision_start|>": 151652
|
||||
}
|
||||
89
checkpoint-626/chat_template.jinja
Normal file
89
checkpoint-626/chat_template.jinja
Normal file
@@ -0,0 +1,89 @@
|
||||
{%- if tools %}
|
||||
{{- '<|im_start|>system\n' }}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- messages[0].content + '\n\n' }}
|
||||
{%- endif %}
|
||||
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||
{%- for tool in tools %}
|
||||
{{- "\n" }}
|
||||
{{- tool | tojson }}
|
||||
{%- endfor %}
|
||||
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||
{%- else %}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
||||
{%- for message in messages[::-1] %}
|
||||
{%- set index = (messages|length - 1) - loop.index0 %}
|
||||
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
||||
{%- set ns.multi_step_tool = false %}
|
||||
{%- set ns.last_query_index = index %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- for message in messages %}
|
||||
{%- if message.content is string %}
|
||||
{%- set content = message.content %}
|
||||
{%- else %}
|
||||
{%- set content = '' %}
|
||||
{%- endif %}
|
||||
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
||||
{%- elif message.role == "assistant" %}
|
||||
{%- set reasoning_content = '' %}
|
||||
{%- if message.reasoning_content is string %}
|
||||
{%- set reasoning_content = message.reasoning_content %}
|
||||
{%- else %}
|
||||
{%- if '</think>' in content %}
|
||||
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
||||
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- if loop.index0 > ns.last_query_index %}
|
||||
{%- if loop.last or (not loop.last and reasoning_content) %}
|
||||
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- if message.tool_calls %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if (loop.first and content) or (not loop.first) %}
|
||||
{{- '\n' }}
|
||||
{%- endif %}
|
||||
{%- if tool_call.function %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '<tool_call>\n{"name": "' }}
|
||||
{{- tool_call.name }}
|
||||
{{- '", "arguments": ' }}
|
||||
{%- if tool_call.arguments is string %}
|
||||
{{- tool_call.arguments }}
|
||||
{%- else %}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{%- endif %}
|
||||
{{- '}\n</tool_call>' }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif message.role == "tool" %}
|
||||
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
||||
{{- '<|im_start|>user' }}
|
||||
{%- endif %}
|
||||
{{- '\n<tool_response>\n' }}
|
||||
{{- content }}
|
||||
{{- '\n</tool_response>' }}
|
||||
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|im_start|>assistant\n' }}
|
||||
{%- if enable_thinking is defined and enable_thinking is false %}
|
||||
{{- '<think>\n\n</think>\n\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
151388
checkpoint-626/merges.txt
Normal file
151388
checkpoint-626/merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
checkpoint-626/optimizer.pt
Normal file
3
checkpoint-626/optimizer.pt
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:04d5b5571b31b403982b6d8809ea39456915692db9b39c85337a0180f6cb66ad
|
||||
size 34141428
|
||||
3
checkpoint-626/rng_state.pth
Normal file
3
checkpoint-626/rng_state.pth
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e740bb3ac1ad7f3018eb6fbda16dd7cb5d53f2ee2e045b20fae4130d548ea79b
|
||||
size 14244
|
||||
3
checkpoint-626/scheduler.pt
Normal file
3
checkpoint-626/scheduler.pt
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:46d020344d4cdb625fa3ccdabe828dfdae277b492bb5282dbb9f16edd7c73161
|
||||
size 1064
|
||||
25
checkpoint-626/special_tokens_map.json
Normal file
25
checkpoint-626/special_tokens_map.json
Normal file
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "<|endoftext|>"
|
||||
}
|
||||
BIN
checkpoint-626/tokenizer.json
(Stored with Git LFS)
Normal file
BIN
checkpoint-626/tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
239
checkpoint-626/tokenizer_config.json
Normal file
239
checkpoint-626/tokenizer_config.json
Normal file
@@ -0,0 +1,239 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151665": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151666": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151667": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151668": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 131072,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null
|
||||
}
|
||||
654
checkpoint-626/trainer_state.json
Normal file
654
checkpoint-626/trainer_state.json
Normal file
@@ -0,0 +1,654 @@
|
||||
{
|
||||
"best_global_step": null,
|
||||
"best_metric": null,
|
||||
"best_model_checkpoint": null,
|
||||
"epoch": 2.0,
|
||||
"eval_steps": 500,
|
||||
"global_step": 626,
|
||||
"is_hyper_param_search": false,
|
||||
"is_local_process_zero": true,
|
||||
"is_world_process_zero": true,
|
||||
"log_history": [
|
||||
{
|
||||
"entropy": 0.6765631157904863,
|
||||
"epoch": 0.032,
|
||||
"grad_norm": 2.875,
|
||||
"learning_rate": 9.473684210526316e-05,
|
||||
"loss": 3.33,
|
||||
"mean_token_accuracy": 0.5463392566889524,
|
||||
"num_tokens": 32197.0,
|
||||
"step": 10
|
||||
},
|
||||
{
|
||||
"entropy": 1.1614421313628553,
|
||||
"epoch": 0.064,
|
||||
"grad_norm": 1.7734375,
|
||||
"learning_rate": 0.0002,
|
||||
"loss": 1.6539,
|
||||
"mean_token_accuracy": 0.6597556989639998,
|
||||
"num_tokens": 64756.0,
|
||||
"step": 20
|
||||
},
|
||||
{
|
||||
"entropy": 0.939408042281866,
|
||||
"epoch": 0.096,
|
||||
"grad_norm": 0.60546875,
|
||||
"learning_rate": 0.0001998660954234636,
|
||||
"loss": 0.7808,
|
||||
"mean_token_accuracy": 0.781595965847373,
|
||||
"num_tokens": 95410.0,
|
||||
"step": 30
|
||||
},
|
||||
{
|
||||
"entropy": 0.9574495874345302,
|
||||
"epoch": 0.128,
|
||||
"grad_norm": 0.55078125,
|
||||
"learning_rate": 0.00019946474030256675,
|
||||
"loss": 0.859,
|
||||
"mean_token_accuracy": 0.7595490481704473,
|
||||
"num_tokens": 123619.0,
|
||||
"step": 40
|
||||
},
|
||||
{
|
||||
"entropy": 0.8130528301000595,
|
||||
"epoch": 0.16,
|
||||
"grad_norm": 0.498046875,
|
||||
"learning_rate": 0.00019879700950305957,
|
||||
"loss": 0.7411,
|
||||
"mean_token_accuracy": 0.8014852415770293,
|
||||
"num_tokens": 156066.0,
|
||||
"step": 50
|
||||
},
|
||||
{
|
||||
"entropy": 0.8076014310121536,
|
||||
"epoch": 0.192,
|
||||
"grad_norm": 0.67578125,
|
||||
"learning_rate": 0.00019786469126914098,
|
||||
"loss": 0.8028,
|
||||
"mean_token_accuracy": 0.7921789325773716,
|
||||
"num_tokens": 186345.0,
|
||||
"step": 60
|
||||
},
|
||||
{
|
||||
"entropy": 0.7697046212852001,
|
||||
"epoch": 0.224,
|
||||
"grad_norm": 0.58984375,
|
||||
"learning_rate": 0.0001966702824343772,
|
||||
"loss": 0.7332,
|
||||
"mean_token_accuracy": 0.7981064647436142,
|
||||
"num_tokens": 219449.0,
|
||||
"step": 70
|
||||
},
|
||||
{
|
||||
"entropy": 0.7365534473210573,
|
||||
"epoch": 0.256,
|
||||
"grad_norm": 0.66015625,
|
||||
"learning_rate": 0.0001952169817349528,
|
||||
"loss": 0.6818,
|
||||
"mean_token_accuracy": 0.8138540856540203,
|
||||
"num_tokens": 252398.0,
|
||||
"step": 80
|
||||
},
|
||||
{
|
||||
"entropy": 0.8327326748520136,
|
||||
"epoch": 0.288,
|
||||
"grad_norm": 0.447265625,
|
||||
"learning_rate": 0.0001935086812431626,
|
||||
"loss": 0.7288,
|
||||
"mean_token_accuracy": 0.7888761077076196,
|
||||
"num_tokens": 281833.0,
|
||||
"step": 90
|
||||
},
|
||||
{
|
||||
"entropy": 0.8304124176502228,
|
||||
"epoch": 0.32,
|
||||
"grad_norm": 0.57421875,
|
||||
"learning_rate": 0.0001915499559440855,
|
||||
"loss": 0.7666,
|
||||
"mean_token_accuracy": 0.7884823590517044,
|
||||
"num_tokens": 313194.0,
|
||||
"step": 100
|
||||
},
|
||||
{
|
||||
"entropy": 0.7173997653648257,
|
||||
"epoch": 0.352,
|
||||
"grad_norm": 0.609375,
|
||||
"learning_rate": 0.00018934605148335608,
|
||||
"loss": 0.6474,
|
||||
"mean_token_accuracy": 0.8159435257315636,
|
||||
"num_tokens": 345629.0,
|
||||
"step": 110
|
||||
},
|
||||
{
|
||||
"entropy": 0.7204551868140697,
|
||||
"epoch": 0.384,
|
||||
"grad_norm": 0.6640625,
|
||||
"learning_rate": 0.00018690287011884508,
|
||||
"loss": 0.674,
|
||||
"mean_token_accuracy": 0.8059780243784189,
|
||||
"num_tokens": 379281.0,
|
||||
"step": 120
|
||||
},
|
||||
{
|
||||
"entropy": 0.7626429490745068,
|
||||
"epoch": 0.416,
|
||||
"grad_norm": 0.470703125,
|
||||
"learning_rate": 0.00018422695491387284,
|
||||
"loss": 0.6938,
|
||||
"mean_token_accuracy": 0.8045202873647213,
|
||||
"num_tokens": 410187.0,
|
||||
"step": 130
|
||||
},
|
||||
{
|
||||
"entropy": 0.7266103811562061,
|
||||
"epoch": 0.448,
|
||||
"grad_norm": 0.484375,
|
||||
"learning_rate": 0.00018132547221428675,
|
||||
"loss": 0.7094,
|
||||
"mean_token_accuracy": 0.8129661556333303,
|
||||
"num_tokens": 442476.0,
|
||||
"step": 140
|
||||
},
|
||||
{
|
||||
"entropy": 0.7414866480976343,
|
||||
"epoch": 0.48,
|
||||
"grad_norm": 0.6875,
|
||||
"learning_rate": 0.00017820619245633115,
|
||||
"loss": 0.6545,
|
||||
"mean_token_accuracy": 0.8073488272726536,
|
||||
"num_tokens": 472224.0,
|
||||
"step": 150
|
||||
},
|
||||
{
|
||||
"entropy": 0.7822258766740561,
|
||||
"epoch": 0.512,
|
||||
"grad_norm": 0.54296875,
|
||||
"learning_rate": 0.00017487746935670776,
|
||||
"loss": 0.7011,
|
||||
"mean_token_accuracy": 0.795686560869217,
|
||||
"num_tokens": 501232.0,
|
||||
"step": 160
|
||||
},
|
||||
{
|
||||
"entropy": 0.745093465037644,
|
||||
"epoch": 0.544,
|
||||
"grad_norm": 0.55078125,
|
||||
"learning_rate": 0.00017134821754055782,
|
||||
"loss": 0.7028,
|
||||
"mean_token_accuracy": 0.8048942796885967,
|
||||
"num_tokens": 531476.0,
|
||||
"step": 170
|
||||
},
|
||||
{
|
||||
"entropy": 0.7522185202687979,
|
||||
"epoch": 0.576,
|
||||
"grad_norm": 0.55078125,
|
||||
"learning_rate": 0.00016762788866728,
|
||||
"loss": 0.7073,
|
||||
"mean_token_accuracy": 0.8059838317334652,
|
||||
"num_tokens": 564807.0,
|
||||
"step": 180
|
||||
},
|
||||
{
|
||||
"entropy": 0.7345199611037969,
|
||||
"epoch": 0.608,
|
||||
"grad_norm": 0.62890625,
|
||||
"learning_rate": 0.0001637264461181213,
|
||||
"loss": 0.6915,
|
||||
"mean_token_accuracy": 0.80965431407094,
|
||||
"num_tokens": 597440.0,
|
||||
"step": 190
|
||||
},
|
||||
{
|
||||
"entropy": 0.8049266938120127,
|
||||
"epoch": 0.64,
|
||||
"grad_norm": 0.486328125,
|
||||
"learning_rate": 0.0001596543383133303,
|
||||
"loss": 0.7237,
|
||||
"mean_token_accuracy": 0.7935581188648939,
|
||||
"num_tokens": 630050.0,
|
||||
"step": 200
|
||||
},
|
||||
{
|
||||
"entropy": 0.6652943708002568,
|
||||
"epoch": 0.672,
|
||||
"grad_norm": 0.52734375,
|
||||
"learning_rate": 0.00015542247073033116,
|
||||
"loss": 0.6553,
|
||||
"mean_token_accuracy": 0.8136730916798115,
|
||||
"num_tokens": 660867.0,
|
||||
"step": 210
|
||||
},
|
||||
{
|
||||
"entropy": 0.6704742344096303,
|
||||
"epoch": 0.704,
|
||||
"grad_norm": 0.55078125,
|
||||
"learning_rate": 0.00015104217669785713,
|
||||
"loss": 0.6572,
|
||||
"mean_token_accuracy": 0.8189906783401966,
|
||||
"num_tokens": 694781.0,
|
||||
"step": 220
|
||||
},
|
||||
{
|
||||
"entropy": 0.8146899126470089,
|
||||
"epoch": 0.736,
|
||||
"grad_norm": 0.6015625,
|
||||
"learning_rate": 0.0001465251870442586,
|
||||
"loss": 0.7881,
|
||||
"mean_token_accuracy": 0.7911321885883809,
|
||||
"num_tokens": 727207.0,
|
||||
"step": 230
|
||||
},
|
||||
{
|
||||
"entropy": 0.6836955199018121,
|
||||
"epoch": 0.768,
|
||||
"grad_norm": 0.53515625,
|
||||
"learning_rate": 0.00014188359868127134,
|
||||
"loss": 0.6743,
|
||||
"mean_token_accuracy": 0.8147166304290294,
|
||||
"num_tokens": 760836.0,
|
||||
"step": 240
|
||||
},
|
||||
{
|
||||
"entropy": 0.8043613195419311,
|
||||
"epoch": 0.8,
|
||||
"grad_norm": 0.48828125,
|
||||
"learning_rate": 0.00013712984220737938,
|
||||
"loss": 0.7463,
|
||||
"mean_token_accuracy": 0.7957966670393943,
|
||||
"num_tokens": 794412.0,
|
||||
"step": 250
|
||||
},
|
||||
{
|
||||
"entropy": 0.6497382273897528,
|
||||
"epoch": 0.832,
|
||||
"grad_norm": 0.5703125,
|
||||
"learning_rate": 0.00013227664861753454,
|
||||
"loss": 0.6431,
|
||||
"mean_token_accuracy": 0.8244783975183964,
|
||||
"num_tokens": 827099.0,
|
||||
"step": 260
|
||||
},
|
||||
{
|
||||
"entropy": 0.8308647315949201,
|
||||
"epoch": 0.864,
|
||||
"grad_norm": 0.609375,
|
||||
"learning_rate": 0.00012733701520838681,
|
||||
"loss": 0.7665,
|
||||
"mean_token_accuracy": 0.7912245698273181,
|
||||
"num_tokens": 858402.0,
|
||||
"step": 270
|
||||
},
|
||||
{
|
||||
"entropy": 0.6155038874596357,
|
||||
"epoch": 0.896,
|
||||
"grad_norm": 0.54296875,
|
||||
"learning_rate": 0.00012232417077033414,
|
||||
"loss": 0.6085,
|
||||
"mean_token_accuracy": 0.8311799973249435,
|
||||
"num_tokens": 887820.0,
|
||||
"step": 280
|
||||
},
|
||||
{
|
||||
"entropy": 0.7490381442010403,
|
||||
"epoch": 0.928,
|
||||
"grad_norm": 0.4921875,
|
||||
"learning_rate": 0.00011725154015961088,
|
||||
"loss": 0.6872,
|
||||
"mean_token_accuracy": 0.8019387792795897,
|
||||
"num_tokens": 918679.0,
|
||||
"step": 290
|
||||
},
|
||||
{
|
||||
"entropy": 0.7262276472523809,
|
||||
"epoch": 0.96,
|
||||
"grad_norm": 0.458984375,
|
||||
"learning_rate": 0.0001121327083452942,
|
||||
"loss": 0.7096,
|
||||
"mean_token_accuracy": 0.8102037977427244,
|
||||
"num_tokens": 951444.0,
|
||||
"step": 300
|
||||
},
|
||||
{
|
||||
"entropy": 0.6726594571024179,
|
||||
"epoch": 0.992,
|
||||
"grad_norm": 0.5625,
|
||||
"learning_rate": 0.00010698138402751314,
|
||||
"loss": 0.6115,
|
||||
"mean_token_accuracy": 0.8189524404704571,
|
||||
"num_tokens": 983476.0,
|
||||
"step": 310
|
||||
},
|
||||
{
|
||||
"entropy": 0.6149586082289094,
|
||||
"epoch": 1.0224,
|
||||
"grad_norm": 0.515625,
|
||||
"learning_rate": 0.00010181136292429528,
|
||||
"loss": 0.5614,
|
||||
"mean_token_accuracy": 0.840733151686819,
|
||||
"num_tokens": 1014089.0,
|
||||
"step": 320
|
||||
},
|
||||
{
|
||||
"entropy": 0.6150874266400933,
|
||||
"epoch": 1.0544,
|
||||
"grad_norm": 0.6328125,
|
||||
"learning_rate": 9.663649082537075e-05,
|
||||
"loss": 0.5826,
|
||||
"mean_token_accuracy": 0.8344197846949101,
|
||||
"num_tokens": 1048175.0,
|
||||
"step": 330
|
||||
},
|
||||
{
|
||||
"entropy": 0.688701249845326,
|
||||
"epoch": 1.0864,
|
||||
"grad_norm": 0.63671875,
|
||||
"learning_rate": 9.147062651188034e-05,
|
||||
"loss": 0.582,
|
||||
"mean_token_accuracy": 0.822768672183156,
|
||||
"num_tokens": 1077167.0,
|
||||
"step": 340
|
||||
},
|
||||
{
|
||||
"entropy": 0.6307795088738203,
|
||||
"epoch": 1.1184,
|
||||
"grad_norm": 0.7890625,
|
||||
"learning_rate": 8.632760464129085e-05,
|
||||
"loss": 0.6128,
|
||||
"mean_token_accuracy": 0.8314752034842968,
|
||||
"num_tokens": 1106931.0,
|
||||
"step": 350
|
||||
},
|
||||
{
|
||||
"entropy": 0.5494781306013465,
|
||||
"epoch": 1.1504,
|
||||
"grad_norm": 0.69921875,
|
||||
"learning_rate": 8.122119869691636e-05,
|
||||
"loss": 0.5115,
|
||||
"mean_token_accuracy": 0.8432619094848632,
|
||||
"num_tokens": 1140267.0,
|
||||
"step": 360
|
||||
},
|
||||
{
|
||||
"entropy": 0.5744671380147338,
|
||||
"epoch": 1.1824,
|
||||
"grad_norm": 0.703125,
|
||||
"learning_rate": 7.616508410126881e-05,
|
||||
"loss": 0.5383,
|
||||
"mean_token_accuracy": 0.8436760693788529,
|
||||
"num_tokens": 1174016.0,
|
||||
"step": 370
|
||||
},
|
||||
{
|
||||
"entropy": 0.7073690647259354,
|
||||
"epoch": 1.2144,
|
||||
"grad_norm": 0.68359375,
|
||||
"learning_rate": 7.11728015920253e-05,
|
||||
"loss": 0.6142,
|
||||
"mean_token_accuracy": 0.815117885172367,
|
||||
"num_tokens": 1203362.0,
|
||||
"step": 380
|
||||
},
|
||||
{
|
||||
"entropy": 0.6476075932383537,
|
||||
"epoch": 1.2464,
|
||||
"grad_norm": 0.59375,
|
||||
"learning_rate": 6.625772095869278e-05,
|
||||
"loss": 0.576,
|
||||
"mean_token_accuracy": 0.8274026662111282,
|
||||
"num_tokens": 1233733.0,
|
||||
"step": 390
|
||||
},
|
||||
{
|
||||
"entropy": 0.6079700440168381,
|
||||
"epoch": 1.2784,
|
||||
"grad_norm": 0.75390625,
|
||||
"learning_rate": 6.143300523708828e-05,
|
||||
"loss": 0.5926,
|
||||
"mean_token_accuracy": 0.8361447125673294,
|
||||
"num_tokens": 1266408.0,
|
||||
"step": 400
|
||||
},
|
||||
{
|
||||
"entropy": 0.710476384870708,
|
||||
"epoch": 1.3104,
|
||||
"grad_norm": 0.62890625,
|
||||
"learning_rate": 5.6711575457523904e-05,
|
||||
"loss": 0.6673,
|
||||
"mean_token_accuracy": 0.8070973232388496,
|
||||
"num_tokens": 1297692.0,
|
||||
"step": 410
|
||||
},
|
||||
{
|
||||
"entropy": 0.6220435939729214,
|
||||
"epoch": 1.3424,
|
||||
"grad_norm": 0.58984375,
|
||||
"learning_rate": 5.210607604110537e-05,
|
||||
"loss": 0.535,
|
||||
"mean_token_accuracy": 0.8349618587642909,
|
||||
"num_tokens": 1328647.0,
|
||||
"step": 420
|
||||
},
|
||||
{
|
||||
"entropy": 0.636304665543139,
|
||||
"epoch": 1.3744,
|
||||
"grad_norm": 0.71875,
|
||||
"learning_rate": 4.762884093681439e-05,
|
||||
"loss": 0.5681,
|
||||
"mean_token_accuracy": 0.8337738730013371,
|
||||
"num_tokens": 1360726.0,
|
||||
"step": 430
|
||||
},
|
||||
{
|
||||
"entropy": 0.5927035486325621,
|
||||
"epoch": 1.4064,
|
||||
"grad_norm": 0.640625,
|
||||
"learning_rate": 4.329186059006498e-05,
|
||||
"loss": 0.5686,
|
||||
"mean_token_accuracy": 0.843344721570611,
|
||||
"num_tokens": 1392566.0,
|
||||
"step": 440
|
||||
},
|
||||
{
|
||||
"entropy": 0.5271363547071815,
|
||||
"epoch": 1.4384000000000001,
|
||||
"grad_norm": 0.78515625,
|
||||
"learning_rate": 3.9106749831192655e-05,
|
||||
"loss": 0.4787,
|
||||
"mean_token_accuracy": 0.8514938339591026,
|
||||
"num_tokens": 1423983.0,
|
||||
"step": 450
|
||||
},
|
||||
{
|
||||
"entropy": 0.5616443024948239,
|
||||
"epoch": 1.4704,
|
||||
"grad_norm": 0.609375,
|
||||
"learning_rate": 3.50847167698759e-05,
|
||||
"loss": 0.562,
|
||||
"mean_token_accuracy": 0.8439631327986717,
|
||||
"num_tokens": 1457630.0,
|
||||
"step": 460
|
||||
},
|
||||
{
|
||||
"entropy": 0.6092825196683407,
|
||||
"epoch": 1.5024,
|
||||
"grad_norm": 0.55078125,
|
||||
"learning_rate": 3.123653277879256e-05,
|
||||
"loss": 0.5474,
|
||||
"mean_token_accuracy": 0.8354076690971851,
|
||||
"num_tokens": 1487297.0,
|
||||
"step": 470
|
||||
},
|
||||
{
|
||||
"entropy": 0.6257630828768015,
|
||||
"epoch": 1.5344,
|
||||
"grad_norm": 0.7265625,
|
||||
"learning_rate": 2.7572503646897817e-05,
|
||||
"loss": 0.5658,
|
||||
"mean_token_accuracy": 0.8328265182673931,
|
||||
"num_tokens": 1520243.0,
|
||||
"step": 480
|
||||
},
|
||||
{
|
||||
"entropy": 0.6298206310719252,
|
||||
"epoch": 1.5664,
|
||||
"grad_norm": 0.65625,
|
||||
"learning_rate": 2.410244197957815e-05,
|
||||
"loss": 0.5689,
|
||||
"mean_token_accuracy": 0.8329555809497833,
|
||||
"num_tokens": 1552070.0,
|
||||
"step": 490
|
||||
},
|
||||
{
|
||||
"entropy": 0.639691986143589,
|
||||
"epoch": 1.5984,
|
||||
"grad_norm": 0.81640625,
|
||||
"learning_rate": 2.0835640919595878e-05,
|
||||
"loss": 0.5927,
|
||||
"mean_token_accuracy": 0.834860210865736,
|
||||
"num_tokens": 1583470.0,
|
||||
"step": 500
|
||||
},
|
||||
{
|
||||
"entropy": 0.6124603090807795,
|
||||
"epoch": 1.6303999999999998,
|
||||
"grad_norm": 0.6875,
|
||||
"learning_rate": 1.7780849259202394e-05,
|
||||
"loss": 0.5623,
|
||||
"mean_token_accuracy": 0.8385609880089759,
|
||||
"num_tokens": 1615277.0,
|
||||
"step": 510
|
||||
},
|
||||
{
|
||||
"entropy": 0.5668893689289689,
|
||||
"epoch": 1.6623999999999999,
|
||||
"grad_norm": 0.625,
|
||||
"learning_rate": 1.494624801007144e-05,
|
||||
"loss": 0.5323,
|
||||
"mean_token_accuracy": 0.8450584053993225,
|
||||
"num_tokens": 1648050.0,
|
||||
"step": 520
|
||||
},
|
||||
{
|
||||
"entropy": 0.6057403447106481,
|
||||
"epoch": 1.6944,
|
||||
"grad_norm": 0.63671875,
|
||||
"learning_rate": 1.2339428493801386e-05,
|
||||
"loss": 0.5769,
|
||||
"mean_token_accuracy": 0.835369348526001,
|
||||
"num_tokens": 1680674.0,
|
||||
"step": 530
|
||||
},
|
||||
{
|
||||
"entropy": 0.6277687439695001,
|
||||
"epoch": 1.7264,
|
||||
"grad_norm": 0.78125,
|
||||
"learning_rate": 9.967372011660813e-06,
|
||||
"loss": 0.5623,
|
||||
"mean_token_accuracy": 0.8294578112661839,
|
||||
"num_tokens": 1710429.0,
|
||||
"step": 540
|
||||
},
|
||||
{
|
||||
"entropy": 0.6762315159663558,
|
||||
"epoch": 1.7584,
|
||||
"grad_norm": 0.6796875,
|
||||
"learning_rate": 7.836431148025003e-06,
|
||||
"loss": 0.5577,
|
||||
"mean_token_accuracy": 0.8203907363116741,
|
||||
"num_tokens": 1739709.0,
|
||||
"step": 550
|
||||
},
|
||||
{
|
||||
"entropy": 0.6461215754970908,
|
||||
"epoch": 1.7904,
|
||||
"grad_norm": 0.71875,
|
||||
"learning_rate": 5.9523127575733415e-06,
|
||||
"loss": 0.5774,
|
||||
"mean_token_accuracy": 0.8332192912697792,
|
||||
"num_tokens": 1770745.0,
|
||||
"step": 560
|
||||
},
|
||||
{
|
||||
"entropy": 0.6354038525372744,
|
||||
"epoch": 1.8224,
|
||||
"grad_norm": 0.73046875,
|
||||
"learning_rate": 4.320062681810167e-06,
|
||||
"loss": 0.6049,
|
||||
"mean_token_accuracy": 0.8269469507038594,
|
||||
"num_tokens": 1801462.0,
|
||||
"step": 570
|
||||
},
|
||||
{
|
||||
"entropy": 0.6516931420192122,
|
||||
"epoch": 1.8544,
|
||||
"grad_norm": 0.6796875,
|
||||
"learning_rate": 2.9440522358394227e-06,
|
||||
"loss": 0.5778,
|
||||
"mean_token_accuracy": 0.8247631914913655,
|
||||
"num_tokens": 1834635.0,
|
||||
"step": 580
|
||||
},
|
||||
{
|
||||
"entropy": 0.6212354129180312,
|
||||
"epoch": 1.8864,
|
||||
"grad_norm": 0.76171875,
|
||||
"learning_rate": 1.8279665015826342e-06,
|
||||
"loss": 0.546,
|
||||
"mean_token_accuracy": 0.8374237649142742,
|
||||
"num_tokens": 1865151.0,
|
||||
"step": 590
|
||||
},
|
||||
{
|
||||
"entropy": 0.6352507598698139,
|
||||
"epoch": 1.9184,
|
||||
"grad_norm": 0.70703125,
|
||||
"learning_rate": 9.747944587923163e-07,
|
||||
"loss": 0.6198,
|
||||
"mean_token_accuracy": 0.8281367566436529,
|
||||
"num_tokens": 1897057.0,
|
||||
"step": 600
|
||||
},
|
||||
{
|
||||
"entropy": 0.5361566325649619,
|
||||
"epoch": 1.9504000000000001,
|
||||
"grad_norm": 0.84375,
|
||||
"learning_rate": 3.86820980290481e-07,
|
||||
"loss": 0.5049,
|
||||
"mean_token_accuracy": 0.8522287331521511,
|
||||
"num_tokens": 1930326.0,
|
||||
"step": 610
|
||||
},
|
||||
{
|
||||
"entropy": 0.5473375553265214,
|
||||
"epoch": 1.9824000000000002,
|
||||
"grad_norm": 0.65625,
|
||||
"learning_rate": 6.562071287022908e-08,
|
||||
"loss": 0.5049,
|
||||
"mean_token_accuracy": 0.8522140845656395,
|
||||
"num_tokens": 1964962.0,
|
||||
"step": 620
|
||||
}
|
||||
],
|
||||
"logging_steps": 10,
|
||||
"max_steps": 626,
|
||||
"num_input_tokens_seen": 0,
|
||||
"num_train_epochs": 2,
|
||||
"save_steps": 500,
|
||||
"stateful_callbacks": {
|
||||
"TrainerControl": {
|
||||
"args": {
|
||||
"should_epoch_stop": false,
|
||||
"should_evaluate": false,
|
||||
"should_log": false,
|
||||
"should_save": true,
|
||||
"should_training_stop": true
|
||||
},
|
||||
"attributes": {}
|
||||
}
|
||||
},
|
||||
"total_flos": 4.345811216383795e+16,
|
||||
"train_batch_size": 1,
|
||||
"trial_name": null,
|
||||
"trial_params": null
|
||||
}
|
||||
3
checkpoint-626/training_args.bin
Normal file
3
checkpoint-626/training_args.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b390207fb9a20b960f3ed1741d1294d45a9e140cce1e74163aca8b95afc1fb74
|
||||
size 5944
|
||||
1
checkpoint-626/vocab.json
Normal file
1
checkpoint-626/vocab.json
Normal file
File diff suppressed because one or more lines are too long
28
merged/added_tokens.json
Normal file
28
merged/added_tokens.json
Normal file
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"</think>": 151668,
|
||||
"</tool_call>": 151658,
|
||||
"</tool_response>": 151666,
|
||||
"<think>": 151667,
|
||||
"<tool_call>": 151657,
|
||||
"<tool_response>": 151665,
|
||||
"<|box_end|>": 151649,
|
||||
"<|box_start|>": 151648,
|
||||
"<|endoftext|>": 151643,
|
||||
"<|file_sep|>": 151664,
|
||||
"<|fim_middle|>": 151660,
|
||||
"<|fim_pad|>": 151662,
|
||||
"<|fim_prefix|>": 151659,
|
||||
"<|fim_suffix|>": 151661,
|
||||
"<|im_end|>": 151645,
|
||||
"<|im_start|>": 151644,
|
||||
"<|image_pad|>": 151655,
|
||||
"<|object_ref_end|>": 151647,
|
||||
"<|object_ref_start|>": 151646,
|
||||
"<|quad_end|>": 151651,
|
||||
"<|quad_start|>": 151650,
|
||||
"<|repo_name|>": 151663,
|
||||
"<|video_pad|>": 151656,
|
||||
"<|vision_end|>": 151653,
|
||||
"<|vision_pad|>": 151654,
|
||||
"<|vision_start|>": 151652
|
||||
}
|
||||
89
merged/chat_template.jinja
Normal file
89
merged/chat_template.jinja
Normal file
@@ -0,0 +1,89 @@
|
||||
{%- if tools %}
|
||||
{{- '<|im_start|>system\n' }}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- messages[0].content + '\n\n' }}
|
||||
{%- endif %}
|
||||
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||
{%- for tool in tools %}
|
||||
{{- "\n" }}
|
||||
{{- tool | tojson }}
|
||||
{%- endfor %}
|
||||
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||
{%- else %}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
||||
{%- for message in messages[::-1] %}
|
||||
{%- set index = (messages|length - 1) - loop.index0 %}
|
||||
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
||||
{%- set ns.multi_step_tool = false %}
|
||||
{%- set ns.last_query_index = index %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- for message in messages %}
|
||||
{%- if message.content is string %}
|
||||
{%- set content = message.content %}
|
||||
{%- else %}
|
||||
{%- set content = '' %}
|
||||
{%- endif %}
|
||||
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
||||
{%- elif message.role == "assistant" %}
|
||||
{%- set reasoning_content = '' %}
|
||||
{%- if message.reasoning_content is string %}
|
||||
{%- set reasoning_content = message.reasoning_content %}
|
||||
{%- else %}
|
||||
{%- if '</think>' in content %}
|
||||
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
||||
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- if loop.index0 > ns.last_query_index %}
|
||||
{%- if loop.last or (not loop.last and reasoning_content) %}
|
||||
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- if message.tool_calls %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if (loop.first and content) or (not loop.first) %}
|
||||
{{- '\n' }}
|
||||
{%- endif %}
|
||||
{%- if tool_call.function %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '<tool_call>\n{"name": "' }}
|
||||
{{- tool_call.name }}
|
||||
{{- '", "arguments": ' }}
|
||||
{%- if tool_call.arguments is string %}
|
||||
{{- tool_call.arguments }}
|
||||
{%- else %}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{%- endif %}
|
||||
{{- '}\n</tool_call>' }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif message.role == "tool" %}
|
||||
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
||||
{{- '<|im_start|>user' }}
|
||||
{%- endif %}
|
||||
{{- '\n<tool_response>\n' }}
|
||||
{{- content }}
|
||||
{{- '\n</tool_response>' }}
|
||||
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|im_start|>assistant\n' }}
|
||||
{%- if enable_thinking is defined and enable_thinking is false %}
|
||||
{{- '<think>\n\n</think>\n\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
68
merged/config.json
Normal file
68
merged/config.json
Normal file
@@ -0,0 +1,68 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Qwen3ForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"dtype": "float16",
|
||||
"eos_token_id": 151645,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 2560,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 9728,
|
||||
"layer_types": [
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention"
|
||||
],
|
||||
"max_position_embeddings": 40960,
|
||||
"max_window_layers": 36,
|
||||
"model_type": "qwen3",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 36,
|
||||
"num_key_value_heads": 8,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "4.57.6",
|
||||
"use_cache": true,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936
|
||||
}
|
||||
6
merged/generation_config.json
Normal file
6
merged/generation_config.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 151643,
|
||||
"eos_token_id": 151645,
|
||||
"transformers_version": "4.57.6"
|
||||
}
|
||||
151388
merged/merges.txt
Normal file
151388
merged/merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
merged/model-00001-of-00002.safetensors
Normal file
3
merged/model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:800ecf7962b5e007276e82d1bc686516d56d9c5c01e4f58e25c2b3aefe28e790
|
||||
size 4967215128
|
||||
3
merged/model-00002-of-00002.safetensors
Normal file
3
merged/model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:50b33872e07963c729267c0aadfc5f64a5163d26fe98cd46b8d802f6e4265775
|
||||
size 3077766464
|
||||
406
merged/model.safetensors.index.json
Normal file
406
merged/model.safetensors.index.json
Normal file
@@ -0,0 +1,406 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_parameters": 4022468096,
|
||||
"total_size": 8044936192
|
||||
},
|
||||
"weight_map": {
|
||||
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.norm.weight": "model-00002-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
25
merged/special_tokens_map.json
Normal file
25
merged/special_tokens_map.json
Normal file
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "<|endoftext|>"
|
||||
}
|
||||
BIN
merged/tokenizer.json
(Stored with Git LFS)
Normal file
BIN
merged/tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
239
merged/tokenizer_config.json
Normal file
239
merged/tokenizer_config.json
Normal file
@@ -0,0 +1,239 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151665": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151666": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151667": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151668": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 131072,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null
|
||||
}
|
||||
1
merged/vocab.json
Normal file
1
merged/vocab.json
Normal file
File diff suppressed because one or more lines are too long
50
tests/QnA1.txt
Normal file
50
tests/QnA1.txt
Normal file
@@ -0,0 +1,50 @@
|
||||
Q: 47x83
|
||||
A: To multiply 47 by 83, we can break down the multiplication into smaller parts:
|
||||
|
||||
First, multiply 47 by 3 (the ones place of 83):
|
||||
47 x 3 = 141
|
||||
|
||||
Then, multiply 47 by 8 (the tens place of 83):
|
||||
47 x 80 = 3760
|
||||
|
||||
Now, add the two results together:
|
||||
141 + 3760 = 3901
|
||||
|
||||
So, 47 x 83 equals 3901.
|
||||
|
||||
|
||||
Q: my name is mike // what is my name
|
||||
A: Hello, Mike! How can I assist you today? // Your name is Mike.
|
||||
|
||||
Q: lsit 3 fruits
|
||||
A 1. Apple
|
||||
2. Banana
|
||||
3. Orange
|
||||
|
||||
|
||||
Q: who made you
|
||||
A: I was developed by Alibaba Cloud's Tongyi Lab. My name is Qwen.
|
||||
|
||||
Q: write a short python script to check if a number is prime
|
||||
A: `def is_prime(n):
|
||||
if n <= 1:
|
||||
return False
|
||||
if n == 2:
|
||||
return True
|
||||
if n % 2 == 0:
|
||||
return False
|
||||
|
||||
i = 3
|
||||
while i * i <= n:
|
||||
if n % i == 0:
|
||||
return False
|
||||
i += 2
|
||||
|
||||
return True
|
||||
|
||||
# Example usage:
|
||||
num = int(input("Enter a number: "))
|
||||
if is_prime(num):
|
||||
print(f"{num} is a prime number.")
|
||||
else:
|
||||
print(f"{num} is not a prime number.")`
|
||||
17
tests/QnA2.txt
Normal file
17
tests/QnA2.txt
Normal file
@@ -0,0 +1,17 @@
|
||||
Q: A train leaves Chicago at 60mph, another leaves NYC at 80mph, they're 800 miles apart, when do they meet?
|
||||
A: Let's reason step-by-step:
|
||||
1. The train from Chicago is traveling at 60mph, and the train from NYC is traveling at 80mph.
|
||||
2. When two objects move towards each other, their speeds are added together. In this case, the combined speed is 60mph + 80mph = 140mph.
|
||||
3. They are 800 miles apart when they start, and we want to find out how long it takes for them to cover this distance at their combined speed.
|
||||
4. To find the time it takes, we divide the distance by the combined speed: 800 miles / 140mph = approximately 5.71 hours.
|
||||
5. Since there are 60 minutes in an hour, we can convert 0.71 hours to minutes: 0.71 * 60 ≈ 42.6 minutes.
|
||||
6. So, they will meet after approximately 5 hours and 43 minutes.
|
||||
|
||||
Final answer: The trains will meet after approximately 5 hours and 43 minutes.
|
||||
|
||||
|
||||
Q: Write a haiku about quantum physics where every line starts with Q
|
||||
A: Quantum phenomena
|
||||
Quantum entanglement's strange bond,
|
||||
Quantum mysteries.
|
||||
|
||||
132
tests/conv-test.txt
Normal file
132
tests/conv-test.txt
Normal file
@@ -0,0 +1,132 @@
|
||||
> hi
|
||||
|
||||
Hello! How can I assist you today?
|
||||
|
||||
[ Prompt: 69.3 t/s | Generation: 76.2 t/s ]
|
||||
|
||||
> who are you
|
||||
|
||||
I am Qwen, a large-scale language model independently developed by Alibaba Cloud's Tongyi Lab. I support 100 languages, including but not limited to Chinese, English, German, French, Spanish, and many more. I can answer questions, create text such as writing stories, official documents, emails, scripts, perform logical reasoning, code generation, and even express opinions and play games. If you have any questions or need help, please feel free to ask me!
|
||||
|
||||
[ Prompt: 1093.4 t/s | Generation: 71.3 t/s ]
|
||||
|
||||
> what can you do
|
||||
|
||||
I can do the following:
|
||||
|
||||
1. Answer questions: I can answer questions on a wide range of topics, from general knowledge to specific details.
|
||||
|
||||
2. Create text: I can write stories, official documents, emails, scripts, and more, based on the input you provide.
|
||||
|
||||
3. Logical reasoning: I can analyze situations and provide logical conclusions.
|
||||
|
||||
4. Programming: I can write code in various programming languages, such as Python, Java, C++, etc.
|
||||
|
||||
5. Express opinions: I can express my opinions on various topics, although they are based on the information I've been trained on.
|
||||
|
||||
6. Play games: I can engage in text-based games or puzzles with you.
|
||||
|
||||
7. Translate languages: Since I support over 100 languages, I can translate text between these languages.
|
||||
|
||||
Remember, while I strive to provide accurate and helpful responses, I am an AI and may not have all the information at my disposal. If you have a question or need help with something, please feel free to ask me!
|
||||
|
||||
[ Prompt: 676.9 t/s | Generation: 71.5 t/s ]
|
||||
|
||||
> how do you work
|
||||
|
||||
I work by processing input and generating output based on the data I was trained on. Here's a step-by-step overview of how I work:
|
||||
|
||||
1. Input: You provide me with input, which could be a question, statement, instruction, etc.
|
||||
|
||||
2. Tokenization: The input is broken down into smaller units, called tokens. These tokens can be words, subwords, or characters.
|
||||
|
||||
3. Embedding: Each token is converted into a numerical representation called an embedding. This embedding captures the meaning and context of the token.
|
||||
|
||||
4. Processing: The embeddings are then processed through multiple layers of the model. This process involves complex mathematical operations that allow the model to understand the context and meaning of the input.
|
||||
|
||||
5. Output: The final output is a sequence of tokens that form a response to your input. This response is then converted back into a human-readable format.
|
||||
|
||||
6. Training: The model is trained on a vast amount of text data, which allows it to learn patterns, relationships, and facts. The more data it has, the better it can understand and generate text.
|
||||
|
||||
7. Optimization: The model is constantly being optimized and improved through techniques like fine-tuning, transfer learning, and reinforcement learning.
|
||||
|
||||
8. Deployment: Once the model is trained and optimized, it can be deployed to provide services like answering questions, creating text, and more.
|
||||
|
||||
In summary, I work by taking in input, converting it into numerical representations, processing these representations through multiple layers of the model, and then generating an output based on the processed information. The more data I have and the more I'm trained on, the better I can understand and respond to your queries.
|
||||
|
||||
[ Prompt: 660.5 t/s | Generation: 70.1 t/s ]
|
||||
|
||||
> tell me a joke
|
||||
|
||||
Why don't scientists trust atoms? Because they make up everything!
|
||||
|
||||
[ Prompt: 610.8 t/s | Generation: 73.6 t/s ]
|
||||
|
||||
> what is the capital of france
|
||||
|
||||
The capital of France is Paris.
|
||||
|
||||
[ Prompt: 757.7 t/s | Generation: 73.8 t/s ]
|
||||
|
||||
> germany?
|
||||
|
||||
The capital of Germany is Berlin.
|
||||
|
||||
[ Prompt: 980.9 t/s | Generation: 75.1 t/s ]
|
||||
|
||||
> tell me another joke
|
||||
|
||||
Why don't scientists trust atoms? Because they make up everything!
|
||||
|
||||
[ Prompt: 302.3 t/s | Generation: 72.9 t/s ]
|
||||
|
||||
> you already said that one
|
||||
|
||||
I apologize for the repetition. Here's another one:
|
||||
|
||||
Why did the scarecrow win an award? Because he was outstanding in his field!
|
||||
|
||||
[ Prompt: 686.0 t/s | Generation: 71.4 t/s ]
|
||||
|
||||
> whats 5x12
|
||||
|
||||
The product of 5 and 12 is 60.
|
||||
|
||||
[ Prompt: 844.9 t/s | Generation: 72.9 t/s ]
|
||||
|
||||
> how do you know
|
||||
|
||||
I know this because I have been trained on a vast amount of data that includes mathematical concepts and operations. I have been taught to perform calculations like multiplication, addition, subtraction, and division. So, I can determine that 5 multiplied by 12 equals 60.
|
||||
|
||||
[ Prompt: 662.0 t/s | Generation: 69.9 t/s ]
|
||||
|
||||
> whats your name again?
|
||||
|
||||
My name is Qwen, a large-scale language model independently developed by Alibaba Cloud's Tongyi Lab.
|
||||
|
||||
[ Prompt: 737.7 t/s | Generation: 70.6 t/s ]
|
||||
|
||||
> what was the first joke you told me
|
||||
|
||||
The first joke I told you was: "Why don't scientists trust atoms? Because they make up everything!"
|
||||
|
||||
[ Prompt: 885.0 t/s | Generation: 70.5 t/s ]
|
||||
|
||||
> when did i ask the capital of italy
|
||||
|
||||
I don't have access to your conversation history, so I don't know when you asked about the capital of Italy.
|
||||
|
||||
[ Prompt: 941.7 t/s | Generation: 70.7 t/s ]
|
||||
|
||||
> when did i ask you the capital of italy
|
||||
|
||||
I don't have access to your conversation history, so I don't know when you asked about the capital of Italy.
|
||||
|
||||
[ Prompt: 1780.8 t/s | Generation: 70.5 t/s ]
|
||||
|
||||
> what is it
|
||||
|
||||
The capital of Italy is Rome.
|
||||
|
||||
[ Prompt: 590.4 t/s | Generation: 75.7 t/s ]
|
||||
|
||||
Reference in New Issue
Block a user