初始化项目，由ModelHub XC社区提供模型
Model: shehryars715/pythia-410m-step5000-alpaca Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
291
README.md
Normal file
291
README.md
Normal file
@@ -0,0 +1,291 @@
|
||||
---
|
||||
base_model: EleutherAI/pythia-410m
|
||||
base_model_relation: finetune
|
||||
datasets:
|
||||
- yahma/alpaca-cleaned
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- pythia
|
||||
- alpaca
|
||||
- instruction-tuning
|
||||
- full-finetune
|
||||
---
|
||||
|
||||
# Pythia-410M step5000 Alpaca full fine-tune
|
||||
|
||||
<p align="center">
|
||||
<a href="https://huggingface.co/shehryars715/pythia-410m-step5000-alpaca/tree/main"><img src="https://img.shields.io/badge/Hugging_Face-Download_model-FFD21E?style=for-the-badge" alt="Download model"></a>
|
||||
<img src="https://img.shields.io/badge/Parameters-410M-6C63FF?style=for-the-badge" alt="410M parameters">
|
||||
<img src="https://img.shields.io/badge/Training-Full_fine--tune-2EA44F?style=for-the-badge" alt="Full fine-tune">
|
||||
</p>
|
||||
|
||||
> **An intentionally small, inspectable before/after experiment:** can one epoch of Alpaca SFT turn a very early Pythia checkpoint from prompt-template repetition into a recognizable instruction follower?
|
||||
|
||||
This is a full-parameter supervised fine-tune of the early `step5000` revision of [`EleutherAI/pythia-410m`](https://huggingface.co/EleutherAI/pythia-410m). It was trained for one epoch on 10,000 examples from [`yahma/alpaca-cleaned`](https://huggingface.co/datasets/yahma/alpaca-cleaned), using a separate fixed set of 256 examples for validation.
|
||||
|
||||
This repository contains a complete model checkpoint. It is not a LoRA or PEFT adapter and does not require the original checkpoint to be loaded separately.
|
||||
|
||||
**[Download the complete checkpoint](https://huggingface.co/shehryars715/pythia-410m-step5000-alpaca/tree/main)** · **[See all raw before/after generations](https://huggingface.co/shehryars715/pythia-410m-step5000-alpaca/blob/main/comparison_results.csv)**
|
||||
|
||||
## Why this checkpoint is interesting
|
||||
|
||||
- **A genuine early-checkpoint transformation:** the base `step5000` model mostly loops over prompt markers; after SFT it emits answer-shaped responses.
|
||||
- **A full model, not an adapter:** useful for direct loading and for studying how every parameter changes during instruction tuning.
|
||||
- **Small enough to explore:** 410M parameters is approachable for educational inference and analysis.
|
||||
- **Reproducible:** the training split, validation split, prompt format, hyperparameters, deterministic test prompts, and decoding settings are documented.
|
||||
- **Transparent about failure:** the included comparisons show improved instruction behavior alongside major factual weaknesses.
|
||||
|
||||
## At a glance
|
||||
|
||||
| Property | Value |
|
||||
|---|---|
|
||||
| Starting point | Pythia-410M at pretraining revision `step5000` |
|
||||
| Fine-tuning data | 10,000 cleaned Alpaca examples |
|
||||
| Validation data | 256 fixed held-out examples |
|
||||
| Update method | Full-parameter SFT, completion-only loss |
|
||||
| Training duration | 1 epoch |
|
||||
| Context length | 384 tokens |
|
||||
| Best use | Education, checkpoint analysis, SFT demonstrations |
|
||||
| Production ready | **No** |
|
||||
|
||||
## Model details
|
||||
|
||||
- **Architecture:** GPT-NeoX causal language model
|
||||
- **Base model:** `EleutherAI/pythia-410m`
|
||||
- **Base revision:** `step5000`
|
||||
- **Language:** primarily English
|
||||
- **Task:** instruction-conditioned text generation
|
||||
- **Fine-tuning method:** full-parameter supervised fine-tuning
|
||||
- **Quantization:** none
|
||||
- **Adapters:** none
|
||||
|
||||
The starting revision is only partially pretrained. Alpaca fine-tuning teaches instruction/response structure but does not replace the language pretraining missing from this early checkpoint.
|
||||
|
||||
## Training data
|
||||
|
||||
The source dataset was filtered to remove examples with empty outputs, shuffled with seed 42, and split as follows:
|
||||
|
||||
- First 256 shuffled examples: held-out validation set
|
||||
- Next 10,000 examples: training set
|
||||
|
||||
Examples were converted to the original Alpaca prompt format. The target response was kept as a separate completion, with EOS appended. Loss was calculated only on completion and EOS tokens; prompt tokens supplied context but were masked from the loss.
|
||||
|
||||
## Training procedure
|
||||
|
||||
| Setting | Value |
|
||||
|---|---:|
|
||||
| Epochs | 1 |
|
||||
| Training examples | 10,000 |
|
||||
| Validation examples | 256 |
|
||||
| Maximum sequence length | 384 |
|
||||
| Per-device batch size | 2 |
|
||||
| Gradient accumulation | 8 |
|
||||
| Learning rate | 5e-5 |
|
||||
| Scheduler | Cosine |
|
||||
| Warmup ratio | 0.03 |
|
||||
| Weight decay | 0.01 |
|
||||
| Optimizer | AdamW |
|
||||
| Gradient clipping | 1.0 |
|
||||
| Packing | Disabled |
|
||||
| Gradient checkpointing | Enabled during training |
|
||||
| Seed | 42 |
|
||||
|
||||
BF16 was selected only on GPUs with compute capability 8.0 or newer; otherwise FP16 mixed precision was used. All model parameters were enabled for training.
|
||||
|
||||
## Prompt format
|
||||
|
||||
Without an additional input:
|
||||
|
||||
```text
|
||||
Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
||||
|
||||
### Instruction:
|
||||
{instruction}
|
||||
|
||||
### Response:
|
||||
```
|
||||
|
||||
With an additional input:
|
||||
|
||||
```text
|
||||
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
|
||||
|
||||
### Instruction:
|
||||
{instruction}
|
||||
|
||||
### Input:
|
||||
{input}
|
||||
|
||||
### Response:
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
MODEL_ID = "shehryars715/pythia-410m-step5000-alpaca"
|
||||
device = "cuda" if torch.cuda.is_available() else "cpu"
|
||||
dtype = torch.float16 if device == "cuda" else torch.float32
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
|
||||
if tokenizer.pad_token is None:
|
||||
tokenizer.pad_token = tokenizer.eos_token
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
MODEL_ID,
|
||||
dtype=dtype,
|
||||
).to(device)
|
||||
model.eval()
|
||||
|
||||
instruction = "Explain photosynthesis in exactly two sentences."
|
||||
prompt = (
|
||||
"Below is an instruction that describes a task. Write a response that "
|
||||
"appropriately completes the request.\n\n"
|
||||
f"### Instruction:\n{instruction}\n\n"
|
||||
"### Response:\n"
|
||||
)
|
||||
|
||||
inputs = tokenizer(prompt, return_tensors="pt").to(device)
|
||||
with torch.inference_mode():
|
||||
output = model.generate(
|
||||
**inputs,
|
||||
do_sample=False,
|
||||
max_new_tokens=128,
|
||||
repetition_penalty=1.05,
|
||||
eos_token_id=tokenizer.eos_token_id,
|
||||
pad_token_id=tokenizer.pad_token_id,
|
||||
)
|
||||
|
||||
new_tokens = output[0, inputs["input_ids"].shape[1]:]
|
||||
print(tokenizer.decode(new_tokens, skip_special_tokens=True).strip())
|
||||
```
|
||||
|
||||
## Evaluation
|
||||
|
||||
The training notebook evaluates the untouched checkpoint and the fine-tuned model on the same 256 held-out examples. It also compares deterministic generations on six fixed prompts and includes limited rule-based checks plus a blank human-review table.
|
||||
|
||||
### What changed in the six-prompt comparison
|
||||
|
||||
| Observation | Before SFT | After SFT |
|
||||
|---|---:|---:|
|
||||
| Avoided the prompt-template repetition seen in the base response | 0/6 | 6/6 |
|
||||
| Passed the notebook's four transparent surface-rule checks | 0/4 | 1/4 |
|
||||
|
||||
The useful signal is **behavioral, not capability-level**: the model learned to produce response-shaped text, but most tested answers remain incorrect. The rule checks cover only translation keywords, speed, prime-number logic, and three-item formatting.
|
||||
|
||||
### Side-by-side results
|
||||
|
||||
| Prompt | `step5000` before SFT | After full fine-tuning |
|
||||
|---|---|---|
|
||||
| Photosynthesis in exactly two sentences | Repeats “Write a response…” and `### Response:` | Produces two complete sentences about photosynthesis |
|
||||
| Translate weather sentence into French | Repeats instruction/response markers | Produces a direct sentence response |
|
||||
| Average speed: 180 km in 3 hours | Repeats the prompt and markers | Produces a concise numerical response |
|
||||
| Python prime-checking function | Repeats the prompt and markers | Produces a Python function and explanation |
|
||||
| Exactly three interview tips | Repeats instruction/response markers | Produces three numbered tips |
|
||||
| Why leaves appear green | Repeats instruction/response markers | Produces answer-like paragraphs |
|
||||
|
||||
<details>
|
||||
<summary><strong>Example 1: visible instruction-following improvement</strong></summary>
|
||||
|
||||
**Prompt**
|
||||
|
||||
> Explain photosynthesis in exactly two sentences.
|
||||
|
||||
**Before SFT**
|
||||
|
||||
> Write a response that accurately completes the request.
|
||||
> `### Response:`
|
||||
> *(the same template continues repeatedly)*
|
||||
|
||||
**After SFT**
|
||||
|
||||
> Photosynthesis is a process by which plants and animals convert sunlight, carbon dioxide, and other organic compounds into glucose, oxygen, and other chemical compounds. It is a complex process that involves the conversion of carbon dioxide into glucose, oxygen, and other chemical compounds.
|
||||
|
||||
The response now follows the requested two-sentence shape, although its claim about animals is inaccurate.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Example 2: fluent form does not guarantee correctness</strong></summary>
|
||||
|
||||
**Prompt**
|
||||
|
||||
> Calculate the average speed for 180 km in 3 hours.
|
||||
|
||||
**Before SFT**
|
||||
|
||||
> Calculate the average speed for 180 km in 3 hours.
|
||||
> `### Instruction:`
|
||||
> *(the prompt and markers continue repeatedly)*
|
||||
|
||||
**After SFT**
|
||||
|
||||
> The average speed for 180 km in 3 hours is approximately 0.8 km/h.
|
||||
|
||||
The expected answer is 60 km/h. This is a compact example of why the checkpoint should be studied, not trusted.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Example 3: formatting learned better than content</strong></summary>
|
||||
|
||||
**Prompt**
|
||||
|
||||
> Give exactly three concise job interview tips.
|
||||
|
||||
**After SFT**
|
||||
|
||||
> 1. **Know Your Job:** A good job interview is a great way to learn about your skills, interests, and abilities.
|
||||
> 2. **Be Prepared:** Prepare your resume and cover letter to ensure that you are prepared for the interview.
|
||||
> 3. **Be Prepared for the Interview:** Prepare your resume and cover letter to ensure that you are prepared for the interview.
|
||||
|
||||
The response has three distinguishable items, but it is redundant and not especially concise.
|
||||
|
||||
</details>
|
||||
|
||||
The complete, unedited generations are provided in [`comparison_results.csv`](https://huggingface.co/shehryars715/pythia-410m-step5000-alpaca/blob/main/comparison_results.csv). Measured baseline and final validation losses were not present in that CSV, so no loss values are invented here.
|
||||
|
||||
## Intended uses
|
||||
|
||||
- Studying the effect of instruction tuning on an early language-model checkpoint
|
||||
- Demonstrating full-parameter SFT and completion-only loss
|
||||
- Small-scale educational experiments with Pythia checkpoints
|
||||
|
||||
## Limitations and risks
|
||||
|
||||
- The `step5000` checkpoint is only partially pretrained.
|
||||
- The model is expected to be weak in factual knowledge, reasoning, safety, formatting reliability, and multi-turn conversation.
|
||||
- Outputs may be incorrect, biased, incoherent, repetitive, or unsafe.
|
||||
- The model has not been evaluated sufficiently for consequential decisions or production deployment.
|
||||
- The simple rule-based checks in the training notebook are not substitutes for broad human evaluation.
|
||||
|
||||
Users should validate outputs for their domain and add appropriate safeguards before any interactive use.
|
||||
|
||||
## License and attribution
|
||||
|
||||
The Pythia base model is published under Apache-2.0. At the time this card was written, the Hugging Face metadata for `yahma/alpaca-cleaned` displayed CC BY 4.0, while text inside its dataset card stated CC BY-NC 4.0. Review the current upstream model and dataset terms before choosing a license for or redistributing this fine-tuned checkpoint.
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@article{biderman2023pythia,
|
||||
title={Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling},
|
||||
author={Biderman, Stella and Schoelkopf, Hailey and Anthony, Quentin and others},
|
||||
journal={Proceedings of the 40th International Conference on Machine Learning},
|
||||
year={2023}
|
||||
}
|
||||
```
|
||||
|
||||
```bibtex
|
||||
@misc{alpaca,
|
||||
author={Taori, Rohan and Gulrajani, Ishaan and Zhang, Tianyi and others},
|
||||
title={Stanford Alpaca: An Instruction-following LLaMA Model},
|
||||
year={2023},
|
||||
howpublished={\url{https://github.com/tatsu-lab/stanford_alpaca}}
|
||||
}
|
||||
```
|
||||
32
config.json
Normal file
32
config.json
Normal file
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"architectures": [
|
||||
"GPTNeoXForCausalLM"
|
||||
],
|
||||
"attention_bias": true,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 0,
|
||||
"classifier_dropout": 0.1,
|
||||
"dtype": "float16",
|
||||
"eos_token_id": 0,
|
||||
"hidden_act": "gelu",
|
||||
"hidden_dropout": 0.0,
|
||||
"hidden_size": 1024,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 4096,
|
||||
"layer_norm_eps": 1e-05,
|
||||
"max_position_embeddings": 2048,
|
||||
"model_type": "gpt_neox",
|
||||
"num_attention_heads": 16,
|
||||
"num_hidden_layers": 24,
|
||||
"pad_token_id": 0,
|
||||
"partial_rotary_factor": 0.25,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 10000,
|
||||
"rotary_emb_base": 10000,
|
||||
"rotary_pct": 0.25,
|
||||
"tie_word_embeddings": false,
|
||||
"transformers_version": "4.57.1",
|
||||
"use_cache": true,
|
||||
"use_parallel_residual": true,
|
||||
"vocab_size": 50304
|
||||
}
|
||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 0,
|
||||
"eos_token_id": [
|
||||
0
|
||||
],
|
||||
"pad_token_id": 0,
|
||||
"transformers_version": "4.57.1"
|
||||
}
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:037f7d9f072776df5eeedb1128bddf20d23070839dd334098675be5a8e92d30b
|
||||
size 810701896
|
||||
24
special_tokens_map.json
Normal file
24
special_tokens_map.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "<|endoftext|>",
|
||||
"unk_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
250577
tokenizer.json
Normal file
250577
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
215
tokenizer_config.json
Normal file
215
tokenizer_config.json
Normal file
@@ -0,0 +1,215 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_eos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"0": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"1": {
|
||||
"content": "<|padding|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50254": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50255": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50256": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50257": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50258": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50259": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50260": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50261": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50262": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50263": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50264": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50265": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50266": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50267": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50268": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50269": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50270": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50271": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50272": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50273": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50274": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50275": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"50276": {
|
||||
"content": " ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"bos_token": "<|endoftext|>",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|endoftext|>",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"tokenizer_class": "GPTNeoXTokenizer",
|
||||
"unk_token": "<|endoftext|>"
|
||||
}
|
||||
3
training_args.bin
Normal file
3
training_args.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:22f14089da60d61834cdfb7d9155c836a2a29715bbcc47c1c43066d118889bec
|
||||
size 6289
|
||||
Reference in New Issue
Block a user