初始化项目，由ModelHub XC社区提供模型
Model: GODELEV/Test-1-3000 Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
508
README.md
Normal file
508
README.md
Normal file
@@ -0,0 +1,508 @@
|
|||||||
|
---
|
||||||
|
license: mit
|
||||||
|
datasets:
|
||||||
|
- roneneldan/TinyStories
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
tags:
|
||||||
|
- text-generation-inference
|
||||||
|
new_version: GODELEV/Test-1-4000
|
||||||
|
---
|
||||||
|
# Test-1-3000 — A 190M Parameter Narrative Intelligence Engine
|
||||||
|
|
||||||
|
<p align="center">
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
</p>
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Overview
|
||||||
|
|
||||||
|
**Test-1-3000** is a compact yet remarkably capable decoder-only Transformer language model built upon the modern **Llama architecture**.
|
||||||
|
|
||||||
|
The project explores an important question in language model research:
|
||||||
|
|
||||||
|
> *How much narrative reasoning, coherence, and world understanding can emerge inside a small model when trained correctly?*
|
||||||
|
|
||||||
|
Despite containing only **190.55 million parameters**, Test-1-3000 demonstrates surprisingly advanced:
|
||||||
|
|
||||||
|
- Narrative continuity
|
||||||
|
- Character persistence
|
||||||
|
- Long-range memory consistency
|
||||||
|
- Emotional progression
|
||||||
|
- Logical event sequencing
|
||||||
|
- Contextual storytelling stability
|
||||||
|
|
||||||
|
The model was trained specifically for **short-form narrative intelligence**, focusing on coherent storytelling rather than broad internet-scale memorization.
|
||||||
|
|
||||||
|
Unlike many small models that generate fragmented or repetitive text, Test-1-3000 learns to maintain:
|
||||||
|
|
||||||
|
- causal relationships,
|
||||||
|
- stable story worlds,
|
||||||
|
- emotional trajectories,
|
||||||
|
- and meaningful resolutions across long contexts.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Key Highlights
|
||||||
|
|
||||||
|
| Feature | Description |
|
||||||
|
|---|---|
|
||||||
|
| Architecture | Llama-based Decoder-only Transformer |
|
||||||
|
| Parameters | 190.55 Million |
|
||||||
|
| Context Length | 2048 Tokens |
|
||||||
|
| Final Training Step | 3000 |
|
||||||
|
| Final Training Loss | **0.8516** |
|
||||||
|
| Attention Optimization | Flash Attention 2 |
|
||||||
|
| Compilation | `torch.compile` |
|
||||||
|
| Precision | bfloat16 Mixed Precision |
|
||||||
|
| Positional Encoding | Rotary Positional Embeddings (RoPE) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#What Makes Test-1-3000 Special?
|
||||||
|
|
||||||
|
Most compact language models struggle with:
|
||||||
|
|
||||||
|
- maintaining consistency,
|
||||||
|
- remembering earlier events,
|
||||||
|
- resolving story arcs,
|
||||||
|
- and avoiding repetition.
|
||||||
|
|
||||||
|
Test-1-3000 was trained with a different objective philosophy:
|
||||||
|
|
||||||
|
## Narrative Intelligence First
|
||||||
|
|
||||||
|
Instead of optimizing for broad factual memorization, the model focuses on:
|
||||||
|
|
||||||
|
- temporal continuity,
|
||||||
|
- event causality,
|
||||||
|
- emotional logic,
|
||||||
|
- and narrative closure.
|
||||||
|
|
||||||
|
This creates a surprisingly stable storytelling engine capable of generating coherent multi-paragraph narratives with strong thematic flow.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Model Architecture
|
||||||
|
|
||||||
|
Test-1-3000 follows a modern efficient Transformer design optimized for both:
|
||||||
|
|
||||||
|
- training stability,
|
||||||
|
- and inference throughput.
|
||||||
|
|
||||||
|
The architecture borrows heavily from the proven Llama design philosophy while remaining lightweight enough for experimentation and rapid iteration.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Technical Specifications
|
||||||
|
|
||||||
|
| Feature | Specification |
|
||||||
|
|---|---|
|
||||||
|
| Model Type | Decoder-only Transformer |
|
||||||
|
| Hidden Dimension | 768 |
|
||||||
|
| Layers (Depth) | 12 |
|
||||||
|
| Attention Heads | 12 |
|
||||||
|
| Intermediate Size | 3072 |
|
||||||
|
| Activation Function | SwiGLU |
|
||||||
|
| Normalization | RMSNorm |
|
||||||
|
| Vocabulary Size | 50,257 |
|
||||||
|
| Tokenizer | GPT-2 Tokenizer |
|
||||||
|
| Context Window | 2048 Tokens |
|
||||||
|
| Precision | bfloat16 |
|
||||||
|
| Attention Backend | Flash Attention 2 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Positional Understanding with RoPE
|
||||||
|
|
||||||
|
Test-1-3000 uses **Rotary Positional Embeddings (RoPE)** to maintain precise token relationship awareness throughout long contexts.
|
||||||
|
|
||||||
|
This allows the model to:
|
||||||
|
|
||||||
|
- track entities across paragraphs,
|
||||||
|
- preserve story continuity,
|
||||||
|
- maintain dialogue references,
|
||||||
|
- and understand long-range dependencies efficiently.
|
||||||
|
|
||||||
|
For a model of this scale, the 2048-token context window provides unusually strong narrative memory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#The Evolution of Learning
|
||||||
|
|
||||||
|
Training Test-1-3000 revealed clear emergent phases of cognitive development.
|
||||||
|
|
||||||
|
The model did not merely memorize text patterns — it progressively developed increasingly sophisticated representations of narrative structure and world dynamics.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#The Lexical Phase
|
||||||
|
## *(Steps 0 → 250)*
|
||||||
|
|
||||||
|
At the beginning of training, the model learned the statistical foundations of language.
|
||||||
|
|
||||||
|
It discovered:
|
||||||
|
|
||||||
|
- common sentence structures,
|
||||||
|
- punctuation behavior,
|
||||||
|
- frequent vocabulary patterns,
|
||||||
|
- and story-opening syntax.
|
||||||
|
|
||||||
|
During this phase, phrases such as:
|
||||||
|
|
||||||
|
> "Once upon a time"
|
||||||
|
|
||||||
|
became strong narrative anchors.
|
||||||
|
|
||||||
|
The model began constructing basic grammatical fluency but still lacked deeper logical understanding.
|
||||||
|
|
||||||
|
### Characteristics
|
||||||
|
|
||||||
|
- High repetition
|
||||||
|
- Weak memory
|
||||||
|
- Poor event continuity
|
||||||
|
- Basic syntax acquisition
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# The Relational Phase
|
||||||
|
## *(Steps 250 → 1000)*
|
||||||
|
|
||||||
|
The model started connecting concepts together into meaningful relationships.
|
||||||
|
|
||||||
|
It learned:
|
||||||
|
|
||||||
|
- object interactions,
|
||||||
|
- spatial reasoning,
|
||||||
|
- basic causality,
|
||||||
|
- and action consistency.
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
- parks imply trees and playing,
|
||||||
|
- rain implies umbrellas or wetness,
|
||||||
|
- sadness often precedes comfort or resolution.
|
||||||
|
|
||||||
|
The training loss rapidly decreased below **1.5**, signaling major improvements in structural reasoning.
|
||||||
|
|
||||||
|
### Emergent Behaviors
|
||||||
|
|
||||||
|
- Scene consistency
|
||||||
|
- Character-action alignment
|
||||||
|
- Basic emotional logic
|
||||||
|
- Improved descriptive continuity
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# The Coherence Phase
|
||||||
|
## *(Steps 1000 → 2000)*
|
||||||
|
|
||||||
|
This phase marked the emergence of true narrative stabilization.
|
||||||
|
|
||||||
|
The model learned:
|
||||||
|
|
||||||
|
- story pacing,
|
||||||
|
- setup/payoff relationships,
|
||||||
|
- conflict resolution,
|
||||||
|
- and multi-sentence thematic continuity.
|
||||||
|
|
||||||
|
Stories no longer collapsed into unrelated fragments.
|
||||||
|
|
||||||
|
Instead, the model began maintaining:
|
||||||
|
|
||||||
|
- stable goals,
|
||||||
|
- emotional arcs,
|
||||||
|
- and logical conclusions.
|
||||||
|
|
||||||
|
If a story introduced a problem:
|
||||||
|
|
||||||
|
> "Lily was lonely."
|
||||||
|
|
||||||
|
the model increasingly learned to produce meaningful emotional resolutions later in the text.
|
||||||
|
|
||||||
|
### Major Improvements
|
||||||
|
|
||||||
|
- Long-range memory
|
||||||
|
- Reduced contradiction
|
||||||
|
- Better endings
|
||||||
|
- Stronger narrative flow
|
||||||
|
- Lower hallucination frequency
|
||||||
|
|
||||||
|
Final loss at this stage:
|
||||||
|
|
||||||
|
| Step | Loss |
|
||||||
|
|---|---|
|
||||||
|
| 2000 | **1.27** |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# The Emergent Narrative Intelligence Phase
|
||||||
|
## *(Steps 2000 → 3000)*
|
||||||
|
|
||||||
|
This final stage represented a major leap in generative sophistication.
|
||||||
|
|
||||||
|
Rather than simply maintaining coherence, the model began exhibiting signs of:
|
||||||
|
|
||||||
|
- implicit world modeling,
|
||||||
|
- narrative anticipation,
|
||||||
|
- emotional persistence,
|
||||||
|
- and latent planning behavior.
|
||||||
|
|
||||||
|
The model increasingly understood that stories possess:
|
||||||
|
|
||||||
|
- momentum,
|
||||||
|
- consequences,
|
||||||
|
- emotional gravity,
|
||||||
|
- and thematic closure.
|
||||||
|
|
||||||
|
Characters began behaving more consistently across long contexts.
|
||||||
|
|
||||||
|
Events earlier in stories influenced future generations more reliably.
|
||||||
|
|
||||||
|
The model also became significantly better at:
|
||||||
|
|
||||||
|
- avoiding repetitive loops,
|
||||||
|
- maintaining tone,
|
||||||
|
- preserving narrative identity,
|
||||||
|
- and generating cleaner transitions between scenes.
|
||||||
|
|
||||||
|
### Emergent Capabilities
|
||||||
|
|
||||||
|
- Multi-event causal chaining
|
||||||
|
- Persistent emotional tone
|
||||||
|
- Improved dialogue continuity
|
||||||
|
- Better conflict resolution
|
||||||
|
- Reduced topic drift
|
||||||
|
- More natural pacing
|
||||||
|
- Stronger thematic stability
|
||||||
|
|
||||||
|
Most importantly:
|
||||||
|
|
||||||
|
> The model began generating stories that feel intentionally written rather than statistically assembled.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#Final Training Statistics
|
||||||
|
|
||||||
|
| Metric | Value |
|
||||||
|
|---|---|
|
||||||
|
| Final Step | 3000 |
|
||||||
|
| Final Loss | **0.8516** |
|
||||||
|
| Training Stability | Excellent |
|
||||||
|
| Gradient Behavior | Stable |
|
||||||
|
| Divergence Events | None Observed |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Training Configuration
|
||||||
|
|
||||||
|
## Hyperparameters
|
||||||
|
|
||||||
|
| Parameter | Value |
|
||||||
|
|---|---|
|
||||||
|
| Optimizer | AdamW |
|
||||||
|
| Betas | β₁=0.9, β₂=0.95 |
|
||||||
|
| Learning Rate | 5e-4 |
|
||||||
|
| Scheduler | OneCycleLR |
|
||||||
|
| Weight Decay | 0.01 |
|
||||||
|
| Precision | bfloat16 |
|
||||||
|
| Compilation | torch.compile |
|
||||||
|
| Attention Optimization | Flash Attention 2 |
|
||||||
|
| Effective Batch Size | ~262,144 Tokens / Step |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Dataset
|
||||||
|
|
||||||
|
## TinyStories (2M)
|
||||||
|
|
||||||
|
Test-1-3000 was trained on the **TinyStories** dataset.
|
||||||
|
|
||||||
|
TinyStories is uniquely valuable because it isolates:
|
||||||
|
|
||||||
|
- narrative structure,
|
||||||
|
- reasoning,
|
||||||
|
- consistency,
|
||||||
|
- and causality
|
||||||
|
|
||||||
|
without the overwhelming informational noise of the open web.
|
||||||
|
|
||||||
|
The stories use:
|
||||||
|
|
||||||
|
- child-level vocabulary,
|
||||||
|
- but professionally structured narrative composition.
|
||||||
|
|
||||||
|
This creates an ideal environment for studying emergent reasoning inside small language models.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Training Philosophy
|
||||||
|
|
||||||
|
The project intentionally prioritizes:
|
||||||
|
|
||||||
|
- coherence over memorization,
|
||||||
|
- reasoning over factual retrieval,
|
||||||
|
- and narrative intelligence over benchmark chasing.
|
||||||
|
|
||||||
|
The goal is not merely to create a chatbot.
|
||||||
|
|
||||||
|
The goal is to study:
|
||||||
|
|
||||||
|
> how structured cognition emerges inside compact neural systems.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
#Usage — Quick Start
|
||||||
|
|
||||||
|
Install dependencies:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install transformers torch accelerate
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Inference Example
|
||||||
|
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||||
|
|
||||||
|
model_path = "GODELEV/Test-1-3000"
|
||||||
|
|
||||||
|
# Load Tokenizer and Model
|
||||||
|
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
||||||
|
|
||||||
|
model = AutoModelForCausalLM.from_pretrained(
|
||||||
|
model_path,
|
||||||
|
torch_dtype=torch.bfloat16,
|
||||||
|
device_map="auto"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Prompt
|
||||||
|
prompt = "Once upon a time, Tom found a blue car."
|
||||||
|
|
||||||
|
inputs = tokenizer(
|
||||||
|
prompt,
|
||||||
|
return_tensors="pt"
|
||||||
|
).to(model.device)
|
||||||
|
|
||||||
|
# Generate
|
||||||
|
output = model.generate(
|
||||||
|
**inputs,
|
||||||
|
max_new_tokens=200,
|
||||||
|
temperature=0.7,
|
||||||
|
top_p=0.9,
|
||||||
|
repetition_penalty=1.1,
|
||||||
|
do_sample=True,
|
||||||
|
eos_token_id=tokenizer.eos_token_id,
|
||||||
|
pad_token_id=tokenizer.pad_token_id
|
||||||
|
)
|
||||||
|
|
||||||
|
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Recommended Generation Settings
|
||||||
|
|
||||||
|
| Parameter | Recommended |
|
||||||
|
|---|---|
|
||||||
|
| Temperature | 0.7 |
|
||||||
|
| Top-p | 0.9 |
|
||||||
|
| Repetition Penalty | 1.1 |
|
||||||
|
| Max Tokens | 128–512 |
|
||||||
|
| Sampling | Enabled |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Observed Emergent Behaviors
|
||||||
|
|
||||||
|
During evaluation, the model demonstrated:
|
||||||
|
|
||||||
|
- Character persistence
|
||||||
|
- Goal-oriented progression
|
||||||
|
- Emotional continuity
|
||||||
|
- Environmental consistency
|
||||||
|
- Contextual callbacks
|
||||||
|
- Story resolution awareness
|
||||||
|
|
||||||
|
These behaviors are especially notable given the model's relatively small parameter count.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Limitations
|
||||||
|
|
||||||
|
Although highly capable for its size, Test-1-3000 still has limitations:
|
||||||
|
|
||||||
|
- Limited factual world knowledge
|
||||||
|
- Occasional repetition in very long generations
|
||||||
|
- Reduced reasoning performance outside storytelling domains
|
||||||
|
- Less stable beyond trained narrative styles
|
||||||
|
|
||||||
|
The model is optimized specifically for:
|
||||||
|
|
||||||
|
> coherent short-form storytelling.
|
||||||
|
|
||||||
|
---
|
||||||
|
``
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# 📜 Citation
|
||||||
|
|
||||||
|
```bibtex
|
||||||
|
@misc{test13000,
|
||||||
|
title={Test-1-3000: A 190M Parameter Narrative Intelligence Engine},
|
||||||
|
author={GODELEV},
|
||||||
|
year={2026},
|
||||||
|
note={Compact narrative-focused language model trained on TinyStories}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# License
|
||||||
|
|
||||||
|
This project is intended for:
|
||||||
|
|
||||||
|
- research,
|
||||||
|
- experimentation,
|
||||||
|
- educational use,
|
||||||
|
- and open exploration of compact language models.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Final Thoughts
|
||||||
|
|
||||||
|
Test-1-3000 demonstrates that meaningful narrative intelligence can emerge inside surprisingly small neural systems when training is focused, clean, and structurally optimized.
|
||||||
|
|
||||||
|
At only **190M parameters**, the model exhibits behaviors often associated with significantly larger systems:
|
||||||
|
|
||||||
|
- narrative planning,
|
||||||
|
- emotional continuity,
|
||||||
|
- causal consistency,
|
||||||
|
- and coherent resolution generation.
|
||||||
|
|
||||||
|
The project serves as both:
|
||||||
|
|
||||||
|
- a practical storytelling model,
|
||||||
|
- and an experiment in emergent cognition within compact architectures.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
<p align="center">
|
||||||
|
|
||||||
|
### “Small models are not weak models.
|
||||||
|
### They are compressed intelligence waiting to emerge.”
|
||||||
|
|
||||||
|
</p>
|
||||||
|
````
|
||||||
32
config.json
Normal file
32
config.json
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
{
|
||||||
|
"architectures": [
|
||||||
|
"LlamaForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_bias": false,
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"bos_token_id": 1,
|
||||||
|
"dtype": "float32",
|
||||||
|
"eos_token_id": 2,
|
||||||
|
"head_dim": 64,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 896,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 2432,
|
||||||
|
"max_position_embeddings": 2048,
|
||||||
|
"mlp_bias": false,
|
||||||
|
"model_type": "llama",
|
||||||
|
"num_attention_heads": 14,
|
||||||
|
"num_hidden_layers": 12,
|
||||||
|
"num_key_value_heads": 2,
|
||||||
|
"pad_token_id": null,
|
||||||
|
"pretraining_tp": 1,
|
||||||
|
"rms_norm_eps": 1e-05,
|
||||||
|
"rope_parameters": {
|
||||||
|
"rope_theta": 10000.0,
|
||||||
|
"rope_type": "default"
|
||||||
|
},
|
||||||
|
"tie_word_embeddings": false,
|
||||||
|
"transformers_version": "5.6.2",
|
||||||
|
"use_cache": false,
|
||||||
|
"vocab_size": 50257
|
||||||
|
}
|
||||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
{
|
||||||
|
"_from_model_config": true,
|
||||||
|
"bos_token_id": 1,
|
||||||
|
"eos_token_id": 2,
|
||||||
|
"output_attentions": false,
|
||||||
|
"output_hidden_states": false,
|
||||||
|
"transformers_version": "5.6.2",
|
||||||
|
"use_cache": false
|
||||||
|
}
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:e1dcdfb41c634038eea67bfbd7c01de8fea575aee2432c89b8eb23cd6ea3d817
|
||||||
|
size 762210848
|
||||||
250306
tokenizer.json
Normal file
250306
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
13
tokenizer_config.json
Normal file
13
tokenizer_config.json
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"add_prefix_space": false,
|
||||||
|
"backend": "tokenizers",
|
||||||
|
"bos_token": "<|endoftext|>",
|
||||||
|
"eos_token": "<|endoftext|>",
|
||||||
|
"errors": "replace",
|
||||||
|
"is_local": false,
|
||||||
|
"local_files_only": false,
|
||||||
|
"model_max_length": 2048,
|
||||||
|
"pad_token": "<|endoftext|>",
|
||||||
|
"tokenizer_class": "GPT2Tokenizer",
|
||||||
|
"unk_token": "<|endoftext|>"
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user