初始化项目，由ModelHub XC社区提供模型

Model: OPENGCM/Hydrion-v1-Base
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,88 @@
+---
+license: apache-2.0
+language:
+- en
+tags:
+- text-generation
+- causal-lm
+- chatml
+- from-scratch
+- hydrion
+- opengcm
+pipeline_tag: text-generation
+---
+
+
+
+![image](https://cdn-uploads.huggingface.co/production/uploads/69c842686cf758859915159c/KRCENFQ0W7H9B2GruxyA9.png)
+
+Hydrion is a 114M-parameter causal language model, pretrained from scratch and fine-tuned for chat, built on a single RTX 3060 plus a handful of rented A100 hours.
+
+This repo (`OpenGCM/Hydrion-Base`) is the base model, non-chat ready version. The instruction model (chat formatting) is available at [`OpenGCM/Hydrion-SFT`](https://huggingface.co/OPENGCM/Hydrion-SFT).
+## Model Details
+
+- **Architecture:** Llama-style decoder-only transformer (RMSNorm, rotary position embeddings, SwiGLU MLP, grouped-query attention)
+- **Parameters:** 114.1M
+- **Layers:** 12
+- **Hidden size:** 768
+- **Attention heads:** 12 (4 KV heads, GQA)
+- **Context length:** 1024 tokens
+- **Tokenizer:** [`EleutherAI/gpt-neox-20b`](https://huggingface.co/EleutherAI/gpt-neox-20b)
+- **License:** Apache 2.0
+
+## Training
+
+Hydrion was trained in two pretraining stages.
+
+1. **Initial pretraining** — ~2B tokens on a FineWeb-Edu / Wikipedia mix, trained on a single RTX 3060 (12GB).
+2. **Continued pretraining** — an additional ~0.5B tokens on a more diverse mix (FineWeb-Edu, Wikipedia, TinyStories, a code subset, and Dolly), run on a rented A100 to broaden register and topic coverage beyond pure web/encyclopedic text.
+
+Total pretraining exposure: roughly **2.5 billion tokens**.
+
+## Benchmarks
+
+Evaluated with [`lm-evaluation-harness`](https://github.com/EleutherAI/lm-evaluation-harness) on the base (pre-SFT) checkpoint:
+
+| Benchmark | Metric | Score |
+|---|---|---|
+| BLiMP | acc | 80.08% |
+| ARC-Easy | acc | 47.26% |
+| ARC-Easy | acc_norm | 43.39% |
+| WikiText-2 | byte_perplexity | 2.04 |
+| WikiText-2 | bits_per_byte | 1.03 |
+| WikiText-2 | word_perplexity | 45.02 |
+
+Grammatical judgment (BLiMP) is comparable to models trained on far larger token budgets; factual/reasoning performance (ARC-Easy) is meaningfully weaker, consistent with the relatively small pretraining corpus.
+
+## Usage
+
+```python
+import torch
+from transformers import AutoTokenizer, LlamaForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("OPENGCM/Hydrion-Base")
+model = LlamaForCausalLM.from_pretrained("OPENGCM/Hydrion-Base", torch_dtype=torch.bfloat16).cuda()
+model.eval()
+
+prompt = "What is the capital of"
+inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
+
+with torch.no_grad():
+    output = model.generate(
+        **inputs,
+        max_new_tokens=150,
+        do_sample=True,
+        temperature=0.7,
+        top_p=0.9,
+        repetition_penalty=1.3,
+        no_repeat_ngram_size=3,
+        eos_token_id=tokenizer.convert_tokens_to_ids("<|im_end|>"),
+    )
+
+response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print(response)
+```
+
+## Limitations
+
+Hydrion is a small model trained on a modest token budget (~2.5B tokens, versus the trillions used by comparable production small models). It should **not** be relied on for factual accuracy. It reliably produces fluent, grammatically well-formed English and responds in a conversational chat format, but frequently states incorrect facts, fabricates names/dates/attributions, and performs poorly at arithmetic and multi-step reasoning. Treat outputs as unreliable by default — this model is best understood as a demonstration of a working from-scratch training pipeline rather than a usable knowledge source or assistant.

config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 2048,
+  "max_position_embeddings": 1024,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "num_key_value_heads": 4,
+  "pad_token_id": null,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.14.1",
+  "use_cache": false,
+  "vocab_size": 50280
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "transformers_version": "5.14.1",
+  "use_cache": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:687bb38cefff29c8afea2eca7d10c4c3ab837df5adf1f693523156b7e8a6f4c2
+size 456539080

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:203ba0d49e64969894b8d194a9cbdac1a993e216a5a5b7e2087ea40299afc0f4
+size 913148619

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:250560ab3d528161ab3659b120def6e4a9ab4b457e3399603bbcfa40db3efc90
+size 14645

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e452e68919a7d4317856b783f9067e86378e9d516b2bad5f08c7cd7e3d232da
+size 1465

tokenizer_config.json

@@ -0,0 +1,14 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|padding|>",
+  "tokenizer_class": "GPTNeoXTokenizer",
+  "trim_offsets": true,
+  "unk_token": "<|endoftext|>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4eb93e1e55e9d6fca1896e9767712278eb9d586eba1b578fdf8402cc93bc1d48
+size 5201