初始化项目，由ModelHub XC社区提供模型

Model: BananaMind/BananaMind-1.5-Base
Source: Original Platform

.gitattributes

@@ -0,0 +1,38 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+banner.png filter=lfs diff=lfs merge=lfs -text
+chart_parameter_size.png filter=lfs diff=lfs merge=lfs -text
+benchmarks.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,156 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+datasets:
+- HuggingFaceFW/fineweb-edu
+tags:
+- causal-lm
+- language-model
+- base-model
+- small-language-model
+- bananamind
+- from-scratch
+- pytorch
+- safetensors
+- llama
+---
+
+![banner](banner.png)
+
+# BananaMind-1.5-Base
+
+BananaMind-1.5-Base is a small English causal language model trained from scratch by BananaMind.
+
+It is our first fully pretrained medium model
+
+
+## Model Details
+
+| Field | Value |
+|---|---:|
+| Parameters | 75,054,720 |
+| Architecture | Llama-style decoder-only Transformer |
+| Layers | 12 |
+| Hidden size | 640 |
+| Intermediate size | 1728 |
+| Attention heads | 10 |
+| KV heads | 5 |
+| Context length | 4096 tokens |
+| Vocabulary size | 32,000 |
+| Tokenizer | Custom byte-level BPE |
+| Training tokens | ~27B tokens |
+| Precision | BF16 training, safetensors release |
+| Model type | Base causal LM |
+| Training Cost | 103.31$(PLEASE LIKE THIS IS SO EXPENSIVE) |
+| Training GPU | RTX Pro 6000 |
+
+![Benchmarks](benchmarks.png)
+A instruction tuned version is coming very soon.
+
+## Usage
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+repo = "BananaMind/BananaMind-1.5-Base"
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+dtype = torch.float16 if torch.cuda.is_available() else torch.float32
+
+tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=False)
+
+model = AutoModelForCausalLM.from_pretrained(
+    repo,
+    trust_remote_code=False,
+    dtype=dtype,
+).to(device)
+
+model.eval()
+
+prompt = "The color of the sky is blue. The color of a banana is"
+inputs = tok(prompt, return_tensors="pt").to(device)
+
+with torch.no_grad():
+    out = model.generate(
+        **inputs,
+        max_new_tokens=16,
+        do_sample=True,
+        temperature=0.7,
+        top_p=0.9,
+        pad_token_id=tok.eos_token_id,
+        eos_token_id=tok.eos_token_id,
+    )
+
+print(tok.decode(out[0], skip_special_tokens=True))
+```
+
+## Generation Settings
+
+Recommended starting settings:
+
+```python
+temperature = 0.7
+top_p = 0.9
+max_new_tokens = 64
+```
+
+For deterministic sanity tests:
+
+```python
+do_sample = False
+max_new_tokens = 8
+```
+
+## Training
+
+BananaMind-1.5-Base was trained from scratch on approximately 27B tokens of FineWeb-Edu-style English web text.
+
+The model uses a custom 32k byte-level BPE tokenizer and a compact Llama-style architecture with grouped-query attention.
+
+## Architecture
+
+BananaMind-1.5-Base uses a compact Llama-style decoder architecture:
+
+- 12 Transformer layers
+- 640 hidden size
+- 1728 intermediate size
+- 10 attention heads
+- 5 key-value heads
+- grouped-query attention
+- SiLU activation
+- RMSNorm
+- tied input/output embeddings
+- 4096 token context length
+
+## Evaluation
+
+Our model performs very good in comparison to other models:
+
+| Model | HellaSwag | ARC-Easy | ARC-Challenge | PIQA | ArithMark-2.0 | Average |
+|---|---:|---:|---:|---:|---:|---:|
+| BananaMind-1.5-Base | 30.91% | 42.38% | 23.98% | 60.55% | 26.68% | 36.90% |
+| Gemma 3 IT 270M | 37.70% | - | - | 66.20% | - | - |
+| Zupra-1.6-Instruct-Ultra-Exp | 29.66% | 34.41% | 25.51% | 59.74% | 30.44% | 35.95% |
+| KeyLM 75M | 29.66% | 35.73% | 23.98% | 60.50% | 25.80% | 35.13% |
+| GPT-2 124M | 31.26% | 39.35% | 22.35% | 62.08% | 26.48% | 36.30% |
+
+![Benchmarks](benchmarks.png)
+
+
+## Parameter vs Size
+![Parameter vs size Chart](chart_parameter_size.png)
+
+## Citation
+
+```bibtex
+@misc{bananamind15base,
+  title = {BananaMind-1.5-Base},
+  author = {BananaMind},
+  year = {2026},
+  publisher = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/BananaMind/BananaMind-1.5-Base}}
+}
+```

banner.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7eb8ae768a03c39bcc20ce6797940b46622da985d3a633ed895e86ed3bc46933
+size 1430992

benchmarks.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65f9c43c45c6ec8de70802da3035b4c6459fa5b293aa759af50a75f194404dde
+size 1067819

chart_parameter_size.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:736c51ae1b1836d1726d393aba5e538ab65f440d8857afe6e06bc3c2cde84f96
+size 1106607

config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 640,
+  "initializer_range": 0.02,
+  "intermediate_size": 1728,
+  "max_position_embeddings": 4096,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 10,
+  "num_hidden_layers": 12,
+  "num_key_value_heads": 5,
+  "pad_token_id": 1,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.12.1",
+  "use_cache": false,
+  "vocab_size": 32000
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 1,
+  "transformers_version": "5.12.1",
+  "use_cache": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3357333a0011e8b9bf9c7b9c7a5515be66c591c915d28da446cedc9f6b11c06f
+size 150121664

tokenizer_config.json

@@ -0,0 +1,12 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "is_local": true,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>"
+}

training_summary.json

@@ -0,0 +1,15 @@
+{
+  "final_step": 274555,
+  "supervised_tokens_seen": 26983265400,
+  "actual_target_supervised_tokens": 26983343205,
+  "target_supervised_tokens_requested": 27000000000,
+  "params": 75054720,
+  "label_policy": "labels=input_ids; LlamaForCausalLM shifts internally",
+  "data_order": "sequential contiguous token blocks",
+  "block_size": 4096,
+  "supervised_targets_per_sequence": 4095,
+  "batch_size": 24,
+  "grad_accum": 1,
+  "param_dtype": "bf16",
+  "compute_dtype": "bf16 autocast"
+}