初始化项目，由ModelHub XC社区提供模型

Model: BananaMind/BananaMind-1.5-Base
Source: Original Platform

README.md

@@ -0,0 +1,156 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+datasets:
+- HuggingFaceFW/fineweb-edu
+tags:
+- causal-lm
+- language-model
+- base-model
+- small-language-model
+- bananamind
+- from-scratch
+- pytorch
+- safetensors
+- llama
+---
+
+![banner](banner.png)
+
+# BananaMind-1.5-Base
+
+BananaMind-1.5-Base is a small English causal language model trained from scratch by BananaMind.
+
+It is our first fully pretrained medium model
+
+
+## Model Details
+
+| Field | Value |
+|---|---:|
+| Parameters | 75,054,720 |
+| Architecture | Llama-style decoder-only Transformer |
+| Layers | 12 |
+| Hidden size | 640 |
+| Intermediate size | 1728 |
+| Attention heads | 10 |
+| KV heads | 5 |
+| Context length | 4096 tokens |
+| Vocabulary size | 32,000 |
+| Tokenizer | Custom byte-level BPE |
+| Training tokens | ~27B tokens |
+| Precision | BF16 training, safetensors release |
+| Model type | Base causal LM |
+| Training Cost | 103.31$(PLEASE LIKE THIS IS SO EXPENSIVE) |
+| Training GPU | RTX Pro 6000 |
+
+![Benchmarks](benchmarks.png)
+A instruction tuned version is coming very soon.
+
+## Usage
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+repo = "BananaMind/BananaMind-1.5-Base"
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+dtype = torch.float16 if torch.cuda.is_available() else torch.float32
+
+tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=False)
+
+model = AutoModelForCausalLM.from_pretrained(
+    repo,
+    trust_remote_code=False,
+    dtype=dtype,
+).to(device)
+
+model.eval()
+
+prompt = "The color of the sky is blue. The color of a banana is"
+inputs = tok(prompt, return_tensors="pt").to(device)
+
+with torch.no_grad():
+    out = model.generate(
+        **inputs,
+        max_new_tokens=16,
+        do_sample=True,
+        temperature=0.7,
+        top_p=0.9,
+        pad_token_id=tok.eos_token_id,
+        eos_token_id=tok.eos_token_id,
+    )
+
+print(tok.decode(out[0], skip_special_tokens=True))
+```
+
+## Generation Settings
+
+Recommended starting settings:
+
+```python
+temperature = 0.7
+top_p = 0.9
+max_new_tokens = 64
+```
+
+For deterministic sanity tests:
+
+```python
+do_sample = False
+max_new_tokens = 8
+```
+
+## Training
+
+BananaMind-1.5-Base was trained from scratch on approximately 27B tokens of FineWeb-Edu-style English web text.
+
+The model uses a custom 32k byte-level BPE tokenizer and a compact Llama-style architecture with grouped-query attention.
+
+## Architecture
+
+BananaMind-1.5-Base uses a compact Llama-style decoder architecture:
+
+- 12 Transformer layers
+- 640 hidden size
+- 1728 intermediate size
+- 10 attention heads
+- 5 key-value heads
+- grouped-query attention
+- SiLU activation
+- RMSNorm
+- tied input/output embeddings
+- 4096 token context length
+
+## Evaluation
+
+Our model performs very good in comparison to other models:
+
+| Model | HellaSwag | ARC-Easy | ARC-Challenge | PIQA | ArithMark-2.0 | Average |
+|---|---:|---:|---:|---:|---:|---:|
+| BananaMind-1.5-Base | 30.91% | 42.38% | 23.98% | 60.55% | 26.68% | 36.90% |
+| Gemma 3 IT 270M | 37.70% | - | - | 66.20% | - | - |
+| Zupra-1.6-Instruct-Ultra-Exp | 29.66% | 34.41% | 25.51% | 59.74% | 30.44% | 35.95% |
+| KeyLM 75M | 29.66% | 35.73% | 23.98% | 60.50% | 25.80% | 35.13% |
+| GPT-2 124M | 31.26% | 39.35% | 22.35% | 62.08% | 26.48% | 36.30% |
+
+![Benchmarks](benchmarks.png)
+
+
+## Parameter vs Size
+![Parameter vs size Chart](chart_parameter_size.png)
+
+## Citation
+
+```bibtex
+@misc{bananamind15base,
+  title = {BananaMind-1.5-Base},
+  author = {BananaMind},
+  year = {2026},
+  publisher = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/BananaMind/BananaMind-1.5-Base}}
+}
+```