language, license, pipeline_tag, tags, library_name, datasets

language	license	pipeline_tag	tags	library_name	datasets
en	apache-2.0	text-generation	llama causal-lm pretrained chytrej base tiny	transformers	HuggingFaceFW/fineweb-edu

Chytrej2-Mini

A fully custom pretrained language model built from scratch on the LLaMA architecture trained on 2B tokens of the FineWeb Edu dataset.

Built by PingVortex Labs.

---

Model Details

• Parameters: 20M
• Context length: 1024 tokens
• Language: English only
• Format: Base model
• Architecture: LLaMA
• License: Apache 2.0

---

Benchmark

• The model achieves score of 35.77% on ARC-Easy benchmark.

---

Usage

from transformers import LlamaForCausalLM, PreTrainedTokenizerFast

model = LlamaForCausalLM.from_pretrained("pvlabs/Chytrej2-Mini")
tokenizer = PreTrainedTokenizerFast.from_pretrained("pvlabs/Chytrej2-Mini")

prompt = "Neural Networks are"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, repetition_penalty=1.3)
print(tokenizer.decode(outputs[0]))

---

Made by PingVortex.