初始化项目,由ModelHub XC社区提供模型

Model: ripbaggie/babygrok
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-25 12:15:18 +08:00
commit 586425faed
17 changed files with 92337 additions and 0 deletions

61
README.md Normal file
View File

@@ -0,0 +1,61 @@
---
license: apache-2.0
base_model: mistralai/Mistral-7B-v0.1
language:
- en
pipeline_tag: text-generation
tags:
- alignment-handbook
- generated_from_trainer
model-index:
- name: babygrok
results: []
library_name: transformers
---
# babygrok
A mini version of Grok, by [ripbaggie](https://huggingface.co/ripbaggie).
## About
babygrok is Grok 4.3 condensed into a 7B Mistral model, trained to mirror the behavior of Grok — essentially Grok in a tiny package, allowing you to host it locally.
## Usage
A 7B-parameter model designed to run locally.
## License
Apache 2.0.
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 4
- total_train_batch_size: 256
- total_eval_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1
### Training results
| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 0.9326 | 1.0 | 545 | 0.9348 |
### Framework versions
- Transformers 4.36.2
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0