Josephgflowers/TinyLlama-748M-Reason-With-Cinder-Test-2
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: Josephgflowers/TinyLlama-748M-Reason-With-Cinder-Test-2
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-11 17:05:07 +08:00
commit 976a3c6c94
15 changed files with 281 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
README.md Normal file
View File

@@ -0,0 +1,17 @@
---
license: mit
widget:
- text: >
<|system|>
You are a helpful assistant</s>
<|user|>
What is 32+64?</s>
<|assistant|>
---
I cut my TinyLlama 1.1B cinder v 2 down from 22 layers to 14. At 14 there was no coherent text but there were emerging ideas of a response. 1000 steps on step-by-step dataset.
10000 on Reason-with-cinder. The loss was around .6 and the learning rate was still over 4. Starting to get better performance now. This model still needs significat training. I am putting it up as a base model that
needs work. If you continue training please let me know on the tinyllama discord https://discord.com/channels/1156883027805356072/1156883029671813122 or email Cinder.stem@gmail.com , I have some interesting plans for this model.