初始化项目，由ModelHub XC社区提供模型
Model: elinas/Chronos-Gold-12B-1.0 Source: Original Platform
This commit is contained in:
ModelHub XC
192
README.md
Normal file
192
README.md
Normal file
@@ -0,0 +1,192 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
library_name: transformers
|
||||
tags:
|
||||
- general-purpose
|
||||
- roleplay
|
||||
- storywriting
|
||||
- merge
|
||||
- finetune
|
||||
base_model: mistralai/Mistral-Nemo-Base-2407
|
||||
model-index:
|
||||
- name: Chronos-Gold-12B-1.0
|
||||
results:
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: IFEval (0-Shot)
|
||||
type: HuggingFaceH4/ifeval
|
||||
args:
|
||||
num_few_shot: 0
|
||||
metrics:
|
||||
- type: inst_level_strict_acc and prompt_level_strict_acc
|
||||
value: 31.66
|
||||
name: strict accuracy
|
||||
source:
|
||||
url: >-
|
||||
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: BBH (3-Shot)
|
||||
type: BBH
|
||||
args:
|
||||
num_few_shot: 3
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: 35.91
|
||||
name: normalized accuracy
|
||||
source:
|
||||
url: >-
|
||||
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: MATH Lvl 5 (4-Shot)
|
||||
type: hendrycks/competition_math
|
||||
args:
|
||||
num_few_shot: 4
|
||||
metrics:
|
||||
- type: exact_match
|
||||
value: 4.38
|
||||
name: exact match
|
||||
source:
|
||||
url: >-
|
||||
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: GPQA (0-shot)
|
||||
type: Idavidrein/gpqa
|
||||
args:
|
||||
num_few_shot: 0
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: 9.06
|
||||
name: acc_norm
|
||||
source:
|
||||
url: >-
|
||||
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: MuSR (0-shot)
|
||||
type: TAUR-Lab/MuSR
|
||||
args:
|
||||
num_few_shot: 0
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: 19.42
|
||||
name: acc_norm
|
||||
source:
|
||||
url: >-
|
||||
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: MMLU-PRO (5-shot)
|
||||
type: TIGER-Lab/MMLU-Pro
|
||||
config: main
|
||||
split: test
|
||||
args:
|
||||
num_few_shot: 5
|
||||
metrics:
|
||||
- type: acc
|
||||
value: 27.98
|
||||
name: accuracy
|
||||
source:
|
||||
url: >-
|
||||
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=elinas/Chronos-Gold-12B-1.0
|
||||
name: Open LLM Leaderboard
|
||||
---
|
||||
|
||||
# Chronos Gold 12B-1.0
|
||||
|
||||
|
||||
|
||||
Chronos Gold 12B 1.0 is a very unique model that applies to domain areas such as
|
||||
general chatbot functionatliy, *roleplay*, and storywriting. The model has been observed to write up to 2250 tokens in a single sequence. The model was trained at a
|
||||
sequence length of 16384 (16k) and will still retain the *apparent* 128k context length from Mistral-Nemo, though it deteriorates over time like regular Nemo does based on
|
||||
the [RULER Test](https://github.com/hsiehjackson/RULER?tab=readme-ov-file#-ruler-whats-the-real-context-size-of-your-long-context-language-models)
|
||||
|
||||
As a result, is recommended to keep your sequence length max at 16384, or you will experience performance degredation.
|
||||
|
||||
The base model is `mistralai/Mistral-Nemo-Base-2407` which was heavily modified to produce a more coherent model, comparable to much larger models.
|
||||
|
||||
**Chronos Gold 12B-1.0** re-creates the uniqueness of the original Chronos with significiantly enhanced prompt adherence (following), coherence, a modern dataset, as well as supporting a majority of "character card" formats in applications like SillyTavern.
|
||||
|
||||
It went through an iterative and objective merge process as my previous models and was further finetuned on a dataset curated for it.
|
||||
|
||||
The specifics of the model will not be disclosed at the time due to dataset ownership.
|
||||
|
||||
## Instruct Template
|
||||
|
||||
This model uses `ChatML` - below is an example. It is a preset in many frontends.
|
||||
|
||||
```
|
||||
<|im_start|>system
|
||||
A system prompt describing how you'd like your bot to act.<|im_end|>
|
||||
<|im_start|>user
|
||||
Hello there!<|im_end|>
|
||||
<|im_start|>assistant
|
||||
I can assist you or we can discuss other things?<|im_end|>
|
||||
<|im_start|>user
|
||||
I was wondering how transformers work?<|im_end|>
|
||||
<|im_start|>assistant
|
||||
```
|
||||
|
||||
## Quantization
|
||||
|
||||
#### LlamaCPP
|
||||
[GGUFs from @bartowski](https://huggingface.co/bartowski/Chronos-Gold-12B-1.0-GGUF)
|
||||
|
||||
#### Exllama2
|
||||
[4.5bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-4.5bpw-h6-exl2)
|
||||
|
||||
[5.5bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-5.5bpw-h6-exl2)
|
||||
|
||||
[6.5bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-6.5bpw-h6-exl2)
|
||||
|
||||
[8.0bpw by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-8.0bpw-h8-exl2)
|
||||
|
||||
#### FP8
|
||||
[FP8 Quant by @Pyroserenus](https://huggingface.co/Pyroserenus/Chronos-Gold-12B-1.0-FP8)
|
||||
|
||||
## Sampling Settings
|
||||
Nemo is a bit sensitive to high temperatures, so I use lower. Here are my settings:
|
||||
```
|
||||
Temp - 0.7 (0.9 max)
|
||||
Presence Penalty - 1.0
|
||||
Repetition Penalty range - 2800
|
||||
Min P - 0.10
|
||||
```
|
||||
|
||||
## Additional Details
|
||||
This model was created by **elinas** on discord. Thank you to [@kalomaze](https://huggingface.co/kalomaze) for providing a model that made this merge possible.
|
||||
|
||||
This is one of multiple models to come out in the series by size and model architecture, so look forward to it!
|
||||
|
||||
Contact me on Discord for inquiries.
|
||||
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
|
||||
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_elinas__Chronos-Gold-12B-1.0)
|
||||
|
||||
| Metric |Value|
|
||||
|-------------------|----:|
|
||||
|Avg. |21.40|
|
||||
|IFEval (0-Shot) |31.66|
|
||||
|BBH (3-Shot) |35.91|
|
||||
|MATH Lvl 5 (4-Shot)| 4.38|
|
||||
|GPQA (0-shot) | 9.06|
|
||||
|MuSR (0-shot) |19.42|
|
||||
|MMLU-PRO (5-shot) |27.98|
|
||||
Reference in New Issue
Block a user