137 lines
4.5 KiB
Markdown
137 lines
4.5 KiB
Markdown
---
|
|
license: apache-2.0
|
|
datasets:
|
|
- PocketDoc/Dans-MemoryCore-CoreCurriculum-Small
|
|
- PocketDoc/Dans-Prosemaxx-Gutenberg
|
|
- PocketDoc/Dans-Prosemaxx-Cowriter-S
|
|
- PocketDoc/Dans-Prosemaxx-Adventure
|
|
- PocketDoc/Dans-Prosemaxx-Opus-Writing
|
|
- PocketDoc/Dans-Assistantmaxx-Sharegpt
|
|
- PocketDoc/Dans-Assistantmaxx-OpenAssistant2
|
|
- PocketDoc/Dans-Assistantmaxx-Opus-instruct-1
|
|
- PocketDoc/Dans-Assistantmaxx-Opus-instruct-2
|
|
- PocketDoc/Dans-Assistantmaxx-Opus-instruct-3
|
|
- PocketDoc/Dans-Assistantmaxx-Opus-Multi-Instruct
|
|
- PocketDoc/Dans-Assistantmaxx-sonnetorca-subset
|
|
- PocketDoc/Dans-Assistantmaxx-NoRobots
|
|
- AquaV/Energetic-Materials-Sharegpt
|
|
- AquaV/Chemical-Biological-Safety-Applications-Sharegpt
|
|
- AquaV/US-Army-Survival-Sharegpt
|
|
- AquaV/Resistance-Sharegpt
|
|
- AquaV/Interrogation-Sharegpt
|
|
- AquaV/Multi-Environment-Operations-Sharegpt
|
|
- PocketDoc/Dans-Mathmaxx
|
|
- PJMixers/Math-Multiturn-1K-ShareGPT
|
|
- PocketDoc/Dans-Benchmaxx
|
|
- PocketDoc/Dans-Codemaxx-LeetCode
|
|
- PocketDoc/Dans-Codemaxx-CodeFeedback-Conversations
|
|
- PocketDoc/Dans-Codemaxx-CodeFeedback-SingleTurn
|
|
- PocketDoc/Dans-Taskmaxx
|
|
- PocketDoc/Dans-Taskmaxx-DataPrepper
|
|
- PocketDoc/Dans-Taskmaxx-ConcurrentQA-Reworked
|
|
- PocketDoc/Dans-Systemmaxx
|
|
- PocketDoc/Dans-Toolmaxx-Agent
|
|
- PocketDoc/Dans-Toolmaxx-ShellCommands
|
|
- PocketDoc/Dans-ASCIIMaxx-Wordart
|
|
- PocketDoc/Dans-Personamaxx
|
|
- PocketDoc/DansTestYard
|
|
- PocketDoc/Dans-Logicmaxx-Skunkworks
|
|
language:
|
|
- en
|
|
base_model:
|
|
- meta-llama/Llama-3.1-8B
|
|
- Dans-DiscountModels/Meta-Llama-3.1-8B-ChatML
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- chemistry
|
|
- biology
|
|
- code
|
|
- climate
|
|
- text-generation-inference
|
|
---
|
|
|
|
## What is it?
|
|
|
|
This model is intended to be multifarious in its capabilities and should be quite capable at both co-writing and roleplay as well as find itself quite at home performing sentiment analysis or summarization as part of a pipeline. It has been trained on a wide array of one shot instructions, multi turn instructions, role playing scenarios, text adventure games, co-writing, and much more. The full dataset is publicly available and can be found in the datasets section of the model page.
|
|
|
|
There has not been any form of harmfulness alignment done on this model, please take the appropriate precautions when using it in a production environment.
|
|
|
|
|
|
## Prompting
|
|
|
|
The model has been trained on standard "ChatML" format prompting, an example of which is shown below:
|
|
|
|
```
|
|
<|im_start|>system
|
|
system prompt<|im_end|>
|
|
<|im_start|>user
|
|
Hi there!<|im_end|>
|
|
<|im_start|>assistant
|
|
Nice to meet you!<|im_end|>
|
|
<|im_start|>user
|
|
Can I ask a question?<|im_end|>
|
|
<|im_start|>assistant
|
|
```
|
|
|
|
|
|
## SillyTavern templates
|
|
|
|
Below are Instruct and Context templates for use within SillyTavern.
|
|
|
|
<details><summary>context template</summary>
|
|
|
|
```yaml
|
|
{
|
|
"story_string": "<|im_start|>system\n{{#if system}}{{system}}\n{{/if}}{{#if wiBefore}}{{wiBefore}}\n{{/if}}{{#if description}}{{description}}\n{{/if}}{{#if personality}}{{char}}'s personality: {{personality}}\n{{/if}}{{#if scenario}}Scenario: {{scenario}}\n{{/if}}{{#if wiAfter}}{{wiAfter}}\n{{/if}}{{#if persona}}{{persona}}\n{{/if}}{{trim}}<|im_end|>\n",
|
|
"example_separator": "",
|
|
"chat_start": "",
|
|
"use_stop_strings": false,
|
|
"allow_jailbreak": false,
|
|
"always_force_name2": false,
|
|
"trim_sentences": false,
|
|
"include_newline": false,
|
|
"single_line": false,
|
|
"name": "Dan-ChatML"
|
|
}
|
|
```
|
|
|
|
</details><br>
|
|
<details><summary>instruct template</summary>
|
|
|
|
```yaml
|
|
{
|
|
"system_prompt": "Write {{char}}'s actions and dialogue, user will write {{user}}'s.",
|
|
"input_sequence": "<|im_start|>user\n",
|
|
"output_sequence": "<|im_start|>assistant\n",
|
|
"first_output_sequence": "",
|
|
"last_output_sequence": "",
|
|
"system_sequence_prefix": "",
|
|
"system_sequence_suffix": "",
|
|
"stop_sequence": "<|im_end|>",
|
|
"wrap": false,
|
|
"macro": true,
|
|
"names": false,
|
|
"names_force_groups": false,
|
|
"activation_regex": "",
|
|
"skip_examples": false,
|
|
"output_suffix": "<|im_end|>\n",
|
|
"input_suffix": "<|im_end|>\n",
|
|
"system_sequence": "<|im_start|>system\n",
|
|
"system_suffix": "<|im_end|>\n",
|
|
"user_alignment_message": "",
|
|
"last_system_sequence": "",
|
|
"system_same_as_user": false,
|
|
"first_input_sequence": "",
|
|
"last_input_sequence": "",
|
|
"name": "Dan-ChatML"
|
|
}
|
|
```
|
|
|
|
</details><br>
|
|
|
|
|
|
## Training
|
|
|
|
This model was full finetuned for 4 epochs on 8x H100 equating to 21 hours.
|
|
|
|
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl) |