初始化项目,由ModelHub XC社区提供模型
Model: simonko912/simllama-1.1-instruct Source: Original Platform
This commit is contained in:
62
README.md
Normal file
62
README.md
Normal file
@@ -0,0 +1,62 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
tags:
|
||||
- simllama
|
||||
- llama
|
||||
- smol-smoltalk
|
||||
- slm
|
||||
- 400m
|
||||
datasets:
|
||||
- HuggingFaceTB/smol-smoltalk
|
||||
- exnivo/tinybrain-instruct-sft-200k
|
||||
- OpenAssistant/oasst2
|
||||
- OpenAssistant/oasst1
|
||||
- drwlf/medra-thinking-768
|
||||
- togethercomputer/llama-instruct
|
||||
- interview-eval/MATH
|
||||
- RLAIF/mbpp
|
||||
---
|
||||
Checkpoint info: Loss around 1.8 when training. Epoch 2. More data unavaiable due to my trainer bugging out.<br>
|
||||
Loss on whole dataset: 2.0506176433255594 <br>
|
||||
Perplexity on whole dataset: 7.77 <br>
|
||||
Base model: https://huggingface.co/simonko912/simllama-1.1-base <br>
|
||||
|
||||
| Dataset | Loss | Perplexity |
|
||||
|----------------|-------|------------|
|
||||
| WikiText-2 | 3.188 | 24.24 |
|
||||
| Penn Treebank | 4.059 | 57.94 |
|
||||
| LAMBADA | 3.656 | 38.70 |
|
||||
| C4 | 3.302 | 27.16 |
|
||||
|
||||
Context: 4096 tokens total <br>
|
||||
Size: 0.4B <br>
|
||||
Extra info: Dense model, llama like arch. <br>
|
||||
|
||||
### Examples
|
||||
````
|
||||
User: Make a hello world python program [/INST]
|
||||
Full prompt format (leave empty for raw input):
|
||||
|
||||
Assistant: Here's an example Python program that takes in a string as input and prints it:
|
||||
```
|
||||
python
|
||||
def hello_world():
|
||||
print("Hello, world!")
|
||||
```
|
||||
````
|
||||
````
|
||||
User: What is the capital of france? [/INST]
|
||||
Full prompt format (leave empty for raw input):
|
||||
|
||||
Assistant: The capital of France is Paris.
|
||||
***
|
||||
Paris is famous for its historical landmarks and cultural institutions. Here are some of the most popular tourist attractions:
|
||||
* Clock Tower: This iconic clock tower is one of the most famous in the world, and it's also one of the most famous clocks in the world.
|
||||
* Notre Dame Cathedral: A beautiful Gothic cathedral with intricate carvings and a stunning view of the city.
|
||||
* La Plaza de la Seine: A beautiful and historic landmark, it's one of the most famous landmarks in the world.
|
||||
* Vatican City: A UNESCO World Heritage Site, the Vatican City is one of the world's largest and most visited tourist destinations.
|
||||
* Santa Cruz de la Catedral: A historic city with a rich history and culture, it's one of the most famous and vibrant cities in the world.
|
||||
* Santa Monica de los Nevis: A historic city with a rich history and cultural significance, it's one of the most famous tourist spots in the world.
|
||||
````
|
||||
Reference in New Issue
Block a user