ShieldX/manovyadh-1.1B-v1-chat
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: ShieldX/manovyadh-1.1B-v1-chat
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-23 09:19:17 +08:00
commit 62640c8600
10 changed files with 93882 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

349
README.md Normal file
View File

@@ -0,0 +1,349 @@
---
library_name: transformers
tags:
- medical
- trl
- trainer
license: apache-2.0
thumbnail: https://huggingface.co/ShieldX/manovyadh-1.1B-v1-chat/blob/main/manovyadh.png
datasets:
- ShieldX/manovyadh-3.5k
language:
- en
metrics:
- accuracy
pipeline_tag: text-generation
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
widget:
- text: >
###SYSTEM: You are an AI assistant that helps people cope with stress and improve their mental health. User will tell you about their feelings and challenges. Your task is to listen empathetically and offer helpful suggestions. While responding, think about the user’s needs and goals and show compassion and support
###USER: I don't know how to tell someone how I feel about them. How can I get better at expressing how I feel??
###ASSISTANT:
model-index:
- name: manovyadh-1.1B-v1-chat
results:
- task:
type: text-generation
dataset:
name: ai2_arc
type: arc
metrics:
- name: pass@1
type: pass@1
value: 35.92
source:
name: Open LLM Leaderboard
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
- task:
type: text-generation
dataset:
name: hellaswag
type: hellaswag
metrics:
- name: pass@1
type: pass@1
value: 60.03
source:
name: Open LLM Leaderboard
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
- task:
type: text-generation
dataset:
name: truthful_qa
type: truthful_qa
metrics:
- name: pass@1
type: pass@1
value: 39.17
source:
name: Open LLM Leaderboard
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
- task:
type: text-generation
dataset:
name: winogrande
type: winogrande
metrics:
- name: pass@1
type: pass@1
value: 61.09
source:
name: Open LLM Leaderboard
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
---
# Uploaded model
- **Developed by:** ShieldX
- **License:** apache-2.0
- **Finetuned from model :** TinyLlama/TinyLlama-1.1B-Chat-v1.0
<style>
img{
width: 40vw;
height: auto;
margin: 0 auto;
display: flex;
align-items: center;
justify-content: center;
}
</style>
# ShieldX/manovyadh-1.1B-v1
Introducing ManoVyadh, A finetuned version of TinyLlama 1.1B Chat on Mental Health Counselling Dataset.
<img class="custom-image" src="manovyadh.png" alt="BongLlama">
# Model Details
## Model Description
ManoVyadh is a LLM for mental health counselling.
# Uses
## Direct Use
- base model for further finetuning
- for fun
## Downstream Use
- can be deployed with api
- used to create webapp or app to show demo
## Out-of-Scope Use
- cannot be used for production purpose
- not to be applied in real life health purpose
- cannot be used to generate text for research or academic purposes
# Usage
```
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoConfig
tokenizer = AutoTokenizer.from_pretrained("ShieldX/manovyadh-1.1B-v1-chat")
model = AutoModelForCausalLM.from_pretrained("ShieldX/manovyadh-1.1B-v1-chat").to("cuda")
config = AutoConfig.from_pretrained("ShieldX/manovyadh-1.1B-v1-chat")
def format_prompt(q):
return f"""###SYSTEM: You are an AI assistant that helps people cope with stress and improve their mental health. User will tell you about their feelings and challenges. Your task is to listen empathetically and offer helpful suggestions. While responding, think about the user’s needs and goals and show compassion and support
###USER: {q}
###ASSISTANT:"""
prompt = format_prompt("I've never been able to talk with my parents. My parents are in their sixties while I am a teenager. I love both of them but not their personalities. I feel that they do not take me seriously whenever I talk about a serious event in my life. If my dad doesn’t believe me, then my mom goes along with my dad and acts like she doesn’t believe me either. I’m a pansexual, but I can’t trust my own parents. I've fought depression and won; however, stress and anxiety are killing me. I feel that my friends don't listen to me. I know they have their own problems, which I do my best to help with. But they don't always try to help me with mine, when I really need them. I feel as if my childhood has been taken from me. I feel as if I have no one whom I can trust.")
import torch
from transformers import GenerationConfig, TextStreamer
from time import perf_counter
# Check for GPU availability
if torch.cuda.is_available():
device = "cuda"
else:
device = "cpu"
# Move model and inputs to the GPU (if available)
model.to(device)
inputs = tokenizer(prompt, return_tensors="pt").to(device)
streamer = TextStreamer(tokenizer)
generation_config = GenerationConfig(
penalty_alpha=0.6,
do_sample=True,
top_k=5,
temperature=0.5,
repetition_penalty=1.2,
max_new_tokens=256,
streamer=streamer,
pad_token_id=tokenizer.eos_token_id
)
start_time = perf_counter()
outputs = model.generate(**inputs, generation_config=generation_config)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
output_time = perf_counter() - start_time
print(f"Time taken for inference: {round(output_time, 2)} seconds")
```
# Bias, Risks, and Limitations
Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
# Training Details
# Model Examination
We will be further finetuning this model on large dataset to see how it performs
# Environmental Impact
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
- **Hardware Type:** 1 X Tesla T4
- **Hours used:** 0.48
- **Cloud Provider:** Google Colab
- **Compute Region:** India
# Technical Specifications
## Model Architecture and Objective
Finetuned on Tiny-Llama 1.1B Chat model
### Hardware
1 X Tesla T4
# training
This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) on [ShieldX/manovyadh-3.5k](https://huggingface.co/datasets/ShieldX/manovyadh-3.5k) dataset.
It achieves the following results on the evaluation set:
- Loss: 1.8587
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2.5e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 400
- mixed_precision_training: Native AMP
-
### Training results
| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.5894 | 0.01 | 5 | 2.5428 |
| 2.5283 | 0.02 | 10 | 2.5240 |
| 2.5013 | 0.03 | 15 | 2.5033 |
| 2.378 | 0.05 | 20 | 2.4770 |
| 2.3735 | 0.06 | 25 | 2.4544 |
| 2.3894 | 0.07 | 30 | 2.4335 |
| 2.403 | 0.08 | 35 | 2.4098 |
| 2.3719 | 0.09 | 40 | 2.3846 |
| 2.3691 | 0.1 | 45 | 2.3649 |
| 2.3088 | 0.12 | 50 | 2.3405 |
| 2.3384 | 0.13 | 55 | 2.3182 |
| 2.2577 | 0.14 | 60 | 2.2926 |
| 2.245 | 0.15 | 65 | 2.2702 |
| 2.1389 | 0.16 | 70 | 2.2457 |
| 2.1482 | 0.17 | 75 | 2.2176 |
| 2.1567 | 0.18 | 80 | 2.1887 |
| 2.1533 | 0.2 | 85 | 2.1616 |
| 2.0629 | 0.21 | 90 | 2.1318 |
| 2.1068 | 0.22 | 95 | 2.0995 |
| 2.0196 | 0.23 | 100 | 2.0740 |
| 2.062 | 0.24 | 105 | 2.0461 |
| 1.9436 | 0.25 | 110 | 2.0203 |
| 1.9348 | 0.26 | 115 | 1.9975 |
| 1.8803 | 0.28 | 120 | 1.9747 |
| 1.9108 | 0.29 | 125 | 1.9607 |
| 1.7826 | 0.3 | 130 | 1.9506 |
| 1.906 | 0.31 | 135 | 1.9374 |
| 1.8745 | 0.32 | 140 | 1.9300 |
| 1.8634 | 0.33 | 145 | 1.9232 |
| 1.8561 | 0.35 | 150 | 1.9183 |
| 1.8371 | 0.36 | 155 | 1.9147 |
| 1.8006 | 0.37 | 160 | 1.9106 |
| 1.8941 | 0.38 | 165 | 1.9069 |
| 1.8456 | 0.39 | 170 | 1.9048 |
| 1.8525 | 0.4 | 175 | 1.9014 |
| 1.8475 | 0.41 | 180 | 1.8998 |
| 1.8255 | 0.43 | 185 | 1.8962 |
| 1.9358 | 0.44 | 190 | 1.8948 |
| 1.758 | 0.45 | 195 | 1.8935 |
| 1.7859 | 0.46 | 200 | 1.8910 |
| 1.8412 | 0.47 | 205 | 1.8893 |
| 1.835 | 0.48 | 210 | 1.8875 |
| 1.8739 | 0.49 | 215 | 1.8860 |
| 1.9397 | 0.51 | 220 | 1.8843 |
| 1.8187 | 0.52 | 225 | 1.8816 |
| 1.8174 | 0.53 | 230 | 1.8807 |
| 1.8 | 0.54 | 235 | 1.8794 |
| 1.7736 | 0.55 | 240 | 1.8772 |
| 1.7429 | 0.56 | 245 | 1.8778 |
| 1.8024 | 0.58 | 250 | 1.8742 |
| 1.8431 | 0.59 | 255 | 1.8731 |
| 1.7692 | 0.6 | 260 | 1.8706 |
| 1.8084 | 0.61 | 265 | 1.8698 |
| 1.7602 | 0.62 | 270 | 1.8705 |
| 1.7751 | 0.63 | 275 | 1.8681 |
| 1.7403 | 0.64 | 280 | 1.8672 |
| 1.8078 | 0.66 | 285 | 1.8648 |
| 1.8464 | 0.67 | 290 | 1.8648 |
| 1.7853 | 0.68 | 295 | 1.8651 |
| 1.8546 | 0.69 | 300 | 1.8643 |
| 1.8319 | 0.7 | 305 | 1.8633 |
| 1.7908 | 0.71 | 310 | 1.8614 |
| 1.738 | 0.72 | 315 | 1.8625 |
| 1.8868 | 0.74 | 320 | 1.8630 |
| 1.7744 | 0.75 | 325 | 1.8621 |
| 1.8292 | 0.76 | 330 | 1.8609 |
| 1.7905 | 0.77 | 335 | 1.8623 |
| 1.7652 | 0.78 | 340 | 1.8610 |
| 1.8371 | 0.79 | 345 | 1.8611 |
| 1.7024 | 0.81 | 350 | 1.8593 |
| 1.7328 | 0.82 | 355 | 1.8593 |
| 1.7376 | 0.83 | 360 | 1.8606 |
| 1.747 | 0.84 | 365 | 1.8601 |
| 1.7777 | 0.85 | 370 | 1.8602 |
| 1.8701 | 0.86 | 375 | 1.8598 |
| 1.7165 | 0.87 | 380 | 1.8579 |
| 1.779 | 0.89 | 385 | 1.8588 |
| 1.8536 | 0.9 | 390 | 1.8583 |
| 1.7263 | 0.91 | 395 | 1.8582 |
| 1.7983 | 0.92 | 400 | 1.8587 |
### Framework versions
- PEFT 0.7.1
- Transformers 4.37.1
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1
# Citation
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
**BibTeX:**
```
@misc{ShieldX/manovyadh-1.1B-v1-chat,
url={[https://huggingface.co/ShieldX/manovyadh-1.1B-v1-chat](https://huggingface.co/ShieldX/manovyadh-1.1B-v1-chat)},
title={ManoVyadh},
author={Rohan Shaw},
year={2024}, month={Jan}
}
```
# Model Card Authors
ShieldX a.k.a Rohan Shaw
# Model Card Contact
email : rohanshaw.dev@gmail.com

28
config.json Normal file
View File

@@ -0,0 +1,28 @@
{
"_name_or_path": "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 2048,
"initializer_range": 0.02,
"intermediate_size": 5632,
"max_position_embeddings": 2048,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 22,
"num_key_value_heads": 4,
"pretraining_tp": 1,
"rms_norm_eps": 1e-05,
"rope_scaling": null,
"rope_theta": 10000.0,
"tie_word_embeddings": false,
"torch_dtype": "float32",
"transformers_version": "4.37.1",
"use_cache": true,
"vocab_size": 32000
}

3
events.out.tfevents.1706250479.4b392ab000ca.1055.0 Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9730db3908179ef9ee343c7f685b470bdffb0abcf101a4405bec2c1e0f800f27
size 39218

7
generation_config.json Normal file
View File

@@ -0,0 +1,7 @@
{
"bos_token_id": 1,
"eos_token_id": 2,
"max_length": 2048,
"pad_token_id": 0,
"transformers_version": "4.37.1"
}

BIN
manovyadh.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 972 KiB

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:463bbb331c95baa0e80cc548e84251c999d71f631f8a40accc0a46a2a897c5a8
size 4400216536

24
special_tokens_map.json Normal file
View File

@@ -0,0 +1,24 @@
{
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": "</s>",
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

93391
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

42
tokenizer_config.json Normal file
View File

@@ -0,0 +1,42 @@
{
"add_bos_token": true,
"add_eos_token": false,
"added_tokens_decoder": {
"0": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"bos_token": "<s>",
"chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n' + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
"clean_up_tokenization_spaces": false,
"eos_token": "</s>",
"legacy": false,
"model_max_length": 2048,
"pad_token": "</s>",
"padding_side": "right",
"sp_model_kwargs": {},
"tokenizer_class": "LlamaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": false
}