AmberYifan/capsdnum-marin-8b-base-code_ppl_b4000_s0
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: AmberYifan/capsdnum-marin-8b-base-code_ppl_b4000_s0
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-21 09:17:10 +08:00
commit d4946838d7
21 changed files with 455 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
.gitattributes vendored Normal file
View File

@@ -0,0 +1,37 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
checkpoint-63/tokenizer.json filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

61
README.md Normal file
View File

@@ -0,0 +1,61 @@
---
library_name: transformers
license: other
base_model: marin-community/marin-8b-base
tags:
- llama-factory
- full
- generated_from_trainer
model-index:
- name: marin-8b-base_code_ppl_b4000_s0
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# marin-8b-base_code_ppl_b4000_s0
This model is a fine-tuned version of [marin-community/marin-8b-base](https://huggingface.co/marin-community/marin-8b-base) on the capsd_marin-8b-base-n80000-opc__mix_code_ppl_b4000_s0 dataset.
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 0
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 8
- total_train_batch_size: 64
- total_eval_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 0.03
- num_epochs: 1
### Training results
### Framework versions
- Transformers 5.7.0
- Pytorch 2.13.0+cu130
- Datasets 4.0.0
- Tokenizers 0.22.2

8
all_results.json Normal file
View File

@@ -0,0 +1,8 @@
{
"epoch": 1.0,
"total_flos": 1164137594880.0,
"train_loss": 0.5609567505972726,
"train_runtime": 1966.3573,
"train_samples_per_second": 2.034,
"train_steps_per_second": 0.032
}

7
chat_template.jinja Normal file
View File

@@ -0,0 +1,7 @@
{{ '<|begin_of_text|>' }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}{% if system_message is defined %}{{ '<|start_header_id|>system<|end_header_id|>
' + system_message + '<|eot_id|>' }}{% endif %}{% for message in loop_messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|start_header_id|>user<|end_header_id|>
' + content + '<|eot_id|><|start_header_id|>assistant<|end_header_id|>
' }}{% elif message['role'] == 'assistant' %}{{ content + '<|eot_id|>' }}{% endif %}{% endfor %}

7
checkpoint-63/chat_template.jinja Normal file
View File

@@ -0,0 +1,7 @@
{{ '<|begin_of_text|>' }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}{% if system_message is defined %}{{ '<|start_header_id|>system<|end_header_id|>
' + system_message + '<|eot_id|>' }}{% endif %}{% for message in loop_messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|start_header_id|>user<|end_header_id|>
' + content + '<|eot_id|><|start_header_id|>assistant<|end_header_id|>
' }}{% elif message['role'] == 'assistant' %}{{ content + '<|eot_id|>' }}{% endif %}{% endfor %}

51
checkpoint-63/config.json Normal file
View File

@@ -0,0 +1,51 @@
{
"_attn_implementation_autoset": false,
"add_cross_attention": false,
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 128000,
"cross_attention_hidden_size": null,
"decoder_start_token_id": 128000,
"dtype": "bfloat16",
"eos_token_id": 128009,
"finetuning_task": null,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"is_decoder": false,
"max_position_embeddings": 4096,
"mlp_bias": false,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"num_key_value_heads": 8,
"pad_token_id": 128009,
"prefix": null,
"pretraining_tp": 1,
"pruned_heads": {},
"rms_norm_eps": 1e-05,
"rope_parameters": {
"factor": 8.0,
"high_freq_factor": 4.0,
"low_freq_factor": 1.0,
"original_max_position_embeddings": 8192,
"rope_theta": 500000,
"rope_type": "llama3"
},
"sep_token_id": null,
"task_specific_params": null,
"tf_legacy_loss": false,
"tie_encoder_decoder": false,
"tie_word_embeddings": false,
"tokenizer_class": null,
"torchscript": false,
"transformers_version": "5.7.0",
"use_bfloat16": false,
"use_cache": false,
"vocab_size": 128256
}

37
checkpoint-63/generation_config.json Normal file
View File

@@ -0,0 +1,37 @@
{
"_from_model_config": true,
"begin_suppress_tokens": [
128000,
128001
],
"bos_token_id": 128000,
"decoder_start_token_id": 128000,
"diversity_penalty": 0.0,
"do_sample": false,
"early_stopping": false,
"encoder_no_repeat_ngram_size": 0,
"eos_token_id": [
128009,
128001
],
"length_penalty": 1.0,
"max_length": 20,
"min_length": 0,
"no_repeat_ngram_size": 0,
"num_beam_groups": 1,
"num_beams": 1,
"num_return_sequences": 1,
"output_attentions": false,
"output_hidden_states": false,
"output_scores": false,
"pad_token_id": 128009,
"remove_invalid_values": false,
"repetition_penalty": 1.0,
"return_dict_in_generate": false,
"temperature": 1.0,
"top_k": 50,
"top_p": 1.0,
"transformers_version": "5.7.0",
"typical_p": 1.0,
"use_cache": true
}

3
checkpoint-63/model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3f2a8c9f22f0612decd091ec6e5e0f51ebdd9ac8c1fb59962073e9c79cedf0f2
size 16060556616

BIN
checkpoint-63/tokenizer.json (Stored with Git LFS) Normal file
View File

Binary file not shown.

20
checkpoint-63/tokenizer_config.json Normal file
View File

@@ -0,0 +1,20 @@
{
"backend": "tokenizers",
"bos_token": "<|begin_of_text|>",
"clean_up_tokenization_spaces": true,
"eos_token": "<|eot_id|>",
"extra_special_tokens": [
"<|eom_id|>"
],
"is_local": false,
"local_files_only": false,
"model_input_names": [
"input_ids",
"attention_mask"
],
"model_max_length": 131072,
"pad_token": "<|eot_id|>",
"padding_side": "right",
"split_special_tokens": false,
"tokenizer_class": "TokenizersBackend"
}

41
checkpoint-63/trainer_state.json Normal file
View File

@@ -0,0 +1,41 @@
{
"best_global_step": null,
"best_metric": null,
"best_model_checkpoint": null,
"epoch": 1.0,
"eval_steps": 500,
"global_step": 63,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"epoch": 0.8,
"grad_norm": 0.8434610737312371,
"learning_rate": 1.2443403456474017e-06,
"loss": 0.575324478149414,
"step": 50
}
],
"logging_steps": 50,
"max_steps": 63,
"num_input_tokens_seen": 0,
"num_train_epochs": 1,
"save_steps": 500,
"stateful_callbacks": {
"TrainerControl": {
"args": {
"should_epoch_stop": false,
"should_evaluate": false,
"should_log": false,
"should_save": true,
"should_training_stop": true
},
"attributes": {}
}
},
"total_flos": 1164137594880.0,
"train_batch_size": 2,
"trial_name": null,
"trial_params": null
}

3
checkpoint-63/training_args.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3baa351764226b048c642b49fda7d29ab1ffa2cc3c8192cc1046d41c15909aea
size 7761

51
config.json Normal file
View File

@@ -0,0 +1,51 @@
{
"_attn_implementation_autoset": false,
"add_cross_attention": false,
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 128000,
"cross_attention_hidden_size": null,
"decoder_start_token_id": 128000,
"dtype": "bfloat16",
"eos_token_id": 128009,
"finetuning_task": null,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"is_decoder": false,
"max_position_embeddings": 4096,
"mlp_bias": false,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"num_key_value_heads": 8,
"pad_token_id": 128009,
"prefix": null,
"pretraining_tp": 1,
"pruned_heads": {},
"rms_norm_eps": 1e-05,
"rope_parameters": {
"factor": 8.0,
"high_freq_factor": 4.0,
"low_freq_factor": 1.0,
"original_max_position_embeddings": 8192,
"rope_theta": 500000,
"rope_type": "llama3"
},
"sep_token_id": null,
"task_specific_params": null,
"tf_legacy_loss": false,
"tie_encoder_decoder": false,
"tie_word_embeddings": false,
"tokenizer_class": null,
"torchscript": false,
"transformers_version": "5.7.0",
"use_bfloat16": false,
"use_cache": false,
"vocab_size": 128256
}

37
generation_config.json Normal file
View File

@@ -0,0 +1,37 @@
{
"_from_model_config": true,
"begin_suppress_tokens": [
128000,
128001
],
"bos_token_id": 128000,
"decoder_start_token_id": 128000,
"diversity_penalty": 0.0,
"do_sample": false,
"early_stopping": false,
"encoder_no_repeat_ngram_size": 0,
"eos_token_id": [
128009,
128001
],
"length_penalty": 1.0,
"max_length": 20,
"min_length": 0,
"no_repeat_ngram_size": 0,
"num_beam_groups": 1,
"num_beams": 1,
"num_return_sequences": 1,
"output_attentions": false,
"output_hidden_states": false,
"output_scores": false,
"pad_token_id": 128009,
"remove_invalid_values": false,
"repetition_penalty": 1.0,
"return_dict_in_generate": false,
"temperature": 1.0,
"top_k": 50,
"top_p": 1.0,
"transformers_version": "5.7.0",
"typical_p": 1.0,
"use_cache": true
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3f2a8c9f22f0612decd091ec6e5e0f51ebdd9ac8c1fb59962073e9c79cedf0f2
size 16060556616

BIN
tokenizer.json (Stored with Git LFS) Normal file
View File

Binary file not shown.

20
tokenizer_config.json Normal file
View File

@@ -0,0 +1,20 @@
{
"backend": "tokenizers",
"bos_token": "<|begin_of_text|>",
"clean_up_tokenization_spaces": true,
"eos_token": "<|eot_id|>",
"extra_special_tokens": [
"<|eom_id|>"
],
"is_local": false,
"local_files_only": false,
"model_input_names": [
"input_ids",
"attention_mask"
],
"model_max_length": 131072,
"pad_token": "<|eot_id|>",
"padding_side": "right",
"split_special_tokens": false,
"tokenizer_class": "TokenizersBackend"
}

8
train_results.json Normal file
View File

@@ -0,0 +1,8 @@
{
"epoch": 1.0,
"total_flos": 1164137594880.0,
"train_loss": 0.5609567505972726,
"train_runtime": 1966.3573,
"train_samples_per_second": 2.034,
"train_steps_per_second": 0.032
}

2
trainer_log.jsonl Normal file
View File

@@ -0,0 +1,2 @@
{"current_steps": 50, "total_steps": 63, "loss": 0.575324478149414, "lr": 1.2443403456474017e-06, "epoch": 0.8, "percentage": 79.37, "elapsed_time": "0:26:01", "remaining_time": "0:06:46"}
{"current_steps": 63, "total_steps": 63, "epoch": 1.0, "percentage": 100.0, "elapsed_time": "0:32:46", "remaining_time": "0:00:00"}

50
trainer_state.json Normal file
View File

@@ -0,0 +1,50 @@
{
"best_global_step": null,
"best_metric": null,
"best_model_checkpoint": null,
"epoch": 1.0,
"eval_steps": 500,
"global_step": 63,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"epoch": 0.8,
"grad_norm": 0.8434610737312371,
"learning_rate": 1.2443403456474017e-06,
"loss": 0.575324478149414,
"step": 50
},
{
"epoch": 1.0,
"step": 63,
"total_flos": 1164137594880.0,
"train_loss": 0.5609567505972726,
"train_runtime": 1966.3573,
"train_samples_per_second": 2.034,
"train_steps_per_second": 0.032
}
],
"logging_steps": 50,
"max_steps": 63,
"num_input_tokens_seen": 0,
"num_train_epochs": 1,
"save_steps": 500,
"stateful_callbacks": {
"TrainerControl": {
"args": {
"should_epoch_stop": false,
"should_evaluate": false,
"should_log": false,
"should_save": true,
"should_training_stop": true
},
"attributes": {}
}
},
"total_flos": 1164137594880.0,
"train_batch_size": 2,
"trial_name": null,
"trial_params": null
}

3
training_args.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3baa351764226b048c642b49fda7d29ab1ffa2cc3c8192cc1046d41c15909aea
size 7761