初始化项目,由ModelHub XC社区提供模型
Model: TIGER-Lab/One-Shot-CFT-Math-Llama-3B Source: Original Platform
This commit is contained in:
commit
82c7bbbbc2
14 files changed
+3181
No files matched your search
@@ -0,0 +1,49 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||
*.tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
*.db* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ark* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.gguf* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ggml filter=lfs diff=lfs merge=lfs -text
|
||||
*.llamafile* filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
base_model:
|
||||
- meta-llama/Llama-3.2-3B-Instruct
|
||||
language:
|
||||
- en
|
||||
tags:
|
||||
- One-Shot-CFT
|
||||
pipeline_tag: text-generation
|
||||
library_name: transformers
|
||||
license: cc-by-4.0
|
||||
---
|
||||
|
||||
# One-Shot-CFT: Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/TIGER-AI-Lab/One-Shot-CFT" target="_blank">💻 Code</a> |
|
||||
<a href="https://arxiv.org/pdf/2506.03295" target="_blank">📄 Paper</a> |
|
||||
<a href="https://huggingface.co/datasets/TIGER-Lab/One-Shot-CFT-Data" target="_blank">📊 Dataset</a> |
|
||||
<a href="https://huggingface.co/collections/TIGER-Lab/one-shot-cft-683fbb4d2bcf698dbea8fb21" target="_blank">🤗 Model</a> |
|
||||
<a href="https://tiger-ai-lab.github.io/One-Shot-CFT/" target="_blank">🌐 Project Page</a>
|
||||
</p>
|
||||
|
||||
|
||||
|
||||
## 🧠 Overview
|
||||
|
||||
One-Shot Critique Fine-Tuning (CFT) is a simple, robust, and compute-efficient training paradigm for unleashing the reasoning capabilities of pretrained LLMs in both mathematical and logical domains. By leveraging critiques on just one problem, One-Shot CFT enables models like Qwen and LLaMA to match or even outperform reinforcement learning, while using 20× less compute.
|
||||
|
||||
Instead of learning from reference answers (as in supervised fine-tuning) or reward signals (as in reinforcement learning), One-Shot CFT enables models to learn from critiques of diverse solutions to a single problem, enhancing their exposure to varied reasoning patterns and mitigating overfitting. This exposes the LLMs to multiple perspectives and error types, thereby more effectively unleashing their reasoning potential.
|
||||
|
||||
|
||||
## ✨ Key Highlights
|
||||
|
||||
- **Unleashes Reasoning with One Example:** One-Shot CFT uses critiques of diverse model-generated solutions to a single problem to significantly boost performance across math and logic tasks. For example, with just 5 GPU hours of training on Qwen2.5-Math-7B, One-Shot CFT achieves an average improvement of +15% on six math benchmarks and +16% on three logic reasoning benchmarks.
|
||||
- **Outperforms RLVR and Full SFT with 20× Less Compute:** One-Shot CFT outperforms both one-shot Reinforcement Learning with Verifiable Rewards (RLVR) and full-dataset supervised fine-tuning, while requiring only 5 GPU hours on a 7B model—offering a much more efficient and stable training alternative.
|
||||
- **Robust Across Seeds and Model Scales:** One-Shot CFT remains effective across different seed problem choices and model sizes—from 1.5B to 14B parameters—demonstrating strong generalization and scalability.
|
||||
|
||||
**This specific model is the One-Shot CFT variant trained based on [Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) with [DSR-CFT-p0](https://huggingface.co/datasets/TIGER-Lab/One-Shot-CFT-Data) dataset.**
|
||||
|
||||
|
||||
## Main Results
|
||||
|
||||
<p align="center">
|
||||
<img src="https://cdn-uploads.huggingface.co/production/uploads/636a35eff8d9af4aea181608/DCxRSdeDrv-Db4VLuEl0T.png" alt="CFT Performance Comparison" width="1100"/>
|
||||
</p>
|
||||
|
||||
<p align="center"><em>
|
||||
One-shot CFT consistently improves mathematical and logical reasoning.
|
||||
<strong>Left:</strong> Average accuracy on six mathematical reasoning benchmarks for Qwen and LLaMA models, comparing base, SFT, RLVR, and CFT with only one training example.
|
||||
<strong>Right:</strong> In-domain accuracy on three logic reasoning benchmarks (BBEH subtasks) for Qwen2.5-Math-7B.
|
||||
Across both domains, CFT with a single problem significantly outperforms standard SFT and matches or exceeds reinforcement learning with much lower compute.
|
||||
</em></p>
|
||||
|
||||
|
||||
## Citation
|
||||
|
||||
If you find our work helpful, please cite it as:
|
||||
|
||||
```bibtex
|
||||
@article{wang2025unleashing,
|
||||
title={Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem},
|
||||
author={Wang, Yubo and Nie, Ping and Zou, Kai and Wu, Lijun and Chen, Wenhu},
|
||||
journal={arXiv preprint arXiv:2506.03295},
|
||||
year={2025}
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,366 @@
|
||||
{
|
||||
"model": "/data/yubo/models/Llama-3.2-3B-Instruct",
|
||||
"model_type": "llama3_2",
|
||||
"model_revision": null,
|
||||
"task_type": "causal_lm",
|
||||
"torch_dtype": "bfloat16",
|
||||
"attn_impl": "flash_attn",
|
||||
"num_labels": null,
|
||||
"problem_type": null,
|
||||
"rope_scaling": null,
|
||||
"device_map": null,
|
||||
"max_memory": {},
|
||||
"local_repo_path": null,
|
||||
"template": "llama3_2",
|
||||
"system": null,
|
||||
"max_length": null,
|
||||
"truncation_strategy": "delete",
|
||||
"max_pixels": null,
|
||||
"agent_template": null,
|
||||
"norm_bbox": null,
|
||||
"response_prefix": null,
|
||||
"padding_side": "right",
|
||||
"loss_scale": "default",
|
||||
"sequence_parallel_size": 1,
|
||||
"use_chat_template": true,
|
||||
"template_backend": "swift",
|
||||
"dataset": [
|
||||
"/data/yubo/CriticCoT/dsr_scripts_0516/dsr_one-shot_train_data_0514_p0.jsonl"
|
||||
],
|
||||
"val_dataset": [],
|
||||
"split_dataset_ratio": 0.0,
|
||||
"data_seed": 42,
|
||||
"dataset_num_proc": 4,
|
||||
"dataset_shuffle": true,
|
||||
"val_dataset_shuffle": false,
|
||||
"streaming": false,
|
||||
"interleave_prob": null,
|
||||
"stopping_strategy": "first_exhausted",
|
||||
"shuffle_buffer_size": 1000,
|
||||
"enable_cache": false,
|
||||
"download_mode": "reuse_dataset_if_exists",
|
||||
"columns": {},
|
||||
"strict": false,
|
||||
"remove_unused_columns": false,
|
||||
"model_name": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"model_author": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"custom_dataset_info": [],
|
||||
"quant_method": null,
|
||||
"quant_bits": null,
|
||||
"hqq_axis": null,
|
||||
"bnb_4bit_compute_dtype": "bfloat16",
|
||||
"bnb_4bit_quant_type": "nf4",
|
||||
"bnb_4bit_use_double_quant": true,
|
||||
"bnb_4bit_quant_storage": null,
|
||||
"max_new_tokens": 64,
|
||||
"temperature": 0.0,
|
||||
"top_k": null,
|
||||
"top_p": null,
|
||||
"repetition_penalty": null,
|
||||
"num_beams": 1,
|
||||
"stream": false,
|
||||
"stop_words": [],
|
||||
"logprobs": false,
|
||||
"top_logprobs": null,
|
||||
"ckpt_dir": null,
|
||||
"load_dataset_config": null,
|
||||
"lora_modules": [],
|
||||
"tuner_backend": "peft",
|
||||
"train_type": "full",
|
||||
"adapters": [],
|
||||
"external_plugins": [],
|
||||
"seed": 42,
|
||||
"model_kwargs": {},
|
||||
"load_args": false,
|
||||
"load_data_args": false,
|
||||
"use_hf": true,
|
||||
"hub_token": null,
|
||||
"custom_register_path": [],
|
||||
"ignore_args_error": false,
|
||||
"use_swift_lora": false,
|
||||
"output_dir": "/data/yubo/CriticCoT/ms-swift/output_models_dsr_llama-3b_ins_0516_p0/v0-20250516-192104",
|
||||
"overwrite_output_dir": false,
|
||||
"do_train": false,
|
||||
"do_eval": false,
|
||||
"do_predict": false,
|
||||
"eval_strategy": "steps",
|
||||
"prediction_loss_only": false,
|
||||
"per_device_train_batch_size": 1,
|
||||
"per_device_eval_batch_size": 1,
|
||||
"per_gpu_train_batch_size": null,
|
||||
"per_gpu_eval_batch_size": null,
|
||||
"gradient_accumulation_steps": 128,
|
||||
"eval_accumulation_steps": null,
|
||||
"eval_delay": 0,
|
||||
"torch_empty_cache_steps": null,
|
||||
"learning_rate": 5e-06,
|
||||
"weight_decay": 0.05,
|
||||
"adam_beta1": 0.9,
|
||||
"adam_beta2": 0.95,
|
||||
"adam_epsilon": 1e-08,
|
||||
"max_grad_norm": 1.0,
|
||||
"num_train_epochs": 40.0,
|
||||
"max_steps": -1,
|
||||
"lr_scheduler_type": "cosine",
|
||||
"lr_scheduler_kwargs": null,
|
||||
"warmup_ratio": 0.2,
|
||||
"warmup_steps": 0,
|
||||
"log_level": "passive",
|
||||
"log_level_replica": "warning",
|
||||
"log_on_each_node": true,
|
||||
"logging_dir": "/data/yubo/CriticCoT/ms-swift/output_models_dsr_llama-3b_ins_0516_p0/v0-20250516-192104/runs",
|
||||
"logging_strategy": "steps",
|
||||
"logging_first_step": true,
|
||||
"logging_steps": 1,
|
||||
"logging_nan_inf_filter": true,
|
||||
"save_strategy": "steps",
|
||||
"save_steps": 2.0,
|
||||
"save_total_limit": null,
|
||||
"save_safetensors": true,
|
||||
"save_on_each_node": false,
|
||||
"save_only_model": true,
|
||||
"restore_callback_states_from_checkpoint": false,
|
||||
"no_cuda": false,
|
||||
"use_cpu": false,
|
||||
"use_mps_device": false,
|
||||
"jit_mode_eval": false,
|
||||
"use_ipex": false,
|
||||
"bf16": true,
|
||||
"fp16": false,
|
||||
"fp16_opt_level": "O1",
|
||||
"half_precision_backend": "auto",
|
||||
"bf16_full_eval": false,
|
||||
"fp16_full_eval": false,
|
||||
"tf32": null,
|
||||
"local_rank": 0,
|
||||
"ddp_backend": "nccl",
|
||||
"tpu_num_cores": null,
|
||||
"tpu_metrics_debug": false,
|
||||
"debug": null,
|
||||
"dataloader_drop_last": false,
|
||||
"eval_steps": 2.0,
|
||||
"dataloader_num_workers": 4,
|
||||
"dataloader_prefetch_factor": null,
|
||||
"past_index": -1,
|
||||
"run_name": "/data/yubo/CriticCoT/ms-swift/output_models_dsr_llama-3b_ins_0516_p0/v0-20250516-192104",
|
||||
"disable_tqdm": null,
|
||||
"label_names": null,
|
||||
"load_best_model_at_end": false,
|
||||
"metric_for_best_model": "loss",
|
||||
"greater_is_better": false,
|
||||
"ignore_data_skip": false,
|
||||
"fsdp": "",
|
||||
"fsdp_min_num_params": 0,
|
||||
"fsdp_config": null,
|
||||
"tp_size": 0,
|
||||
"fsdp_transformer_layer_cls_to_wrap": null,
|
||||
"accelerator_config": {
|
||||
"dispatch_batches": false
|
||||
},
|
||||
"deepspeed": {
|
||||
"fp16": {
|
||||
"enabled": "auto",
|
||||
"loss_scale": 0,
|
||||
"loss_scale_window": 1000,
|
||||
"initial_scale_power": 16,
|
||||
"hysteresis": 2,
|
||||
"min_loss_scale": 1
|
||||
},
|
||||
"bf16": {
|
||||
"enabled": "auto"
|
||||
},
|
||||
"zero_optimization": {
|
||||
"stage": 3,
|
||||
"offload_optimizer": {
|
||||
"device": "none",
|
||||
"pin_memory": true
|
||||
},
|
||||
"offload_param": {
|
||||
"device": "none",
|
||||
"pin_memory": true
|
||||
},
|
||||
"overlap_comm": false,
|
||||
"contiguous_gradients": true,
|
||||
"sub_group_size": 1000000000.0,
|
||||
"reduce_bucket_size": "auto",
|
||||
"zero_quantized_weights": false,
|
||||
"zero_quantized_gradients": false,
|
||||
"stage3_prefetch_bucket_size": "auto",
|
||||
"stage3_param_persistence_threshold": "auto",
|
||||
"stage3_max_live_parameters": 1000000000.0,
|
||||
"stage3_max_reuse_distance": 1000000000.0,
|
||||
"stage3_gather_16bit_weights_on_model_save": true
|
||||
},
|
||||
"gradient_accumulation_steps": "auto",
|
||||
"gradient_clipping": "auto",
|
||||
"steps_per_print": 2000,
|
||||
"train_batch_size": "auto",
|
||||
"train_micro_batch_size_per_gpu": "auto",
|
||||
"wall_clock_breakdown": false
|
||||
},
|
||||
"label_smoothing_factor": 0.0,
|
||||
"optim": "adamw_torch",
|
||||
"optim_args": null,
|
||||
"adafactor": false,
|
||||
"group_by_length": false,
|
||||
"length_column_name": "length",
|
||||
"report_to": [
|
||||
"none"
|
||||
],
|
||||
"ddp_find_unused_parameters": null,
|
||||
"ddp_bucket_cap_mb": null,
|
||||
"ddp_broadcast_buffers": null,
|
||||
"dataloader_pin_memory": true,
|
||||
"dataloader_persistent_workers": false,
|
||||
"skip_memory_metrics": true,
|
||||
"use_legacy_prediction_loop": false,
|
||||
"push_to_hub": false,
|
||||
"resume_from_checkpoint": null,
|
||||
"hub_model_id": null,
|
||||
"hub_strategy": "every_save",
|
||||
"hub_private_repo": null,
|
||||
"hub_always_push": false,
|
||||
"gradient_checkpointing": true,
|
||||
"gradient_checkpointing_kwargs": null,
|
||||
"include_inputs_for_metrics": false,
|
||||
"include_for_metrics": [],
|
||||
"eval_do_concat_batches": true,
|
||||
"fp16_backend": "auto",
|
||||
"push_to_hub_model_id": null,
|
||||
"push_to_hub_organization": null,
|
||||
"push_to_hub_token": null,
|
||||
"mp_parameters": "",
|
||||
"auto_find_batch_size": false,
|
||||
"full_determinism": false,
|
||||
"torchdynamo": null,
|
||||
"ray_scope": "last",
|
||||
"ddp_timeout": 1800,
|
||||
"torch_compile": false,
|
||||
"torch_compile_backend": null,
|
||||
"torch_compile_mode": null,
|
||||
"include_tokens_per_second": false,
|
||||
"include_num_input_tokens_seen": false,
|
||||
"neftune_noise_alpha": null,
|
||||
"optim_target_modules": null,
|
||||
"batch_eval_metrics": false,
|
||||
"eval_on_start": false,
|
||||
"use_liger_kernel": false,
|
||||
"eval_use_gather_object": false,
|
||||
"average_tokens_across_devices": false,
|
||||
"sortish_sampler": false,
|
||||
"predict_with_generate": false,
|
||||
"generation_max_length": null,
|
||||
"generation_num_beams": null,
|
||||
"generation_config": null,
|
||||
"check_model": true,
|
||||
"acc_strategy": "token",
|
||||
"train_dataloader_shuffle": true,
|
||||
"metric_warmup_step": 0,
|
||||
"fsdp_num": 1,
|
||||
"acc_steps": 1,
|
||||
"eval_use_evalscope": false,
|
||||
"eval_datasets": [],
|
||||
"eval_limit": null,
|
||||
"eval_datasets_args": null,
|
||||
"eval_generation_config": null,
|
||||
"freeze_parameters": [],
|
||||
"freeze_parameters_ratio": 0.0,
|
||||
"trainable_parameters": [],
|
||||
"freeze_llm": false,
|
||||
"freeze_vit": false,
|
||||
"freeze_aligner": false,
|
||||
"target_modules": [
|
||||
"all-linear"
|
||||
],
|
||||
"target_regex": null,
|
||||
"modules_to_save": [],
|
||||
"lora_rank": 8,
|
||||
"lora_alpha": 32,
|
||||
"lora_dropout": 0.05,
|
||||
"lora_bias": "none",
|
||||
"lora_dtype": null,
|
||||
"lorap_lr_ratio": null,
|
||||
"use_rslora": false,
|
||||
"use_dora": false,
|
||||
"lora_ga_batch_size": 2,
|
||||
"lora_ga_iters": 2,
|
||||
"lora_ga_max_length": 1024,
|
||||
"lora_ga_direction": "ArB2r",
|
||||
"lora_ga_scale": "stable",
|
||||
"lora_ga_stable_gamma": 16,
|
||||
"init_weights": true,
|
||||
"fourier_n_frequency": 2000,
|
||||
"fourier_scaling": 300.0,
|
||||
"boft_block_size": 4,
|
||||
"boft_block_num": 0,
|
||||
"boft_n_butterfly_factor": 1,
|
||||
"boft_dropout": 0.0,
|
||||
"vera_rank": 256,
|
||||
"vera_projection_prng_key": 0,
|
||||
"vera_dropout": 0.0,
|
||||
"vera_d_initial": 0.1,
|
||||
"adapter_act": "gelu",
|
||||
"adapter_length": 128,
|
||||
"use_galore": false,
|
||||
"galore_target_modules": null,
|
||||
"galore_rank": 128,
|
||||
"galore_update_proj_gap": 50,
|
||||
"galore_scale": 1.0,
|
||||
"galore_proj_type": "std",
|
||||
"galore_optim_per_parameter": false,
|
||||
"galore_with_embedding": false,
|
||||
"galore_quantization": false,
|
||||
"galore_proj_quant": false,
|
||||
"galore_proj_bits": 4,
|
||||
"galore_proj_group_size": 256,
|
||||
"galore_cos_threshold": 0.4,
|
||||
"galore_gamma_proj": 2,
|
||||
"galore_queue_size": 5,
|
||||
"adalora_target_r": 8,
|
||||
"adalora_init_r": 12,
|
||||
"adalora_tinit": 0,
|
||||
"adalora_tfinal": 0,
|
||||
"adalora_deltaT": 1,
|
||||
"adalora_beta1": 0.85,
|
||||
"adalora_beta2": 0.85,
|
||||
"adalora_orth_reg_weight": 0.5,
|
||||
"llamapro_num_new_blocks": 4,
|
||||
"llamapro_num_groups": null,
|
||||
"lisa_activated_layers": 0,
|
||||
"lisa_step_interval": 20,
|
||||
"reft_layer_key": null,
|
||||
"reft_layers": null,
|
||||
"reft_rank": 4,
|
||||
"reft_intervention_type": "LoreftIntervention",
|
||||
"reft_args": null,
|
||||
"swanlab_token": null,
|
||||
"swanlab_project": null,
|
||||
"swanlab_workspace": null,
|
||||
"swanlab_exp_name": null,
|
||||
"swanlab_mode": "cloud",
|
||||
"add_version": true,
|
||||
"resume_only_model": false,
|
||||
"create_checkpoint_symlink": false,
|
||||
"packing": false,
|
||||
"lazy_tokenize": false,
|
||||
"loss_type": null,
|
||||
"optimizer": null,
|
||||
"metric": null,
|
||||
"zero_hpz_partition_size": null,
|
||||
"rank": 0,
|
||||
"global_world_size": 4,
|
||||
"local_world_size": 4,
|
||||
"model_suffix": "Llama-3.2-3B-Instruct",
|
||||
"model_info": "ModelInfo(model_type='llama3_2', model_dir='/data/yubo/models/Llama-3.2-3B-Instruct', torch_dtype=torch.bfloat16, max_model_len=131072, quant_method=None, quant_bits=None, rope_scaling={'factor': 32.0, 'high_freq_factor': 4.0, 'low_freq_factor': 1.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}, config=None, task_type='causal_lm', num_labels=None)",
|
||||
"model_meta": "ModelMeta(model_type='llama3_2', model_groups=[ModelGroup(models=[Model(ms_model_id='LLM-Research/Llama-3.2-1B', hf_model_id='meta-llama/Llama-3.2-1B', model_path=None, ms_revision=None, hf_revision=None), Model(ms_model_id='LLM-Research/Llama-3.2-3B', hf_model_id='meta-llama/Llama-3.2-3B', model_path=None, ms_revision=None, hf_revision=None), Model(ms_model_id='LLM-Research/Llama-3.2-1B-Instruct', hf_model_id='meta-llama/Llama-3.2-1B-Instruct', model_path=None, ms_revision=None, hf_revision=None), Model(ms_model_id='LLM-Research/Llama-3.2-3B-Instruct', hf_model_id='meta-llama/Llama-3.2-3B-Instruct', model_path=None, ms_revision=None, hf_revision=None)], ignore_patterns=None, requires=None, tags=[]), ModelGroup(models=[Model(ms_model_id='LLM-Research/Llama-3.3-70B-Instruct', hf_model_id='meta-llama/Llama-3.3-70B-Instruct', model_path=None, ms_revision=None, hf_revision=None), Model(ms_model_id='unsloth/Llama-3.3-70B-Instruct-bnb-4bit', hf_model_id='unsloth/Llama-3.3-70B-Instruct-bnb-4bit', model_path=None, ms_revision=None, hf_revision=None)], ignore_patterns=None, requires=None, tags=[])], template='llama3_2', get_function=<function get_model_tokenizer_with_flash_attn at 0x727c58a05fc0>, model_arch='llama', architectures=['LlamaForCausalLM'], additional_saved_files=[], torch_dtype=None, is_multimodal=False, is_reward=False, task_type=None, ignore_patterns=None, requires=['transformers>=4.43'], tags=[])",
|
||||
"model_dir": "/data/yubo/models/Llama-3.2-3B-Instruct",
|
||||
"hub": "<class 'swift.hub.hub.HFHub'>",
|
||||
"evaluation_strategy": "steps",
|
||||
"training_args": "Seq2SeqTrainingArguments(output_dir='/data/yubo/CriticCoT/ms-swift/output_models_dsr_llama-3b_ins_0516_p0/v0-20250516-192104', overwrite_output_dir=False, do_train=False, do_eval=True, do_predict=False, eval_strategy=<IntervalStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=1, per_device_eval_batch_size=1, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=128, eval_accumulation_steps=None, eval_delay=0, torch_empty_cache_steps=None, learning_rate=5e-06, weight_decay=0.05, adam_beta1=0.9, adam_beta2=0.95, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=40.0, max_steps=-1, lr_scheduler_type=<SchedulerType.COSINE: 'cosine'>, lr_scheduler_kwargs=None, warmup_ratio=0.2, warmup_steps=0, log_level='passive', log_level_replica='warning', log_on_each_node=True, logging_dir='/data/yubo/CriticCoT/ms-swift/output_models_dsr_llama-3b_ins_0516_p0/v0-20250516-192104/runs', logging_strategy=<IntervalStrategy.STEPS: 'steps'>, logging_first_step=True, logging_steps=1, logging_nan_inf_filter=True, save_strategy=<SaveStrategy.STEPS: 'steps'>, save_steps=2, save_total_limit=None, save_safetensors=True, save_on_each_node=False, save_only_model=True, restore_callback_states_from_checkpoint=False, no_cuda=False, use_cpu=False, use_mps_device=False, seed=42, data_seed=42, jit_mode_eval=False, use_ipex=False, bf16=True, fp16=False, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=0, ddp_backend='nccl', tpu_num_cores=None, tpu_metrics_debug=False, debug=[], dataloader_drop_last=False, eval_steps=2, dataloader_num_workers=4, dataloader_prefetch_factor=10, past_index=-1, run_name='/data/yubo/CriticCoT/ms-swift/output_models_dsr_llama-3b_ins_0516_p0/v0-20250516-192104', disable_tqdm=False, remove_unused_columns=False, label_names=None, load_best_model_at_end=False, metric_for_best_model='loss', greater_is_better=False, ignore_data_skip=False, fsdp=[], fsdp_min_num_params=0, fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, tp_size=0, fsdp_transformer_layer_cls_to_wrap=None, accelerator_config=AcceleratorConfig(split_batches=False, dispatch_batches=False, even_batches=True, use_seedable_sampler=True, non_blocking=False, gradient_accumulation_kwargs=None, use_configured_state=False), deepspeed={'fp16': {'enabled': 'auto', 'loss_scale': 0, 'loss_scale_window': 1000, 'initial_scale_power': 16, 'hysteresis': 2, 'min_loss_scale': 1}, 'bf16': {'enabled': 'auto'}, 'zero_optimization': {'stage': 3, 'offload_optimizer': {'device': 'none', 'pin_memory': True}, 'offload_param': {'device': 'none', 'pin_memory': True}, 'overlap_comm': False, 'contiguous_gradients': True, 'sub_group_size': 1000000000.0, 'reduce_bucket_size': 'auto', 'zero_quantized_weights': False, 'zero_quantized_gradients': False, 'stage3_prefetch_bucket_size': 'auto', 'stage3_param_persistence_threshold': 'auto', 'stage3_max_live_parameters': 1000000000.0, 'stage3_max_reuse_distance': 1000000000.0, 'stage3_gather_16bit_weights_on_model_save': True}, 'gradient_accumulation_steps': 'auto', 'gradient_clipping': 'auto', 'steps_per_print': 2000, 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'wall_clock_breakdown': False}, label_smoothing_factor=0.0, optim=<OptimizerNames.ADAMW_TORCH: 'adamw_torch'>, optim_args=None, adafactor=False, group_by_length=False, length_column_name='length', report_to=[], ddp_find_unused_parameters=None, ddp_bucket_cap_mb=None, ddp_broadcast_buffers=None, dataloader_pin_memory=True, dataloader_persistent_workers=False, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=False, resume_from_checkpoint=None, hub_model_id=None, hub_strategy=<HubStrategy.EVERY_SAVE: 'every_save'>, hub_token=None, hub_private_repo=None, hub_always_push=False, gradient_checkpointing=True, gradient_checkpointing_kwargs=None, include_inputs_for_metrics=False, include_for_metrics=[], eval_do_concat_batches=True, fp16_backend='auto', push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, include_tokens_per_second=None, include_num_input_tokens_seen=None, neftune_noise_alpha=None, optim_target_modules=None, batch_eval_metrics=False, eval_on_start=False, use_liger_kernel=False, eval_use_gather_object=False, average_tokens_across_devices=None, sortish_sampler=False, predict_with_generate=False, generation_max_length=None, generation_num_beams=None, generation_config=None, check_model=True, acc_strategy='token', train_dataloader_shuffle=True, metric_warmup_step=0, fsdp_num=1, acc_steps=1, eval_use_evalscope=False, eval_datasets=[], eval_limit=None, eval_datasets_args=None, eval_generation_config=None, train_type=Line truncated
|
||||
}
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 128000,
|
||||
"eos_token_id": [
|
||||
128001,
|
||||
128008,
|
||||
128009
|
||||
],
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 3072,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 8192,
|
||||
"max_position_embeddings": 131072,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 24,
|
||||
"num_hidden_layers": 28,
|
||||
"num_key_value_heads": 8,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": {
|
||||
"factor": 32.0,
|
||||
"high_freq_factor": 4.0,
|
||||
"low_freq_factor": 1.0,
|
||||
"original_max_position_embeddings": 8192,
|
||||
"rope_type": "llama3"
|
||||
},
|
||||
"rope_theta": 500000.0,
|
||||
"tie_word_embeddings": true,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.51.3",
|
||||
"use_cache": false,
|
||||
"vocab_size": 128256
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"framework": "pytorch", "task": "others", "allow_remote": true}
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"bos_token_id": 128000,
|
||||
"do_sample": true,
|
||||
"eos_token_id": [
|
||||
128001,
|
||||
128008,
|
||||
128009
|
||||
],
|
||||
"temperature": 0.6,
|
||||
"top_p": 0.9,
|
||||
"transformers_version": "4.51.3"
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:930d90d2785d572c61845e7b4dbcb8ca95dc190534ca7b36adeb841bc4798f84
|
||||
size 4965799096
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:687359552b1578c07637c742ec86ec8a14b634b57695140988c766652eab374e
|
||||
size 1459729952
|
||||
@@ -0,0 +1,261 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 6425499648
|
||||
},
|
||||
"weight_map": {
|
||||
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.norm.weight": "model-00002-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<|begin_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|eot_id|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "<|eot_id|>"
|
||||
}
|
||||
LFS
BIN
Binary file not shown.
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,294 @@
|
||||
{
|
||||
"best_global_step": null,
|
||||
"best_metric": null,
|
||||
"best_model_checkpoint": null,
|
||||
"epoch": 25.771084337349397,
|
||||
"eval_steps": 2,
|
||||
"global_step": 26,
|
||||
"is_hyper_param_search": false,
|
||||
"is_local_process_zero": true,
|
||||
"is_world_process_zero": true,
|
||||
"log_history": [
|
||||
{
|
||||
"epoch": 0.7710843373493976,
|
||||
"grad_norm": 6.121489677872157,
|
||||
"learning_rate": 6.25e-07,
|
||||
"loss": 0.8753013014793396,
|
||||
"memory(GiB)": 34.86,
|
||||
"step": 1,
|
||||
"token_acc": 0.7918330258556598,
|
||||
"train_speed(iter/s)": 0.009904
|
||||
},
|
||||
{
|
||||
"epoch": 1.7710843373493976,
|
||||
"grad_norm": 11.509424437532436,
|
||||
"learning_rate": 1.25e-06,
|
||||
"loss": 1.762073278427124,
|
||||
"memory(GiB)": 39.41,
|
||||
"step": 2,
|
||||
"token_acc": 0.7978716452742124,
|
||||
"train_speed(iter/s)": 0.00936
|
||||
},
|
||||
{
|
||||
"epoch": 2.7710843373493974,
|
||||
"grad_norm": 11.323944309901117,
|
||||
"learning_rate": 1.8750000000000003e-06,
|
||||
"loss": 1.7230606079101562,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 3,
|
||||
"token_acc": 0.7989877731008218,
|
||||
"train_speed(iter/s)": 0.008908
|
||||
},
|
||||
{
|
||||
"epoch": 3.7710843373493974,
|
||||
"grad_norm": 10.87196352874565,
|
||||
"learning_rate": 2.5e-06,
|
||||
"loss": 1.7452430725097656,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 4,
|
||||
"token_acc": 0.7974636739751764,
|
||||
"train_speed(iter/s)": 0.008888
|
||||
},
|
||||
{
|
||||
"epoch": 4.771084337349397,
|
||||
"grad_norm": 11.053879691862349,
|
||||
"learning_rate": 3.125e-06,
|
||||
"loss": 1.683917760848999,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 5,
|
||||
"token_acc": 0.7954782471812833,
|
||||
"train_speed(iter/s)": 0.008692
|
||||
},
|
||||
{
|
||||
"epoch": 5.771084337349397,
|
||||
"grad_norm": 8.606854760903413,
|
||||
"learning_rate": 3.7500000000000005e-06,
|
||||
"loss": 1.5291715860366821,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 6,
|
||||
"token_acc": 0.8125722279666687,
|
||||
"train_speed(iter/s)": 0.008684
|
||||
},
|
||||
{
|
||||
"epoch": 6.771084337349397,
|
||||
"grad_norm": 5.022068500005259,
|
||||
"learning_rate": 4.3750000000000005e-06,
|
||||
"loss": 1.410496473312378,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 7,
|
||||
"token_acc": 0.8070179394950782,
|
||||
"train_speed(iter/s)": 0.008583
|
||||
},
|
||||
{
|
||||
"epoch": 7.771084337349397,
|
||||
"grad_norm": 3.8329268469171702,
|
||||
"learning_rate": 5e-06,
|
||||
"loss": 1.1963456869125366,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 8,
|
||||
"token_acc": 0.8275855412383573,
|
||||
"train_speed(iter/s)": 0.008616
|
||||
},
|
||||
{
|
||||
"epoch": 8.771084337349398,
|
||||
"grad_norm": 3.848141778586357,
|
||||
"learning_rate": 4.987961816680493e-06,
|
||||
"loss": 1.1539512872695923,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 9,
|
||||
"token_acc": 0.8452060931899642,
|
||||
"train_speed(iter/s)": 0.00854
|
||||
},
|
||||
{
|
||||
"epoch": 9.771084337349398,
|
||||
"grad_norm": 2.977196631037463,
|
||||
"learning_rate": 4.9519632010080765e-06,
|
||||
"loss": 1.0900822877883911,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 10,
|
||||
"token_acc": 0.8439449530665865,
|
||||
"train_speed(iter/s)": 0.008583
|
||||
},
|
||||
{
|
||||
"epoch": 10.771084337349398,
|
||||
"grad_norm": 2.3240379853396145,
|
||||
"learning_rate": 4.8923508393305224e-06,
|
||||
"loss": 0.9584915637969971,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 11,
|
||||
"token_acc": 0.8541569662165658,
|
||||
"train_speed(iter/s)": 0.00854
|
||||
},
|
||||
{
|
||||
"epoch": 11.771084337349398,
|
||||
"grad_norm": 1.7059344045170224,
|
||||
"learning_rate": 4.809698831278217e-06,
|
||||
"loss": 0.9206792116165161,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 12,
|
||||
"token_acc": 0.8550325931866718,
|
||||
"train_speed(iter/s)": 0.00856
|
||||
},
|
||||
{
|
||||
"epoch": 12.771084337349398,
|
||||
"grad_norm": 1.7886326192292616,
|
||||
"learning_rate": 4.704803160870888e-06,
|
||||
"loss": 0.8803208470344543,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 13,
|
||||
"token_acc": 0.8565676850786719,
|
||||
"train_speed(iter/s)": 0.008514
|
||||
},
|
||||
{
|
||||
"epoch": 13.771084337349398,
|
||||
"grad_norm": 1.5286406890043707,
|
||||
"learning_rate": 4.578674030756364e-06,
|
||||
"loss": 0.8406718969345093,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 14,
|
||||
"token_acc": 0.868490055655166,
|
||||
"train_speed(iter/s)": 0.008553
|
||||
},
|
||||
{
|
||||
"epoch": 14.771084337349398,
|
||||
"grad_norm": 1.4093835831424686,
|
||||
"learning_rate": 4.432526133406843e-06,
|
||||
"loss": 0.816148042678833,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 15,
|
||||
"token_acc": 0.8801949289867506,
|
||||
"train_speed(iter/s)": 0.008514
|
||||
},
|
||||
{
|
||||
"epoch": 15.771084337349398,
|
||||
"grad_norm": 1.3680984858266587,
|
||||
"learning_rate": 4.267766952966369e-06,
|
||||
"loss": 0.7781298756599426,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 16,
|
||||
"token_acc": 0.8775519188228432,
|
||||
"train_speed(iter/s)": 0.008531
|
||||
},
|
||||
{
|
||||
"epoch": 16.771084337349397,
|
||||
"grad_norm": 0.6513969535166108,
|
||||
"learning_rate": 4.085983210409114e-06,
|
||||
"loss": 0.7328703999519348,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 17,
|
||||
"token_acc": 0.8854784825706624,
|
||||
"train_speed(iter/s)": 0.008507
|
||||
},
|
||||
{
|
||||
"epoch": 17.771084337349397,
|
||||
"grad_norm": 1.1321914535679016,
|
||||
"learning_rate": 3.888925582549006e-06,
|
||||
"loss": 0.7167081832885742,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 18,
|
||||
"token_acc": 0.8828302499188575,
|
||||
"train_speed(iter/s)": 0.008528
|
||||
},
|
||||
{
|
||||
"epoch": 18.771084337349397,
|
||||
"grad_norm": 1.1087830646957209,
|
||||
"learning_rate": 3.6784918420649952e-06,
|
||||
"loss": 0.6928962469100952,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 19,
|
||||
"token_acc": 0.8914687444586997,
|
||||
"train_speed(iter/s)": 0.008501
|
||||
},
|
||||
{
|
||||
"epoch": 19.771084337349397,
|
||||
"grad_norm": 1.0244408604059563,
|
||||
"learning_rate": 3.4567085809127247e-06,
|
||||
"loss": 0.6718354821205139,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 20,
|
||||
"token_acc": 0.8931382342286962,
|
||||
"train_speed(iter/s)": 0.008518
|
||||
},
|
||||
{
|
||||
"epoch": 20.771084337349397,
|
||||
"grad_norm": 0.9684342265578457,
|
||||
"learning_rate": 3.225711693136156e-06,
|
||||
"loss": 0.64753657579422,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 21,
|
||||
"token_acc": 0.898327751680115,
|
||||
"train_speed(iter/s)": 0.008492
|
||||
},
|
||||
{
|
||||
"epoch": 21.771084337349397,
|
||||
"grad_norm": 0.8695314329605501,
|
||||
"learning_rate": 2.9877258050403214e-06,
|
||||
"loss": 0.6080504655838013,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 22,
|
||||
"token_acc": 0.8969131371141421,
|
||||
"train_speed(iter/s)": 0.008511
|
||||
},
|
||||
{
|
||||
"epoch": 22.771084337349397,
|
||||
"grad_norm": 0.7610645886404945,
|
||||
"learning_rate": 2.7450428508239024e-06,
|
||||
"loss": 0.5871363878250122,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 23,
|
||||
"token_acc": 0.9013859215427465,
|
||||
"train_speed(iter/s)": 0.008488
|
||||
},
|
||||
{
|
||||
"epoch": 23.771084337349397,
|
||||
"grad_norm": 0.838874811475282,
|
||||
"learning_rate": 2.5e-06,
|
||||
"loss": 0.6137609481811523,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 24,
|
||||
"token_acc": 0.9088375088841507,
|
||||
"train_speed(iter/s)": 0.008512
|
||||
},
|
||||
{
|
||||
"epoch": 24.771084337349397,
|
||||
"grad_norm": 0.7953361813418657,
|
||||
"learning_rate": 2.2549571491760985e-06,
|
||||
"loss": 0.6176888942718506,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 25,
|
||||
"token_acc": 0.9058841092793619,
|
||||
"train_speed(iter/s)": 0.008488
|
||||
},
|
||||
{
|
||||
"epoch": 25.771084337349397,
|
||||
"grad_norm": 0.8068676839609372,
|
||||
"learning_rate": 2.01227419495968e-06,
|
||||
"loss": 0.5883712768554688,
|
||||
"memory(GiB)": 42.29,
|
||||
"step": 26,
|
||||
"token_acc": 0.9068480043739748,
|
||||
"train_speed(iter/s)": 0.008491
|
||||
}
|
||||
],
|
||||
"logging_steps": 1,
|
||||
"max_steps": 40,
|
||||
"num_input_tokens_seen": 0,
|
||||
"num_train_epochs": 40,
|
||||
"save_steps": 2,
|
||||
"stateful_callbacks": {
|
||||
"TrainerControl": {
|
||||
"args": {
|
||||
"should_epoch_stop": false,
|
||||
"should_evaluate": false,
|
||||
"should_log": false,
|
||||
"should_save": true,
|
||||
"should_training_stop": false
|
||||
},
|
||||
"attributes": {}
|
||||
}
|
||||
},
|
||||
"total_flos": 22664558936064.0,
|
||||
"train_batch_size": 1,
|
||||
"trial_name": null,
|
||||
"trial_params": null
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:40c82bd5b17ce662a75098c6135442ddb3f08f8656f3aa621c929074a2479028
|
||||
size 8312
|
||||
Reference in new issue
Block a user