初始化项目，由ModelHub XC社区提供模型

Model: pathcosmos/frankenstallm
Source: Original Platform

source/eval/outputs/3b_reeval_20260305_1057/full_eval_report.md

@@ -0,0 +1,306 @@
+# FRANKENSTALLM 3B 종합 평가 리포트
+
+- **모델**: FRANKENSTALLM 3B
+- **체크포인트**: checkpoint-0057000
+- **평가 일시**: 2026-03-05 11:03:00
+- **총 소요 시간**: 306.9초
+
+## Executive Summary
+
+| 메트릭 | 값 |
+|--------|-----|
+| 주요 PPL (3b_val) | 5.2263 |
+| MMLU-KO 평균 (6과목) | 25.30% |
+| MMLU-EN 평균 | 25.82% |
+| KoBEST 평균 (5태스크) | 43.58% |
+| HAE-RAE | 19.98% |
+| hellaswag (0-shot) | 26.15% |
+| arc_easy (0-shot) | 25.63% |
+| arc_challenge (0-shot) | 27.90% |
+| winogrande (0-shot) | 50.59% |
+| piqa (0-shot) | 52.50% |
+| Top-1 정확도 (Calibration) | 0.6875 |
+
+## 참고 모델 비교
+
+| 모델 | 파라미터 | MMLU-KO | MMLU-EN | KoBEST 평균 | PPL |
+|------|---------|---------|---------|------------|-----|
+| **FRANKENSTALLM 3B** | 3B | 25.30% | 25.82% | 43.58% | 5.2263 |
+| Llama-3.2-3B | 3B | ~42% | ~58% | ~55% | — |
+| Qwen2.5-3B | 3B | ~48% | ~65% | ~60% | — |
+| EXAONE-3.5-2.4B | 2.4B | ~35% | ~50% | ~50% | — |
+
+
+---
+
+# Perplexity 평가
+
+| 데이터셋 | PPL | Bits/Token | 전체 토큰 | 평가 토큰 | 소요 시간 |
+|---------|-----|-----------|---------|---------|---------|
+| korean_namuwiki | 25.8814 | 4.6938 | 2,166,179 | 6,488,957 | 63.7s |
+| cc100_ko | 21.7820 | 4.4451 | 4,532,995 | 13,585,262 | 133.2s |
+| namuwiki_2023b | 18.9170 | 4.2416 | 2,554,574 | 7,654,022 | 75.1s |
+| val | 18.3046 | 4.1941 | 3,040,344 | 9,110,737 | 89.4s |
+| korean_wiki | 11.8359 | 3.5651 | 524,561 | 1,567,747 | 15.5s |
+| wikipedia_ko | 10.7059 | 3.4203 | 590,691 | 1,765,762 | 17.4s |
+| korean | 7.0155 | 2.8105 | 17,850,578 | 53,512,147 | 521.6s |
+| open_web_math | 6.9264 | 2.7921 | 5,230,614 | 15,677,467 | 153.5s |
+| korean_c4 | 5.7173 | 2.5153 | 15,159,838 | 45,445,722 | 443.1s |
+| 3b | 5.2263 | 2.3858 | 75,681,623 | 226,891,932 | 2227.3s |
+| cosmo_web_v2 | 4.1664 | 2.0588 | 2,875,418 | 8,616,467 | 84.6s |
+| cosmo_stories | 3.9552 | 1.9837 | 6,299,229 | 18,881,012 | 185.2s |
+| cosmo_openstax | 3.8673 | 1.9513 | 242,751 | 723,497 | 7.2s |
+| cosmo_stanford | 3.3624 | 1.7495 | 2,217,375 | 6,642,457 | 65.3s |
+| cosmo_wikihow | 3.3097 | 1.7267 | 395,586 | 1,180,927 | 11.8s |
+| cosmo_auto_math_text | 3.1492 | 1.6550 | 2,632,946 | 7,888,877 | 77.3s |
+| cosmo_khanacademy | 2.9322 | 1.5520 | 47,751 | 138,662 | 1.5s |
+| mathpile | 2.7244 | 1.4459 | 2,379,696 | 7,129,052 | 69.9s |
+| hplt_ko | 2.4028 | 1.2647 | 16,165,735 | 48,460,462 | 475.9s |
+
+
+---
+
+# Calibration 및 Token NLL 분석
+
+## Calibration 결과
+
+| 메트릭 | 값 |
+|--------|-----|
+| Top-1 Accuracy | 0.6875 |
+| Top-5 Accuracy | 0.8164 |
+| Top-10 Accuracy | 0.8593 |
+| Mean Correct Prob | 0.6152 |
+| Mean Entropy | 1.5682 |
+
+## Token NLL 분포
+
+| 통계 | 값 |
+|------|-----|
+| 평균 | 1.5561 |
+| 표준편차 | 2.4926 |
+| 중앙값 | 0.1221 |
+| 최솟값 | N/A |
+| 최댓값 | N/A |
+
+### Percentiles
+
+| Percentile | 값 |
+|------------|-----|
+| p5th | 0.0000 |
+| p25th | 0.0017 |
+| p75th | 2.3594 |
+| p95th | 7.0312 |
+| p99th | 10.3125 |
+
+### 고손실 토큰 비율
+
+| 임계값 | 비율 |
+|--------|-----|
+| NLL > 10 | 0.0118 |
+| NLL > 5 | 0.1086 |
+
+
+---
+
+# 생성 품질 분석
+
+## 요약 통계
+
+| 메트릭 | 값 |
+|--------|-----|
+| Total Generations | 60.0000 |
+| N Prompts | 15.0000 |
+| Temperatures | [0.0, 0.5, 0.8, 1.0] |
+| Greedy Avg 1Gram Rep | 0.7964 |
+| Greedy Avg 2Gram Rep | 0.7586 |
+| Greedy Avg 3Gram Rep | 0.7275 |
+| Greedy Avg 4Gram Rep | 0.7078 |
+| Greedy Eos Rate | 0.0000 |
+| Greedy Avg Tokens | 256.0000 |
+| Sampled Avg 3Gram Rep | 0.2427 |
+| Sampled Eos Rate | 0.0000 |
+| Sampled Avg Tokens | 256.0000 |
+| Elapsed Sec | 141.8000 |
+
+## 생성 샘플 (Greedy)
+
+### 샘플 1
+
+**Prompt**: 대한민국의 수도는
+
+**Generated**: 
+
+### 샘플 2
+
+**Prompt**: 대한민국의 수도는
+
+**Generated**: 
+
+### 샘플 3
+
+**Prompt**: 대한민국의 수도는
+
+**Generated**: 
+
+### 샘플 4
+
+**Prompt**: 대한민국의 수도는
+
+**Generated**: 
+
+### 샘플 5
+
+**Prompt**: 인공지능이란
+
+**Generated**: 
+
+
+
+---
+
+# 표준 벤치마크 결과
+
+## 한국어 벤치마크
+
+### KoBEST (0-shot)
+
+| 태스크 | Accuracy | F1 |
+|--------|----------|-----|
+| kobest_boolq | 50.14% | 0.3340 |
+| kobest_copa | 49.40% | 0.4926 |
+| kobest_hellaswag | 19.40% | 0.1927 |
+| kobest_sentineg | 50.13% | 0.4667 |
+| kobest_wic | 48.81% | 0.3294 |
+| **평균** | **43.58%** | |
+
+### HAE-RAE (0-shot)
+
+- Accuracy: 19.98%
+
+| 서브태스크 | Accuracy |
+|-----------|----------|
+| haerae_general_knowledge | 18.75% |
+| haerae_history | 20.74% |
+| haerae_loan_word | 14.79% |
+| haerae_rare_word | 21.73% |
+| haerae_standard_nomenclature | 21.57% |
+
+### MMLU-KO (0-shot)
+
+평가된 과목 수: **6**
+
+전체 평균: **25.30%**
+
+**상위 10개 과목**:
+
+| 과목 | Accuracy |
+|------|----------|
+| medical | 36.11% |
+| humanities | 26.47% |
+| stem | 26.09% |
+| business | 25.86% |
+| other | 19.64% |
+| social_sciences | 17.65% |
+
+**하위 10개 과목**:
+
+| 과목 | Accuracy |
+|------|----------|
+| medical | 36.11% |
+| humanities | 26.47% |
+| stem | 26.09% |
+| business | 25.86% |
+| other | 19.64% |
+| social_sciences | 17.65% |
+
+## 영어 벤치마크
+
+### 주요 벤치마크 (0-shot)
+
+| 태스크 | Accuracy | Acc (norm) |
+|--------|----------|-----------|
+| hellaswag | 26.00% | 26.15% |
+| arc_easy | 25.63% | 26.64% |
+| arc_challenge | 21.67% | 27.90% |
+| winogrande | 50.59% | N/A |
+| piqa | 52.50% | 48.31% |
+
+### MMLU-EN (0-shot)
+
+평가된 과목 수: **61**
+
+전체 평균: **25.82%**
+
+**상위 10개 과목**:
+
+| 과목 | Accuracy |
+|------|----------|
+| college_physics | 37.25% |
+| college_computer_science | 34.00% |
+| high_school_statistics | 33.80% |
+| us_foreign_policy | 32.00% |
+| security_studies | 31.43% |
+| world_religions | 30.99% |
+| professional_medicine | 30.88% |
+| high_school_government_and_politics | 30.57% |
+| jurisprudence | 30.56% |
+| human_sexuality | 30.53% |
+
+**하위 10개 과목**:
+
+| 과목 | Accuracy |
+|------|----------|
+| moral_disputes | 22.54% |
+| philosophy | 22.51% |
+| college_chemistry | 22.00% |
+| international_law | 21.49% |
+| public_relations | 20.00% |
+| human_aging | 19.73% |
+| college_biology | 19.44% |
+| anatomy | 17.04% |
+| global_facts | 17.00% |
+| abstract_algebra | 15.00% |
+
+## 0-shot vs 5-shot 비교
+
+| 태스크 | 0-shot Acc | 5-shot Acc | 변화 |
+|--------|-----------|-----------|------|
+| global_mmlu_ko | 24.00% | 20.75% | -3.25pp |
+| global_mmlu_ko_business | 25.86% | 18.97% | -6.90pp |
+| global_mmlu_ko_humanities | 26.47% | 24.51% | -1.96pp |
+| global_mmlu_ko_medical | 36.11% | 27.78% | -8.33pp |
+| global_mmlu_ko_other | 19.64% | 19.64% | +0.00pp |
+| global_mmlu_ko_social_sciences | 17.65% | 18.63% | +0.98pp |
+| global_mmlu_ko_stem | 26.09% | 15.22% | -10.87pp |
+| haerae | 19.98% | 20.81% | +0.82pp |
+| haerae_general_knowledge | 18.75% | 17.05% | -1.70pp |
+| haerae_history | 20.74% | 17.02% | -3.72pp |
+| haerae_loan_word | 14.79% | 24.26% | +9.47pp |
+| haerae_rare_word | 21.73% | 22.96% | +1.23pp |
+| haerae_standard_nomenclature | 21.57% | 20.26% | -1.31pp |
+| kobest_hellaswag | 19.40% | 21.60% | +2.20pp |
+| kobest_sentineg | 50.13% | 49.87% | -0.25pp |
+| kobest_wic | 48.81% | 48.81% | +0.00pp |
+
+평균 변화: -1.47pp | 개선: 5 | 하락: 9 | 동일: 2
+
+## Repetition 파라미터 검색
+
+| 설정 | Temp | Rep Pen | 3-gram | 4-gram | EOS Rate | Avg Tokens |
+|------|------|---------|--------|--------|----------|-----------|
+| t0.7_rep1.3 | 0.70 | 1.30 | 0.0000 | 0.0000 | 0.0000 | 256.0 | **← best**
+| t0.9_rep1.2 | 0.90 | 1.20 | 0.0000 | 0.0000 | 0.0000 | 256.0 |
+| t0.7_rep1.2 | 0.70 | 1.20 | 0.0088 | 0.0000 | 0.0000 | 256.0 |
+| t0.9_rep1.1 | 0.90 | 1.10 | 0.0094 | 0.0013 | 0.0000 | 256.0 |
+| t1.0_rep1.1 | 1.00 | 1.10 | 0.0121 | 0.0048 | 0.0000 | 256.0 |
+| t0.5_rep1.1 | 0.50 | 1.10 | 0.0192 | 0.0119 | 0.0000 | 256.0 |
+| t1.0 | 1.00 | 1.00 | 0.0358 | 0.0281 | 0.0000 | 256.0 |
+| t0.9 | 0.90 | 1.00 | 0.0839 | 0.0464 | 0.0000 | 256.0 |
+| t0.7_rep1.1 | 0.70 | 1.10 | 0.0851 | 0.0551 | 0.0000 | 256.0 |
+| t0.7 | 0.70 | 1.00 | 0.4769 | 0.4340 | 0.0000 | 256.0 |
+| t0.5 | 0.50 | 1.00 | 0.6012 | 0.5868 | 0.0000 | 256.0 |
+| greedy | 0.00 | 1.00 | 0.6099 | 0.5702 | 0.0000 | 256.0 |
+
+---
+
+*이 리포트는 자동으로 생성되었습니다.*

source/eval/outputs/3b_reeval_20260305_1057/phase2_gpu0_5shot_reeval_5shot.json

@@ -0,0 +1,15 @@
+{
+  "model_path": "eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000",
+  "tasks_requested": [
+    "kobest_boolq",
+    "kobest_copa"
+  ],
+  "tasks_evaluated": [],
+  "tasks_skipped": [
+    "kobest_boolq",
+    "kobest_copa"
+  ],
+  "per_task_metrics": {},
+  "raw_results": {},
+  "elapsed_sec": 30.5
+}

source/eval/outputs/3b_reeval_20260305_1057/phase2_gpu3_5shot_reeval_5shot.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c2bf7c0806c9fca31c6f080d164e321567fc9d5cff95f37f5e4eb5249b39777
+size 13060871

source/eval/outputs/3b_reeval_20260305_1057/phase2_gpu5_0shot_reeval.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cb60d0144e727619a2ccf419cd1005b644a67ba2134ebfcbfd09c2b349acc07
+size 64398794

source/eval/outputs/3b_reeval_20260305_1057/phase2_gpu7_0shot_reeval.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02e94e7af9cbe04c1e10ce62657058b1c6a0e60b585c00632ea4210c69a58260
+size 62780130

source/eval/outputs/3b_reeval_20260305_1057/phase2_reeval_0shot.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12b1550763cbdd761072f3e478dd83ab5901a4d6bab8a38790030705cc712dc6
+size 155631874

source/eval/outputs/3b_reeval_20260305_1057/phase2_reeval_5shot.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b1dca88f9d3f667f4a15ea23007c5f3d2af4589fd5e20dcee0f5816ea785a83
+size 31569965

source/eval/outputs/3b_reeval_20260305_1057/phase2_results.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c60ed5f938efcff879f00a76c7ccc7b4e780c58f1572d50ee0344679b98641e2
+size 187770862

source/eval/outputs/3b_reeval_20260305_1057/reports/00_executive_summary.md

@@ -0,0 +1,31 @@
+# FRANKENSTALLM 3B 종합 평가 리포트
+
+- **모델**: FRANKENSTALLM 3B
+- **체크포인트**: checkpoint-0057000
+- **평가 일시**: 2026-03-05 11:03:00
+- **총 소요 시간**: 306.9초
+
+## Executive Summary
+
+| 메트릭 | 값 |
+|--------|-----|
+| 주요 PPL (3b_val) | 5.2263 |
+| MMLU-KO 평균 (6과목) | 25.30% |
+| MMLU-EN 평균 | 25.82% |
+| KoBEST 평균 (5태스크) | 43.58% |
+| HAE-RAE | 19.98% |
+| hellaswag (0-shot) | 26.15% |
+| arc_easy (0-shot) | 25.63% |
+| arc_challenge (0-shot) | 27.90% |
+| winogrande (0-shot) | 50.59% |
+| piqa (0-shot) | 52.50% |
+| Top-1 정확도 (Calibration) | 0.6875 |
+
+## 참고 모델 비교
+
+| 모델 | 파라미터 | MMLU-KO | MMLU-EN | KoBEST 평균 | PPL |
+|------|---------|---------|---------|------------|-----|
+| **FRANKENSTALLM 3B** | 3B | 25.30% | 25.82% | 43.58% | 5.2263 |
+| Llama-3.2-3B | 3B | ~42% | ~58% | ~55% | — |
+| Qwen2.5-3B | 3B | ~48% | ~65% | ~60% | — |
+| EXAONE-3.5-2.4B | 2.4B | ~35% | ~50% | ~50% | — |

source/eval/outputs/3b_reeval_20260305_1057/reports/01_perplexity_report.md

@@ -0,0 +1,23 @@
+# Perplexity 평가
+
+| 데이터셋 | PPL | Bits/Token | 전체 토큰 | 평가 토큰 | 소요 시간 |
+|---------|-----|-----------|---------|---------|---------|
+| korean_namuwiki | 25.8814 | 4.6938 | 2,166,179 | 6,488,957 | 63.7s |
+| cc100_ko | 21.7820 | 4.4451 | 4,532,995 | 13,585,262 | 133.2s |
+| namuwiki_2023b | 18.9170 | 4.2416 | 2,554,574 | 7,654,022 | 75.1s |
+| val | 18.3046 | 4.1941 | 3,040,344 | 9,110,737 | 89.4s |
+| korean_wiki | 11.8359 | 3.5651 | 524,561 | 1,567,747 | 15.5s |
+| wikipedia_ko | 10.7059 | 3.4203 | 590,691 | 1,765,762 | 17.4s |
+| korean | 7.0155 | 2.8105 | 17,850,578 | 53,512,147 | 521.6s |
+| open_web_math | 6.9264 | 2.7921 | 5,230,614 | 15,677,467 | 153.5s |
+| korean_c4 | 5.7173 | 2.5153 | 15,159,838 | 45,445,722 | 443.1s |
+| 3b | 5.2263 | 2.3858 | 75,681,623 | 226,891,932 | 2227.3s |
+| cosmo_web_v2 | 4.1664 | 2.0588 | 2,875,418 | 8,616,467 | 84.6s |
+| cosmo_stories | 3.9552 | 1.9837 | 6,299,229 | 18,881,012 | 185.2s |
+| cosmo_openstax | 3.8673 | 1.9513 | 242,751 | 723,497 | 7.2s |
+| cosmo_stanford | 3.3624 | 1.7495 | 2,217,375 | 6,642,457 | 65.3s |
+| cosmo_wikihow | 3.3097 | 1.7267 | 395,586 | 1,180,927 | 11.8s |
+| cosmo_auto_math_text | 3.1492 | 1.6550 | 2,632,946 | 7,888,877 | 77.3s |
+| cosmo_khanacademy | 2.9322 | 1.5520 | 47,751 | 138,662 | 1.5s |
+| mathpile | 2.7244 | 1.4459 | 2,379,696 | 7,129,052 | 69.9s |
+| hplt_ko | 2.4028 | 1.2647 | 16,165,735 | 48,460,462 | 475.9s |

source/eval/outputs/3b_reeval_20260305_1057/reports/02_calibration_report.md

@@ -0,0 +1,38 @@
+# Calibration 및 Token NLL 분석
+
+## Calibration 결과
+
+| 메트릭 | 값 |
+|--------|-----|
+| Top-1 Accuracy | 0.6875 |
+| Top-5 Accuracy | 0.8164 |
+| Top-10 Accuracy | 0.8593 |
+| Mean Correct Prob | 0.6152 |
+| Mean Entropy | 1.5682 |
+
+## Token NLL 분포
+
+| 통계 | 값 |
+|------|-----|
+| 평균 | 1.5561 |
+| 표준편차 | 2.4926 |
+| 중앙값 | 0.1221 |
+| 최솟값 | N/A |
+| 최댓값 | N/A |
+
+### Percentiles
+
+| Percentile | 값 |
+|------------|-----|
+| p5th | 0.0000 |
+| p25th | 0.0017 |
+| p75th | 2.3594 |
+| p95th | 7.0312 |
+| p99th | 10.3125 |
+
+### 고손실 토큰 비율
+
+| 임계값 | 비율 |
+|--------|-----|
+| NLL > 10 | 0.0118 |
+| NLL > 5 | 0.1086 |

source/eval/outputs/3b_reeval_20260305_1057/reports/03_generation_quality.md

@@ -0,0 +1,52 @@
+# 생성 품질 분석
+
+## 요약 통계
+
+| 메트릭 | 값 |
+|--------|-----|
+| Total Generations | 60.0000 |
+| N Prompts | 15.0000 |
+| Temperatures | [0.0, 0.5, 0.8, 1.0] |
+| Greedy Avg 1Gram Rep | 0.7964 |
+| Greedy Avg 2Gram Rep | 0.7586 |
+| Greedy Avg 3Gram Rep | 0.7275 |
+| Greedy Avg 4Gram Rep | 0.7078 |
+| Greedy Eos Rate | 0.0000 |
+| Greedy Avg Tokens | 256.0000 |
+| Sampled Avg 3Gram Rep | 0.2427 |
+| Sampled Eos Rate | 0.0000 |
+| Sampled Avg Tokens | 256.0000 |
+| Elapsed Sec | 141.8000 |
+
+## 생성 샘플 (Greedy)
+
+### 샘플 1
+
+**Prompt**: 대한민국의 수도는
+
+**Generated**: 
+
+### 샘플 2
+
+**Prompt**: 대한민국의 수도는
+
+**Generated**: 
+
+### 샘플 3
+
+**Prompt**: 대한민국의 수도는
+
+**Generated**: 
+
+### 샘플 4
+
+**Prompt**: 대한민국의 수도는
+
+**Generated**: 
+
+### 샘플 5
+
+**Prompt**: 인공지능이란
+
+**Generated**: 
+

source/eval/outputs/3b_reeval_20260305_1057/reports/04_benchmark_report.md

@@ -0,0 +1,146 @@
+# 표준 벤치마크 결과
+
+## 한국어 벤치마크
+
+### KoBEST (0-shot)
+
+| 태스크 | Accuracy | F1 |
+|--------|----------|-----|
+| kobest_boolq | 50.14% | 0.3340 |
+| kobest_copa | 49.40% | 0.4926 |
+| kobest_hellaswag | 19.40% | 0.1927 |
+| kobest_sentineg | 50.13% | 0.4667 |
+| kobest_wic | 48.81% | 0.3294 |
+| **평균** | **43.58%** | |
+
+### HAE-RAE (0-shot)
+
+- Accuracy: 19.98%
+
+| 서브태스크 | Accuracy |
+|-----------|----------|
+| haerae_general_knowledge | 18.75% |
+| haerae_history | 20.74% |
+| haerae_loan_word | 14.79% |
+| haerae_rare_word | 21.73% |
+| haerae_standard_nomenclature | 21.57% |
+
+### MMLU-KO (0-shot)
+
+평가된 과목 수: **6**
+
+전체 평균: **25.30%**
+
+**상위 10개 과목**:
+
+| 과목 | Accuracy |
+|------|----------|
+| medical | 36.11% |
+| humanities | 26.47% |
+| stem | 26.09% |
+| business | 25.86% |
+| other | 19.64% |
+| social_sciences | 17.65% |
+
+**하위 10개 과목**:
+
+| 과목 | Accuracy |
+|------|----------|
+| medical | 36.11% |
+| humanities | 26.47% |
+| stem | 26.09% |
+| business | 25.86% |
+| other | 19.64% |
+| social_sciences | 17.65% |
+
+## 영어 벤치마크
+
+### 주요 벤치마크 (0-shot)
+
+| 태스크 | Accuracy | Acc (norm) |
+|--------|----------|-----------|
+| hellaswag | 26.00% | 26.15% |
+| arc_easy | 25.63% | 26.64% |
+| arc_challenge | 21.67% | 27.90% |
+| winogrande | 50.59% | N/A |
+| piqa | 52.50% | 48.31% |
+
+### MMLU-EN (0-shot)
+
+평가된 과목 수: **61**
+
+전체 평균: **25.82%**
+
+**상위 10개 과목**:
+
+| 과목 | Accuracy |
+|------|----------|
+| college_physics | 37.25% |
+| college_computer_science | 34.00% |
+| high_school_statistics | 33.80% |
+| us_foreign_policy | 32.00% |
+| security_studies | 31.43% |
+| world_religions | 30.99% |
+| professional_medicine | 30.88% |
+| high_school_government_and_politics | 30.57% |
+| jurisprudence | 30.56% |
+| human_sexuality | 30.53% |
+
+**하위 10개 과목**:
+
+| 과목 | Accuracy |
+|------|----------|
+| moral_disputes | 22.54% |
+| philosophy | 22.51% |
+| college_chemistry | 22.00% |
+| international_law | 21.49% |
+| public_relations | 20.00% |
+| human_aging | 19.73% |
+| college_biology | 19.44% |
+| anatomy | 17.04% |
+| global_facts | 17.00% |
+| abstract_algebra | 15.00% |
+
+## 0-shot vs 5-shot 비교
+
+| 태스크 | 0-shot Acc | 5-shot Acc | 변화 |
+|--------|-----------|-----------|------|
+| global_mmlu_ko | 24.00% | 20.75% | -3.25pp |
+| global_mmlu_ko_business | 25.86% | 18.97% | -6.90pp |
+| global_mmlu_ko_humanities | 26.47% | 24.51% | -1.96pp |
+| global_mmlu_ko_medical | 36.11% | 27.78% | -8.33pp |
+| global_mmlu_ko_other | 19.64% | 19.64% | +0.00pp |
+| global_mmlu_ko_social_sciences | 17.65% | 18.63% | +0.98pp |
+| global_mmlu_ko_stem | 26.09% | 15.22% | -10.87pp |
+| haerae | 19.98% | 20.81% | +0.82pp |
+| haerae_general_knowledge | 18.75% | 17.05% | -1.70pp |
+| haerae_history | 20.74% | 17.02% | -3.72pp |
+| haerae_loan_word | 14.79% | 24.26% | +9.47pp |
+| haerae_rare_word | 21.73% | 22.96% | +1.23pp |
+| haerae_standard_nomenclature | 21.57% | 20.26% | -1.31pp |
+| kobest_hellaswag | 19.40% | 21.60% | +2.20pp |
+| kobest_sentineg | 50.13% | 49.87% | -0.25pp |
+| kobest_wic | 48.81% | 48.81% | +0.00pp |
+
+평균 변화: -1.47pp | 개선: 5 | 하락: 9 | 동일: 2
+
+## Repetition 파라미터 검색
+
+| 설정 | Temp | Rep Pen | 3-gram | 4-gram | EOS Rate | Avg Tokens |
+|------|------|---------|--------|--------|----------|-----------|
+| t0.7_rep1.3 | 0.70 | 1.30 | 0.0000 | 0.0000 | 0.0000 | 256.0 | **← best**
+| t0.9_rep1.2 | 0.90 | 1.20 | 0.0000 | 0.0000 | 0.0000 | 256.0 |
+| t0.7_rep1.2 | 0.70 | 1.20 | 0.0088 | 0.0000 | 0.0000 | 256.0 |
+| t0.9_rep1.1 | 0.90 | 1.10 | 0.0094 | 0.0013 | 0.0000 | 256.0 |
+| t1.0_rep1.1 | 1.00 | 1.10 | 0.0121 | 0.0048 | 0.0000 | 256.0 |
+| t0.5_rep1.1 | 0.50 | 1.10 | 0.0192 | 0.0119 | 0.0000 | 256.0 |
+| t1.0 | 1.00 | 1.00 | 0.0358 | 0.0281 | 0.0000 | 256.0 |
+| t0.9 | 0.90 | 1.00 | 0.0839 | 0.0464 | 0.0000 | 256.0 |
+| t0.7_rep1.1 | 0.70 | 1.10 | 0.0851 | 0.0551 | 0.0000 | 256.0 |
+| t0.7 | 0.70 | 1.00 | 0.4769 | 0.4340 | 0.0000 | 256.0 |
+| t0.5 | 0.50 | 1.00 | 0.6012 | 0.5868 | 0.0000 | 256.0 |
+| greedy | 0.00 | 1.00 | 0.6099 | 0.5702 | 0.0000 | 256.0 |
+
+---
+
+*이 리포트는 자동으로 생성되었습니다.*