kikiyaa/Mistral-7B-dpo-full-tuned
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
Mistral-7B-dpo-full-tuned/debug.log

35 lines
154 KiB
Plaintext
Raw Permalink Normal View History

初始化项目，由ModelHub XC社区提供模型 Model: kikiyaa/Mistral-7B-dpo-full-tuned Source: Original Platform
2026-06-13 17:10:39 +08:00
tokenizer_config.json: 0%| | 0.00/996 [00:00<?, ?B/s] tokenizer_config.json: 100%|████████████████████████████████████████████| 996/996 [00:00<00:00, 1.57MB/s]
tokenizer.model: 0%| | 0.00/493k [00:00<?, ?B/s] tokenizer.model: 0%| | 0.00/493k [00:00<?, ?B/s] tokenizer.model: 0%| | 0.00/493k [00:00<?, ?B/s] tokenizer.model: 0%| | 0.00/493k [00:00<?, ?B/s] tokenizer.model: 0%| | 0.00/493k [00:00<?, ?B/s] tokenizer.model: 0%| | 0.00/493k [00:01<?, ?B/s] tokenizer.model: 0%| | 0.00/493k [00:01<?, ?B/s] tokenizer.model: 0%| | 0.00/493k [00:01<?, ?B/s] tokenizer.model: 0%| | 0.00/493k [00:01<?, ?B/s] tokenizer.model: 100%|████████████████████████████████████████████████| 493k/493k [00:01<00:00, 2.47MB/s] tokenizer.model: 100%|█████████████████████████████████████████████████| 493k/493k [00:01<00:00, 272kB/s]
tokenizer.json: 0.00B [00:00, ?B/s] tokenizer.json: 1.80MB [00:00, 9.80MB/s] tokenizer.json: 1.80MB [00:00, 9.66MB/s]
special_tokens_map.json: 0%| | 0.00/414 [00:00<?, ?B/s] special_tokens_map.json: 100%|███████████████████████████████████████████| 414/414 [00:00<00:00, 763kB/s]
Downloading (incomplete total...): 0.00B [00:00, ?B/s]
Fetching 0 files: 0it [00:00, ?it/s] Fetching 0 files: 0it [00:00, ?it/s]
Download complete: : 0.00B [00:00, ?B/s]
README.md: 0%| | 0.00/348 [00:00<?, ?B/s] README.md: 100%|████████████████████████████████████████████████████████| 348/348 [00:00<00:00, 1.35MB/s]
Download complete: : 0.00B [00:01, ?B/s]
data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:02<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:02<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/4.77M [00:02<?, ?B/s] data/train-00000-of-00001.parquet: 100%|████████████████████████████| 4.77M/4.77M [00:02<00:00, 23.8MB/s] data/train-00000-of-00001.parquet: 100%|████████████████████████████| 4.77M/4.77M [00:02<00:00, 1.83MB/s]
Generating train split: 0%| | 0/5684 [00:00<?, ? examples/s] Generating train split: 100%|█████████████████████████████| 5684/5684 [00:00<00:00, 125797.03 examples/s]
Downloading (incomplete total...): 0.00B [00:00, ?B/s]
Fetching 0 files: 0it [00:00, ?it/s] Fetching 0 files: 0it [00:00, ?it/s]
Download complete: : 0.00B [00:00, ?B/s]
README.md: 0%| | 0.00/348 [00:00<?, ?B/s] README.md: 100%|█████████████████████████████████████████████████████████| 348/348 [00:00<00:00, 802kB/s]
Download complete: : 0.00B [00:02, ?B/s]
data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:00<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:01<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:02<?, ?B/s] data/train-00000-of-00001.parquet: 0%| | 0.00/2.49M [00:02<?, ?B/s] data/train-00000-of-00001.parquet: 100%|████████████████████████████| 2.49M/2.49M [00:02<00:00, 12.4MB/s] data/train-00000-of-00001.parquet: 100%|████████████████████████████| 2.49M/2.49M [00:02<00:00, 1.03MB/s]
Generating train split: 0%| | 0/6555 [00:00<?, ? examples/s] Generating train split: 100%|█████████████████████████████| 6555/6555 [00:00<00:00, 249865.16 examples/s]
Mapping RL Dataset (num_proc=96): 0%| | 0/5684 [00:00<?, ? examples/s] Mapping RL Dataset (num_proc=96): 1%|▎ | 60/5684 [00:01<02:30, 37.44 examples/s] Mapping RL Dataset (num_proc=96): 2%|▌ | 120/5684 [00:01<01:21, 67.97 examples/s] Mapping RL Dataset (num_proc=96): 3%|▊ | 180/5684 [00:02<00:56, 97.52 examples/s] Mapping RL Dataset (num_proc=96): 4%|▉ | 240/5684 [00:02<00:47, 115.61 examples/s] Mapping RL Dataset (num_proc=96): 5%|█▏ | 300/5684 [00:03<00:42, 125.99 examples/s] Mapping RL Dataset (num_proc=96): 6%|█▍ | 360/5684 [00:03<00:36, 144.78 examples/s] Mapping RL Dataset (num_proc=96): 7%|█▋ | 420/5684 [00:03<00:34, 150.53 examples/s] Mapping RL Dataset (num_proc=96): 8%|█▉ | 480/5684 [00:04<00:32, 161.11 examples/s] Mapping RL Dataset (num_proc=96): 10%|██▏ | 540/5684 [00:04<00:31, 163.12 examples/s] Mapping RL Dataset (num_proc=96): 11%|██▍ | 600/5684 [00:04<00:29, 170.18 examples/s] Mapping RL Dataset (num_proc=96): 12%|██▋ | 660/5684 [00:05<00:29, 170.06 examples/s] Mapping RL Dataset (num_proc=96): 13%|██▉ | 720/5684 [00:05<00:27, 178.20 examples/s] Mapping RL Dataset (num_proc=96): 14%|███▏ | 780/5684 [00:05<00:28, 169.61 examples/s] Mapping RL Dataset (num_proc=96): 15%|███▍ | 840/5684 [00:06<00:28, 167.61 examples/s] Mapping RL Dataset (num_proc=96): 16%|███▋ | 900/5684 [00:06<00:27, 173.75 examples/s] Mapping RL Dataset (num_proc=96): 17%|███▉ | 960/5684 [00:06<00:25, 182.04 examples/s] Mapping RL Dataset (num_proc=96): 18%|███▉ | 1020/5684 [00:07<00:27, 169.76 examples/s] Mapping RL Dataset (num_proc=96): 19%|████▏ | 1080/5684 [00:07<00:26, 172.79 examples/s] Mapping RL Dataset (num_proc=96): 20%|████▍ | 1140/5684 [00:07<00:26, 173.31 examples/s] Mapping RL Dataset (num_proc=96): 21%|████▋ | 1200/5684 [00:08<00:25, 174.30 examples/s] Mapping RL Dataset (num_proc=96): 22%|████▊ | 1259/5684 [00:08<00:25, 174.87 examples/s] Mapping RL Dataset (num_proc=96): 23%|█████ | 1318/5684 [00:08<00:25, 174.61 examples/s] Mapping RL Dataset (num_proc=96): 24%|█████▎ | 1377/5684 [00:09<00:24, 173.40 examples/s] Mapping RL Dataset (num_proc=96): 25%|█████▌ | 1436/5684 [00:09<00:24, 173.13 examples/s] Mapping RL Dataset (num_proc=96): 26%|█████▊ | 1495/5684 [00:09<00:24, 172.20 examples/s] Mapping RL Dataset (num_proc=96): 27%|██████ | 1554/5684 [00:10<00:24, 170.07 examples/s] Mapping RL Dataset (num_proc=96): 28%|██████▏ | 1613/5684 [00:10<00:23, 172.30 examples/s] Mapping RL Dataset (num_proc=96): 29%|██████▍ | 1672/5684 [00:10<00:23, 172.46 examples/s] Mapping RL Dataset (num_proc=96): 30%|██████▋ | 1731/5684 [00:11<00:23, 169.17 examples/s] Mapping RL Dataset (num_proc=96): 31%|██████▉ | 1790/5684 [00:11<00:22, 173.66 examples/s] Mapping RL Dataset (num_proc=96): 33%|███████▏ | 1849/5684 [00:11<00:22, 172.97 examples/s] Mapping RL Dataset (num_proc=96): 34%|███████▍ | 1908/5684 [00:12<00:22, 168.41 examples/s] Mapping RL Dataset (num_proc=96): 35%|███████▌ | 1967/5684 [00:12<00:21, 173.78 examples/s] Mapping RL Dataset (num_proc=96): 36%|███████▊ | 2026/5684 [00:12<00:21, 170.24 examples/s] Mapping RL Dataset (num_proc=96): 37%|████████ |
Dropping Long Sequences (num_proc=96): 0%| | 0/5684 [00:00<?, ? examples/s] Dropping Long Sequences (num_proc=96): 1%|▏ | 60/5684 [00:01<02:46, 33.86 examples/s] Dropping Long Sequences (num_proc=96): 2%|▍ | 120/5684 [00:02<01:26, 64.03 examples/s] Dropping Long Sequences (num_proc=96): 3%|▌ | 180/5684 [00:02<01:00, 90.74 examples/s] Dropping Long Sequences (num_proc=96): 4%|▊ | 240/5684 [00:02<00:48, 111.84 examples/s] Dropping Long Sequences (num_proc=96): 5%|▉ | 300/5684 [00:03<00:41, 130.25 examples/s] Dropping Long Sequences (num_proc=96): 6%|█▏ | 360/5684 [00:03<00:38, 139.46 examples/s] Dropping Long Sequences (num_proc=96): 7%|█▎ | 420/5684 [00:03<00:36, 143.45 examples/s] Dropping Long Sequences (num_proc=96): 8%|█▌ | 480/5684 [00:04<00:34, 151.62 examples/s] Dropping Long Sequences (num_proc=96): 10%|█▋ | 540/5684 [00:04<00:31, 161.14 examples/s] Dropping Long Sequences (num_proc=96): 11%|█▉ | 600/5684 [00:04<00:32, 157.70 examples/s] Dropping Long Sequences (num_proc=96): 12%|██ | 660/5684 [00:05<00:31, 161.50 examples/s] Dropping Long Sequences (num_proc=96): 13%|██▎ | 720/5684 [00:05<00:29, 166.83 examples/s] Dropping Long Sequences (num_proc=96): 14%|██▍ | 780/5684 [00:05<00:29, 168.52 examples/s] Dropping Long Sequences (num_proc=96): 15%|██▋ | 840/5684 [00:06<00:28, 170.74 examples/s] Dropping Long Sequences (num_proc=96): 16%|██▊ | 900/5684 [00:06<00:28, 169.08 examples/s] Dropping Long Sequences (num_proc=96): 17%|███ | 960/5684 [00:07<00:27, 172.79 examples/s] Dropping Long Sequences (num_proc=96): 18%|███ | 1020/5684 [00:07<00:26, 173.18 examples/s] Dropping Long Sequences (num_proc=96): 19%|███▏ | 1080/5684 [00:07<00:26, 173.66 examples/s] Dropping Long Sequences (num_proc=96): 20%|███▍ | 1140/5684 [00:08<00:26, 172.76 examples/s] Dropping Long Sequences (num_proc=96): 21%|███▌ | 1200/5684 [00:08<00:25, 172.75 examples/s] Dropping Long Sequences (num_proc=96): 22%|███▊ | 1259/5684 [00:08<00:25, 171.57 examples/s] Dropping Long Sequences (num_proc=96): 23%|███▉ | 1318/5684 [00:09<00:25, 171.13 examples/s] Dropping Long Sequences (num_proc=96): 24%|████ | 1377/5684 [00:09<00:24, 172.65 examples/s] Dropping Long Sequences (num_proc=96): 25%|████▎ | 1436/5684 [00:09<00:24, 170.40 examples/s] Dropping Long Sequences (num_proc=96): 26%|████▍ | 1495/5684 [00:10<00:24, 171.88 examples/s] Dropping Long Sequences (num_proc=96): 27%|████▋ | 1554/5684 [00:10<00:23, 172.38 examples/s] Dropping Long Sequences (num_proc=96): 28%|████▊ | 1613/5684 [00:10<00:23, 172.23 examples/s] Dropping Long Sequences (num_proc=96): 29%|█████ | 1672/5684 [00:11<00:23, 172.73 examples/s] Dropping Long Sequences (num_proc=96): 30%|█████▏ | 1731/5684 [00:11<00:23, 171.65 examples/s] Dropping Long Sequences (num_proc=96): 31%|█████▎ | 1790/5684 [00:11<00:22, 175.28 examples/s] Dropping Long Sequences (num_proc=96): 33%|█████▌ | 1849/5684 [00:12<00:22, 169.79 examples/s] Dropping Long Sequences (num_proc=96): 34%|█████▋ | 1908/5684 [00:12<00:21, 173.50 examples/s] Dropping Long Sequences (num_proc=96): 35%|█████▉ | 1967/5684 [00:12<00:21, 171.01 examples/s] Dropping Long Sequences (num_proc=96): 36%|██████ | 2026/5684 [00:13<00:21, 170.56 examples/s] Dropping Long Sequences (num_proc=96): 37%|██████▏ | 2085/5684 [00:13<00:21, 169.67 examples/s] Dropping Long Sequences
Mapping RL Dataset (num_proc=96): 0%| | 0/6555 [00:00<?, ? examples/s] Mapping RL Dataset (num_proc=96): 1%|▎ | 69/6555 [00:01<02:37, 41.12 examples/s] Mapping RL Dataset (num_proc=96): 2%|▌ | 138/6555 [00:02<01:24, 76.05 examples/s] Mapping RL Dataset (num_proc=96): 3%|▋ | 207/6555 [00:02<00:59, 106.95 examples/s] Mapping RL Dataset (num_proc=96): 4%|▉ | 276/6555 [00:02<00:48, 129.59 examples/s] Mapping RL Dataset (num_proc=96): 5%|█▏ | 345/6555 [00:03<00:41, 151.03 examples/s] Mapping RL Dataset (num_proc=96): 6%|█▍ | 414/6555 [00:03<00:37, 162.75 examples/s] Mapping RL Dataset (num_proc=96): 7%|█▋ | 483/6555 [00:03<00:34, 174.50 examples/s] Mapping RL Dataset (num_proc=96): 8%|█▉ | 552/6555 [00:04<00:33, 181.82 examples/s] Mapping RL Dataset (num_proc=96): 9%|██▏ | 621/6555 [00:04<00:31, 186.10 examples/s] Mapping RL Dataset (num_proc=96): 11%|██▍ | 690/6555 [00:04<00:31, 188.16 examples/s] Mapping RL Dataset (num_proc=96): 12%|██▋ | 759/6555 [00:05<00:31, 181.98 examples/s] Mapping RL Dataset (num_proc=96): 13%|██▉ | 828/6555 [00:05<00:30, 188.17 examples/s] Mapping RL Dataset (num_proc=96): 14%|███▏ | 897/6555 [00:05<00:30, 187.80 examples/s] Mapping RL Dataset (num_proc=96): 15%|███▍ | 966/6555 [00:06<00:28, 195.52 examples/s] Mapping RL Dataset (num_proc=96): 16%|███▍ | 1035/6555 [00:06<00:28, 191.61 examples/s] Mapping RL Dataset (num_proc=96): 17%|███▋ | 1104/6555 [00:07<00:29, 184.66 examples/s] Mapping RL Dataset (num_proc=96): 18%|███▉ | 1173/6555 [00:07<00:27, 198.03 examples/s] Mapping RL Dataset (num_proc=96): 19%|████▏ | 1242/6555 [00:07<00:26, 204.10 examples/s] Mapping RL Dataset (num_proc=96): 20%|████▍ | 1311/6555 [00:07<00:26, 199.87 examples/s] Mapping RL Dataset (num_proc=96): 21%|████▋ | 1380/6555 [00:08<00:25, 202.44 examples/s] Mapping RL Dataset (num_proc=96): 22%|████▊ | 1449/6555 [00:08<00:26, 192.72 examples/s] Mapping RL Dataset (num_proc=96): 23%|█████ | 1518/6555 [00:09<00:25, 194.67 examples/s] Mapping RL Dataset (num_proc=96): 24%|█████▎ | 1587/6555 [00:09<00:25, 195.62 examples/s] Mapping RL Dataset (num_proc=96): 25%|█████▌ | 1656/6555 [00:09<00:24, 196.89 examples/s] Mapping RL Dataset (num_proc=96): 26%|█████▊ | 1725/6555 [00:10<00:24, 197.90 examples/s] Mapping RL Dataset (num_proc=96): 27%|██████ | 1794/6555 [00:10<00:24, 197.79 examples/s] Mapping RL Dataset (num_proc=96): 28%|██████▎ | 1863/6555 [00:10<00:22, 205.36 examples/s] Mapping RL Dataset (num_proc=96): 29%|██████▍ | 1931/6555 [00:11<00:23, 196.52 examples/s] Mapping RL Dataset (num_proc=96): 30%|██████▋ | 1999/6555 [00:11<00:23, 197.33 examples/s] Mapping RL Dataset (num_proc=96): 32%|██████▉ | 2067/6555 [00:11<00:23, 192.50 examples/s] Mapping RL Dataset (num_proc=96): 33%|███████▏ | 2135/6555 [00:12<00:22, 197.13 examples/s] Mapping RL Dataset (num_proc=96): 34%|███████▍ | 2203/6555 [00:12<00:21, 200.05 examples/s] Mapping RL Dataset (num_proc=96): 35%|███████▌ | 2271/6555 [00:12<00:21, 195.06 examples/s] Mapping RL Dataset (num_proc=96): 36%|███████▊ | 2339/6555 [00:13<00:21, 194.67 examples/s] Mapping RL Dataset (num_proc=96): 37%|████████ |
Dropping Long Sequences (num_proc=96): 0%| | 0/6555 [00:00<?, ? examples/s] Dropping Long Sequences (num_proc=96): 1%|▏ | 69/6555 [00:01<02:45, 39.17 examples/s] Dropping Long Sequences (num_proc=96): 2%|▍ | 138/6555 [00:02<01:26, 74.09 examples/s] Dropping Long Sequences (num_proc=96): 3%|▌ | 207/6555 [00:02<01:01, 103.94 examples/s] Dropping Long Sequences (num_proc=96): 4%|▊ | 276/6555 [00:02<00:50, 125.26 examples/s] Dropping Long Sequences (num_proc=96): 5%|▉ | 345/6555 [00:03<00:41, 147.91 examples/s] Dropping Long Sequences (num_proc=96): 6%|█▏ | 414/6555 [00:03<00:37, 163.14 examples/s] Dropping Long Sequences (num_proc=96): 7%|█▎ | 483/6555 [00:03<00:35, 172.51 examples/s] Dropping Long Sequences (num_proc=96): 8%|█▌ | 552/6555 [00:04<00:33, 177.56 examples/s] Dropping Long Sequences (num_proc=96): 9%|█▋ | 621/6555 [00:04<00:31, 188.45 examples/s] Dropping Long Sequences (num_proc=96): 11%|█▉ | 690/6555 [00:04<00:30, 192.88 examples/s] Dropping Long Sequences (num_proc=96): 12%|██ | 759/6555 [00:05<00:30, 192.79 examples/s] Dropping Long Sequences (num_proc=96): 13%|██▎ | 828/6555 [00:05<00:29, 194.68 examples/s] Dropping Long Sequences (num_proc=96): 14%|██▍ | 897/6555 [00:05<00:28, 196.38 examples/s] Dropping Long Sequences (num_proc=96): 15%|██▋ | 966/6555 [00:06<00:28, 195.45 examples/s] Dropping Long Sequences (num_proc=96): 16%|██▋ | 1035/6555 [00:06<00:28, 196.46 examples/s] Dropping Long Sequences (num_proc=96): 17%|██▊ | 1104/6555 [00:06<00:27, 199.43 examples/s] Dropping Long Sequences (num_proc=96): 18%|███ | 1173/6555 [00:07<00:27, 198.62 examples/s] Dropping Long Sequences (num_proc=96): 19%|███▏ | 1242/6555 [00:07<00:27, 192.34 examples/s] Dropping Long Sequences (num_proc=96): 20%|███▍ | 1311/6555 [00:07<00:26, 200.26 examples/s] Dropping Long Sequences (num_proc=96): 21%|███▌ | 1380/6555 [00:08<00:25, 200.50 examples/s] Dropping Long Sequences (num_proc=96): 22%|███▊ | 1449/6555 [00:08<00:25, 204.19 examples/s] Dropping Long Sequences (num_proc=96): 23%|███▉ | 1518/6555 [00:09<00:25, 193.81 examples/s] Dropping Long Sequences (num_proc=96): 24%|████ | 1587/6555 [00:09<00:25, 194.13 examples/s] Dropping Long Sequences (num_proc=96): 25%|████▎ | 1656/6555 [00:09<00:25, 195.38 examples/s] Dropping Long Sequences (num_proc=96): 26%|████▍ | 1725/6555 [00:10<00:24, 194.40 examples/s] Dropping Long Sequences (num_proc=96): 27%|████▋ | 1794/6555 [00:10<00:24, 196.80 examples/s] Dropping Long Sequences (num_proc=96): 28%|████▊ | 1863/6555 [00:10<00:23, 197.01 examples/s] Dropping Long Sequences (num_proc=96): 29%|█████ | 1931/6555 [00:11<00:23, 195.98 examples/s] Dropping Long Sequences (num_proc=96): 30%|█████▏ | 1999/6555 [00:11<00:23, 193.20 examples/s] Dropping Long Sequences (num_proc=96): 32%|█████▎ | 2067/6555 [00:11<00:22, 197.28 examples/s] Dropping Long Sequences (num_proc=96): 33%|█████▌ | 2135/6555 [00:12<00:22, 195.67 examples/s] Dropping Long Sequences (num_proc=96): 34%|█████▋ | 2203/6555 [00:12<00:22, 196.21 examples/s] Dropping Long Sequences (num_proc=96): 35%|█████▉ | 2271/6555 [00:12<00:21, 195.26 examples/s] Dropping Long Sequences (num_proc=96): 36%|██████ | 2339/6555 [00:13<00:21, 195.91 examples/s] Dropping Long Sequences (num_proc=96): 37%|██████▏ | 2407/6555 [00:13<00:21, 196.10 examples/s] Dropping Long Sequences
Saving the dataset (0/47 shards): 0%| | 0/12239 [00:00<?, ? examples/s] Saving the dataset (0/47 shards): 2%|▍ | 261/12239 [00:02<02:00, 99.39 examples/s] Saving the dataset (1/47 shards): 2%|▍ | 261/12239 [00:02<02:00, 99.39 examples/s] Saving the dataset (2/47 shards): 4%|▉ | 522/12239 [00:02<01:57, 99.39 examples/s] Saving the dataset (3/47 shards): 6%|█▍ | 783/12239 [00:02<01:55, 99.39 examples/s] Saving the dataset (4/47 shards): 9%|█▉ | 1044/12239 [00:02<01:52, 99.39 examples/s] Saving the dataset (5/47 shards): 11%|██▎ | 1305/12239 [00:02<01:50, 99.39 examples/s] Saving the dataset (6/47 shards): 13%|██▊ | 1566/12239 [00:02<01:47, 99.39 examples/s] Saving the dataset (7/47 shards): 15%|███▎ | 1827/12239 [00:02<01:44, 99.39 examples/s] Saving the dataset (8/47 shards): 17%|███▊ | 2088/12239 [00:02<01:42, 99.39 examples/s] Saving the dataset (9/47 shards): 19%|████▏ | 2349/12239 [00:02<01:39, 99.39 examples/s] Saving the dataset (10/47 shards): 21%|████▍ | 2610/12239 [00:02<01:36, 99.39 examples/s] Saving the dataset (11/47 shards): 23%|████▉ | 2871/12239 [00:02<01:34, 99.39 examples/s] Saving the dataset (12/47 shards): 26%|█████▎ | 3132/12239 [00:02<01:31, 99.39 examples/s] Saving the dataset (13/47 shards): 28%|█████▊ | 3393/12239 [00:02<01:28, 99.39 examples/s] Saving the dataset (14/47 shards): 30%|██████▎ | 3654/12239 [00:02<01:26, 99.39 examples/s] Saving the dataset (15/47 shards): 32%|██████▋ | 3915/12239 [00:02<01:23, 99.39 examples/s] Saving the dataset (16/47 shards): 34%|███████▏ | 4176/12239 [00:02<01:21, 99.39 examples/s] Saving the dataset (17/47 shards): 36%|███████▌ | 4437/12239 [00:02<01:18, 99.39 examples/s] Saving the dataset (18/47 shards): 38%|████████ | 4698/12239 [00:02<01:15, 99.39 examples/s] Saving the dataset (19/47 shards): 41%|████████▌ | 4959/12239 [00:02<01:13, 99.39 examples/s] Saving the dataset (20/47 shards): 43%|████████▉ | 5219/12239 [00:02<01:10, 99.39 examples/s] Saving the dataset (21/47 shards): 45%|█████████▍ | 5479/12239 [00:02<01:08, 99.39 examples/s] Saving the dataset (22/47 shards): 47%|█████████▊ | 5739/12239 [00:02<01:05, 99.39 examples/s] Saving the dataset (23/47 shards): 49%|██████████▎ | 5999/12239 [00:02<01:02, 99.39 examples/s] Saving the dataset (24/47 shards): 51%|██████████▋ | 6259/12239 [00:02<01:00, 99.39 examples/s] Saving the dataset (25/47 shards): 53%|███████████▏ | 6519/12239 [00:02<00:57, 99.39 examples/s] Saving the dataset (26/47 shards): 55%|███████████▋ | 6779/12239 [00:02<00:54, 99.39 examples/s] Saving the dataset (27/47 shards): 58%|████████████ | 7039/12239 [00:02<00:52, 99.39 examples/s] Saving the dataset (28/47 shards): 60%|████████████▌ | 7299/12239 [00:02<00:49, 99.39 examples/s] Saving the dataset (29/47 shards): 62%|████████████▉ | 7559/12239 [00:02<00:47, 99.39 examples/s] Saving the dataset (30/47 shards): 64%|█████████████▍ | 7819/12239 [00:02<00:44, 99.39 examples/s] Saving the dataset (31/47 shards): 66%|█████████████▊ | 8079/12239 [00:02<00:41, 99.39 examples/s] Saving the dataset (32/47 shards): 68%|██████████████▎ | 8339/12239 [00:02<00:39, 99.39
model.safetensors.index.json: 0.00B [00:00, ?B/s] model.safetensors.index.json: 25.1kB [00:00, 22.2MB/s]
Downloading (incomplete total...): 0.00B [00:00, ?B/s]
Fetching 2 files: 0%| | 0/2 [00:00<?, ?it/s] Downloading (incomplete total...): 0%| | 0.00/9.94G [00:00<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:00<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:00<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:00<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:00<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:01<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:01<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:01<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:01<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:01<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:02<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:02<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:02<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:02<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:02<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:03<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:03<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:03<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:03<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:03<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:04<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:04<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:04<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:04<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:04<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:05<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:05<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:05<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:05<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:05<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:06<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:06<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:06<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:06<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:06<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:07<?, ?B/s] Downloading (incomplete total...): 0%| | 0.00/14.5G [00:07<?, ?B/s] Downloading (incomplete total...): 0%|
Fetching 2 files: 50%|█████████████████████████▌ | 1/2 [01:55<01:55, 115.61s/it] Fetching 2 files: 100%|████████████████████████████████████████████████████| 2/2 [01:55<00:00, 57.81s/it]
Download complete: 100%|████████████████████████████████████████████| 14.5G/14.5G [01:55<00:00, 1.27GB/s] Download complete: 100%|█████████████████████████████████████████████| 14.5G/14.5G [01:55<00:00, 125MB/s]
Loading weights: 0%| | 0/291 [00:00<?, ?it/s] Loading weights: 1%|▋ | 4/291 [00:00<00:08, 34.12it/s] Loading weights: 5%|██▌ | 15/291 [00:00<00:03, 70.11it/s] Loading weights: 8%|████ | 24/291 [00:00<00:03, 73.08it/s] Loading weights: 11%|█████▋ | 33/291 [00:00<00:03, 71.41it/s] Loading weights: 14%|███████▏ | 42/291 [00:00<00:03, 70.51it/s] Loading weights: 18%|████████▊ | 51/291 [00:00<00:03, 69.79it/s] Loading weights: 21%|██████████▎ | 60/291 [00:00<00:03, 70.30it/s] Loading weights: 24%|███████████▊ | 69/291 [00:00<00:03, 70.26it/s] Loading weights: 27%|█████████████▍ | 78/291 [00:01<00:03, 70.16it/s] Loading weights: 30%|██████████████▉ | 87/291 [00:01<00:02, 70.15it/s] Loading weights: 33%|████████████████▍ | 96/291 [00:01<00:02, 70.19it/s] Loading weights: 36%|█████████████████▋ | 105/291 [00:01<00:02, 70.66it/s] Loading weights: 39%|███████████████████▏ | 114/291 [00:01<00:02, 71.39it/s] Loading weights: 42%|████████████████████▋ | 123/291 [00:01<00:02, 71.24it/s] Loading weights: 45%|██████████████████████▏ | 132/291 [00:01<00:02, 70.75it/s] Loading weights: 48%|███████████████████████▋ | 141/291 [00:02<00:02, 70.52it/s] Loading weights: 52%|█████████████████████████▎ | 150/291 [00:02<00:01, 71.30it/s] Loading weights: 55%|██████████████████████████▊ | 159/291 [00:02<00:01, 70.72it/s] Loading weights: 58%|████████████████████████████▎ | 168/291 [00:02<00:01, 70.33it/s] Loading weights: 61%|█████████████████████████████▊ | 177/291 [00:02<00:01, 70.18it/s] Loading weights: 64%|███████████████████████████████▎ | 186/291 [00:02<00:01, 70.04it/s] Loading weights: 67%|████████████████████████████████▊ | 195/291 [00:02<00:01, 69.81it/s] Loading weights: 70%|██████████████████████████████████▎ | 204/291 [00:02<00:01, 69.73it/s] Loading weights: 73%|███████████████████████████████████▊ | 213/291 [00:03<00:01, 69.80it/s] Loading weights: 76%|█████████████████████████████████████▍ | 222/291 [00:03<00:00, 69.93it/s] Loading weights: 79%|██████████████████████████████████████▉ | 231/291 [00:03<00:00, 69.96it/s] Loading weights: 82%|████████████████████████████████████████▍ | 240/291 [00:03<
Loading weights: 0%| | 0/291 [00:00<?, ?it/s] Loading weights: 2%|█ | 6/291 [00:00<00:05, 52.48it/s] Loading weights: 5%|██▌ | 15/291 [00:00<00:03, 71.56it/s] Loading weights: 8%|████ | 24/291 [00:00<00:03, 71.72it/s] Loading weights: 11%|█████▋ | 33/291 [00:00<00:03, 70.60it/s] Loading weights: 14%|███████▏ | 42/291 [00:00<00:03, 70.27it/s] Loading weights: 18%|████████▊ | 51/291 [00:00<00:03, 69.66it/s] Loading weights: 21%|██████████▎ | 60/291 [00:00<00:03, 69.77it/s] Loading weights: 24%|███████████▊ | 69/291 [00:00<00:03, 69.14it/s] Loading weights: 27%|█████████████▍ | 78/291 [00:01<00:03, 68.04it/s] Loading weights: 30%|██████████████▊ | 86/291 [00:01<00:02, 70.10it/s] Loading weights: 33%|████████████████▎ | 95/291 [00:01<00:02, 68.70it/s] Loading weights: 35%|█████████████████▎ | 103/291 [00:01<00:02, 71.47it/s] Loading weights: 38%|██████████████████▊ | 112/291 [00:01<00:02, 69.77it/s] Loading weights: 42%|████████████████████▎ | 121/291 [00:01<00:02, 68.61it/s] Loading weights: 44%|█████████████████████▌ | 128/291 [00:01<00:02, 68.90it/s] Loading weights: 46%|██████████████████████▋ | 135/291 [00:01<00:02, 65.03it/s] Loading weights: 49%|███████████████████████▉ | 142/291 [00:02<00:02, 62.63it/s] Loading weights: 52%|█████████████████████████▎ | 150/291 [00:02<00:02, 61.64it/s] Loading weights: 54%|██████████████████████████▌ | 158/291 [00:02<00:02, 66.26it/s] Loading weights: 57%|███████████████████████████▉ | 166/291 [00:02<00:01, 68.82it/s] Loading weights: 59%|█████████████████████████████▏ | 173/291 [00:02<00:01, 68.31it/s] Loading weights: 62%|██████████████████████████████▎ | 180/291 [00:02<00:01, 64.66it/s] Loading weights: 64%|███████████████████████████████▍ | 187/291 [00:02<00:01, 61.70it/s] Loading weights: 67%|████████████████████████████████▋ | 194/291 [00:02<00:01, 63.24it/s] Loading weights: 69%|██████████████████████████████████ | 202/291 [00:02<00:01, 67.04it/s] Loading weights: 72%|███████████████████████████████████▏ | 209/291 [00:03<00:01, 67.14it/s] Loading weights: 74%|████████████████████████████████████▎ | 216/291 [00:03<00:01, 63.63it/s] Loading weights: 77%|███████<E29688>
Adding EOS to train dataset (num_proc=96): 0%| | 0/11627 [00:00<?, ? examples/s] Adding EOS to train dataset (num_proc=96): 1%|▏ | 122/11627 [00:02<03:29, 55.03 examples/s] Adding EOS to train dataset (num_proc=96): 5%|▋ | 610/11627 [00:02<00:32, 335.85 examples/s] Adding EOS to train dataset (num_proc=96): 9%|█▏ | 1098/11627 [00:02<00:15, 659.66 examples/s] Adding EOS to train dataset (num_proc=96): 13%|█▌ | 1462/11627 [00:02<00:11, 913.75 examples/s] Adding EOS to train dataset (num_proc=96): 16%|█▋ | 1826/11627 [00:02<00:07, 1229.71 examples/s] Adding EOS to train dataset (num_proc=96): 20%|██▏ | 2310/11627 [00:02<00:05, 1706.16 examples/s] Adding EOS to train dataset (num_proc=96): 25%|██▊ | 2915/11627 [00:03<00:03, 2346.71 examples/s] Adding EOS to train dataset (num_proc=96): 100%|██████████| 11627/11627 [00:03<00:00, 3513.06 examples/s]
Tokenizing train dataset (num_proc=96): 0%| | 0/11627 [00:00<?, ? examples/s] Tokenizing train dataset (num_proc=96): 0%| | 52/11627 [00:02<09:56, 19.41 examples/s] Tokenizing train dataset (num_proc=96): 2%|▎ | 181/11627 [00:03<02:37, 72.78 examples/s] Tokenizing train dataset (num_proc=96): 3%|▍ | 305/11627 [00:03<01:31, 124.02 examples/s] Tokenizing train dataset (num_proc=96): 4%|▌ | 420/11627 [00:03<01:07, 166.38 examples/s] Tokenizing train dataset (num_proc=96): 5%|▊ | 547/11627 [00:04<00:52, 211.39 examples/s] Tokenizing train dataset (num_proc=96): 6%|▉ | 666/11627 [00:04<00:45, 243.26 examples/s] Tokenizing train dataset (num_proc=96): 7%|█ | 788/11627 [00:04<00:39, 276.71 examples/s] Tokenizing train dataset (num_proc=96): 8%|█▏ | 907/11627 [00:05<00:37, 287.04 examples/s] Tokenizing train dataset (num_proc=96): 9%|█▎ | 1033/11627 [00:05<00:34, 306.70 examples/s] Tokenizing train dataset (num_proc=96): 10%|█▍ | 1148/11627 [00:05<00:33, 314.77 examples/s] Tokenizing train dataset (num_proc=96): 11%|█▋ | 1272/11627 [00:06<00:31, 325.01 examples/s] Tokenizing train dataset (num_proc=96): 12%|█▊ | 1403/11627 [00:06<00:29, 350.19 examples/s] Tokenizing train dataset (num_proc=96): 13%|█▉ | 1522/11627 [00:06<00:29, 345.57 examples/s] Tokenizing train dataset (num_proc=96): 14%|██ | 1639/11627 [00:07<00:30, 328.47 examples/s] Tokenizing train dataset (num_proc=96): 15%|██▎ | 1765/11627 [00:07<00:28, 343.00 examples/s] Tokenizing train dataset (num_proc=96): 16%|██▍ | 1886/11627 [00:08<00:29, 328.89 examples/s] Tokenizing train dataset (num_proc=96): 17%|██▌ | 2010/11627 [00:08<00:28, 337.40 examples/s] Tokenizing train dataset (num_proc=96): 18%|██▋ | 2123/11627 [00:08<00:27, 341.79 examples/s] Tokenizing train dataset (num_proc=96): 19%|██▉ | 2243/11627 [00:09<00:28, 333.79 examples/s] Tokenizing train dataset (num_proc=96): 20%|███ | 2369/11627 [00:09<00:26, 343.37 examples/s] Tokenizing train dataset (num_proc=96): 21%|███▏ | 2492/11627 [00:09<00:26, 342.63 examples/s] Tokenizing train dataset (num_proc=96): 22%|███▎ | 2609/11627 [00:10<00:26, 341.92 examples/s] Tokenizing train dataset (num_proc=96): 23%|███▌ | 2731/11627 [00:10<00:25, 345.62 examples/s] Tokenizing train dataset (num_proc=96): 25%|███▋ | 2850/11627 [00:10<00:25, 345.30 examples/s] Tokenizing train dataset (num_proc=96): 26%|███▊ | 2977/11627 [00:11<00:24, 356.89 examples/s] Tokenizing train dataset (num_proc=96): 27%|███▉ | 3090/11627 [00:11<00:25, 328.45 examples/s] Tokenizing train dataset (num_proc=96): 28%|████▏ | 3214/11627 [00:11<00:24, 339.73 examples/s] Tokenizing train dataset (num_proc=96): 29%|████▎ | 3338/11627 [00:12<00:23, 345.68 examples/s] Tokenizing train dataset (num_proc=96): 30%|████▍ | 3458/11627 [00:12<00:23, 346.38 examples/s] Tokenizing train dataset (num_proc=96): 31%|████▌ | 3584/11627 [00:12<00:22, 358.29 examples/s] Tokenizing train dataset (num_proc=96): 32%|████▊ | 3703/11627 [00:13<00:22, 345.43 examples/s] Tokenizing train dataset (num_proc=96): 33%|████▉ | 3817/11627 [00:13<00:22, 340.29 examples/s] Tokenizing train dataset (num_proc=96): 34%|█████ | 3947/11627 [00:13<00:21, 352.58 examples/s] Tokenizing train dataset (num_proc=96): 35%|█████▏ | 4063/11627 [00:14<00:21, 344.79 examples/s] Tokenizing train dataset (num_proc=96): 36%|█████▍ | 4175/11627 [00:14<00:22, 338.50 examples/s] Tokenizing train dataset (num_proc=96): 37%|█████
Adding EOS to eval dataset (num_proc=96): 0%| | 0/612 [00:00<?, ? examples/s] Adding EOS to eval dataset (num_proc=96): 1%|▏ | 7/612 [00:02<03:08, 3.20 examples/s] Adding EOS to eval dataset (num_proc=96): 6%|█ | 35/612 [00:02<00:29, 19.49 examples/s] Adding EOS to eval dataset (num_proc=96): 10%|█▊ | 63/612 [00:02<00:14, 39.05 examples/s] Adding EOS to eval dataset (num_proc=96): 15%|██▋ | 91/612 [00:02<00:08, 61.49 examples/s] Adding EOS to eval dataset (num_proc=96): 19%|███▎ | 119/612 [00:02<00:05, 84.60 examples/s] Adding EOS to eval dataset (num_proc=96): 23%|███▋ | 140/612 [00:02<00:04, 101.96 examples/s] Adding EOS to eval dataset (num_proc=96): 29%|████▌ | 175/612 [00:02<00:03, 143.60 examples/s] Adding EOS to eval dataset (num_proc=96): 68%|██████████▊ | 414/612 [00:03<00:00, 577.40 examples/s] Adding EOS to eval dataset (num_proc=96): 100%|████████████████| 612/612 [00:03<00:00, 183.57 examples/s]
Tokenizing eval dataset (num_proc=96): 0%| | 0/612 [00:00<?, ? examples/s] Tokenizing eval dataset (num_proc=96): 1%|▎ | 7/612 [00:02<03:53, 2.59 examples/s] Tokenizing eval dataset (num_proc=96): 2%|▍ | 14/612 [00:03<01:53, 5.29 examples/s] Tokenizing eval dataset (num_proc=96): 3%|▋ | 21/612 [00:03<01:16, 7.72 examples/s] Tokenizing eval dataset (num_proc=96): 5%|▉ | 28/612 [00:03<00:55, 10.52 examples/s] Tokenizing eval dataset (num_proc=96): 6%|█▏ | 35/612 [00:04<00:45, 12.57 examples/s] Tokenizing eval dataset (num_proc=96): 7%|█▍ | 42/612 [00:04<00:39, 14.27 examples/s] Tokenizing eval dataset (num_proc=96): 8%|█▋ | 49/612 [00:04<00:35, 15.75 examples/s] Tokenizing eval dataset (num_proc=96): 9%|█▉ | 56/612 [00:05<00:32, 17.09 examples/s] Tokenizing eval dataset (num_proc=96): 10%|██▏ | 63/612 [00:05<00:30, 17.87 examples/s] Tokenizing eval dataset (num_proc=96): 11%|██▍ | 70/612 [00:05<00:29, 18.67 examples/s] Tokenizing eval dataset (num_proc=96): 13%|██▋ | 77/612 [00:06<00:27, 19.32 examples/s] Tokenizing eval dataset (num_proc=96): 14%|██▉ | 84/612 [00:06<00:28, 18.49 examples/s] Tokenizing eval dataset (num_proc=96): 15%|███ | 91/612 [00:06<00:28, 18.53 examples/s] Tokenizing eval dataset (num_proc=96): 16%|███▎ | 98/612 [00:07<00:26, 19.25 examples/s] Tokenizing eval dataset (num_proc=96): 17%|███▍ | 105/612 [00:07<00:25, 19.59 examples/s] Tokenizing eval dataset (num_proc=96): 18%|███▋ | 112/612 [00:08<00:25, 19.63 examples/s] Tokenizing eval dataset (num_proc=96): 19%|███▉ | 119/612 [00:08<00:24, 19.74 examples/s] Tokenizing eval dataset (num_proc=96): 21%|████ | 126/612 [00:08<00:24, 19.78 examples/s] Tokenizing eval dataset (num_proc=96): 22%|████▎ | 133/612 [00:09<00:23, 20.19 examples/s] Tokenizing eval dataset (num_proc=96): 23%|████▌ | 140/612 [00:09<00:23, 19.84 examples/s] Tokenizing eval dataset (num_proc=96): 24%|████▊ | 147/612 [00:09<00:23, 19.93 examples/s] Tokenizing eval dataset (num_proc=96): 25%|█████ | 154/612 [00:10<00:23, 19.82 examples/s] Tokenizing eval dataset (num_proc=96): 26%|█████▎ | 161/612 [00:10<00:22, 20.09 examples/s] Tokenizing eval dataset (num_proc=96): 27%|█████▍ | 168/612 [00:10<00:22, 19.51 examples/s] Tokenizing eval dataset (num_proc=96): 29%|█████▋ | 175/612 [00:11<00:21, 19.94 examples/s] Tokenizing eval dataset (num_proc=96): 30%|█████▉ | 182/612 [00:11<00:21, 19.59 examples/s] Tokenizing eval dataset (num_proc=96): 31%|██████▏ | 189/612 [00:11<00:20, 20.26 examples/s] Tokenizing eval dataset (num_proc=96): 32%|██████▍ | 196/612 [00:12<00:21, 19.24 examples/s] Tokenizing eval dataset (num_proc=96): 33%|██████▋ | 203/612 [00:12<00:21, 19.29 examples/s] Tokenizing eval dataset (num_proc=96): 34%|██████▊ | 210/612 [00:12<00:20, 19.56 examples/s] Tokenizing eval dataset (num_proc=96): 35%|███████ | 217/612 [00:13<00:20, 19.48 examples/s] Tokenizing eval dataset (num_proc=96): 37%|███████▎ | 224/612 [00:13<00:19, 19.77 examples/s] Tokenizing eval dataset (num_proc=96): 38%|███████▌ | 231/612 [00:14<00:19, 19.83 examples/s] Tokenizing eval dataset (num_proc=96): 39%|███████▊ | 238/612 [00:14<00:18, 19.82 examples/s] Tokenizing eval dataset (num_proc=96): 40%|████████ | 24
Reference in New Issue Copy Permalink