初始化项目,由ModelHub XC社区提供模型
Model: laion/ablation-pymethods2test-seqnorm-15-8B Source: Original Platform
This commit is contained in:
29
training_logs/20260608_210110_metrics_job_639470.csv
Normal file
29
training_logs/20260608_210110_metrics_job_639470.csv
Normal file
@@ -0,0 +1,29 @@
|
||||
async/discard_rate,async/discarded_count,async/effective_batch_groups,async/effective_batch_samples,async/staleness_max,async/staleness_mean,async/staleness_min,async/staleness_ratio,generate/avg_num_tokens,generate/avg_tokens_non_zero_rewards,generate/avg_tokens_zero_rewards,generate/max_num_tokens,generate/min_num_tokens,generate/std_num_tokens,loss/avg_final_rewards,loss/avg_raw_advantages,loss/avg_raw_advantages_abs,policy/final_loss,policy/log_ratio_abs_max,policy/log_ratio_abs_mean,policy/log_ratio_abs_p99,policy/log_ratio_abs_pos00,policy/log_ratio_abs_pos10,policy/log_ratio_abs_pos20,policy/log_ratio_abs_pos30,policy/log_ratio_abs_pos40,policy/log_ratio_abs_pos50,policy/log_ratio_abs_pos60,policy/log_ratio_abs_pos70,policy/log_ratio_abs_pos80,policy/log_ratio_abs_pos90,policy/n_tokens_dp_gt_10pct,policy/n_tokens_dp_gt_1pct,policy/n_tokens_dp_gt_50pct,policy/policy_entropy,policy/policy_loss,policy/policy_lr,policy/policy_update_steps,policy/ppo_clip_ratio,policy/raw_grad_norm,reward/avg_pass_at_8,reward/avg_raw_reward,system/process_rss_gb,system/process_vms_gb,system/ram_available_gb,system/ram_percent,system/ram_total_gb,system/ram_used_gb,timing/compute_advantages_and_returns,timing/convert_to_training_input,timing/fwd_logprobs_values_reward,timing/policy_train,timing/run_training,timing/step,timing/sync_weights,timing/train_critic_and_policy,timing/wait_for_generation_buffer,trainer/epoch,trainer/global_step,batch_errors/total_batches,batch_errors/total_instances,batch_errors/total_successful,batch_errors/total_failed,batch_errors/total_masked,batch_errors/avg_VerifierTimeoutError,batch_errors/total_VerifierTimeoutError,batch_errors/avg_ContextLengthExceededError,batch_errors/total_ContextLengthExceededError,batch_errors/avg_AgentTimeoutError,batch_errors/total_AgentTimeoutError,batch_errors/avg_InvalidChatHistory,batch_errors/total_InvalidChatHistory,batch_errors/avg_AgentSetupTimeoutError,batch_errors/total_AgentSetupTimeoutError,timing/cleanup_old_checkpoints,timing/save_checkpoints,timing/save_hf_model,batch_errors/avg_DaytonaAuthenticationError,batch_errors/total_DaytonaAuthenticationError,batch_errors/avg_RuntimeError,batch_errors/total_RuntimeError,batch_errors/avg_Timeout,batch_errors/total_Timeout
|
||||
0.0,0,64,512,0,0.0,0,0.0,4204.6133,3980.0401,4997.5752,27880,1,2753.0192,0.7793,-0.0112,0.1056,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1167,0.0,0.0,1.0,0.0,0.0,0.8438,0.7793,12.2631,50.3984,585.1565,31.8,857.9676,272.8111,0.1765,4.0466,31.755,151.813,183.9451,2948.9375,21.9869,152.0133,2738.9586,0,1,125,1000,979,14,16,0.032,4.0,0.08,10,0.008,1.0,0.04,5,0.008,1.0,,,,,,,,,
|
||||
0.0,0,64,512,1,0.8125,0,0.8125,5306.6934,5657.698,4984.6105,31779,1,4492.8588,0.4785,0.0293,0.129,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1394,-0.0,0.0,1.0,0.0,0.0,0.625,0.4785,12.7607,50.6826,576.8641,32.8,857.9676,281.1035,0.0756,4.7377,39.2039,228.2368,267.76,1838.1447,22.259,228.4801,1543.3857,0,2,58,464,418,20,39,0.10344827586206896,6.0,0.1896551724137931,11,0.06896551724137931,4.0,0.5517241379310345,32,0.017241379310344827,1.0,11.9772,23.0244,,,,,,,
|
||||
0.0,0,64,512,2,1.6406,1,1.0,5116.0566,5485.2443,4989.1181,31748,1,5080.3634,0.2559,0.0172,0.1066,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1377,-0.0,0.0,1.0,0.0,0.0,0.3594,0.2559,12.9185,50.7878,575.0807,33.0,857.9676,282.8868,0.1134,4.8467,36.4005,233.6667,270.4372,828.5581,22.9816,233.9229,530.29,0,3,63,504,481,13,20,0.031746031746031744,2.0,0.12698412698412698,8,0.015873015873015872,1.0,0.1746031746031746,11,0.015873015873015872,1.0,,,,,,,,,
|
||||
0.0,0,64,512,3,1.7188,0,0.8594,4765.7383,4558.2788,4995.3951,31815,1,4154.9884,0.5254,-0.0022,0.0986,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1334,0.0,0.0,1.0,0.0,0.0,0.625,0.5254,13.0012,50.838,574.6276,33.0,857.9676,283.34,0.0786,4.6187,33.8366,202.053,236.1893,1643.3116,23.5376,202.2738,1378.9625,0,4,61,488,453,15,29,0.08196721311475409,5.0,0.14754098360655737,9,0.03278688524590164,2.0,0.29508196721311475,18,0.01639344262295082,1.0,0.007,11.8014,,,,,,,
|
||||
0.0,0,64,512,4,1.8594,0,0.8906,4864.7266,5293.4787,4615.9444,31633,1,4252.8392,0.3672,0.0121,0.1054,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1324,-0.0,0.0,1.0,0.0,0.0,0.4844,0.3672,13.094,50.8868,574.0699,33.1,857.9676,283.8977,0.0636,4.5964,36.9763,209.4955,246.7664,1591.9611,23.9136,209.726,1316.6818,0,5,58,464,442,15,22,,,0.22413793103448276,13,0.017241379310344827,1.0,0.1724137931034483,10,,,,,8.9801,,,,,,
|
||||
0.0,0,64,512,4,2.125,0,0.9688,5790.6836,5796.5498,5785.0421,31796,1,5065.3921,0.4902,0.0026,0.1038,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1329,-0.0,0.0,1.0,0.0,0.0,0.5781,0.4902,13.1595,50.9821,573.3127,33.2,857.9676,284.6549,0.0652,4.8477,38.9466,242.4056,281.6225,968.7238,25.1848,242.6104,657.0663,0,6,62,496,464,15,21,0.1774193548387097,11.0,0.12903225806451613,8,0.06451612903225806,4.0,0.1774193548387097,11,,,0.0074,12.5554,,,,,,,
|
||||
0.0,0,64,512,4,1.8906,0,0.9219,5076.3281,4580.6522,5480.6028,31841,1,4655.3477,0.4492,-0.0017,0.0668,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1365,0.0,0.0,1.0,0.0,0.0,0.5625,0.4492,13.1735,50.9979,572.762,33.2,857.9676,285.2056,0.0629,4.6355,34.4,220.7394,255.4282,1394.3532,22.6359,220.965,1111.6508,0,7,63,504,478,15,25,0.015873015873015872,1.0,0.12698412698412698,8,,,0.2698412698412698,17,,,,,,,,,,,
|
||||
0.0,0,64,512,4,2.1094,0,0.8906,6032.8496,5776.6962,6253.6073,31766,1,5542.4923,0.4629,-0.0002,0.1325,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1434,0.0,0.0,1.0,0.0,0.0,0.625,0.4629,13.1891,51.0035,571.7978,33.4,857.9676,286.1698,0.0656,4.7759,41.5188,266.819,308.6626,1826.314,24.5452,267.0779,1488.327,0,8,62,496,461,18,32,0.04838709677419355,3.0,0.14516129032258066,9,0.08064516129032258,5.0,0.2903225806451613,18,,,0.0078,13.2713,,,,,,,
|
||||
0.0,0,64,512,4,1.9219,0,0.9062,5214.0684,4609.1528,5655.4932,31811,1,4602.8614,0.4219,-0.006,0.1127,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.138,0.0,0.0,1.0,0.0,0.0,0.5625,0.4219,13.1974,51.0071,571.1542,33.4,857.9676,286.8134,0.0623,4.7115,36.4913,219.5474,256.316,1383.5879,21.2289,219.762,1101.328,0,9,61,488,459,16,27,0.03278688524590164,2.0,0.13114754098360656,8,0.01639344262295082,1.0,0.29508196721311475,18,,,,,,,,,,,
|
||||
0.0,0,64,512,3,1.8438,0,0.9531,4883.5527,4809.5297,4946.8478,31765,1,4293.3557,0.4609,0.0117,0.1251,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1369,-0.0,0.0,1.0,0.0,0.0,0.5938,0.4609,14.1407,51.0665,564.7126,34.2,857.9676,293.255,0.0946,4.751,37.1062,205.8864,243.3256,1131.1312,24.7025,206.1244,858.3488,0,10,61,488,460,15,23,0.08196721311475409,5.0,0.09836065573770492,6,0.01639344262295082,1.0,0.26229508196721313,16,,,0.0075,12.9056,8.8306,,,,,,
|
||||
0.0,0,64,512,3,1.7812,0,0.9062,4549.2324,4530.1362,4568.4784,31819,1,3818.3797,0.502,0.0195,0.157,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1388,-0.0,0.0,1.0,0.0,0.0,0.6719,0.502,14.1364,51.0617,563.7548,34.3,857.9676,294.2128,0.0871,4.7176,34.0193,184.8761,219.1954,1502.0577,23.6183,185.0886,1254.5217,0,11,63,504,474,17,29,0.015873015873015872,1.0,0.15873015873015872,10,,,0.30158730158730157,19,,,,,,,,,,,
|
||||
0.0,0,64,512,4,2.1562,0,0.875,5296.7969,4698.2873,5624.0785,31847,1,4178.7928,0.3535,0.0064,0.0936,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1414,-0.0,0.0,1.0,0.0,0.0,0.4375,0.3535,14.1368,51.059,563.1395,34.4,857.9676,294.8281,0.0852,4.7881,36.8212,203.0982,240.2245,1268.0385,21.8286,203.3178,1001.1932,0,12,63,504,453,20,42,0.14285714285714285,9.0,0.31746031746031744,20,0.1111111111111111,7.0,0.23809523809523808,15,,,0.0075,12.3179,,,,,,,
|
||||
0.0,0,64,512,4,1.9688,0,0.8438,4750.25,4750.5154,4750.0386,31807,1,4514.3945,0.4434,-0.0169,0.1051,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.124,0.0,0.0,1.0,0.0,0.0,0.5938,0.4434,14.1379,51.1242,562.4609,34.4,857.9676,295.5067,0.0878,4.6646,34.6581,200.5881,235.5732,1478.8349,21.9149,200.8269,1216.6776,0,13,64,512,461,19,42,0.125,8.0,0.109375,7,0.171875,11.0,0.375,24,0.015625,1.0,,,,,,,,,
|
||||
0.0,0,64,512,4,1.8281,0,0.9062,4798.502,4904.2,4721.9865,20821,1,4021.2985,0.4199,0.0107,0.1478,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1326,-0.0,0.0,1.0,0.0,0.0,0.6094,0.4199,13.9293,50.9799,562.1901,34.5,857.9676,295.7775,0.0499,3.3116,33.3015,194.2488,227.806,1529.1316,20.1055,194.4542,1277.9038,0,14,61,488,477,7,11,,,0.03278688524590164,2,0.03278688524590164,2.0,0.14754098360655737,9,,,0.0075,13.2883,,,,,,,
|
||||
0.0,0,64,512,4,1.7969,0,0.8906,5064.6367,4590.1661,5650.9913,31722,1,3612.7922,0.5527,0.0086,0.1079,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1393,-0.0,0.0,1.0,0.0,0.0,0.625,0.5527,14.5292,51.5609,560.838,34.6,857.9676,297.1296,0.1119,4.7729,30.2506,177.5966,208.1631,1397.2363,22.9002,177.8003,1161.3976,0,15,59,472,368,33,68,0.3559322033898305,21.0,0.2033898305084746,12,0.8135593220338984,48.0,0.2542372881355932,15,0.23728813559322035,14.0,,,8.7705,0.01694915254237288,1.0,,,,
|
||||
0.0,0,64,512,4,2.125,0,0.8906,4274.1406,4573.4162,4121.413,31809,1,4387.0225,0.3379,0.0301,0.1177,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1111,-0.0,0.0,1.0,0.0,0.0,0.5,0.3379,14.5318,51.5632,560.3275,34.7,857.9676,297.6401,0.1121,4.5533,30.9684,197.875,229.1899,2136.448,25.0381,198.109,1877.6589,0,16,60,480,435,18,36,0.13333333333333333,8.0,0.35,21,0.016666666666666666,1.0,0.36666666666666664,22,0.016666666666666666,1.0,0.0077,13.2783,,,,,,,
|
||||
0.0,0,64,512,4,2.0,0,0.9688,4734.5156,5008.1053,4573.0807,20674,1,3774.2358,0.3711,0.0059,0.0758,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1229,-0.0,0.0,1.0,0.0,0.0,0.4531,0.3711,14.6272,51.7075,560.1359,34.7,857.9676,297.8317,0.081,3.4166,30.0522,179.5794,209.9197,1116.7953,19.6398,179.7862,883.8122,0,17,57,456,435,11,19,0.017543859649122806,1.0,0.15789473684210525,9,0.017543859649122806,1.0,0.24561403508771928,14,0.017543859649122806,1.0,,,,,,,,,
|
||||
0.0,0,64,512,4,1.7031,0,0.8281,4471.5098,4470.4851,4472.3791,31766,1,3869.9902,0.459,0.0336,0.1621,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1329,-0.0,0.0,1.0,0.0,0.0,0.6094,0.459,14.7954,51.8759,559.9256,34.7,857.9676,298.042,0.1127,4.6919,30.7942,184.653,215.7859,2036.849,23.8428,184.8787,1792.5224,0,18,58,464,419,26,40,0.08620689655172414,5.0,0.22413793103448276,13,,,0.4827586206896552,28,,,0.0073,13.3234,,,,,,,
|
||||
0.0,0,64,512,5,2.0312,0,0.9531,4740.6719,4578.8382,4823.2596,31842,1,4293.5827,0.3379,-0.002,0.1209,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1469,0.0,0.0,1.0,0.0,0.0,0.4688,0.3379,14.7972,51.8773,559.5532,34.8,857.9676,298.4144,0.1151,4.6856,35.8461,193.9452,230.1248,1344.6664,23.0847,194.1632,1086.7632,0,19,60,480,461,9,16,0.03333333333333333,2.0,0.016666666666666666,1,0.06666666666666667,4.0,0.23333333333333334,14,,,,,,,,0.016666666666666666,1.0,,
|
||||
0.0,0,64,512,4,1.7031,0,0.8594,4880.1914,4838.5936,4911.2833,31680,1,3718.5085,0.4277,0.0027,0.0566,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.142,-0.0,0.0,1.0,0.0,0.0,0.5,0.4277,14.8422,51.875,558.8903,34.9,857.9676,299.0773,0.1142,4.6474,32.816,178.6089,211.7893,1843.2328,20.1983,178.8585,1606.5913,0,20,64,512,471,19,41,,,0.171875,11,,,0.46875,30,,,0.0073,12.4239,8.7038,,,,,,
|
||||
0.0,0,64,512,3,1.7969,0,0.9062,4588.334,4493.3441,4642.5307,31717,1,3962.6762,0.3633,0.0029,0.1589,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1395,0.0,0.0,1.0,0.0,0.0,0.5156,0.3633,14.8433,51.876,558.6581,34.9,857.9676,299.3095,0.1126,4.5229,33.0721,192.653,226.0719,1421.8711,21.8044,192.8868,1169.4651,0,21,57,456,428,15,26,0.017543859649122806,1.0,0.07017543859649122,4,0.017543859649122806,1.0,0.38596491228070173,22,0.017543859649122806,1.0,,,,,,,,,
|
||||
0.0,0,64,512,4,1.7656,0,0.875,4870.873,5241.0396,4629.6677,31739,1,4219.8385,0.3945,0.0013,0.1305,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1352,0.0,0.0,1.0,0.0,0.0,0.5312,0.3945,14.8431,51.9402,558.3341,34.9,857.9676,299.6335,0.1085,4.6972,33.7789,196.7163,230.8154,1494.0327,21.1779,196.9276,1237.3356,0,22,61,488,449,18,38,0.01639344262295082,1.0,0.13114754098360656,8,,,0.5245901639344263,32,,,0.0075,12.276,,,,,,,
|
||||
0.0,0,64,512,3,1.7344,0,0.875,4640.7969,4846.2115,4500.25,31768,1,4044.2257,0.4062,0.0139,0.096,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1285,-0.0,0.0,1.0,0.0,0.0,0.5156,0.4062,15.2382,52.3806,557.8073,35.0,857.9676,300.1603,0.1114,4.6277,30.466,192.71,223.5157,1293.9556,24.0146,192.938,1041.7923,0,23,63,504,445,19,46,0.1746031746031746,11.0,0.14285714285714285,9,0.06349206349206349,4.0,0.5238095238095238,33,,,,,,0.015873015873015872,1.0,,,0.015873015873015872,1.0
|
||||
0.0,0,64,512,5,2.0781,0,0.9219,4341.1699,5158.0,3913.3065,31757,1,4123.9554,0.3438,0.0106,0.1051,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1264,-0.0,0.0,1.0,0.0,0.0,0.4531,0.3438,15.2377,52.3799,556.5636,35.1,857.9676,301.404,0.1106,4.5859,32.4505,188.7186,221.4767,1641.4602,21.8273,188.9153,1393.5632,0,24,60,480,451,12,25,0.06666666666666667,4.0,0.16666666666666666,10,0.11666666666666667,7.0,0.15,9,,,0.0076,11.67,,,,,,,
|
||||
0.0,0,64,512,4,1.5625,0,0.8281,4450.6797,4570.1978,4319.4057,22934,1,3289.9179,0.5234,0.0061,0.096,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1266,-0.0,0.0,1.0,0.0,0.0,0.6406,0.5234,15.0594,52.2661,557.0922,35.1,857.9676,300.8754,0.0845,3.4762,30.0472,159.5092,189.8417,1090.4692,20.7836,159.7097,876.3605,0,25,62,496,455,15,26,0.24193548387096775,15.0,0.20967741935483872,13,0.06451612903225806,4.0,0.16129032258064516,10,,,,,8.7814,,,,,,
|
||||
0.0,0,64,512,4,2.0,0,0.9375,5077.3633,4987.4731,5170.1071,31764,1,4875.2722,0.5078,0.0036,0.0731,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1211,-0.0,0.0,1.0,0.0,0.0,0.5781,0.5078,15.2418,52.4453,556.3601,35.2,857.9676,301.6075,0.1143,4.8282,36.5507,227.4206,264.3124,1169.9258,21.9494,227.6469,878.8315,0,26,63,504,483,12,19,0.031746031746031744,2.0,0.06349206349206349,4,0.031746031746031744,2.0,0.20634920634920634,13,,,0.0084,11.212,,,,,,,
|
||||
0.0,0,64,512,4,1.8281,0,0.875,4673.9082,4273.9848,5096.3173,31755,1,3635.6011,0.5137,0.0092,0.0966,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1351,-0.0,0.0,1.0,0.0,0.0,0.6406,0.5137,15.2407,52.4443,555.6974,35.2,857.9676,302.2701,0.1106,4.6972,30.1187,177.3769,207.8291,1096.7705,23.892,177.5995,860.3458,0,27,61,488,449,15,37,0.01639344262295082,1.0,0.19672131147540983,12,0.04918032786885246,3.0,0.3770491803278688,23,,,,,,0.01639344262295082,1.0,,,,
|
||||
0.0,0,64,512,4,1.9688,0,0.8594,4745.1426,4807.2651,4700.1717,31484,1,3794.1737,0.4199,-0.0062,0.1298,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1313,0.0,0.0,1.0,0.0,0.0,0.5469,0.4199,15.2377,52.5056,554.7346,35.3,857.9676,303.233,0.1099,4.5698,33.2127,183.796,217.3223,1415.0287,21.5982,183.9994,1171.5333,0,28,59,472,421,19,41,0.1694915254237288,10.0,0.11864406779661017,7,0.0847457627118644,5.0,0.5084745762711864,30,,,0.0085,13.2233,,,,,,,
|
||||
|
22
training_logs/20260608_210110_metrics_job_639471.csv
Normal file
22
training_logs/20260608_210110_metrics_job_639471.csv
Normal file
@@ -0,0 +1,22 @@
|
||||
async/discard_rate,async/discarded_count,async/effective_batch_groups,async/effective_batch_samples,async/staleness_max,async/staleness_mean,async/staleness_min,async/staleness_ratio,generate/avg_num_tokens,generate/avg_tokens_non_zero_rewards,generate/avg_tokens_zero_rewards,generate/max_num_tokens,generate/min_num_tokens,generate/std_num_tokens,loss/avg_final_rewards,loss/avg_raw_advantages,loss/avg_raw_advantages_abs,policy/final_loss,policy/log_ratio_abs_max,policy/log_ratio_abs_mean,policy/log_ratio_abs_p99,policy/log_ratio_abs_pos00,policy/log_ratio_abs_pos10,policy/log_ratio_abs_pos20,policy/log_ratio_abs_pos30,policy/log_ratio_abs_pos40,policy/log_ratio_abs_pos50,policy/log_ratio_abs_pos60,policy/log_ratio_abs_pos70,policy/log_ratio_abs_pos80,policy/log_ratio_abs_pos90,policy/n_tokens_dp_gt_10pct,policy/n_tokens_dp_gt_1pct,policy/n_tokens_dp_gt_50pct,policy/policy_entropy,policy/policy_loss,policy/policy_lr,policy/policy_update_steps,policy/ppo_clip_ratio,policy/raw_grad_norm,reward/avg_pass_at_8,reward/avg_raw_reward,system/process_rss_gb,system/process_vms_gb,system/ram_available_gb,system/ram_percent,system/ram_total_gb,system/ram_used_gb,timing/compute_advantages_and_returns,timing/convert_to_training_input,timing/fwd_logprobs_values_reward,timing/policy_train,timing/run_training,timing/step,timing/sync_weights,timing/train_critic_and_policy,timing/wait_for_generation_buffer,trainer/epoch,trainer/global_step,batch_errors/total_batches,batch_errors/total_instances,batch_errors/total_successful,batch_errors/total_failed,batch_errors/total_masked,batch_errors/avg_ContextLengthExceededError,batch_errors/total_ContextLengthExceededError,batch_errors/avg_InvalidChatHistory,batch_errors/total_InvalidChatHistory,batch_errors/avg_VerifierTimeoutError,batch_errors/total_VerifierTimeoutError,batch_errors/avg_AgentTimeoutError,batch_errors/total_AgentTimeoutError,batch_errors/avg_AgentSetupTimeoutError,batch_errors/total_AgentSetupTimeoutError,timing/cleanup_old_checkpoints,timing/save_checkpoints,timing/save_hf_model,batch_errors/avg_DaytonaAuthenticationError,batch_errors/total_DaytonaAuthenticationError,batch_errors/avg_DaytonaError,batch_errors/total_DaytonaError
|
||||
0.0,0,64,512,4,0.5312,0,0.2344,5278.6973,5088.7018,5517.2379,25239,1,3818.4836,0.5566,-0.004,0.1129,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1241,0.0,0.0,1.0,0.0,0.0,0.6481,0.5566,10.4951,50.4392,584.3245,31.9,857.9675,273.643,0.1302,3.8894,37.1114,202.1707,239.6029,4371.2665,22.4624,202.3605,4105.3115,0,29,52,416,374,16,39,0.11538461538461539,6,0.7884615384615384,41,0.019230769230769232,1.0,0.019230769230769232,1.0,0.038461538461538464,2.0,,,,,,,
|
||||
0.0,0,64,512,1,0.9844,0,0.9844,4944.7676,5091.439,4875.6466,31704,1,4244.5182,0.3203,0.008,0.0845,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1335,-0.0,0.0,1.0,0.0,0.0,0.4219,0.3203,12.8603,50.865,578.3782,32.6,857.9675,279.5893,0.0666,4.6428,32.9006,192.3422,225.5807,695.9013,22.4198,192.613,443.2517,0,30,57,456,414,19,35,0.08771929824561403,5,0.7017543859649122,40,0.12280701754385964,7.0,0.05263157894736842,3.0,,,10.7945,19.7607,6.6515,,,,
|
||||
0.0,0,64,512,2,1.7188,0,0.9844,4114.9512,3919.9412,4244.1136,31730,1,4194.079,0.3984,0.0198,0.0875,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.125,-0.0,0.0,1.0,0.0,0.0,0.5312,0.3984,13.8393,50.8987,577.0171,32.7,857.9675,280.9504,0.0636,4.4873,29.5848,193.0235,222.9079,1107.8622,24.0469,193.259,856.417,0,31,64,512,493,10,15,0.03125,2,0.203125,13,0.046875,3.0,,,0.015625,1.0,,,,,,,
|
||||
0.0,0,64,512,3,1.7656,0,0.8594,5001.7891,4406.6449,5429.1745,31817,1,4145.4506,0.418,-0.0054,0.0829,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.14,0.0,0.0,1.0,0.0,0.0,0.5,0.418,14.1603,51.2707,576.1389,32.8,857.9675,281.8287,0.0622,4.6552,38.8123,204.3063,243.3941,2021.8598,23.3742,204.5192,1750.4335,0,32,61,488,437,20,46,0.08196721311475409,5,0.639344262295082,39,0.06557377049180328,4.0,0.04918032786885246,3.0,0.01639344262295082,1.0,0.0081,7.8836,,,,,
|
||||
0.0,0,64,512,4,2.0312,0,0.9375,4818.6543,4762.1027,4841.2131,31726,1,5036.9689,0.2852,0.0145,0.1036,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1325,-0.0,0.0,1.0,0.0,0.0,0.4062,0.2852,14.1077,51.196,575.3264,32.9,857.9675,282.6411,0.0656,4.6631,33.8051,229.0206,263.108,1289.5434,22.5459,229.237,999.2225,0,33,61,488,465,10,21,0.11475409836065574,7,0.22950819672131148,14,,,0.03278688524590164,2.0,0.03278688524590164,2.0,,,,,,,
|
||||
0.0,0,64,512,4,1.75,0,0.9375,4567.793,4373.0586,4762.5273,31639,1,3628.674,0.5,0.0075,0.1492,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1348,-0.0,0.0,1.0,0.0,0.0,0.6406,0.5,14.1472,51.2188,573.8872,33.1,857.9675,284.0803,0.0665,4.5609,28.5178,173.7767,202.5827,1301.8005,22.2062,173.9981,1072.4469,0,34,59,472,441,17,23,0.13559322033898305,8,0.2542372881355932,15,0.13559322033898305,8.0,,,,,0.0083,8.411,,,,,
|
||||
0.0,0,64,512,3,1.9688,0,0.8906,5203.1992,4785.8119,5475.1742,31726,1,4610.7692,0.3945,0.0015,0.1334,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1392,-0.0,0.0,1.0,0.0,0.0,0.5781,0.3945,14.2407,51.2979,572.6302,33.3,857.9675,285.3373,0.0655,4.7104,36.7892,210.5074,247.6007,1785.0733,22.4758,210.7457,1510.2823,0,35,61,488,460,13,27,0.11475409836065574,7,0.32786885245901637,20,,,,,0.01639344262295082,1.0,,,7.3123,,,,
|
||||
0.0,0,64,512,4,1.9219,0,0.9219,5044.2012,5116.1058,4961.4202,31779,1,4480.1027,0.5352,-0.0054,0.0842,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.135,0.0,0.0,1.0,0.0,0.0,0.625,0.5352,14.2407,51.3575,572.3281,33.3,857.9675,285.6394,0.0627,4.8504,36.6108,213.1635,250.063,1218.0837,21.7718,213.3891,941.3942,0,36,60,480,441,17,39,0.18333333333333332,11,0.43333333333333335,26,,,0.03333333333333333,2.0,,,0.0077,8.4733,,,,,
|
||||
0.0,0,64,512,3,1.75,0,0.8906,4737.1152,4212.0859,5262.1445,30999,1,4007.6604,0.5,0.0051,0.1255,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1275,-0.0,0.0,1.0,0.0,0.0,0.625,0.5,14.3788,51.4694,571.8417,33.3,857.9675,286.1258,0.0673,4.553,31.2988,186.6306,218.2242,1048.2685,23.4353,186.8577,802.0521,0,37,64,512,479,17,31,0.15625,10,0.296875,19,0.015625,1.0,0.03125,2.0,0.015625,1.0,,,,,,,
|
||||
0.0,0,64,512,4,2.0781,0,0.9062,5100.459,5243.36,5008.8558,31758,1,4289.2275,0.3906,0.0276,0.1256,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1362,-0.0,0.0,1.0,0.0,0.0,0.5312,0.3906,14.4257,51.5521,570.7955,33.5,857.9675,287.172,0.0653,4.7056,32.7387,201.981,235.0431,1731.9565,21.626,202.2386,1470.5772,0,38,59,472,444,14,24,0.06779661016949153,4,0.3898305084745763,23,0.05084745762711865,3.0,0.03389830508474576,2.0,0.01694915254237288,1.0,0.0086,9.2793,,,,,
|
||||
0.0,0,64,512,4,1.7031,0,0.9531,5043.1562,4965.0356,5119.4672,31823,1,4607.3975,0.4941,0.0074,0.126,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1292,-0.0,0.0,1.0,0.0,0.0,0.625,0.4941,14.4556,51.6204,570.5818,33.5,857.9675,287.3857,0.0686,4.688,40.8057,219.6135,260.7417,1244.3571,27.5955,219.867,951.3271,0,39,61,488,452,14,36,0.08196721311475409,5,0.45901639344262296,28,,,0.04918032786885246,3.0,,,,,,,,,
|
||||
0.0,0,64,512,4,1.875,0,0.9219,4419.4766,4218.5217,4583.3759,31827,1,4168.3604,0.4492,0.0001,0.0823,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1279,0.0,0.0,1.0,0.0,0.0,0.5625,0.4492,14.7014,51.8757,569.6165,33.6,857.9675,288.3511,0.0662,4.5612,30.5522,192.2327,223.0663,1248.4529,24.1425,192.4475,996.6791,0,40,61,488,466,11,17,0.08196721311475409,5,0.18032786885245902,11,0.08196721311475409,5.0,0.01639344262295082,1.0,,,0.0084,8.4746,6.1621,,,,
|
||||
0.0,0,64,512,4,1.8281,0,0.875,4378.8047,4390.5302,4370.3165,20422,1,3374.562,0.4199,0.0026,0.1097,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1403,-0.0,0.0,1.0,0.0,0.0,0.5469,0.4199,14.6059,51.8282,569.7355,33.6,857.9675,288.2321,0.0477,3.2238,29.9727,163.6158,193.973,1448.2179,23.0063,163.9523,1228.0101,0,41,63,504,473,17,31,0.15873015873015872,10,0.3333333333333333,21,,,,,,,,,,,,,
|
||||
0.0,0,64,512,4,1.9688,0,0.8906,5440.5508,5273.5025,5545.0222,31794,1,4314.333,0.3848,-0.0019,0.0687,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1387,0.0,0.0,1.0,0.0,0.0,0.4688,0.3848,14.9666,52.2508,569.2278,33.7,857.9675,288.7397,0.065,4.781,34.9912,208.4372,243.726,1518.1162,21.834,208.6693,1247.7718,0,42,60,480,440,14,32,0.16666666666666666,10,0.35,21,0.13333333333333333,8.0,0.016666666666666666,1.0,,,0.0078,8.9859,,,,,
|
||||
0.0,0,64,512,4,1.5469,0,0.7969,4309.9902,4170.6036,4471.7257,31730,1,3959.7796,0.5371,0.017,0.0942,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1249,-0.0,0.0,1.0,0.0,0.0,0.625,0.5371,15.3689,52.4981,568.5267,33.7,857.9675,289.4408,0.0633,4.577,30.0185,183.5674,213.8767,1281.3491,21.6632,183.7946,1041.2281,0,43,63,504,468,14,27,0.06349206349206349,4,0.31746031746031744,20,0.15873015873015872,10.0,0.047619047619047616,3.0,,,,,,,,,
|
||||
0.0,0,64,512,4,1.7969,0,0.9219,4686.1582,4334.2927,4921.1173,31718,1,4093.6999,0.4004,0.0024,0.0855,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1345,-0.0,0.0,1.0,0.0,0.0,0.5,0.4004,15.3116,52.6276,568.5529,33.7,857.9675,289.4147,0.0649,4.5947,33.5947,192.7301,226.6108,1070.9411,23.6635,192.9508,816.0677,0,44,64,512,404,33,66,0.03125,2,0.203125,13,0.25,16.0,0.796875,51.0,0.40625,26.0,0.0082,8.7248,,,,,
|
||||
0.0,0,64,512,4,2.1875,0,0.9062,4488.0605,4766.2674,4327.9846,31776,1,4643.8272,0.3652,0.0198,0.102,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.115,-0.0,0.0,1.0,0.0,0.0,0.4688,0.3652,15.2923,52.5649,567.6223,33.8,857.9675,290.3453,0.0648,4.5433,38.2594,212.1464,250.6847,2427.0584,19.2772,212.3601,2152.5489,0,45,59,472,281,51,126,0.1016949152542373,6,0.0847457627118644,5,0.6949152542372882,41.0,2.2033898305084745,130.0,0.559322033898305,33.0,,,6.1293,,,,
|
||||
0.0,0,64,512,4,2.1406,0,0.9844,2831.2832,4734.6768,1934.2816,19126,1,3422.9025,0.3203,0.0741,0.1736,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0819,-0.0,0.0,1.0,0.0,0.0,0.5156,0.3203,15.0547,52.2833,567.6797,33.8,857.9675,290.2878,0.0433,2.9003,21.3716,139.1872,160.9597,2780.6994,19.1556,139.5444,2597.6796,0,46,57,456,174,56,114,0.05263157894736842,3,0.17543859649122806,10,2.5789473684210527,147.0,1.9473684210526316,111.0,0.7192982456140351,41.0,0.0085,55.7189,,0.03508771929824561,2.0,,
|
||||
0.0,0,64,512,4,2.0,0,0.9531,1948.959,3929.527,1614.3425,20028,1,3361.197,0.1445,0.0796,0.2262,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.057,-0.0,0.0,1.0,0.0,0.0,0.4375,0.1445,14.9425,52.2209,567.6288,33.8,857.9675,290.3387,0.0488,2.7631,25.3613,134.4423,160.065,4012.6647,18.8904,134.6545,3830.9431,0,47,60,480,251,53,87,0.06666666666666667,4,0.18333333333333332,11,1.8666666666666667,112.0,1.3333333333333333,80.0,0.6166666666666667,37.0,,,,,,0.016666666666666666,1.0
|
||||
0.0,0,64,512,3,1.5469,0,0.875,1713.2109,3120.7785,1084.9746,12999,1,2205.1024,0.3086,0.1337,0.1884,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0686,-0.0,0.0,1.0,0.0,0.0,0.5312,0.3086,14.818,52.0276,568.1573,33.8,857.9675,289.8102,0.0314,1.9562,15.2563,98.0173,113.4611,3641.5565,17.9428,98.173,3508.1929,0,48,64,512,309,57,117,0.171875,11,0.296875,19,1.34375,86.0,1.359375,87.0,0.09375,6.0,0.0082,8.6866,,0.015625,1.0,,
|
||||
0.0,0,64,512,7,1.625,0,0.9531,2370.7871,3573.7063,1978.1244,31763,1,3467.8194,0.2461,0.0916,0.1688,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0845,-0.0,0.0,1.0,0.0,0.0,0.4531,0.2461,15.1041,52.3137,567.8245,33.8,857.9675,290.143,0.065,4.2255,18.8418,139.4689,158.5744,3585.1599,19.0904,139.6672,3403.2676,0,49,39,312,191,35,70,0.07692307692307693,3,0.41025641025641024,16,1.1794871794871795,46.0,1.3076923076923077,51.0,0.10256410256410256,4.0,,,,0.02564102564102564,1.0,,
|
||||
|
18
training_logs/20260608_210110_metrics_job_639473.csv
Normal file
18
training_logs/20260608_210110_metrics_job_639473.csv
Normal file
@@ -0,0 +1,18 @@
|
||||
async/discard_rate,async/discarded_count,async/effective_batch_groups,async/effective_batch_samples,async/staleness_max,async/staleness_mean,async/staleness_min,async/staleness_ratio,generate/avg_num_tokens,generate/avg_tokens_non_zero_rewards,generate/avg_tokens_zero_rewards,generate/max_num_tokens,generate/min_num_tokens,generate/std_num_tokens,loss/avg_final_rewards,loss/avg_raw_advantages,loss/avg_raw_advantages_abs,policy/final_loss,policy/log_ratio_abs_max,policy/log_ratio_abs_mean,policy/log_ratio_abs_p99,policy/log_ratio_abs_pos00,policy/log_ratio_abs_pos10,policy/log_ratio_abs_pos20,policy/log_ratio_abs_pos30,policy/log_ratio_abs_pos40,policy/log_ratio_abs_pos50,policy/log_ratio_abs_pos60,policy/log_ratio_abs_pos70,policy/log_ratio_abs_pos80,policy/log_ratio_abs_pos90,policy/n_tokens_dp_gt_10pct,policy/n_tokens_dp_gt_1pct,policy/n_tokens_dp_gt_50pct,policy/policy_entropy,policy/policy_loss,policy/policy_lr,policy/policy_update_steps,policy/ppo_clip_ratio,policy/raw_grad_norm,reward/avg_pass_at_8,reward/avg_raw_reward,system/process_rss_gb,system/process_vms_gb,system/ram_available_gb,system/ram_percent,system/ram_total_gb,system/ram_used_gb,timing/compute_advantages_and_returns,timing/convert_to_training_input,timing/fwd_logprobs_values_reward,timing/policy_train,timing/run_training,timing/step,timing/sync_weights,timing/train_critic_and_policy,timing/wait_for_generation_buffer,trainer/epoch,trainer/global_step,batch_errors/total_batches,batch_errors/total_instances,batch_errors/total_successful,batch_errors/total_failed,batch_errors/total_masked,batch_errors/avg_InvalidChatHistory,batch_errors/total_InvalidChatHistory,batch_errors/avg_AgentSetupTimeoutError,batch_errors/total_AgentSetupTimeoutError,batch_errors/avg_AgentTimeoutError,batch_errors/total_AgentTimeoutError,batch_errors/avg_ContextLengthExceededError,batch_errors/total_ContextLengthExceededError,batch_errors/avg_VerifierTimeoutError,batch_errors/total_VerifierTimeoutError,timing/cleanup_old_checkpoints,timing/save_checkpoints,timing/save_hf_model,batch_errors/avg_DaytonaError,batch_errors/total_DaytonaError,batch_errors/avg_DaytonaAuthenticationError,batch_errors/total_DaytonaAuthenticationError
|
||||
0.0,0,64,512,0,0.0,0,0.0,2574.8984,3414.6373,2066.8433,14422,1,2612.5608,0.377,0.1212,0.1534,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0885,-0.0,0.0,1.0,0.0,0.0,0.5625,0.377,7.261,50.0898,573.4373,33.2,857.9675,284.5302,0.0455,2.2784,38.4903,117.0694,155.8179,9181.4558,19.7276,117.2818,9003.6315,0,49,60,480,400,37,48,0.21666666666666667,13,0.18333333333333332,11.0,0.38333333333333336,23.0,0.26666666666666666,16,0.4666666666666667,28.0,,,,,,,
|
||||
0.0,0,64,512,1,1.0,1,1.0,3732.3652,3769.9299,3715.7521,31808,1,4078.5665,0.3066,0.024,0.0998,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.117,-0.0,0.0,1.0,0.0,0.0,0.4531,0.3066,12.5825,50.4959,566.5182,34.0,857.9675,291.4493,0.0686,4.4722,26.4647,178.2315,205.0064,611.3836,22.5264,178.4728,379.3765,0,50,61,488,454,16,7,0.06557377049180328,4,0.03278688524590164,2.0,0.01639344262295082,1.0,0.01639344262295082,1,0.4098360655737705,25.0,73.2928,82.7069,20.5797,0.01639344262295082,1.0,,
|
||||
0.0,0,64,512,2,1.5312,0,0.875,4180.8438,3657.6311,4591.0279,17255,1,3125.6494,0.4395,0.0131,0.0883,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1311,-0.0,0.0,1.0,0.0,0.0,0.5312,0.4395,12.5143,50.4529,565.7725,34.1,857.9675,292.1951,0.0393,2.8336,28.712,150.3515,179.3291,1837.5348,23.3846,150.5774,1631.9841,0,51,63,504,477,13,26,0.2222222222222222,14,,,0.015873015873015872,1.0,0.14285714285714285,9,0.015873015873015872,1.0,,,,0.031746031746031744,2.0,,
|
||||
0.0,0,64,512,3,1.6719,0,0.8906,4772.627,4960.8151,4609.1642,31652,1,3769.5395,0.4648,0.014,0.0786,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1376,-0.0,0.0,1.0,0.0,0.0,0.5781,0.4648,12.8545,50.8127,566.1333,34.0,857.9675,291.8342,0.0654,5.1247,31.4543,178.3831,210.1787,1732.7267,22.7374,178.6587,1494.6839,0,52,61,488,431,19,52,0.639344262295082,39,,,0.11475409836065574,7.0,0.13114754098360656,8,0.06557377049180328,4.0,0.0085,8.3109,,0.06557377049180328,4.0,0.01639344262295082,1.0
|
||||
0.0,0,64,512,4,2.0156,0,0.9531,4045.3574,4180.3436,3982.3123,31771,1,3952.5092,0.3184,0.0176,0.1401,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1307,-0.0,0.0,1.0,0.0,0.0,0.5,0.3184,13.2115,51.1639,565.3646,34.1,857.9675,292.603,0.0634,4.4954,29.8155,172.7807,202.884,1262.3478,25.742,173.0047,1029.2236,0,53,65,520,492,13,26,0.27692307692307694,18,,,0.046153846153846156,3.0,0.03076923076923077,2,0.03076923076923077,2.0,,,,0.046153846153846156,3.0,,
|
||||
0.0,0,64,512,3,1.7031,0,0.8906,5088.5273,4488.8068,5495.5508,31800,1,4521.5771,0.4043,0.0049,0.0916,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1336,0.0,0.0,1.0,0.0,0.0,0.4844,0.4043,13.4076,51.4641,565.8401,34.0,857.9675,292.1274,0.0649,4.8001,35.4838,216.8179,252.5977,1903.4231,22.6662,217.0486,1623.3547,0,54,61,488,453,13,31,0.4262295081967213,26,,,0.01639344262295082,1.0,0.08196721311475409,5,0.06557377049180328,4.0,0.0084,7.7525,,,,,
|
||||
0.0,0,64,512,3,1.75,0,0.8906,4631.4766,5009.6139,4385.0774,31812,1,4169.8675,0.3945,0.0182,0.0826,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1331,-0.0,0.0,1.0,0.0,0.0,0.4844,0.3945,13.6974,51.6048,565.2818,34.1,857.9675,292.6857,0.0706,4.5962,31.4269,196.5331,228.3023,1296.0607,24.1405,196.8045,1039.0185,0,55,61,488,457,12,23,0.2786885245901639,17,,,0.03278688524590164,2.0,0.06557377049180328,4,0.13114754098360656,8.0,,,6.2341,0.03278688524590164,2.0,,
|
||||
0.0,0,64,512,3,1.875,0,0.9219,5294.6016,4989.8677,5472.9133,31704,1,4638.6034,0.3691,-0.0048,0.1111,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1325,0.0,0.0,1.0,0.0,0.0,0.4844,0.3691,13.7494,51.6116,565.5437,34.1,857.9675,292.4238,0.0867,4.6832,38.8346,224.59,263.8149,1565.5302,22.4001,224.8932,1274.6283,0,56,63,504,478,11,24,0.25396825396825395,16,,,0.031746031746031744,2.0,0.1111111111111111,7,0.031746031746031744,2.0,0.0081,7.9573,,,,,
|
||||
0.0,0,64,512,3,1.7188,0,0.9062,4950.9766,4560.3436,5262.1123,31843,1,4774.4938,0.4434,-0.0031,0.134,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1306,0.0,0.0,1.0,0.0,0.0,0.5781,0.4434,13.8882,51.8104,564.4661,34.2,857.9675,293.5014,0.0892,4.6537,34.0791,223.8635,258.2931,1726.4682,23.134,224.1245,1440.3826,0,57,63,504,466,17,31,0.38095238095238093,24,,,0.047619047619047616,3.0,0.06349206349206349,4,0.1111111111111111,7.0,,,,,,,
|
||||
0.0,0,64,512,3,1.6094,0,0.9375,4331.5684,4241.4459,4400.5586,31744,1,3845.2618,0.4336,0.0047,0.1354,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1298,-0.0,0.0,1.0,0.0,0.0,0.5781,0.4336,14.0239,51.9366,564.886,34.2,857.9675,293.0815,0.0936,4.5707,30.7528,183.0933,214.1963,1407.8184,24.3511,183.3495,1164.6952,0,58,63,504,475,13,29,0.3333333333333333,21,,,0.031746031746031744,2.0,0.07936507936507936,5,,,0.0073,7.6926,,0.015873015873015872,1.0,,
|
||||
0.0,0,64,512,3,1.5156,0,0.7812,4725.0371,3898.9237,5431.4239,31792,1,4496.5728,0.4609,0.0244,0.0844,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1322,-0.0,0.0,1.0,0.0,0.0,0.5625,0.4609,14.0273,51.9401,563.9426,34.3,857.9675,294.025,0.0865,4.6121,34.988,210.5671,245.9236,1490.2312,26.3782,210.8487,1213.311,0,59,61,488,465,10,22,0.3114754098360656,19,0.01639344262295082,1.0,0.08196721311475409,5.0,0.01639344262295082,1,0.06557377049180328,4.0,,,,,,,
|
||||
0.0,0,64,512,4,1.5781,0,0.8125,4552.8398,4327.259,4769.7778,31726,1,3951.5205,0.4902,0.0058,0.1092,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1243,-0.0,0.0,1.0,0.0,0.0,0.625,0.4902,14.0216,51.9988,563.3038,34.3,857.9675,294.6637,0.1176,4.6409,34.6955,189.8302,224.9402,1325.7249,23.1612,190.1267,1072.9765,0,60,58,464,413,16,40,0.5689655172413793,33,,,0.08620689655172414,5.0,0.13793103448275862,8,0.1896551724137931,11.0,0.0086,7.7063,6.7543,,,,
|
||||
0.0,0,64,512,4,1.6719,0,0.8438,4229.166,4585.0703,3873.2617,31580,1,3910.4259,0.5,0.0132,0.1619,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1226,-0.0,0.0,1.0,0.0,0.0,0.6562,0.5,14.0175,52.0583,562.8052,34.4,857.9675,295.1624,0.097,4.5178,31.3756,184.0054,215.7393,1404.1596,25.5141,184.2663,1158.3824,0,61,62,496,465,15,27,0.25806451612903225,16,,,0.06451612903225806,4.0,0.12903225806451613,8,0.06451612903225806,4.0,,,,,,,
|
||||
0.0,0,64,512,5,1.7656,0,0.8594,4619.9727,4131.9717,5074.8264,31689,1,4163.141,0.4824,-0.0109,0.0868,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1226,0.0,0.0,1.0,0.0,0.0,0.5625,0.4824,14.4709,52.4935,562.3414,34.5,857.9675,295.6262,0.1201,4.6276,38.4473,199.087,237.8926,1122.7358,23.8958,199.3249,856.3148,0,62,61,488,476,11,10,0.09836065573770492,6,0.01639344262295082,1.0,,,0.08196721311475409,5,0.01639344262295082,1.0,0.0085,7.8196,,,,,
|
||||
0.0,0,64,512,4,2.3594,0,0.9688,5134.4629,4500.9897,5524.1388,31785,1,3916.1129,0.3809,0.0077,0.1039,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1386,-0.0,0.0,1.0,0.0,0.0,0.4844,0.3809,14.3535,52.3757,561.5363,34.6,857.9675,296.4312,0.1201,4.8417,33.4082,196.2872,230.1101,1242.3513,24.6441,196.5814,982.7475,0,63,62,496,467,16,25,0.3225806451612903,20,,,0.03225806451612903,2.0,0.06451612903225806,4,0.06451612903225806,4.0,,,,,,,
|
||||
0.0,0,64,512,4,2.1562,0,0.9062,5462.8105,5057.0452,5720.7891,31766,1,5070.3185,0.3887,0.0044,0.069,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1309,-0.0,0.0,1.0,0.0,0.0,0.4844,0.3887,14.3912,52.3806,561.2214,34.6,857.9675,296.7462,0.1168,4.7044,38.1819,238.6575,277.2112,1523.0599,23.148,238.912,1217.9877,0,64,61,488,456,15,24,0.2786885245901639,17,,,0.03278688524590164,2.0,0.08196721311475409,5,0.13114754098360656,8.0,0.0084,9.6022,,,,,
|
||||
0.0,0,64,512,4,1.7188,0,0.8438,5144.3984,4651.443,5569.2364,31772,1,5404.6743,0.4629,0.0039,0.0697,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1304,-0.0,0.0,1.0,0.0,0.0,0.5469,0.4629,14.7312,52.7578,560.1469,34.7,857.9675,297.8206,0.1115,4.7518,42.0515,251.3932,293.8211,1513.4404,21.9402,251.6576,1192.9212,0,65,59,472,430,17,35,0.423728813559322,25,,,0.05084745762711865,3.0,0.11864406779661017,7,0.11864406779661017,7.0,,,6.0896,,,,
|
||||
|
17
training_logs/20260608_210110_metrics_job_639474.csv
Normal file
17
training_logs/20260608_210110_metrics_job_639474.csv
Normal file
@@ -0,0 +1,17 @@
|
||||
async/discard_rate,async/discarded_count,async/effective_batch_groups,async/effective_batch_samples,async/staleness_max,async/staleness_mean,async/staleness_min,async/staleness_ratio,generate/avg_num_tokens,generate/avg_tokens_non_zero_rewards,generate/avg_tokens_zero_rewards,generate/max_num_tokens,generate/min_num_tokens,generate/std_num_tokens,loss/avg_final_rewards,loss/avg_raw_advantages,loss/avg_raw_advantages_abs,policy/final_loss,policy/log_ratio_abs_max,policy/log_ratio_abs_mean,policy/log_ratio_abs_p99,policy/log_ratio_abs_pos00,policy/log_ratio_abs_pos10,policy/log_ratio_abs_pos20,policy/log_ratio_abs_pos30,policy/log_ratio_abs_pos40,policy/log_ratio_abs_pos50,policy/log_ratio_abs_pos60,policy/log_ratio_abs_pos70,policy/log_ratio_abs_pos80,policy/log_ratio_abs_pos90,policy/n_tokens_dp_gt_10pct,policy/n_tokens_dp_gt_1pct,policy/n_tokens_dp_gt_50pct,policy/policy_entropy,policy/policy_loss,policy/policy_lr,policy/policy_update_steps,policy/ppo_clip_ratio,policy/raw_grad_norm,reward/avg_pass_at_8,reward/avg_raw_reward,system/process_rss_gb,system/process_vms_gb,system/ram_available_gb,system/ram_percent,system/ram_total_gb,system/ram_used_gb,timing/compute_advantages_and_returns,timing/convert_to_training_input,timing/fwd_logprobs_values_reward,timing/policy_train,timing/run_training,timing/save_hf_model,timing/step,timing/sync_weights,timing/train_critic_and_policy,timing/wait_for_generation_buffer,trainer/epoch,trainer/global_step,batch_errors/total_batches,batch_errors/total_instances,batch_errors/total_successful,batch_errors/total_failed,batch_errors/total_masked,batch_errors/avg_ContextLengthExceededError,batch_errors/total_ContextLengthExceededError,batch_errors/avg_InvalidChatHistory,batch_errors/total_InvalidChatHistory,batch_errors/avg_AgentTimeoutError,batch_errors/total_AgentTimeoutError,timing/cleanup_old_checkpoints,timing/save_checkpoints,batch_errors/avg_VerifierTimeoutError,batch_errors/total_VerifierTimeoutError,batch_errors/avg_DaytonaAuthenticationError,batch_errors/total_DaytonaAuthenticationError,batch_errors/avg_AgentSetupTimeoutError,batch_errors/total_AgentSetupTimeoutError,batch_errors/avg_DaytonaError,batch_errors/total_DaytonaError
|
||||
0.0,0,64,512,4,0.5469,0,0.25,4862.9727,4513.1443,5350.1168,31704,1,3892.5586,0.582,0.0105,0.1354,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1296,-0.0,0.0,1.0,0.0,0.0,0.6727,0.582,12.6295,50.548,561.5298,34.6,857.9675,296.4377,0.1256,4.65,69.6756,200.9736,271.0452,36.4673,4741.4969,23.1178,201.2433,4442.6837,0,65,37,296,277,13,19,0.40540540540540543,15,0.8918918918918919,33,0.08108108108108109,3.0,,,,,,,,,,
|
||||
0.0,0,64,512,1,1.0,1,1.0,5066.0879,4639.2486,5299.4955,31799,1,5073.213,0.3535,0.0067,0.1205,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1309,-0.0,0.0,1.0,0.0,0.0,0.5,0.3535,12.7806,50.6748,627.205,26.9,857.9675,230.7626,0.0762,4.6149,37.9156,235.294,273.5463,,605.1264,24.0294,235.5538,302.928,0,66,60,480,440,16,29,0.15,9,0.25,15,0.13333333333333333,8.0,42.5487,50.8663,0.18333333333333332,11.0,,,,,,
|
||||
0.0,0,64,512,2,1.6719,0,0.9531,4734.041,5337.0905,4314.702,31845,1,3896.281,0.4102,0.0173,0.1281,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1301,-0.0,0.0,1.0,0.0,0.0,0.5156,0.4102,13.7043,50.7106,622.134,27.5,857.9675,235.8336,0.0668,4.5589,29.9015,188.6862,218.8452,,1183.017,23.4914,188.8764,936.117,0,67,63,504,355,47,91,0.1111111111111111,7,0.20634920634920634,13,1.1587301587301588,73.0,,,0.3333333333333333,21.0,0.031746031746031744,2.0,0.5555555555555556,35.0,,
|
||||
0.0,0,64,512,3,2.2969,0,0.9219,4292.0527,5313.4359,3989.5165,31752,1,4965.0243,0.2285,0.0361,0.1284,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1007,-0.0,0.0,1.0,0.0,0.0,0.3594,0.2285,13.7788,50.7601,620.5532,27.7,857.9675,237.4144,0.0656,4.7095,31.3857,214.771,246.5058,,5067.1626,19.1151,215.0541,4796.8269,0,68,61,488,276,57,125,0.04918032786885246,3,0.18032786885245902,11,1.8524590163934427,113.0,0.0088,8.3963,0.7868852459016393,48.0,0.03278688524590164,2.0,0.6065573770491803,37.0,0.01639344262295082,1.0
|
||||
0.0,0,64,512,4,2.0312,0,0.9844,2689.0547,4481.0945,2097.9143,16225,1,3499.4794,0.248,0.064,0.1772,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.078,-0.0,0.0,1.0,0.0,0.0,0.5,0.248,13.5363,50.52,618.4584,27.9,857.9675,239.5091,0.0562,2.4372,19.9013,140.6377,160.7959,,2845.5649,19.1755,140.8378,2663.1483,0,69,56,448,218,54,89,0.125,7,0.07142857142857142,4,1.8392857142857142,103.0,,,2.4464285714285716,137.0,,,0.30357142857142855,17.0,,
|
||||
0.0,0,64,512,3,1.8125,0,0.9531,1819.6953,3456.2381,1285.487,16720,1,2446.8912,0.2461,0.1768,0.2232,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0662,-0.0,0.0,1.0,0.0,0.0,0.5469,0.2461,13.6124,50.5903,620.05,27.7,857.9675,237.9175,0.0369,2.372,14.1649,102.8323,117.1881,8.0763,2600.3087,18.8825,102.9859,2461.8638,0,70,58,464,355,32,36,0.08620689655172414,5,0.13793103448275862,8,0.39655172413793105,23.0,0.0078,17.8975,1.3620689655172413,79.0,,,0.05172413793103448,3.0,0.017241379310344827,1.0
|
||||
0.0,0,64,512,3,1.4375,0,0.9219,3140.1328,3793.3929,2632.0417,31776,1,3649.7812,0.4375,0.0669,0.1491,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0942,-0.0,0.0,1.0,0.0,0.0,0.6094,0.4375,13.8658,50.8432,615.5933,28.2,857.9675,242.3742,0.0646,4.3413,26.1997,152.3269,178.7983,,1848.2381,19.4477,152.5336,1645.6483,0,71,54,432,346,33,32,0.1111111111111111,6,0.2222222222222222,12,0.42592592592592593,23.0,,,0.7962962962962963,43.0,0.018518518518518517,1.0,0.3333333333333333,18.0,0.018518518518518517,1.0
|
||||
0.0,0,64,512,4,1.7344,0,0.9062,3757.4688,3691.0625,3792.253,31727,1,3971.7082,0.3438,0.0297,0.0921,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1108,-0.0,0.0,1.0,0.0,0.0,0.4531,0.3438,13.8605,50.8377,617.5437,28.0,857.9675,240.4238,0.0696,4.3975,27.0854,166.2132,193.5822,,3286.0053,18.9345,166.4269,3069.0865,0,72,64,512,326,56,112,0.171875,11,0.1875,12,1.421875,91.0,0.008,82.4332,0.5625,36.0,0.015625,1.0,0.578125,37.0,0.015625,1.0
|
||||
0.0,0,64,512,4,1.9844,0,0.9375,3025.4746,4412.386,2628.2186,31668,1,3575.8498,0.2227,0.0329,0.1432,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0848,-0.0,0.0,1.0,0.0,0.0,0.3594,0.2227,13.8521,50.8289,615.4798,28.3,857.9675,242.4877,0.0891,4.3699,23.2891,149.6715,173.2579,,2698.0374,18.0666,149.8792,2502.3368,0,73,60,480,337,49,48,0.03333333333333333,2,0.2,12,0.6166666666666667,37.0,,,1.4166666666666667,85.0,0.016666666666666666,1.0,0.15,9.0,0.016666666666666666,1.0
|
||||
0.0,0,64,512,4,1.7031,0,0.9375,3082.4863,3622.4722,2871.1875,31787,1,3677.9858,0.2812,0.0609,0.1265,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0914,-0.0,0.0,1.0,0.0,0.0,0.4844,0.2812,13.8518,50.893,614.1329,28.4,857.9675,243.8346,0.0647,4.3386,23.0836,150.3708,173.7147,,1731.4236,19.0162,150.566,1534.3491,0,74,60,480,414,31,29,0.06666666666666667,4,0.3,18,0.1,6.0,0.0081,8.0639,0.5666666666666667,34.0,0.016666666666666666,1.0,0.05,3.0,0.016666666666666666,1.0
|
||||
0.0,0,64,512,4,1.8281,1,1.0,3949.6445,4464.3611,3450.7654,25386,1,3103.6693,0.4922,0.0245,0.1625,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1116,-0.0,0.0,1.0,0.0,0.0,0.6406,0.4922,13.816,50.7961,612.8476,28.6,857.9675,245.1199,0.0558,3.786,27.4207,148.2987,175.969,5.8565,1591.7732,20.678,148.4921,1391.3251,0,75,61,488,446,18,27,0.08196721311475409,5,0.32786885245901637,20,,,,,0.2459016393442623,15.0,,,,,0.03278688524590164,2.0
|
||||
0.0,0,64,512,3,2.2344,2,1.0,4071.6719,3826.4317,4208.0821,31779,1,3359.4605,0.3574,0.0041,0.1367,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.122,-0.0,0.0,1.0,0.0,0.0,0.5312,0.3574,14.0157,51.0362,611.0781,28.8,857.9675,246.8895,0.0659,4.5298,25.7335,158.3383,184.3538,,1915.8699,20.5904,158.5541,1706.3756,0,76,64,512,469,18,35,0.125,8,0.453125,29,0.046875,3.0,0.0081,8.0145,0.125,8.0,,,,,0.015625,1.0
|
||||
0.0,0,64,512,3,3.0,3,1.0,4338.9863,4198.9942,4409.1877,31653,1,3668.1638,0.334,-0.002,0.0705,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1292,0.0,0.0,1.0,0.0,0.0,0.4375,0.334,14.2828,51.3484,609.1068,29.0,857.9675,248.8608,0.0664,4.5496,30.742,170.001,201.0159,,1171.6576,21.017,170.2071,945.0538,0,77,64,512,430,36,63,0.640625,41,0.3125,20,0.03125,2.0,,,0.296875,19.0,,,,,,
|
||||
0.0,0,64,512,4,4.0,4,1.0,4903.0371,5346.2614,4811.0472,23634,1,4141.1011,0.1719,0.0046,0.11,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1267,-0.0,0.0,1.0,0.0,0.0,0.3438,0.1719,14.4365,51.4758,607.8784,29.1,857.9675,250.0891,0.0511,3.6578,32.3641,188.0312,220.6809,,1996.031,19.1079,188.2653,1752.5626,0,78,93,744,701,23,38,0.2903225806451613,27,0.08602150537634409,8,,,0.0078,10.0346,0.043010752688172046,4.0,,,0.010752688172043012,1.0,0.03225806451612903,3.0
|
||||
0.0,0,64,512,0,0.0,0,0.0,3705.6719,3357.3361,4578.8973,26629,1,2424.7377,0.7148,0.0124,0.095,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1105,-0.0,0.0,1.0,0.0,0.0,0.7812,0.7148,14.4194,51.4582,606.2361,29.3,857.9675,251.7314,0.0598,3.8365,25.5076,130.5443,156.312,,3207.1053,20.9174,130.7441,3026.017,1,79,60,480,452,12,21,0.1,6,0.16666666666666666,10,,,,,0.11666666666666667,7.0,,,,,0.1,6.0
|
||||
0.0,0,64,512,1,0.8281,0,0.8281,4801.8926,4507.3876,5101.0354,21141,1,3692.5927,0.5039,0.0145,0.122,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1225,-0.0,0.0,1.0,0.0,0.0,0.625,0.5039,14.3202,51.3997,605.6434,29.4,857.9675,252.3241,0.0464,3.3871,30.1146,181.7158,212.1128,5.9826,1758.6001,22.0893,181.9515,1521.007,1,80,64,512,455,31,55,0.078125,5,0.53125,34,0.09375,6.0,0.0078,11.1322,0.03125,2.0,,,,,0.171875,11.0
|
||||
|
2
training_logs/20260608_210110_metrics_job_639475.csv
Normal file
2
training_logs/20260608_210110_metrics_job_639475.csv
Normal file
@@ -0,0 +1,2 @@
|
||||
async/discard_rate,async/discarded_count,async/effective_batch_groups,async/effective_batch_samples,async/staleness_max,async/staleness_mean,async/staleness_min,async/staleness_ratio,generate/avg_num_tokens,generate/avg_tokens_non_zero_rewards,generate/avg_tokens_zero_rewards,generate/max_num_tokens,generate/min_num_tokens,generate/std_num_tokens,loss/avg_final_rewards,loss/avg_raw_advantages,loss/avg_raw_advantages_abs,policy/final_loss,policy/log_ratio_abs_max,policy/log_ratio_abs_mean,policy/log_ratio_abs_p99,policy/log_ratio_abs_pos00,policy/log_ratio_abs_pos10,policy/log_ratio_abs_pos20,policy/log_ratio_abs_pos30,policy/log_ratio_abs_pos40,policy/log_ratio_abs_pos50,policy/log_ratio_abs_pos60,policy/log_ratio_abs_pos70,policy/log_ratio_abs_pos80,policy/log_ratio_abs_pos90,policy/n_tokens_dp_gt_10pct,policy/n_tokens_dp_gt_1pct,policy/n_tokens_dp_gt_50pct,policy/policy_entropy,policy/policy_loss,policy/policy_lr,policy/policy_update_steps,policy/ppo_clip_ratio,policy/raw_grad_norm,reward/avg_pass_at_8,reward/avg_raw_reward,system/process_rss_gb,system/process_vms_gb,system/ram_available_gb,system/ram_percent,system/ram_total_gb,system/ram_used_gb,timing/cleanup_old_checkpoints,timing/compute_advantages_and_returns,timing/convert_to_training_input,timing/fwd_logprobs_values_reward,timing/policy_train,timing/run_training,timing/save_checkpoints,timing/step,timing/sync_weights,timing/train_critic_and_policy,timing/wait_for_generation_buffer,trainer/epoch,trainer/global_step,batch_errors/total_batches,batch_errors/total_instances,batch_errors/total_successful,batch_errors/total_failed,batch_errors/total_masked
|
||||
0.0,0,64,512,0,0.0,0,0.0,3726.2617,3403.3817,4792.5798,19853,1,2212.5652,0.7676,-0.0047,0.0761,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1104,0.0,0.0,1.0,0.0,0.0,0.8281,0.7676,8.8246,50.3819,593.4822,30.8,857.9675,264.4853,15.0154,0.125,3.2035,27.7363,124.398,152.4472,24.9398,2771.6856,22.7908,124.5855,2593.2438,1,82,9,72,72,0,0
|
||||
|
461
training_logs/20260608_210110_metrics_report.md
Normal file
461
training_logs/20260608_210110_metrics_report.md
Normal file
@@ -0,0 +1,461 @@
|
||||
# SkyRL Training Metrics Analysis
|
||||
|
||||
Generated from 5 log files
|
||||
|
||||
## Overview
|
||||
|
||||
| Log File | Total Steps | Metric Blocks | Final Reward (mean) | Final Reward (max) | Total Time (s) |
|
||||
|----------|-------------|---------------|---------------------|-------------------|----------------|
|
||||
| job_639470 | 28 | 28 | 0.4418 | 0.7793 | 41831.8 |
|
||||
| job_639471 | 49 | 21 | 0.3985 | 0.5566 | 40830.2 |
|
||||
| job_639473 | 65 | 17 | 0.4187 | 0.5000 | 32146.5 |
|
||||
| job_639474 | 80 | 16 | 0.3705 | 0.7148 | 38247.4 |
|
||||
| job_639475 | 82 | 1 | 0.7676 | 0.7676 | 2771.7 |
|
||||
|
||||
## Async Metrics
|
||||
|
||||
| | Mean | Std | Min | Max | Count |
|
||||
|:------------------------------|-----------:|---------:|------:|------:|--------:|
|
||||
| async/discard_rate | 0 | 0 | 0 | 0 | 83 |
|
||||
| async/discarded_count | 0 | 0 | 0 | 0 | 83 |
|
||||
| async/effective_batch_groups | 64 | 0 | 64 | 64 | 83 |
|
||||
| async/effective_batch_samples | 512 | 0 | 512 | 512 | 83 |
|
||||
| async/staleness_max | 3.3494 | 1.22408 | 0 | 7 | 83 |
|
||||
| async/staleness_mean | 1.71348 | 0.591401 | 0 | 4 | 83 |
|
||||
| async/staleness_min | 0.156627 | 0.614498 | 0 | 4 | 83 |
|
||||
| async/staleness_ratio | 0.851277 | 0.224556 | 0 | 1 | 83 |
|
||||
|
||||
## Generate Metrics
|
||||
|
||||
| | Mean | Std | Min | Max | Count |
|
||||
|:-------------------------------------|---------:|---------:|---------:|---------:|--------:|
|
||||
| generate/avg_num_tokens | 4458.85 | 862.392 | 1713.21 | 6032.85 | 83 |
|
||||
| generate/avg_tokens_non_zero_rewards | 4543.78 | 578.607 | 3120.78 | 5796.55 | 83 |
|
||||
| generate/avg_tokens_zero_rewards | 4503.36 | 1081.35 | 1084.97 | 6253.61 | 83 |
|
||||
| generate/max_num_tokens | 29333.3 | 4980.39 | 12999 | 31847 | 83 |
|
||||
| generate/min_num_tokens | 1 | 0 | 1 | 1 | 83 |
|
||||
| generate/std_num_tokens | 4009.1 | 673.611 | 2205.1 | 5542.49 | 83 |
|
||||
|
||||
## Loss Metrics
|
||||
|
||||
| | Mean | Std | Min | Max | Count |
|
||||
|:----------------------------|----------:|----------:|--------:|-------:|--------:|
|
||||
| loss/avg_final_rewards | 0.416272 | 0.112108 | 0.1445 | 0.7793 | 83 |
|
||||
| loss/avg_raw_advantages | 0.0180807 | 0.0320128 | -0.0169 | 0.1768 | 83 |
|
||||
| loss/avg_raw_advantages_abs | 0.115995 | 0.0339654 | 0.0566 | 0.2262 | 83 |
|
||||
|
||||
## Policy Metrics
|
||||
|
||||
| | Mean | Std | Min | Max | Count |
|
||||
|:----------------------------|---------:|----------:|-------:|-------:|--------:|
|
||||
| policy/final_loss | 0 | 0 | -0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_max | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_mean | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_p99 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos00 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos10 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos20 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos30 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos40 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos50 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos60 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos70 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos80 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/log_ratio_abs_pos90 | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/n_tokens_dp_gt_10pct | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/n_tokens_dp_gt_1pct | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/n_tokens_dp_gt_50pct | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/policy_entropy | 0.123722 | 0.0189336 | 0.057 | 0.1469 | 83 |
|
||||
| policy/policy_loss | 0 | 0 | -0 | 0 | 83 |
|
||||
| policy/policy_lr | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/policy_update_steps | 1 | 0 | 1 | 1 | 83 |
|
||||
| policy/ppo_clip_ratio | 0 | 0 | 0 | 0 | 83 |
|
||||
| policy/raw_grad_norm | 0 | 0 | 0 | 0 | 83 |
|
||||
|
||||
## Reward Metrics
|
||||
|
||||
| | Mean | Std | Min | Max | Count |
|
||||
|:----------------------|---------:|----------:|-------:|-------:|--------:|
|
||||
| reward/avg_pass_at_8 | 0.54622 | 0.0939465 | 0.3438 | 0.8438 | 83 |
|
||||
| reward/avg_raw_reward | 0.416272 | 0.112108 | 0.1445 | 0.7793 | 83 |
|
||||
|
||||
## System Metrics
|
||||
|
||||
| | Mean | Std | Min | Max | Count |
|
||||
|:------------------------|---------:|-------------:|---------:|---------:|--------:|
|
||||
| system/process_rss_gb | 13.911 | 1.27121 | 7.261 | 15.3689 | 83 |
|
||||
| system/process_vms_gb | 51.4434 | 0.673945 | 50.0898 | 52.7578 | 83 |
|
||||
| system/ram_available_gb | 575.799 | 19.8564 | 554.735 | 627.205 | 83 |
|
||||
| system/ram_percent | 32.8867 | 2.31763 | 26.9 | 35.3 | 83 |
|
||||
| system/ram_total_gb | 857.968 | 4.75679e-05 | 857.967 | 857.968 | 83 |
|
||||
| system/ram_used_gb | 282.169 | 19.8564 | 230.763 | 303.233 | 83 |
|
||||
|
||||
## Timing Metrics
|
||||
|
||||
| | Mean | Std | Min | Max | Count |
|
||||
|:--------------------------------------|-------------:|------------:|---------:|----------:|--------:|
|
||||
| timing/compute_advantages_and_returns | 0.0802976 | 0.026931 | 0.0314 | 0.1765 | 83 |
|
||||
| timing/convert_to_training_input | 4.31686 | 0.687255 | 1.9562 | 5.1247 | 83 |
|
||||
| timing/fwd_logprobs_values_reward | 32.4571 | 6.87409 | 14.1649 | 69.6756 | 83 |
|
||||
| timing/policy_train | 187.948 | 32.7461 | 98.0173 | 266.819 | 83 |
|
||||
| timing/run_training | 220.717 | 37.8457 | 113.461 | 308.663 | 83 |
|
||||
| timing/step | 1877.44 | 1202.31 | 605.126 | 9181.46 | 83 |
|
||||
| timing/sync_weights | 22.2263 | 2.01847 | 17.9428 | 27.5955 | 83 |
|
||||
| timing/train_critic_and_policy | 188.179 | 32.7593 | 98.173 | 267.078 | 83 |
|
||||
| timing/wait_for_generation_buffer | 1630.18 | 1218.35 | 302.928 | 9003.63 | 83 |
|
||||
| timing/cleanup_old_checkpoints | 3.75404 | 13.3134 | 0.007 | 73.2928 | 41 |
|
||||
| timing/save_checkpoints | 16.8853 | 18.0611 | 7.6926 | 82.7069 | 41 |
|
||||
| timing/save_hf_model | 9.786 | 7.68081 | 5.8565 | 36.4673 | 17 |
|
||||
|
||||
## Trainer Metrics
|
||||
|
||||
| | Mean | Std | Min | Max | Count |
|
||||
|:--------------------|-----------:|----------:|------:|------:|--------:|
|
||||
| trainer/epoch | 0.0361446 | 0.187784 | 0 | 1 | 83 |
|
||||
| trainer/global_step | 41.3976 | 23.4229 | 1 | 82 | 83 |
|
||||
|
||||
## Batch_Errors Metrics
|
||||
|
||||
| | Mean | Std | Min | Max | Count |
|
||||
|:----------------------------------------------|------------:|-------------:|-----------:|-------------:|--------:|
|
||||
| batch_errors/total_batches | 60.8554 | 10.6749 | 9 | 125 | 83 |
|
||||
| batch_errors/total_instances | 486.843 | 85.3996 | 72 | 1000 | 83 |
|
||||
| batch_errors/total_successful | 429.048 | 104.407 | 72 | 979 | 83 |
|
||||
| batch_errors/total_failed | 21.1807 | 13.1716 | 0 | 57 | 83 |
|
||||
| batch_errors/total_masked | 38.6506 | 26.3954 | 0 | 126 | 83 |
|
||||
| batch_errors/avg_VerifierTimeoutError | 0.309534 | 0.530966 | 0.015625 | 2.57895 | 72 |
|
||||
| batch_errors/total_VerifierTimeoutError | 18.1389 | 30.3984 | 1 | 147 | 72 |
|
||||
| batch_errors/avg_ContextLengthExceededError | 0.129623 | 0.092254 | 0.0163934 | 0.640625 | 82 |
|
||||
| batch_errors/total_ContextLengthExceededError | 7.93902 | 5.80623 | 1 | 41 | 82 |
|
||||
| batch_errors/avg_AgentTimeoutError | 0.298439 | 0.547499 | 0.008 | 2.20339 | 69 |
|
||||
| batch_errors/total_AgentTimeoutError | 17.5362 | 32.125 | 1 | 130 | 69 |
|
||||
| batch_errors/avg_InvalidChatHistory | 0.312427 | 0.162863 | 0.04 | 0.891892 | 82 |
|
||||
| batch_errors/total_InvalidChatHistory | 18.6585 | 8.79284 | 4 | 41 | 82 |
|
||||
| batch_errors/avg_AgentSetupTimeoutError | 0.168594 | 0.226175 | 0.008 | 0.719298 | 35 |
|
||||
| batch_errors/total_AgentSetupTimeoutError | 10.0857 | 13.6411 | 1 | 41 | 35 |
|
||||
| batch_errors/avg_DaytonaAuthenticationError | 0.0210748 | 0.00742164 | 0.015625 | 0.0350877 | 13 |
|
||||
| batch_errors/total_DaytonaAuthenticationError | 1.23077 | 0.438529 | 1 | 2 | 13 |
|
||||
| batch_errors/avg_RuntimeError | 0.0166667 | nan | 0.0166667 | 0.0166667 | 1 |
|
||||
| batch_errors/total_RuntimeError | 1 | nan | 1 | 1 | 1 |
|
||||
| batch_errors/avg_Timeout | 0.015873 | nan | 0.015873 | 0.015873 | 1 |
|
||||
| batch_errors/total_Timeout | 1 | nan | 1 | 1 | 1 |
|
||||
| batch_errors/avg_DaytonaError | 0.0377139 | 0.0400871 | 0.015625 | 0.171875 | 18 |
|
||||
| batch_errors/total_DaytonaError | 2.38889 | 2.54694 | 1 | 11 | 18 |
|
||||
|
||||
## Training Progression by Log
|
||||
|
||||
### job_639470
|
||||
|
||||
| Step | Reward | Pass@8 | KL | Loss | Step Time (s) | Gen Wait (s) |
|
||||
|------|--------|--------|-----|------|---------------|-------------|
|
||||
| 1 | 0.7793 | 0.8438 | 0.000000 | 0.0000 | 2948.9 | 2739.0 |
|
||||
| 2 | 0.4785 | 0.6250 | 0.000000 | -0.0000 | 1838.1 | 1543.4 |
|
||||
| 3 | 0.2559 | 0.3594 | 0.000000 | -0.0000 | 828.6 | 530.3 |
|
||||
| 4 | 0.5254 | 0.6250 | 0.000000 | 0.0000 | 1643.3 | 1379.0 |
|
||||
| 5 | 0.3672 | 0.4844 | 0.000000 | -0.0000 | 1592.0 | 1316.7 |
|
||||
| 6 | 0.4902 | 0.5781 | 0.000000 | -0.0000 | 968.7 | 657.1 |
|
||||
| 7 | 0.4492 | 0.5625 | 0.000000 | 0.0000 | 1394.4 | 1111.7 |
|
||||
| 8 | 0.4629 | 0.6250 | 0.000000 | 0.0000 | 1826.3 | 1488.3 |
|
||||
| 9 | 0.4219 | 0.5625 | 0.000000 | 0.0000 | 1383.6 | 1101.3 |
|
||||
| 10 | 0.4609 | 0.5938 | 0.000000 | -0.0000 | 1131.1 | 858.3 |
|
||||
| 11 | 0.5020 | 0.6719 | 0.000000 | -0.0000 | 1502.1 | 1254.5 |
|
||||
| 12 | 0.3535 | 0.4375 | 0.000000 | -0.0000 | 1268.0 | 1001.2 |
|
||||
| 13 | 0.4434 | 0.5938 | 0.000000 | 0.0000 | 1478.8 | 1216.7 |
|
||||
| 14 | 0.4199 | 0.6094 | 0.000000 | -0.0000 | 1529.1 | 1277.9 |
|
||||
| 15 | 0.5527 | 0.6250 | 0.000000 | -0.0000 | 1397.2 | 1161.4 |
|
||||
| 16 | 0.3379 | 0.5000 | 0.000000 | -0.0000 | 2136.4 | 1877.7 |
|
||||
| 17 | 0.3711 | 0.4531 | 0.000000 | -0.0000 | 1116.8 | 883.8 |
|
||||
| 18 | 0.4590 | 0.6094 | 0.000000 | -0.0000 | 2036.8 | 1792.5 |
|
||||
| 19 | 0.3379 | 0.4688 | 0.000000 | 0.0000 | 1344.7 | 1086.8 |
|
||||
| 20 | 0.4277 | 0.5000 | 0.000000 | -0.0000 | 1843.2 | 1606.6 |
|
||||
| 21 | 0.3633 | 0.5156 | 0.000000 | 0.0000 | 1421.9 | 1169.5 |
|
||||
| 22 | 0.3945 | 0.5312 | 0.000000 | 0.0000 | 1494.0 | 1237.3 |
|
||||
| 23 | 0.4062 | 0.5156 | 0.000000 | -0.0000 | 1294.0 | 1041.8 |
|
||||
| 24 | 0.3438 | 0.4531 | 0.000000 | -0.0000 | 1641.5 | 1393.6 |
|
||||
| 25 | 0.5234 | 0.6406 | 0.000000 | -0.0000 | 1090.5 | 876.4 |
|
||||
| 26 | 0.5078 | 0.5781 | 0.000000 | -0.0000 | 1169.9 | 878.8 |
|
||||
| 27 | 0.5137 | 0.6406 | 0.000000 | -0.0000 | 1096.8 | 860.3 |
|
||||
| 28 | 0.4199 | 0.5469 | 0.000000 | 0.0000 | 1415.0 | 1171.5 |
|
||||
|
||||
### job_639471
|
||||
|
||||
| Step | Reward | Pass@8 | KL | Loss | Step Time (s) | Gen Wait (s) |
|
||||
|------|--------|--------|-----|------|---------------|-------------|
|
||||
| 29 | 0.5566 | 0.6481 | 0.000000 | 0.0000 | 4371.3 | 4105.3 |
|
||||
| 30 | 0.3203 | 0.4219 | 0.000000 | -0.0000 | 695.9 | 443.3 |
|
||||
| 31 | 0.3984 | 0.5312 | 0.000000 | -0.0000 | 1107.9 | 856.4 |
|
||||
| 32 | 0.4180 | 0.5000 | 0.000000 | 0.0000 | 2021.9 | 1750.4 |
|
||||
| 33 | 0.2852 | 0.4062 | 0.000000 | -0.0000 | 1289.5 | 999.2 |
|
||||
| 34 | 0.5000 | 0.6406 | 0.000000 | -0.0000 | 1301.8 | 1072.4 |
|
||||
| 35 | 0.3945 | 0.5781 | 0.000000 | -0.0000 | 1785.1 | 1510.3 |
|
||||
| 36 | 0.5352 | 0.6250 | 0.000000 | 0.0000 | 1218.1 | 941.4 |
|
||||
| 37 | 0.5000 | 0.6250 | 0.000000 | -0.0000 | 1048.3 | 802.1 |
|
||||
| 38 | 0.3906 | 0.5312 | 0.000000 | -0.0000 | 1732.0 | 1470.6 |
|
||||
| 39 | 0.4941 | 0.6250 | 0.000000 | -0.0000 | 1244.4 | 951.3 |
|
||||
| 40 | 0.4492 | 0.5625 | 0.000000 | 0.0000 | 1248.5 | 996.7 |
|
||||
| 41 | 0.4199 | 0.5469 | 0.000000 | -0.0000 | 1448.2 | 1228.0 |
|
||||
| 42 | 0.3848 | 0.4688 | 0.000000 | 0.0000 | 1518.1 | 1247.8 |
|
||||
| 43 | 0.5371 | 0.6250 | 0.000000 | -0.0000 | 1281.3 | 1041.2 |
|
||||
| 44 | 0.4004 | 0.5000 | 0.000000 | -0.0000 | 1070.9 | 816.1 |
|
||||
| 45 | 0.3652 | 0.4688 | 0.000000 | -0.0000 | 2427.1 | 2152.5 |
|
||||
| 46 | 0.3203 | 0.5156 | 0.000000 | -0.0000 | 2780.7 | 2597.7 |
|
||||
| 47 | 0.1445 | 0.4375 | 0.000000 | -0.0000 | 4012.7 | 3830.9 |
|
||||
| 48 | 0.3086 | 0.5312 | 0.000000 | -0.0000 | 3641.6 | 3508.2 |
|
||||
| 49 | 0.2461 | 0.4531 | 0.000000 | -0.0000 | 3585.2 | 3403.3 |
|
||||
|
||||
### job_639473
|
||||
|
||||
| Step | Reward | Pass@8 | KL | Loss | Step Time (s) | Gen Wait (s) |
|
||||
|------|--------|--------|-----|------|---------------|-------------|
|
||||
| 49 | 0.3770 | 0.5625 | 0.000000 | -0.0000 | 9181.5 | 9003.6 |
|
||||
| 50 | 0.3066 | 0.4531 | 0.000000 | -0.0000 | 611.4 | 379.4 |
|
||||
| 51 | 0.4395 | 0.5312 | 0.000000 | -0.0000 | 1837.5 | 1632.0 |
|
||||
| 52 | 0.4648 | 0.5781 | 0.000000 | -0.0000 | 1732.7 | 1494.7 |
|
||||
| 53 | 0.3184 | 0.5000 | 0.000000 | -0.0000 | 1262.3 | 1029.2 |
|
||||
| 54 | 0.4043 | 0.4844 | 0.000000 | 0.0000 | 1903.4 | 1623.4 |
|
||||
| 55 | 0.3945 | 0.4844 | 0.000000 | -0.0000 | 1296.1 | 1039.0 |
|
||||
| 56 | 0.3691 | 0.4844 | 0.000000 | 0.0000 | 1565.5 | 1274.6 |
|
||||
| 57 | 0.4434 | 0.5781 | 0.000000 | 0.0000 | 1726.5 | 1440.4 |
|
||||
| 58 | 0.4336 | 0.5781 | 0.000000 | -0.0000 | 1407.8 | 1164.7 |
|
||||
| 59 | 0.4609 | 0.5625 | 0.000000 | -0.0000 | 1490.2 | 1213.3 |
|
||||
| 60 | 0.4902 | 0.6250 | 0.000000 | -0.0000 | 1325.7 | 1073.0 |
|
||||
| 61 | 0.5000 | 0.6562 | 0.000000 | -0.0000 | 1404.2 | 1158.4 |
|
||||
| 62 | 0.4824 | 0.5625 | 0.000000 | 0.0000 | 1122.7 | 856.3 |
|
||||
| 63 | 0.3809 | 0.4844 | 0.000000 | -0.0000 | 1242.4 | 982.7 |
|
||||
| 64 | 0.3887 | 0.4844 | 0.000000 | -0.0000 | 1523.1 | 1218.0 |
|
||||
| 65 | 0.4629 | 0.5469 | 0.000000 | -0.0000 | 1513.4 | 1192.9 |
|
||||
|
||||
### job_639474
|
||||
|
||||
| Step | Reward | Pass@8 | KL | Loss | Step Time (s) | Gen Wait (s) |
|
||||
|------|--------|--------|-----|------|---------------|-------------|
|
||||
| 65 | 0.5820 | 0.6727 | 0.000000 | -0.0000 | 4741.5 | 4442.7 |
|
||||
| 66 | 0.3535 | 0.5000 | 0.000000 | -0.0000 | 605.1 | 302.9 |
|
||||
| 67 | 0.4102 | 0.5156 | 0.000000 | -0.0000 | 1183.0 | 936.1 |
|
||||
| 68 | 0.2285 | 0.3594 | 0.000000 | -0.0000 | 5067.2 | 4796.8 |
|
||||
| 69 | 0.2480 | 0.5000 | 0.000000 | -0.0000 | 2845.6 | 2663.1 |
|
||||
| 70 | 0.2461 | 0.5469 | 0.000000 | -0.0000 | 2600.3 | 2461.9 |
|
||||
| 71 | 0.4375 | 0.6094 | 0.000000 | -0.0000 | 1848.2 | 1645.6 |
|
||||
| 72 | 0.3438 | 0.4531 | 0.000000 | -0.0000 | 3286.0 | 3069.1 |
|
||||
| 73 | 0.2227 | 0.3594 | 0.000000 | -0.0000 | 2698.0 | 2502.3 |
|
||||
| 74 | 0.2812 | 0.4844 | 0.000000 | -0.0000 | 1731.4 | 1534.3 |
|
||||
| 75 | 0.4922 | 0.6406 | 0.000000 | -0.0000 | 1591.8 | 1391.3 |
|
||||
| 76 | 0.3574 | 0.5312 | 0.000000 | -0.0000 | 1915.9 | 1706.4 |
|
||||
| 77 | 0.3340 | 0.4375 | 0.000000 | 0.0000 | 1171.7 | 945.1 |
|
||||
| 78 | 0.1719 | 0.3438 | 0.000000 | -0.0000 | 1996.0 | 1752.6 |
|
||||
| 79 | 0.7148 | 0.7812 | 0.000000 | -0.0000 | 3207.1 | 3026.0 |
|
||||
| 80 | 0.5039 | 0.6250 | 0.000000 | -0.0000 | 1758.6 | 1521.0 |
|
||||
|
||||
### job_639475
|
||||
|
||||
| Step | Reward | Pass@8 | KL | Loss | Step Time (s) | Gen Wait (s) |
|
||||
|------|--------|--------|-----|------|---------------|-------------|
|
||||
| 82 | 0.7676 | 0.8281 | 0.000000 | 0.0000 | 2771.7 | 2593.2 |
|
||||
|
||||
## Timing Analysis
|
||||
|
||||
### Average Time Breakdown (% of step time)
|
||||
|
||||
| Component | Avg % of Step Time |
|
||||
|-----------|-------------------|
|
||||
| wait_for_generation_buffer | 83.2% |
|
||||
| run_training | 15.0% |
|
||||
| train_critic_and_policy | 12.8% |
|
||||
| policy_train | 12.8% |
|
||||
| fwd_logprobs_values_reward | 2.2% |
|
||||
| sync_weights | 1.5% |
|
||||
| save_checkpoints | 1.3% |
|
||||
| save_hf_model | 0.7% |
|
||||
| cleanup_old_checkpoints | 0.5% |
|
||||
| convert_to_training_input | 0.3% |
|
||||
| compute_advantages_and_returns | 0.0% |
|
||||
|
||||
## Cross-Log Comparison
|
||||
|
||||
| Log | Avg Reward | Pass@8 | Step Time (s) | Gen Wait Time (s) | Avg Tokens | Staleness |
|
||||
|-----|------|------|------|------|------|------|
|
||||
| job_639470 | 0.4418 | 0.5625 | 1493.9938 | 1232.6167 | 4864.4295 | 1.7768 |
|
||||
| job_639471 | 0.3985 | 0.5353 | 1944.2947 | 1701.1955 | 4306.7650 | 1.7485 |
|
||||
| job_639473 | 0.4187 | 0.5386 | 1890.9678 | 1633.8600 | 4557.1723 | 1.6259 |
|
||||
| job_639474 | 0.3705 | 0.5225 | 2390.4636 | 2168.5831 | 3890.0232 | 1.7568 |
|
||||
| job_639475 | 0.7676 | 0.8281 | 2771.6856 | 2593.2438 | 3726.2617 | 0.0000 |
|
||||
|
||||
## vLLM Inference Engine Analysis
|
||||
|
||||
Metrics from vLLM stat loggers (V1LoggingStatLoggerFixed).
|
||||
|
||||
> **Note**: Ray deduplicates similar log messages with `[repeated Nx across cluster]`,
|
||||
> so we typically capture stats from one engine per timestamp. The stats shown are
|
||||
> **per-engine** values. Multiply by num_inference_engines for cluster-wide estimates.
|
||||
|
||||
### Summary by Log (Per-Engine Stats)
|
||||
|
||||
| Log | Avg Running/Engine | Avg Waiting/Engine | Avg Gen Throughput/Engine | Avg KV Cache % | Avg Prefix Hit % |
|
||||
|-----|-------------------|-------------------|--------------------------|----------------|------------------|
|
||||
| job_632522 | 8.5 | 0.0 | 380.3 tok/s | 35.3% | 86.8% |
|
||||
| job_632523 | 9.7 | 0.0 | 433.1 tok/s | 40.9% | 87.6% |
|
||||
| job_632524 | 10.4 | 0.0 | 457.7 tok/s | 43.2% | 86.4% |
|
||||
| job_632525 | 9.1 | 0.0 | 404.4 tok/s | 37.4% | 88.5% |
|
||||
| job_639470 | 11.5 | 0.4 | 492.9 tok/s | 51.0% | 73.8% |
|
||||
| job_639471 | 8.8 | 0.4 | 374.1 tok/s | 39.0% | 72.1% |
|
||||
| job_639473 | 10.4 | 0.3 | 443.9 tok/s | 46.0% | 75.4% |
|
||||
| job_639474 | 7.2 | 0.1 | 315.1 tok/s | 31.4% | 81.6% |
|
||||
| job_639475 | 7.6 | 0.0 | 338.0 tok/s | 31.2% | 84.6% |
|
||||
| job_639476 | 3.4 | 0.0 | 150.5 tok/s | 8.7% | 67.5% |
|
||||
|
||||
### Utilization Analysis (Per-Engine)
|
||||
|
||||
Key indicators of inference engine utilization:
|
||||
|
||||
- **Running requests/engine**: Concurrent requests being processed by each engine
|
||||
- **Waiting requests**: Requests queued (0 = engine not saturated, has spare capacity)
|
||||
- **Generation throughput**: Decode tokens/sec per engine
|
||||
- 8B model on H100 can do **1000+ tok/s** when saturated
|
||||
- If seeing <300 tok/s with 0 waiting, engine is **starved for requests**
|
||||
|
||||
#### job_632522
|
||||
|
||||
- **Running requests/engine**: avg=8.5, max=21
|
||||
- **Waiting requests**: avg=0.0, max=0
|
||||
- **Generation throughput/engine**: avg=380.3 tok/s, max=731.5 tok/s
|
||||
- **KV cache usage**: avg=35.3%
|
||||
- **Prefix cache hit rate**: avg=86.8%
|
||||
- ℹ️ **Moderate utilization**
|
||||
|
||||
#### job_632523
|
||||
|
||||
- **Running requests/engine**: avg=9.7, max=21
|
||||
- **Waiting requests**: avg=0.0, max=1
|
||||
- **Generation throughput/engine**: avg=433.1 tok/s, max=779.6 tok/s
|
||||
- **KV cache usage**: avg=40.9%
|
||||
- **Prefix cache hit rate**: avg=87.6%
|
||||
- ✅ **Well-utilized**: Engines saturated (waiting > 0)
|
||||
|
||||
#### job_632524
|
||||
|
||||
- **Running requests/engine**: avg=10.4, max=24
|
||||
- **Waiting requests**: avg=0.0, max=8
|
||||
- **Generation throughput/engine**: avg=457.7 tok/s, max=1016.5 tok/s
|
||||
- **KV cache usage**: avg=43.2%
|
||||
- **Prefix cache hit rate**: avg=86.4%
|
||||
- ✅ **Well-utilized**: Engines saturated (waiting > 0)
|
||||
|
||||
#### job_632525
|
||||
|
||||
- **Running requests/engine**: avg=9.1, max=24
|
||||
- **Waiting requests**: avg=0.0, max=0
|
||||
- **Generation throughput/engine**: avg=404.4 tok/s, max=845.3 tok/s
|
||||
- **KV cache usage**: avg=37.4%
|
||||
- **Prefix cache hit rate**: avg=88.5%
|
||||
- ℹ️ **Moderate utilization**
|
||||
|
||||
#### job_639470
|
||||
|
||||
- **Running requests/engine**: avg=11.5, max=32
|
||||
- **Waiting requests**: avg=0.4, max=31
|
||||
- **Generation throughput/engine**: avg=492.9 tok/s, max=1265.8 tok/s
|
||||
- **KV cache usage**: avg=51.0%
|
||||
- **Prefix cache hit rate**: avg=73.8%
|
||||
- ✅ **Well-utilized**: Engines saturated (waiting > 0)
|
||||
|
||||
#### job_639471
|
||||
|
||||
- **Running requests/engine**: avg=8.8, max=24
|
||||
- **Waiting requests**: avg=0.4, max=39
|
||||
- **Generation throughput/engine**: avg=374.1 tok/s, max=939.4 tok/s
|
||||
- **KV cache usage**: avg=39.0%
|
||||
- **Prefix cache hit rate**: avg=72.1%
|
||||
- ✅ **Well-utilized**: Engines saturated (waiting > 0)
|
||||
|
||||
#### job_639473
|
||||
|
||||
- **Running requests/engine**: avg=10.4, max=24
|
||||
- **Waiting requests**: avg=0.3, max=22
|
||||
- **Generation throughput/engine**: avg=443.9 tok/s, max=958.4 tok/s
|
||||
- **KV cache usage**: avg=46.0%
|
||||
- **Prefix cache hit rate**: avg=75.4%
|
||||
- ✅ **Well-utilized**: Engines saturated (waiting > 0)
|
||||
|
||||
#### job_639474
|
||||
|
||||
- **Running requests/engine**: avg=7.2, max=24
|
||||
- **Waiting requests**: avg=0.1, max=16
|
||||
- **Generation throughput/engine**: avg=315.1 tok/s, max=974.2 tok/s
|
||||
- **KV cache usage**: avg=31.4%
|
||||
- **Prefix cache hit rate**: avg=81.6%
|
||||
- ✅ **Well-utilized**: Engines saturated (waiting > 0)
|
||||
|
||||
#### job_639475
|
||||
|
||||
- **Running requests/engine**: avg=7.6, max=20
|
||||
- **Waiting requests**: avg=0.0, max=0
|
||||
- **Generation throughput/engine**: avg=338.0 tok/s, max=782.0 tok/s
|
||||
- **KV cache usage**: avg=31.2%
|
||||
- **Prefix cache hit rate**: avg=84.6%
|
||||
- ℹ️ **Moderate utilization**
|
||||
|
||||
#### job_639476
|
||||
|
||||
- **Running requests/engine**: avg=3.4, max=9
|
||||
- **Waiting requests**: avg=0.0, max=0
|
||||
- **Generation throughput/engine**: avg=150.5 tok/s, max=425.9 tok/s
|
||||
- **KV cache usage**: avg=8.7%
|
||||
- **Prefix cache hit rate**: avg=67.5%
|
||||
- ⚠️ **Underutilized**: Engines starved for requests (0 waiting, avg 3.4 running)
|
||||
- Bottleneck is likely upstream (environment execution, not inference)
|
||||
|
||||
## Trial-Level Analysis (from result.json)
|
||||
|
||||
Total trials parsed: 46308
|
||||
|
||||
### Turn Count Statistics
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Mean | 5.4 |
|
||||
| Median | 5.0 |
|
||||
| Std | 2.6 |
|
||||
| Min | 1 |
|
||||
| Max | 137 |
|
||||
| Count | 45918 |
|
||||
|
||||
### Exception Distribution
|
||||
|
||||
| Exception Type | Count | % |
|
||||
|---------------|-------|---|
|
||||
| No exception | 31188 | 67.3% |
|
||||
| AgentTimeoutError | 12483 | 27.0% |
|
||||
| VerifierTimeoutError | 1536 | 3.3% |
|
||||
| ContextLengthExceededError | 671 | 1.4% |
|
||||
| AgentSetupTimeoutError | 366 | 0.8% |
|
||||
| DaytonaError | 46 | 0.1% |
|
||||
| DaytonaAuthenticationError | 16 | 0.0% |
|
||||
| Timeout | 1 | 0.0% |
|
||||
| RuntimeError | 1 | 0.0% |
|
||||
|
||||
### Turn Count by Exception Type
|
||||
|
||||
| Exception Type | Mean Turns | Median Turns | Count |
|
||||
|---------------|-----------|-------------|-------|
|
||||
| RuntimeError | 13.0 | 13.0 | 1 |
|
||||
| No exception | 6.0 | 5.0 | 31188 |
|
||||
| VerifierTimeoutError | 5.7 | 5.0 | 1536 |
|
||||
| ContextLengthExceededError | 4.8 | 4.0 | 671 |
|
||||
| AgentTimeoutError | 3.9 | 4.0 | 12483 |
|
||||
| DaytonaAuthenticationError | 3.3 | 3.0 | 16 |
|
||||
| DaytonaError | 3.1 | 2.5 | 22 |
|
||||
| Timeout | 2.0 | 2.0 | 1 |
|
||||
|
||||
### Turn Count by Outcome
|
||||
|
||||
| Outcome | Mean Turns | Median Turns | Count |
|
||||
|---------|-----------|-------------|-------|
|
||||
| Success | 5.6 | 5.0 | 19074 |
|
||||
| Failure | 5.3 | 5.0 | 23341 |
|
||||
|
||||
### Reward Summary
|
||||
|
||||
- Mean reward: 0.4497
|
||||
- Success rate: 45.0%
|
||||
- Trials with reward data: 42415
|
||||
|
||||
84
training_logs/20260608_210110_metrics_table.csv
Normal file
84
training_logs/20260608_210110_metrics_table.csv
Normal file
@@ -0,0 +1,84 @@
|
||||
log_file,async/discard_rate,async/discarded_count,async/effective_batch_groups,async/effective_batch_samples,async/staleness_max,async/staleness_mean,async/staleness_min,async/staleness_ratio,generate/avg_num_tokens,generate/avg_tokens_non_zero_rewards,generate/avg_tokens_zero_rewards,generate/max_num_tokens,generate/min_num_tokens,generate/std_num_tokens,loss/avg_final_rewards,loss/avg_raw_advantages,loss/avg_raw_advantages_abs,policy/final_loss,policy/log_ratio_abs_max,policy/log_ratio_abs_mean,policy/log_ratio_abs_p99,policy/log_ratio_abs_pos00,policy/log_ratio_abs_pos10,policy/log_ratio_abs_pos20,policy/log_ratio_abs_pos30,policy/log_ratio_abs_pos40,policy/log_ratio_abs_pos50,policy/log_ratio_abs_pos60,policy/log_ratio_abs_pos70,policy/log_ratio_abs_pos80,policy/log_ratio_abs_pos90,policy/n_tokens_dp_gt_10pct,policy/n_tokens_dp_gt_1pct,policy/n_tokens_dp_gt_50pct,policy/policy_entropy,policy/policy_loss,policy/policy_lr,policy/policy_update_steps,policy/ppo_clip_ratio,policy/raw_grad_norm,reward/avg_pass_at_8,reward/avg_raw_reward,system/process_rss_gb,system/process_vms_gb,system/ram_available_gb,system/ram_percent,system/ram_total_gb,system/ram_used_gb,timing/compute_advantages_and_returns,timing/convert_to_training_input,timing/fwd_logprobs_values_reward,timing/policy_train,timing/run_training,timing/step,timing/sync_weights,timing/train_critic_and_policy,timing/wait_for_generation_buffer,trainer/epoch,trainer/global_step,batch_errors/total_batches,batch_errors/total_instances,batch_errors/total_successful,batch_errors/total_failed,batch_errors/total_masked,batch_errors/avg_VerifierTimeoutError,batch_errors/total_VerifierTimeoutError,batch_errors/avg_ContextLengthExceededError,batch_errors/total_ContextLengthExceededError,batch_errors/avg_AgentTimeoutError,batch_errors/total_AgentTimeoutError,batch_errors/avg_InvalidChatHistory,batch_errors/total_InvalidChatHistory,batch_errors/avg_AgentSetupTimeoutError,batch_errors/total_AgentSetupTimeoutError,timing/cleanup_old_checkpoints,timing/save_checkpoints,timing/save_hf_model,batch_errors/avg_DaytonaAuthenticationError,batch_errors/total_DaytonaAuthenticationError,batch_errors/avg_RuntimeError,batch_errors/total_RuntimeError,batch_errors/avg_Timeout,batch_errors/total_Timeout,batch_errors/avg_DaytonaError,batch_errors/total_DaytonaError,global_step
|
||||
job_639470,0.0,0,64,512,0,0.0,0,0.0,4204.6133,3980.0401,4997.5752,27880,1,2753.0192,0.7793,-0.0112,0.1056,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1167,0.0,0.0,1.0,0.0,0.0,0.8438,0.7793,12.2631,50.3984,585.1565,31.8,857.9676,272.8111,0.1765,4.0466,31.755,151.813,183.9451,2948.9375,21.9869,152.0133,2738.9586,0,1,125,1000,979,14,16,0.032,4.0,0.08,10.0,0.008,1.0,0.04,5.0,0.008,1.0,,,,,,,,,,,,1
|
||||
job_639470,0.0,0,64,512,1,0.8125,0,0.8125,5306.6934,5657.698,4984.6105,31779,1,4492.8588,0.4785,0.0293,0.129,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1394,-0.0,0.0,1.0,0.0,0.0,0.625,0.4785,12.7607,50.6826,576.8641,32.8,857.9676,281.1035,0.0756,4.7377,39.2039,228.2368,267.76,1838.1447,22.259,228.4801,1543.3857,0,2,58,464,418,20,39,0.10344827586206896,6.0,0.1896551724137931,11.0,0.06896551724137931,4.0,0.5517241379310345,32.0,0.017241379310344827,1.0,11.9772,23.0244,,,,,,,,,,2
|
||||
job_639470,0.0,0,64,512,2,1.6406,1,1.0,5116.0566,5485.2443,4989.1181,31748,1,5080.3634,0.2559,0.0172,0.1066,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1377,-0.0,0.0,1.0,0.0,0.0,0.3594,0.2559,12.9185,50.7878,575.0807,33.0,857.9676,282.8868,0.1134,4.8467,36.4005,233.6667,270.4372,828.5581,22.9816,233.9229,530.29,0,3,63,504,481,13,20,0.031746031746031744,2.0,0.12698412698412698,8.0,0.015873015873015872,1.0,0.1746031746031746,11.0,0.015873015873015872,1.0,,,,,,,,,,,,3
|
||||
job_639470,0.0,0,64,512,3,1.7188,0,0.8594,4765.7383,4558.2788,4995.3951,31815,1,4154.9884,0.5254,-0.0022,0.0986,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1334,0.0,0.0,1.0,0.0,0.0,0.625,0.5254,13.0012,50.838,574.6276,33.0,857.9676,283.34,0.0786,4.6187,33.8366,202.053,236.1893,1643.3116,23.5376,202.2738,1378.9625,0,4,61,488,453,15,29,0.08196721311475409,5.0,0.14754098360655737,9.0,0.03278688524590164,2.0,0.29508196721311475,18.0,0.01639344262295082,1.0,0.007,11.8014,,,,,,,,,,4
|
||||
job_639470,0.0,0,64,512,4,1.8594,0,0.8906,4864.7266,5293.4787,4615.9444,31633,1,4252.8392,0.3672,0.0121,0.1054,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1324,-0.0,0.0,1.0,0.0,0.0,0.4844,0.3672,13.094,50.8868,574.0699,33.1,857.9676,283.8977,0.0636,4.5964,36.9763,209.4955,246.7664,1591.9611,23.9136,209.726,1316.6818,0,5,58,464,442,15,22,,,0.22413793103448276,13.0,0.017241379310344827,1.0,0.1724137931034483,10.0,,,,,8.9801,,,,,,,,,5
|
||||
job_639470,0.0,0,64,512,4,2.125,0,0.9688,5790.6836,5796.5498,5785.0421,31796,1,5065.3921,0.4902,0.0026,0.1038,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1329,-0.0,0.0,1.0,0.0,0.0,0.5781,0.4902,13.1595,50.9821,573.3127,33.2,857.9676,284.6549,0.0652,4.8477,38.9466,242.4056,281.6225,968.7238,25.1848,242.6104,657.0663,0,6,62,496,464,15,21,0.1774193548387097,11.0,0.12903225806451613,8.0,0.06451612903225806,4.0,0.1774193548387097,11.0,,,0.0074,12.5554,,,,,,,,,,6
|
||||
job_639470,0.0,0,64,512,4,1.8906,0,0.9219,5076.3281,4580.6522,5480.6028,31841,1,4655.3477,0.4492,-0.0017,0.0668,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1365,0.0,0.0,1.0,0.0,0.0,0.5625,0.4492,13.1735,50.9979,572.762,33.2,857.9676,285.2056,0.0629,4.6355,34.4,220.7394,255.4282,1394.3532,22.6359,220.965,1111.6508,0,7,63,504,478,15,25,0.015873015873015872,1.0,0.12698412698412698,8.0,,,0.2698412698412698,17.0,,,,,,,,,,,,,,7
|
||||
job_639470,0.0,0,64,512,4,2.1094,0,0.8906,6032.8496,5776.6962,6253.6073,31766,1,5542.4923,0.4629,-0.0002,0.1325,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1434,0.0,0.0,1.0,0.0,0.0,0.625,0.4629,13.1891,51.0035,571.7978,33.4,857.9676,286.1698,0.0656,4.7759,41.5188,266.819,308.6626,1826.314,24.5452,267.0779,1488.327,0,8,62,496,461,18,32,0.04838709677419355,3.0,0.14516129032258066,9.0,0.08064516129032258,5.0,0.2903225806451613,18.0,,,0.0078,13.2713,,,,,,,,,,8
|
||||
job_639470,0.0,0,64,512,4,1.9219,0,0.9062,5214.0684,4609.1528,5655.4932,31811,1,4602.8614,0.4219,-0.006,0.1127,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.138,0.0,0.0,1.0,0.0,0.0,0.5625,0.4219,13.1974,51.0071,571.1542,33.4,857.9676,286.8134,0.0623,4.7115,36.4913,219.5474,256.316,1383.5879,21.2289,219.762,1101.328,0,9,61,488,459,16,27,0.03278688524590164,2.0,0.13114754098360656,8.0,0.01639344262295082,1.0,0.29508196721311475,18.0,,,,,,,,,,,,,,9
|
||||
job_639470,0.0,0,64,512,3,1.8438,0,0.9531,4883.5527,4809.5297,4946.8478,31765,1,4293.3557,0.4609,0.0117,0.1251,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1369,-0.0,0.0,1.0,0.0,0.0,0.5938,0.4609,14.1407,51.0665,564.7126,34.2,857.9676,293.255,0.0946,4.751,37.1062,205.8864,243.3256,1131.1312,24.7025,206.1244,858.3488,0,10,61,488,460,15,23,0.08196721311475409,5.0,0.09836065573770492,6.0,0.01639344262295082,1.0,0.26229508196721313,16.0,,,0.0075,12.9056,8.8306,,,,,,,,,10
|
||||
job_639470,0.0,0,64,512,3,1.7812,0,0.9062,4549.2324,4530.1362,4568.4784,31819,1,3818.3797,0.502,0.0195,0.157,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1388,-0.0,0.0,1.0,0.0,0.0,0.6719,0.502,14.1364,51.0617,563.7548,34.3,857.9676,294.2128,0.0871,4.7176,34.0193,184.8761,219.1954,1502.0577,23.6183,185.0886,1254.5217,0,11,63,504,474,17,29,0.015873015873015872,1.0,0.15873015873015872,10.0,,,0.30158730158730157,19.0,,,,,,,,,,,,,,11
|
||||
job_639470,0.0,0,64,512,4,2.1562,0,0.875,5296.7969,4698.2873,5624.0785,31847,1,4178.7928,0.3535,0.0064,0.0936,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1414,-0.0,0.0,1.0,0.0,0.0,0.4375,0.3535,14.1368,51.059,563.1395,34.4,857.9676,294.8281,0.0852,4.7881,36.8212,203.0982,240.2245,1268.0385,21.8286,203.3178,1001.1932,0,12,63,504,453,20,42,0.14285714285714285,9.0,0.31746031746031744,20.0,0.1111111111111111,7.0,0.23809523809523808,15.0,,,0.0075,12.3179,,,,,,,,,,12
|
||||
job_639470,0.0,0,64,512,4,1.9688,0,0.8438,4750.25,4750.5154,4750.0386,31807,1,4514.3945,0.4434,-0.0169,0.1051,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.124,0.0,0.0,1.0,0.0,0.0,0.5938,0.4434,14.1379,51.1242,562.4609,34.4,857.9676,295.5067,0.0878,4.6646,34.6581,200.5881,235.5732,1478.8349,21.9149,200.8269,1216.6776,0,13,64,512,461,19,42,0.125,8.0,0.109375,7.0,0.171875,11.0,0.375,24.0,0.015625,1.0,,,,,,,,,,,,13
|
||||
job_639470,0.0,0,64,512,4,1.8281,0,0.9062,4798.502,4904.2,4721.9865,20821,1,4021.2985,0.4199,0.0107,0.1478,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1326,-0.0,0.0,1.0,0.0,0.0,0.6094,0.4199,13.9293,50.9799,562.1901,34.5,857.9676,295.7775,0.0499,3.3116,33.3015,194.2488,227.806,1529.1316,20.1055,194.4542,1277.9038,0,14,61,488,477,7,11,,,0.03278688524590164,2.0,0.03278688524590164,2.0,0.14754098360655737,9.0,,,0.0075,13.2883,,,,,,,,,,14
|
||||
job_639470,0.0,0,64,512,4,1.7969,0,0.8906,5064.6367,4590.1661,5650.9913,31722,1,3612.7922,0.5527,0.0086,0.1079,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1393,-0.0,0.0,1.0,0.0,0.0,0.625,0.5527,14.5292,51.5609,560.838,34.6,857.9676,297.1296,0.1119,4.7729,30.2506,177.5966,208.1631,1397.2363,22.9002,177.8003,1161.3976,0,15,59,472,368,33,68,0.3559322033898305,21.0,0.2033898305084746,12.0,0.8135593220338984,48.0,0.2542372881355932,15.0,0.23728813559322035,14.0,,,8.7705,0.01694915254237288,1.0,,,,,,,15
|
||||
job_639470,0.0,0,64,512,4,2.125,0,0.8906,4274.1406,4573.4162,4121.413,31809,1,4387.0225,0.3379,0.0301,0.1177,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1111,-0.0,0.0,1.0,0.0,0.0,0.5,0.3379,14.5318,51.5632,560.3275,34.7,857.9676,297.6401,0.1121,4.5533,30.9684,197.875,229.1899,2136.448,25.0381,198.109,1877.6589,0,16,60,480,435,18,36,0.13333333333333333,8.0,0.35,21.0,0.016666666666666666,1.0,0.36666666666666664,22.0,0.016666666666666666,1.0,0.0077,13.2783,,,,,,,,,,16
|
||||
job_639470,0.0,0,64,512,4,2.0,0,0.9688,4734.5156,5008.1053,4573.0807,20674,1,3774.2358,0.3711,0.0059,0.0758,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1229,-0.0,0.0,1.0,0.0,0.0,0.4531,0.3711,14.6272,51.7075,560.1359,34.7,857.9676,297.8317,0.081,3.4166,30.0522,179.5794,209.9197,1116.7953,19.6398,179.7862,883.8122,0,17,57,456,435,11,19,0.017543859649122806,1.0,0.15789473684210525,9.0,0.017543859649122806,1.0,0.24561403508771928,14.0,0.017543859649122806,1.0,,,,,,,,,,,,17
|
||||
job_639470,0.0,0,64,512,4,1.7031,0,0.8281,4471.5098,4470.4851,4472.3791,31766,1,3869.9902,0.459,0.0336,0.1621,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1329,-0.0,0.0,1.0,0.0,0.0,0.6094,0.459,14.7954,51.8759,559.9256,34.7,857.9676,298.042,0.1127,4.6919,30.7942,184.653,215.7859,2036.849,23.8428,184.8787,1792.5224,0,18,58,464,419,26,40,0.08620689655172414,5.0,0.22413793103448276,13.0,,,0.4827586206896552,28.0,,,0.0073,13.3234,,,,,,,,,,18
|
||||
job_639470,0.0,0,64,512,5,2.0312,0,0.9531,4740.6719,4578.8382,4823.2596,31842,1,4293.5827,0.3379,-0.002,0.1209,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1469,0.0,0.0,1.0,0.0,0.0,0.4688,0.3379,14.7972,51.8773,559.5532,34.8,857.9676,298.4144,0.1151,4.6856,35.8461,193.9452,230.1248,1344.6664,23.0847,194.1632,1086.7632,0,19,60,480,461,9,16,0.03333333333333333,2.0,0.016666666666666666,1.0,0.06666666666666667,4.0,0.23333333333333334,14.0,,,,,,,,0.016666666666666666,1.0,,,,,19
|
||||
job_639470,0.0,0,64,512,4,1.7031,0,0.8594,4880.1914,4838.5936,4911.2833,31680,1,3718.5085,0.4277,0.0027,0.0566,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.142,-0.0,0.0,1.0,0.0,0.0,0.5,0.4277,14.8422,51.875,558.8903,34.9,857.9676,299.0773,0.1142,4.6474,32.816,178.6089,211.7893,1843.2328,20.1983,178.8585,1606.5913,0,20,64,512,471,19,41,,,0.171875,11.0,,,0.46875,30.0,,,0.0073,12.4239,8.7038,,,,,,,,,20
|
||||
job_639470,0.0,0,64,512,3,1.7969,0,0.9062,4588.334,4493.3441,4642.5307,31717,1,3962.6762,0.3633,0.0029,0.1589,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1395,0.0,0.0,1.0,0.0,0.0,0.5156,0.3633,14.8433,51.876,558.6581,34.9,857.9676,299.3095,0.1126,4.5229,33.0721,192.653,226.0719,1421.8711,21.8044,192.8868,1169.4651,0,21,57,456,428,15,26,0.017543859649122806,1.0,0.07017543859649122,4.0,0.017543859649122806,1.0,0.38596491228070173,22.0,0.017543859649122806,1.0,,,,,,,,,,,,21
|
||||
job_639470,0.0,0,64,512,4,1.7656,0,0.875,4870.873,5241.0396,4629.6677,31739,1,4219.8385,0.3945,0.0013,0.1305,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1352,0.0,0.0,1.0,0.0,0.0,0.5312,0.3945,14.8431,51.9402,558.3341,34.9,857.9676,299.6335,0.1085,4.6972,33.7789,196.7163,230.8154,1494.0327,21.1779,196.9276,1237.3356,0,22,61,488,449,18,38,0.01639344262295082,1.0,0.13114754098360656,8.0,,,0.5245901639344263,32.0,,,0.0075,12.276,,,,,,,,,,22
|
||||
job_639470,0.0,0,64,512,3,1.7344,0,0.875,4640.7969,4846.2115,4500.25,31768,1,4044.2257,0.4062,0.0139,0.096,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1285,-0.0,0.0,1.0,0.0,0.0,0.5156,0.4062,15.2382,52.3806,557.8073,35.0,857.9676,300.1603,0.1114,4.6277,30.466,192.71,223.5157,1293.9556,24.0146,192.938,1041.7923,0,23,63,504,445,19,46,0.1746031746031746,11.0,0.14285714285714285,9.0,0.06349206349206349,4.0,0.5238095238095238,33.0,,,,,,0.015873015873015872,1.0,,,0.015873015873015872,1.0,,,23
|
||||
job_639470,0.0,0,64,512,5,2.0781,0,0.9219,4341.1699,5158.0,3913.3065,31757,1,4123.9554,0.3438,0.0106,0.1051,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1264,-0.0,0.0,1.0,0.0,0.0,0.4531,0.3438,15.2377,52.3799,556.5636,35.1,857.9676,301.404,0.1106,4.5859,32.4505,188.7186,221.4767,1641.4602,21.8273,188.9153,1393.5632,0,24,60,480,451,12,25,0.06666666666666667,4.0,0.16666666666666666,10.0,0.11666666666666667,7.0,0.15,9.0,,,0.0076,11.67,,,,,,,,,,24
|
||||
job_639470,0.0,0,64,512,4,1.5625,0,0.8281,4450.6797,4570.1978,4319.4057,22934,1,3289.9179,0.5234,0.0061,0.096,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1266,-0.0,0.0,1.0,0.0,0.0,0.6406,0.5234,15.0594,52.2661,557.0922,35.1,857.9676,300.8754,0.0845,3.4762,30.0472,159.5092,189.8417,1090.4692,20.7836,159.7097,876.3605,0,25,62,496,455,15,26,0.24193548387096775,15.0,0.20967741935483872,13.0,0.06451612903225806,4.0,0.16129032258064516,10.0,,,,,8.7814,,,,,,,,,25
|
||||
job_639470,0.0,0,64,512,4,2.0,0,0.9375,5077.3633,4987.4731,5170.1071,31764,1,4875.2722,0.5078,0.0036,0.0731,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1211,-0.0,0.0,1.0,0.0,0.0,0.5781,0.5078,15.2418,52.4453,556.3601,35.2,857.9676,301.6075,0.1143,4.8282,36.5507,227.4206,264.3124,1169.9258,21.9494,227.6469,878.8315,0,26,63,504,483,12,19,0.031746031746031744,2.0,0.06349206349206349,4.0,0.031746031746031744,2.0,0.20634920634920634,13.0,,,0.0084,11.212,,,,,,,,,,26
|
||||
job_639470,0.0,0,64,512,4,1.8281,0,0.875,4673.9082,4273.9848,5096.3173,31755,1,3635.6011,0.5137,0.0092,0.0966,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1351,-0.0,0.0,1.0,0.0,0.0,0.6406,0.5137,15.2407,52.4443,555.6974,35.2,857.9676,302.2701,0.1106,4.6972,30.1187,177.3769,207.8291,1096.7705,23.892,177.5995,860.3458,0,27,61,488,449,15,37,0.01639344262295082,1.0,0.19672131147540983,12.0,0.04918032786885246,3.0,0.3770491803278688,23.0,,,,,,0.01639344262295082,1.0,,,,,,,27
|
||||
job_639470,0.0,0,64,512,4,1.9688,0,0.8594,4745.1426,4807.2651,4700.1717,31484,1,3794.1737,0.4199,-0.0062,0.1298,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1313,0.0,0.0,1.0,0.0,0.0,0.5469,0.4199,15.2377,52.5056,554.7346,35.3,857.9676,303.233,0.1099,4.5698,33.2127,183.796,217.3223,1415.0287,21.5982,183.9994,1171.5333,0,28,59,472,421,19,41,0.1694915254237288,10.0,0.11864406779661017,7.0,0.0847457627118644,5.0,0.5084745762711864,30.0,,,0.0085,13.2233,,,,,,,,,,28
|
||||
job_639471,0.0,0,64,512,4,0.5312,0,0.2344,5278.6973,5088.7018,5517.2379,25239,1,3818.4836,0.5566,-0.004,0.1129,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1241,0.0,0.0,1.0,0.0,0.0,0.6481,0.5566,10.4951,50.4392,584.3245,31.9,857.9675,273.643,0.1302,3.8894,37.1114,202.1707,239.6029,4371.2665,22.4624,202.3605,4105.3115,0,29,52,416,374,16,39,0.019230769230769232,1.0,0.11538461538461539,6.0,0.019230769230769232,1.0,0.7884615384615384,41.0,0.038461538461538464,2.0,,,,,,,,,,,,29
|
||||
job_639471,0.0,0,64,512,1,0.9844,0,0.9844,4944.7676,5091.439,4875.6466,31704,1,4244.5182,0.3203,0.008,0.0845,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1335,-0.0,0.0,1.0,0.0,0.0,0.4219,0.3203,12.8603,50.865,578.3782,32.6,857.9675,279.5893,0.0666,4.6428,32.9006,192.3422,225.5807,695.9013,22.4198,192.613,443.2517,0,30,57,456,414,19,35,0.12280701754385964,7.0,0.08771929824561403,5.0,0.05263157894736842,3.0,0.7017543859649122,40.0,,,10.7945,19.7607,6.6515,,,,,,,,,30
|
||||
job_639471,0.0,0,64,512,2,1.7188,0,0.9844,4114.9512,3919.9412,4244.1136,31730,1,4194.079,0.3984,0.0198,0.0875,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.125,-0.0,0.0,1.0,0.0,0.0,0.5312,0.3984,13.8393,50.8987,577.0171,32.7,857.9675,280.9504,0.0636,4.4873,29.5848,193.0235,222.9079,1107.8622,24.0469,193.259,856.417,0,31,64,512,493,10,15,0.046875,3.0,0.03125,2.0,,,0.203125,13.0,0.015625,1.0,,,,,,,,,,,,31
|
||||
job_639471,0.0,0,64,512,3,1.7656,0,0.8594,5001.7891,4406.6449,5429.1745,31817,1,4145.4506,0.418,-0.0054,0.0829,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.14,0.0,0.0,1.0,0.0,0.0,0.5,0.418,14.1603,51.2707,576.1389,32.8,857.9675,281.8287,0.0622,4.6552,38.8123,204.3063,243.3941,2021.8598,23.3742,204.5192,1750.4335,0,32,61,488,437,20,46,0.06557377049180328,4.0,0.08196721311475409,5.0,0.04918032786885246,3.0,0.639344262295082,39.0,0.01639344262295082,1.0,0.0081,7.8836,,,,,,,,,,32
|
||||
job_639471,0.0,0,64,512,4,2.0312,0,0.9375,4818.6543,4762.1027,4841.2131,31726,1,5036.9689,0.2852,0.0145,0.1036,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1325,-0.0,0.0,1.0,0.0,0.0,0.4062,0.2852,14.1077,51.196,575.3264,32.9,857.9675,282.6411,0.0656,4.6631,33.8051,229.0206,263.108,1289.5434,22.5459,229.237,999.2225,0,33,61,488,465,10,21,,,0.11475409836065574,7.0,0.03278688524590164,2.0,0.22950819672131148,14.0,0.03278688524590164,2.0,,,,,,,,,,,,33
|
||||
job_639471,0.0,0,64,512,4,1.75,0,0.9375,4567.793,4373.0586,4762.5273,31639,1,3628.674,0.5,0.0075,0.1492,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1348,-0.0,0.0,1.0,0.0,0.0,0.6406,0.5,14.1472,51.2188,573.8872,33.1,857.9675,284.0803,0.0665,4.5609,28.5178,173.7767,202.5827,1301.8005,22.2062,173.9981,1072.4469,0,34,59,472,441,17,23,0.13559322033898305,8.0,0.13559322033898305,8.0,,,0.2542372881355932,15.0,,,0.0083,8.411,,,,,,,,,,34
|
||||
job_639471,0.0,0,64,512,3,1.9688,0,0.8906,5203.1992,4785.8119,5475.1742,31726,1,4610.7692,0.3945,0.0015,0.1334,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1392,-0.0,0.0,1.0,0.0,0.0,0.5781,0.3945,14.2407,51.2979,572.6302,33.3,857.9675,285.3373,0.0655,4.7104,36.7892,210.5074,247.6007,1785.0733,22.4758,210.7457,1510.2823,0,35,61,488,460,13,27,,,0.11475409836065574,7.0,,,0.32786885245901637,20.0,0.01639344262295082,1.0,,,7.3123,,,,,,,,,35
|
||||
job_639471,0.0,0,64,512,4,1.9219,0,0.9219,5044.2012,5116.1058,4961.4202,31779,1,4480.1027,0.5352,-0.0054,0.0842,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.135,0.0,0.0,1.0,0.0,0.0,0.625,0.5352,14.2407,51.3575,572.3281,33.3,857.9675,285.6394,0.0627,4.8504,36.6108,213.1635,250.063,1218.0837,21.7718,213.3891,941.3942,0,36,60,480,441,17,39,,,0.18333333333333332,11.0,0.03333333333333333,2.0,0.43333333333333335,26.0,,,0.0077,8.4733,,,,,,,,,,36
|
||||
job_639471,0.0,0,64,512,3,1.75,0,0.8906,4737.1152,4212.0859,5262.1445,30999,1,4007.6604,0.5,0.0051,0.1255,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1275,-0.0,0.0,1.0,0.0,0.0,0.625,0.5,14.3788,51.4694,571.8417,33.3,857.9675,286.1258,0.0673,4.553,31.2988,186.6306,218.2242,1048.2685,23.4353,186.8577,802.0521,0,37,64,512,479,17,31,0.015625,1.0,0.15625,10.0,0.03125,2.0,0.296875,19.0,0.015625,1.0,,,,,,,,,,,,37
|
||||
job_639471,0.0,0,64,512,4,2.0781,0,0.9062,5100.459,5243.36,5008.8558,31758,1,4289.2275,0.3906,0.0276,0.1256,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1362,-0.0,0.0,1.0,0.0,0.0,0.5312,0.3906,14.4257,51.5521,570.7955,33.5,857.9675,287.172,0.0653,4.7056,32.7387,201.981,235.0431,1731.9565,21.626,202.2386,1470.5772,0,38,59,472,444,14,24,0.05084745762711865,3.0,0.06779661016949153,4.0,0.03389830508474576,2.0,0.3898305084745763,23.0,0.01694915254237288,1.0,0.0086,9.2793,,,,,,,,,,38
|
||||
job_639471,0.0,0,64,512,4,1.7031,0,0.9531,5043.1562,4965.0356,5119.4672,31823,1,4607.3975,0.4941,0.0074,0.126,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1292,-0.0,0.0,1.0,0.0,0.0,0.625,0.4941,14.4556,51.6204,570.5818,33.5,857.9675,287.3857,0.0686,4.688,40.8057,219.6135,260.7417,1244.3571,27.5955,219.867,951.3271,0,39,61,488,452,14,36,,,0.08196721311475409,5.0,0.04918032786885246,3.0,0.45901639344262296,28.0,,,,,,,,,,,,,,39
|
||||
job_639471,0.0,0,64,512,4,1.875,0,0.9219,4419.4766,4218.5217,4583.3759,31827,1,4168.3604,0.4492,0.0001,0.0823,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1279,0.0,0.0,1.0,0.0,0.0,0.5625,0.4492,14.7014,51.8757,569.6165,33.6,857.9675,288.3511,0.0662,4.5612,30.5522,192.2327,223.0663,1248.4529,24.1425,192.4475,996.6791,0,40,61,488,466,11,17,0.08196721311475409,5.0,0.08196721311475409,5.0,0.01639344262295082,1.0,0.18032786885245902,11.0,,,0.0084,8.4746,6.1621,,,,,,,,,40
|
||||
job_639471,0.0,0,64,512,4,1.8281,0,0.875,4378.8047,4390.5302,4370.3165,20422,1,3374.562,0.4199,0.0026,0.1097,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1403,-0.0,0.0,1.0,0.0,0.0,0.5469,0.4199,14.6059,51.8282,569.7355,33.6,857.9675,288.2321,0.0477,3.2238,29.9727,163.6158,193.973,1448.2179,23.0063,163.9523,1228.0101,0,41,63,504,473,17,31,,,0.15873015873015872,10.0,,,0.3333333333333333,21.0,,,,,,,,,,,,,,41
|
||||
job_639471,0.0,0,64,512,4,1.9688,0,0.8906,5440.5508,5273.5025,5545.0222,31794,1,4314.333,0.3848,-0.0019,0.0687,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1387,0.0,0.0,1.0,0.0,0.0,0.4688,0.3848,14.9666,52.2508,569.2278,33.7,857.9675,288.7397,0.065,4.781,34.9912,208.4372,243.726,1518.1162,21.834,208.6693,1247.7718,0,42,60,480,440,14,32,0.13333333333333333,8.0,0.16666666666666666,10.0,0.016666666666666666,1.0,0.35,21.0,,,0.0078,8.9859,,,,,,,,,,42
|
||||
job_639471,0.0,0,64,512,4,1.5469,0,0.7969,4309.9902,4170.6036,4471.7257,31730,1,3959.7796,0.5371,0.017,0.0942,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1249,-0.0,0.0,1.0,0.0,0.0,0.625,0.5371,15.3689,52.4981,568.5267,33.7,857.9675,289.4408,0.0633,4.577,30.0185,183.5674,213.8767,1281.3491,21.6632,183.7946,1041.2281,0,43,63,504,468,14,27,0.15873015873015872,10.0,0.06349206349206349,4.0,0.047619047619047616,3.0,0.31746031746031744,20.0,,,,,,,,,,,,,,43
|
||||
job_639471,0.0,0,64,512,4,1.7969,0,0.9219,4686.1582,4334.2927,4921.1173,31718,1,4093.6999,0.4004,0.0024,0.0855,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1345,-0.0,0.0,1.0,0.0,0.0,0.5,0.4004,15.3116,52.6276,568.5529,33.7,857.9675,289.4147,0.0649,4.5947,33.5947,192.7301,226.6108,1070.9411,23.6635,192.9508,816.0677,0,44,64,512,404,33,66,0.25,16.0,0.03125,2.0,0.796875,51.0,0.203125,13.0,0.40625,26.0,0.0082,8.7248,,,,,,,,,,44
|
||||
job_639471,0.0,0,64,512,4,2.1875,0,0.9062,4488.0605,4766.2674,4327.9846,31776,1,4643.8272,0.3652,0.0198,0.102,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.115,-0.0,0.0,1.0,0.0,0.0,0.4688,0.3652,15.2923,52.5649,567.6223,33.8,857.9675,290.3453,0.0648,4.5433,38.2594,212.1464,250.6847,2427.0584,19.2772,212.3601,2152.5489,0,45,59,472,281,51,126,0.6949152542372882,41.0,0.1016949152542373,6.0,2.2033898305084745,130.0,0.0847457627118644,5.0,0.559322033898305,33.0,,,6.1293,,,,,,,,,45
|
||||
job_639471,0.0,0,64,512,4,2.1406,0,0.9844,2831.2832,4734.6768,1934.2816,19126,1,3422.9025,0.3203,0.0741,0.1736,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0819,-0.0,0.0,1.0,0.0,0.0,0.5156,0.3203,15.0547,52.2833,567.6797,33.8,857.9675,290.2878,0.0433,2.9003,21.3716,139.1872,160.9597,2780.6994,19.1556,139.5444,2597.6796,0,46,57,456,174,56,114,2.5789473684210527,147.0,0.05263157894736842,3.0,1.9473684210526316,111.0,0.17543859649122806,10.0,0.7192982456140351,41.0,0.0085,55.7189,,0.03508771929824561,2.0,,,,,,,46
|
||||
job_639471,0.0,0,64,512,4,2.0,0,0.9531,1948.959,3929.527,1614.3425,20028,1,3361.197,0.1445,0.0796,0.2262,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.057,-0.0,0.0,1.0,0.0,0.0,0.4375,0.1445,14.9425,52.2209,567.6288,33.8,857.9675,290.3387,0.0488,2.7631,25.3613,134.4423,160.065,4012.6647,18.8904,134.6545,3830.9431,0,47,60,480,251,53,87,1.8666666666666667,112.0,0.06666666666666667,4.0,1.3333333333333333,80.0,0.18333333333333332,11.0,0.6166666666666667,37.0,,,,,,,,,,0.016666666666666666,1.0,47
|
||||
job_639471,0.0,0,64,512,3,1.5469,0,0.875,1713.2109,3120.7785,1084.9746,12999,1,2205.1024,0.3086,0.1337,0.1884,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0686,-0.0,0.0,1.0,0.0,0.0,0.5312,0.3086,14.818,52.0276,568.1573,33.8,857.9675,289.8102,0.0314,1.9562,15.2563,98.0173,113.4611,3641.5565,17.9428,98.173,3508.1929,0,48,64,512,309,57,117,1.34375,86.0,0.171875,11.0,1.359375,87.0,0.296875,19.0,0.09375,6.0,0.0082,8.6866,,0.015625,1.0,,,,,,,48
|
||||
job_639471,0.0,0,64,512,7,1.625,0,0.9531,2370.7871,3573.7063,1978.1244,31763,1,3467.8194,0.2461,0.0916,0.1688,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0845,-0.0,0.0,1.0,0.0,0.0,0.4531,0.2461,15.1041,52.3137,567.8245,33.8,857.9675,290.143,0.065,4.2255,18.8418,139.4689,158.5744,3585.1599,19.0904,139.6672,3403.2676,0,49,39,312,191,35,70,1.1794871794871795,46.0,0.07692307692307693,3.0,1.3076923076923077,51.0,0.41025641025641024,16.0,0.10256410256410256,4.0,,,,0.02564102564102564,1.0,,,,,,,49
|
||||
job_639473,0.0,0,64,512,0,0.0,0,0.0,2574.8984,3414.6373,2066.8433,14422,1,2612.5608,0.377,0.1212,0.1534,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0885,-0.0,0.0,1.0,0.0,0.0,0.5625,0.377,7.261,50.0898,573.4373,33.2,857.9675,284.5302,0.0455,2.2784,38.4903,117.0694,155.8179,9181.4558,19.7276,117.2818,9003.6315,0,49,60,480,400,37,48,0.4666666666666667,28.0,0.26666666666666666,16.0,0.38333333333333336,23.0,0.21666666666666667,13.0,0.18333333333333332,11.0,,,,,,,,,,,,49
|
||||
job_639473,0.0,0,64,512,1,1.0,1,1.0,3732.3652,3769.9299,3715.7521,31808,1,4078.5665,0.3066,0.024,0.0998,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.117,-0.0,0.0,1.0,0.0,0.0,0.4531,0.3066,12.5825,50.4959,566.5182,34.0,857.9675,291.4493,0.0686,4.4722,26.4647,178.2315,205.0064,611.3836,22.5264,178.4728,379.3765,0,50,61,488,454,16,7,0.4098360655737705,25.0,0.01639344262295082,1.0,0.01639344262295082,1.0,0.06557377049180328,4.0,0.03278688524590164,2.0,73.2928,82.7069,20.5797,,,,,,,0.01639344262295082,1.0,50
|
||||
job_639473,0.0,0,64,512,2,1.5312,0,0.875,4180.8438,3657.6311,4591.0279,17255,1,3125.6494,0.4395,0.0131,0.0883,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1311,-0.0,0.0,1.0,0.0,0.0,0.5312,0.4395,12.5143,50.4529,565.7725,34.1,857.9675,292.1951,0.0393,2.8336,28.712,150.3515,179.3291,1837.5348,23.3846,150.5774,1631.9841,0,51,63,504,477,13,26,0.015873015873015872,1.0,0.14285714285714285,9.0,0.015873015873015872,1.0,0.2222222222222222,14.0,,,,,,,,,,,,0.031746031746031744,2.0,51
|
||||
job_639473,0.0,0,64,512,3,1.6719,0,0.8906,4772.627,4960.8151,4609.1642,31652,1,3769.5395,0.4648,0.014,0.0786,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1376,-0.0,0.0,1.0,0.0,0.0,0.5781,0.4648,12.8545,50.8127,566.1333,34.0,857.9675,291.8342,0.0654,5.1247,31.4543,178.3831,210.1787,1732.7267,22.7374,178.6587,1494.6839,0,52,61,488,431,19,52,0.06557377049180328,4.0,0.13114754098360656,8.0,0.11475409836065574,7.0,0.639344262295082,39.0,,,0.0085,8.3109,,0.01639344262295082,1.0,,,,,0.06557377049180328,4.0,52
|
||||
job_639473,0.0,0,64,512,4,2.0156,0,0.9531,4045.3574,4180.3436,3982.3123,31771,1,3952.5092,0.3184,0.0176,0.1401,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1307,-0.0,0.0,1.0,0.0,0.0,0.5,0.3184,13.2115,51.1639,565.3646,34.1,857.9675,292.603,0.0634,4.4954,29.8155,172.7807,202.884,1262.3478,25.742,173.0047,1029.2236,0,53,65,520,492,13,26,0.03076923076923077,2.0,0.03076923076923077,2.0,0.046153846153846156,3.0,0.27692307692307694,18.0,,,,,,,,,,,,0.046153846153846156,3.0,53
|
||||
job_639473,0.0,0,64,512,3,1.7031,0,0.8906,5088.5273,4488.8068,5495.5508,31800,1,4521.5771,0.4043,0.0049,0.0916,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1336,0.0,0.0,1.0,0.0,0.0,0.4844,0.4043,13.4076,51.4641,565.8401,34.0,857.9675,292.1274,0.0649,4.8001,35.4838,216.8179,252.5977,1903.4231,22.6662,217.0486,1623.3547,0,54,61,488,453,13,31,0.06557377049180328,4.0,0.08196721311475409,5.0,0.01639344262295082,1.0,0.4262295081967213,26.0,,,0.0084,7.7525,,,,,,,,,,54
|
||||
job_639473,0.0,0,64,512,3,1.75,0,0.8906,4631.4766,5009.6139,4385.0774,31812,1,4169.8675,0.3945,0.0182,0.0826,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1331,-0.0,0.0,1.0,0.0,0.0,0.4844,0.3945,13.6974,51.6048,565.2818,34.1,857.9675,292.6857,0.0706,4.5962,31.4269,196.5331,228.3023,1296.0607,24.1405,196.8045,1039.0185,0,55,61,488,457,12,23,0.13114754098360656,8.0,0.06557377049180328,4.0,0.03278688524590164,2.0,0.2786885245901639,17.0,,,,,6.2341,,,,,,,0.03278688524590164,2.0,55
|
||||
job_639473,0.0,0,64,512,3,1.875,0,0.9219,5294.6016,4989.8677,5472.9133,31704,1,4638.6034,0.3691,-0.0048,0.1111,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1325,0.0,0.0,1.0,0.0,0.0,0.4844,0.3691,13.7494,51.6116,565.5437,34.1,857.9675,292.4238,0.0867,4.6832,38.8346,224.59,263.8149,1565.5302,22.4001,224.8932,1274.6283,0,56,63,504,478,11,24,0.031746031746031744,2.0,0.1111111111111111,7.0,0.031746031746031744,2.0,0.25396825396825395,16.0,,,0.0081,7.9573,,,,,,,,,,56
|
||||
job_639473,0.0,0,64,512,3,1.7188,0,0.9062,4950.9766,4560.3436,5262.1123,31843,1,4774.4938,0.4434,-0.0031,0.134,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1306,0.0,0.0,1.0,0.0,0.0,0.5781,0.4434,13.8882,51.8104,564.4661,34.2,857.9675,293.5014,0.0892,4.6537,34.0791,223.8635,258.2931,1726.4682,23.134,224.1245,1440.3826,0,57,63,504,466,17,31,0.1111111111111111,7.0,0.06349206349206349,4.0,0.047619047619047616,3.0,0.38095238095238093,24.0,,,,,,,,,,,,,,57
|
||||
job_639473,0.0,0,64,512,3,1.6094,0,0.9375,4331.5684,4241.4459,4400.5586,31744,1,3845.2618,0.4336,0.0047,0.1354,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1298,-0.0,0.0,1.0,0.0,0.0,0.5781,0.4336,14.0239,51.9366,564.886,34.2,857.9675,293.0815,0.0936,4.5707,30.7528,183.0933,214.1963,1407.8184,24.3511,183.3495,1164.6952,0,58,63,504,475,13,29,,,0.07936507936507936,5.0,0.031746031746031744,2.0,0.3333333333333333,21.0,,,0.0073,7.6926,,,,,,,,0.015873015873015872,1.0,58
|
||||
job_639473,0.0,0,64,512,3,1.5156,0,0.7812,4725.0371,3898.9237,5431.4239,31792,1,4496.5728,0.4609,0.0244,0.0844,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1322,-0.0,0.0,1.0,0.0,0.0,0.5625,0.4609,14.0273,51.9401,563.9426,34.3,857.9675,294.025,0.0865,4.6121,34.988,210.5671,245.9236,1490.2312,26.3782,210.8487,1213.311,0,59,61,488,465,10,22,0.06557377049180328,4.0,0.01639344262295082,1.0,0.08196721311475409,5.0,0.3114754098360656,19.0,0.01639344262295082,1.0,,,,,,,,,,,,59
|
||||
job_639473,0.0,0,64,512,4,1.5781,0,0.8125,4552.8398,4327.259,4769.7778,31726,1,3951.5205,0.4902,0.0058,0.1092,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1243,-0.0,0.0,1.0,0.0,0.0,0.625,0.4902,14.0216,51.9988,563.3038,34.3,857.9675,294.6637,0.1176,4.6409,34.6955,189.8302,224.9402,1325.7249,23.1612,190.1267,1072.9765,0,60,58,464,413,16,40,0.1896551724137931,11.0,0.13793103448275862,8.0,0.08620689655172414,5.0,0.5689655172413793,33.0,,,0.0086,7.7063,6.7543,,,,,,,,,60
|
||||
job_639473,0.0,0,64,512,4,1.6719,0,0.8438,4229.166,4585.0703,3873.2617,31580,1,3910.4259,0.5,0.0132,0.1619,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1226,-0.0,0.0,1.0,0.0,0.0,0.6562,0.5,14.0175,52.0583,562.8052,34.4,857.9675,295.1624,0.097,4.5178,31.3756,184.0054,215.7393,1404.1596,25.5141,184.2663,1158.3824,0,61,62,496,465,15,27,0.06451612903225806,4.0,0.12903225806451613,8.0,0.06451612903225806,4.0,0.25806451612903225,16.0,,,,,,,,,,,,,,61
|
||||
job_639473,0.0,0,64,512,5,1.7656,0,0.8594,4619.9727,4131.9717,5074.8264,31689,1,4163.141,0.4824,-0.0109,0.0868,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1226,0.0,0.0,1.0,0.0,0.0,0.5625,0.4824,14.4709,52.4935,562.3414,34.5,857.9675,295.6262,0.1201,4.6276,38.4473,199.087,237.8926,1122.7358,23.8958,199.3249,856.3148,0,62,61,488,476,11,10,0.01639344262295082,1.0,0.08196721311475409,5.0,,,0.09836065573770492,6.0,0.01639344262295082,1.0,0.0085,7.8196,,,,,,,,,,62
|
||||
job_639473,0.0,0,64,512,4,2.3594,0,0.9688,5134.4629,4500.9897,5524.1388,31785,1,3916.1129,0.3809,0.0077,0.1039,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1386,-0.0,0.0,1.0,0.0,0.0,0.4844,0.3809,14.3535,52.3757,561.5363,34.6,857.9675,296.4312,0.1201,4.8417,33.4082,196.2872,230.1101,1242.3513,24.6441,196.5814,982.7475,0,63,62,496,467,16,25,0.06451612903225806,4.0,0.06451612903225806,4.0,0.03225806451612903,2.0,0.3225806451612903,20.0,,,,,,,,,,,,,,63
|
||||
job_639473,0.0,0,64,512,4,2.1562,0,0.9062,5462.8105,5057.0452,5720.7891,31766,1,5070.3185,0.3887,0.0044,0.069,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1309,-0.0,0.0,1.0,0.0,0.0,0.4844,0.3887,14.3912,52.3806,561.2214,34.6,857.9675,296.7462,0.1168,4.7044,38.1819,238.6575,277.2112,1523.0599,23.148,238.912,1217.9877,0,64,61,488,456,15,24,0.13114754098360656,8.0,0.08196721311475409,5.0,0.03278688524590164,2.0,0.2786885245901639,17.0,,,0.0084,9.6022,,,,,,,,,,64
|
||||
job_639473,0.0,0,64,512,4,1.7188,0,0.8438,5144.3984,4651.443,5569.2364,31772,1,5404.6743,0.4629,0.0039,0.0697,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1304,-0.0,0.0,1.0,0.0,0.0,0.5469,0.4629,14.7312,52.7578,560.1469,34.7,857.9675,297.8206,0.1115,4.7518,42.0515,251.3932,293.8211,1513.4404,21.9402,251.6576,1192.9212,0,65,59,472,430,17,35,0.11864406779661017,7.0,0.11864406779661017,7.0,0.05084745762711865,3.0,0.423728813559322,25.0,,,,,6.0896,,,,,,,,,65
|
||||
job_639474,0.0,0,64,512,4,0.5469,0,0.25,4862.9727,4513.1443,5350.1168,31704,1,3892.5586,0.582,0.0105,0.1354,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1296,-0.0,0.0,1.0,0.0,0.0,0.6727,0.582,12.6295,50.548,561.5298,34.6,857.9675,296.4377,0.1256,4.65,69.6756,200.9736,271.0452,4741.4969,23.1178,201.2433,4442.6837,0,65,37,296,277,13,19,,,0.40540540540540543,15.0,0.08108108108108109,3.0,0.8918918918918919,33.0,,,,,36.4673,,,,,,,,,65
|
||||
job_639474,0.0,0,64,512,1,1.0,1,1.0,5066.0879,4639.2486,5299.4955,31799,1,5073.213,0.3535,0.0067,0.1205,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1309,-0.0,0.0,1.0,0.0,0.0,0.5,0.3535,12.7806,50.6748,627.205,26.9,857.9675,230.7626,0.0762,4.6149,37.9156,235.294,273.5463,605.1264,24.0294,235.5538,302.928,0,66,60,480,440,16,29,0.18333333333333332,11.0,0.15,9.0,0.13333333333333333,8.0,0.25,15.0,,,42.5487,50.8663,,,,,,,,,,66
|
||||
job_639474,0.0,0,64,512,2,1.6719,0,0.9531,4734.041,5337.0905,4314.702,31845,1,3896.281,0.4102,0.0173,0.1281,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1301,-0.0,0.0,1.0,0.0,0.0,0.5156,0.4102,13.7043,50.7106,622.134,27.5,857.9675,235.8336,0.0668,4.5589,29.9015,188.6862,218.8452,1183.017,23.4914,188.8764,936.117,0,67,63,504,355,47,91,0.3333333333333333,21.0,0.1111111111111111,7.0,1.1587301587301588,73.0,0.20634920634920634,13.0,0.5555555555555556,35.0,,,,0.031746031746031744,2.0,,,,,,,67
|
||||
job_639474,0.0,0,64,512,3,2.2969,0,0.9219,4292.0527,5313.4359,3989.5165,31752,1,4965.0243,0.2285,0.0361,0.1284,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1007,-0.0,0.0,1.0,0.0,0.0,0.3594,0.2285,13.7788,50.7601,620.5532,27.7,857.9675,237.4144,0.0656,4.7095,31.3857,214.771,246.5058,5067.1626,19.1151,215.0541,4796.8269,0,68,61,488,276,57,125,0.7868852459016393,48.0,0.04918032786885246,3.0,1.8524590163934427,113.0,0.18032786885245902,11.0,0.6065573770491803,37.0,0.0088,8.3963,,0.03278688524590164,2.0,,,,,0.01639344262295082,1.0,68
|
||||
job_639474,0.0,0,64,512,4,2.0312,0,0.9844,2689.0547,4481.0945,2097.9143,16225,1,3499.4794,0.248,0.064,0.1772,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.078,-0.0,0.0,1.0,0.0,0.0,0.5,0.248,13.5363,50.52,618.4584,27.9,857.9675,239.5091,0.0562,2.4372,19.9013,140.6377,160.7959,2845.5649,19.1755,140.8378,2663.1483,0,69,56,448,218,54,89,2.4464285714285716,137.0,0.125,7.0,1.8392857142857142,103.0,0.07142857142857142,4.0,0.30357142857142855,17.0,,,,,,,,,,,,69
|
||||
job_639474,0.0,0,64,512,3,1.8125,0,0.9531,1819.6953,3456.2381,1285.487,16720,1,2446.8912,0.2461,0.1768,0.2232,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0662,-0.0,0.0,1.0,0.0,0.0,0.5469,0.2461,13.6124,50.5903,620.05,27.7,857.9675,237.9175,0.0369,2.372,14.1649,102.8323,117.1881,2600.3087,18.8825,102.9859,2461.8638,0,70,58,464,355,32,36,1.3620689655172413,79.0,0.08620689655172414,5.0,0.39655172413793105,23.0,0.13793103448275862,8.0,0.05172413793103448,3.0,0.0078,17.8975,8.0763,,,,,,,0.017241379310344827,1.0,70
|
||||
job_639474,0.0,0,64,512,3,1.4375,0,0.9219,3140.1328,3793.3929,2632.0417,31776,1,3649.7812,0.4375,0.0669,0.1491,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0942,-0.0,0.0,1.0,0.0,0.0,0.6094,0.4375,13.8658,50.8432,615.5933,28.2,857.9675,242.3742,0.0646,4.3413,26.1997,152.3269,178.7983,1848.2381,19.4477,152.5336,1645.6483,0,71,54,432,346,33,32,0.7962962962962963,43.0,0.1111111111111111,6.0,0.42592592592592593,23.0,0.2222222222222222,12.0,0.3333333333333333,18.0,,,,0.018518518518518517,1.0,,,,,0.018518518518518517,1.0,71
|
||||
job_639474,0.0,0,64,512,4,1.7344,0,0.9062,3757.4688,3691.0625,3792.253,31727,1,3971.7082,0.3438,0.0297,0.0921,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1108,-0.0,0.0,1.0,0.0,0.0,0.4531,0.3438,13.8605,50.8377,617.5437,28.0,857.9675,240.4238,0.0696,4.3975,27.0854,166.2132,193.5822,3286.0053,18.9345,166.4269,3069.0865,0,72,64,512,326,56,112,0.5625,36.0,0.171875,11.0,1.421875,91.0,0.1875,12.0,0.578125,37.0,0.008,82.4332,,0.015625,1.0,,,,,0.015625,1.0,72
|
||||
job_639474,0.0,0,64,512,4,1.9844,0,0.9375,3025.4746,4412.386,2628.2186,31668,1,3575.8498,0.2227,0.0329,0.1432,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0848,-0.0,0.0,1.0,0.0,0.0,0.3594,0.2227,13.8521,50.8289,615.4798,28.3,857.9675,242.4877,0.0891,4.3699,23.2891,149.6715,173.2579,2698.0374,18.0666,149.8792,2502.3368,0,73,60,480,337,49,48,1.4166666666666667,85.0,0.03333333333333333,2.0,0.6166666666666667,37.0,0.2,12.0,0.15,9.0,,,,0.016666666666666666,1.0,,,,,0.016666666666666666,1.0,73
|
||||
job_639474,0.0,0,64,512,4,1.7031,0,0.9375,3082.4863,3622.4722,2871.1875,31787,1,3677.9858,0.2812,0.0609,0.1265,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0914,-0.0,0.0,1.0,0.0,0.0,0.4844,0.2812,13.8518,50.893,614.1329,28.4,857.9675,243.8346,0.0647,4.3386,23.0836,150.3708,173.7147,1731.4236,19.0162,150.566,1534.3491,0,74,60,480,414,31,29,0.5666666666666667,34.0,0.06666666666666667,4.0,0.1,6.0,0.3,18.0,0.05,3.0,0.0081,8.0639,,0.016666666666666666,1.0,,,,,0.016666666666666666,1.0,74
|
||||
job_639474,0.0,0,64,512,4,1.8281,1,1.0,3949.6445,4464.3611,3450.7654,25386,1,3103.6693,0.4922,0.0245,0.1625,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1116,-0.0,0.0,1.0,0.0,0.0,0.6406,0.4922,13.816,50.7961,612.8476,28.6,857.9675,245.1199,0.0558,3.786,27.4207,148.2987,175.969,1591.7732,20.678,148.4921,1391.3251,0,75,61,488,446,18,27,0.2459016393442623,15.0,0.08196721311475409,5.0,,,0.32786885245901637,20.0,,,,,5.8565,,,,,,,0.03278688524590164,2.0,75
|
||||
job_639474,0.0,0,64,512,3,2.2344,2,1.0,4071.6719,3826.4317,4208.0821,31779,1,3359.4605,0.3574,0.0041,0.1367,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.122,-0.0,0.0,1.0,0.0,0.0,0.5312,0.3574,14.0157,51.0362,611.0781,28.8,857.9675,246.8895,0.0659,4.5298,25.7335,158.3383,184.3538,1915.8699,20.5904,158.5541,1706.3756,0,76,64,512,469,18,35,0.125,8.0,0.125,8.0,0.046875,3.0,0.453125,29.0,,,0.0081,8.0145,,,,,,,,0.015625,1.0,76
|
||||
job_639474,0.0,0,64,512,3,3.0,3,1.0,4338.9863,4198.9942,4409.1877,31653,1,3668.1638,0.334,-0.002,0.0705,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1292,0.0,0.0,1.0,0.0,0.0,0.4375,0.334,14.2828,51.3484,609.1068,29.0,857.9675,248.8608,0.0664,4.5496,30.742,170.001,201.0159,1171.6576,21.017,170.2071,945.0538,0,77,64,512,430,36,63,0.296875,19.0,0.640625,41.0,0.03125,2.0,0.3125,20.0,,,,,,,,,,,,,,77
|
||||
job_639474,0.0,0,64,512,4,4.0,4,1.0,4903.0371,5346.2614,4811.0472,23634,1,4141.1011,0.1719,0.0046,0.11,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1267,-0.0,0.0,1.0,0.0,0.0,0.3438,0.1719,14.4365,51.4758,607.8784,29.1,857.9675,250.0891,0.0511,3.6578,32.3641,188.0312,220.6809,1996.031,19.1079,188.2653,1752.5626,0,78,93,744,701,23,38,0.043010752688172046,4.0,0.2903225806451613,27.0,,,0.08602150537634409,8.0,0.010752688172043012,1.0,0.0078,10.0346,,,,,,,,0.03225806451612903,3.0,78
|
||||
job_639474,0.0,0,64,512,0,0.0,0,0.0,3705.6719,3357.3361,4578.8973,26629,1,2424.7377,0.7148,0.0124,0.095,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1105,-0.0,0.0,1.0,0.0,0.0,0.7812,0.7148,14.4194,51.4582,606.2361,29.3,857.9675,251.7314,0.0598,3.8365,25.5076,130.5443,156.312,3207.1053,20.9174,130.7441,3026.017,1,79,60,480,452,12,21,0.11666666666666667,7.0,0.1,6.0,,,0.16666666666666666,10.0,,,,,,,,,,,,0.1,6.0,79
|
||||
job_639474,0.0,0,64,512,1,0.8281,0,0.8281,4801.8926,4507.3876,5101.0354,21141,1,3692.5927,0.5039,0.0145,0.122,-0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1225,-0.0,0.0,1.0,0.0,0.0,0.625,0.5039,14.3202,51.3997,605.6434,29.4,857.9675,252.3241,0.0464,3.3871,30.1146,181.7158,212.1128,1758.6001,22.0893,181.9515,1521.007,1,80,64,512,455,31,55,0.03125,2.0,0.078125,5.0,0.09375,6.0,0.53125,34.0,,,0.0078,11.1322,5.9826,,,,,,,0.171875,11.0,80
|
||||
job_639475,0.0,0,64,512,0,0.0,0,0.0,3726.2617,3403.3817,4792.5798,19853,1,2212.5652,0.7676,-0.0047,0.0761,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1104,0.0,0.0,1.0,0.0,0.0,0.8281,0.7676,8.8246,50.3819,593.4822,30.8,857.9675,264.4853,0.125,3.2035,27.7363,124.398,152.4472,2771.6856,22.7908,124.5855,2593.2438,1,82,9,72,72,0,0,,,,,,,,,,,15.0154,24.9398,,,,,,,,,,82
|
||||
|
3
training_logs/20260608_210110_reward_vs_steps.png
Normal file
3
training_logs/20260608_210110_reward_vs_steps.png
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1946a5bbcc8d2618f627be0492949dc41f3b22d566fa9f1aa985d5fcfc57ca9e
|
||||
size 287397
|
||||
46309
training_logs/20260608_210110_trial_results.csv
Normal file
46309
training_logs/20260608_210110_trial_results.csv
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9c9e5138ecdec47939baa6f4dbae58c063bd41dc565960475f71aa27113f016b
|
||||
size 124229
|
||||
592
training_logs/20260608_210110_vllm_metrics_job_632522.csv
Normal file
592
training_logs/20260608_210110_vllm_metrics_job_632522.csv
Normal file
@@ -0,0 +1,592 @@
|
||||
datetime_str,n_engines_reporting,unique_ips,total_prompt_throughput_tokens_per_sec,total_generation_throughput_tokens_per_sec,total_running_requests,total_waiting_requests,avg_prompt_throughput_per_engine,avg_generation_throughput_per_engine,avg_running_requests_per_engine,avg_waiting_requests_per_engine,avg_gpu_kv_cache_usage_pct,avg_prefix_cache_hit_rate_pct,min_running_requests,max_running_requests,min_generation_throughput,max_generation_throughput
|
||||
06-06 15:25:08,1,1,16.2,0.0,1,0,16.2,0.0,1.0,0.0,0.3,0.0,1,1,0.0,0.0
|
||||
06-06 15:25:13,1,1,0.0,49.8,1,0,0.0,49.8,1.0,0.0,0.4,0.0,1,1,49.8,49.8
|
||||
06-06 15:25:19,1,1,0.0,50.4,1,0,0.0,50.4,1.0,0.0,0.4,0.0,1,1,50.4,50.4
|
||||
06-06 15:25:24,1,1,0.0,50.4,1,0,0.0,50.4,1.0,0.0,0.5,0.0,1,1,50.4,50.4
|
||||
06-06 15:25:29,1,1,0.0,50.4,1,0,0.0,50.4,1.0,0.0,0.6,0.0,1,1,50.4,50.4
|
||||
06-06 15:25:34,1,1,0.0,50.5,1,0,0.0,50.5,1.0,0.0,0.6,0.0,1,1,50.5,50.5
|
||||
06-06 15:25:39,1,1,0.0,50.5,1,0,0.0,50.5,1.0,0.0,0.7,0.0,1,1,50.5,50.5
|
||||
06-06 15:25:44,1,1,0.0,49.9,1,0,0.0,49.9,1.0,0.0,0.8,0.0,1,1,49.9,49.9
|
||||
06-06 15:25:49,1,1,0.0,49.7,1,0,0.0,49.7,1.0,0.0,0.8,0.0,1,1,49.7,49.7
|
||||
06-06 15:25:54,1,1,0.0,50.3,1,0,0.0,50.3,1.0,0.0,0.9,0.0,1,1,50.3,50.3
|
||||
06-06 15:25:59,1,1,0.0,49.9,1,0,0.0,49.9,1.0,0.0,1.0,0.0,1,1,49.9,49.9
|
||||
06-06 15:26:04,1,1,0.0,49.9,1,0,0.0,49.9,1.0,0.0,1.1,0.0,1,1,49.9,49.9
|
||||
06-06 15:26:09,1,1,0.0,49.8,1,0,0.0,49.8,1.0,0.0,1.1,0.0,1,1,49.8,49.8
|
||||
06-06 15:26:14,1,1,227.3,88.0,4,0,227.3,88.0,4.0,0.0,1.5,45.9,4,4,88.0,88.0
|
||||
06-06 15:26:19,1,1,0.0,172.5,3,0,0.0,172.5,3.0,0.0,1.6,45.9,3,3,172.5,172.5
|
||||
06-06 15:26:24,1,1,34.3,166.9,3,0,34.3,166.9,3.0,0.0,1.8,56.9,3,3,166.9,166.9
|
||||
06-06 15:26:29,1,1,33.0,167.9,4,0,33.0,167.9,4.0,0.0,2.3,64.3,4,4,167.9,167.9
|
||||
06-06 15:26:34,1,1,63.5,163.7,4,0,63.5,163.7,4.0,0.0,2.6,69.4,4,4,163.7,163.7
|
||||
06-06 15:26:39,1,1,59.1,164.9,4,0,59.1,164.9,4.0,0.0,2.9,73.9,4,4,164.9,164.9
|
||||
06-06 15:26:44,1,1,0.0,178.6,4,0,0.0,178.6,4.0,0.0,3.2,73.9,4,4,178.6,178.6
|
||||
06-06 15:26:49,1,1,0.0,178.2,3,0,0.0,178.2,3.0,0.0,2.9,73.9,3,3,178.2,178.2
|
||||
06-06 15:26:54,1,1,345.1,145.7,4,0,345.1,145.7,4.0,0.0,3.7,73.1,4,4,145.7,145.7
|
||||
06-06 15:26:59,1,1,0.0,178.6,4,0,0.0,178.6,4.0,0.0,3.9,73.1,4,4,178.6,178.6
|
||||
06-06 15:27:04,1,1,0.0,177.8,4,0,0.0,177.8,4.0,0.0,4.2,73.1,4,4,177.8,177.8
|
||||
06-06 15:27:09,1,1,68.2,212.6,5,0,68.2,212.6,5.0,0.0,4.6,72.8,5,5,212.6,212.6
|
||||
06-06 15:27:14,1,1,185.4,204.9,5,0,185.4,204.9,5.0,0.0,5.1,73.8,5,5,204.9,204.9
|
||||
06-06 15:27:19,1,1,100.6,207.6,5,0,100.6,207.6,5.0,0.0,5.5,76.1,5,5,207.6,207.6
|
||||
06-06 15:27:24,1,1,0.0,221.8,5,0,0.0,221.8,5.0,0.0,5.9,76.1,5,5,221.8,221.8
|
||||
06-06 15:27:29,1,1,0.0,222.3,5,0,0.0,222.3,5.0,0.0,6.2,76.1,5,5,222.3,222.3
|
||||
06-06 15:27:34,1,1,116.6,202.3,5,0,116.6,202.3,5.0,0.0,6.6,78.3,5,5,202.3,202.3
|
||||
06-06 15:27:39,1,1,0.0,224.2,5,0,0.0,224.2,5.0,0.0,6.9,78.3,5,5,224.2,224.2
|
||||
06-06 15:27:44,1,1,0.0,180.4,4,0,0.0,180.4,4.0,0.0,5.8,78.3,4,4,180.4,180.4
|
||||
06-06 15:27:49,1,1,0.0,178.9,4,0,0.0,178.9,4.0,0.0,6.0,78.3,4,4,178.9,178.9
|
||||
06-06 15:27:54,1,1,0.0,179.0,4,0,0.0,179.0,4.0,0.0,6.3,78.3,4,4,179.0,179.0
|
||||
06-06 15:27:59,1,1,0.0,168.4,3,0,0.0,168.4,3.0,0.0,5.8,78.3,3,3,168.4,168.4
|
||||
06-06 15:28:04,1,1,0.0,126.8,3,0,0.0,126.8,3.0,0.0,5.3,77.5,3,3,126.8,126.8
|
||||
06-06 15:28:09,1,1,0.0,126.9,3,0,0.0,126.9,3.0,0.0,5.5,77.5,3,3,126.9,126.9
|
||||
06-06 15:28:14,1,1,8.2,126.4,4,0,8.2,126.4,4.0,0.0,5.7,78.6,4,4,126.4,126.4
|
||||
06-06 15:28:19,1,1,0.0,168.0,4,0,0.0,168.0,4.0,0.0,5.9,78.6,4,4,168.0,168.0
|
||||
06-06 15:28:24,1,1,0.0,168.4,4,0,0.0,168.4,4.0,0.0,6.2,78.6,4,4,168.4,168.4
|
||||
06-06 15:28:29,1,1,0.0,168.1,4,0,0.0,168.1,4.0,0.0,6.4,78.6,4,4,168.1,168.1
|
||||
06-06 15:28:34,1,1,52.2,50.2,1,0,52.2,50.2,1.0,0.0,0.6,81.2,1,1,50.2,50.2
|
||||
06-06 15:28:40,1,1,0.0,171.2,3,0,0.0,171.2,3.0,0.0,5.4,89.5,3,3,171.2,171.2
|
||||
06-06 15:28:45,1,1,41.7,162.0,4,0,41.7,162.0,4.0,0.0,6.0,89.5,4,4,162.0,162.0
|
||||
06-06 15:28:50,1,1,0.0,161.4,3,0,0.0,161.4,3.0,0.0,5.8,89.5,3,3,161.4,161.4
|
||||
06-06 15:28:55,1,1,56.6,153.5,4,0,56.6,153.5,4.0,0.0,6.5,89.5,4,4,153.5,153.5
|
||||
06-06 15:29:00,1,1,23.0,165.7,4,0,23.0,165.7,4.0,0.0,6.8,89.7,4,4,165.7,165.7
|
||||
06-06 15:29:05,1,1,0.0,180.3,4,0,0.0,180.3,4.0,0.0,7.0,89.7,4,4,180.3,180.3
|
||||
06-06 15:29:10,1,1,0.0,141.6,3,0,0.0,141.6,3.0,0.0,6.5,89.7,3,3,141.6,141.6
|
||||
06-06 15:29:15,1,1,0.0,135.6,3,0,0.0,135.6,3.0,0.0,6.7,89.7,3,3,135.6,135.6
|
||||
06-06 15:29:20,1,1,120.8,176.1,4,0,120.8,176.1,4.0,0.0,7.1,89.1,4,4,176.1,176.1
|
||||
06-06 15:29:25,1,1,0.0,180.2,4,0,0.0,180.2,4.0,0.0,7.4,89.1,4,4,180.2,180.2
|
||||
06-06 15:29:30,1,1,0.0,179.7,4,0,0.0,179.7,4.0,0.0,7.6,89.1,4,4,179.7,179.7
|
||||
06-06 15:29:35,1,1,0.0,179.6,4,0,0.0,179.6,4.0,0.0,7.9,89.1,4,4,179.6,179.6
|
||||
06-06 15:29:40,1,1,0.0,180.0,4,0,0.0,180.0,4.0,0.0,8.1,89.1,4,4,180.0,180.0
|
||||
06-06 15:29:45,1,1,0.0,180.2,4,0,0.0,180.2,4.0,0.0,8.4,89.1,4,4,180.2,180.2
|
||||
06-06 15:29:50,1,1,0.0,180.4,4,0,0.0,180.4,4.0,0.0,8.6,89.1,4,4,180.4,180.4
|
||||
06-06 15:29:55,1,1,0.0,180.7,4,0,0.0,180.7,4.0,0.0,8.9,89.1,4,4,180.7,180.7
|
||||
06-06 15:30:00,1,1,0.0,180.5,4,0,0.0,180.5,4.0,0.0,9.1,89.1,4,4,180.5,180.5
|
||||
06-06 15:30:05,1,1,0.0,168.3,3,0,0.0,168.3,3.0,0.0,8.6,89.1,3,3,168.3,168.3
|
||||
06-06 15:30:10,1,1,138.2,177.4,4,0,138.2,177.4,4.0,0.0,9.3,88.7,4,4,177.4,177.4
|
||||
06-06 15:30:15,1,1,22.7,170.5,4,0,22.7,170.5,4.0,0.0,9.6,88.9,4,4,170.5,170.5
|
||||
06-06 15:30:20,1,1,0.0,180.6,4,0,0.0,180.6,4.0,0.0,9.8,88.9,4,4,180.6,180.6
|
||||
06-06 15:30:25,1,1,0.0,48.0,1,0,0.0,48.0,1.0,0.0,0.7,67.8,1,1,48.0,48.0
|
||||
06-06 15:30:30,1,1,88.8,31.1,1,0,88.8,31.1,1.0,0.0,0.8,74.5,1,1,31.1,31.1
|
||||
06-06 15:30:35,1,1,77.2,31.7,1,0,77.2,31.7,1.0,0.0,1.0,79.1,1,1,31.7,31.7
|
||||
06-06 15:30:40,1,1,114.1,56.5,2,0,114.1,56.5,2.0,0.0,1.2,76.5,2,2,56.5,56.5
|
||||
06-06 15:30:45,1,1,73.5,75.3,2,0,73.5,75.3,2.0,0.0,1.4,80.3,2,2,75.3,75.3
|
||||
06-06 15:30:50,1,1,0.0,92.6,2,0,0.0,92.6,2.0,0.0,7.6,83.1,2,2,92.6,92.6
|
||||
06-06 15:30:55,1,1,0.0,92.4,2,0,0.0,92.4,2.0,0.0,7.7,83.1,2,2,92.4,92.4
|
||||
06-06 15:31:00,1,1,0.0,92.9,2,0,0.0,92.9,2.0,0.0,7.8,83.1,2,2,92.9,92.9
|
||||
06-06 15:31:05,1,1,0.0,92.9,2,0,0.0,92.9,2.0,0.0,8.0,83.1,2,2,92.9,92.9
|
||||
06-06 15:31:10,1,1,0.0,93.1,2,0,0.0,93.1,2.0,0.0,8.1,83.1,2,2,93.1,93.1
|
||||
06-06 15:31:15,1,1,0.0,93.2,2,0,0.0,93.2,2.0,0.0,8.2,83.1,2,2,93.2,93.2
|
||||
06-06 15:31:20,1,1,0.0,92.9,2,0,0.0,92.9,2.0,0.0,8.4,83.1,2,2,92.9,92.9
|
||||
06-06 15:31:25,1,1,61.4,115.2,3,0,61.4,115.2,3.0,0.0,8.6,82.8,3,3,115.2,115.2
|
||||
06-06 15:31:31,1,1,70.0,161.1,4,0,70.0,161.1,4.0,0.0,8.9,82.4,4,4,161.1,161.1
|
||||
06-06 15:31:36,1,1,0.0,186.9,4,0,0.0,186.9,4.0,0.0,9.2,82.4,4,4,186.9,186.9
|
||||
06-06 15:31:41,1,1,0.0,178.8,3,0,0.0,178.8,3.0,0.0,9.2,82.4,3,3,178.8,178.8
|
||||
06-06 15:31:46,1,1,0.0,140.2,3,0,0.0,140.2,3.0,0.0,9.4,82.4,3,3,140.2,140.2
|
||||
06-06 15:31:51,1,1,0.0,139.8,3,0,0.0,139.8,3.0,0.0,9.6,82.4,3,3,139.8,139.8
|
||||
06-06 15:31:56,1,1,0.0,140.3,3,0,0.0,140.3,3.0,0.0,9.8,82.4,3,3,140.3,140.3
|
||||
06-06 15:32:01,1,1,0.0,89.1,2,0,0.0,89.1,2.0,0.0,6.3,87.1,2,2,89.1,89.1
|
||||
06-06 15:32:06,1,1,0.0,89.4,2,0,0.0,89.4,2.0,0.0,6.4,87.1,2,2,89.4,89.4
|
||||
06-06 15:32:11,1,1,0.0,89.2,2,0,0.0,89.2,2.0,0.0,6.5,87.1,2,2,89.2,89.2
|
||||
06-06 15:32:16,1,1,0.0,89.0,2,0,0.0,89.0,2.0,0.0,6.6,87.1,2,2,89.0,89.0
|
||||
06-06 15:32:21,1,1,0.0,89.0,2,0,0.0,89.0,2.0,0.0,6.8,87.1,2,2,89.0,89.0
|
||||
06-06 15:32:26,1,1,0.0,89.2,2,0,0.0,89.2,2.0,0.0,6.9,87.1,2,2,89.2,89.2
|
||||
06-06 15:32:31,1,1,79.0,92.6,2,0,79.0,92.6,2.0,0.0,5.2,89.5,2,2,92.6,92.6
|
||||
06-06 15:32:36,1,1,0.0,87.4,2,0,0.0,87.4,2.0,0.0,5.3,89.5,2,2,87.4,87.4
|
||||
06-06 15:32:41,1,1,32.9,122.8,3,0,32.9,122.8,3.0,0.0,5.7,89.5,3,3,122.8,122.8
|
||||
06-06 15:32:46,1,1,0.0,103.5,2,0,0.0,103.5,2.0,0.0,5.5,89.5,2,2,103.5,103.5
|
||||
06-06 15:32:51,1,1,0.0,87.0,2,0,0.0,87.0,2.0,0.0,5.6,89.5,2,2,87.0,87.0
|
||||
06-06 15:32:56,1,1,0.0,87.4,2,0,0.0,87.4,2.0,0.0,5.8,89.5,2,2,87.4,87.4
|
||||
06-06 15:33:01,1,1,174.4,60.2,2,0,174.4,60.2,2.0,0.0,5.8,89.0,2,2,60.2,60.2
|
||||
06-06 15:33:06,1,1,0.0,71.4,1,0,0.0,71.4,1.0,0.0,5.4,89.0,1,1,71.4,71.4
|
||||
06-06 15:33:11,1,1,25.4,71.3,2,0,25.4,71.3,2.0,0.0,6.0,89.1,2,2,71.3,71.3
|
||||
06-06 15:33:16,1,1,0.0,87.1,2,0,0.0,87.1,2.0,0.0,6.1,89.1,2,2,87.1,87.1
|
||||
06-06 15:33:21,1,1,0.0,87.3,2,0,0.0,87.3,2.0,0.0,6.3,89.1,2,2,87.3,87.3
|
||||
06-06 15:33:26,1,1,0.0,87.3,2,0,0.0,87.3,2.0,0.0,6.4,89.1,2,2,87.3,87.3
|
||||
06-06 15:33:31,1,1,0.0,87.5,2,0,0.0,87.5,2.0,0.0,6.5,89.1,2,2,87.5,87.5
|
||||
06-06 15:33:36,1,1,0.0,87.3,2,0,0.0,87.3,2.0,0.0,6.6,89.1,2,2,87.3,87.3
|
||||
06-06 15:33:41,1,1,0.0,87.3,2,0,0.0,87.3,2.0,0.0,6.8,89.1,2,2,87.3,87.3
|
||||
06-06 15:33:46,1,1,0.0,87.4,2,0,0.0,87.4,2.0,0.0,6.9,89.1,2,2,87.4,87.4
|
||||
06-06 15:33:51,1,1,0.0,87.2,2,0,0.0,87.2,2.0,0.0,7.0,89.1,2,2,87.2,87.2
|
||||
06-06 15:33:56,1,1,0.0,81.0,1,0,0.0,81.0,1.0,0.0,6.0,89.1,1,1,81.0,81.0
|
||||
06-06 15:34:01,1,1,51.2,63.2,2,0,51.2,63.2,2.0,0.0,7.3,89.3,2,2,63.2,63.2
|
||||
06-06 15:34:06,1,1,0.0,86.5,2,0,0.0,86.5,2.0,0.0,7.4,89.3,2,2,86.5,86.5
|
||||
06-06 15:34:11,1,1,0.0,86.9,2,0,0.0,86.9,2.0,0.0,7.5,89.3,2,2,86.9,86.9
|
||||
06-06 15:34:16,1,1,0.0,86.9,2,0,0.0,86.9,2.0,0.0,7.6,89.3,2,2,86.9,86.9
|
||||
06-06 15:34:21,1,1,37.1,104.9,3,0,37.1,104.9,3.0,0.0,8.1,89.3,3,3,104.9,104.9
|
||||
06-06 15:34:26,1,1,0.0,112.3,2,0,0.0,112.3,2.0,0.0,7.9,89.3,2,2,112.3,112.3
|
||||
06-06 15:34:31,1,1,0.0,86.8,2,0,0.0,86.8,2.0,0.0,8.0,89.3,2,2,86.8,86.8
|
||||
06-06 15:34:36,1,1,0.0,86.9,2,0,0.0,86.9,2.0,0.0,8.1,89.3,2,2,86.9,86.9
|
||||
06-06 15:34:41,1,1,0.0,87.0,2,0,0.0,87.0,2.0,0.0,8.2,89.3,2,2,87.0,87.0
|
||||
06-06 15:34:46,1,1,79.5,120.4,3,0,79.5,120.4,3.0,0.0,8.5,89.1,3,3,120.4,120.4
|
||||
06-06 15:34:51,1,1,0.0,150.6,2,0,0.0,150.6,2.0,0.0,10.8,85.0,2,2,150.6,150.6
|
||||
06-06 15:34:56,1,1,22.9,140.9,3,0,22.9,140.9,3.0,0.0,12.2,86.2,3,3,140.9,140.9
|
||||
06-06 15:35:01,1,1,0.0,143.2,2,0,0.0,143.2,2.0,0.0,11.1,86.2,2,2,143.2,143.2
|
||||
06-06 15:35:06,1,1,123.2,108.9,3,0,123.2,108.9,3.0,0.0,11.4,85.4,3,3,108.9,108.9
|
||||
06-06 15:35:11,1,1,0.0,152.1,3,0,0.0,152.1,3.0,0.0,11.7,85.4,3,3,152.1,152.1
|
||||
06-06 15:35:16,1,1,68.2,244.4,5,0,68.2,244.4,5.0,0.0,12.1,85.3,5,5,244.4,244.4
|
||||
06-06 15:35:21,1,1,0.0,252.3,5,0,0.0,252.3,5.0,0.0,12.4,85.3,5,5,252.3,252.3
|
||||
06-06 15:35:27,1,1,83.0,216.1,5,0,83.0,216.1,5.0,0.0,12.9,85.1,5,5,216.1,216.1
|
||||
06-06 15:35:32,1,1,0.0,137.1,3,0,0.0,137.1,3.0,0.0,13.3,84.3,3,3,137.1,137.1
|
||||
06-06 15:35:37,1,1,69.8,107.1,3,0,69.8,107.1,3.0,0.0,13.6,84.3,3,3,107.1,107.1
|
||||
06-06 15:35:42,1,1,0.0,137.0,3,0,0.0,137.0,3.0,0.0,13.8,84.3,3,3,137.0,137.0
|
||||
06-06 15:35:47,1,1,189.7,75.9,2,0,189.7,75.9,2.0,0.0,3.9,87.1,2,2,75.9,75.9
|
||||
06-06 15:35:52,1,1,87.3,60.6,2,0,87.3,60.6,2.0,0.0,4.1,87.1,2,2,60.6,60.6
|
||||
06-06 15:35:57,1,1,58.9,92.2,3,0,58.9,92.2,3.0,0.0,4.3,86.9,3,3,92.2,92.2
|
||||
06-06 15:36:02,1,1,21.2,111.1,3,0,21.2,111.1,3.0,0.0,4.5,87.3,3,3,111.1,111.1
|
||||
06-06 15:36:07,1,1,161.0,93.6,1,0,161.0,93.6,1.0,0.0,3.7,88.2,1,1,93.6,93.6
|
||||
06-06 15:36:12,1,1,0.0,168.0,4,0,0.0,168.0,4.0,0.0,19.9,89.7,4,4,168.0,168.0
|
||||
06-06 15:36:17,1,1,0.0,168.5,4,0,0.0,168.5,4.0,0.0,20.1,89.7,4,4,168.5,168.5
|
||||
06-06 15:36:22,1,1,0.0,91.9,2,0,0.0,91.9,2.0,0.0,14.3,84.6,2,2,91.9,91.9
|
||||
06-06 15:36:27,1,1,3.2,202.7,5,0,3.2,202.7,5.0,0.0,20.7,90.0,5,5,202.7,202.7
|
||||
06-06 15:36:32,1,1,155.1,237.2,6,0,155.1,237.2,6.0,0.0,21.9,89.6,6,6,237.2,237.2
|
||||
06-06 15:36:37,1,1,0.0,252.2,6,0,0.0,252.2,6.0,0.0,22.3,89.6,6,6,252.2,252.2
|
||||
06-06 15:36:42,1,1,113.9,413.4,8,0,113.9,413.4,8.0,0.0,43.1,89.1,8,8,413.4,413.4
|
||||
06-06 15:36:47,1,1,31.1,397.2,8,0,31.1,397.2,8.0,0.0,42.2,89.1,8,8,397.2,397.2
|
||||
06-06 15:36:52,1,1,22.5,369.1,8,0,22.5,369.1,8.0,0.0,35.5,89.4,8,8,369.1,369.1
|
||||
06-06 15:36:57,1,1,95.3,364.7,8,0,95.3,364.7,8.0,0.0,34.3,89.4,8,8,364.7,364.7
|
||||
06-06 15:37:02,1,1,79.2,409.9,9,0,79.2,409.9,9.0,0.0,35.0,89.2,9,9,409.9,409.9
|
||||
06-06 15:37:07,1,1,35.8,403.3,9,0,35.8,403.3,9.0,0.0,35.6,89.2,9,9,403.3,403.3
|
||||
06-06 15:37:12,1,1,0.0,226.0,5,0,0.0,226.0,5.0,0.0,18.2,90.0,5,5,226.0,226.0
|
||||
06-06 15:37:17,1,1,0.0,220.3,5,0,0.0,220.3,5.0,0.0,18.5,90.0,5,5,220.3,220.3
|
||||
06-06 15:37:22,1,1,0.0,217.2,4,0,0.0,217.2,4.0,0.0,17.0,92.6,4,4,217.2,217.2
|
||||
06-06 15:37:27,1,1,151.2,209.6,5,0,151.2,209.6,5.0,0.0,18.8,92.6,5,5,209.6,209.6
|
||||
06-06 15:37:32,1,1,79.0,267.0,6,0,79.0,267.0,6.0,0.0,19.3,92.5,6,6,267.0,267.0
|
||||
06-06 15:37:37,1,1,102.7,312.2,7,0,102.7,312.2,7.0,0.0,29.0,89.2,7,7,312.2,312.2
|
||||
06-06 15:37:42,1,1,36.0,304.1,6,0,36.0,304.1,6.0,0.0,20.7,89.3,6,6,304.1,304.1
|
||||
06-06 15:37:47,1,1,0.0,269.0,6,0,0.0,269.0,6.0,0.0,20.8,92.4,6,6,269.0,269.0
|
||||
06-06 15:37:52,1,1,0.0,253.7,5,0,0.0,253.7,5.0,0.0,19.3,92.4,5,5,253.7,253.7
|
||||
06-06 15:37:57,1,1,166.4,242.8,5,0,166.4,242.8,5.0,0.0,19.7,92.3,5,5,242.8,242.8
|
||||
06-06 15:38:02,1,1,155.6,269.9,6,0,155.6,269.9,6.0,0.0,22.3,92.3,6,6,269.9,269.9
|
||||
06-06 15:38:07,1,1,216.5,309.3,8,0,216.5,309.3,8.0,0.0,23.3,89.0,8,8,309.3,309.3
|
||||
06-06 15:38:12,1,1,303.4,386.5,9,0,303.4,386.5,9.0,0.0,30.1,93.7,9,9,386.5,386.5
|
||||
06-06 15:38:17,1,1,154.5,453.4,10,0,154.5,453.4,10.0,0.0,30.1,93.6,10,10,453.4,453.4
|
||||
06-06 15:38:22,1,1,105.5,478.2,10,0,105.5,478.2,10.0,0.0,32.4,93.6,10,10,478.2,478.2
|
||||
06-06 15:38:27,1,1,148.1,369.0,8,0,148.1,369.0,8.0,0.0,30.7,93.5,8,8,369.0,369.0
|
||||
06-06 15:38:32,1,1,412.3,462.5,11,0,412.3,462.5,11.0,0.0,36.6,94.1,11,11,462.5,462.5
|
||||
06-06 15:38:37,1,1,67.6,503.9,11,0,67.6,503.9,11.0,0.0,37.4,94.1,11,11,503.9,503.9
|
||||
06-06 15:38:42,1,1,0.0,471.9,9,0,0.0,471.9,9.0,0.0,37.3,94.1,9,9,471.9,471.9
|
||||
06-06 15:38:47,1,1,73.0,471.6,10,0,73.0,471.6,10.0,0.0,38.6,94.1,10,10,471.6,471.6
|
||||
06-06 15:38:52,1,1,0.8,512.1,11,0,0.8,512.1,11.0,0.0,39.3,94.1,11,11,512.1,512.1
|
||||
06-06 15:38:57,1,1,22.8,433.3,9,0,22.8,433.3,9.0,0.0,28.3,94.1,9,9,433.3,433.3
|
||||
06-06 15:39:02,1,1,230.4,501.8,11,0,230.4,501.8,11.0,0.0,34.3,94.1,11,11,501.8,501.8
|
||||
06-06 15:39:07,1,1,0.0,508.7,10,0,0.0,508.7,10.0,0.0,34.1,94.1,10,10,508.7,508.7
|
||||
06-06 15:39:13,1,1,52.7,400.5,7,0,52.7,400.5,7.0,0.0,16.9,91.4,7,7,400.5,400.5
|
||||
06-06 15:39:18,1,1,39.6,395.8,9,0,39.6,395.8,9.0,0.0,19.1,91.5,9,9,395.8,395.8
|
||||
06-06 15:39:23,1,1,7.2,412.8,9,0,7.2,412.8,9.0,0.0,19.6,91.6,9,9,412.8,412.8
|
||||
06-06 15:39:28,1,1,63.8,410.4,9,0,63.8,410.4,9.0,0.0,19.2,91.5,9,9,410.4,410.4
|
||||
06-06 15:39:33,1,1,76.3,473.0,10,0,76.3,473.0,10.0,0.0,20.4,91.6,10,10,473.0,473.0
|
||||
06-06 15:39:38,1,1,43.3,485.8,9,0,43.3,485.8,9.0,0.0,20.4,91.6,9,9,485.8,485.8
|
||||
06-06 15:39:43,1,1,9.2,452.5,10,0,9.2,452.5,10.0,0.0,22.3,91.7,10,10,452.5,452.5
|
||||
06-06 15:39:48,1,1,75.3,495.6,10,0,75.3,495.6,10.0,0.0,21.8,91.7,10,10,495.6,495.6
|
||||
06-06 15:39:53,1,1,25.0,478.2,9,0,25.0,478.2,9.0,0.0,22.3,91.8,9,9,478.2,478.2
|
||||
06-06 15:39:58,1,1,29.1,455.6,10,0,29.1,455.6,10.0,0.0,24.3,91.8,10,10,455.6,455.6
|
||||
06-06 15:40:03,1,1,0.0,472.5,9,0,0.0,472.5,9.0,0.0,23.5,91.8,9,9,472.5,472.5
|
||||
06-06 15:40:08,1,1,0.0,442.8,9,0,0.0,442.8,9.0,0.0,24.1,91.8,9,9,442.8,442.8
|
||||
06-06 15:40:13,1,1,0.0,277.5,6,0,0.0,277.5,6.0,0.0,26.6,83.8,6,6,277.5,277.5
|
||||
06-06 15:40:18,1,1,0.0,273.7,5,0,0.0,273.7,5.0,0.0,24.4,83.8,5,5,273.7,273.7
|
||||
06-06 15:40:23,1,1,256.0,257.4,6,0,256.0,257.4,6.0,0.0,27.7,84.1,6,6,257.4,257.4
|
||||
06-06 15:40:28,1,1,0.0,244.7,5,0,0.0,244.7,5.0,0.0,25.0,84.1,5,5,244.7,244.7
|
||||
06-06 15:40:33,1,1,0.0,227.5,5,0,0.0,227.5,5.0,0.0,25.3,84.1,5,5,227.5,227.5
|
||||
06-06 15:40:38,1,1,0.0,208.4,4,0,0.0,208.4,4.0,0.0,16.7,84.1,4,4,208.4,208.4
|
||||
06-06 15:40:43,1,1,95.2,194.4,5,0,95.2,194.4,5.0,0.0,17.1,83.9,5,5,194.4,194.4
|
||||
06-06 15:40:48,1,1,0.0,228.3,4,0,0.0,228.3,4.0,0.0,17.2,83.9,4,4,228.3,228.3
|
||||
06-06 15:40:53,1,1,40.6,191.4,5,0,40.6,191.4,5.0,0.0,17.7,84.0,5,5,191.4,191.4
|
||||
06-06 15:40:58,1,1,0.0,230.7,5,0,0.0,230.7,5.0,0.0,18.1,84.0,5,5,230.7,230.7
|
||||
06-06 15:41:03,1,1,0.0,230.6,5,0,0.0,230.6,5.0,0.0,18.4,84.0,5,5,230.6,230.6
|
||||
06-06 15:41:08,1,1,0.0,230.3,5,0,0.0,230.3,5.0,0.0,18.7,84.0,5,5,230.3,230.3
|
||||
06-06 15:41:13,1,1,96.2,344.3,8,0,96.2,344.3,8.0,0.0,27.9,88.6,8,8,344.3,344.3
|
||||
06-06 15:41:18,1,1,0.0,358.9,8,0,0.0,358.9,8.0,0.0,28.4,88.6,8,8,358.9,358.9
|
||||
06-06 15:41:23,1,1,0.0,353.9,7,0,0.0,353.9,7.0,0.0,27.8,88.6,7,7,353.9,353.9
|
||||
06-06 15:41:28,1,1,23.9,364.4,9,0,23.9,364.4,9.0,0.0,29.5,89.0,9,9,364.4,364.4
|
||||
06-06 15:41:33,1,1,0.0,387.3,8,0,0.0,387.3,8.0,0.0,29.9,89.0,8,8,387.3,387.3
|
||||
06-06 15:41:38,1,1,34.5,361.5,8,0,34.5,361.5,8.0,0.0,29.2,89.0,8,8,361.5,361.5
|
||||
06-06 15:41:43,1,1,0.0,356.8,8,0,0.0,356.8,8.0,0.0,29.7,89.0,8,8,356.8,356.8
|
||||
06-06 15:41:48,1,1,0.0,356.9,8,0,0.0,356.9,8.0,0.0,30.2,89.0,8,8,356.9,356.9
|
||||
06-06 15:41:53,1,1,0.0,356.5,8,0,0.0,356.5,8.0,0.0,30.7,89.0,8,8,356.5,356.5
|
||||
06-06 15:41:58,1,1,0.0,356.7,8,0,0.0,356.7,8.0,0.0,31.2,89.0,8,8,356.7,356.7
|
||||
06-06 15:42:03,1,1,108.6,282.1,7,0,108.6,282.1,7.0,0.0,14.5,91.5,7,7,282.1,282.1
|
||||
06-06 15:42:08,1,1,8.8,333.0,7,0,8.8,333.0,7.0,0.0,14.7,91.5,7,7,333.0,333.0
|
||||
06-06 15:42:13,1,1,0.0,330.9,6,0,0.0,330.9,6.0,0.0,15.1,91.5,6,6,330.9,330.9
|
||||
06-06 15:42:18,1,1,132.4,362.7,9,0,132.4,362.7,9.0,0.0,33.4,88.9,9,9,362.7,362.7
|
||||
06-06 15:42:24,1,1,0.0,238.3,5,0,0.0,238.3,5.0,0.0,15.4,91.5,5,5,238.3,238.3
|
||||
06-06 15:42:29,1,1,0.0,238.9,5,0,0.0,238.9,5.0,0.0,15.7,91.5,5,5,238.9,238.9
|
||||
06-06 15:42:34,1,1,0.0,238.8,5,0,0.0,238.8,5.0,0.0,16.1,91.5,5,5,238.8,238.8
|
||||
06-06 15:42:39,1,1,0.0,239.1,5,0,0.0,239.1,5.0,0.0,16.4,91.5,5,5,239.1,239.1
|
||||
06-06 15:42:44,1,1,0.0,239.0,5,0,0.0,239.0,5.0,0.0,16.7,91.5,5,5,239.0,239.0
|
||||
06-06 15:42:49,1,1,60.1,251.5,6,0,60.1,251.5,6.0,0.0,17.5,91.5,6,6,251.5,251.5
|
||||
06-06 15:42:54,1,1,0.0,288.3,6,0,0.0,288.3,6.0,0.0,17.9,91.5,6,6,288.3,288.3
|
||||
06-06 15:42:59,1,1,0.0,287.5,6,0,0.0,287.5,6.0,0.0,18.3,91.5,6,6,287.5,287.5
|
||||
06-06 15:43:04,1,1,43.8,251.9,6,0,43.8,251.9,6.0,0.0,18.3,91.4,6,6,251.9,251.9
|
||||
06-06 15:43:09,1,1,49.5,315.6,7,0,49.5,315.6,7.0,0.0,19.1,91.4,7,7,315.6,315.6
|
||||
06-06 15:43:14,1,1,0.0,300.0,6,0,0.0,300.0,6.0,0.0,19.1,91.4,6,6,300.0,300.0
|
||||
06-06 15:43:19,1,1,0.0,286.5,6,0,0.0,286.5,6.0,0.0,19.5,91.4,6,6,286.5,286.5
|
||||
06-06 15:43:24,1,1,0.0,287.3,6,0,0.0,287.3,6.0,0.0,19.9,91.4,6,6,287.3,287.3
|
||||
06-06 15:43:29,1,1,0.0,285.6,6,0,0.0,285.6,6.0,0.0,20.3,91.4,6,6,285.6,285.6
|
||||
06-06 15:43:34,1,1,135.3,306.2,7,0,135.3,306.2,7.0,0.0,21.5,91.3,7,7,306.2,306.2
|
||||
06-06 15:43:39,1,1,60.1,345.1,8,0,60.1,345.1,8.0,0.0,22.5,91.3,8,8,345.1,345.1
|
||||
06-06 15:43:44,1,1,0.0,382.5,8,0,0.0,382.5,8.0,0.0,23.0,91.3,8,8,382.5,382.5
|
||||
06-06 15:43:49,1,1,0.0,361.6,7,0,0.0,361.6,7.0,0.0,22.9,91.3,7,7,361.6,361.6
|
||||
06-06 15:43:54,1,1,0.0,333.5,7,0,0.0,333.5,7.0,0.0,23.3,91.3,7,7,333.5,333.5
|
||||
06-06 15:43:59,1,1,0.0,297.0,6,0,0.0,297.0,6.0,0.0,22.7,91.3,6,6,297.0,297.0
|
||||
06-06 15:44:04,1,1,0.0,286.7,6,0,0.0,286.7,6.0,0.0,23.1,91.3,6,6,286.7,286.7
|
||||
06-06 15:44:09,1,1,71.0,308.2,7,0,71.0,308.2,7.0,0.0,23.6,91.2,7,7,308.2,308.2
|
||||
06-06 15:44:14,1,1,0.0,609.9,14,0,0.0,609.9,14.0,0.0,52.3,88.1,14,14,609.9,609.9
|
||||
06-06 15:44:19,1,1,109.1,595.8,14,0,109.1,595.8,14.0,0.0,53.3,88.0,14,14,595.8,595.8
|
||||
06-06 15:44:24,1,1,0.0,613.1,14,0,0.0,613.1,14.0,0.0,54.1,88.0,14,14,613.1,613.1
|
||||
06-06 15:44:29,1,1,35.1,593.2,14,0,35.1,593.2,14.0,0.0,55.0,88.1,14,14,593.2,593.2
|
||||
06-06 15:44:34,1,1,0.0,611.8,14,0,0.0,611.8,14.0,0.0,55.9,88.1,14,14,611.8,611.8
|
||||
06-06 15:44:39,1,1,22.6,600.2,14,0,22.6,600.2,14.0,0.0,56.7,88.2,14,14,600.2,600.2
|
||||
06-06 15:44:44,1,1,0.0,611.2,14,0,0.0,611.2,14.0,0.0,57.6,88.2,14,14,611.2,611.2
|
||||
06-06 15:44:49,1,1,0.0,223.6,5,0,0.0,223.6,5.0,0.0,20.3,91.0,5,5,223.6,223.6
|
||||
06-06 15:44:54,1,1,0.0,223.2,5,0,0.0,223.2,5.0,0.0,20.6,91.0,5,5,223.2,223.2
|
||||
06-06 15:44:59,1,1,0.0,223.2,5,0,0.0,223.2,5.0,0.0,20.9,91.0,5,5,223.2,223.2
|
||||
06-06 15:45:04,1,1,0.0,223.6,5,0,0.0,223.6,5.0,0.0,21.2,91.0,5,5,223.6,223.6
|
||||
06-06 15:45:09,1,1,0.0,223.6,5,0,0.0,223.6,5.0,0.0,21.5,91.0,5,5,223.6,223.6
|
||||
06-06 15:45:14,1,1,0.0,223.3,5,0,0.0,223.3,5.0,0.0,21.8,91.0,5,5,223.3,223.3
|
||||
06-06 15:45:19,1,1,0.0,188.6,4,0,0.0,188.6,4.0,0.0,19.8,91.0,4,4,188.6,188.6
|
||||
06-06 15:45:24,1,1,200.3,164.2,4,0,200.3,164.2,4.0,0.0,20.3,91.1,4,4,164.2,164.2
|
||||
06-06 15:45:29,1,1,0.0,176.0,3,0,0.0,176.0,3.0,0.0,16.6,91.1,3,3,176.0,176.0
|
||||
06-06 15:45:34,1,1,73.5,153.1,4,0,73.5,153.1,4.0,0.0,20.9,91.4,4,4,153.1,153.1
|
||||
06-06 15:45:40,1,1,0.0,399.2,9,0,0.0,399.2,9.0,0.0,48.2,90.8,9,9,399.2,399.2
|
||||
06-06 15:45:45,1,1,0.0,399.1,9,0,0.0,399.1,9.0,0.0,48.8,90.8,9,9,399.1,399.1
|
||||
06-06 15:45:50,1,1,0.0,399.3,9,0,0.0,399.3,9.0,0.0,49.3,90.8,9,9,399.3,399.3
|
||||
06-06 15:45:55,1,1,0.0,396.8,9,0,0.0,396.8,9.0,0.0,49.9,90.8,9,9,396.8,396.8
|
||||
06-06 15:46:00,1,1,0.0,349.5,7,0,0.0,349.5,7.0,0.0,42.4,88.8,7,7,349.5,349.5
|
||||
06-06 15:46:05,1,1,0.0,349.5,7,0,0.0,349.5,7.0,0.0,42.9,88.8,7,7,349.5,349.5
|
||||
06-06 15:46:10,1,1,67.2,367.3,8,0,67.2,367.3,8.0,0.0,43.5,88.7,8,8,367.3,367.3
|
||||
06-06 15:46:15,1,1,0.0,399.7,8,0,0.0,399.7,8.0,0.0,44.0,88.7,8,8,399.7,399.7
|
||||
06-06 15:46:20,1,1,0.0,399.0,8,0,0.0,399.0,8.0,0.0,44.6,88.7,8,8,399.0,399.0
|
||||
06-06 15:46:25,1,1,0.0,357.5,7,0,0.0,357.5,7.0,0.0,44.8,88.7,7,7,357.5,357.5
|
||||
06-06 15:46:30,1,1,90.3,398.6,8,0,90.3,398.6,8.0,0.0,45.6,88.6,8,8,398.6,398.6
|
||||
06-06 15:46:35,1,1,0.0,350.7,7,0,0.0,350.7,7.0,0.0,45.8,88.6,7,7,350.7,350.7
|
||||
06-06 15:46:40,1,1,116.0,424.4,9,0,116.0,424.4,9.0,0.0,49.0,88.9,9,9,424.4,424.4
|
||||
06-06 15:46:45,1,1,0.0,450.7,9,0,0.0,450.7,9.0,0.0,49.6,88.9,9,9,450.7,450.7
|
||||
06-06 15:46:50,1,1,0.8,461.7,10,0,0.8,461.7,10.0,0.0,50.3,88.9,10,10,461.7,461.7
|
||||
06-06 15:46:55,1,1,0.0,499.2,10,0,0.0,499.2,10.0,0.0,51.0,88.9,10,10,499.2,499.2
|
||||
06-06 15:47:00,1,1,0.0,497.4,10,0,0.0,497.4,10.0,0.0,51.7,88.9,10,10,497.4,497.4
|
||||
06-06 15:47:05,1,1,0.0,416.7,8,0,0.0,416.7,8.0,0.0,40.7,88.9,8,8,416.7,416.7
|
||||
06-06 15:47:10,1,1,1.6,431.9,10,0,1.6,431.9,10.0,0.0,40.2,89.2,10,10,431.9,431.9
|
||||
06-06 15:47:15,1,1,0.0,397.3,8,0,0.0,397.3,8.0,0.0,41.9,88.9,8,8,397.3,397.3
|
||||
06-06 15:47:20,1,1,0.0,451.8,9,0,0.0,451.8,9.0,0.0,41.0,89.5,9,9,451.8,451.8
|
||||
06-06 15:47:25,1,1,101.3,591.2,13,0,101.3,591.2,13.0,0.0,52.5,86.2,13,13,591.2,591.2
|
||||
06-06 15:47:30,1,1,36.4,597.3,14,0,36.4,597.3,14.0,0.0,53.7,86.3,14,14,597.3,597.3
|
||||
06-06 15:47:35,1,1,64.1,597.8,14,0,64.1,597.8,14.0,0.0,54.6,86.3,14,14,597.8,597.8
|
||||
06-06 15:47:40,1,1,0.0,472.9,10,0,0.0,472.9,10.0,0.0,49.9,88.7,10,10,472.9,472.9
|
||||
06-06 15:47:45,1,1,0.0,427.5,9,0,0.0,427.5,9.0,0.0,50.0,88.7,9,9,427.5,427.5
|
||||
06-06 15:47:50,1,1,0.0,646.7,14,0,0.0,646.7,14.0,0.0,56.3,86.3,14,14,646.7,646.7
|
||||
06-06 15:47:55,1,1,0.0,612.4,14,0,0.0,612.4,14.0,0.0,57.2,86.3,14,14,612.4,612.4
|
||||
06-06 15:48:00,1,1,0.0,613.3,14,0,0.0,613.3,14.0,0.0,58.0,86.3,14,14,613.3,613.3
|
||||
06-06 15:48:05,1,1,119.4,389.2,9,0,119.4,389.2,9.0,0.0,36.4,90.7,9,9,389.2,389.2
|
||||
06-06 15:48:10,1,1,269.8,412.0,10,0,269.8,412.0,10.0,0.0,37.5,90.6,10,10,412.0,412.0
|
||||
06-06 15:48:15,1,1,77.3,471.2,11,0,77.3,471.2,11.0,0.0,38.3,90.5,11,11,471.2,471.2
|
||||
06-06 15:48:20,1,1,9.2,525.7,12,0,9.2,525.7,12.0,0.0,54.4,88.9,12,12,525.7,525.7
|
||||
06-06 15:48:25,1,1,315.4,450.8,10,0,315.4,450.8,10.0,0.0,39.1,90.5,10,10,450.8,450.8
|
||||
06-06 15:48:30,1,1,148.8,450.5,11,0,148.8,450.5,11.0,0.0,40.8,90.4,11,11,450.5,450.5
|
||||
06-06 15:48:35,1,1,334.7,461.6,12,0,334.7,461.6,12.0,0.0,42.0,90.6,12,12,461.6,461.6
|
||||
06-06 15:48:40,1,1,30.6,487.6,12,0,30.6,487.6,12.0,0.0,41.1,90.9,12,12,487.6,487.6
|
||||
06-06 15:48:45,1,1,266.0,520.4,12,0,266.0,520.4,12.0,0.0,42.0,90.9,12,12,520.4,520.4
|
||||
06-06 15:48:50,1,1,61.8,507.6,11,0,61.8,507.6,11.0,0.0,42.5,90.9,11,11,507.6,507.6
|
||||
06-06 15:48:55,1,1,149.2,449.9,11,0,149.2,449.9,11.0,0.0,45.0,91.0,11,11,449.9,449.9
|
||||
06-06 15:49:00,1,1,0.0,377.3,8,0,0.0,377.3,8.0,0.0,32.5,86.6,8,8,377.3,377.3
|
||||
06-06 15:49:05,1,1,0.0,376.6,8,0,0.0,376.6,8.0,0.0,33.0,86.6,8,8,376.6,376.6
|
||||
06-06 15:49:10,1,1,13.2,697.2,18,0,13.2,697.2,18.0,0.0,80.9,89.1,18,18,697.2,697.2
|
||||
06-06 15:49:15,1,1,147.6,673.7,17,0,147.6,673.7,17.0,0.0,73.3,89.2,17,17,673.7,673.7
|
||||
06-06 15:49:21,1,1,0.0,316.3,7,0,0.0,316.3,7.0,0.0,36.8,89.1,7,7,316.3,316.3
|
||||
06-06 15:49:26,1,1,90.6,324.1,8,0,90.6,324.1,8.0,0.0,39.7,89.3,8,8,324.1,324.1
|
||||
06-06 15:49:31,1,1,0.0,356.7,8,0,0.0,356.7,8.0,0.0,40.2,89.3,8,8,356.7,356.7
|
||||
06-06 15:49:36,1,1,0.0,334.2,7,0,0.0,334.2,7.0,0.0,38.1,89.3,7,7,334.2,334.2
|
||||
06-06 15:49:41,1,1,85.8,327.3,8,0,85.8,327.3,8.0,0.0,41.2,89.5,8,8,327.3,327.3
|
||||
06-06 15:49:46,1,1,0.0,340.7,7,0,0.0,340.7,7.0,0.0,39.0,89.5,7,7,340.7,340.7
|
||||
06-06 15:49:51,1,1,33.1,328.5,8,0,33.1,328.5,8.0,0.0,42.2,89.7,8,8,328.5,328.5
|
||||
06-06 15:49:56,1,1,0.0,354.7,8,0,0.0,354.7,8.0,0.0,42.7,89.7,8,8,354.7,354.7
|
||||
06-06 15:50:01,1,1,0.0,312.5,7,0,0.0,312.5,7.0,0.0,40.3,89.7,7,7,312.5,312.5
|
||||
06-06 15:50:06,1,1,23.3,344.1,8,0,23.3,344.1,8.0,0.0,43.6,90.0,8,8,344.1,344.1
|
||||
06-06 15:50:11,1,1,0.0,357.0,8,0,0.0,357.0,8.0,0.0,44.1,90.0,8,8,357.0,357.0
|
||||
06-06 15:50:16,1,1,0.0,313.6,7,0,0.0,313.6,7.0,0.0,41.6,90.0,7,7,313.6,313.6
|
||||
06-06 15:50:21,1,1,0.0,312.0,7,0,0.0,312.0,7.0,0.0,42.1,90.0,7,7,312.0,312.0
|
||||
06-06 15:50:26,1,1,0.0,312.5,7,0,0.0,312.5,7.0,0.0,42.5,90.0,7,7,312.5,312.5
|
||||
06-06 15:50:31,1,1,65.2,340.5,8,0,65.2,340.5,8.0,0.0,43.1,90.0,8,8,340.5,340.5
|
||||
06-06 15:50:36,1,1,0.0,335.2,6,0,0.0,335.2,6.0,0.0,34.4,90.0,6,6,335.2,335.2
|
||||
06-06 15:50:41,1,1,43.4,292.9,7,0,43.4,292.9,7.0,0.0,35.0,89.9,7,7,292.9,292.9
|
||||
06-06 15:50:46,1,1,42.0,322.0,8,0,42.0,322.0,8.0,0.0,35.6,89.9,8,8,322.0,322.0
|
||||
06-06 15:50:51,1,1,62.1,326.0,8,0,62.1,326.0,8.0,0.0,36.2,89.9,8,8,326.0,326.0
|
||||
06-06 15:50:56,1,1,0.0,354.8,8,0,0.0,354.8,8.0,0.0,36.7,89.9,8,8,354.8,354.8
|
||||
06-06 15:51:01,1,1,9.2,410.3,10,0,9.2,410.3,10.0,0.0,30.2,88.8,10,10,410.3,410.3
|
||||
06-06 15:51:06,1,1,185.1,402.1,10,0,185.1,402.1,10.0,0.0,31.0,88.8,10,10,402.1,402.1
|
||||
06-06 15:51:11,1,1,0.0,425.1,9,0,0.0,425.1,9.0,0.0,31.4,88.8,9,9,425.1,425.1
|
||||
06-06 15:51:16,1,1,78.1,413.5,10,0,78.1,413.5,10.0,0.0,32.3,88.7,10,10,413.5,413.5
|
||||
06-06 15:51:21,1,1,0.0,423.8,9,0,0.0,423.8,9.0,0.0,30.9,88.7,9,9,423.8,423.8
|
||||
06-06 15:51:26,1,1,23.5,430.4,10,0,23.5,430.4,10.0,0.0,33.5,88.9,10,10,430.4,430.4
|
||||
06-06 15:51:31,1,1,0.0,423.5,9,0,0.0,423.5,9.0,0.0,33.6,88.9,9,9,423.5,423.5
|
||||
06-06 15:51:36,1,1,191.8,450.4,10,0,191.8,450.4,10.0,0.0,32.8,88.8,10,10,450.4,450.4
|
||||
06-06 15:51:41,1,1,0.0,435.6,10,0,0.0,435.6,10.0,0.0,33.4,88.8,10,10,435.6,435.6
|
||||
06-06 15:51:46,1,1,0.0,468.4,10,0,0.0,468.4,10.0,0.0,41.5,92.0,10,10,468.4,468.4
|
||||
06-06 15:51:51,1,1,67.8,462.3,10,0,67.8,462.3,10.0,0.0,33.4,91.9,10,10,462.3,462.3
|
||||
06-06 15:51:56,1,1,0.0,468.4,10,0,0.0,468.4,10.0,0.0,34.0,91.9,10,10,468.4,468.4
|
||||
06-06 15:52:01,1,1,0.0,466.6,10,0,0.0,466.6,10.0,0.0,34.7,91.9,10,10,466.6,466.6
|
||||
06-06 15:52:06,1,1,0.0,467.1,10,0,0.0,467.1,10.0,0.0,35.3,91.9,10,10,467.1,467.1
|
||||
06-06 15:52:11,1,1,63.8,434.5,10,0,63.8,434.5,10.0,0.0,35.7,91.9,10,10,434.5,434.5
|
||||
06-06 15:52:16,1,1,0.0,467.7,10,0,0.0,467.7,10.0,0.0,36.3,91.9,10,10,467.7,467.7
|
||||
06-06 15:52:21,1,1,0.0,453.9,9,0,0.0,453.9,9.0,0.0,36.7,91.9,9,9,453.9,453.9
|
||||
06-06 15:52:26,1,1,49.7,443.1,10,0,49.7,443.1,10.0,0.0,37.6,91.9,10,10,443.1,443.1
|
||||
06-06 15:52:31,1,1,61.4,432.7,9,0,61.4,432.7,9.0,0.0,37.2,91.8,9,9,432.7,432.7
|
||||
06-06 15:52:36,1,1,226.3,451.0,10,0,226.3,451.0,10.0,0.0,39.1,91.7,10,10,451.0,451.0
|
||||
06-06 15:52:41,1,1,0.0,465.5,10,0,0.0,465.5,10.0,0.0,39.7,91.7,10,10,465.5,465.5
|
||||
06-06 15:52:46,1,1,0.0,464.5,10,0,0.0,464.5,10.0,0.0,40.4,91.7,10,10,464.5,464.5
|
||||
06-06 15:52:51,1,1,0.0,441.3,9,0,0.0,441.3,9.0,0.0,40.4,91.7,9,9,441.3,441.3
|
||||
06-06 15:52:56,1,1,196.1,456.8,10,0,196.1,456.8,10.0,0.0,41.8,88.6,10,10,456.8,456.8
|
||||
06-06 15:53:01,1,1,42.5,401.9,9,0,42.5,401.9,9.0,0.0,37.8,89.0,9,9,401.9,401.9
|
||||
06-06 15:53:06,1,1,71.9,414.8,9,0,71.9,414.8,9.0,0.0,29.9,89.0,9,9,414.8,414.8
|
||||
06-06 15:53:11,1,1,65.1,381.6,9,0,65.1,381.6,9.0,0.0,31.4,89.1,9,9,381.6,381.6
|
||||
06-06 15:53:16,1,1,36.1,405.6,9,0,36.1,405.6,9.0,0.0,37.1,91.8,9,9,405.6,405.6
|
||||
06-06 15:53:22,1,1,0.0,380.7,8,0,0.0,380.7,8.0,0.0,34.9,91.8,8,8,380.7,380.7
|
||||
06-06 15:53:27,1,1,0.0,380.1,8,0,0.0,380.1,8.0,0.0,35.4,91.8,8,8,380.1,380.1
|
||||
06-06 15:53:32,1,1,73.4,358.8,8,0,73.4,358.8,8.0,0.0,36.0,92.0,8,8,358.8,358.8
|
||||
06-06 15:53:37,1,1,0.0,380.9,8,0,0.0,380.9,8.0,0.0,36.5,92.0,8,8,380.9,380.9
|
||||
06-06 15:53:42,1,1,0.0,380.3,8,0,0.0,380.3,8.0,0.0,37.1,92.0,8,8,380.3,380.3
|
||||
06-06 15:53:47,1,1,45.7,393.5,9,0,45.7,393.5,9.0,0.0,40.4,92.2,9,9,393.5,393.5
|
||||
06-06 15:53:52,1,1,0.0,426.0,9,0,0.0,426.0,9.0,0.0,41.0,92.2,9,9,426.0,426.0
|
||||
06-06 15:53:57,1,1,0.0,420.9,8,0,0.0,420.9,8.0,0.0,37.0,92.2,8,8,420.9,420.9
|
||||
06-06 15:54:02,1,1,113.9,410.7,9,0,113.9,410.7,9.0,0.0,42.3,92.4,9,9,410.7,410.7
|
||||
06-06 15:54:07,1,1,0.0,234.1,5,0,0.0,234.1,5.0,0.0,23.1,88.1,5,5,234.1,234.1
|
||||
06-06 15:54:12,1,1,8.6,258.2,6,0,8.6,258.2,6.0,0.0,23.6,88.1,6,6,258.2,258.2
|
||||
06-06 15:54:17,1,1,0.0,277.2,6,0,0.0,277.2,6.0,0.0,24.0,88.1,6,6,277.2,277.2
|
||||
06-06 15:54:22,1,1,0.0,276.3,6,0,0.0,276.3,6.0,0.0,24.4,88.1,6,6,276.3,276.3
|
||||
06-06 15:54:27,1,1,0.0,274.9,6,0,0.0,274.9,6.0,0.0,24.7,88.1,6,6,274.9,274.9
|
||||
06-06 15:54:32,1,1,0.0,423.9,8,0,0.0,423.9,8.0,0.0,33.2,90.1,8,8,423.9,423.9
|
||||
06-06 15:54:37,1,1,0.0,390.0,8,0,0.0,390.0,8.0,0.0,33.8,90.1,8,8,390.0,390.0
|
||||
06-06 15:54:42,1,1,0.0,392.9,8,0,0.0,392.9,8.0,0.0,34.3,90.1,8,8,392.9,392.9
|
||||
06-06 15:54:47,1,1,0.0,393.2,8,0,0.0,393.2,8.0,0.0,34.8,90.1,8,8,393.2,393.2
|
||||
06-06 15:54:52,1,1,0.0,392.6,8,0,0.0,392.6,8.0,0.0,35.4,90.1,8,8,392.6,392.6
|
||||
06-06 15:54:57,1,1,0.0,391.4,8,0,0.0,391.4,8.0,0.0,35.9,90.1,8,8,391.4,391.4
|
||||
06-06 15:55:02,1,1,0.0,391.7,8,0,0.0,391.7,8.0,0.0,36.5,90.1,8,8,391.7,391.7
|
||||
06-06 15:55:07,1,1,0.0,391.1,8,0,0.0,391.1,8.0,0.0,37.0,90.1,8,8,391.1,391.1
|
||||
06-06 15:55:12,1,1,89.0,410.5,9,0,89.0,410.5,9.0,0.0,38.3,90.1,9,9,410.5,410.5
|
||||
06-06 15:55:17,1,1,134.6,438.1,10,0,134.6,438.1,10.0,0.0,41.3,90.1,10,10,438.1,438.1
|
||||
06-06 15:55:22,1,1,11.2,461.9,10,0,11.2,461.9,10.0,0.0,41.3,90.2,10,10,461.9,461.9
|
||||
06-06 15:55:27,1,1,0.0,488.1,10,0,0.0,488.1,10.0,0.0,42.0,90.2,10,10,488.1,488.1
|
||||
06-06 15:55:32,1,1,8.4,494.6,11,0,8.4,494.6,11.0,0.0,42.8,90.2,11,11,494.6,494.6
|
||||
06-06 15:55:37,1,1,9.0,389.7,9,0,9.0,389.7,9.0,0.0,35.8,87.1,9,9,389.7,389.7
|
||||
06-06 15:55:42,1,1,0.0,434.0,9,0,0.0,434.0,9.0,0.0,36.4,87.1,9,9,434.0,434.0
|
||||
06-06 15:55:47,1,1,0.0,393.9,8,0,0.0,393.9,8.0,0.0,36.8,87.1,8,8,393.9,393.9
|
||||
06-06 15:55:52,1,1,10.8,412.2,9,0,10.8,412.2,9.0,0.0,37.4,87.1,9,9,412.2,412.2
|
||||
06-06 15:55:57,1,1,0.0,433.9,9,0,0.0,433.9,9.0,0.0,38.1,87.1,9,9,433.9,433.9
|
||||
06-06 15:56:02,1,1,0.0,434.1,9,0,0.0,434.1,9.0,0.0,38.7,87.1,9,9,434.1,434.1
|
||||
06-06 15:56:07,1,1,0.0,418.2,8,0,0.0,418.2,8.0,0.0,38.9,87.1,8,8,418.2,418.2
|
||||
06-06 15:56:12,1,1,133.1,424.6,9,0,133.1,424.6,9.0,0.0,40.0,87.1,9,9,424.6,424.6
|
||||
06-06 15:56:17,1,1,0.0,436.7,9,0,0.0,436.7,9.0,0.0,40.6,87.1,9,9,436.7,436.7
|
||||
06-06 15:56:23,1,1,263.3,214.0,7,0,263.3,214.0,7.0,0.0,26.0,90.8,7,7,214.0,214.0
|
||||
06-06 15:56:28,1,1,34.7,354.7,8,0,34.7,354.7,8.0,0.0,26.8,90.8,8,8,354.7,354.7
|
||||
06-06 15:56:33,1,1,50.9,351.3,8,0,50.9,351.3,8.0,0.0,25.9,90.7,8,8,351.3,351.3
|
||||
06-06 15:56:38,1,1,0.0,362.7,8,0,0.0,362.7,8.0,0.0,26.4,90.7,8,8,362.7,362.7
|
||||
06-06 15:56:43,1,1,141.5,290.3,7,0,141.5,290.3,7.0,0.0,36.6,91.2,7,7,290.3,290.3
|
||||
06-06 15:56:48,1,1,0.0,287.7,6,0,0.0,287.7,6.0,0.0,35.2,91.2,6,6,287.7,287.7
|
||||
06-06 15:56:53,1,1,0.0,268.5,6,0,0.0,268.5,6.0,0.0,35.5,91.2,6,6,268.5,268.5
|
||||
06-06 15:56:58,1,1,0.0,268.8,6,0,0.0,268.8,6.0,0.0,35.9,91.2,6,6,268.8,268.8
|
||||
06-06 15:57:03,1,1,0.0,225.1,5,0,0.0,225.1,5.0,0.0,27.5,91.2,5,5,225.1,225.1
|
||||
06-06 15:57:08,1,1,0.0,224.7,5,0,0.0,224.7,5.0,0.0,27.8,91.2,5,5,224.7,224.7
|
||||
06-06 15:57:13,1,1,21.6,250.4,6,0,21.6,250.4,6.0,0.0,27.2,90.7,6,6,250.4,250.4
|
||||
06-06 15:57:18,1,1,0.0,223.9,5,0,0.0,223.9,5.0,0.0,28.4,91.2,5,5,223.9,223.9
|
||||
06-06 15:57:23,1,1,0.0,224.4,5,0,0.0,224.4,5.0,0.0,28.7,91.2,5,5,224.4,224.4
|
||||
06-06 15:57:28,1,1,0.0,422.7,9,0,0.0,422.7,9.0,0.0,43.0,90.7,9,9,422.7,422.7
|
||||
06-06 15:57:33,1,1,0.0,396.8,9,0,0.0,396.8,9.0,0.0,43.6,90.7,9,9,396.8,396.8
|
||||
06-06 15:57:38,1,1,0.0,396.9,9,0,0.0,396.9,9.0,0.0,44.1,90.7,9,9,396.9,396.9
|
||||
06-06 15:57:43,1,1,8.2,418.2,9,0,8.2,418.2,9.0,0.0,43.3,90.7,9,9,418.2,418.2
|
||||
06-06 15:57:48,1,1,0.0,342.7,7,0,0.0,342.7,7.0,0.0,38.2,90.7,7,7,342.7,342.7
|
||||
06-06 15:57:53,1,1,0.0,310.1,7,0,0.0,310.1,7.0,0.0,38.6,90.7,7,7,310.1,310.1
|
||||
06-06 15:57:58,1,1,415.8,385.6,9,0,415.8,385.6,9.0,0.0,41.9,90.5,9,9,385.6,385.6
|
||||
06-06 15:58:03,1,1,495.9,425.7,10,0,495.9,425.7,10.0,0.0,48.5,90.5,10,10,425.7,425.7
|
||||
06-06 15:58:08,1,1,54.9,460.0,11,0,54.9,460.0,11.0,0.0,49.4,90.5,11,11,460.0,460.0
|
||||
06-06 15:58:13,1,1,0.0,475.4,10,0,0.0,475.4,10.0,0.0,49.7,90.5,10,10,475.4,475.4
|
||||
06-06 15:58:18,1,1,33.5,445.9,11,0,33.5,445.9,11.0,0.0,50.6,90.5,11,11,445.9,445.9
|
||||
06-06 15:58:23,1,1,0.0,481.4,11,0,0.0,481.4,11.0,0.0,51.3,90.5,11,11,481.4,481.4
|
||||
06-06 15:58:28,1,1,0.0,676.0,15,0,0.0,676.0,15.0,0.0,62.6,90.0,15,15,676.0,676.0
|
||||
06-06 15:58:33,1,1,0.0,460.6,10,0,0.0,460.6,10.0,0.0,50.0,90.5,10,10,460.6,460.6
|
||||
06-06 15:58:38,1,1,0.0,438.9,11,0,0.0,438.9,11.0,0.0,53.7,90.4,11,11,438.9,438.9
|
||||
06-06 15:58:43,1,1,339.3,477.5,11,0,339.3,477.5,11.0,0.0,54.3,90.4,11,11,477.5,477.5
|
||||
06-06 15:58:48,1,1,0.0,482.1,11,0,0.0,482.1,11.0,0.0,55.0,90.4,11,11,482.1,482.1
|
||||
06-06 15:58:53,1,1,0.0,617.1,13,0,0.0,617.1,13.0,0.0,59.0,89.5,13,13,617.1,617.1
|
||||
06-06 15:58:58,1,1,66.4,627.0,14,0,66.4,627.0,14.0,0.0,60.0,89.4,14,14,627.0,627.0
|
||||
06-06 15:59:03,1,1,8.6,625.4,14,0,8.6,625.4,14.0,0.0,60.8,89.4,14,14,625.4,625.4
|
||||
06-06 15:59:08,1,1,34.6,631.7,14,0,34.6,631.7,14.0,0.0,61.7,89.4,14,14,631.7,631.7
|
||||
06-06 15:59:13,1,1,0.0,633.3,14,0,0.0,633.3,14.0,0.0,62.6,89.4,14,14,633.3,633.3
|
||||
06-06 15:59:19,1,1,7.8,658.9,15,0,7.8,658.9,15.0,0.0,63.6,89.5,15,15,658.9,658.9
|
||||
06-06 15:59:24,1,1,0.0,678.1,15,0,0.0,678.1,15.0,0.0,64.5,89.5,15,15,678.1,678.1
|
||||
06-06 15:59:29,1,1,34.1,712.4,16,0,34.1,712.4,16.0,0.0,65.7,89.5,16,16,712.4,712.4
|
||||
06-06 15:59:34,1,1,0.0,661.9,18,0,0.0,661.9,18.0,0.0,73.7,88.0,18,18,661.9,661.9
|
||||
06-06 15:59:39,1,1,0.0,630.3,18,0,0.0,630.3,18.0,0.0,74.6,88.0,18,18,630.3,630.3
|
||||
06-06 15:59:44,1,1,0.0,629.8,18,0,0.0,629.8,18.0,0.0,75.5,88.0,18,18,629.8,629.8
|
||||
06-06 15:59:49,1,1,64.8,642.0,19,0,64.8,642.0,19.0,0.0,76.5,87.9,19,19,642.0,642.0
|
||||
06-06 15:59:54,1,1,59.0,665.3,20,0,59.0,665.3,20.0,0.0,77.5,87.9,20,20,665.3,665.3
|
||||
06-06 15:59:59,1,1,125.9,705.9,20,0,125.9,705.9,20.0,0.0,79.3,87.9,20,20,705.9,705.9
|
||||
06-06 16:00:04,1,1,0.0,666.8,19,0,0.0,666.8,19.0,0.0,80.1,87.9,19,19,666.8,666.8
|
||||
06-06 16:00:09,1,1,0.0,632.0,18,0,0.0,632.0,18.0,0.0,79.8,87.9,18,18,632.0,632.0
|
||||
06-06 16:00:14,1,1,0.0,598.6,17,0,0.0,598.6,17.0,0.0,79.9,87.9,17,17,598.6,598.6
|
||||
06-06 16:00:19,1,1,8.8,627.6,18,0,8.8,627.6,18.0,0.0,80.8,87.9,18,18,627.6,627.6
|
||||
06-06 16:00:24,1,1,34.3,642.3,19,0,34.3,642.3,19.0,0.0,81.9,87.9,19,19,642.3,642.3
|
||||
06-06 16:00:29,1,1,291.1,691.8,21,0,291.1,691.8,21.0,0.0,78.8,88.1,21,21,691.8,691.8
|
||||
06-06 16:00:34,1,1,0.0,473.0,10,0,0.0,473.0,10.0,0.0,58.5,90.4,10,10,473.0,473.0
|
||||
06-06 16:00:39,1,1,0.0,454.0,10,0,0.0,454.0,10.0,0.0,59.1,90.4,10,10,454.0,454.0
|
||||
06-06 16:00:44,1,1,224.9,530.6,14,0,224.9,530.6,14.0,0.0,68.1,90.6,14,14,530.6,530.6
|
||||
06-06 16:00:49,1,1,0.0,594.4,13,0,0.0,594.4,13.0,0.0,67.3,90.6,13,13,594.4,594.4
|
||||
06-06 16:00:54,1,1,66.4,568.6,12,0,66.4,568.6,12.0,0.0,62.2,90.6,12,12,568.6,568.6
|
||||
06-06 16:00:59,1,1,0.0,526.8,11,0,0.0,526.8,11.0,0.0,62.8,90.6,11,11,526.8,526.8
|
||||
06-06 16:01:04,1,1,0.0,494.6,11,0,0.0,494.6,11.0,0.0,63.5,90.6,11,11,494.6,494.6
|
||||
06-06 16:01:09,1,1,0.0,469.2,10,0,0.0,469.2,10.0,0.0,55.2,90.6,10,10,469.2,469.2
|
||||
06-06 16:01:14,1,1,70.1,514.5,12,0,70.1,514.5,12.0,0.0,58.8,90.6,12,12,514.5,514.5
|
||||
06-06 16:01:19,1,1,110.3,583.4,13,0,110.3,583.4,13.0,0.0,58.7,90.7,13,13,583.4,583.4
|
||||
06-06 16:01:24,1,1,0.0,558.5,12,0,0.0,558.5,12.0,0.0,57.7,90.7,12,12,558.5,558.5
|
||||
06-06 16:01:29,1,1,35.7,584.0,13,0,35.7,584.0,13.0,0.0,58.7,90.7,13,13,584.0,584.0
|
||||
06-06 16:01:34,1,1,78.0,599.7,14,0,78.0,599.7,14.0,0.0,62.7,90.7,14,14,599.7,599.7
|
||||
06-06 16:01:39,1,1,0.0,624.1,14,0,0.0,624.1,14.0,0.0,63.5,90.7,14,14,624.1,624.1
|
||||
06-06 16:01:44,1,1,0.0,575.2,12,0,0.0,575.2,12.0,0.0,52.2,90.7,12,12,575.2,575.2
|
||||
06-06 16:01:49,1,1,0.0,537.7,12,0,0.0,537.7,12.0,0.0,53.0,90.7,12,12,537.7,537.7
|
||||
06-06 16:01:55,1,1,0.0,466.1,9,0,0.0,466.1,9.0,0.0,39.9,91.1,9,9,466.1,466.1
|
||||
06-06 16:02:00,1,1,214.7,494.4,11,0,214.7,494.4,11.0,0.0,42.5,91.0,11,11,494.4,494.4
|
||||
06-06 16:02:05,1,1,0.0,507.8,9,0,0.0,507.8,9.0,0.0,41.2,91.0,9,9,507.8,507.8
|
||||
06-06 16:02:10,1,1,0.0,449.8,9,0,0.0,449.8,9.0,0.0,41.8,91.0,9,9,449.8,449.8
|
||||
06-06 16:02:15,1,1,102.4,446.8,10,0,102.4,446.8,10.0,0.0,41.7,89.5,10,10,446.8,446.8
|
||||
06-06 16:02:20,1,1,0.0,460.0,10,0,0.0,460.0,10.0,0.0,42.3,89.5,10,10,460.0,460.0
|
||||
06-06 16:02:25,1,1,185.9,427.0,10,0,185.9,427.0,10.0,0.0,43.8,89.4,10,10,427.0,427.0
|
||||
06-06 16:02:30,1,1,80.2,392.2,10,0,80.2,392.2,10.0,0.0,37.0,91.1,10,10,392.2,392.2
|
||||
06-06 16:02:35,1,1,22.1,574.5,13,0,22.1,574.5,13.0,0.0,47.4,89.5,13,13,574.5,574.5
|
||||
06-06 16:02:40,1,1,0.0,559.3,13,0,0.0,559.3,13.0,0.0,72.8,90.5,13,13,559.3,559.3
|
||||
06-06 16:02:45,1,1,0.0,559.4,13,0,0.0,559.4,13.0,0.0,73.6,90.5,13,13,559.4,559.4
|
||||
06-06 16:02:50,1,1,82.4,599.9,14,0,82.4,599.9,14.0,0.0,74.6,90.5,14,14,599.9,599.9
|
||||
06-06 16:02:55,1,1,74.3,574.1,13,0,74.3,574.1,13.0,0.0,70.6,90.5,13,13,574.1,574.1
|
||||
06-06 16:03:00,1,1,81.4,586.1,14,0,81.4,586.1,14.0,0.0,71.5,90.4,14,14,586.1,586.1
|
||||
06-06 16:03:05,1,1,0.0,598.7,14,0,0.0,598.7,14.0,0.0,72.4,90.4,14,14,598.7,598.7
|
||||
06-06 16:03:10,1,1,0.0,598.7,14,0,0.0,598.7,14.0,0.0,73.2,90.4,14,14,598.7,598.7
|
||||
06-06 16:03:15,1,1,0.0,576.9,13,0,0.0,576.9,13.0,0.0,73.7,90.4,13,13,576.9,576.9
|
||||
06-06 16:03:20,1,1,0.0,556.5,13,0,0.0,556.5,13.0,0.0,74.5,90.4,13,13,556.5,556.5
|
||||
06-06 16:03:25,1,1,0.0,555.6,13,0,0.0,555.6,13.0,0.0,75.3,90.4,13,13,555.6,555.6
|
||||
06-06 16:03:30,1,1,0.4,590.5,14,0,0.4,590.5,14.0,0.0,76.1,90.4,14,14,590.5,590.5
|
||||
06-06 16:03:35,1,1,0.0,601.7,14,0,0.0,601.7,14.0,0.0,76.9,90.4,14,14,601.7,601.7
|
||||
06-06 16:03:40,1,1,141.3,665.3,16,0,141.3,665.3,16.0,0.0,78.4,90.4,16,16,665.3,665.3
|
||||
06-06 16:03:45,1,1,0.0,671.8,15,0,0.0,671.8,15.0,0.0,79.0,90.4,15,15,671.8,671.8
|
||||
06-06 16:03:50,1,1,0.0,645.0,15,0,0.0,645.0,15.0,0.0,79.9,90.4,15,15,645.0,645.0
|
||||
06-06 16:03:55,1,1,0.0,626.6,14,0,0.0,626.6,14.0,0.0,71.9,90.4,14,14,626.6,626.6
|
||||
06-06 16:04:00,1,1,0.0,601.9,14,0,0.0,601.9,14.0,0.0,72.8,90.4,14,14,601.9,601.9
|
||||
06-06 16:04:05,1,1,0.0,614.5,13,0,0.0,614.5,13.0,0.0,61.3,92.3,13,13,614.5,614.5
|
||||
06-06 16:04:10,1,1,0.0,614.3,13,0,0.0,614.3,13.0,0.0,62.2,92.3,13,13,614.3,614.3
|
||||
06-06 16:04:15,1,1,60.4,626.2,14,0,60.4,626.2,14.0,0.0,63.1,92.3,14,14,626.2,626.2
|
||||
06-06 16:04:20,1,1,0.0,663.0,14,0,0.0,663.0,14.0,0.0,64.1,92.3,14,14,663.0,663.0
|
||||
06-06 16:04:25,1,1,1.0,702.0,15,0,1.0,702.0,15.0,0.0,65.0,92.3,15,15,702.0,702.0
|
||||
06-06 16:04:31,1,1,0.0,711.4,15,0,0.0,711.4,15.0,0.0,66.0,92.3,15,15,711.4,711.4
|
||||
06-06 16:04:36,1,1,104.7,731.5,16,0,104.7,731.5,16.0,0.0,67.6,92.3,16,16,731.5,731.5
|
||||
06-06 16:04:41,1,1,56.3,710.9,17,0,56.3,710.9,17.0,0.0,69.1,92.2,17,17,710.9,710.9
|
||||
06-06 16:04:46,1,1,0.0,704.8,16,0,0.0,704.8,16.0,0.0,61.2,92.2,16,16,704.8,704.8
|
||||
06-06 16:04:51,1,1,125.4,728.7,17,0,125.4,728.7,17.0,0.0,61.8,92.2,17,17,728.7,728.7
|
||||
06-06 16:04:55,1,1,3.2,570.5,13,0,3.2,570.5,13.0,0.0,64.6,90.4,13,13,570.5,570.5
|
||||
06-06 16:05:00,1,1,0.0,558.4,13,0,0.0,558.4,13.0,0.0,65.4,90.4,13,13,558.4,558.4
|
||||
06-06 16:05:06,1,1,0.0,709.1,15,0,0.0,709.1,15.0,0.0,55.8,92.2,15,15,709.1,709.1
|
||||
06-06 16:05:11,1,1,0.0,685.0,14,0,0.0,685.0,14.0,0.0,48.1,92.2,14,14,685.0,685.0
|
||||
06-06 16:05:15,1,1,76.7,476.8,12,0,76.7,476.8,12.0,0.0,50.0,90.3,12,12,476.8,476.8
|
||||
06-06 16:05:21,1,1,0.0,660.8,14,0,0.0,660.8,14.0,0.0,49.9,92.2,14,14,660.8,660.8
|
||||
06-06 16:05:26,1,1,129.3,704.9,19,0,129.3,704.9,19.0,0.0,90.2,90.3,19,19,704.9,704.9
|
||||
06-06 16:05:31,1,1,0.0,712.4,18,0,0.0,712.4,18.0,0.0,82.5,90.3,18,18,712.4,712.4
|
||||
06-06 16:05:36,1,1,0.0,648.9,17,0,0.0,648.9,17.0,0.0,82.7,90.3,17,17,648.9,648.9
|
||||
06-06 16:05:41,1,1,145.0,669.7,17,0,145.0,669.7,17.0,0.0,82.3,90.2,17,17,669.7,669.7
|
||||
06-06 16:05:46,1,1,0.0,683.9,15,0,0.0,683.9,15.0,0.0,82.1,90.2,15,15,683.9,683.9
|
||||
06-06 16:05:51,1,1,0.0,666.9,15,0,0.0,666.9,15.0,0.0,83.1,90.2,15,15,666.9,666.9
|
||||
06-06 16:05:56,1,1,0.0,667.2,15,0,0.0,667.2,15.0,0.0,84.0,90.2,15,15,667.2,667.2
|
||||
06-06 16:06:01,1,1,0.0,667.7,15,0,0.0,667.7,15.0,0.0,84.9,90.2,15,15,667.7,667.7
|
||||
06-06 16:06:06,1,1,0.6,692.3,16,0,0.6,692.3,16.0,0.0,85.9,90.3,16,16,692.3,692.3
|
||||
06-06 16:06:11,1,1,149.6,660.6,17,0,149.6,660.6,17.0,0.0,88.6,90.3,17,17,660.6,660.6
|
||||
06-06 16:06:16,1,1,24.4,526.9,14,0,24.4,526.9,14.0,0.0,61.9,90.2,14,14,526.9,526.9
|
||||
06-06 16:06:21,1,1,0.0,728.3,19,0,0.0,728.3,19.0,0.0,91.9,90.2,19,19,728.3,728.3
|
||||
06-06 16:06:26,1,1,0.0,535.0,13,0,0.0,535.0,13.0,0.0,63.3,90.2,13,13,535.0,535.0
|
||||
06-06 16:06:31,1,1,0.0,683.8,18,0,0.0,683.8,18.0,0.0,93.6,90.2,18,18,683.8,683.8
|
||||
06-06 16:06:36,1,1,144.7,533.6,12,0,144.7,533.6,12.0,0.0,44.1,90.2,12,12,533.6,533.6
|
||||
06-06 16:06:41,1,1,0.0,557.7,12,0,0.0,557.7,12.0,0.0,44.9,90.2,12,12,557.7,557.7
|
||||
06-06 16:06:46,1,1,0.0,558.6,12,0,0.0,558.6,12.0,0.0,45.7,90.2,12,12,558.6,558.6
|
||||
06-06 16:06:51,1,1,0.0,558.2,12,0,0.0,558.2,12.0,0.0,46.4,90.2,12,12,558.2,558.2
|
||||
06-06 16:06:56,1,1,0.0,558.0,12,0,0.0,558.0,12.0,0.0,47.2,90.2,12,12,558.0,558.0
|
||||
06-06 16:07:01,1,1,0.0,558.0,12,0,0.0,558.0,12.0,0.0,48.0,90.2,12,12,558.0,558.0
|
||||
06-06 16:07:06,1,1,55.5,555.0,12,0,55.5,555.0,12.0,0.0,48.9,90.2,12,12,555.0,555.0
|
||||
06-06 16:07:11,1,1,83.3,575.4,13,0,83.3,575.4,13.0,0.0,50.5,90.2,13,13,575.4,575.4
|
||||
06-06 16:07:16,1,1,0.0,606.5,13,0,0.0,606.5,13.0,0.0,51.4,90.2,13,13,606.5,606.5
|
||||
06-06 16:07:21,1,1,1.4,609.4,14,0,1.4,609.4,14.0,0.0,52.2,90.2,14,14,609.4,609.4
|
||||
06-06 16:07:26,1,1,0.0,494.3,12,0,0.0,494.3,12.0,0.0,62.6,90.2,12,12,494.3,494.3
|
||||
06-06 16:07:31,1,1,158.0,567.0,14,0,158.0,567.0,14.0,0.0,65.2,90.2,14,14,567.0,567.0
|
||||
06-06 16:07:36,1,1,0.0,588.8,12,0,0.0,588.8,12.0,0.0,45.9,90.2,12,12,588.8,588.8
|
||||
06-06 16:07:41,1,1,87.2,573.9,12,0,87.2,573.9,12.0,0.0,39.1,90.2,12,12,573.9,573.9
|
||||
06-06 16:07:46,1,1,18.8,578.0,12,0,18.8,578.0,12.0,0.0,39.1,90.2,12,12,578.0,578.0
|
||||
06-06 16:07:51,1,1,0.0,560.0,12,0,0.0,560.0,12.0,0.0,39.9,90.2,12,12,560.0,560.0
|
||||
06-06 16:07:56,1,1,0.0,559.9,12,0,0.0,559.9,12.0,0.0,40.7,90.2,12,12,559.9,559.9
|
||||
06-06 16:08:01,1,1,11.0,596.5,13,0,11.0,596.5,13.0,0.0,41.7,90.3,13,13,596.5,596.5
|
||||
06-06 16:08:06,1,1,9.6,631.2,13,0,9.6,631.2,13.0,0.0,41.7,90.3,13,13,631.2,631.2
|
||||
06-06 16:08:11,1,1,0.0,570.3,12,0,0.0,570.3,12.0,0.0,42.3,90.3,12,12,570.3,570.3
|
||||
06-06 16:08:16,1,1,0.0,524.4,11,0,0.0,524.4,11.0,0.0,41.3,90.3,11,11,524.4,524.4
|
||||
06-06 16:08:21,1,1,57.4,530.2,12,0,57.4,530.2,12.0,0.0,42.2,90.2,12,12,530.2,530.2
|
||||
06-06 16:08:26,1,1,0.0,551.1,11,0,0.0,551.1,11.0,0.0,42.4,90.2,11,11,551.1,551.1
|
||||
06-06 16:08:31,1,1,0.0,480.9,10,0,0.0,480.9,10.0,0.0,42.9,90.2,10,10,480.9,480.9
|
||||
06-06 16:08:36,1,1,363.8,511.0,12,0,363.8,511.0,12.0,0.0,45.3,90.1,12,12,511.0,511.0
|
||||
06-06 16:08:41,1,1,102.0,589.2,13,0,102.0,589.2,13.0,0.0,48.0,90.2,13,13,589.2,589.2
|
||||
06-06 16:08:46,1,1,0.0,454.1,11,0,0.0,454.1,11.0,0.0,50.0,90.3,11,11,454.1,454.1
|
||||
06-06 16:08:51,1,1,100.7,488.9,12,0,100.7,488.9,12.0,0.0,51.3,90.3,12,12,488.9,488.9
|
||||
06-06 16:08:56,1,1,0.0,494.3,12,0,0.0,494.3,12.0,0.0,52.0,90.3,12,12,494.3,494.3
|
||||
06-06 16:09:01,1,1,98.6,657.6,15,0,98.6,657.6,15.0,0.0,51.9,90.1,15,15,657.6,657.6
|
||||
06-06 16:09:06,1,1,39.8,703.9,15,0,39.8,703.9,15.0,0.0,53.8,90.1,15,15,703.9,703.9
|
||||
06-06 16:09:11,1,1,0.0,691.1,14,0,0.0,691.1,14.0,0.0,54.3,90.1,14,14,691.1,691.1
|
||||
06-06 16:09:16,1,1,0.0,651.7,14,0,0.0,651.7,14.0,0.0,55.2,90.1,14,14,651.7,651.7
|
||||
06-06 16:09:21,1,1,0.0,653.1,14,0,0.0,653.1,14.0,0.0,56.1,90.1,14,14,653.1,653.1
|
||||
06-06 16:09:26,1,1,0.0,566.6,11,0,0.0,566.6,11.0,0.0,52.2,90.1,11,11,566.6,566.6
|
||||
06-06 16:09:31,1,1,120.6,524.6,12,0,120.6,524.6,12.0,0.0,53.6,90.1,12,12,524.6,524.6
|
||||
06-06 16:09:36,1,1,0.0,560.2,12,0,0.0,560.2,12.0,0.0,54.4,90.1,12,12,560.2,560.2
|
||||
06-06 16:09:42,1,1,0.0,680.7,15,0,0.0,680.7,15.0,0.0,63.1,91.3,15,15,680.7,680.7
|
||||
06-06 16:09:47,1,1,0.0,608.9,13,0,0.0,608.9,13.0,0.0,60.0,91.3,13,13,608.9,608.9
|
||||
06-06 16:09:52,1,1,8.2,612.9,14,0,8.2,612.9,14.0,0.0,60.9,91.3,14,14,612.9,612.9
|
||||
06-06 16:09:57,1,1,0.0,616.4,14,0,0.0,616.4,14.0,0.0,61.8,91.3,14,14,616.4,616.4
|
||||
06-06 16:10:02,1,1,0.0,597.5,13,0,0.0,597.5,13.0,0.0,53.8,91.3,13,13,597.5,597.5
|
||||
06-06 16:10:07,1,1,0.0,537.7,11,0,0.0,537.7,11.0,0.0,56.0,91.0,11,11,537.7,537.7
|
||||
06-06 16:10:12,1,1,0.0,539.5,11,0,0.0,539.5,11.0,0.0,56.8,91.0,11,11,539.5,539.5
|
||||
06-06 16:10:17,1,1,2.2,541.6,12,0,2.2,541.6,12.0,0.0,57.5,91.0,12,12,541.6,541.6
|
||||
06-06 16:10:22,1,1,9.6,600.8,13,0,9.6,600.8,13.0,0.0,58.5,91.0,13,13,600.8,600.8
|
||||
06-06 16:10:27,1,1,0.0,635.4,13,0,0.0,635.4,13.0,0.0,59.3,91.0,13,13,635.4,635.4
|
||||
06-06 16:10:32,1,1,44.9,637.0,12,0,44.9,637.0,12.0,0.0,45.0,91.1,12,12,637.0,637.0
|
||||
06-06 16:10:37,1,1,0.0,492.3,12,0,0.0,492.3,12.0,0.0,62.7,90.3,12,12,492.3,492.3
|
||||
06-06 16:10:42,1,1,132.2,565.9,11,0,132.2,565.9,11.0,0.0,44.6,91.0,11,11,565.9,565.9
|
||||
06-06 16:10:47,1,1,0.0,496.3,10,0,0.0,496.3,10.0,0.0,36.3,91.0,10,10,496.3,496.3
|
||||
06-06 16:10:52,1,1,0.0,492.1,10,0,0.0,492.1,10.0,0.0,37.0,91.0,10,10,492.1,492.1
|
||||
06-06 16:10:57,1,1,0.0,445.8,9,0,0.0,445.8,9.0,0.0,36.5,91.0,9,9,445.8,445.8
|
||||
06-06 16:11:02,1,1,0.0,443.6,9,0,0.0,443.6,9.0,0.0,37.2,91.0,9,9,443.6,443.6
|
||||
06-06 16:11:07,1,1,0.0,454.9,11,0,0.0,454.9,11.0,0.0,58.5,90.3,11,11,454.9,454.9
|
||||
06-06 16:11:12,1,1,0.0,443.1,9,0,0.0,443.1,9.0,0.0,38.4,91.0,9,9,443.1,443.1
|
||||
06-06 16:11:17,1,1,0.0,370.9,8,0,0.0,370.9,8.0,0.0,40.0,89.9,8,8,370.9,370.9
|
||||
06-06 16:11:22,1,1,0.0,441.7,9,0,0.0,441.7,9.0,0.0,39.6,91.0,9,9,441.7,441.7
|
||||
06-06 16:11:27,1,1,33.9,425.0,10,0,33.9,425.0,10.0,0.0,42.3,89.8,10,10,425.0,425.0
|
||||
06-06 16:11:32,1,1,0.0,461.2,10,0,0.0,461.2,10.0,0.0,42.9,89.8,10,10,461.2,461.2
|
||||
06-06 16:11:37,1,1,0.0,460.9,10,0,0.0,460.9,10.0,0.0,43.6,89.8,10,10,460.9,460.9
|
||||
06-06 16:11:42,1,1,0.0,451.0,9,0,0.0,451.0,9.0,0.0,42.9,89.8,9,9,451.0,451.0
|
||||
06-06 16:11:47,1,1,0.0,414.5,9,0,0.0,414.5,9.0,0.0,43.5,89.8,9,9,414.5,414.5
|
||||
06-06 16:11:52,1,1,27.5,417.9,10,0,27.5,417.9,10.0,0.0,44.3,89.8,10,10,417.9,417.9
|
||||
06-06 16:11:57,1,1,8.0,473.3,10,0,8.0,473.3,10.0,0.0,44.5,89.9,10,10,473.3,473.3
|
||||
06-06 16:12:02,1,1,0.0,460.8,10,0,0.0,460.8,10.0,0.0,45.2,89.9,10,10,460.8,460.8
|
||||
06-06 16:12:07,1,1,0.0,460.4,10,0,0.0,460.4,10.0,0.0,45.8,89.9,10,10,460.4,460.4
|
||||
06-06 16:12:12,1,1,0.0,481.7,11,0,0.0,481.7,11.0,0.0,57.7,90.3,11,11,481.7,481.7
|
||||
06-06 16:12:17,1,1,0.0,504.9,11,0,0.0,504.9,11.0,0.0,48.6,89.9,11,11,504.9,504.9
|
||||
06-06 16:12:22,1,1,16.8,513.7,12,0,16.8,513.7,12.0,0.0,50.9,90.0,12,12,513.7,513.7
|
||||
06-06 16:12:27,1,1,0.0,546.5,12,0,0.0,546.5,12.0,0.0,51.7,90.0,12,12,546.5,546.5
|
||||
06-06 16:12:32,1,1,103.2,478.9,11,0,103.2,478.9,11.0,0.0,53.9,91.7,11,11,478.9,478.9
|
||||
06-06 16:12:37,1,1,0.0,470.5,10,0,0.0,470.5,10.0,0.0,49.6,89.9,10,10,470.5,470.5
|
||||
06-06 16:12:42,1,1,268.8,496.6,12,0,268.8,496.6,12.0,0.0,33.4,91.7,12,12,496.6,496.6
|
||||
06-06 16:12:47,1,1,0.0,518.0,11,0,0.0,518.0,11.0,0.0,33.3,91.7,11,11,518.0,518.0
|
||||
06-06 16:12:52,1,1,0.0,496.9,11,0,0.0,496.9,11.0,0.0,34.0,91.7,11,11,496.9,496.9
|
||||
06-06 16:12:57,1,1,0.0,411.4,9,0,0.0,411.4,9.0,0.0,52.0,89.4,9,9,411.4,411.4
|
||||
06-06 16:13:02,1,1,200.9,417.1,10,0,200.9,417.1,10.0,0.0,55.9,89.4,10,10,417.1,417.1
|
||||
06-06 16:13:07,1,1,68.3,452.7,10,0,68.3,452.7,10.0,0.0,53.2,89.4,10,10,452.7,452.7
|
||||
06-06 16:13:12,1,1,0.0,405.8,9,0,0.0,405.8,9.0,0.0,35.6,91.7,9,9,405.8,405.8
|
||||
06-06 16:13:17,1,1,95.6,457.4,10,0,95.6,457.4,10.0,0.0,55.1,89.4,10,10,457.4,457.4
|
||||
06-06 16:13:22,1,1,10.8,461.9,11,0,10.8,461.9,11.0,0.0,55.7,89.4,11,11,461.9,461.9
|
||||
06-06 16:13:27,1,1,0.0,468.0,10,0,0.0,468.0,10.0,0.0,55.5,89.4,10,10,468.0,468.0
|
||||
06-06 16:13:32,1,1,0.0,457.0,10,0,0.0,457.0,10.0,0.0,56.2,89.4,10,10,457.0,457.0
|
||||
06-06 16:13:37,1,1,0.6,463.5,10,0,0.6,463.5,10.0,0.0,47.9,89.4,10,10,463.5,463.5
|
||||
06-06 16:13:42,1,1,125.3,507.5,12,0,125.3,507.5,12.0,0.0,52.3,89.5,12,12,507.5,507.5
|
||||
06-06 16:13:47,1,1,2.0,558.1,12,0,2.0,558.1,12.0,0.0,49.5,89.5,12,12,558.1,558.1
|
||||
06-06 16:13:52,1,1,1.6,545.0,12,0,1.6,545.0,12.0,0.0,42.1,89.5,12,12,545.0,545.0
|
||||
06-06 16:13:57,1,1,0.0,536.2,12,0,0.0,536.2,12.0,0.0,42.8,89.5,12,12,536.2,536.2
|
||||
06-06 16:14:03,1,1,0.0,596.9,11,0,0.0,596.9,11.0,0.0,42.0,89.0,11,11,596.9,596.9
|
||||
06-06 16:14:08,1,1,0.0,516.6,10,0,0.0,516.6,10.0,0.0,40.9,89.0,10,10,516.6,516.6
|
||||
06-06 16:14:13,1,1,0.0,503.1,10,0,0.0,503.1,10.0,0.0,41.6,89.0,10,10,503.1,503.1
|
||||
06-06 16:14:18,1,1,0.0,502.3,10,0,0.0,502.3,10.0,0.0,42.3,89.0,10,10,502.3,502.3
|
||||
06-06 16:14:23,1,1,0.0,500.8,10,0,0.0,500.8,10.0,0.0,43.0,89.0,10,10,500.8,500.8
|
||||
06-06 16:14:28,1,1,0.0,500.3,10,0,0.0,500.3,10.0,0.0,43.7,89.0,10,10,500.3,500.3
|
||||
06-06 16:14:30,1,1,0.0,365.9,9,0,0.0,365.9,9.0,0.0,42.8,90.3,9,9,365.9,365.9
|
||||
|
519
training_logs/20260608_210110_vllm_metrics_job_632523.csv
Normal file
519
training_logs/20260608_210110_vllm_metrics_job_632523.csv
Normal file
@@ -0,0 +1,519 @@
|
||||
datetime_str,n_engines_reporting,unique_ips,total_prompt_throughput_tokens_per_sec,total_generation_throughput_tokens_per_sec,total_running_requests,total_waiting_requests,avg_prompt_throughput_per_engine,avg_generation_throughput_per_engine,avg_running_requests_per_engine,avg_waiting_requests_per_engine,avg_gpu_kv_cache_usage_pct,avg_prefix_cache_hit_rate_pct,min_running_requests,max_running_requests,min_generation_throughput,max_generation_throughput
|
||||
06-06 16:22:45,1,1,14.0,0.0,1,0,14.0,0.0,1.0,0.0,0.3,0.0,1,1,0.0,0.0
|
||||
06-06 16:22:50,1,1,0.0,49.8,1,0,0.0,49.8,1.0,0.0,0.3,0.0,1,1,49.8,49.8
|
||||
06-06 16:22:55,1,1,10.5,0.0,1,0,10.5,0.0,1.0,0.0,0.3,0.0,1,1,0.0,0.0
|
||||
06-06 16:23:00,1,1,0.0,48.4,1,0,0.0,48.4,1.0,0.0,0.3,0.0,1,1,48.4,48.4
|
||||
06-06 16:23:05,1,1,130.9,129.7,3,0,130.9,129.7,3.0,0.0,1.0,57.4,3,3,129.7,129.7
|
||||
06-06 16:23:10,1,1,33.8,87.6,1,0,33.8,87.6,1.0,0.0,0.5,67.1,1,1,87.6,87.6
|
||||
06-06 16:23:15,1,1,113.3,122.5,3,0,113.3,122.5,3.0,0.0,1.5,78.7,3,3,122.5,122.5
|
||||
06-06 16:23:20,1,1,36.2,117.7,2,0,36.2,117.7,2.0,0.0,1.2,81.3,2,2,117.7,117.7
|
||||
06-06 16:23:25,1,1,0.0,98.5,2,0,0.0,98.5,2.0,0.0,1.4,81.3,2,2,98.5,98.5
|
||||
06-06 16:23:30,1,1,0.0,91.5,1,0,0.0,91.5,1.0,0.0,1.0,81.3,1,1,91.5,91.5
|
||||
06-06 16:23:35,1,1,138.2,73.7,2,0,138.2,73.7,2.0,0.0,1.8,80.9,2,2,73.7,73.7
|
||||
06-06 16:23:40,1,1,0.0,96.4,1,0,0.0,96.4,1.0,0.0,1.0,80.9,1,1,96.4,96.4
|
||||
06-06 16:23:45,1,1,50.2,84.0,2,0,50.2,84.0,2.0,0.0,2.1,83.2,2,2,84.0,84.0
|
||||
06-06 16:23:50,1,1,34.3,44.3,1,0,34.3,44.3,1.0,0.0,1.3,85.3,1,1,44.3,44.3
|
||||
06-06 16:23:55,1,1,141.0,92.1,2,0,141.0,92.1,2.0,0.0,2.5,85.3,2,2,92.1,92.1
|
||||
06-06 16:24:01,1,1,33.3,84.5,2,0,33.3,84.5,2.0,0.0,0.6,55.2,2,2,84.5,84.5
|
||||
06-06 16:24:06,1,1,64.7,131.8,2,0,64.7,131.8,2.0,0.0,0.7,57.6,2,2,131.8,131.8
|
||||
06-06 16:24:11,1,1,110.1,109.9,3,0,110.1,109.9,3.0,0.0,1.2,68.8,3,3,109.9,109.9
|
||||
06-06 16:24:16,1,1,79.5,112.9,3,0,79.5,112.9,3.0,0.0,1.5,72.2,3,3,112.9,112.9
|
||||
06-06 16:24:21,1,1,100.1,119.6,2,0,100.1,119.6,2.0,0.0,1.2,74.6,2,2,119.6,119.6
|
||||
06-06 16:24:26,1,1,101.9,119.1,3,0,101.9,119.1,3.0,0.0,2.1,79.1,3,3,119.1,119.1
|
||||
06-06 16:24:31,1,1,0.0,123.7,2,0,0.0,123.7,2.0,0.0,1.6,79.1,2,2,123.7,123.7
|
||||
06-06 16:24:36,1,1,135.6,61.8,1,0,135.6,61.8,1.0,0.0,1.2,79.9,1,1,61.8,61.8
|
||||
06-06 16:24:41,1,1,58.8,79.8,2,0,58.8,79.8,2.0,0.0,2.0,81.3,2,2,79.8,79.8
|
||||
06-06 16:24:46,1,1,140.7,131.5,3,0,140.7,131.5,3.0,0.0,2.4,82.0,3,3,131.5,131.5
|
||||
06-06 16:24:51,1,1,0.0,145.3,3,0,0.0,145.3,3.0,0.0,2.6,82.0,3,3,145.3,145.3
|
||||
06-06 16:24:56,1,1,0.0,115.4,1,0,0.0,115.4,1.0,0.0,0.6,82.0,1,1,115.4,115.4
|
||||
06-06 16:25:01,1,1,94.5,110.5,3,0,94.5,110.5,3.0,0.0,4.0,82.9,3,3,110.5,110.5
|
||||
06-06 16:25:06,1,1,0.0,133.7,3,0,0.0,133.7,3.0,0.0,4.2,82.9,3,3,133.7,133.7
|
||||
06-06 16:25:11,1,1,207.0,100.0,2,0,207.0,100.0,2.0,0.0,3.2,82.8,2,2,100.0,100.0
|
||||
06-06 16:25:16,1,1,167.4,161.3,4,0,167.4,161.3,4.0,0.0,5.1,83.5,4,4,161.3,161.3
|
||||
06-06 16:25:21,1,1,34.1,162.2,4,0,34.1,162.2,4.0,0.0,5.3,83.7,4,4,162.2,162.2
|
||||
06-06 16:25:26,1,1,0.0,177.9,4,0,0.0,177.9,4.0,0.0,5.6,83.7,4,4,177.9,177.9
|
||||
06-06 16:25:31,1,1,0.0,178.2,4,0,0.0,178.2,4.0,0.0,5.8,83.7,4,4,178.2,178.2
|
||||
06-06 16:25:36,1,1,0.0,163.9,3,0,0.0,163.9,3.0,0.0,5.6,83.7,3,3,163.9,163.9
|
||||
06-06 16:25:41,1,1,85.2,176.5,4,0,85.2,176.5,4.0,0.0,6.4,83.7,4,4,176.5,176.5
|
||||
06-06 16:25:46,1,1,0.0,177.5,4,0,0.0,177.5,4.0,0.0,6.7,83.7,4,4,177.5,177.5
|
||||
06-06 16:25:51,1,1,89.9,182.4,5,0,89.9,182.4,5.0,0.0,7.1,83.2,5,5,182.4,182.4
|
||||
06-06 16:25:56,1,1,0.0,203.1,4,0,0.0,203.1,4.0,0.0,6.6,83.2,4,4,203.1,203.1
|
||||
06-06 16:26:01,1,1,235.6,185.2,4,0,235.6,185.2,4.0,0.0,5.8,83.0,4,4,185.2,185.2
|
||||
06-06 16:26:06,1,1,302.7,241.9,6,0,302.7,241.9,6.0,0.0,8.7,83.3,6,6,241.9,241.9
|
||||
06-06 16:26:11,1,1,133.7,144.7,3,0,133.7,144.7,3.0,0.0,3.9,82.3,3,3,144.7,144.7
|
||||
06-06 16:26:16,1,1,33.1,166.0,4,0,33.1,166.0,4.0,0.0,4.3,82.6,4,4,166.0,166.0
|
||||
06-06 16:26:21,1,1,0.0,192.1,4,0,0.0,192.1,4.0,0.0,4.6,82.6,4,4,192.1,192.1
|
||||
06-06 16:26:26,1,1,66.7,161.8,4,0,66.7,161.8,4.0,0.0,4.6,82.0,4,4,161.8,161.8
|
||||
06-06 16:26:31,1,1,0.0,340.2,7,0,0.0,340.2,7.0,0.0,14.5,86.0,7,7,340.2,340.2
|
||||
06-06 16:26:36,1,1,0.0,340.7,7,0,0.0,340.7,7.0,0.0,15.0,86.0,7,7,340.7,340.7
|
||||
06-06 16:26:41,1,1,0.0,340.0,7,0,0.0,340.0,7.0,0.0,15.5,86.0,7,7,340.0,340.0
|
||||
06-06 16:26:46,1,1,0.0,310.6,6,0,0.0,310.6,6.0,0.0,15.2,86.0,6,6,310.6,310.6
|
||||
06-06 16:26:51,1,1,229.8,331.0,7,0,229.8,331.0,7.0,0.0,16.7,85.6,7,7,331.0,331.0
|
||||
06-06 16:26:56,1,1,0.0,340.1,7,0,0.0,340.1,7.0,0.0,17.2,85.6,7,7,340.1,340.1
|
||||
06-06 16:27:01,1,1,0.0,339.4,7,0,0.0,339.4,7.0,0.0,17.6,85.6,7,7,339.4,339.4
|
||||
06-06 16:27:06,1,1,0.0,339.3,7,0,0.0,339.3,7.0,0.0,18.1,85.6,7,7,339.3,339.3
|
||||
06-06 16:27:11,1,1,0.0,338.9,7,0,0.0,338.9,7.0,0.0,18.6,85.6,7,7,338.9,338.9
|
||||
06-06 16:27:16,1,1,174.0,331.9,8,0,174.0,331.9,8.0,0.0,14.7,88.6,8,8,331.9,331.9
|
||||
06-06 16:27:21,1,1,21.0,346.8,8,0,21.0,346.8,8.0,0.0,15.2,88.9,8,8,346.8,346.8
|
||||
06-06 16:27:26,1,1,0.0,333.8,7,0,0.0,333.8,7.0,0.0,11.3,88.9,7,7,333.8,333.8
|
||||
06-06 16:27:31,1,1,179.3,381.8,9,0,179.3,381.8,9.0,0.0,16.5,89.4,9,9,381.8,381.8
|
||||
06-06 16:27:37,1,1,0.0,243.3,5,0,0.0,243.3,5.0,0.0,10.3,86.5,5,5,243.3,243.3
|
||||
06-06 16:27:42,1,1,0.0,196.8,4,0,0.0,196.8,4.0,0.0,9.1,86.5,4,4,196.8,196.8
|
||||
06-06 16:27:47,1,1,0.0,194.8,4,0,0.0,194.8,4.0,0.0,9.3,86.5,4,4,194.8,194.8
|
||||
06-06 16:27:52,1,1,53.0,228.0,5,0,53.0,228.0,5.0,0.0,11.2,87.0,5,5,228.0,228.0
|
||||
06-06 16:27:57,1,1,116.0,236.7,5,0,116.0,236.7,5.0,0.0,9.4,90.4,5,5,236.7,236.7
|
||||
06-06 16:28:02,1,1,128.5,179.5,4,0,128.5,179.5,4.0,0.0,8.5,86.6,4,4,179.5,179.5
|
||||
06-06 16:28:07,1,1,0.0,120.3,2,0,0.0,120.3,2.0,0.0,7.0,86.6,2,2,120.3,120.3
|
||||
06-06 16:28:12,1,1,158.1,103.7,4,0,158.1,103.7,4.0,0.0,9.1,86.8,4,4,103.7,103.7
|
||||
06-06 16:28:17,1,1,0.0,188.5,3,0,0.0,188.5,3.0,0.0,7.8,86.8,3,3,188.5,188.5
|
||||
06-06 16:28:22,1,1,0.0,151.4,3,0,0.0,151.4,3.0,0.0,8.0,86.8,3,3,151.4,151.4
|
||||
06-06 16:28:27,1,1,0.0,151.6,3,0,0.0,151.6,3.0,0.0,8.3,86.8,3,3,151.6,151.6
|
||||
06-06 16:28:32,1,1,132.5,375.0,10,0,132.5,375.0,10.0,0.0,23.8,90.5,10,10,375.0,375.0
|
||||
06-06 16:28:37,1,1,86.3,428.9,9,0,86.3,428.9,9.0,0.0,24.2,90.5,9,9,428.9,428.9
|
||||
06-06 16:28:42,1,1,21.1,355.5,7,0,21.1,355.5,7.0,0.0,21.8,90.6,7,7,355.5,355.5
|
||||
06-06 16:28:47,1,1,261.5,327.9,7,0,261.5,327.9,7.0,0.0,22.0,90.3,7,7,327.9,327.9
|
||||
06-06 16:28:52,1,1,0.0,270.8,5,0,0.0,270.8,5.0,0.0,8.7,90.1,5,5,270.8,270.8
|
||||
06-06 16:28:57,1,1,278.6,223.4,6,0,278.6,223.4,6.0,0.0,7.7,89.6,6,6,223.4,223.4
|
||||
06-06 16:29:02,1,1,235.0,315.4,9,0,235.0,315.4,9.0,0.0,11.1,89.7,9,9,315.4,315.4
|
||||
06-06 16:29:07,1,1,0.0,370.9,6,0,0.0,370.9,6.0,0.0,10.3,89.7,6,6,370.9,370.9
|
||||
06-06 16:29:12,1,1,32.7,310.2,7,0,32.7,310.2,7.0,0.0,10.9,89.7,7,7,310.2,310.2
|
||||
06-06 16:29:17,1,1,45.1,307.0,7,0,45.1,307.0,7.0,0.0,11.4,89.7,7,7,307.0,307.0
|
||||
06-06 16:29:22,1,1,92.5,185.5,5,0,92.5,185.5,5.0,0.0,11.7,87.1,5,5,185.5,185.5
|
||||
06-06 16:29:27,1,1,0.0,226.2,5,0,0.0,226.2,5.0,0.0,12.1,87.1,5,5,226.2,226.2
|
||||
06-06 16:29:32,1,1,0.0,228.0,5,0,0.0,228.0,5.0,0.0,12.4,87.1,5,5,228.0,228.0
|
||||
06-06 16:29:37,1,1,0.0,227.7,5,0,0.0,227.7,5.0,0.0,12.7,87.1,5,5,227.7,227.7
|
||||
06-06 16:29:42,1,1,0.0,227.8,5,0,0.0,227.8,5.0,0.0,13.0,87.1,5,5,227.8,227.8
|
||||
06-06 16:29:47,1,1,0.0,191.1,4,0,0.0,191.1,4.0,0.0,12.9,87.1,4,4,191.1,191.1
|
||||
06-06 16:29:52,1,1,0.0,181.1,4,0,0.0,181.1,4.0,0.0,13.1,87.1,4,4,181.1,181.1
|
||||
06-06 16:29:57,1,1,0.0,469.8,10,0,0.0,469.8,10.0,0.0,34.5,87.6,10,10,469.8,469.8
|
||||
06-06 16:30:02,1,1,111.6,463.2,11,0,111.6,463.2,11.0,0.0,35.3,87.5,11,11,463.2,463.2
|
||||
06-06 16:30:07,1,1,0.0,178.7,4,0,0.0,178.7,4.0,0.0,13.9,87.1,4,4,178.7,178.7
|
||||
06-06 16:30:12,1,1,73.1,221.3,5,0,73.1,221.3,5.0,0.0,14.4,87.4,5,5,221.3,221.3
|
||||
06-06 16:30:17,1,1,0.0,503.3,11,0,0.0,503.3,11.0,0.0,37.4,87.5,11,11,503.3,503.3
|
||||
06-06 16:30:22,1,1,0.0,494.6,10,0,0.0,494.6,10.0,0.0,37.7,87.5,10,10,494.6,494.6
|
||||
06-06 16:30:27,1,1,21.8,468.7,11,0,21.8,468.7,11.0,0.0,39.6,87.7,11,11,468.7,468.7
|
||||
06-06 16:30:32,1,1,0.0,502.1,11,0,0.0,502.1,11.0,0.0,40.3,87.7,11,11,502.1,502.1
|
||||
06-06 16:30:37,1,1,0.0,501.5,11,0,0.0,501.5,11.0,0.0,41.0,87.7,11,11,501.5,501.5
|
||||
06-06 16:30:43,1,1,0.0,502.4,11,0,0.0,502.4,11.0,0.0,41.7,87.7,11,11,502.4,502.4
|
||||
06-06 16:30:48,1,1,206.7,569.4,13,0,206.7,569.4,13.0,0.0,43.0,87.5,13,13,569.4,569.4
|
||||
06-06 16:30:53,1,1,0.0,569.3,12,0,0.0,569.3,12.0,0.0,43.3,87.5,12,12,569.3,569.3
|
||||
06-06 16:30:58,1,1,0.0,548.5,12,0,0.0,548.5,12.0,0.0,44.1,87.5,12,12,548.5,548.5
|
||||
06-06 16:31:03,1,1,0.0,234.7,5,0,0.0,234.7,5.0,0.0,23.8,87.0,5,5,234.7,234.7
|
||||
06-06 16:31:08,1,1,0.0,234.2,5,0,0.0,234.2,5.0,0.0,24.2,87.0,5,5,234.2,234.2
|
||||
06-06 16:31:13,1,1,0.0,234.8,5,0,0.0,234.8,5.0,0.0,24.5,87.0,5,5,234.8,234.8
|
||||
06-06 16:31:18,1,1,0.0,239.5,5,0,0.0,239.5,5.0,0.0,22.1,88.0,5,5,239.5,239.5
|
||||
06-06 16:31:23,1,1,0.0,239.9,5,0,0.0,239.9,5.0,0.0,22.4,88.0,5,5,239.9,239.9
|
||||
06-06 16:31:28,1,1,22.8,237.9,5,0,22.8,237.9,5.0,0.0,12.8,87.4,5,5,237.9,237.9
|
||||
06-06 16:31:33,1,1,0.0,244.4,5,0,0.0,244.4,5.0,0.0,13.2,87.4,5,5,244.4,244.4
|
||||
06-06 16:31:38,1,1,0.0,197.1,4,0,0.0,197.1,4.0,0.0,12.0,87.4,4,4,197.1,197.1
|
||||
06-06 16:31:43,1,1,0.0,320.7,7,0,0.0,320.7,7.0,0.0,29.6,85.8,7,7,320.7,320.7
|
||||
06-06 16:31:48,1,1,0.0,320.6,7,0,0.0,320.6,7.0,0.0,30.0,85.8,7,7,320.6,320.6
|
||||
06-06 16:31:53,1,1,67.4,364.3,8,0,67.4,364.3,8.0,0.0,30.6,85.7,8,8,364.3,364.3
|
||||
06-06 16:31:58,1,1,0.0,337.8,7,0,0.0,337.8,7.0,0.0,30.9,85.7,7,7,337.8,337.8
|
||||
06-06 16:32:03,1,1,192.5,368.1,9,0,192.5,368.1,9.0,0.0,35.0,86.2,9,9,368.1,368.1
|
||||
06-06 16:32:08,1,1,0.0,409.4,8,0,0.0,409.4,8.0,0.0,33.7,86.2,8,8,409.4,409.4
|
||||
06-06 16:32:13,1,1,0.0,351.9,7,0,0.0,351.9,7.0,0.0,32.3,86.2,7,7,351.9,351.9
|
||||
06-06 16:32:19,1,1,0.0,362.6,8,0,0.0,362.6,8.0,0.0,38.1,88.8,8,8,362.6,362.6
|
||||
06-06 16:32:24,1,1,0.0,362.4,8,0,0.0,362.4,8.0,0.0,38.6,88.8,8,8,362.4,362.4
|
||||
06-06 16:32:29,1,1,102.4,367.8,9,0,102.4,367.8,9.0,0.0,45.1,89.4,9,9,367.8,367.8
|
||||
06-06 16:32:34,1,1,0.0,407.1,9,0,0.0,407.1,9.0,0.0,45.7,89.4,9,9,407.1,407.1
|
||||
06-06 16:32:39,1,1,0.0,406.3,9,0,0.0,406.3,9.0,0.0,46.2,89.4,9,9,406.3,406.3
|
||||
06-06 16:32:43,1,1,113.1,328.8,8,0,113.1,328.8,8.0,0.0,29.5,88.3,8,8,328.8,328.8
|
||||
06-06 16:32:49,1,1,0.0,187.6,4,0,0.0,187.6,4.0,0.0,13.2,89.2,4,4,187.6,187.6
|
||||
06-06 16:32:54,1,1,0.0,187.4,4,0,0.0,187.4,4.0,0.0,13.5,89.2,4,4,187.4,187.4
|
||||
06-06 16:32:59,1,1,0.0,187.7,4,0,0.0,187.7,4.0,0.0,13.8,89.2,4,4,187.7,187.7
|
||||
06-06 16:33:04,1,1,35.4,446.6,10,0,35.4,446.6,10.0,0.0,49.4,89.3,10,10,446.6,446.6
|
||||
06-06 16:33:09,1,1,0.0,187.6,4,0,0.0,187.6,4.0,0.0,14.3,89.2,4,4,187.6,187.6
|
||||
06-06 16:33:14,1,1,86.1,211.6,5,0,86.1,211.6,5.0,0.0,14.7,89.1,5,5,211.6,211.6
|
||||
06-06 16:33:19,1,1,0.0,211.4,4,0,0.0,211.4,4.0,0.0,14.8,89.1,4,4,211.4,211.4
|
||||
06-06 16:33:24,1,1,0.0,450.6,10,0,0.0,450.6,10.0,0.0,51.9,89.3,10,10,450.6,450.6
|
||||
06-06 16:33:29,1,1,0.0,280.1,6,0,0.0,280.1,6.0,0.0,16.6,89.2,6,6,280.1,280.1
|
||||
06-06 16:33:34,1,1,0.0,280.8,6,0,0.0,280.8,6.0,0.0,17.0,89.2,6,6,280.8,280.8
|
||||
06-06 16:33:39,1,1,0.0,275.1,5,0,0.0,275.1,5.0,0.0,16.3,89.2,5,5,275.1,275.1
|
||||
06-06 16:33:44,1,1,0.0,233.8,5,0,0.0,233.8,5.0,0.0,16.7,89.2,5,5,233.8,233.8
|
||||
06-06 16:33:49,1,1,9.4,248.9,6,0,9.4,248.9,6.0,0.0,24.3,90.3,6,6,248.9,248.9
|
||||
06-06 16:33:54,1,1,0.0,401.5,7,0,0.0,401.5,7.0,0.0,37.0,85.5,7,7,401.5,401.5
|
||||
06-06 16:33:59,1,1,0.0,320.7,7,0,0.0,320.7,7.0,0.0,37.4,85.5,7,7,320.7,320.7
|
||||
06-06 16:34:04,1,1,72.3,353.9,8,0,72.3,353.9,8.0,0.0,38.0,85.4,8,8,353.9,353.9
|
||||
06-06 16:34:09,1,1,183.9,451.2,9,0,183.9,451.2,9.0,0.0,38.9,85.3,9,9,451.2,451.2
|
||||
06-06 16:34:14,1,1,0.0,377.2,7,0,0.0,377.2,7.0,0.0,38.7,85.3,7,7,377.2,377.2
|
||||
06-06 16:34:19,1,1,36.0,351.9,7,0,36.0,351.9,7.0,0.0,39.2,85.3,7,7,351.9,351.9
|
||||
06-06 16:34:24,1,1,74.6,267.9,6,0,74.6,267.9,6.0,0.0,16.0,85.4,6,6,267.9,267.9
|
||||
06-06 16:34:29,1,1,0.0,287.1,6,0,0.0,287.1,6.0,0.0,16.4,85.4,6,6,287.1,287.1
|
||||
06-06 16:34:34,1,1,108.9,319.9,8,0,108.9,319.9,8.0,0.0,18.6,85.7,8,8,319.9,319.9
|
||||
06-06 16:34:40,1,1,71.5,410.9,9,0,71.5,410.9,9.0,0.0,20.2,85.8,9,9,410.9,410.9
|
||||
06-06 16:34:45,1,1,0.0,430.8,9,0,0.0,430.8,9.0,0.0,20.8,85.8,9,9,430.8,430.8
|
||||
06-06 16:34:50,1,1,45.4,424.3,8,0,45.4,424.3,8.0,0.0,20.0,86.1,8,8,424.3,424.3
|
||||
06-06 16:34:55,1,1,69.1,396.7,9,0,69.1,396.7,9.0,0.0,20.7,86.0,9,9,396.7,396.7
|
||||
06-06 16:35:00,1,1,0.0,414.9,7,0,0.0,414.9,7.0,0.0,20.5,86.0,7,7,414.9,414.9
|
||||
06-06 16:35:05,1,1,523.2,396.0,11,0,523.2,396.0,11.0,0.0,24.7,85.9,11,11,396.0,396.0
|
||||
06-06 16:35:10,1,1,119.0,551.2,11,0,119.0,551.2,11.0,0.0,25.7,85.9,11,11,551.2,551.2
|
||||
06-06 16:35:15,1,1,26.7,491.7,10,0,26.7,491.7,10.0,0.0,44.5,89.7,10,10,491.7,491.7
|
||||
06-06 16:35:20,1,1,12.2,426.6,11,0,12.2,426.6,11.0,0.0,46.2,89.7,11,11,426.6,426.6
|
||||
06-06 16:35:25,1,1,27.1,471.4,11,0,27.1,471.4,11.0,0.0,46.6,89.8,11,11,471.4,471.4
|
||||
06-06 16:35:30,1,1,0.0,465.7,11,0,0.0,465.7,11.0,0.0,47.3,89.8,11,11,465.7,465.7
|
||||
06-06 16:35:35,1,1,96.6,441.5,10,0,96.6,441.5,10.0,0.0,46.9,89.8,10,10,441.5,441.5
|
||||
06-06 16:35:40,1,1,41.0,436.2,10,0,41.0,436.2,10.0,0.0,47.6,89.8,10,10,436.2,436.2
|
||||
06-06 16:35:45,1,1,105.8,423.1,11,0,105.8,423.1,11.0,0.0,49.2,89.9,11,11,423.1,423.1
|
||||
06-06 16:35:50,1,1,21.0,464.6,12,0,21.0,464.6,12.0,0.0,53.0,90.0,12,12,464.6,464.6
|
||||
06-06 16:35:55,1,1,112.0,564.5,14,0,112.0,564.5,14.0,0.0,56.0,90.2,14,14,564.5,564.5
|
||||
06-06 16:36:00,1,1,27.3,445.8,10,0,27.3,445.8,10.0,0.0,30.9,85.6,10,10,445.8,445.8
|
||||
06-06 16:36:05,1,1,0.0,440.9,9,0,0.0,440.9,9.0,0.0,51.7,87.3,9,9,440.9,440.9
|
||||
06-06 16:36:10,1,1,0.0,439.0,9,0,0.0,439.0,9.0,0.0,52.3,87.3,9,9,439.0,439.0
|
||||
06-06 16:36:15,1,1,0.0,439.2,9,0,0.0,439.2,9.0,0.0,52.9,87.3,9,9,439.2,439.2
|
||||
06-06 16:36:20,1,1,79.2,447.8,10,0,79.2,447.8,10.0,0.0,53.7,87.2,10,10,447.8,447.8
|
||||
06-06 16:36:25,1,1,0.0,467.6,8,0,0.0,467.6,8.0,0.0,45.2,87.2,8,8,467.6,467.6
|
||||
06-06 16:36:30,1,1,0.0,393.8,8,0,0.0,393.8,8.0,0.0,45.7,87.2,8,8,393.8,393.8
|
||||
06-06 16:36:35,1,1,0.0,392.2,8,0,0.0,392.2,8.0,0.0,46.3,87.2,8,8,392.2,392.2
|
||||
06-06 16:36:40,1,1,104.0,426.7,9,0,104.0,426.7,9.0,0.0,47.0,87.1,9,9,426.7,426.7
|
||||
06-06 16:36:45,1,1,113.7,442.1,9,0,113.7,442.1,9.0,0.0,47.9,87.0,9,9,442.1,442.1
|
||||
06-06 16:36:50,1,1,64.4,481.9,10,0,64.4,481.9,10.0,0.0,48.6,86.9,10,10,481.9,481.9
|
||||
06-06 16:36:55,1,1,10.6,490.7,10,0,10.6,490.7,10.0,0.0,49.2,87.0,10,10,490.7,490.7
|
||||
06-06 16:37:00,1,1,10.2,495.5,11,0,10.2,495.5,11.0,0.0,50.0,87.0,11,11,495.5,495.5
|
||||
06-06 16:37:05,1,1,0.0,450.2,9,0,0.0,450.2,9.0,0.0,41.1,87.0,9,9,450.2,450.2
|
||||
06-06 16:37:10,1,1,3.2,367.1,7,0,3.2,367.1,7.0,0.0,32.2,87.0,7,7,367.1,367.1
|
||||
06-06 16:37:15,1,1,0.0,346.8,7,0,0.0,346.8,7.0,0.0,32.7,87.0,7,7,346.8,346.8
|
||||
06-06 16:37:20,1,1,0.0,299.1,6,0,0.0,299.1,6.0,0.0,32.9,87.0,6,6,299.1,299.1
|
||||
06-06 16:37:25,1,1,0.0,295.5,6,0,0.0,295.5,6.0,0.0,33.3,87.0,6,6,295.5,295.5
|
||||
06-06 16:37:30,1,1,34.6,341.0,7,0,34.6,341.0,7.0,0.0,34.0,87.0,7,7,341.0,341.0
|
||||
06-06 16:37:35,1,1,11.6,401.2,9,0,11.6,401.2,9.0,0.0,34.7,87.1,9,9,401.2,401.2
|
||||
06-06 16:37:40,1,1,48.3,434.9,8,0,48.3,434.9,8.0,0.0,35.0,87.1,8,8,434.9,434.9
|
||||
06-06 16:37:45,1,1,0.0,360.9,7,0,0.0,360.9,7.0,0.0,26.6,87.1,7,7,360.9,360.9
|
||||
06-06 16:37:50,1,1,46.3,358.1,7,0,46.3,358.1,7.0,0.0,21.0,87.9,7,7,358.1,358.1
|
||||
06-06 16:37:55,1,1,0.0,327.4,7,0,0.0,327.4,7.0,0.0,21.5,87.9,7,7,327.4,327.4
|
||||
06-06 16:38:00,1,1,236.9,385.3,8,0,236.9,385.3,8.0,0.0,29.2,86.9,8,8,385.3,385.3
|
||||
06-06 16:38:05,1,1,34.9,407.6,9,0,34.9,407.6,9.0,0.0,30.0,86.9,9,9,407.6,407.6
|
||||
06-06 16:38:10,1,1,130.3,472.6,10,0,130.3,472.6,10.0,0.0,31.1,86.8,10,10,472.6,472.6
|
||||
06-06 16:38:15,1,1,45.3,488.8,10,0,45.3,488.8,10.0,0.0,31.6,86.8,10,10,488.8,488.8
|
||||
06-06 16:38:20,1,1,0.0,492.4,10,0,0.0,492.4,10.0,0.0,32.3,86.8,10,10,492.4,492.4
|
||||
06-06 16:38:25,1,1,43.3,472.9,9,0,43.3,472.9,9.0,0.0,31.7,86.8,9,9,472.9,472.9
|
||||
06-06 16:38:30,1,1,0.0,443.6,9,0,0.0,443.6,9.0,0.0,32.3,86.8,9,9,443.6,443.6
|
||||
06-06 16:38:36,1,1,0.0,441.4,9,0,0.0,441.4,9.0,0.0,32.9,86.8,9,9,441.4,441.4
|
||||
06-06 16:38:41,1,1,34.9,222.5,5,0,34.9,222.5,5.0,0.0,9.6,91.1,5,5,222.5,222.5
|
||||
06-06 16:38:46,1,1,0.0,264.1,6,0,0.0,264.1,6.0,0.0,31.5,88.1,6,6,264.1,264.1
|
||||
06-06 16:38:51,1,1,0.0,264.5,6,0,0.0,264.5,6.0,0.0,31.9,88.1,6,6,264.5,264.5
|
||||
06-06 16:38:56,1,1,135.9,302.8,8,0,135.9,302.8,8.0,0.0,35.0,88.2,8,8,302.8,302.8
|
||||
06-06 16:39:01,1,1,0.0,334.8,7,0,0.0,334.8,7.0,0.0,34.5,88.2,7,7,334.8,334.8
|
||||
06-06 16:39:06,1,1,41.8,321.5,8,0,41.8,321.5,8.0,0.0,27.7,88.4,8,8,321.5,321.5
|
||||
06-06 16:39:11,1,1,0.0,348.6,8,0,0.0,348.6,8.0,0.0,28.2,88.4,8,8,348.6,348.6
|
||||
06-06 16:39:16,1,1,146.9,348.5,9,0,146.9,348.5,9.0,0.0,29.6,88.3,9,9,348.5,348.5
|
||||
06-06 16:39:21,1,1,0.0,393.4,9,0,0.0,393.4,9.0,0.0,30.2,88.3,9,9,393.4,393.4
|
||||
06-06 16:39:26,1,1,118.2,638.1,14,0,118.2,638.1,14.0,0.0,59.9,88.8,14,14,638.1,638.1
|
||||
06-06 16:39:31,1,1,75.3,657.2,14,0,75.3,657.2,14.0,0.0,57.6,88.8,14,14,657.2,657.2
|
||||
06-06 16:39:36,1,1,0.0,442.2,10,0,0.0,442.2,10.0,0.0,51.8,88.7,10,10,442.2,442.2
|
||||
06-06 16:39:41,1,1,318.8,478.4,11,0,318.8,478.4,11.0,0.0,57.2,88.8,11,11,478.4,478.4
|
||||
06-06 16:39:46,1,1,0.0,485.0,11,0,0.0,485.0,11.0,0.0,57.8,88.8,11,11,485.0,485.0
|
||||
06-06 16:39:51,1,1,56.9,502.0,12,0,56.9,502.0,12.0,0.0,59.0,88.8,12,12,502.0,502.0
|
||||
06-06 16:39:56,1,1,0.0,529.3,12,0,0.0,529.3,12.0,0.0,59.7,88.8,12,12,529.3,529.3
|
||||
06-06 16:40:01,1,1,53.4,505.0,11,0,53.4,505.0,11.0,0.0,55.3,88.8,11,11,505.0,505.0
|
||||
06-06 16:40:06,1,1,0.0,474.3,10,0,0.0,474.3,10.0,0.0,55.5,88.8,10,10,474.3,474.3
|
||||
06-06 16:40:11,1,1,178.4,442.7,10,0,178.4,442.7,10.0,0.0,39.2,88.9,10,10,442.7,442.7
|
||||
06-06 16:40:16,1,1,435.1,465.8,12,0,435.1,465.8,12.0,0.0,57.6,88.4,12,12,465.8,465.8
|
||||
06-06 16:40:21,1,1,0.0,527.9,12,0,0.0,527.9,12.0,0.0,58.4,88.4,12,12,527.9,527.9
|
||||
06-06 16:40:26,1,1,0.0,529.6,12,0,0.0,529.6,12.0,0.0,59.1,88.4,12,12,529.6,529.6
|
||||
06-06 16:40:32,1,1,86.0,468.1,10,0,86.0,468.1,10.0,0.0,30.7,88.6,10,10,468.1,468.1
|
||||
06-06 16:40:37,1,1,163.0,478.4,11,0,163.0,478.4,11.0,0.0,32.8,88.6,11,11,478.4,478.4
|
||||
06-06 16:40:42,1,1,200.0,546.9,12,0,200.0,546.9,12.0,0.0,34.7,88.5,12,12,546.9,546.9
|
||||
06-06 16:40:47,1,1,191.3,560.0,13,0,191.3,560.0,13.0,0.0,36.7,88.3,13,13,560.0,560.0
|
||||
06-06 16:40:52,1,1,0.0,596.3,13,0,0.0,596.3,13.0,0.0,37.6,88.3,13,13,596.3,596.3
|
||||
06-06 16:40:57,1,1,0.0,547.9,11,0,0.0,547.9,11.0,0.0,35.7,88.3,11,11,547.9,547.9
|
||||
06-06 16:41:01,1,1,2.2,687.9,15,0,2.2,687.9,15.0,0.0,63.9,88.9,15,15,687.9,687.9
|
||||
06-06 16:41:07,1,1,0.0,461.0,10,0,0.0,461.0,10.0,0.0,35.6,88.3,10,10,461.0,461.0
|
||||
06-06 16:41:12,1,1,0.0,445.9,9,0,0.0,445.9,9.0,0.0,34.6,88.3,9,9,445.9,445.9
|
||||
06-06 16:41:17,1,1,0.0,413.5,9,0,0.0,413.5,9.0,0.0,35.2,88.3,9,9,413.5,413.5
|
||||
06-06 16:41:22,1,1,0.0,414.6,9,0,0.0,414.6,9.0,0.0,35.8,88.3,9,9,414.6,414.6
|
||||
06-06 16:41:27,1,1,0.0,414.5,9,0,0.0,414.5,9.0,0.0,36.4,88.3,9,9,414.5,414.5
|
||||
06-06 16:41:32,1,1,0.0,414.1,9,0,0.0,414.1,9.0,0.0,37.0,88.3,9,9,414.1,414.1
|
||||
06-06 16:41:37,1,1,0.0,414.1,9,0,0.0,414.1,9.0,0.0,37.5,88.3,9,9,414.1,414.1
|
||||
06-06 16:41:42,1,1,0.0,414.7,9,0,0.0,414.7,9.0,0.0,38.1,88.3,9,9,414.7,414.7
|
||||
06-06 16:41:47,1,1,0.0,415.2,9,0,0.0,415.2,9.0,0.0,38.7,88.3,9,9,415.2,415.2
|
||||
06-06 16:41:52,1,1,0.0,414.8,9,0,0.0,414.8,9.0,0.0,39.3,88.3,9,9,414.8,414.8
|
||||
06-06 16:41:57,1,1,150.7,446.5,11,0,150.7,446.5,11.0,0.0,42.8,88.5,11,11,446.5,446.5
|
||||
06-06 16:42:02,1,1,0.0,506.7,11,0,0.0,506.7,11.0,0.0,43.5,88.5,11,11,506.7,506.7
|
||||
06-06 16:42:07,1,1,178.8,509.4,11,0,178.8,509.4,11.0,0.0,43.9,88.5,11,11,509.4,509.4
|
||||
06-06 16:42:12,1,1,0.0,389.7,9,0,0.0,389.7,9.0,0.0,26.8,92.2,9,9,389.7,389.7
|
||||
06-06 16:42:17,1,1,0.0,390.5,9,0,0.0,390.5,9.0,0.0,27.4,92.2,9,9,390.5,390.5
|
||||
06-06 16:42:22,1,1,0.0,394.7,9,0,0.0,394.7,9.0,0.0,27.9,92.2,9,9,394.7,394.7
|
||||
06-06 16:42:27,1,1,0.0,391.7,9,0,0.0,391.7,9.0,0.0,28.5,92.2,9,9,391.7,391.7
|
||||
06-06 16:42:32,1,1,0.0,389.7,9,0,0.0,389.7,9.0,0.0,29.0,92.2,9,9,389.7,389.7
|
||||
06-06 16:42:37,1,1,0.0,390.3,9,0,0.0,390.3,9.0,0.0,29.5,92.2,9,9,390.3,390.3
|
||||
06-06 16:42:42,1,1,40.9,401.4,10,0,40.9,401.4,10.0,0.0,31.4,92.2,10,10,401.4,401.4
|
||||
06-06 16:42:47,1,1,0.0,433.3,10,0,0.0,433.3,10.0,0.0,32.0,92.2,10,10,433.3,433.3
|
||||
06-06 16:42:52,1,1,0.0,459.6,10,0,0.0,459.6,10.0,0.0,55.4,88.3,10,10,459.6,459.6
|
||||
06-06 16:42:57,1,1,174.0,460.1,11,0,174.0,460.1,11.0,0.0,57.5,88.2,11,11,460.1,460.1
|
||||
06-06 16:43:02,1,1,0.0,505.4,11,0,0.0,505.4,11.0,0.0,58.2,88.2,11,11,505.4,505.4
|
||||
06-06 16:43:07,1,1,0.0,479.1,10,0,0.0,479.1,10.0,0.0,57.3,88.2,10,10,479.1,479.1
|
||||
06-06 16:43:12,1,1,0.0,460.8,10,0,0.0,460.8,10.0,0.0,58.0,88.2,10,10,460.8,460.8
|
||||
06-06 16:43:17,1,1,117.3,468.1,11,0,117.3,468.1,11.0,0.0,61.9,88.5,11,11,468.1,468.1
|
||||
06-06 16:43:22,1,1,89.4,491.6,11,0,89.4,491.6,11.0,0.0,59.4,88.4,11,11,491.6,491.6
|
||||
06-06 16:43:27,1,1,0.0,507.4,11,0,0.0,507.4,11.0,0.0,60.1,88.4,11,11,507.4,507.4
|
||||
06-06 16:43:32,1,1,0.0,506.9,11,0,0.0,506.9,11.0,0.0,60.8,88.4,11,11,506.9,506.9
|
||||
06-06 16:43:37,1,1,0.0,506.3,11,0,0.0,506.3,11.0,0.0,61.5,88.4,11,11,506.3,506.3
|
||||
06-06 16:43:42,1,1,0.0,505.9,11,0,0.0,505.9,11.0,0.0,62.2,88.4,11,11,505.9,505.9
|
||||
06-06 16:43:47,1,1,0.0,505.7,11,0,0.0,505.7,11.0,0.0,62.9,88.4,11,11,505.7,505.7
|
||||
06-06 16:43:52,1,1,0.0,505.8,11,0,0.0,505.8,11.0,0.0,63.6,88.4,11,11,505.8,505.8
|
||||
06-06 16:43:57,1,1,211.1,537.7,13,0,211.1,537.7,13.0,0.0,66.6,88.4,13,13,537.7,537.7
|
||||
06-06 16:44:02,1,1,0.0,598.4,13,0,0.0,598.4,13.0,0.0,67.5,88.4,13,13,598.4,598.4
|
||||
06-06 16:44:07,1,1,0.0,574.1,12,0,0.0,574.1,12.0,0.0,67.7,88.4,12,12,574.1,574.1
|
||||
06-06 16:44:12,1,1,0.0,552.9,12,0,0.0,552.9,12.0,0.0,68.4,88.4,12,12,552.9,552.9
|
||||
06-06 16:44:18,1,1,0.0,272.3,6,0,0.0,272.3,6.0,0.0,29.1,88.8,6,6,272.3,272.3
|
||||
06-06 16:44:23,1,1,0.0,272.8,6,0,0.0,272.8,6.0,0.0,29.5,88.8,6,6,272.8,272.8
|
||||
06-06 16:44:28,1,1,0.0,272.7,6,0,0.0,272.7,6.0,0.0,29.9,88.8,6,6,272.7,272.7
|
||||
06-06 16:44:33,1,1,0.0,272.3,5,0,0.0,272.3,5.0,0.0,28.9,88.8,5,5,272.3,272.3
|
||||
06-06 16:44:38,1,1,72.0,266.8,6,0,72.0,266.8,6.0,0.0,31.1,88.9,6,6,266.8,266.8
|
||||
06-06 16:44:42,1,1,79.5,518.1,12,0,79.5,518.1,12.0,0.0,72.6,88.7,12,12,518.1,518.1
|
||||
06-06 16:44:48,1,1,0.0,272.9,6,0,0.0,272.9,6.0,0.0,31.8,88.9,6,6,272.9,272.9
|
||||
06-06 16:44:53,1,1,78.7,294.7,7,0,78.7,294.7,7.0,0.0,32.4,88.8,7,7,294.7,294.7
|
||||
06-06 16:44:58,1,1,0.0,317.6,7,0,0.0,317.6,7.0,0.0,32.8,88.8,7,7,317.6,317.6
|
||||
06-06 16:45:03,1,1,0.0,318.7,7,0,0.0,318.7,7.0,0.0,33.3,88.8,7,7,318.7,318.7
|
||||
06-06 16:45:08,1,1,0.0,318.1,7,0,0.0,318.1,7.0,0.0,33.7,88.8,7,7,318.1,318.1
|
||||
06-06 16:45:13,1,1,0.0,317.8,7,0,0.0,317.8,7.0,0.0,34.1,88.8,7,7,317.8,317.8
|
||||
06-06 16:45:18,1,1,0.0,215.9,4,0,0.0,215.9,4.0,0.0,16.8,93.8,4,4,215.9,215.9
|
||||
06-06 16:45:23,1,1,0.0,192.9,4,0,0.0,192.9,4.0,0.0,17.1,93.8,4,4,192.9,192.9
|
||||
06-06 16:45:28,1,1,0.0,192.8,4,0,0.0,192.8,4.0,0.0,17.3,93.8,4,4,192.8,192.8
|
||||
06-06 16:45:33,1,1,0.0,193.0,4,0,0.0,193.0,4.0,0.0,17.6,93.8,4,4,193.0,193.0
|
||||
06-06 16:45:38,1,1,141.6,228.5,5,0,141.6,228.5,5.0,0.0,21.7,93.8,5,5,228.5,228.5
|
||||
06-06 16:45:43,1,1,0.0,207.0,4,0,0.0,207.0,4.0,0.0,13.1,93.8,4,4,207.0,207.0
|
||||
06-06 16:45:48,1,1,41.1,229.8,5,0,41.1,229.8,5.0,0.0,14.1,93.8,5,5,229.8,229.8
|
||||
06-06 16:45:53,1,1,0.0,195.4,4,0,0.0,195.4,4.0,0.0,10.4,93.8,4,4,195.4,195.4
|
||||
06-06 16:45:58,1,1,0.0,193.4,4,0,0.0,193.4,4.0,0.0,10.7,93.8,4,4,193.4,193.4
|
||||
06-06 16:46:03,1,1,0.0,193.7,4,0,0.0,193.7,4.0,0.0,11.0,93.8,4,4,193.7,193.7
|
||||
06-06 16:46:08,1,1,0.0,193.7,4,0,0.0,193.7,4.0,0.0,11.2,93.8,4,4,193.7,193.7
|
||||
06-06 16:46:13,1,1,0.0,193.8,4,0,0.0,193.8,4.0,0.0,11.5,93.8,4,4,193.8,193.8
|
||||
06-06 16:46:18,1,1,0.0,193.8,4,0,0.0,193.8,4.0,0.0,11.8,93.8,4,4,193.8,193.8
|
||||
06-06 16:46:23,1,1,0.0,193.0,4,0,0.0,193.0,4.0,0.0,12.0,93.8,4,4,193.0,193.0
|
||||
06-06 16:46:28,1,1,0.0,194.0,4,0,0.0,194.0,4.0,0.0,12.3,93.8,4,4,194.0,194.0
|
||||
06-06 16:46:33,1,1,0.0,193.3,4,0,0.0,193.3,4.0,0.0,12.6,93.8,4,4,193.3,193.3
|
||||
06-06 16:46:38,1,1,0.0,193.5,4,0,0.0,193.5,4.0,0.0,12.9,93.8,4,4,193.5,193.5
|
||||
06-06 16:46:43,1,1,0.0,458.8,10,0,0.0,458.8,10.0,0.0,34.0,87.7,10,10,458.8,458.8
|
||||
06-06 16:46:48,1,1,0.0,457.9,10,0,0.0,457.9,10.0,0.0,34.6,87.7,10,10,457.9,457.9
|
||||
06-06 16:46:53,1,1,0.0,458.5,10,0,0.0,458.5,10.0,0.0,35.3,87.7,10,10,458.5,458.5
|
||||
06-06 16:46:58,1,1,0.0,458.1,10,0,0.0,458.1,10.0,0.0,35.9,87.7,10,10,458.1,458.1
|
||||
06-06 16:47:03,1,1,0.0,459.4,10,0,0.0,459.4,10.0,0.0,36.5,87.7,10,10,459.4,459.4
|
||||
06-06 16:47:08,1,1,0.0,457.7,10,0,0.0,457.7,10.0,0.0,37.2,87.7,10,10,457.7,457.7
|
||||
06-06 16:47:13,1,1,0.0,458.2,10,0,0.0,458.2,10.0,0.0,37.8,87.7,10,10,458.2,458.2
|
||||
06-06 16:47:18,1,1,0.0,459.3,10,0,0.0,459.3,10.0,0.0,38.5,87.7,10,10,459.3,459.3
|
||||
06-06 16:47:23,1,1,0.0,631.8,13,0,0.0,631.8,13.0,0.0,75.1,90.7,13,13,631.8,631.8
|
||||
06-06 16:47:28,1,1,0.0,615.0,12,0,0.0,615.0,12.0,0.0,75.0,90.7,12,12,615.0,615.0
|
||||
06-06 16:47:33,1,1,0.0,579.2,12,0,0.0,579.2,12.0,0.0,75.9,90.7,12,12,579.2,579.2
|
||||
06-06 16:47:38,1,1,0.0,574.5,12,0,0.0,574.5,12.0,0.0,76.7,90.7,12,12,574.5,574.5
|
||||
06-06 16:47:43,1,1,21.8,618.3,14,0,21.8,618.3,14.0,0.0,77.7,90.7,14,14,618.3,618.3
|
||||
06-06 16:47:48,1,1,0.0,642.5,13,0,0.0,642.5,13.0,0.0,78.3,90.7,13,13,642.5,642.5
|
||||
06-06 16:47:54,1,1,0.0,603.0,12,0,0.0,603.0,12.0,0.0,70.2,90.7,12,12,603.0,603.0
|
||||
06-06 16:47:59,1,1,90.8,625.4,13,0,90.8,625.4,13.0,0.0,73.3,90.8,13,13,625.4,625.4
|
||||
06-06 16:48:04,1,1,0.0,502.4,11,0,0.0,502.4,11.0,0.0,59.2,93.7,11,11,502.4,502.4
|
||||
06-06 16:48:09,1,1,0.0,503.4,11,0,0.0,503.4,11.0,0.0,59.9,93.7,11,11,503.4,503.4
|
||||
06-06 16:48:14,1,1,518.3,491.9,11,0,518.3,491.9,11.0,0.0,64.2,93.6,11,11,491.9,491.9
|
||||
06-06 16:48:19,1,1,45.4,500.7,11,0,45.4,500.7,11.0,0.0,61.1,93.6,11,11,500.7,500.7
|
||||
06-06 16:48:24,1,1,0.0,499.6,11,0,0.0,499.6,11.0,0.0,61.8,93.6,11,11,499.6,499.6
|
||||
06-06 16:48:29,1,1,0.0,481.9,10,0,0.0,481.9,10.0,0.0,53.6,93.6,10,10,481.9,481.9
|
||||
06-06 16:48:34,1,1,13.4,517.5,12,0,13.4,517.5,12.0,0.0,55.5,93.6,12,12,517.5,517.5
|
||||
06-06 16:48:39,1,1,0.0,545.4,12,0,0.0,545.4,12.0,0.0,56.2,93.6,12,12,545.4,545.4
|
||||
06-06 16:48:44,1,1,0.0,502.3,11,0,0.0,502.3,11.0,0.0,56.3,93.6,11,11,502.3,502.3
|
||||
06-06 16:48:49,1,1,0.0,499.3,11,0,0.0,499.3,11.0,0.0,57.1,93.6,11,11,499.3,499.3
|
||||
06-06 16:48:54,1,1,49.5,504.5,12,0,49.5,504.5,12.0,0.0,58.6,93.6,12,12,504.5,504.5
|
||||
06-06 16:48:59,1,1,0.0,485.1,8,0,0.0,485.1,8.0,0.0,31.4,88.9,8,8,485.1,485.1
|
||||
06-06 16:49:04,1,1,0.0,383.2,8,0,0.0,383.2,8.0,0.0,31.9,88.9,8,8,383.2,383.2
|
||||
06-06 16:49:09,1,1,66.8,416.1,9,0,66.8,416.1,9.0,0.0,34.3,89.0,9,9,416.1,416.1
|
||||
06-06 16:49:14,1,1,0.0,430.6,9,0,0.0,430.6,9.0,0.0,34.9,89.0,9,9,430.6,430.6
|
||||
06-06 16:49:19,1,1,40.5,456.1,10,0,40.5,456.1,10.0,0.0,36.3,89.0,10,10,456.1,456.1
|
||||
06-06 16:49:24,1,1,127.5,510.6,12,0,127.5,510.6,12.0,0.0,38.4,89.0,12,12,510.6,510.6
|
||||
06-06 16:49:29,1,1,205.2,572.2,13,0,205.2,572.2,13.0,0.0,42.4,89.1,13,13,572.2,572.2
|
||||
06-06 16:49:34,1,1,0.0,553.4,10,0,0.0,553.4,10.0,0.0,38.0,89.1,10,10,553.4,553.4
|
||||
06-06 16:49:39,1,1,0.0,479.1,10,0,0.0,479.1,10.0,0.0,38.7,89.1,10,10,479.1,479.1
|
||||
06-06 16:49:44,1,1,0.0,424.5,8,0,0.0,424.5,8.0,0.0,36.9,89.1,8,8,424.5,424.5
|
||||
06-06 16:49:49,1,1,0.0,384.4,8,0,0.0,384.4,8.0,0.0,37.4,89.1,8,8,384.4,384.4
|
||||
06-06 16:49:54,1,1,163.1,321.5,7,0,163.1,321.5,7.0,0.0,26.9,89.9,7,7,321.5,321.5
|
||||
06-06 16:49:59,1,1,27.9,386.9,8,0,27.9,386.9,8.0,0.0,25.7,90.0,8,8,386.9,386.9
|
||||
06-06 16:50:04,1,1,0.0,380.3,7,0,0.0,380.3,7.0,0.0,26.1,90.0,7,7,380.3,380.3
|
||||
06-06 16:50:09,1,1,9.8,345.6,8,0,9.8,345.6,8.0,0.0,31.3,90.1,8,8,345.6,345.6
|
||||
06-06 16:50:14,1,1,223.5,652.5,15,0,223.5,652.5,15.0,0.0,59.0,88.6,15,15,652.5,652.5
|
||||
06-06 16:50:19,1,1,0.0,718.3,15,0,0.0,718.3,15.0,0.0,60.0,88.6,15,15,718.3,718.3
|
||||
06-06 16:50:24,1,1,0.0,720.0,15,0,0.0,720.0,15.0,0.0,61.0,88.6,15,15,720.0,720.0
|
||||
06-06 16:50:29,1,1,0.0,690.2,13,0,0.0,690.2,13.0,0.0,52.0,88.6,13,13,690.2,690.2
|
||||
06-06 16:50:34,1,1,0.0,580.7,12,0,0.0,580.7,12.0,0.0,48.8,88.6,12,12,580.7,580.7
|
||||
06-06 16:50:39,1,1,0.0,576.4,12,0,0.0,576.4,12.0,0.0,49.6,88.6,12,12,576.4,576.4
|
||||
06-06 16:50:44,1,1,9.4,603.5,13,0,9.4,603.5,13.0,0.0,50.6,88.7,13,13,603.5,603.5
|
||||
06-06 16:50:49,1,1,0.0,600.5,12,0,0.0,600.5,12.0,0.0,51.2,88.7,12,12,600.5,600.5
|
||||
06-06 16:50:54,1,1,105.9,591.6,13,0,105.9,591.6,13.0,0.0,53.2,88.7,13,13,591.6,591.6
|
||||
06-06 16:50:59,1,1,0.0,622.6,13,0,0.0,622.6,13.0,0.0,54.1,88.7,13,13,622.6,622.6
|
||||
06-06 16:51:04,1,1,8.0,629.2,14,0,8.0,629.2,14.0,0.0,55.0,88.7,14,14,629.2,629.2
|
||||
06-06 16:51:09,1,1,35.1,701.9,15,0,35.1,701.9,15.0,0.0,56.2,88.7,15,15,701.9,701.9
|
||||
06-06 16:51:14,1,1,0.0,718.3,15,0,0.0,718.3,15.0,0.0,57.2,88.7,15,15,718.3,718.3
|
||||
06-06 16:51:19,1,1,78.3,694.8,15,0,78.3,694.8,15.0,0.0,57.9,88.6,15,15,694.8,694.8
|
||||
06-06 16:51:24,1,1,0.0,717.5,15,0,0.0,717.5,15.0,0.0,58.9,88.6,15,15,717.5,717.5
|
||||
06-06 16:51:29,1,1,75.7,647.4,14,0,75.7,647.4,14.0,0.0,58.5,88.6,14,14,647.4,647.4
|
||||
06-06 16:51:34,1,1,0.0,646.8,13,0,0.0,646.8,13.0,0.0,59.0,88.6,13,13,646.8,646.8
|
||||
06-06 16:51:39,1,1,365.4,647.7,14,0,365.4,647.7,14.0,0.0,65.4,88.8,14,14,647.7,647.7
|
||||
06-06 16:51:44,1,1,0.0,362.2,8,0,0.0,362.2,8.0,0.0,38.2,90.4,8,8,362.2,362.2
|
||||
06-06 16:51:50,1,1,69.3,354.9,8,0,69.3,354.9,8.0,0.0,46.3,91.1,8,8,354.9,354.9
|
||||
06-06 16:51:55,1,1,0.0,541.5,11,0,0.0,541.5,11.0,0.0,51.1,90.3,11,11,541.5,541.5
|
||||
06-06 16:52:00,1,1,43.7,553.6,12,0,43.7,553.6,12.0,0.0,52.1,90.3,12,12,553.6,553.6
|
||||
06-06 16:52:05,1,1,0.0,562.1,12,0,0.0,562.1,12.0,0.0,52.9,90.3,12,12,562.1,562.1
|
||||
06-06 16:52:10,1,1,84.1,564.7,13,0,84.1,564.7,13.0,0.0,48.6,90.4,13,13,564.7,564.7
|
||||
06-06 16:52:15,1,1,125.0,551.2,12,0,125.0,551.2,12.0,0.0,51.0,90.4,12,12,551.2,551.2
|
||||
06-06 16:52:20,1,1,90.4,507.1,11,0,90.4,507.1,11.0,0.0,45.9,90.4,11,11,507.1,507.1
|
||||
06-06 16:52:25,1,1,11.6,551.9,12,0,11.6,551.9,12.0,0.0,46.7,90.4,12,12,551.9,551.9
|
||||
06-06 16:52:30,1,1,0.0,555.4,12,0,0.0,555.4,12.0,0.0,47.5,90.4,12,12,555.4,555.4
|
||||
06-06 16:52:35,1,1,0.0,556.8,12,0,0.0,556.8,12.0,0.0,48.2,90.4,12,12,556.8,556.8
|
||||
06-06 16:52:40,1,1,113.6,535.1,12,0,113.6,535.1,12.0,0.0,40.8,90.4,12,12,535.1,535.1
|
||||
06-06 16:52:45,1,1,21.8,567.1,13,0,21.8,567.1,13.0,0.0,45.1,90.5,13,13,567.1,567.1
|
||||
06-06 16:52:50,1,1,0.0,601.7,13,0,0.0,601.7,13.0,0.0,46.0,90.5,13,13,601.7,601.7
|
||||
06-06 16:52:55,1,1,143.2,568.6,12,0,143.2,568.6,12.0,0.0,34.6,90.5,12,12,568.6,568.6
|
||||
06-06 16:53:00,1,1,0.0,556.9,12,0,0.0,556.9,12.0,0.0,35.4,90.5,12,12,556.9,556.9
|
||||
06-06 16:53:05,1,1,0.0,556.8,12,0,0.0,556.8,12.0,0.0,36.1,90.5,12,12,556.8,556.8
|
||||
06-06 16:53:10,1,1,46.6,580.6,14,0,46.6,580.6,14.0,0.0,42.9,90.7,14,14,580.6,580.6
|
||||
06-06 16:53:15,1,1,0.0,619.5,13,0,0.0,619.5,13.0,0.0,43.1,90.7,13,13,619.5,619.5
|
||||
06-06 16:53:20,1,1,0.0,603.3,13,0,0.0,603.3,13.0,0.0,43.9,90.7,13,13,603.3,603.3
|
||||
06-06 16:53:25,1,1,0.0,540.2,11,0,0.0,540.2,11.0,0.0,38.4,90.7,11,11,540.2,540.2
|
||||
06-06 16:53:30,1,1,13.2,552.3,13,0,13.2,552.3,13.0,0.0,40.0,90.7,13,13,552.3,552.3
|
||||
06-06 16:53:35,1,1,0.0,581.2,12,0,0.0,581.2,12.0,0.0,39.9,90.7,12,12,581.2,581.2
|
||||
06-06 16:53:40,1,1,0.0,534.0,11,0,0.0,534.0,11.0,0.0,40.4,90.7,11,11,534.0,534.0
|
||||
06-06 16:53:45,1,1,10.6,541.4,12,0,10.6,541.4,12.0,0.0,41.3,90.7,12,12,541.4,541.4
|
||||
06-06 16:53:50,1,1,0.0,557.7,12,0,0.0,557.7,12.0,0.0,42.1,90.7,12,12,557.7,557.7
|
||||
06-06 16:53:55,1,1,0.0,555.6,12,0,0.0,555.6,12.0,0.0,42.9,90.7,12,12,555.6,555.6
|
||||
06-06 16:54:00,1,1,0.0,555.7,12,0,0.0,555.7,12.0,0.0,43.6,90.7,12,12,555.7,555.7
|
||||
06-06 16:54:05,1,1,0.0,558.4,12,0,0.0,558.4,12.0,0.0,44.4,90.7,12,12,558.4,558.4
|
||||
06-06 16:54:10,1,1,27.3,723.3,19,0,27.3,723.3,19.0,0.0,87.3,89.7,19,19,723.3,723.3
|
||||
06-06 16:54:15,1,1,0.0,558.2,12,0,0.0,558.2,12.0,0.0,46.2,90.7,12,12,558.2,558.2
|
||||
06-06 16:54:20,1,1,0.0,554.9,12,0,0.0,554.9,12.0,0.0,47.0,90.7,12,12,554.9,554.9
|
||||
06-06 16:54:25,1,1,0.0,676.0,17,0,0.0,676.0,17.0,0.0,80.5,89.6,17,17,676.0,676.0
|
||||
06-06 16:54:30,1,1,0.0,664.3,17,0,0.0,664.3,17.0,0.0,81.4,89.6,17,17,664.3,664.3
|
||||
06-06 16:54:35,1,1,0.0,642.3,13,0,0.0,642.3,13.0,0.0,50.1,90.7,13,13,642.3,642.3
|
||||
06-06 16:54:40,1,1,46.8,636.6,14,0,46.8,636.6,14.0,0.0,51.2,90.7,14,14,636.6,636.6
|
||||
06-06 16:54:45,1,1,220.5,677.7,15,0,220.5,677.7,15.0,0.0,52.6,90.6,15,15,677.7,677.7
|
||||
06-06 16:54:50,1,1,0.0,666.0,14,0,0.0,666.0,14.0,0.0,53.0,90.6,14,14,666.0,666.0
|
||||
06-06 16:54:55,1,1,0.0,647.2,14,0,0.0,647.2,14.0,0.0,53.9,90.6,14,14,647.2,647.2
|
||||
06-06 16:55:00,1,1,67.0,328.7,7,0,67.0,328.7,7.0,0.0,27.6,90.9,7,7,328.7,328.7
|
||||
06-06 16:55:05,1,1,0.0,602.0,13,0,0.0,602.0,13.0,0.0,55.1,90.6,13,13,602.0,602.0
|
||||
06-06 16:55:10,1,1,0.0,705.8,18,0,0.0,705.8,18.0,0.0,82.0,89.7,18,18,705.8,705.8
|
||||
06-06 16:55:15,1,1,0.8,736.4,19,0,0.8,736.4,19.0,0.0,83.1,89.7,19,19,736.4,736.4
|
||||
06-06 16:55:20,1,1,61.5,738.9,20,0,61.5,738.9,20.0,0.0,84.2,89.7,20,20,738.9,738.9
|
||||
06-06 16:55:25,1,1,0.0,748.3,19,0,0.0,748.3,19.0,0.0,83.9,89.7,19,19,748.3,748.3
|
||||
06-06 16:55:30,1,1,0.0,738.5,19,0,0.0,738.5,19.0,0.0,84.9,89.7,19,19,738.5,738.5
|
||||
06-06 16:55:35,1,1,0.0,738.2,19,0,0.0,738.2,19.0,0.0,85.9,89.7,19,19,738.2,738.2
|
||||
06-06 16:55:40,1,1,0.0,738.9,19,0,0.0,738.9,19.0,0.0,87.0,89.7,19,19,738.9,738.9
|
||||
06-06 16:55:45,1,1,21.3,757.4,19,0,21.3,757.4,19.0,0.0,89.8,89.7,19,19,757.4,757.4
|
||||
06-06 16:55:50,1,1,369.3,741.9,21,0,369.3,741.9,21.0,0.0,91.4,89.6,21,21,741.9,741.9
|
||||
06-06 16:55:55,1,1,0.0,779.6,20,0,0.0,779.6,20.0,0.0,90.2,89.6,20,20,779.6,779.6
|
||||
06-06 16:56:00,1,1,0.0,772.5,20,0,0.0,772.5,20.0,0.0,91.3,89.6,20,20,772.5,772.5
|
||||
06-06 16:56:05,1,1,0.0,741.0,19,0,0.0,741.0,19.0,0.0,91.7,89.6,19,19,741.0,741.0
|
||||
06-06 16:56:10,1,1,0.0,729.2,19,0,0.0,729.2,19.0,0.0,92.7,89.6,19,19,729.2,729.2
|
||||
06-06 16:56:15,1,1,0.0,724.7,19,0,0.0,724.7,19.0,0.0,93.7,89.6,19,19,724.7,724.7
|
||||
06-06 16:56:20,1,1,0.0,719.7,19,0,0.0,719.7,19.0,0.0,94.7,89.6,19,19,719.7,719.7
|
||||
06-06 16:56:25,1,1,0.0,714.5,19,0,0.0,714.5,19.0,0.0,95.7,89.6,19,19,714.5,714.5
|
||||
06-06 16:56:30,1,1,115.1,733.3,20,0,115.1,733.3,20.0,0.0,96.9,89.6,20,20,733.3,733.3
|
||||
06-06 16:56:35,1,1,0.0,734.2,19,0,0.0,734.2,19.0,0.0,89.0,89.6,19,19,734.2,734.2
|
||||
06-06 16:56:40,1,1,316.8,736.4,20,0,316.8,736.4,20.0,0.0,90.5,89.4,20,20,736.4,736.4
|
||||
06-06 16:56:46,1,1,574.1,759.7,21,0,574.1,759.7,21.0,0.0,92.5,89.2,21,21,759.7,759.7
|
||||
06-06 16:56:51,1,1,0.0,763.8,20,0,0.0,763.8,20.0,0.0,93.0,89.2,20,20,763.8,763.8
|
||||
06-06 16:56:56,1,1,0.0,752.6,20,0,0.0,752.6,20.0,0.0,94.1,89.2,20,20,752.6,752.6
|
||||
06-06 16:57:01,1,1,10.8,769.1,21,0,10.8,769.1,21.0,0.0,95.2,89.2,21,21,769.1,769.1
|
||||
06-06 16:57:06,1,1,0.0,777.2,21,0,0.0,777.2,21.0,0.0,96.2,89.2,21,21,777.2,777.2
|
||||
06-06 16:57:11,1,1,0.0,760.4,19,0,0.0,760.4,19.0,0.0,94.5,89.2,19,19,760.4,760.4
|
||||
06-06 16:57:16,1,1,0.0,705.7,19,0,0.0,705.7,19.0,0.0,95.5,89.2,19,19,705.7,705.7
|
||||
06-06 16:57:21,1,1,0.0,700.0,19,0,0.0,700.0,19.0,0.0,96.5,89.2,19,19,700.0,700.0
|
||||
06-06 16:57:26,1,1,3.2,725.4,20,0,3.2,725.4,20.0,0.0,97.5,89.3,20,20,725.4,725.4
|
||||
06-06 16:57:31,1,1,0.0,719.5,20,0,0.0,719.5,20.0,0.0,98.5,89.3,20,20,719.5,719.5
|
||||
06-06 16:57:36,1,1,0.0,719.8,19,0,0.0,719.8,19.0,0.0,90.7,89.3,19,19,719.8,719.8
|
||||
06-06 16:57:41,1,1,643.8,727.1,20,0,643.8,727.1,20.0,0.0,92.6,89.0,20,20,727.1,727.1
|
||||
06-06 16:57:46,1,1,453.8,758.9,21,0,453.8,758.9,21.0,0.0,94.3,88.8,21,21,758.9,758.9
|
||||
06-06 16:57:51,1,1,1650.8,738.8,21,0,1650.8,738.8,21.0,0.0,96.6,88.2,21,21,738.8,738.8
|
||||
06-06 16:57:56,1,1,0.0,759.1,21,0,0.0,759.1,21.0,0.0,97.7,88.2,21,21,759.1,759.1
|
||||
06-06 16:58:01,1,1,0.0,754.5,21,0,0.0,754.5,21.0,0.0,98.7,88.2,21,21,754.5,754.5
|
||||
06-06 16:58:06,1,1,0.0,748.7,21,0,0.0,748.7,21.0,0.0,99.8,88.2,21,21,748.7,748.7
|
||||
06-06 16:58:11,1,1,0.0,701.9,19,0,0.0,701.9,19.0,0.0,99.4,88.2,19,19,701.9,701.9
|
||||
06-06 16:58:16,1,1,0.0,678.4,18,1,0.0,678.4,18.0,1.0,97.8,88.2,18,18,678.4,678.4
|
||||
06-06 16:58:21,1,1,25.0,425.5,10,0,25.0,425.5,10.0,0.0,47.4,91.5,10,10,425.5,425.5
|
||||
06-06 16:58:26,1,1,0.0,453.7,10,0,0.0,453.7,10.0,0.0,48.0,91.5,10,10,453.7,453.7
|
||||
06-06 16:58:31,1,1,0.0,734.6,18,0,0.0,734.6,18.0,0.0,86.6,87.8,18,18,734.6,734.6
|
||||
06-06 16:58:36,1,1,0.0,694.0,18,0,0.0,694.0,18.0,0.0,87.5,87.8,18,18,694.0,694.0
|
||||
06-06 16:58:41,1,1,0.0,680.1,17,0,0.0,680.1,17.0,0.0,79.6,87.8,17,17,680.1,680.1
|
||||
06-06 16:58:46,1,1,0.0,680.9,16,0,0.0,680.9,16.0,0.0,79.2,87.8,16,16,680.9,680.9
|
||||
06-06 16:58:51,1,1,0.0,672.1,16,0,0.0,672.1,16.0,0.0,80.2,87.8,16,16,672.1,672.1
|
||||
06-06 16:58:56,1,1,0.0,667.5,15,0,0.0,667.5,15.0,0.0,72.2,87.8,15,15,667.5,667.5
|
||||
06-06 16:59:01,1,1,0.0,667.7,15,0,0.0,667.7,15.0,0.0,73.1,87.8,15,15,667.7,667.7
|
||||
06-06 16:59:06,1,1,657.2,698.1,16,0,657.2,698.1,16.0,0.0,75.0,87.6,16,16,698.1,698.1
|
||||
06-06 16:59:11,1,1,0.0,689.6,15,0,0.0,689.6,15.0,0.0,74.9,87.6,15,15,689.6,689.6
|
||||
06-06 16:59:16,1,1,46.8,672.8,16,0,46.8,672.8,16.0,0.0,76.1,87.6,16,16,672.8,672.8
|
||||
06-06 16:59:21,1,1,0.0,715.7,16,0,0.0,715.7,16.0,0.0,77.1,87.6,16,16,715.7,715.7
|
||||
06-06 16:59:26,1,1,0.0,715.7,16,0,0.0,715.7,16.0,0.0,78.1,87.6,16,16,715.7,715.7
|
||||
06-06 16:59:31,1,1,0.0,696.8,15,0,0.0,696.8,15.0,0.0,78.7,87.6,15,15,696.8,696.8
|
||||
06-06 16:59:36,1,1,0.0,672.0,15,0,0.0,672.0,15.0,0.0,79.6,87.6,15,15,672.0,672.0
|
||||
06-06 16:59:41,1,1,67.3,314.0,7,0,67.3,314.0,7.0,0.0,25.3,91.5,7,7,314.0,314.0
|
||||
06-06 16:59:46,1,1,0.0,305.8,6,0,0.0,305.8,6.0,0.0,22.6,91.5,6,6,305.8,305.8
|
||||
06-06 16:59:51,1,1,0.0,670.2,15,0,0.0,670.2,15.0,0.0,75.2,87.5,15,15,670.2,670.2
|
||||
06-06 16:59:56,1,1,0.0,669.6,15,0,0.0,669.6,15.0,0.0,76.1,87.5,15,15,669.6,669.6
|
||||
06-06 17:00:01,1,1,22.6,690.4,16,0,22.6,690.4,16.0,0.0,78.1,87.6,16,16,690.4,690.4
|
||||
06-06 17:00:06,1,1,81.1,683.2,17,0,81.1,683.2,17.0,0.0,79.6,87.6,17,17,683.2,683.2
|
||||
06-06 17:00:11,1,1,0.0,656.3,14,0,0.0,656.3,14.0,0.0,76.8,87.6,14,14,656.3,656.3
|
||||
06-06 17:00:16,1,1,98.9,617.8,15,0,98.9,617.8,15.0,0.0,77.8,87.5,15,15,617.8,617.8
|
||||
06-06 17:00:21,1,1,0.0,647.5,15,0,0.0,647.5,15.0,0.0,78.7,87.5,15,15,647.5,647.5
|
||||
06-06 17:00:26,1,1,0.0,397.9,9,0,0.0,397.9,9.0,0.0,30.7,90.2,9,9,397.9,397.9
|
||||
06-06 17:00:31,1,1,11.0,410.2,10,0,11.0,410.2,10.0,0.0,31.3,90.2,10,10,410.2,410.2
|
||||
06-06 17:00:36,1,1,10.2,481.6,11,0,10.2,481.6,11.0,0.0,32.0,90.2,11,11,481.6,481.6
|
||||
06-06 17:00:41,1,1,0.0,461.6,10,0,0.0,461.6,10.0,0.0,31.0,90.2,10,10,461.6,461.6
|
||||
06-06 17:00:46,1,1,30.1,493.1,11,0,30.1,493.1,11.0,0.0,53.2,91.2,11,11,493.1,493.1
|
||||
06-06 17:00:51,1,1,13.0,489.1,11,0,13.0,489.1,11.0,0.0,51.2,91.3,11,11,489.1,489.1
|
||||
06-06 17:00:56,1,1,0.0,408.6,7,0,0.0,408.6,7.0,0.0,26.7,90.0,7,7,408.6,408.6
|
||||
06-06 17:01:01,1,1,58.3,306.5,7,0,58.3,306.5,7.0,0.0,26.5,90.0,7,7,306.5,306.5
|
||||
06-06 17:01:06,1,1,92.6,314.5,7,0,92.6,314.5,7.0,0.0,27.7,89.9,7,7,314.5,314.5
|
||||
06-06 17:01:12,1,1,76.7,338.8,7,0,76.7,338.8,7.0,0.0,28.3,90.0,7,7,338.8,338.8
|
||||
06-06 17:01:17,1,1,86.2,367.2,8,0,86.2,367.2,8.0,0.0,29.7,89.9,8,8,367.2,367.2
|
||||
06-06 17:01:22,1,1,33.2,386.7,8,0,33.2,386.7,8.0,0.0,29.5,89.9,8,8,386.7,386.7
|
||||
06-06 17:01:27,1,1,0.0,656.3,14,0,0.0,656.3,14.0,0.0,63.9,89.7,14,14,656.3,656.3
|
||||
06-06 17:01:32,1,1,212.3,626.2,15,0,212.3,626.2,15.0,0.0,65.2,89.6,15,15,626.2,626.2
|
||||
06-06 17:01:37,1,1,0.0,632.4,13,0,0.0,632.4,13.0,0.0,64.9,89.6,13,13,632.4,632.4
|
||||
06-06 17:01:42,1,1,0.0,579.8,13,0,0.0,579.8,13.0,0.0,65.7,89.6,13,13,579.8,579.8
|
||||
06-06 17:01:47,1,1,0.0,578.6,13,0,0.0,578.6,13.0,0.0,66.5,89.6,13,13,578.6,578.6
|
||||
06-06 17:01:52,1,1,0.0,528.6,11,0,0.0,528.6,11.0,0.0,65.0,89.6,11,11,528.6,528.6
|
||||
06-06 17:01:57,1,1,24.2,525.6,12,0,24.2,525.6,12.0,0.0,66.3,89.6,12,12,525.6,525.6
|
||||
06-06 17:02:02,1,1,0.0,535.4,12,0,0.0,535.4,12.0,0.0,67.0,89.6,12,12,535.4,535.4
|
||||
06-06 17:02:07,1,1,0.0,535.9,12,0,0.0,535.9,12.0,0.0,67.8,89.6,12,12,535.9,535.9
|
||||
06-06 17:02:12,1,1,74.9,544.4,13,0,74.9,544.4,13.0,0.0,69.1,89.6,13,13,544.4,544.4
|
||||
06-06 17:02:17,1,1,0.0,581.6,13,0,0.0,581.6,13.0,0.0,70.0,89.6,13,13,581.6,581.6
|
||||
06-06 17:02:22,1,1,0.0,382.2,8,0,0.0,382.2,8.0,0.0,42.2,89.7,8,8,382.2,382.2
|
||||
06-06 17:02:27,1,1,0.0,382.5,8,0,0.0,382.5,8.0,0.0,42.7,89.7,8,8,382.5,382.5
|
||||
06-06 17:02:32,1,1,0.0,376.1,8,0,0.0,376.1,8.0,0.0,36.5,89.4,8,8,376.1,376.1
|
||||
06-06 17:02:37,1,1,0.0,365.2,8,0,0.0,365.2,8.0,0.0,37.0,89.4,8,8,365.2,365.2
|
||||
06-06 17:02:42,1,1,0.0,365.7,8,0,0.0,365.7,8.0,0.0,37.5,89.4,8,8,365.7,365.7
|
||||
06-06 17:02:47,1,1,0.0,366.1,8,0,0.0,366.1,8.0,0.0,38.0,89.4,8,8,366.1,366.1
|
||||
06-06 17:02:52,1,1,0.0,366.3,8,0,0.0,366.3,8.0,0.0,38.5,89.4,8,8,366.3,366.3
|
||||
06-06 17:02:57,1,1,0.0,366.7,8,0,0.0,366.7,8.0,0.0,39.0,89.4,8,8,366.7,366.7
|
||||
06-06 17:03:02,1,1,0.0,366.2,8,0,0.0,366.2,8.0,0.0,39.5,89.4,8,8,366.2,366.2
|
||||
06-06 17:03:07,1,1,0.0,381.5,8,0,0.0,381.5,8.0,0.0,46.9,89.7,8,8,381.5,381.5
|
||||
06-06 17:03:12,1,1,0.0,382.3,8,0,0.0,382.3,8.0,0.0,47.5,89.7,8,8,382.3,382.3
|
||||
06-06 17:03:17,1,1,0.0,382.7,8,0,0.0,382.7,8.0,0.0,48.0,89.7,8,8,382.7,382.7
|
||||
06-06 17:03:22,1,1,74.4,395.0,9,0,74.4,395.0,9.0,0.0,48.7,89.7,9,9,395.0,395.0
|
||||
06-06 17:03:27,1,1,0.0,421.0,8,0,0.0,421.0,8.0,0.0,49.1,89.7,8,8,421.0,421.0
|
||||
06-06 17:03:32,1,1,0.0,438.9,10,0,0.0,438.9,10.0,0.0,44.2,90.4,10,10,438.9,438.9
|
||||
06-06 17:03:37,1,1,0.0,439.1,10,0,0.0,439.1,10.0,0.0,44.8,90.4,10,10,439.1,439.1
|
||||
06-06 17:03:42,1,1,237.3,459.4,11,0,237.3,459.4,11.0,0.0,49.2,90.4,11,11,459.4,459.4
|
||||
06-06 17:03:47,1,1,0.0,439.4,10,0,0.0,439.4,10.0,0.0,41.0,90.4,10,10,439.4,439.4
|
||||
06-06 17:03:52,1,1,0.0,439.1,10,0,0.0,439.1,10.0,0.0,41.6,90.4,10,10,439.1,439.1
|
||||
06-06 17:03:57,1,1,0.0,438.3,10,0,0.0,438.3,10.0,0.0,42.2,90.4,10,10,438.3,438.3
|
||||
06-06 17:04:02,1,1,45.0,446.6,11,0,45.0,446.6,11.0,0.0,43.6,90.4,11,11,446.6,446.6
|
||||
06-06 17:04:07,1,1,29.9,484.6,12,0,29.9,484.6,12.0,0.0,46.0,90.5,12,12,484.6,484.6
|
||||
06-06 17:04:12,1,1,0.0,493.7,11,0,0.0,493.7,11.0,0.0,45.8,90.5,11,11,493.7,493.7
|
||||
06-06 17:04:17,1,1,0.0,456.2,10,0,0.0,456.2,10.0,0.0,44.6,90.5,10,10,456.2,456.2
|
||||
06-06 17:04:22,1,1,0.0,443.3,10,0,0.0,443.3,10.0,0.0,45.3,90.5,10,10,443.3,443.3
|
||||
06-06 17:04:27,1,1,0.0,444.9,10,0,0.0,444.9,10.0,0.0,45.9,90.5,10,10,444.9,444.9
|
||||
06-06 17:04:32,1,1,0.0,558.5,12,0,0.0,558.5,12.0,0.0,53.4,86.6,12,12,558.5,558.5
|
||||
06-06 17:04:37,1,1,0.0,526.1,11,0,0.0,526.1,11.0,0.0,52.4,86.6,11,11,526.1,526.1
|
||||
06-06 17:04:42,1,1,0.0,466.6,10,0,0.0,466.6,10.0,0.0,50.4,86.6,10,10,466.6,466.6
|
||||
06-06 17:04:47,1,1,0.0,464.7,10,0,0.0,464.7,10.0,0.0,51.0,86.6,10,10,464.7,464.7
|
||||
06-06 17:04:53,1,1,0.0,462.9,10,0,0.0,462.9,10.0,0.0,37.1,88.5,10,10,462.9,462.9
|
||||
06-06 17:04:58,1,1,0.0,459.0,10,0,0.0,459.0,10.0,0.0,37.7,88.5,10,10,459.0,459.0
|
||||
06-06 17:05:03,1,1,0.0,457.3,10,0,0.0,457.3,10.0,0.0,38.4,88.5,10,10,457.3,457.3
|
||||
06-06 17:05:08,1,1,0.0,457.3,10,0,0.0,457.3,10.0,0.0,39.0,88.5,10,10,457.3,457.3
|
||||
06-06 17:05:13,1,1,0.0,456.9,10,0,0.0,456.9,10.0,0.0,39.7,88.5,10,10,456.9,456.9
|
||||
06-06 17:05:18,1,1,0.0,449.6,9,0,0.0,449.6,9.0,0.0,42.9,89.9,9,9,449.6,449.6
|
||||
06-06 17:05:23,1,1,0.0,420.8,9,0,0.0,420.8,9.0,0.0,43.4,89.9,9,9,420.8,420.8
|
||||
06-06 17:05:28,1,1,0.0,417.6,9,0,0.0,417.6,9.0,0.0,44.0,89.9,9,9,417.6,417.6
|
||||
06-06 17:05:33,1,1,0.0,374.5,8,0,0.0,374.5,8.0,0.0,35.6,89.9,8,8,374.5,374.5
|
||||
06-06 17:05:38,1,1,101.3,421.8,9,0,101.3,421.8,9.0,0.0,37.0,89.9,9,9,421.8,421.8
|
||||
06-06 17:05:43,1,1,34.6,458.9,10,0,34.6,458.9,10.0,0.0,37.9,89.9,10,10,458.9,458.9
|
||||
06-06 17:05:48,1,1,133.5,450.7,10,0,133.5,450.7,10.0,0.0,38.2,89.8,10,10,450.7,450.7
|
||||
06-06 17:05:53,1,1,120.1,490.5,10,0,120.1,490.5,10.0,0.0,37.1,89.8,10,10,490.5,490.5
|
||||
06-06 17:05:58,1,1,2.0,502.8,11,0,2.0,502.8,11.0,0.0,38.1,89.8,11,11,502.8,502.8
|
||||
06-06 17:06:02,1,1,0.0,413.8,9,0,0.0,413.8,9.0,0.0,39.6,89.7,9,9,413.8,413.8
|
||||
|
485
training_logs/20260608_210110_vllm_metrics_job_632524.csv
Normal file
485
training_logs/20260608_210110_vllm_metrics_job_632524.csv
Normal file
@@ -0,0 +1,485 @@
|
||||
datetime_str,n_engines_reporting,unique_ips,total_prompt_throughput_tokens_per_sec,total_generation_throughput_tokens_per_sec,total_running_requests,total_waiting_requests,avg_prompt_throughput_per_engine,avg_generation_throughput_per_engine,avg_running_requests_per_engine,avg_waiting_requests_per_engine,avg_gpu_kv_cache_usage_pct,avg_prefix_cache_hit_rate_pct,min_running_requests,max_running_requests,min_generation_throughput,max_generation_throughput
|
||||
06-06 17:13:54,1,1,20.9,0.0,1,0,20.9,0.0,1.0,0.0,0.3,0.0,1,1,0.0,0.0
|
||||
06-06 17:13:59,1,1,0.0,47.4,1,0,0.0,47.4,1.0,0.0,0.4,0.0,1,1,47.4,47.4
|
||||
06-06 17:14:04,1,1,137.5,106.6,3,0,137.5,106.6,3.0,0.0,0.9,54.5,3,3,106.6,106.6
|
||||
06-06 17:14:09,1,1,0.0,131.0,2,0,0.0,131.0,2.0,0.0,0.8,54.5,2,2,131.0,131.0
|
||||
06-06 17:14:14,1,1,34.7,139.4,3,0,34.7,139.4,3.0,0.0,1.3,63.0,3,3,139.4,139.4
|
||||
06-06 17:14:19,1,1,0.0,137.7,2,0,0.0,137.7,2.0,0.0,1.1,63.0,2,2,137.7,137.7
|
||||
06-06 17:14:24,1,1,138.3,111.5,3,0,138.3,111.5,3.0,0.0,1.8,72.6,3,3,111.5,111.5
|
||||
06-06 17:14:29,1,1,29.8,120.7,2,0,29.8,120.7,2.0,0.0,1.4,77.0,2,2,120.7,120.7
|
||||
06-06 17:14:34,1,1,151.7,134.4,3,0,151.7,134.4,3.0,0.0,2.4,77.4,3,3,134.4,134.4
|
||||
06-06 17:14:39,1,1,148.3,119.9,2,0,148.3,119.9,2.0,0.0,2.0,78.8,2,2,119.9,119.9
|
||||
06-06 17:14:44,1,1,87.2,127.4,3,0,87.2,127.4,3.0,0.0,3.1,80.2,3,3,127.4,127.4
|
||||
06-06 17:14:49,1,1,72.2,95.8,2,0,72.2,95.8,2.0,0.0,2.4,82.4,2,2,95.8,95.8
|
||||
06-06 17:14:54,1,1,0.0,91.6,1,0,0.0,91.6,1.0,0.0,1.2,86.8,1,1,91.6,91.6
|
||||
06-06 17:14:59,1,1,17.9,1.5,1,0,17.9,1.5,1.0,0.0,0.3,78.6,1,1,1.5,1.5
|
||||
06-06 17:15:04,1,1,60.3,76.9,2,0,60.3,76.9,2.0,0.0,0.5,77.7,2,2,76.9,76.9
|
||||
06-06 17:15:09,1,1,42.4,70.3,2,0,42.4,70.3,2.0,0.0,0.7,78.8,2,2,70.3,70.3
|
||||
06-06 17:15:14,1,1,0.0,91.5,2,0,0.0,91.5,2.0,0.0,0.8,78.8,2,2,91.5,91.5
|
||||
06-06 17:15:19,1,1,72.0,100.8,3,0,72.0,100.8,3.0,0.0,1.0,77.8,3,3,100.8,100.8
|
||||
06-06 17:15:24,1,1,85.3,151.1,3,0,85.3,151.1,3.0,0.0,1.0,76.5,3,3,151.1,151.1
|
||||
06-06 17:15:29,1,1,95.9,175.8,4,0,95.9,175.8,4.0,0.0,1.5,75.9,4,4,175.8,175.8
|
||||
06-06 17:15:34,1,1,41.6,123.9,2,0,41.6,123.9,2.0,0.0,1.1,76.9,2,2,123.9,123.9
|
||||
06-06 17:15:39,1,1,73.6,210.1,5,0,73.6,210.1,5.0,0.0,5.2,84.8,5,5,210.1,210.1
|
||||
06-06 17:15:44,1,1,84.6,221.9,5,0,84.6,221.9,5.0,0.0,5.4,84.5,5,5,221.9,221.9
|
||||
06-06 17:15:49,1,1,21.5,211.8,4,0,21.5,211.8,4.0,0.0,5.4,85.1,4,4,211.8,211.8
|
||||
06-06 17:15:54,1,1,45.2,236.2,5,0,45.2,236.2,5.0,0.0,6.1,85.3,5,5,236.2,236.2
|
||||
06-06 17:15:59,1,1,22.0,184.0,4,0,22.0,184.0,4.0,0.0,5.6,85.6,4,4,184.0,184.0
|
||||
06-06 17:16:04,1,1,0.0,172.3,3,0,0.0,172.3,3.0,0.0,5.3,85.6,3,3,172.3,172.3
|
||||
06-06 17:16:09,1,1,0.0,144.7,3,0,0.0,144.7,3.0,0.0,5.5,85.6,3,3,144.7,144.7
|
||||
06-06 17:16:14,1,1,0.0,145.2,3,0,0.0,145.2,3.0,0.0,5.7,85.6,3,3,145.2,145.2
|
||||
06-06 17:16:20,1,1,0.0,141.7,3,0,0.0,141.7,3.0,0.0,4.8,88.2,3,3,141.7,141.7
|
||||
06-06 17:16:25,1,1,0.0,142.0,3,0,0.0,142.0,3.0,0.0,5.0,88.2,3,3,142.0,142.0
|
||||
06-06 17:16:30,1,1,69.0,188.3,4,0,69.0,188.3,4.0,0.0,5.4,88.1,4,4,188.3,188.3
|
||||
06-06 17:16:35,1,1,0.0,189.8,4,0,0.0,189.8,4.0,0.0,5.7,88.1,4,4,189.8,189.8
|
||||
06-06 17:16:40,1,1,73.6,168.6,4,0,73.6,168.6,4.0,0.0,6.0,88.0,4,4,168.6,168.6
|
||||
06-06 17:16:45,1,1,0.0,190.3,4,0,0.0,190.3,4.0,0.0,6.3,88.0,4,4,190.3,190.3
|
||||
06-06 17:16:50,1,1,0.0,172.3,3,0,0.0,172.3,3.0,0.0,6.0,88.0,3,3,172.3,172.3
|
||||
06-06 17:16:55,1,1,42.7,187.4,4,0,42.7,187.4,4.0,0.0,6.8,88.0,4,4,187.4,187.4
|
||||
06-06 17:17:00,1,1,179.3,190.9,5,0,179.3,190.9,5.0,0.0,7.3,87.8,5,5,190.9,190.9
|
||||
06-06 17:17:05,1,1,0.0,237.0,5,0,0.0,237.0,5.0,0.0,7.7,87.8,5,5,237.0,237.0
|
||||
06-06 17:17:10,1,1,0.0,237.1,5,0,0.0,237.1,5.0,0.0,8.0,87.8,5,5,237.1,237.1
|
||||
06-06 17:17:15,1,1,95.2,232.5,6,0,95.2,232.5,6.0,0.0,8.5,87.9,6,6,232.5,232.5
|
||||
06-06 17:17:20,1,1,0.0,284.1,6,0,0.0,284.1,6.0,0.0,8.9,87.9,6,6,284.1,284.1
|
||||
06-06 17:17:25,1,1,0.0,257.9,5,0,0.0,257.9,5.0,0.0,8.2,87.9,5,5,257.9,257.9
|
||||
06-06 17:17:30,1,1,0.0,237.6,5,0,0.0,237.6,5.0,0.0,8.5,87.9,5,5,237.6,237.6
|
||||
06-06 17:17:35,1,1,147.5,161.5,4,0,147.5,161.5,4.0,0.0,5.3,81.2,4,4,161.5,161.5
|
||||
06-06 17:17:40,1,1,0.0,189.1,4,0,0.0,189.1,4.0,0.0,5.6,81.2,4,4,189.1,189.1
|
||||
06-06 17:17:45,1,1,0.0,171.3,3,0,0.0,171.3,3.0,0.0,5.1,81.2,3,3,171.3,171.3
|
||||
06-06 17:17:50,1,1,85.3,340.1,7,0,85.3,340.1,7.0,0.0,14.9,86.6,7,7,340.1,340.1
|
||||
06-06 17:17:55,1,1,0.0,289.5,6,0,0.0,289.5,6.0,0.0,14.5,86.6,6,6,289.5,289.5
|
||||
06-06 17:18:00,1,1,47.4,309.7,7,0,47.4,309.7,7.0,0.0,15.8,86.8,7,7,309.7,309.7
|
||||
06-06 17:18:05,1,1,0.0,314.1,7,0,0.0,314.1,7.0,0.0,16.3,86.8,7,7,314.1,314.1
|
||||
06-06 17:18:10,1,1,0.0,300.3,6,0,0.0,300.3,6.0,0.0,13.0,86.8,6,6,300.3,300.3
|
||||
06-06 17:18:15,1,1,187.0,288.9,7,0,187.0,288.9,7.0,0.0,17.3,87.3,7,7,288.9,288.9
|
||||
06-06 17:18:20,1,1,67.1,324.9,7,0,67.1,324.9,7.0,0.0,12.9,87.7,7,7,324.9,324.9
|
||||
06-06 17:18:25,1,1,0.0,311.4,7,0,0.0,311.4,7.0,0.0,18.2,87.3,7,7,311.4,311.4
|
||||
06-06 17:18:30,1,1,0.0,310.9,7,0,0.0,310.9,7.0,0.0,18.7,87.3,7,7,310.9,310.9
|
||||
06-06 17:18:35,1,1,0.0,310.8,7,0,0.0,310.8,7.0,0.0,19.1,87.3,7,7,310.8,310.8
|
||||
06-06 17:18:40,1,1,33.2,216.3,5,0,33.2,216.3,5.0,0.0,7.7,84.1,5,5,216.3,216.3
|
||||
06-06 17:18:45,1,1,0.0,311.7,7,0,0.0,311.7,7.0,0.0,20.1,88.0,7,7,311.7,311.7
|
||||
06-06 17:18:50,1,1,142.2,283.0,7,0,142.2,283.0,7.0,0.0,20.7,88.6,7,7,283.0,283.0
|
||||
06-06 17:18:55,1,1,0.0,311.4,7,0,0.0,311.4,7.0,0.0,21.2,88.6,7,7,311.4,311.4
|
||||
06-06 17:19:00,1,1,0.0,278.6,6,0,0.0,278.6,6.0,0.0,16.8,88.6,6,6,278.6,278.6
|
||||
06-06 17:19:05,1,1,0.0,268.1,6,0,0.0,268.1,6.0,0.0,17.1,88.6,6,6,268.1,268.1
|
||||
06-06 17:19:10,1,1,0.0,268.5,6,0,0.0,268.5,6.0,0.0,17.5,88.6,6,6,268.5,268.5
|
||||
06-06 17:19:15,1,1,0.0,177.9,4,0,0.0,177.9,4.0,0.0,13.4,90.1,4,4,177.9,177.9
|
||||
06-06 17:19:20,1,1,0.0,178.0,4,0,0.0,178.0,4.0,0.0,13.7,90.1,4,4,178.0,178.0
|
||||
06-06 17:19:25,1,1,0.0,177.5,4,0,0.0,177.5,4.0,0.0,13.9,90.1,4,4,177.5,177.5
|
||||
06-06 17:19:30,1,1,0.0,177.7,4,0,0.0,177.7,4.0,0.0,14.2,90.1,4,4,177.7,177.7
|
||||
06-06 17:19:35,1,1,0.0,178.3,4,0,0.0,178.3,4.0,0.0,14.4,90.1,4,4,178.3,178.3
|
||||
06-06 17:19:40,1,1,0.0,178.1,4,0,0.0,178.1,4.0,0.0,14.7,90.1,4,4,178.1,178.1
|
||||
06-06 17:19:45,1,1,0.0,177.8,4,0,0.0,177.8,4.0,0.0,14.9,90.1,4,4,177.8,177.8
|
||||
06-06 17:19:50,1,1,0.0,177.5,4,0,0.0,177.5,4.0,0.0,15.2,90.1,4,4,177.5,177.5
|
||||
06-06 17:19:55,1,1,0.0,177.8,4,0,0.0,177.8,4.0,0.0,15.4,90.1,4,4,177.8,177.8
|
||||
06-06 17:20:00,1,1,0.0,178.2,4,0,0.0,178.2,4.0,0.0,15.7,90.1,4,4,178.2,178.2
|
||||
06-06 17:20:05,1,1,0.0,177.9,4,0,0.0,177.9,4.0,0.0,15.9,90.1,4,4,177.9,177.9
|
||||
06-06 17:20:10,1,1,92.5,180.0,5,0,92.5,180.0,5.0,0.0,16.3,89.9,5,5,180.0,180.0
|
||||
06-06 17:20:15,1,1,88.9,241.0,5,0,88.9,241.0,5.0,0.0,11.9,89.7,5,5,241.0,241.0
|
||||
06-06 17:20:20,1,1,56.5,225.3,5,0,56.5,225.3,5.0,0.0,16.9,90.7,5,5,225.3,225.3
|
||||
06-06 17:20:25,1,1,0.0,347.0,7,0,0.0,347.0,7.0,0.0,23.5,90.1,7,7,347.0,347.0
|
||||
06-06 17:20:30,1,1,138.2,333.1,8,0,138.2,333.1,8.0,0.0,24.2,90.0,8,8,333.1,333.1
|
||||
06-06 17:20:35,1,1,19.2,348.6,8,0,19.2,348.6,8.0,0.0,24.7,90.7,8,8,348.6,348.6
|
||||
06-06 17:20:40,1,1,89.9,341.7,8,0,89.9,341.7,8.0,0.0,25.3,90.6,8,8,341.7,341.7
|
||||
06-06 17:20:45,1,1,0.0,336.7,7,0,0.0,336.7,7.0,0.0,25.4,90.6,7,7,336.7,336.7
|
||||
06-06 17:20:50,1,1,0.0,288.8,6,0,0.0,288.8,6.0,0.0,22.6,85.5,6,6,288.8,288.8
|
||||
06-06 17:20:56,1,1,83.8,267.6,6,0,83.8,267.6,6.0,0.0,22.6,85.3,6,6,267.6,267.6
|
||||
06-06 17:21:01,1,1,77.2,293.8,7,0,77.2,293.8,7.0,0.0,23.3,85.2,7,7,293.8,293.8
|
||||
06-06 17:21:06,1,1,70.1,285.8,6,0,70.1,285.8,6.0,0.0,23.4,85.2,6,6,285.8,285.8
|
||||
06-06 17:21:11,1,1,58.0,265.0,6,0,58.0,265.0,6.0,0.0,23.4,85.1,6,6,265.0,265.0
|
||||
06-06 17:21:16,1,1,123.8,361.2,8,0,123.8,361.2,8.0,0.0,24.6,85.0,8,8,361.2,361.2
|
||||
06-06 17:21:21,1,1,9.6,410.6,9,0,9.6,410.6,9.0,0.0,25.2,85.1,9,9,410.6,410.6
|
||||
06-06 17:21:26,1,1,0.0,429.5,9,0,0.0,429.5,9.0,0.0,25.8,85.1,9,9,429.5,429.5
|
||||
06-06 17:21:31,1,1,79.4,474.3,9,0,79.4,474.3,9.0,0.0,33.1,87.3,9,9,474.3,474.3
|
||||
06-06 17:21:36,1,1,112.0,431.3,10,0,112.0,431.3,10.0,0.0,33.8,87.2,10,10,431.3,431.3
|
||||
06-06 17:21:41,1,1,0.0,474.8,10,0,0.0,474.8,10.0,0.0,34.5,87.2,10,10,474.8,474.8
|
||||
06-06 17:21:46,1,1,0.0,475.1,10,0,0.0,475.1,10.0,0.0,35.2,87.2,10,10,475.1,475.1
|
||||
06-06 17:21:51,1,1,261.6,453.8,10,0,261.6,453.8,10.0,0.0,36.2,87.1,10,10,453.8,453.8
|
||||
06-06 17:21:56,1,1,0.0,451.3,9,0,0.0,451.3,9.0,0.0,34.6,87.1,9,9,451.3,451.3
|
||||
06-06 17:22:01,1,1,0.0,224.6,5,0,0.0,224.6,5.0,0.0,21.1,90.9,5,5,224.6,224.6
|
||||
06-06 17:22:06,1,1,91.9,227.6,6,0,91.9,227.6,6.0,0.0,22.1,90.8,6,6,227.6,227.6
|
||||
06-06 17:22:11,1,1,0.0,269.6,6,0,0.0,269.6,6.0,0.0,22.5,90.8,6,6,269.6,269.6
|
||||
06-06 17:22:16,1,1,0.0,269.6,6,0,0.0,269.6,6.0,0.0,22.9,90.8,6,6,269.6,269.6
|
||||
06-06 17:22:21,1,1,0.0,242.4,5,0,0.0,242.4,5.0,0.0,22.4,90.8,5,5,242.4,242.4
|
||||
06-06 17:22:26,1,1,0.0,224.4,5,0,0.0,224.4,5.0,0.0,22.7,90.8,5,5,224.4,224.4
|
||||
06-06 17:22:31,1,1,0.0,224.6,5,0,0.0,224.6,5.0,0.0,23.0,90.8,5,5,224.6,224.6
|
||||
06-06 17:22:36,1,1,57.5,261.4,6,0,57.5,261.4,6.0,0.0,24.1,90.8,6,6,261.4,261.4
|
||||
06-06 17:22:41,1,1,0.0,269.8,6,0,0.0,269.8,6.0,0.0,24.5,90.8,6,6,269.8,269.8
|
||||
06-06 17:22:46,1,1,0.0,269.6,6,0,0.0,269.6,6.0,0.0,24.9,90.8,6,6,269.6,269.6
|
||||
06-06 17:22:51,1,1,0.0,380.2,7,0,0.0,380.2,7.0,0.0,30.7,91.1,7,7,380.2,380.2
|
||||
06-06 17:22:56,1,1,0.0,223.8,5,0,0.0,223.8,5.0,0.0,29.5,90.5,5,5,223.8,223.8
|
||||
06-06 17:23:01,1,1,0.0,348.0,7,0,0.0,348.0,7.0,0.0,31.7,91.1,7,7,348.0,348.0
|
||||
06-06 17:23:06,1,1,0.0,347.8,7,0,0.0,347.8,7.0,0.0,32.2,91.1,7,7,347.8,347.8
|
||||
06-06 17:23:11,1,1,0.0,347.3,7,0,0.0,347.3,7.0,0.0,32.7,91.1,7,7,347.3,347.3
|
||||
06-06 17:23:16,1,1,0.0,347.4,7,0,0.0,347.4,7.0,0.0,33.1,91.1,7,7,347.4,347.4
|
||||
06-06 17:23:21,1,1,0.0,347.2,7,0,0.0,347.2,7.0,0.0,33.6,91.1,7,7,347.2,347.2
|
||||
06-06 17:23:26,1,1,101.8,348.8,8,0,101.8,348.8,8.0,0.0,34.4,91.0,8,8,348.8,348.8
|
||||
06-06 17:23:31,1,1,0.0,389.2,7,0,0.0,389.2,7.0,0.0,34.6,91.0,7,7,389.2,389.2
|
||||
06-06 17:23:36,1,1,0.0,348.5,7,0,0.0,348.5,7.0,0.0,35.1,91.0,7,7,348.5,348.5
|
||||
06-06 17:23:41,1,1,81.7,374.1,8,0,81.7,374.1,8.0,0.0,35.7,90.9,8,8,374.1,374.1
|
||||
06-06 17:23:46,1,1,0.0,397.3,8,0,0.0,397.3,8.0,0.0,36.3,90.9,8,8,397.3,397.3
|
||||
06-06 17:23:51,1,1,0.0,396.5,8,0,0.0,396.5,8.0,0.0,36.8,90.9,8,8,396.5,396.5
|
||||
06-06 17:23:56,1,1,0.0,396.5,8,0,0.0,396.5,8.0,0.0,37.4,90.9,8,8,396.5,396.5
|
||||
06-06 17:24:01,1,1,0.0,396.9,8,0,0.0,396.9,8.0,0.0,37.9,90.9,8,8,396.9,396.9
|
||||
06-06 17:24:06,1,1,0.0,396.4,8,0,0.0,396.4,8.0,0.0,38.5,90.9,8,8,396.4,396.4
|
||||
06-06 17:24:11,1,1,50.9,569.7,13,0,50.9,569.7,13.0,0.0,56.3,86.9,13,13,569.7,569.7
|
||||
06-06 17:24:16,1,1,46.0,564.7,13,0,46.0,564.7,13.0,0.0,56.3,86.9,13,13,564.7,564.7
|
||||
06-06 17:24:21,1,1,0.0,563.7,13,0,0.0,563.7,13.0,0.0,57.1,86.9,13,13,563.7,563.7
|
||||
06-06 17:24:26,1,1,36.1,600.2,14,0,36.1,600.2,14.0,0.0,58.7,87.0,14,14,600.2,600.2
|
||||
06-06 17:24:31,1,1,0.0,270.3,6,0,0.0,270.3,6.0,0.0,32.9,91.2,6,6,270.3,270.3
|
||||
06-06 17:24:36,1,1,93.2,410.5,9,0,93.2,410.5,9.0,0.0,33.0,90.9,9,9,410.5,410.5
|
||||
06-06 17:24:41,1,1,0.0,444.6,9,0,0.0,444.6,9.0,0.0,33.6,90.9,9,9,444.6,444.6
|
||||
06-06 17:24:46,1,1,0.0,408.1,8,0,0.0,408.1,8.0,0.0,33.9,90.9,8,8,408.1,408.1
|
||||
06-06 17:24:51,1,1,0.0,358.1,7,0,0.0,358.1,7.0,0.0,33.9,90.9,7,7,358.1,358.1
|
||||
06-06 17:24:56,1,1,0.0,312.8,6,0,0.0,312.8,6.0,0.0,33.2,90.9,6,6,312.8,312.8
|
||||
06-06 17:25:01,1,1,168.4,365.7,8,0,168.4,365.7,8.0,0.0,34.6,90.8,8,8,365.7,365.7
|
||||
06-06 17:25:06,1,1,0.0,335.4,6,0,0.0,335.4,6.0,0.0,34.0,90.8,6,6,335.4,335.4
|
||||
06-06 17:25:11,1,1,104.0,372.8,9,0,104.0,372.8,9.0,0.0,35.7,90.9,9,9,372.8,372.8
|
||||
06-06 17:25:16,1,1,0.0,444.4,9,0,0.0,444.4,9.0,0.0,36.3,90.9,9,9,444.4,444.4
|
||||
06-06 17:25:21,1,1,0.0,402.2,6,0,0.0,402.2,6.0,0.0,35.3,90.9,6,6,402.2,402.2
|
||||
06-06 17:25:26,1,1,376.3,546.6,15,0,376.3,546.6,15.0,0.0,47.2,87.8,15,15,546.6,546.6
|
||||
06-06 17:25:31,1,1,0.0,645.1,14,0,0.0,645.1,14.0,0.0,47.5,87.8,14,14,645.1,645.1
|
||||
06-06 17:25:37,1,1,1.0,223.2,6,0,1.0,223.2,6.0,0.0,23.3,84.7,6,6,223.2,223.2
|
||||
06-06 17:25:42,1,1,0.0,260.3,6,0,0.0,260.3,6.0,0.0,23.7,84.7,6,6,260.3,260.3
|
||||
06-06 17:25:47,1,1,29.7,241.3,6,0,29.7,241.3,6.0,0.0,24.1,85.2,6,6,241.3,241.3
|
||||
06-06 17:25:52,1,1,63.5,267.0,7,0,63.5,267.0,7.0,0.0,25.3,85.4,7,7,267.0,267.0
|
||||
06-06 17:25:57,1,1,20.4,295.0,7,0,20.4,295.0,7.0,0.0,26.1,86.2,7,7,295.0,295.0
|
||||
06-06 17:26:02,1,1,0.0,303.9,7,0,0.0,303.9,7.0,0.0,26.6,86.2,7,7,303.9,303.9
|
||||
06-06 17:26:07,1,1,169.1,287.5,7,0,169.1,287.5,7.0,0.0,17.8,85.7,7,7,287.5,287.5
|
||||
06-06 17:26:12,1,1,221.0,303.7,8,0,221.0,303.7,8.0,0.0,19.2,85.6,8,8,303.7,303.7
|
||||
06-06 17:26:17,1,1,0.0,347.6,8,0,0.0,347.6,8.0,0.0,19.7,85.6,8,8,347.6,347.6
|
||||
06-06 17:26:22,1,1,0.0,348.3,8,0,0.0,348.3,8.0,0.0,20.2,85.6,8,8,348.3,348.3
|
||||
06-06 17:26:27,1,1,43.5,369.5,9,0,43.5,369.5,9.0,0.0,21.0,85.6,9,9,369.5,369.5
|
||||
06-06 17:26:32,1,1,0.0,355.8,8,0,0.0,355.8,8.0,0.0,19.7,85.6,8,8,355.8,355.8
|
||||
06-06 17:26:37,1,1,0.0,347.4,8,0,0.0,347.4,8.0,0.0,20.2,85.6,8,8,347.4,347.4
|
||||
06-06 17:26:42,1,1,0.0,334.8,7,0,0.0,334.8,7.0,0.0,20.2,85.6,7,7,334.8,334.8
|
||||
06-06 17:26:47,1,1,134.8,330.0,7,0,134.8,330.0,7.0,0.0,20.6,85.4,7,7,330.0,330.0
|
||||
06-06 17:26:52,1,1,25.9,316.6,8,0,25.9,316.6,8.0,0.0,21.8,85.7,8,8,316.6,316.6
|
||||
06-06 17:26:57,1,1,54.0,363.0,9,0,54.0,363.0,9.0,0.0,23.2,86.0,9,9,363.0,363.0
|
||||
06-06 17:27:02,1,1,0.0,390.5,9,0,0.0,390.5,9.0,0.0,23.7,86.0,9,9,390.5,390.5
|
||||
06-06 17:27:07,1,1,113.5,377.7,9,0,113.5,377.7,9.0,0.0,25.3,86.4,9,9,377.7,377.7
|
||||
06-06 17:27:12,1,1,0.0,391.1,9,0,0.0,391.1,9.0,0.0,25.9,86.4,9,9,391.1,391.1
|
||||
06-06 17:27:17,1,1,121.9,585.3,14,0,121.9,585.3,14.0,0.0,52.4,88.8,14,14,585.3,585.3
|
||||
06-06 17:27:22,1,1,0.0,347.7,8,0,0.0,347.7,8.0,0.0,24.8,86.4,8,8,347.7,347.7
|
||||
06-06 17:27:27,1,1,0.0,348.0,8,0,0.0,348.0,8.0,0.0,25.3,86.4,8,8,348.0,348.0
|
||||
06-06 17:27:32,1,1,22.2,370.5,9,0,22.2,370.5,9.0,0.0,26.7,86.8,9,9,370.5,370.5
|
||||
06-06 17:27:37,1,1,0.0,392.1,9,0,0.0,392.1,9.0,0.0,27.2,86.8,9,9,392.1,392.1
|
||||
06-06 17:27:42,1,1,0.0,415.5,9,0,0.0,415.5,9.0,0.0,41.7,89.6,9,9,415.5,415.5
|
||||
06-06 17:27:48,1,1,86.1,453.8,11,0,86.1,453.8,11.0,0.0,43.4,89.6,11,11,453.8,453.8
|
||||
06-06 17:27:53,1,1,103.9,482.3,11,0,103.9,482.3,11.0,0.0,44.7,89.6,11,11,482.3,482.3
|
||||
06-06 17:27:58,1,1,74.5,427.9,10,0,74.5,427.9,10.0,0.0,29.3,86.5,10,10,427.9,427.9
|
||||
06-06 17:28:03,1,1,0.0,275.6,6,0,0.0,275.6,6.0,0.0,20.5,93.1,6,6,275.6,275.6
|
||||
06-06 17:28:08,1,1,34.9,293.0,7,0,34.9,293.0,7.0,0.0,21.2,93.1,7,7,293.0,293.0
|
||||
06-06 17:28:13,1,1,62.8,297.5,8,0,62.8,297.5,8.0,0.0,22.3,93.1,8,8,297.5,297.5
|
||||
06-06 17:28:18,1,1,0.0,319.3,7,0,0.0,319.3,7.0,0.0,22.2,93.1,7,7,319.3,319.3
|
||||
06-06 17:28:23,1,1,10.0,277.7,6,0,10.0,277.7,6.0,0.0,21.7,93.2,6,6,277.7,277.7
|
||||
06-06 17:28:28,1,1,0.0,189.8,4,0,0.0,189.8,4.0,0.0,21.2,93.2,4,4,189.8,189.8
|
||||
06-06 17:28:33,1,1,100.4,207.4,5,0,100.4,207.4,5.0,0.0,21.9,93.1,5,5,207.4,207.4
|
||||
06-06 17:28:38,1,1,0.0,175.4,4,0,0.0,175.4,4.0,0.0,21.6,93.1,4,4,175.4,175.4
|
||||
06-06 17:28:43,1,1,0.0,444.8,10,0,0.0,444.8,10.0,0.0,42.8,87.2,10,10,444.8,444.8
|
||||
06-06 17:28:48,1,1,0.0,444.2,10,0,0.0,444.2,10.0,0.0,43.4,87.2,10,10,444.2,444.2
|
||||
06-06 17:28:53,1,1,107.6,479.4,11,0,107.6,479.4,11.0,0.0,45.0,87.2,11,11,479.4,479.4
|
||||
06-06 17:28:58,1,1,9.4,500.6,12,0,9.4,500.6,12.0,0.0,45.8,87.2,12,12,500.6,500.6
|
||||
06-06 17:29:03,1,1,0.0,532.8,12,0,0.0,532.8,12.0,0.0,46.6,87.2,12,12,532.8,532.8
|
||||
06-06 17:29:08,1,1,0.0,531.6,12,0,0.0,531.6,12.0,0.0,47.3,87.2,12,12,531.6,531.6
|
||||
06-06 17:29:13,1,1,0.0,361.0,7,0,0.0,361.0,7.0,0.0,37.9,88.1,7,7,361.0,361.0
|
||||
06-06 17:29:18,1,1,156.9,405.9,9,0,156.9,405.9,9.0,0.0,40.6,88.2,9,9,405.9,405.9
|
||||
06-06 17:29:23,1,1,9.2,426.8,9,0,9.2,426.8,9.0,0.0,32.5,88.2,9,9,426.8,426.8
|
||||
06-06 17:29:28,1,1,0.0,428.8,9,0,0.0,428.8,9.0,0.0,33.1,88.2,9,9,428.8,428.8
|
||||
06-06 17:29:33,1,1,0.0,390.0,8,0,0.0,390.0,8.0,0.0,32.0,88.2,8,8,390.0,390.0
|
||||
06-06 17:29:38,1,1,74.9,387.5,9,0,74.9,387.5,9.0,0.0,33.8,88.3,9,9,387.5,387.5
|
||||
06-06 17:29:43,1,1,53.2,438.1,9,0,53.2,438.1,9.0,0.0,36.2,88.5,9,9,438.1,438.1
|
||||
06-06 17:29:49,1,1,327.6,459.1,11,0,327.6,459.1,11.0,0.0,33.8,88.3,11,11,459.1,459.1
|
||||
06-06 17:29:54,1,1,0.0,460.7,10,0,0.0,460.7,10.0,0.0,32.2,88.3,10,10,460.7,460.7
|
||||
06-06 17:29:59,1,1,267.9,436.0,9,0,267.9,436.0,9.0,0.0,31.0,88.2,9,9,436.0,436.0
|
||||
06-06 17:30:04,1,1,39.4,426.1,10,0,39.4,426.1,10.0,0.0,32.8,88.3,10,10,426.1,426.1
|
||||
06-06 17:30:09,1,1,18.8,468.6,10,0,18.8,468.6,10.0,0.0,34.6,88.5,10,10,468.6,468.6
|
||||
06-06 17:30:14,1,1,20.9,480.4,11,0,20.9,480.4,11.0,0.0,35.9,88.6,11,11,480.4,480.4
|
||||
06-06 17:30:19,1,1,0.0,498.5,10,0,0.0,498.5,10.0,0.0,34.6,88.6,10,10,498.5,498.5
|
||||
06-06 17:30:24,1,1,120.3,570.5,13,0,120.3,570.5,13.0,0.0,58.6,89.4,13,13,570.5,570.5
|
||||
06-06 17:30:29,1,1,228.4,589.4,14,0,228.4,589.4,14.0,0.0,58.6,89.3,14,14,589.4,589.4
|
||||
06-06 17:30:34,1,1,0.0,619.0,14,0,0.0,619.0,14.0,0.0,59.5,89.3,14,14,619.0,619.0
|
||||
06-06 17:30:39,1,1,0.0,609.5,12,0,0.0,609.5,12.0,0.0,58.8,89.3,12,12,609.5,609.5
|
||||
06-06 17:30:44,1,1,0.0,520.0,11,0,0.0,520.0,11.0,0.0,59.1,89.3,11,11,520.0,520.0
|
||||
06-06 17:30:49,1,1,0.0,495.0,11,0,0.0,495.0,11.0,0.0,59.8,89.3,11,11,495.0,495.0
|
||||
06-06 17:30:54,1,1,50.4,578.9,14,0,50.4,578.9,14.0,0.0,60.9,89.3,14,14,578.9,578.9
|
||||
06-06 17:30:59,1,1,44.1,645.8,15,0,44.1,645.8,15.0,0.0,62.5,89.4,15,15,645.8,645.8
|
||||
06-06 17:31:04,1,1,0.0,672.7,15,0,0.0,672.7,15.0,0.0,63.4,89.4,15,15,672.7,672.7
|
||||
06-06 17:31:09,1,1,0.8,694.0,16,0,0.8,694.0,16.0,0.0,64.4,89.4,16,16,694.0,694.0
|
||||
06-06 17:31:14,1,1,10.2,707.2,16,0,10.2,707.2,16.0,0.0,64.9,89.5,16,16,707.2,707.2
|
||||
06-06 17:31:19,1,1,123.3,701.3,16,0,123.3,701.3,16.0,0.0,66.6,89.4,16,16,701.3,701.3
|
||||
06-06 17:31:24,1,1,0.0,717.1,16,0,0.0,717.1,16.0,0.0,67.6,89.4,16,16,717.1,717.1
|
||||
06-06 17:31:29,1,1,178.4,661.4,18,0,178.4,661.4,18.0,0.0,69.1,89.3,18,18,661.4,661.4
|
||||
06-06 17:31:34,1,1,0.0,688.0,17,0,0.0,688.0,17.0,0.0,69.5,89.3,17,17,688.0,688.0
|
||||
06-06 17:31:39,1,1,0.0,682.8,15,0,0.0,682.8,15.0,0.0,69.2,89.3,15,15,682.8,682.8
|
||||
06-06 17:31:44,1,1,246.1,685.3,16,0,246.1,685.3,16.0,0.0,70.5,89.1,16,16,685.3,685.3
|
||||
06-06 17:31:49,1,1,0.0,715.2,16,0,0.0,715.2,16.0,0.0,71.5,89.1,16,16,715.2,715.2
|
||||
06-06 17:31:54,1,1,0.0,650.7,14,0,0.0,650.7,14.0,0.0,55.2,89.1,14,14,650.7,650.7
|
||||
06-06 17:31:59,1,1,0.0,584.3,12,0,0.0,584.3,12.0,0.0,46.4,89.1,12,12,584.3,584.3
|
||||
06-06 17:32:04,1,1,79.9,315.6,8,0,79.9,315.6,8.0,0.0,41.8,90.5,8,8,315.6,315.6
|
||||
06-06 17:32:09,1,1,124.0,365.1,9,0,124.0,365.1,9.0,0.0,42.9,90.5,9,9,365.1,365.1
|
||||
06-06 17:32:14,1,1,129.6,414.1,11,0,129.6,414.1,11.0,0.0,44.1,90.4,11,11,414.1,414.1
|
||||
06-06 17:32:19,1,1,124.8,495.7,13,0,124.8,495.7,13.0,0.0,48.6,90.5,13,13,495.7,495.7
|
||||
06-06 17:32:24,1,1,0.0,696.6,16,0,0.0,696.6,16.0,0.0,53.5,88.8,16,16,696.6,696.6
|
||||
06-06 17:32:29,1,1,0.0,472.2,11,0,0.0,472.2,11.0,0.0,47.3,90.5,11,11,472.2,472.2
|
||||
06-06 17:32:34,1,1,34.2,478.5,12,0,34.2,478.5,12.0,0.0,46.0,90.6,12,12,478.5,478.5
|
||||
06-06 17:32:40,1,1,0.0,474.8,10,0,0.0,474.8,10.0,0.0,38.4,88.0,10,10,474.8,474.8
|
||||
06-06 17:32:45,1,1,0.0,453.7,10,0,0.0,453.7,10.0,0.0,39.0,88.0,10,10,453.7,453.7
|
||||
06-06 17:32:50,1,1,134.1,297.7,8,0,134.1,297.7,8.0,0.0,30.0,90.9,8,8,297.7,297.7
|
||||
06-06 17:32:55,1,1,0.0,327.0,7,0,0.0,327.0,7.0,0.0,28.5,90.9,7,7,327.0,327.0
|
||||
06-06 17:33:00,1,1,0.0,315.6,7,0,0.0,315.6,7.0,0.0,28.9,90.9,7,7,315.6,315.6
|
||||
06-06 17:33:05,1,1,21.0,526.7,11,0,21.0,526.7,11.0,0.0,43.6,88.2,11,11,526.7,526.7
|
||||
06-06 17:33:10,1,1,90.5,404.1,10,0,90.5,404.1,10.0,0.0,43.3,89.8,10,10,404.1,404.1
|
||||
06-06 17:33:15,1,1,103.2,473.5,11,0,103.2,473.5,11.0,0.0,44.4,89.7,11,11,473.5,473.5
|
||||
06-06 17:33:20,1,1,108.9,480.2,12,0,108.9,480.2,12.0,0.0,47.1,89.8,12,12,480.2,480.2
|
||||
06-06 17:33:25,1,1,38.7,502.0,12,0,38.7,502.0,12.0,0.0,49.9,89.9,12,12,502.0,502.0
|
||||
06-06 17:33:30,1,1,0.0,517.3,12,0,0.0,517.3,12.0,0.0,50.6,89.9,12,12,517.3,517.3
|
||||
06-06 17:33:35,1,1,0.0,508.3,11,0,0.0,508.3,11.0,0.0,50.6,89.9,11,11,508.3,508.3
|
||||
06-06 17:33:40,1,1,0.0,475.4,11,0,0.0,475.4,11.0,0.0,51.3,89.9,11,11,475.4,475.4
|
||||
06-06 17:33:45,1,1,118.5,492.6,11,0,118.5,492.6,11.0,0.0,49.7,89.9,11,11,492.6,492.6
|
||||
06-06 17:33:50,1,1,0.2,477.5,12,0,0.2,477.5,12.0,0.0,50.4,90.0,12,12,477.5,477.5
|
||||
06-06 17:33:55,1,1,0.0,518.6,12,0,0.0,518.6,12.0,0.0,51.1,90.0,12,12,518.6,518.6
|
||||
06-06 17:34:00,1,1,0.0,514.8,12,0,0.0,514.8,12.0,0.0,51.9,90.0,12,12,514.8,514.8
|
||||
06-06 17:34:05,1,1,0.0,519.1,12,0,0.0,519.1,12.0,0.0,52.6,90.0,12,12,519.1,519.1
|
||||
06-06 17:34:10,1,1,0.0,493.3,11,0,0.0,493.3,11.0,0.0,45.0,90.0,11,11,493.3,493.3
|
||||
06-06 17:34:15,1,1,61.3,504.7,12,0,61.3,504.7,12.0,0.0,45.7,89.9,12,12,504.7,504.7
|
||||
06-06 17:34:20,1,1,0.0,519.3,12,0,0.0,519.3,12.0,0.0,46.5,89.9,12,12,519.3,519.3
|
||||
06-06 17:34:25,1,1,0.0,519.3,12,0,0.0,519.3,12.0,0.0,47.2,89.9,12,12,519.3,519.3
|
||||
06-06 17:34:30,1,1,0.0,518.6,12,0,0.0,518.6,12.0,0.0,47.9,89.9,12,12,518.6,518.6
|
||||
06-06 17:34:35,1,1,0.0,519.2,12,0,0.0,519.2,12.0,0.0,48.6,89.9,12,12,519.2,519.2
|
||||
06-06 17:34:40,1,1,0.0,519.0,12,0,0.0,519.0,12.0,0.0,49.4,89.9,12,12,519.0,519.0
|
||||
06-06 17:34:46,1,1,0.0,627.9,13,0,0.0,627.9,13.0,0.0,63.0,87.5,13,13,627.9,627.9
|
||||
06-06 17:34:51,1,1,9.0,468.8,11,0,9.0,468.8,11.0,0.0,45.2,90.7,11,11,468.8,468.8
|
||||
06-06 17:34:56,1,1,0.0,506.2,11,0,0.0,506.2,11.0,0.0,45.9,90.7,11,11,506.2,506.2
|
||||
06-06 17:35:01,1,1,88.8,533.9,11,0,88.8,533.9,11.0,0.0,46.6,90.6,11,11,533.9,533.9
|
||||
06-06 17:35:06,1,1,0.0,537.2,13,0,0.0,537.2,13.0,0.0,52.1,89.3,13,13,537.2,537.2
|
||||
06-06 17:35:11,1,1,0.0,534.0,12,0,0.0,534.0,12.0,0.0,50.9,89.3,12,12,534.0,534.0
|
||||
06-06 17:35:16,1,1,85.8,528.6,13,0,85.8,528.6,13.0,0.0,52.3,89.3,13,13,528.6,528.6
|
||||
06-06 17:35:21,1,1,231.5,552.9,15,0,231.5,552.9,15.0,0.0,54.2,89.2,15,15,552.9,552.9
|
||||
06-06 17:35:26,1,1,0.0,618.5,15,0,0.0,618.5,15.0,0.0,55.1,89.2,15,15,618.5,618.5
|
||||
06-06 17:35:31,1,1,201.9,631.6,16,0,201.9,631.6,16.0,0.0,59.2,89.2,16,16,631.6,631.6
|
||||
06-06 17:35:36,1,1,0.0,634.2,15,0,0.0,634.2,15.0,0.0,59.4,89.2,15,15,634.2,634.2
|
||||
06-06 17:35:41,1,1,0.0,617.5,15,0,0.0,617.5,15.0,0.0,60.3,89.2,15,15,617.5,617.5
|
||||
06-06 17:35:46,1,1,11.0,655.7,16,0,11.0,655.7,16.0,0.0,61.3,89.2,16,16,655.7,655.7
|
||||
06-06 17:35:51,1,1,0.0,609.5,14,0,0.0,609.5,14.0,0.0,61.7,89.2,14,14,609.5,609.5
|
||||
06-06 17:35:56,1,1,0.0,577.0,14,0,0.0,577.0,14.0,0.0,62.5,89.2,14,14,577.0,577.0
|
||||
06-06 17:36:01,1,1,0.0,578.0,14,0,0.0,578.0,14.0,0.0,63.3,89.2,14,14,578.0,578.0
|
||||
06-06 17:36:06,1,1,0.0,577.5,14,0,0.0,577.5,14.0,0.0,64.1,89.2,14,14,577.5,577.5
|
||||
06-06 17:36:11,1,1,0.0,576.2,14,0,0.0,576.2,14.0,0.0,64.9,89.2,14,14,576.2,576.2
|
||||
06-06 17:36:16,1,1,183.5,590.9,14,0,183.5,590.9,14.0,0.0,65.2,89.2,14,14,590.9,590.9
|
||||
06-06 17:36:21,1,1,0.0,546.0,13,0,0.0,546.0,13.0,0.0,65.0,89.2,13,13,546.0,546.0
|
||||
06-06 17:36:26,1,1,0.0,638.6,14,0,0.0,638.6,14.0,0.0,58.5,90.6,14,14,638.6,638.6
|
||||
06-06 17:36:31,1,1,21.7,641.3,13,0,21.7,641.3,13.0,0.0,61.5,88.5,13,13,641.3,641.3
|
||||
06-06 17:36:36,1,1,0.0,631.0,13,0,0.0,631.0,13.0,0.0,62.4,88.5,13,13,631.0,631.0
|
||||
06-06 17:36:41,1,1,0.0,627.3,13,0,0.0,627.3,13.0,0.0,62.4,88.6,13,13,627.3,627.3
|
||||
06-06 17:36:46,1,1,20.5,603.8,12,0,20.5,603.8,12.0,0.0,62.3,88.6,12,12,603.8,603.8
|
||||
06-06 17:36:51,1,1,70.6,606.9,13,0,70.6,606.9,13.0,0.0,55.5,88.6,13,13,606.9,606.9
|
||||
06-06 17:36:56,1,1,0.0,594.1,12,0,0.0,594.1,12.0,0.0,54.9,88.6,12,12,594.1,594.1
|
||||
06-06 17:37:01,1,1,0.0,554.8,11,0,0.0,554.8,11.0,0.0,54.3,88.6,11,11,554.8,554.8
|
||||
06-06 17:37:06,1,1,0.0,535.6,11,0,0.0,535.6,11.0,0.0,55.1,88.6,11,11,535.6,535.6
|
||||
06-06 17:37:11,1,1,0.0,535.0,11,0,0.0,535.0,11.0,0.0,55.8,88.6,11,11,535.0,535.0
|
||||
06-06 17:37:17,1,1,0.0,416.6,9,0,0.0,416.6,9.0,0.0,51.4,89.7,9,9,416.6,416.6
|
||||
06-06 17:37:22,1,1,155.9,423.7,10,0,155.9,423.7,10.0,0.0,53.6,89.7,10,10,423.7,423.7
|
||||
06-06 17:37:27,1,1,0.0,462.8,10,0,0.0,462.8,10.0,0.0,54.3,89.7,10,10,462.8,462.8
|
||||
06-06 17:37:32,1,1,10.6,469.6,11,0,10.6,469.6,11.0,0.0,55.0,89.7,11,11,469.6,469.6
|
||||
06-06 17:37:37,1,1,0.0,588.0,12,0,0.0,588.0,12.0,0.0,61.1,88.7,12,12,588.0,588.0
|
||||
06-06 17:37:42,1,1,0.0,555.2,11,0,0.0,555.2,11.0,0.0,60.3,88.7,11,11,555.2,555.2
|
||||
06-06 17:37:47,1,1,164.8,542.8,12,0,164.8,542.8,12.0,0.0,61.9,88.6,12,12,542.8,542.8
|
||||
06-06 17:37:52,1,1,10.2,911.8,21,0,10.2,911.8,21.0,0.0,77.7,87.7,21,21,911.8,911.8
|
||||
06-06 17:37:57,1,1,91.2,868.2,21,0,91.2,868.2,21.0,0.0,79.4,87.7,21,21,868.2,868.2
|
||||
06-06 17:38:02,1,1,11.4,832.8,21,0,11.4,832.8,21.0,0.0,80.5,87.8,21,21,832.8,832.8
|
||||
06-06 17:38:07,1,1,0.0,824.5,20,0,0.0,824.5,20.0,0.0,80.0,87.8,20,20,824.5,824.5
|
||||
06-06 17:38:12,1,1,93.2,822.8,20,0,93.2,822.8,20.0,0.0,83.3,87.9,20,20,822.8,822.8
|
||||
06-06 17:38:17,1,1,0.0,819.9,20,0,0.0,819.9,20.0,0.0,84.4,87.9,20,20,819.9,819.9
|
||||
06-06 17:38:22,1,1,0.0,323.7,7,0,0.0,323.7,7.0,0.0,33.3,90.0,7,7,323.7,323.7
|
||||
06-06 17:38:27,1,1,0.0,324.5,7,0,0.0,324.5,7.0,0.0,33.8,90.0,7,7,324.5,324.5
|
||||
06-06 17:38:32,1,1,0.0,602.9,13,0,0.0,602.9,13.0,0.0,57.6,90.1,13,13,602.9,602.9
|
||||
06-06 17:38:37,1,1,303.0,600.1,14,0,303.0,600.1,14.0,0.0,64.0,90.1,14,14,600.1,600.1
|
||||
06-06 17:38:42,1,1,92.2,668.7,15,0,92.2,668.7,15.0,0.0,67.5,90.2,15,15,668.7,668.7
|
||||
06-06 17:38:47,1,1,0.0,695.0,15,0,0.0,695.0,15.0,0.0,68.5,90.2,15,15,695.0,695.0
|
||||
06-06 17:38:52,1,1,45.7,685.0,15,0,45.7,685.0,15.0,0.0,69.0,90.2,15,15,685.0,685.0
|
||||
06-06 17:38:57,1,1,0.0,676.4,14,0,0.0,676.4,14.0,0.0,67.1,90.2,14,14,676.4,676.4
|
||||
06-06 17:39:02,1,1,0.0,651.1,14,0,0.0,651.1,14.0,0.0,68.1,90.2,14,14,651.1,651.1
|
||||
06-06 17:39:07,1,1,50.9,671.7,14,0,50.9,671.7,14.0,0.0,68.9,90.2,14,14,671.7,671.7
|
||||
06-06 17:39:12,1,1,0.0,650.5,14,0,0.0,650.5,14.0,0.0,69.9,90.2,14,14,650.5,650.5
|
||||
06-06 17:39:17,1,1,0.0,643.9,13,0,0.0,643.9,13.0,0.0,70.2,90.2,13,13,643.9,643.9
|
||||
06-06 17:39:22,1,1,0.0,602.4,13,0,0.0,602.4,13.0,0.0,71.1,90.2,13,13,602.4,602.4
|
||||
06-06 17:39:27,1,1,0.0,412.2,9,0,0.0,412.2,9.0,0.0,40.5,90.0,9,9,412.2,412.2
|
||||
06-06 17:39:32,1,1,0.0,413.4,9,0,0.0,413.4,9.0,0.0,41.1,90.0,9,9,413.4,413.4
|
||||
06-06 17:39:37,1,1,0.0,412.6,9,0,0.0,412.6,9.0,0.0,41.7,90.0,9,9,412.6,412.6
|
||||
06-06 17:39:42,1,1,0.0,412.8,9,0,0.0,412.8,9.0,0.0,42.3,90.0,9,9,412.8,412.8
|
||||
06-06 17:39:47,1,1,0.0,549.2,11,0,0.0,549.2,11.0,0.0,53.4,88.2,11,11,549.2,549.2
|
||||
06-06 17:39:53,1,1,0.0,509.8,11,0,0.0,509.8,11.0,0.0,54.1,88.2,11,11,509.8,509.8
|
||||
06-06 17:39:58,1,1,172.4,509.7,12,0,172.4,509.7,12.0,0.0,58.3,88.3,12,12,509.7,509.7
|
||||
06-06 17:40:03,1,1,314.4,550.9,13,0,314.4,550.9,13.0,0.0,65.2,88.4,13,13,550.9,550.9
|
||||
06-06 17:40:08,1,1,0.0,597.9,13,0,0.0,597.9,13.0,0.0,66.0,88.4,13,13,597.9,597.9
|
||||
06-06 17:40:13,1,1,45.4,576.0,13,0,45.4,576.0,13.0,0.0,66.4,88.5,13,13,576.0,576.0
|
||||
06-06 17:40:18,1,1,0.4,620.8,14,0,0.4,620.8,14.0,0.0,67.3,88.5,14,14,620.8,620.8
|
||||
06-06 17:40:23,1,1,376.3,618.6,14,0,376.3,618.6,14.0,0.0,70.0,88.4,14,14,618.6,618.6
|
||||
06-06 17:40:28,1,1,0.0,643.4,13,0,0.0,643.4,13.0,0.0,62.2,88.4,13,13,643.4,643.4
|
||||
06-06 17:40:33,1,1,0.0,598.2,13,0,0.0,598.2,13.0,0.0,63.0,88.4,13,13,598.2,598.2
|
||||
06-06 17:40:38,1,1,0.0,599.3,13,0,0.0,599.3,13.0,0.0,63.9,88.4,13,13,599.3,599.3
|
||||
06-06 17:40:43,1,1,0.0,377.9,8,0,0.0,377.9,8.0,0.0,30.7,89.0,8,8,377.9,377.9
|
||||
06-06 17:40:48,1,1,0.0,350.4,7,0,0.0,350.4,7.0,0.0,30.3,89.0,7,7,350.4,350.4
|
||||
06-06 17:40:53,1,1,89.3,363.3,8,0,89.3,363.3,8.0,0.0,31.8,89.0,8,8,363.3,363.3
|
||||
06-06 17:40:58,1,1,52.8,418.6,9,0,52.8,418.6,9.0,0.0,33.0,89.0,9,9,418.6,418.6
|
||||
06-06 17:41:03,1,1,63.0,441.1,9,0,63.0,441.1,9.0,0.0,34.2,89.1,9,9,441.1,441.1
|
||||
06-06 17:41:08,1,1,0.0,426.9,9,0,0.0,426.9,9.0,0.0,34.8,89.1,9,9,426.9,426.9
|
||||
06-06 17:41:13,1,1,0.0,615.2,13,0,0.0,615.2,13.0,0.0,70.5,66.7,13,13,615.2,615.2
|
||||
06-06 17:41:18,1,1,155.7,645.4,15,0,155.7,645.4,15.0,0.0,71.6,66.7,15,15,645.4,645.4
|
||||
06-06 17:41:23,1,1,0.0,682.6,15,0,0.0,682.6,15.0,0.0,72.5,66.7,15,15,682.6,682.6
|
||||
06-06 17:41:28,1,1,0.0,652.1,14,0,0.0,652.1,14.0,0.0,73.2,66.7,14,14,652.1,652.1
|
||||
06-06 17:41:33,1,1,0.0,601.7,13,0,0.0,601.7,13.0,0.0,65.2,66.7,13,13,601.7,601.7
|
||||
06-06 17:41:38,1,1,84.1,334.6,8,0,84.1,334.6,8.0,0.0,47.5,90.1,8,8,334.6,334.6
|
||||
06-06 17:41:43,1,1,0.0,354.8,7,0,0.0,354.8,7.0,0.0,45.8,90.1,7,7,354.8,354.8
|
||||
06-06 17:41:48,1,1,0.0,316.9,7,0,0.0,316.9,7.0,0.0,46.2,90.1,7,7,316.9,316.9
|
||||
06-06 17:41:53,1,1,2.4,342.4,8,0,2.4,342.4,8.0,0.0,46.7,90.1,8,8,342.4,342.4
|
||||
06-06 17:41:58,1,1,0.0,361.2,8,0,0.0,361.2,8.0,0.0,47.2,90.1,8,8,361.2,361.2
|
||||
06-06 17:42:03,1,1,0.0,360.0,8,0,0.0,360.0,8.0,0.0,47.7,90.1,8,8,360.0,360.0
|
||||
06-06 17:42:08,1,1,0.0,360.1,8,0,0.0,360.1,8.0,0.0,48.2,90.1,8,8,360.1,360.1
|
||||
06-06 17:42:13,1,1,0.0,306.6,6,0,0.0,306.6,6.0,0.0,31.2,90.1,6,6,306.6,306.6
|
||||
06-06 17:42:18,1,1,0.0,270.7,6,0,0.0,270.7,6.0,0.0,31.5,90.1,6,6,270.7,270.7
|
||||
06-06 17:42:23,1,1,0.0,270.8,6,0,0.0,270.8,6.0,0.0,31.9,90.1,6,6,270.8,270.8
|
||||
06-06 17:42:28,1,1,0.0,569.0,13,0,0.0,569.0,13.0,0.0,59.2,88.4,13,13,569.0,569.0
|
||||
06-06 17:42:33,1,1,39.8,573.5,12,0,39.8,573.5,12.0,0.0,52.8,90.9,12,12,573.5,573.5
|
||||
06-06 17:42:38,1,1,80.5,592.6,15,0,80.5,592.6,15.0,0.0,65.5,88.7,15,15,592.6,592.6
|
||||
06-06 17:42:43,1,1,51.0,659.7,15,0,51.0,659.7,15.0,0.0,65.5,88.7,15,15,659.7,659.7
|
||||
06-06 17:42:48,1,1,2.6,669.2,16,0,2.6,669.2,16.0,0.0,66.0,88.7,16,16,669.2,669.2
|
||||
06-06 17:42:53,1,1,0.0,700.5,15,0,0.0,700.5,15.0,0.0,64.9,88.7,15,15,700.5,700.5
|
||||
06-06 17:42:58,1,1,0.0,657.6,15,0,0.0,657.6,15.0,0.0,65.8,88.7,15,15,657.6,657.6
|
||||
06-06 17:43:03,1,1,0.0,653.4,14,0,0.0,653.4,14.0,0.0,63.6,88.7,14,14,653.4,653.4
|
||||
06-06 17:43:08,1,1,0.0,617.4,14,0,0.0,617.4,14.0,0.0,64.4,88.7,14,14,617.4,617.4
|
||||
06-06 17:43:13,1,1,0.0,533.4,12,0,0.0,533.4,12.0,0.0,50.6,91.5,12,12,533.4,533.4
|
||||
06-06 17:43:18,1,1,0.0,616.7,14,0,0.0,616.7,14.0,0.0,66.1,88.7,14,14,616.7,616.7
|
||||
06-06 17:43:23,1,1,23.7,612.8,14,0,23.7,612.8,14.0,0.0,67.0,88.7,14,14,612.8,612.8
|
||||
06-06 17:43:29,1,1,0.0,616.5,14,0,0.0,616.5,14.0,0.0,67.9,88.7,14,14,616.5,616.5
|
||||
06-06 17:43:34,1,1,0.0,577.9,13,0,0.0,577.9,13.0,0.0,68.2,88.7,13,13,577.9,577.9
|
||||
06-06 17:43:39,1,1,17.4,598.2,14,0,17.4,598.2,14.0,0.0,71.1,88.8,14,14,598.2,598.2
|
||||
06-06 17:43:44,1,1,142.9,647.8,15,0,142.9,647.8,15.0,0.0,75.4,88.9,15,15,647.8,647.8
|
||||
06-06 17:43:49,1,1,0.0,607.3,13,0,0.0,607.3,13.0,0.0,72.0,88.9,13,13,607.3,607.3
|
||||
06-06 17:43:54,1,1,0.0,444.0,9,0,0.0,444.0,9.0,0.0,35.4,90.1,9,9,444.0,444.0
|
||||
06-06 17:43:59,1,1,34.2,333.7,8,0,34.2,333.7,8.0,0.0,39.7,92.1,8,8,333.7,333.7
|
||||
06-06 17:44:04,1,1,0.0,222.5,5,0,0.0,222.5,5.0,0.0,22.5,92.2,5,5,222.5,222.5
|
||||
06-06 17:44:09,1,1,0.0,224.0,5,0,0.0,224.0,5.0,0.0,22.8,92.2,5,5,224.0,224.0
|
||||
06-06 17:44:14,1,1,0.0,208.5,4,0,0.0,208.5,4.0,0.0,14.3,92.2,4,4,208.5,208.5
|
||||
06-06 17:44:19,1,1,0.0,178.5,4,0,0.0,178.5,4.0,0.0,14.5,92.2,4,4,178.5,178.5
|
||||
06-06 17:44:24,1,1,0.0,147.1,3,0,0.0,147.1,3.0,0.0,13.7,92.2,3,3,147.1,147.1
|
||||
06-06 17:44:29,1,1,47.0,177.8,4,0,47.0,177.8,4.0,0.0,14.2,92.2,4,4,177.8,177.8
|
||||
06-06 17:44:34,1,1,0.0,178.4,4,0,0.0,178.4,4.0,0.0,14.5,92.2,4,4,178.4,178.4
|
||||
06-06 17:44:39,1,1,0.0,178.4,4,0,0.0,178.4,4.0,0.0,14.7,92.2,4,4,178.4,178.4
|
||||
06-06 17:44:44,1,1,0.0,648.9,17,0,0.0,648.9,17.0,0.0,80.1,91.1,17,17,648.9,648.9
|
||||
06-06 17:44:49,1,1,0.0,651.3,16,0,0.0,651.3,16.0,0.0,80.2,91.1,16,16,651.3,651.3
|
||||
06-06 17:44:54,1,1,0.0,679.7,16,0,0.0,679.7,16.0,0.0,81.2,91.1,16,16,679.7,679.7
|
||||
06-06 17:44:59,1,1,108.9,628.7,17,0,108.9,628.7,17.0,0.0,82.9,91.0,17,17,628.7,628.7
|
||||
06-06 17:45:04,1,1,0.0,679.6,15,0,0.0,679.6,15.0,0.0,60.0,91.6,15,15,679.6,679.6
|
||||
06-06 17:45:09,1,1,93.1,701.0,16,0,93.1,701.0,16.0,0.0,62.0,91.6,16,16,701.0,701.0
|
||||
06-06 17:45:14,1,1,0.0,721.8,15,0,0.0,721.8,15.0,0.0,61.9,91.6,15,15,721.8,721.8
|
||||
06-06 17:45:19,1,1,0.0,682.5,15,0,0.0,682.5,15.0,0.0,62.9,91.6,15,15,682.5,682.5
|
||||
06-06 17:45:24,1,1,19.2,708.5,16,0,19.2,708.5,16.0,0.0,63.2,91.6,16,16,708.5,708.5
|
||||
06-06 17:45:29,1,1,0.0,693.3,15,0,0.0,693.3,15.0,0.0,64.1,91.6,15,15,693.3,693.3
|
||||
06-06 17:45:34,1,1,42.1,706.7,16,0,42.1,706.7,16.0,0.0,67.3,91.7,16,16,706.7,706.7
|
||||
06-06 17:45:39,1,1,0.0,724.7,16,0,0.0,724.7,16.0,0.0,68.3,91.7,16,16,724.7,724.7
|
||||
06-06 17:45:44,1,1,0.0,725.5,16,0,0.0,725.5,16.0,0.0,69.3,91.7,16,16,725.5,725.5
|
||||
06-06 17:45:49,1,1,21.8,690.0,17,0,21.8,690.0,17.0,0.0,71.5,91.7,17,17,690.0,690.0
|
||||
06-06 17:45:55,1,1,0.0,736.3,20,0,0.0,736.3,20.0,0.0,85.9,82.1,20,20,736.3,736.3
|
||||
06-06 17:46:00,1,1,489.0,726.4,20,0,489.0,726.4,20.0,0.0,78.7,81.9,20,20,726.4,726.4
|
||||
06-06 17:46:05,1,1,82.2,733.0,20,0,82.2,733.0,20.0,0.0,80.1,82.0,20,20,733.0,733.0
|
||||
06-06 17:46:10,1,1,0.0,736.0,20,0,0.0,736.0,20.0,0.0,81.2,82.0,20,20,736.0,736.0
|
||||
06-06 17:46:15,1,1,0.6,772.4,21,0,0.6,772.4,21.0,0.0,82.2,82.0,21,21,772.4,772.4
|
||||
06-06 17:46:20,1,1,229.9,769.4,21,0,229.9,769.4,21.0,0.0,85.8,82.2,21,21,769.4,769.4
|
||||
06-06 17:46:25,1,1,0.0,779.7,21,0,0.0,779.7,21.0,0.0,86.9,82.2,21,21,779.7,779.7
|
||||
06-06 17:46:30,1,1,289.9,787.7,22,0,289.9,787.7,22.0,0.0,89.4,82.1,22,22,787.7,787.7
|
||||
06-06 17:46:35,1,1,0.0,787.0,20,0,0.0,787.0,20.0,0.0,87.1,82.1,20,20,787.0,787.0
|
||||
06-06 17:46:40,1,1,207.2,775.0,22,0,207.2,775.0,22.0,0.0,89.7,82.2,22,22,775.0,775.0
|
||||
06-06 17:46:44,1,1,0.0,664.8,17,0,0.0,664.8,17.0,0.0,79.1,91.7,17,17,664.8,664.8
|
||||
06-06 17:46:50,1,1,12.0,813.0,22,0,12.0,813.0,22.0,0.0,92.6,82.3,22,22,813.0,813.0
|
||||
06-06 17:46:55,1,1,38.0,826.4,23,0,38.0,826.4,23.0,0.0,95.2,82.3,23,23,826.4,826.4
|
||||
06-06 17:47:00,1,1,0.0,845.9,23,0,0.0,845.9,23.0,0.0,96.4,82.3,23,23,845.9,845.9
|
||||
06-06 17:47:05,1,1,93.0,864.6,24,0,93.0,864.6,24.0,0.0,99.7,82.4,24,24,864.6,864.6
|
||||
06-06 17:47:10,1,1,202.3,742.7,22,0,202.3,742.7,22.0,0.0,67.5,85.1,22,22,742.7,742.7
|
||||
06-06 17:47:15,1,1,0.0,344.9,7,0,0.0,344.9,7.0,0.0,23.0,89.3,7,7,344.9,344.9
|
||||
06-06 17:47:20,1,1,0.0,345.0,7,0,0.0,345.0,7.0,0.0,23.4,89.3,7,7,345.0,345.0
|
||||
06-06 17:47:25,1,1,56.1,394.2,8,0,56.1,394.2,8.0,0.0,24.6,89.3,8,8,394.2,394.2
|
||||
06-06 17:47:30,1,1,123.7,368.3,7,0,123.7,368.3,7.0,0.0,24.1,89.3,7,7,368.3,368.3
|
||||
06-06 17:47:35,1,1,8.4,351.6,8,0,8.4,351.6,8.0,0.0,24.7,89.3,8,8,351.6,351.6
|
||||
06-06 17:47:40,1,1,0.0,395.3,8,0,0.0,395.3,8.0,0.0,25.3,89.3,8,8,395.3,395.3
|
||||
06-06 17:47:45,1,1,0.0,479.1,11,0,0.0,479.1,11.0,0.0,54.5,88.9,11,11,479.1,479.1
|
||||
06-06 17:47:50,1,1,0.0,479.0,11,0,0.0,479.0,11.0,0.0,55.1,88.9,11,11,479.0,479.0
|
||||
06-06 17:47:55,1,1,0.0,478.3,11,0,0.0,478.3,11.0,0.0,55.8,88.9,11,11,478.3,478.3
|
||||
06-06 17:48:00,1,1,0.0,478.8,11,0,0.0,478.8,11.0,0.0,56.5,88.9,11,11,478.8,478.8
|
||||
06-06 17:48:05,1,1,0.0,480.0,11,0,0.0,480.0,11.0,0.0,57.1,88.9,11,11,480.0,480.0
|
||||
06-06 17:48:10,1,1,0.0,479.1,11,0,0.0,479.1,11.0,0.0,57.8,88.9,11,11,479.1,479.1
|
||||
06-06 17:48:15,1,1,97.4,481.2,12,0,97.4,481.2,12.0,0.0,59.0,88.9,12,12,481.2,481.2
|
||||
06-06 17:48:20,1,1,0.0,523.0,12,0,0.0,523.0,12.0,0.0,59.8,88.9,12,12,523.0,523.0
|
||||
06-06 17:48:25,1,1,0.0,504.7,11,0,0.0,504.7,11.0,0.0,59.8,88.9,11,11,504.7,504.7
|
||||
06-06 17:48:30,1,1,33.6,831.4,23,0,33.6,831.4,23.0,0.0,73.4,85.7,23,23,831.4,831.4
|
||||
06-06 17:48:35,1,1,102.4,857.7,24,1,102.4,857.7,24.0,1.0,75.4,85.7,24,24,857.7,857.7
|
||||
06-06 17:48:40,1,1,9.6,566.8,13,0,9.6,566.8,13.0,0.0,62.0,88.9,13,13,566.8,566.8
|
||||
06-06 17:48:45,1,1,0.0,531.1,12,0,0.0,531.1,12.0,0.0,62.6,88.9,12,12,531.1,531.1
|
||||
06-06 17:48:50,1,1,0.0,483.3,11,0,0.0,483.3,11.0,0.0,63.2,88.9,11,11,483.3,483.3
|
||||
06-06 17:48:55,1,1,75.6,480.6,12,0,75.6,480.6,12.0,0.0,64.6,88.9,12,12,480.6,480.6
|
||||
06-06 17:49:00,1,1,0.0,521.7,12,0,0.0,521.7,12.0,0.0,65.3,88.9,12,12,521.7,521.7
|
||||
06-06 17:49:05,1,1,0.0,521.7,12,0,0.0,521.7,12.0,0.0,66.1,88.9,12,12,521.7,521.7
|
||||
06-06 17:49:10,1,1,106.2,555.9,14,0,106.2,555.9,14.0,0.0,67.2,88.8,14,14,555.9,555.9
|
||||
06-06 17:49:15,1,1,46.1,613.1,14,0,46.1,613.1,14.0,0.0,68.1,88.8,14,14,613.1,613.1
|
||||
06-06 17:49:20,1,1,128.6,641.4,16,0,128.6,641.4,16.0,0.0,69.2,88.7,16,16,641.4,641.4
|
||||
06-06 17:49:25,1,1,0.0,696.0,16,0,0.0,696.0,16.0,0.0,70.2,88.7,16,16,696.0,696.0
|
||||
06-06 17:49:30,1,1,0.0,675.1,15,0,0.0,675.1,15.0,0.0,70.9,88.7,15,15,675.1,675.1
|
||||
06-06 17:49:35,1,1,0.0,644.7,14,0,0.0,644.7,14.0,0.0,63.0,88.7,14,14,644.7,644.7
|
||||
06-06 17:49:40,1,1,0.0,618.1,14,0,0.0,618.1,14.0,0.0,71.7,44.7,14,14,618.1,618.1
|
||||
06-06 17:49:45,1,1,0.0,616.5,14,0,0.0,616.5,14.0,0.0,72.6,44.7,14,14,616.5,616.5
|
||||
06-06 17:49:50,1,1,152.6,668.1,14,0,152.6,668.1,14.0,0.0,67.2,44.9,14,14,668.1,668.1
|
||||
06-06 17:49:56,1,1,29.3,611.1,14,0,29.3,611.1,14.0,0.0,65.8,45.0,14,14,611.1,611.1
|
||||
06-06 17:50:01,1,1,0.0,609.2,14,0,0.0,609.2,14.0,0.0,66.7,45.0,14,14,609.2,609.2
|
||||
06-06 17:50:06,1,1,21.5,635.8,15,0,21.5,635.8,15.0,0.0,70.4,45.2,15,15,635.8,635.8
|
||||
06-06 17:50:11,1,1,50.2,647.1,14,0,50.2,647.1,14.0,0.0,67.6,45.3,14,14,647.1,647.1
|
||||
06-06 17:50:16,1,1,177.2,641.4,16,0,177.2,641.4,16.0,0.0,71.8,45.5,16,16,641.4,641.4
|
||||
06-06 17:50:21,1,1,1.2,698.2,16,0,1.2,698.2,16.0,0.0,72.0,45.6,16,16,698.2,698.2
|
||||
06-06 17:50:26,1,1,0.0,703.5,15,0,0.0,703.5,15.0,0.0,70.4,45.6,15,15,703.5,703.5
|
||||
06-06 17:50:31,1,1,0.0,658.7,15,0,0.0,658.7,15.0,0.0,71.4,45.6,15,15,658.7,658.7
|
||||
06-06 17:50:36,1,1,0.0,878.0,24,7,0.0,878.0,24.0,7.0,95.8,82.7,24,24,878.0,878.0
|
||||
06-06 17:50:41,1,1,0.0,878.4,24,7,0.0,878.4,24.0,7.0,97.0,82.7,24,24,878.4,878.4
|
||||
06-06 17:50:46,1,1,0.0,877.1,24,8,0.0,877.1,24.0,8.0,98.2,82.7,24,24,877.1,877.1
|
||||
06-06 17:50:51,1,1,0.0,530.7,12,0,0.0,530.7,12.0,0.0,56.0,45.9,12,12,530.7,530.7
|
||||
06-06 17:50:56,1,1,0.0,531.3,12,0,0.0,531.3,12.0,0.0,56.7,45.9,12,12,531.3,531.3
|
||||
06-06 17:51:01,1,1,214.0,544.1,13,0,214.0,544.1,13.0,0.0,58.5,45.9,13,13,544.1,544.1
|
||||
06-06 17:51:06,1,1,127.8,556.2,13,0,127.8,556.2,13.0,0.0,59.9,46.0,13,13,556.2,556.2
|
||||
06-06 17:51:11,1,1,0.0,441.3,9,0,0.0,441.3,9.0,0.0,38.4,91.6,9,9,441.3,441.3
|
||||
06-06 17:51:16,1,1,0.0,416.1,9,0,0.0,416.1,9.0,0.0,39.0,91.6,9,9,416.1,416.1
|
||||
06-06 17:51:21,1,1,34.9,435.4,10,0,34.9,435.4,10.0,0.0,44.4,91.7,10,10,435.4,435.4
|
||||
06-06 17:51:26,1,1,99.9,491.8,11,0,99.9,491.8,11.0,0.0,46.4,91.7,11,11,491.8,491.8
|
||||
06-06 17:51:31,1,1,43.2,479.7,11,0,43.2,479.7,11.0,0.0,41.1,91.7,11,11,479.7,479.7
|
||||
06-06 17:51:36,1,1,204.5,608.4,14,0,204.5,608.4,14.0,0.0,43.7,91.7,14,14,608.4,608.4
|
||||
06-06 17:51:41,1,1,50.4,551.1,13,0,50.4,551.1,13.0,0.0,52.7,89.6,13,13,551.1,551.1
|
||||
06-06 17:51:46,1,1,0.0,573.9,13,0,0.0,573.9,13.0,0.0,53.5,89.6,13,13,573.9,573.9
|
||||
06-06 17:51:51,1,1,0.0,557.5,12,0,0.0,557.5,12.0,0.0,46.0,91.7,12,12,557.5,557.5
|
||||
06-06 17:51:56,1,1,84.9,639.4,15,0,84.9,639.4,15.0,0.0,58.2,89.8,15,15,639.4,639.4
|
||||
06-06 17:52:01,1,1,0.0,661.7,15,0,0.0,661.7,15.0,0.0,59.2,89.8,15,15,661.7,661.7
|
||||
06-06 17:52:06,1,1,0.0,650.9,14,0,0.0,650.9,14.0,0.0,59.6,89.8,14,14,650.9,650.9
|
||||
06-06 17:52:11,1,1,0.0,610.1,13,0,0.0,610.1,13.0,0.0,56.1,89.8,13,13,610.1,610.1
|
||||
06-06 17:52:16,1,1,0.0,544.0,12,0,0.0,544.0,12.0,0.0,56.5,89.8,12,12,544.0,544.0
|
||||
06-06 17:52:21,1,1,0.0,531.6,12,0,0.0,531.6,12.0,0.0,57.2,89.8,12,12,531.6,531.6
|
||||
06-06 17:52:26,1,1,51.8,530.0,13,0,51.8,530.0,13.0,0.0,59.7,89.8,13,13,530.0,530.0
|
||||
06-06 17:52:31,1,1,237.6,608.7,14,0,237.6,608.7,14.0,0.0,62.2,89.7,14,14,608.7,608.7
|
||||
06-06 17:52:36,1,1,0.0,620.7,14,0,0.0,620.7,14.0,0.0,63.0,89.7,14,14,620.7,620.7
|
||||
06-06 17:52:41,1,1,0.0,506.5,12,0,0.0,506.5,12.0,0.0,48.8,92.6,12,12,506.5,506.5
|
||||
06-06 17:52:46,1,1,0.0,472.2,11,0,0.0,472.2,11.0,0.0,46.6,92.6,11,11,472.2,472.2
|
||||
06-06 17:52:51,1,1,156.1,512.1,11,0,156.1,512.1,11.0,0.0,41.3,92.0,11,11,512.1,512.1
|
||||
06-06 17:52:56,1,1,11.2,560.5,12,0,11.2,560.5,12.0,0.0,43.2,92.0,12,12,560.5,560.5
|
||||
06-06 17:53:01,1,1,0.0,536.7,11,0,0.0,536.7,11.0,0.0,42.7,92.0,11,11,536.7,536.7
|
||||
06-06 17:53:06,1,1,0.0,475.9,10,0,0.0,475.9,10.0,0.0,42.1,92.0,10,10,475.9,475.9
|
||||
06-06 17:53:11,1,1,0.0,473.4,10,0,0.0,473.4,10.0,0.0,42.8,92.0,10,10,473.4,473.4
|
||||
06-06 17:53:16,1,1,51.0,512.8,11,0,51.0,512.8,11.0,0.0,44.6,92.0,11,11,512.8,512.8
|
||||
06-06 17:53:21,1,1,0.0,522.2,11,0,0.0,522.2,11.0,0.0,45.3,92.0,11,11,522.2,522.2
|
||||
06-06 17:53:26,1,1,0.0,523.4,11,0,0.0,523.4,11.0,0.0,46.0,92.0,11,11,523.4,523.4
|
||||
06-06 17:53:32,1,1,0.0,522.6,11,0,0.0,522.6,11.0,0.0,46.8,92.0,11,11,522.6,522.6
|
||||
06-06 17:53:37,1,1,0.0,496.1,10,0,0.0,496.1,10.0,0.0,46.8,92.0,10,10,496.1,496.1
|
||||
06-06 17:53:42,1,1,14.6,686.3,16,0,14.6,686.3,16.0,0.0,60.3,89.8,16,16,686.3,686.3
|
||||
06-06 17:53:47,1,1,244.6,663.6,16,0,244.6,663.6,16.0,0.0,62.8,89.9,16,16,663.6,663.6
|
||||
06-06 17:53:52,1,1,0.0,701.5,16,0,0.0,701.5,16.0,0.0,63.8,89.9,16,16,701.5,701.5
|
||||
06-06 17:53:57,1,1,0.0,662.6,14,0,0.0,662.6,14.0,0.0,59.8,89.9,14,14,662.6,662.6
|
||||
06-06 17:54:02,1,1,0.0,612.3,14,0,0.0,612.3,14.0,0.0,60.7,89.9,14,14,612.3,612.3
|
||||
06-06 17:54:07,1,1,1.0,611.9,15,0,1.0,611.9,15.0,0.0,61.5,89.9,15,15,611.9,611.9
|
||||
06-06 17:54:12,1,1,106.6,686.2,16,0,106.6,686.2,16.0,0.0,64.4,90.0,16,16,686.2,686.2
|
||||
06-06 17:54:17,1,1,89.6,683.3,16,0,89.6,683.3,16.0,0.0,64.5,89.9,16,16,683.3,683.3
|
||||
06-06 17:54:22,2,2,0.0,1016.5,24,0,0.0,508.25,12.0,0.0,58.0,90.25,9,15,376.5,640.0
|
||||
|
441
training_logs/20260608_210110_vllm_metrics_job_632525.csv
Normal file
441
training_logs/20260608_210110_vllm_metrics_job_632525.csv
Normal file
@@ -0,0 +1,441 @@
|
||||
datetime_str,n_engines_reporting,unique_ips,total_prompt_throughput_tokens_per_sec,total_generation_throughput_tokens_per_sec,total_running_requests,total_waiting_requests,avg_prompt_throughput_per_engine,avg_generation_throughput_per_engine,avg_running_requests_per_engine,avg_waiting_requests_per_engine,avg_gpu_kv_cache_usage_pct,avg_prefix_cache_hit_rate_pct,min_running_requests,max_running_requests,min_generation_throughput,max_generation_throughput
|
||||
06-06 18:02:23,1,1,15.3,0.0,1,0,15.3,0.0,1.0,0.0,0.3,0.0,1,1,0.0,0.0
|
||||
06-06 18:02:28,1,1,75.4,82.2,2,0,75.4,82.2,2.0,0.0,0.5,29.3,2,2,82.2,82.2
|
||||
06-06 18:02:33,1,1,71.2,62.1,2,0,71.2,62.1,2.0,0.0,0.7,63.3,2,2,62.1,62.1
|
||||
06-06 18:02:38,1,1,0.0,91.9,2,0,0.0,91.9,2.0,0.0,0.9,63.3,2,2,91.9,91.9
|
||||
06-06 18:02:43,1,1,67.9,72.3,2,0,67.9,72.3,2.0,0.0,1.1,69.6,2,2,72.3,72.3
|
||||
06-06 18:02:48,1,1,128.3,77.4,2,0,128.3,77.4,2.0,0.0,1.3,72.0,2,2,77.4,77.4
|
||||
06-06 18:02:53,1,1,117.8,76.6,2,0,117.8,76.6,2.0,0.0,1.6,75.1,2,2,76.6,76.6
|
||||
06-06 18:02:58,1,1,104.9,71.6,2,0,104.9,71.6,2.0,0.0,1.8,76.8,2,2,71.6,71.6
|
||||
06-06 18:03:03,1,1,61.6,76.2,2,0,61.6,76.2,2.0,0.0,2.0,79.4,2,2,76.2,76.2
|
||||
06-06 18:03:08,1,1,0.0,81.7,1,0,0.0,81.7,1.0,0.0,1.2,79.4,1,1,81.7,81.7
|
||||
06-06 18:03:13,1,1,23.6,89.1,2,0,23.6,89.1,2.0,0.0,2.3,82.3,2,2,89.1,89.1
|
||||
06-06 18:03:19,1,1,79.9,7.5,1,0,79.9,7.5,1.0,0.0,1.3,74.8,1,1,7.5,7.5
|
||||
06-06 18:03:24,1,1,119.7,87.2,2,0,119.7,87.2,2.0,0.0,1.6,72.3,2,2,87.2,87.2
|
||||
06-06 18:03:29,1,1,0.0,87.8,1,0,0.0,87.8,1.0,0.0,0.5,72.3,1,1,87.8,87.8
|
||||
06-06 18:03:34,1,1,131.9,113.3,3,0,131.9,113.3,3.0,0.0,2.1,78.4,3,3,113.3,113.3
|
||||
06-06 18:03:39,1,1,0.0,140.1,3,0,0.0,140.1,3.0,0.0,2.3,78.4,3,3,140.1,140.1
|
||||
06-06 18:03:44,1,1,0.0,140.0,3,0,0.0,140.0,3.0,0.0,2.4,78.4,3,3,140.0,140.0
|
||||
06-06 18:03:49,1,1,0.0,140.2,3,0,0.0,140.2,3.0,0.0,2.6,78.4,3,3,140.2,140.2
|
||||
06-06 18:03:54,1,1,0.0,139.3,2,0,0.0,139.3,2.0,0.0,2.4,78.4,2,2,139.3,139.3
|
||||
06-06 18:03:59,1,1,108.2,80.9,2,0,108.2,80.9,2.0,0.0,3.1,81.8,2,2,80.9,80.9
|
||||
06-06 18:04:04,1,1,28.3,76.9,2,0,28.3,76.9,2.0,0.0,3.2,85.2,2,2,76.9,76.9
|
||||
06-06 18:04:09,1,1,0.0,95.9,2,0,0.0,95.9,2.0,0.0,3.4,85.2,2,2,95.9,95.9
|
||||
06-06 18:04:14,1,1,0.0,95.8,2,0,0.0,95.8,2.0,0.0,3.5,85.2,2,2,95.8,95.8
|
||||
06-06 18:04:19,1,1,0.0,96.0,2,0,0.0,96.0,2.0,0.0,3.6,85.2,2,2,96.0,96.0
|
||||
06-06 18:04:24,1,1,0.0,95.9,2,0,0.0,95.9,2.0,0.0,3.8,85.2,2,2,95.9,95.9
|
||||
06-06 18:04:29,1,1,0.0,96.1,2,0,0.0,96.1,2.0,0.0,3.9,85.2,2,2,96.1,96.1
|
||||
06-06 18:04:34,1,1,0.0,96.0,2,0,0.0,96.0,2.0,0.0,4.1,85.2,2,2,96.0,96.0
|
||||
06-06 18:04:39,1,1,0.0,96.0,2,0,0.0,96.0,2.0,0.0,4.2,85.2,2,2,96.0,96.0
|
||||
06-06 18:04:44,1,1,0.0,96.0,2,0,0.0,96.0,2.0,0.0,4.3,85.2,2,2,96.0,96.0
|
||||
06-06 18:04:49,1,1,0.0,96.2,2,0,0.0,96.2,2.0,0.0,4.5,85.2,2,2,96.2,96.2
|
||||
06-06 18:04:54,1,1,87.3,132.1,3,0,87.3,132.1,3.0,0.0,4.8,84.2,3,3,132.1,132.1
|
||||
06-06 18:04:59,1,1,0.0,144.6,3,0,0.0,144.6,3.0,0.0,5.0,84.2,3,3,144.6,144.6
|
||||
06-06 18:05:04,1,1,0.0,144.2,3,0,0.0,144.2,3.0,0.0,5.2,84.2,3,3,144.2,144.2
|
||||
06-06 18:05:09,1,1,53.6,116.9,1,0,53.6,116.9,1.0,0.0,2.6,86.5,1,1,116.9,116.9
|
||||
06-06 18:05:14,1,1,132.9,110.6,3,0,132.9,110.6,3.0,0.0,4.7,87.5,3,3,110.6,110.6
|
||||
06-06 18:05:19,1,1,310.2,193.9,4,0,310.2,193.9,4.0,0.0,5.9,79.9,4,4,193.9,193.9
|
||||
06-06 18:05:24,1,1,26.1,227.9,5,0,26.1,227.9,5.0,0.0,6.5,80.5,5,5,227.9,227.9
|
||||
06-06 18:05:30,1,1,151.1,103.2,3,0,151.1,103.2,3.0,0.0,4.1,88.8,3,3,103.2,103.2
|
||||
06-06 18:05:35,1,1,90.0,120.0,4,0,90.0,120.0,4.0,0.0,4.4,89.0,4,4,120.0,120.0
|
||||
06-06 18:05:40,1,1,0.0,172.8,4,0,0.0,172.8,4.0,0.0,4.6,89.0,4,4,172.8,172.8
|
||||
06-06 18:05:45,1,1,0.0,129.7,1,0,0.0,129.7,1.0,0.0,2.6,89.0,1,1,129.7,129.7
|
||||
06-06 18:05:50,1,1,217.3,81.2,3,0,217.3,81.2,3.0,0.0,4.0,88.5,3,3,81.2,81.2
|
||||
06-06 18:05:55,1,1,0.0,130.0,3,0,0.0,130.0,3.0,0.0,4.2,88.5,3,3,130.0,130.0
|
||||
06-06 18:06:00,1,1,0.0,94.9,2,0,0.0,94.9,2.0,0.0,3.9,88.5,2,2,94.9,94.9
|
||||
06-06 18:06:05,1,1,98.5,118.4,3,0,98.5,118.4,3.0,0.0,4.6,88.3,3,3,118.4,118.4
|
||||
06-06 18:06:10,1,1,0.0,129.9,3,0,0.0,129.9,3.0,0.0,4.8,88.3,3,3,129.9,129.9
|
||||
06-06 18:06:15,1,1,0.0,129.8,3,0,0.0,129.8,3.0,0.0,4.9,88.3,3,3,129.8,129.8
|
||||
06-06 18:06:20,1,1,441.6,308.4,7,0,441.6,308.4,7.0,0.0,11.4,86.3,7,7,308.4,308.4
|
||||
06-06 18:06:25,1,1,56.8,302.4,6,0,56.8,302.4,6.0,0.0,8.8,86.8,6,6,302.4,302.4
|
||||
06-06 18:06:30,1,1,190.9,224.2,4,0,190.9,224.2,4.0,0.0,9.1,91.1,4,4,224.2,224.2
|
||||
06-06 18:06:35,1,1,0.0,129.3,3,0,0.0,129.3,3.0,0.0,5.9,88.1,3,3,129.3,129.3
|
||||
06-06 18:06:40,1,1,0.0,238.3,5,0,0.0,238.3,5.0,0.0,10.8,91.2,5,5,238.3,238.3
|
||||
06-06 18:06:45,1,1,117.8,218.2,5,0,117.8,218.2,5.0,0.0,11.3,91.2,5,5,218.2,218.2
|
||||
06-06 18:06:50,1,1,0.0,194.2,4,0,0.0,194.2,4.0,0.0,10.2,91.2,4,4,194.2,194.2
|
||||
06-06 18:06:55,1,1,318.9,222.8,6,0,318.9,222.8,6.0,0.0,12.3,90.9,6,6,222.8,222.8
|
||||
06-06 18:07:00,1,1,0.0,278.9,5,0,0.0,278.9,5.0,0.0,10.9,90.9,5,5,278.9,278.9
|
||||
06-06 18:07:05,1,1,0.0,239.7,5,0,0.0,239.7,5.0,0.0,11.3,90.9,5,5,239.7,239.7
|
||||
06-06 18:07:10,1,1,0.0,238.4,5,0,0.0,238.4,5.0,0.0,11.6,90.9,5,5,238.4,238.4
|
||||
06-06 18:07:15,1,1,206.5,245.4,6,0,206.5,245.4,6.0,0.0,13.9,90.8,6,6,245.4,245.4
|
||||
06-06 18:07:20,1,1,27.3,261.2,6,0,27.3,261.2,6.0,0.0,14.3,90.8,6,6,261.2,261.2
|
||||
06-06 18:07:25,1,1,0.0,311.1,7,0,0.0,311.1,7.0,0.0,19.4,87.3,7,7,311.1,311.1
|
||||
06-06 18:07:30,1,1,0.0,293.1,6,0,0.0,293.1,6.0,0.0,18.3,87.3,6,6,293.1,293.1
|
||||
06-06 18:07:35,1,1,0.0,327.7,7,0,0.0,327.7,7.0,0.0,18.4,90.7,7,7,327.7,327.7
|
||||
06-06 18:07:40,1,1,0.0,279.5,6,0,0.0,279.5,6.0,0.0,15.9,90.7,6,6,279.5,279.5
|
||||
06-06 18:07:45,1,1,79.4,244.1,6,0,79.4,244.1,6.0,0.0,16.3,90.7,6,6,244.1,244.1
|
||||
06-06 18:07:50,1,1,111.6,269.3,7,0,111.6,269.3,7.0,0.0,19.8,90.9,7,7,269.3,269.3
|
||||
06-06 18:07:55,1,1,0.0,303.9,7,0,0.0,303.9,7.0,0.0,20.2,90.9,7,7,303.9,303.9
|
||||
06-06 18:08:00,1,1,131.2,469.7,10,0,131.2,469.7,10.0,0.0,27.2,86.2,10,10,469.7,469.7
|
||||
06-06 18:08:05,1,1,0.0,442.3,10,0,0.0,442.3,10.0,0.0,27.8,86.2,10,10,442.3,442.3
|
||||
06-06 18:08:10,1,1,0.0,444.1,10,0,0.0,444.1,10.0,0.0,28.4,86.2,10,10,444.1,444.1
|
||||
06-06 18:08:15,1,1,0.0,439.8,9,0,0.0,439.8,9.0,0.0,28.7,86.2,9,9,439.8,439.8
|
||||
06-06 18:08:20,1,1,295.8,405.8,10,0,295.8,405.8,10.0,0.0,29.5,85.5,10,10,405.8,405.8
|
||||
06-06 18:08:25,1,1,28.4,405.7,9,0,28.4,405.7,9.0,0.0,27.2,85.6,9,9,405.7,405.7
|
||||
06-06 18:08:30,1,1,44.4,124.5,3,0,44.4,124.5,3.0,0.0,7.3,86.4,3,3,124.5,124.5
|
||||
06-06 18:08:35,1,1,85.0,154.9,4,0,85.0,154.9,4.0,0.0,7.6,86.4,4,4,154.9,154.9
|
||||
06-06 18:08:41,1,1,0.0,220.6,5,0,0.0,220.6,5.0,0.0,21.7,88.0,5,5,220.6,220.6
|
||||
06-06 18:08:46,1,1,0.0,220.4,5,0,0.0,220.4,5.0,0.0,22.0,88.0,5,5,220.4,220.4
|
||||
06-06 18:08:51,1,1,58.7,252.5,6,0,58.7,252.5,6.0,0.0,23.1,88.1,6,6,252.5,252.5
|
||||
06-06 18:08:56,1,1,0.0,230.9,5,0,0.0,230.9,5.0,0.0,22.6,88.1,5,5,230.9,230.9
|
||||
06-06 18:09:01,1,1,0.0,220.6,5,0,0.0,220.6,5.0,0.0,22.9,88.1,5,5,220.6,220.6
|
||||
06-06 18:09:06,1,1,0.0,220.3,5,0,0.0,220.3,5.0,0.0,23.2,88.1,5,5,220.3,220.3
|
||||
06-06 18:09:11,1,1,74.1,253.3,6,0,74.1,253.3,6.0,0.0,23.7,87.9,6,6,253.3,253.3
|
||||
06-06 18:09:16,1,1,0.0,265.3,6,0,0.0,265.3,6.0,0.0,24.0,87.9,6,6,265.3,265.3
|
||||
06-06 18:09:21,1,1,82.6,307.4,7,0,82.6,307.4,7.0,0.0,24.6,87.7,7,7,307.4,307.4
|
||||
06-06 18:09:26,1,1,34.3,295.9,7,0,34.3,295.9,7.0,0.0,25.0,87.8,7,7,295.9,295.9
|
||||
06-06 18:09:31,1,1,0.0,308.5,7,0,0.0,308.5,7.0,0.0,25.5,87.8,7,7,308.5,308.5
|
||||
06-06 18:09:36,1,1,0.0,308.9,7,0,0.0,308.9,7.0,0.0,25.9,87.8,7,7,308.9,308.9
|
||||
06-06 18:09:41,1,1,0.0,299.6,6,0,0.0,299.6,6.0,0.0,25.9,87.8,6,6,299.6,299.6
|
||||
06-06 18:09:46,1,1,102.8,298.2,7,0,102.8,298.2,7.0,0.0,26.9,87.6,7,7,298.2,298.2
|
||||
06-06 18:09:51,1,1,0.0,305.5,7,0,0.0,305.5,7.0,0.0,27.3,87.6,7,7,305.5,305.5
|
||||
06-06 18:09:56,1,1,0.0,303.3,7,0,0.0,303.3,7.0,0.0,27.7,87.6,7,7,303.3,303.3
|
||||
06-06 18:10:01,1,1,140.8,257.5,6,0,140.8,257.5,6.0,0.0,27.5,87.5,6,6,257.5,257.5
|
||||
06-06 18:10:06,1,1,0.0,257.9,6,0,0.0,257.9,6.0,0.0,27.9,87.5,6,6,257.9,257.9
|
||||
06-06 18:10:11,1,1,0.0,258.5,6,0,0.0,258.5,6.0,0.0,28.3,87.5,6,6,258.5,258.5
|
||||
06-06 18:10:16,1,1,0.0,425.3,9,0,0.0,425.3,9.0,0.0,41.4,90.1,9,9,425.3,425.3
|
||||
06-06 18:10:21,1,1,0.0,425.0,9,0,0.0,425.0,9.0,0.0,42.0,90.1,9,9,425.0,425.0
|
||||
06-06 18:10:26,1,1,0.0,424.2,9,0,0.0,424.2,9.0,0.0,42.6,90.1,9,9,424.2,424.2
|
||||
06-06 18:10:31,1,1,272.4,471.3,11,0,272.4,471.3,11.0,0.0,47.0,90.2,11,11,471.3,471.3
|
||||
06-06 18:10:36,1,1,69.4,486.3,10,0,69.4,486.3,10.0,0.0,46.0,90.2,10,10,486.3,486.3
|
||||
06-06 18:10:41,1,1,0.0,461.8,9,0,0.0,461.8,9.0,0.0,45.9,90.2,9,9,461.8,461.8
|
||||
06-06 18:10:46,1,1,59.4,447.8,10,0,59.4,447.8,10.0,0.0,47.4,90.2,10,10,447.8,447.8
|
||||
06-06 18:10:51,1,1,0.0,469.2,10,0,0.0,469.2,10.0,0.0,48.0,90.2,10,10,469.2,469.2
|
||||
06-06 18:10:56,1,1,0.0,469.3,10,0,0.0,469.3,10.0,0.0,48.7,90.2,10,10,469.3,469.3
|
||||
06-06 18:11:01,1,1,0.0,468.8,10,0,0.0,468.8,10.0,0.0,49.3,90.2,10,10,468.8,468.8
|
||||
06-06 18:11:06,1,1,0.0,469.2,10,0,0.0,469.2,10.0,0.0,50.0,90.2,10,10,469.2,469.2
|
||||
06-06 18:11:11,1,1,0.0,467.8,10,0,0.0,467.8,10.0,0.0,50.6,90.2,10,10,467.8,467.8
|
||||
06-06 18:11:16,1,1,0.0,340.1,7,0,0.0,340.1,7.0,0.0,23.8,86.2,7,7,340.1,340.1
|
||||
06-06 18:11:21,1,1,0.0,340.6,7,0,0.0,340.6,7.0,0.0,24.2,86.2,7,7,340.6,340.6
|
||||
06-06 18:11:26,1,1,0.0,340.2,7,0,0.0,340.2,7.0,0.0,24.7,86.2,7,7,340.2,340.2
|
||||
06-06 18:11:31,1,1,45.6,357.2,8,0,45.6,357.2,8.0,0.0,25.5,86.2,8,8,357.2,357.2
|
||||
06-06 18:11:36,1,1,0.0,388.3,8,0,0.0,388.3,8.0,0.0,26.0,86.2,8,8,388.3,388.3
|
||||
06-06 18:11:41,1,1,0.0,390.0,8,0,0.0,390.0,8.0,0.0,26.6,86.2,8,8,390.0,390.0
|
||||
06-06 18:11:46,1,1,0.0,387.4,8,0,0.0,387.4,8.0,0.0,27.1,86.2,8,8,387.4,387.4
|
||||
06-06 18:11:51,1,1,0.0,388.1,8,0,0.0,388.1,8.0,0.0,27.6,86.2,8,8,388.1,388.1
|
||||
06-06 18:11:56,1,1,0.0,388.6,8,0,0.0,388.6,8.0,0.0,28.2,86.2,8,8,388.6,388.6
|
||||
06-06 18:12:01,1,1,0.0,388.6,8,0,0.0,388.6,8.0,0.0,28.7,86.2,8,8,388.6,388.6
|
||||
06-06 18:12:06,1,1,0.0,370.5,7,0,0.0,370.5,7.0,0.0,28.5,86.2,7,7,370.5,370.5
|
||||
06-06 18:12:11,1,1,0.0,338.0,7,0,0.0,338.0,7.0,0.0,29.0,86.2,7,7,338.0,338.0
|
||||
06-06 18:12:16,1,1,0.0,338.9,7,0,0.0,338.9,7.0,0.0,29.4,86.2,7,7,338.9,338.9
|
||||
06-06 18:12:21,1,1,0.0,279.6,6,0,0.0,279.6,6.0,0.0,35.5,88.8,6,6,279.6,279.6
|
||||
06-06 18:12:26,1,1,134.9,344.1,8,0,134.9,344.1,8.0,0.0,36.2,88.6,8,8,344.1,344.1
|
||||
06-06 18:12:31,1,1,0.0,349.9,7,0,0.0,349.9,7.0,0.0,36.5,88.6,7,7,349.9,349.9
|
||||
06-06 18:12:37,1,1,0.0,323.6,7,0,0.0,323.6,7.0,0.0,34.0,87.4,7,7,323.6,323.6
|
||||
06-06 18:12:42,1,1,0.0,324.5,7,0,0.0,324.5,7.0,0.0,34.5,87.4,7,7,324.5,324.5
|
||||
06-06 18:12:47,1,1,39.7,339.8,7,0,39.7,339.8,7.0,0.0,34.8,87.4,7,7,339.8,339.8
|
||||
06-06 18:12:52,1,1,69.0,363.9,8,0,69.0,363.9,8.0,0.0,35.8,87.4,8,8,363.9,363.9
|
||||
06-06 18:12:57,1,1,47.3,420.1,9,0,47.3,420.1,9.0,0.0,44.6,86.9,9,9,420.1,420.1
|
||||
06-06 18:13:02,1,1,128.2,425.6,9,0,128.2,425.6,9.0,0.0,46.2,86.9,9,9,425.6,425.6
|
||||
06-06 18:13:07,1,1,0.0,431.2,9,0,0.0,431.2,9.0,0.0,46.8,86.9,9,9,431.2,431.2
|
||||
06-06 18:13:12,1,1,0.0,186.7,4,0,0.0,186.7,4.0,0.0,19.0,85.4,4,4,186.7,186.7
|
||||
06-06 18:13:17,1,1,0.0,186.6,4,0,0.0,186.6,4.0,0.0,19.2,85.4,4,4,186.6,186.6
|
||||
06-06 18:13:22,1,1,0.0,163.1,3,0,0.0,163.1,3.0,0.0,18.5,85.4,3,3,163.1,163.1
|
||||
06-06 18:13:27,1,1,266.9,184.1,4,0,266.9,184.1,4.0,0.0,20.1,84.9,4,4,184.1,184.1
|
||||
06-06 18:13:32,1,1,21.6,214.5,6,0,21.6,214.5,6.0,0.0,23.5,86.4,6,6,214.5,214.5
|
||||
06-06 18:13:37,1,1,0.0,280.3,6,0,0.0,280.3,6.0,0.0,23.9,86.4,6,6,280.3,280.3
|
||||
06-06 18:13:42,1,1,0.0,250.3,5,0,0.0,250.3,5.0,0.0,22.0,84.1,5,5,250.3,250.3
|
||||
06-06 18:13:47,1,1,35.1,263.8,5,0,35.1,263.8,5.0,0.0,22.2,84.2,5,5,263.8,263.8
|
||||
06-06 18:13:52,1,1,311.6,361.5,8,0,311.6,361.5,8.0,0.0,23.7,83.8,8,8,361.5,361.5
|
||||
06-06 18:13:57,1,1,41.3,368.7,7,0,41.3,368.7,7.0,0.0,23.9,83.9,7,7,368.7,368.7
|
||||
06-06 18:14:02,1,1,0.0,140.1,3,0,0.0,140.1,3.0,0.0,12.8,86.4,3,3,140.1,140.1
|
||||
06-06 18:14:07,1,1,45.1,145.6,4,0,45.1,145.6,4.0,0.0,13.3,86.4,4,4,145.6,145.6
|
||||
06-06 18:14:12,1,1,0.0,187.6,4,0,0.0,187.6,4.0,0.0,13.6,86.4,4,4,187.6,187.6
|
||||
06-06 18:14:17,1,1,0.0,187.5,4,0,0.0,187.5,4.0,0.0,13.8,86.4,4,4,187.5,187.5
|
||||
06-06 18:14:22,1,1,0.0,186.8,4,0,0.0,186.8,4.0,0.0,14.1,86.4,4,4,186.8,186.8
|
||||
06-06 18:14:27,1,1,209.5,624.1,14,0,209.5,624.1,14.0,0.0,60.5,89.5,14,14,624.1,624.1
|
||||
06-06 18:14:32,1,1,71.3,205.6,5,0,71.3,205.6,5.0,0.0,14.7,86.2,5,5,205.6,205.6
|
||||
06-06 18:14:37,1,1,0.0,225.3,4,0,0.0,225.3,4.0,0.0,14.9,86.2,4,4,225.3,225.3
|
||||
06-06 18:14:42,1,1,70.2,543.9,11,0,70.2,543.9,11.0,0.0,51.9,89.9,11,11,543.9,543.9
|
||||
06-06 18:14:47,1,1,22.7,571.1,12,0,22.7,571.1,12.0,0.0,46.5,90.1,12,12,571.1,571.1
|
||||
06-06 18:14:52,1,1,23.0,516.5,11,0,23.0,516.5,11.0,0.0,44.7,90.2,11,11,516.5,516.5
|
||||
06-06 18:14:57,1,1,21.6,532.8,12,0,21.6,532.8,12.0,0.0,48.0,90.4,12,12,532.8,532.8
|
||||
06-06 18:15:02,1,1,0.0,387.6,7,0,0.0,387.6,7.0,0.0,36.1,87.8,7,7,387.6,387.6
|
||||
06-06 18:15:07,1,1,0.0,370.0,6,0,0.0,370.0,6.0,0.0,35.7,87.8,6,6,370.0,370.0
|
||||
06-06 18:15:12,1,1,0.0,322.6,6,0,0.0,322.6,6.0,0.0,36.2,87.8,6,6,322.6,322.6
|
||||
06-06 18:15:18,1,1,24.2,389.4,9,0,24.2,389.4,9.0,0.0,32.8,85.0,9,9,389.4,389.4
|
||||
06-06 18:15:23,1,1,0.0,364.6,7,0,0.0,364.6,7.0,0.0,31.4,85.0,7,7,364.6,364.6
|
||||
06-06 18:15:28,1,1,125.4,423.6,10,0,125.4,423.6,10.0,0.0,34.5,85.8,10,10,423.6,423.6
|
||||
06-06 18:15:33,1,1,0.0,448.9,9,0,0.0,448.9,9.0,0.0,34.9,85.8,9,9,448.9,448.9
|
||||
06-06 18:15:38,1,1,0.0,399.1,8,0,0.0,399.1,8.0,0.0,34.0,85.8,8,8,399.1,399.1
|
||||
06-06 18:15:43,1,1,44.6,395.4,9,0,44.6,395.4,9.0,0.0,34.8,85.8,9,9,395.4,395.4
|
||||
06-06 18:15:48,1,1,0.0,417.8,9,0,0.0,417.8,9.0,0.0,35.4,85.8,9,9,417.8,417.8
|
||||
06-06 18:15:53,1,1,37.5,371.4,8,0,37.5,371.4,8.0,0.0,35.1,86.0,8,8,371.4,371.4
|
||||
06-06 18:15:58,1,1,0.0,336.3,7,0,0.0,336.3,7.0,0.0,34.9,86.0,7,7,336.3,336.3
|
||||
06-06 18:16:03,1,1,84.2,336.0,8,0,84.2,336.0,8.0,0.0,35.8,86.0,8,8,336.0,336.0
|
||||
06-06 18:16:08,1,1,0.0,372.4,7,0,0.0,372.4,7.0,0.0,35.8,86.0,7,7,372.4,372.4
|
||||
06-06 18:16:13,1,1,84.4,332.4,8,0,84.4,332.4,8.0,0.0,37.8,86.3,8,8,332.4,332.4
|
||||
06-06 18:16:18,1,1,51.0,372.3,8,0,51.0,372.3,8.0,0.0,37.4,86.3,8,8,372.3,372.3
|
||||
06-06 18:16:23,1,1,0.0,377.4,8,0,0.0,377.4,8.0,0.0,18.3,86.7,8,8,377.4,377.4
|
||||
06-06 18:16:28,1,1,21.9,422.0,9,0,21.9,422.0,9.0,0.0,19.6,86.8,9,9,422.0,422.0
|
||||
06-06 18:16:33,1,1,9.6,377.5,8,0,9.6,377.5,8.0,0.0,18.7,86.9,8,8,377.5,377.5
|
||||
06-06 18:16:38,1,1,207.8,390.7,8,0,207.8,390.7,8.0,0.0,15.6,86.7,8,8,390.7,390.7
|
||||
06-06 18:16:43,1,1,65.1,381.5,9,0,65.1,381.5,9.0,0.0,17.1,86.8,9,9,381.5,381.5
|
||||
06-06 18:16:48,1,1,0.0,423.8,9,0,0.0,423.8,9.0,0.0,17.7,86.8,9,9,423.8,423.8
|
||||
06-06 18:16:53,1,1,0.0,422.1,8,0,0.0,422.1,8.0,0.0,17.1,86.8,8,8,422.1,422.1
|
||||
06-06 18:16:58,1,1,0.0,367.6,7,0,0.0,367.6,7.0,0.0,32.4,86.9,7,7,367.6,367.6
|
||||
06-06 18:17:03,1,1,148.2,358.1,8,0,148.2,358.1,8.0,0.0,23.0,86.9,8,8,358.1,358.1
|
||||
06-06 18:17:08,1,1,278.3,411.5,9,0,278.3,411.5,9.0,0.0,23.8,86.8,9,9,411.5,411.5
|
||||
06-06 18:17:13,1,1,163.2,424.3,10,0,163.2,424.3,10.0,0.0,24.8,86.9,10,10,424.3,424.3
|
||||
06-06 18:17:18,1,1,68.7,460.0,10,0,68.7,460.0,10.0,0.0,25.0,86.8,10,10,460.0,460.0
|
||||
06-06 18:17:23,1,1,70.1,703.0,16,0,70.1,703.0,16.0,0.0,64.4,91.9,16,16,703.0,703.0
|
||||
06-06 18:17:28,1,1,0.0,718.4,16,0,0.0,718.4,16.0,0.0,65.4,91.9,16,16,718.4,718.4
|
||||
06-06 18:17:33,1,1,0.0,711.4,15,0,0.0,711.4,15.0,0.0,58.2,91.9,15,15,711.4,711.4
|
||||
06-06 18:17:38,1,1,0.0,675.3,15,0,0.0,675.3,15.0,0.0,59.2,91.9,15,15,675.3,675.3
|
||||
06-06 18:17:43,1,1,0.0,674.9,15,0,0.0,674.9,15.0,0.0,60.1,91.9,15,15,674.9,674.9
|
||||
06-06 18:17:48,1,1,0.0,674.8,15,0,0.0,674.8,15.0,0.0,61.1,91.9,15,15,674.8,674.8
|
||||
06-06 18:17:53,1,1,0.0,675.1,15,0,0.0,675.1,15.0,0.0,62.0,91.9,15,15,675.1,675.1
|
||||
06-06 18:17:58,1,1,0.0,645.7,14,0,0.0,645.7,14.0,0.0,62.2,91.9,14,14,645.7,645.7
|
||||
06-06 18:18:03,1,1,0.0,545.9,12,0,0.0,545.9,12.0,0.0,60.3,91.9,12,12,545.9,545.9
|
||||
06-06 18:18:08,1,1,0.0,543.3,12,0,0.0,543.3,12.0,0.0,61.1,91.9,12,12,543.3,543.3
|
||||
06-06 18:18:13,1,1,178.8,549.8,13,0,178.8,549.8,13.0,0.0,63.5,91.8,13,13,549.8,549.8
|
||||
06-06 18:18:18,1,1,0.0,586.5,12,0,0.0,586.5,12.0,0.0,62.6,91.8,12,12,586.5,586.5
|
||||
06-06 18:18:23,1,1,85.8,572.1,13,0,85.8,572.1,13.0,0.0,64.3,91.8,13,13,572.1,572.1
|
||||
06-06 18:18:28,1,1,0.0,586.9,13,0,0.0,586.9,13.0,0.0,65.1,91.8,13,13,586.9,586.9
|
||||
06-06 18:18:33,1,1,37.8,631.0,14,0,37.8,631.0,14.0,0.0,67.2,91.9,14,14,631.0,631.0
|
||||
06-06 18:18:38,1,1,48.0,628.2,14,0,48.0,628.2,14.0,0.0,68.8,91.9,14,14,628.2,628.2
|
||||
06-06 18:18:43,1,1,0.0,621.1,13,0,0.0,621.1,13.0,0.0,68.2,91.9,13,13,621.1,621.1
|
||||
06-06 18:18:48,1,1,0.0,586.4,13,0,0.0,586.4,13.0,0.0,69.1,91.9,13,13,586.4,586.4
|
||||
06-06 18:18:54,1,1,0.0,692.4,13,0,0.0,692.4,13.0,0.0,42.2,89.6,13,13,692.4,692.4
|
||||
06-06 18:18:59,1,1,0.0,614.9,13,0,0.0,614.9,13.0,0.0,43.0,89.6,13,13,614.9,614.9
|
||||
06-06 18:19:04,1,1,0.0,613.2,12,0,0.0,613.2,12.0,0.0,42.3,89.6,12,12,613.2,613.2
|
||||
06-06 18:19:09,1,1,0.0,564.4,11,0,0.0,564.4,11.0,0.0,42.2,89.6,11,11,564.4,564.4
|
||||
06-06 18:19:14,1,1,0.0,480.4,10,0,0.0,480.4,10.0,0.0,42.5,89.6,10,10,480.4,480.4
|
||||
06-06 18:19:19,1,1,0.0,476.6,10,0,0.0,476.6,10.0,0.0,43.2,89.6,10,10,476.6,476.6
|
||||
06-06 18:19:24,1,1,146.5,534.3,12,0,146.5,534.3,12.0,0.0,47.1,89.8,12,12,534.3,534.3
|
||||
06-06 18:19:29,1,1,0.0,537.2,10,0,0.0,537.2,10.0,0.0,37.8,89.8,10,10,537.2,537.2
|
||||
06-06 18:19:34,1,1,0.0,476.9,10,0,0.0,476.9,10.0,0.0,38.5,89.8,10,10,476.9,476.9
|
||||
06-06 18:19:39,1,1,157.8,443.8,10,0,157.8,443.8,10.0,0.0,35.4,87.6,10,10,443.8,443.8
|
||||
06-06 18:19:44,1,1,0.0,462.4,9,0,0.0,462.4,9.0,0.0,38.4,89.8,9,9,462.4,462.4
|
||||
06-06 18:19:49,1,1,0.0,469.5,10,0,0.0,469.5,10.0,0.0,36.2,87.6,10,10,469.5,469.5
|
||||
06-06 18:19:54,1,1,0.0,393.5,9,0,0.0,393.5,9.0,0.0,39.9,90.0,9,9,393.5,393.5
|
||||
06-06 18:19:59,1,1,0.0,391.0,9,0,0.0,391.0,9.0,0.0,40.5,90.0,9,9,391.0,391.0
|
||||
06-06 18:20:04,1,1,241.7,410.9,12,0,241.7,410.9,12.0,0.0,44.9,90.1,12,12,410.9,410.9
|
||||
06-06 18:20:09,1,1,120.7,512.9,12,0,120.7,512.9,12.0,0.0,45.8,90.0,12,12,512.9,512.9
|
||||
06-06 18:20:14,1,1,0.0,497.0,11,0,0.0,497.0,11.0,0.0,45.8,90.0,11,11,497.0,497.0
|
||||
06-06 18:20:19,1,1,0.0,472.8,11,0,0.0,472.8,11.0,0.0,46.4,90.0,11,11,472.8,472.8
|
||||
06-06 18:20:24,1,1,0.0,473.0,11,0,0.0,473.0,11.0,0.0,47.1,90.0,11,11,473.0,473.0
|
||||
06-06 18:20:29,1,1,21.0,512.1,12,0,21.0,512.1,12.0,0.0,48.8,90.1,12,12,512.1,512.1
|
||||
06-06 18:20:34,1,1,0.0,463.7,10,0,0.0,463.7,10.0,0.0,44.3,90.1,10,10,463.7,463.7
|
||||
06-06 18:20:39,1,1,0.0,428.7,10,0,0.0,428.7,10.0,0.0,44.9,90.1,10,10,428.7,428.7
|
||||
06-06 18:20:44,1,1,0.0,228.6,5,0,0.0,228.6,5.0,0.0,17.5,90.0,5,5,228.6,228.6
|
||||
06-06 18:20:49,1,1,0.0,229.2,5,0,0.0,229.2,5.0,0.0,17.8,90.0,5,5,229.2,229.2
|
||||
06-06 18:20:54,1,1,124.9,415.9,9,0,124.9,415.9,9.0,0.0,30.4,86.7,9,9,415.9,415.9
|
||||
06-06 18:20:59,1,1,0.0,363.1,7,0,0.0,363.1,7.0,0.0,29.9,86.7,7,7,363.1,363.1
|
||||
06-06 18:21:04,1,1,0.0,326.5,7,0,0.0,326.5,7.0,0.0,30.4,86.7,7,7,326.5,326.5
|
||||
06-06 18:21:09,1,1,99.2,343.2,8,0,99.2,343.2,8.0,0.0,31.4,86.7,8,8,343.2,343.2
|
||||
06-06 18:21:14,1,1,38.6,396.2,8,0,38.6,396.2,8.0,0.0,31.9,86.7,8,8,396.2,396.2
|
||||
06-06 18:21:19,1,1,0.0,337.4,7,0,0.0,337.4,7.0,0.0,31.7,86.7,7,7,337.4,337.4
|
||||
06-06 18:21:25,1,1,0.0,215.5,5,0,0.0,215.5,5.0,0.0,33.4,89.4,5,5,215.5,215.5
|
||||
06-06 18:21:30,1,1,245.2,217.6,6,0,245.2,217.6,6.0,0.0,35.2,89.2,6,6,217.6,217.6
|
||||
06-06 18:21:35,1,1,11.4,274.0,7,0,11.4,274.0,7.0,0.0,35.7,89.3,7,7,274.0,274.0
|
||||
06-06 18:21:40,1,1,0.0,302.4,7,0,0.0,302.4,7.0,0.0,36.1,89.3,7,7,302.4,302.4
|
||||
06-06 18:21:45,1,1,0.0,261.4,6,0,0.0,261.4,6.0,0.0,36.3,89.3,6,6,261.4,261.4
|
||||
06-06 18:21:50,1,1,0.0,246.3,5,0,0.0,246.3,5.0,0.0,34.9,89.3,5,5,246.3,246.3
|
||||
06-06 18:21:55,1,1,42.0,593.7,13,0,42.0,593.7,13.0,0.0,59.7,91.6,13,13,593.7,593.7
|
||||
06-06 18:22:00,1,1,0.0,559.7,12,0,0.0,559.7,12.0,0.0,59.2,91.6,12,12,559.7,559.7
|
||||
06-06 18:22:05,1,1,0.0,558.1,12,0,0.0,558.1,12.0,0.0,60.0,91.6,12,12,558.1,558.1
|
||||
06-06 18:22:10,1,1,114.3,585.4,13,0,114.3,585.4,13.0,0.0,62.4,91.6,13,13,585.4,585.4
|
||||
06-06 18:22:15,1,1,33.4,646.4,14,0,33.4,646.4,14.0,0.0,64.6,91.6,14,14,646.4,646.4
|
||||
06-06 18:22:20,1,1,0.0,648.2,14,0,0.0,648.2,14.0,0.0,65.5,91.6,14,14,648.2,648.2
|
||||
06-06 18:22:25,1,1,0.0,606.9,12,0,0.0,606.9,12.0,0.0,63.1,91.6,12,12,606.9,606.9
|
||||
06-06 18:22:30,1,1,8.8,585.4,13,0,8.8,585.4,13.0,0.0,64.0,91.6,13,13,585.4,585.4
|
||||
06-06 18:22:35,1,1,0.0,603.3,13,0,0.0,603.3,13.0,0.0,64.8,91.6,13,13,603.3,603.3
|
||||
06-06 18:22:40,1,1,0.0,603.8,13,0,0.0,603.8,13.0,0.0,65.7,91.6,13,13,603.8,603.8
|
||||
06-06 18:22:45,1,1,0.0,603.5,13,0,0.0,603.5,13.0,0.0,66.5,91.6,13,13,603.5,603.5
|
||||
06-06 18:22:50,1,1,20.3,642.7,14,0,20.3,642.7,14.0,0.0,69.2,91.7,14,14,642.7,642.7
|
||||
06-06 18:22:55,1,1,90.1,654.9,14,0,90.1,654.9,14.0,0.0,71.3,91.7,14,14,654.9,654.9
|
||||
06-06 18:23:00,1,1,0.0,649.1,14,0,0.0,649.1,14.0,0.0,72.2,91.7,14,14,649.1,649.1
|
||||
06-06 18:23:05,1,1,0.0,617.9,13,0,0.0,617.9,13.0,0.0,71.5,91.7,13,13,617.9,617.9
|
||||
06-06 18:23:10,1,1,0.0,576.2,12,0,0.0,576.2,12.0,0.0,70.1,91.7,12,12,576.2,576.2
|
||||
06-06 18:23:15,1,1,0.0,558.4,12,0,0.0,558.4,12.0,0.0,70.9,91.7,12,12,558.4,558.4
|
||||
06-06 18:23:20,1,1,0.0,558.4,12,0,0.0,558.4,12.0,0.0,71.7,91.7,12,12,558.4,558.4
|
||||
06-06 18:23:25,1,1,0.0,439.5,8,0,0.0,439.5,8.0,0.0,34.4,90.0,8,8,439.5,439.5
|
||||
06-06 18:23:30,1,1,273.6,332.1,8,0,273.6,332.1,8.0,0.0,36.1,89.9,8,8,332.1,332.1
|
||||
06-06 18:23:35,1,1,149.8,452.1,10,0,149.8,452.1,10.0,0.0,38.1,88.8,10,10,452.1,452.1
|
||||
06-06 18:23:40,1,1,0.0,457.4,9,0,0.0,457.4,9.0,0.0,37.9,88.8,9,9,457.4,457.4
|
||||
06-06 18:23:45,1,1,53.1,725.5,21,0,53.1,725.5,21.0,0.0,78.8,88.0,21,21,725.5,725.5
|
||||
06-06 18:23:50,1,1,0.0,764.4,21,0,0.0,764.4,21.0,0.0,79.9,88.0,21,21,764.4,764.4
|
||||
06-06 18:23:55,1,1,81.3,746.8,21,0,81.3,746.8,21.0,0.0,81.4,88.0,21,21,746.8,746.8
|
||||
06-06 18:24:00,1,1,78.9,790.6,22,0,78.9,790.6,22.0,0.0,82.8,88.0,22,22,790.6,790.6
|
||||
06-06 18:24:05,1,1,3.2,779.2,21,0,3.2,779.2,21.0,0.0,82.0,88.1,21,21,779.2,779.2
|
||||
06-06 18:24:10,1,1,26.7,774.7,22,0,26.7,774.7,22.0,0.0,83.9,88.1,22,22,774.7,774.7
|
||||
06-06 18:24:15,1,1,0.0,791.8,22,0,0.0,791.8,22.0,0.0,85.0,88.1,22,22,791.8,791.8
|
||||
06-06 18:24:20,1,1,138.7,836.1,24,0,138.7,836.1,24.0,0.0,89.2,88.2,24,24,836.1,836.1
|
||||
06-06 18:24:25,1,1,0.0,845.3,22,0,0.0,845.3,22.0,0.0,87.2,88.2,22,22,845.3,845.3
|
||||
06-06 18:24:30,1,1,0.0,794.2,21,0,0.0,794.2,21.0,0.0,87.3,88.2,21,21,794.2,794.2
|
||||
06-06 18:24:36,1,1,0.0,378.2,9,0,0.0,378.2,9.0,0.0,38.4,90.4,9,9,378.2,378.2
|
||||
06-06 18:24:41,1,1,0.0,372.7,9,0,0.0,372.7,9.0,0.0,38.9,90.4,9,9,372.7,372.7
|
||||
06-06 18:24:46,1,1,203.9,428.4,10,0,203.9,428.4,10.0,0.0,42.5,90.5,10,10,428.4,428.4
|
||||
06-06 18:24:51,1,1,0.0,245.6,5,0,0.0,245.6,5.0,0.0,25.2,91.6,5,5,245.6,245.6
|
||||
06-06 18:24:56,1,1,0.0,245.1,5,0,0.0,245.1,5.0,0.0,25.6,91.6,5,5,245.1,245.1
|
||||
06-06 18:25:01,1,1,0.0,244.9,5,0,0.0,244.9,5.0,0.0,25.9,91.6,5,5,244.9,244.9
|
||||
06-06 18:25:06,1,1,35.7,277.8,6,0,35.7,277.8,6.0,0.0,26.9,91.6,6,6,277.8,277.8
|
||||
06-06 18:25:11,1,1,0.0,292.6,6,0,0.0,292.6,6.0,0.0,27.3,91.6,6,6,292.6,292.6
|
||||
06-06 18:25:16,1,1,113.7,373.1,8,0,113.7,373.1,8.0,0.0,28.8,91.6,8,8,373.1,373.1
|
||||
06-06 18:25:21,1,1,0.0,389.0,8,0,0.0,389.0,8.0,0.0,29.4,91.6,8,8,389.0,389.0
|
||||
06-06 18:25:26,1,1,0.0,369.9,7,0,0.0,369.9,7.0,0.0,29.1,91.6,7,7,369.9,369.9
|
||||
06-06 18:25:31,1,1,0.0,343.3,7,0,0.0,343.3,7.0,0.0,29.6,91.6,7,7,343.3,343.3
|
||||
06-06 18:25:36,1,1,0.0,340.6,7,0,0.0,340.6,7.0,0.0,30.1,91.6,7,7,340.6,340.6
|
||||
06-06 18:25:41,1,1,1.6,416.7,10,0,1.6,416.7,10.0,0.0,39.8,87.3,10,10,416.7,416.7
|
||||
06-06 18:25:46,1,1,42.5,273.5,6,0,42.5,273.5,6.0,0.0,31.1,90.1,6,6,273.5,273.5
|
||||
06-06 18:25:51,1,1,9.6,317.8,7,0,9.6,317.8,7.0,0.0,31.6,90.1,7,7,317.8,317.8
|
||||
06-06 18:25:56,1,1,0.0,632.2,17,0,0.0,632.2,17.0,0.0,88.0,89.3,17,17,632.2,632.2
|
||||
06-06 18:26:01,1,1,144.9,651.0,16,0,144.9,651.0,16.0,0.0,87.9,89.2,16,16,651.0,651.0
|
||||
06-06 18:26:06,1,1,432.2,643.0,16,0,432.2,643.0,16.0,0.0,88.7,88.8,16,16,643.0,643.0
|
||||
06-06 18:26:11,1,1,0.0,666.3,16,0,0.0,666.3,16.0,0.0,89.7,88.8,16,16,666.3,666.3
|
||||
06-06 18:26:16,1,1,0.0,661.6,16,0,0.0,661.6,16.0,0.0,90.6,88.8,16,16,661.6,661.6
|
||||
06-06 18:26:21,1,1,0.0,656.4,16,0,0.0,656.4,16.0,0.0,91.5,88.8,16,16,656.4,656.4
|
||||
06-06 18:26:26,1,1,0.0,652.4,16,0,0.0,652.4,16.0,0.0,92.4,88.8,16,16,652.4,652.4
|
||||
06-06 18:26:31,1,1,0.0,488.8,11,0,0.0,488.8,11.0,0.0,59.4,88.8,11,11,488.8,488.8
|
||||
06-06 18:26:36,1,1,0.0,488.2,11,0,0.0,488.2,11.0,0.0,60.1,88.8,11,11,488.2,488.2
|
||||
06-06 18:26:41,1,1,0.0,640.2,16,0,0.0,640.2,16.0,0.0,95.1,88.8,16,16,640.2,640.2
|
||||
06-06 18:26:46,1,1,0.0,635.3,15,0,0.0,635.3,15.0,0.0,87.1,88.8,15,15,635.3,635.3
|
||||
06-06 18:26:51,1,1,217.2,640.0,17,0,217.2,640.0,17.0,0.0,88.3,88.7,17,17,640.0,640.0
|
||||
06-06 18:26:56,1,1,42.8,633.8,18,0,42.8,633.8,18.0,0.0,89.9,88.7,18,18,633.8,633.8
|
||||
06-06 18:27:01,1,1,188.8,656.7,16,0,188.8,656.7,16.0,0.0,90.3,88.6,16,16,656.7,656.7
|
||||
06-06 18:27:06,1,1,0.0,650.4,16,0,0.0,650.4,16.0,0.0,91.3,88.6,16,16,650.4,650.4
|
||||
06-06 18:27:11,1,1,2137.9,586.7,17,0,2137.9,586.7,17.0,0.0,95.1,86.8,17,17,586.7,586.7
|
||||
06-06 18:27:16,1,1,0.0,628.3,14,0,0.0,628.3,14.0,0.0,83.0,86.8,14,14,628.3,628.3
|
||||
06-06 18:27:21,1,1,0.0,492.0,12,0,0.0,492.0,12.0,0.0,49.1,89.3,12,12,492.0,492.0
|
||||
06-06 18:27:26,1,1,0.0,492.4,12,0,0.0,492.4,12.0,0.0,49.8,89.3,12,12,492.4,492.4
|
||||
06-06 18:27:31,1,1,0.0,491.6,12,0,0.0,491.6,12.0,0.0,50.5,89.3,12,12,491.6,491.6
|
||||
06-06 18:27:36,1,1,0.0,464.1,11,0,0.0,464.1,11.0,0.0,49.5,89.3,11,11,464.1,464.1
|
||||
06-06 18:27:41,1,1,0.0,452.0,11,0,0.0,452.0,11.0,0.0,50.1,89.3,11,11,452.0,452.0
|
||||
06-06 18:27:46,1,1,62.2,470.8,12,0,62.2,470.8,12.0,0.0,50.9,89.3,12,12,470.8,470.8
|
||||
06-06 18:27:51,1,1,0.0,477.7,11,0,0.0,477.7,11.0,0.0,51.4,89.3,11,11,477.7,477.7
|
||||
06-06 18:27:56,1,1,0.0,414.7,10,0,0.0,414.7,10.0,0.0,49.1,89.3,10,10,414.7,414.7
|
||||
06-06 18:28:02,1,1,0.0,406.1,10,0,0.0,406.1,10.0,0.0,49.7,89.3,10,10,406.1,406.1
|
||||
06-06 18:28:07,1,1,0.0,406.0,10,0,0.0,406.0,10.0,0.0,50.2,89.3,10,10,406.0,406.0
|
||||
06-06 18:28:12,1,1,0.0,405.9,10,0,0.0,405.9,10.0,0.0,50.8,89.3,10,10,405.9,405.9
|
||||
06-06 18:28:17,1,1,0.0,406.1,10,0,0.0,406.1,10.0,0.0,51.3,89.3,10,10,406.1,406.1
|
||||
06-06 18:28:22,1,1,2.8,443.4,11,0,2.8,443.4,11.0,0.0,52.0,89.3,11,11,443.4,443.4
|
||||
06-06 18:28:27,1,1,0.0,448.6,11,0,0.0,448.6,11.0,0.0,52.6,89.3,11,11,448.6,448.6
|
||||
06-06 18:28:32,1,1,0.0,448.9,11,0,0.0,448.9,11.0,0.0,53.2,89.3,11,11,448.9,448.9
|
||||
06-06 18:28:37,1,1,0.0,455.2,10,0,0.0,455.2,10.0,0.0,36.9,92.4,10,10,455.2,455.2
|
||||
06-06 18:28:42,1,1,0.0,427.4,9,0,0.0,427.4,9.0,0.0,28.5,92.4,9,9,427.4,427.4
|
||||
06-06 18:28:47,1,1,0.0,410.1,9,0,0.0,410.1,9.0,0.0,29.1,92.4,9,9,410.1,410.1
|
||||
06-06 18:28:52,1,1,0.0,410.1,9,0,0.0,410.1,9.0,0.0,29.7,92.4,9,9,410.1,410.1
|
||||
06-06 18:28:57,1,1,0.0,410.0,9,0,0.0,410.0,9.0,0.0,30.2,92.4,9,9,410.0,410.0
|
||||
06-06 18:29:02,1,1,0.0,408.6,9,0,0.0,408.6,9.0,0.0,30.8,92.4,9,9,408.6,408.6
|
||||
06-06 18:29:07,1,1,0.0,408.6,9,0,0.0,408.6,9.0,0.0,31.4,92.4,9,9,408.6,408.6
|
||||
06-06 18:29:12,1,1,0.0,408.5,9,0,0.0,408.5,9.0,0.0,32.0,92.4,9,9,408.5,408.5
|
||||
06-06 18:29:17,1,1,2.8,436.6,10,0,2.8,436.6,10.0,0.0,33.8,92.4,10,10,436.6,436.6
|
||||
06-06 18:29:22,1,1,0.0,453.5,10,0,0.0,453.5,10.0,0.0,34.4,92.4,10,10,453.5,453.5
|
||||
06-06 18:29:27,1,1,0.0,453.2,10,0,0.0,453.2,10.0,0.0,35.0,92.4,10,10,453.2,453.2
|
||||
06-06 18:29:32,1,1,0.0,453.5,10,0,0.0,453.5,10.0,0.0,35.7,92.4,10,10,453.5,453.5
|
||||
06-06 18:29:37,1,1,0.0,453.6,10,0,0.0,453.6,10.0,0.0,36.3,92.4,10,10,453.6,453.6
|
||||
06-06 18:29:42,1,1,42.9,443.9,11,0,42.9,443.9,11.0,0.0,29.0,89.0,11,11,443.9,443.9
|
||||
06-06 18:29:47,1,1,0.0,479.2,10,0,0.0,479.2,10.0,0.0,28.6,89.0,10,10,479.2,479.2
|
||||
06-06 18:29:52,1,1,45.4,440.9,10,0,45.4,440.9,10.0,0.0,27.5,89.1,10,10,440.9,440.9
|
||||
06-06 18:29:57,1,1,20.2,478.1,9,0,20.2,478.1,9.0,0.0,28.1,89.1,9,9,478.1,478.1
|
||||
06-06 18:30:02,1,1,0.0,411.8,8,0,0.0,411.8,8.0,0.0,44.2,92.5,8,8,411.8,411.8
|
||||
06-06 18:30:07,1,1,83.4,391.5,10,0,83.4,391.5,10.0,0.0,46.8,92.6,10,10,391.5,391.5
|
||||
06-06 18:30:12,1,1,35.3,469.6,9,0,35.3,469.6,9.0,0.0,44.3,92.6,9,9,469.6,469.6
|
||||
06-06 18:30:17,1,1,138.0,431.4,10,0,138.0,431.4,10.0,0.0,45.2,92.5,10,10,431.4,431.4
|
||||
06-06 18:30:22,1,1,0.0,501.4,11,0,0.0,501.4,11.0,0.0,42.4,92.3,11,11,501.4,501.4
|
||||
06-06 18:30:27,1,1,0.0,333.7,7,0,0.0,333.7,7.0,0.0,28.6,89.2,7,7,333.7,333.7
|
||||
06-06 18:30:32,1,1,21.3,343.6,8,0,21.3,343.6,8.0,0.0,29.7,89.3,8,8,343.6,343.6
|
||||
06-06 18:30:37,1,1,0.0,460.5,10,0,0.0,460.5,10.0,0.0,48.3,92.5,10,10,460.5,460.5
|
||||
06-06 18:30:42,1,1,0.0,460.2,10,0,0.0,460.2,10.0,0.0,49.0,92.5,10,10,460.2,460.2
|
||||
06-06 18:30:48,1,1,149.2,472.5,9,0,149.2,472.5,9.0,0.0,40.6,92.4,9,9,472.5,472.5
|
||||
06-06 18:30:53,1,1,78.2,426.6,10,0,78.2,426.6,10.0,0.0,41.8,92.4,10,10,426.6,426.6
|
||||
06-06 18:30:58,1,1,62.7,478.1,11,0,62.7,478.1,11.0,0.0,43.1,92.4,11,11,478.1,478.1
|
||||
06-06 18:31:03,1,1,0.0,506.0,11,0,0.0,506.0,11.0,0.0,43.8,92.4,11,11,506.0,506.0
|
||||
06-06 18:31:08,1,1,0.0,506.7,11,0,0.0,506.7,11.0,0.0,44.5,92.4,11,11,506.7,506.7
|
||||
06-06 18:31:13,1,1,25.5,524.9,12,0,25.5,524.9,12.0,0.0,47.4,92.5,12,12,524.9,524.9
|
||||
06-06 18:31:18,1,1,95.5,505.6,11,0,95.5,505.6,11.0,0.0,49.2,92.5,11,11,505.6,505.6
|
||||
06-06 18:31:23,1,1,0.0,504.7,11,0,0.0,504.7,11.0,0.0,49.9,92.5,11,11,504.7,504.7
|
||||
06-06 18:31:28,1,1,0.0,468.9,10,0,0.0,468.9,10.0,0.0,47.6,92.5,10,10,468.9,468.9
|
||||
06-06 18:31:33,1,1,0.0,451.5,9,0,0.0,451.5,9.0,0.0,39.4,92.5,9,9,451.5,451.5
|
||||
06-06 18:31:38,1,1,11.4,394.2,9,0,11.4,394.2,9.0,0.0,31.1,92.5,9,9,394.2,394.2
|
||||
06-06 18:31:43,1,1,92.3,429.4,10,0,92.3,429.4,10.0,0.0,32.7,92.5,10,10,429.4,429.4
|
||||
06-06 18:31:48,1,1,0.0,461.8,10,0,0.0,461.8,10.0,0.0,33.3,92.5,10,10,461.8,461.8
|
||||
06-06 18:31:53,1,1,0.0,461.9,10,0,0.0,461.9,10.0,0.0,34.0,92.5,10,10,461.9,461.9
|
||||
06-06 18:31:58,1,1,0.0,461.1,10,0,0.0,461.1,10.0,0.0,34.6,92.5,10,10,461.1,461.1
|
||||
06-06 18:32:03,1,1,0.0,439.2,9,0,0.0,439.2,9.0,0.0,34.0,92.5,9,9,439.2,439.2
|
||||
06-06 18:32:08,1,1,0.0,414.4,9,0,0.0,414.4,9.0,0.0,34.6,92.5,9,9,414.4,414.4
|
||||
06-06 18:32:13,1,1,0.0,414.0,9,0,0.0,414.0,9.0,0.0,35.2,92.5,9,9,414.0,414.0
|
||||
06-06 18:32:18,1,1,0.0,414.9,9,0,0.0,414.9,9.0,0.0,35.7,92.5,9,9,414.9,414.9
|
||||
06-06 18:32:23,1,1,0.0,414.8,7,0,0.0,414.8,7.0,0.0,17.9,90.6,7,7,414.8,414.8
|
||||
06-06 18:32:28,1,1,22.6,370.7,8,0,22.6,370.7,8.0,0.0,19.1,90.6,8,8,370.7,370.7
|
||||
06-06 18:32:33,1,1,0.0,376.9,8,0,0.0,376.9,8.0,0.0,19.6,90.6,8,8,376.9,376.9
|
||||
06-06 18:32:38,1,1,0.0,340.0,7,0,0.0,340.0,7.0,0.0,19.3,90.6,7,7,340.0,340.0
|
||||
06-06 18:32:43,1,1,0.0,285.1,6,0,0.0,285.1,6.0,0.0,19.3,90.6,6,6,285.1,285.1
|
||||
06-06 18:32:48,1,1,0.0,283.4,6,0,0.0,283.4,6.0,0.0,19.7,90.6,6,6,283.4,283.4
|
||||
06-06 18:32:53,1,1,88.3,346.0,8,0,88.3,346.0,8.0,0.0,20.4,90.6,8,8,346.0,346.0
|
||||
06-06 18:32:58,1,1,0.0,340.9,7,0,0.0,340.9,7.0,0.0,19.2,90.6,7,7,340.9,340.9
|
||||
06-06 18:33:03,1,1,0.0,317.5,6,0,0.0,317.5,6.0,0.0,19.4,90.6,6,6,317.5,317.5
|
||||
06-06 18:33:08,1,1,0.0,283.4,6,0,0.0,283.4,6.0,0.0,19.8,90.6,6,6,283.4,283.4
|
||||
06-06 18:33:13,1,1,0.0,283.7,6,0,0.0,283.7,6.0,0.0,20.2,90.6,6,6,283.7,283.7
|
||||
06-06 18:33:18,1,1,0.0,283.6,6,0,0.0,283.6,6.0,0.0,20.6,90.6,6,6,283.6,283.6
|
||||
06-06 18:33:23,1,1,8.2,284.5,7,0,8.2,284.5,7.0,0.0,21.1,90.6,7,7,284.5,284.5
|
||||
06-06 18:33:28,1,1,82.0,366.2,8,0,82.0,366.2,8.0,0.0,25.5,90.7,8,8,366.2,366.2
|
||||
06-06 18:33:33,1,1,9.4,363.1,8,0,9.4,363.1,8.0,0.0,22.2,90.7,8,8,363.1,363.1
|
||||
06-06 18:33:38,1,1,0.0,343.5,7,0,0.0,343.5,7.0,0.0,22.5,90.7,7,7,343.5,343.5
|
||||
06-06 18:33:43,1,1,0.0,331.0,7,0,0.0,331.0,7.0,0.0,22.9,90.7,7,7,331.0,331.0
|
||||
06-06 18:33:48,1,1,0.0,345.2,7,0,0.0,345.2,7.0,0.0,48.8,91.4,7,7,345.2,345.2
|
||||
06-06 18:33:53,1,1,47.6,333.0,8,0,47.6,333.0,8.0,0.0,49.5,91.4,8,8,333.0,333.0
|
||||
06-06 18:33:58,1,1,0.0,361.8,8,0,0.0,361.8,8.0,0.0,50.0,91.4,8,8,361.8,361.8
|
||||
06-06 18:34:03,1,1,0.2,381.9,9,0,0.2,381.9,9.0,0.0,50.5,91.4,9,9,381.9,381.9
|
||||
06-06 18:34:08,1,1,9.4,441.1,10,0,9.4,441.1,10.0,0.0,51.3,91.4,10,10,441.1,441.1
|
||||
06-06 18:34:13,1,1,0.0,448.9,10,0,0.0,448.9,10.0,0.0,51.9,91.4,10,10,448.9,448.9
|
||||
06-06 18:34:18,1,1,0.0,449.9,10,0,0.0,449.9,10.0,0.0,52.6,91.4,10,10,449.9,449.9
|
||||
06-06 18:34:24,1,1,23.1,573.1,13,0,23.1,573.1,13.0,0.0,53.9,92.2,13,13,573.1,573.1
|
||||
06-06 18:34:29,1,1,0.0,563.8,12,0,0.0,563.8,12.0,0.0,54.2,92.2,12,12,563.8,563.8
|
||||
06-06 18:34:34,1,1,0.0,551.3,12,0,0.0,551.3,12.0,0.0,55.0,92.2,12,12,551.3,551.3
|
||||
06-06 18:34:39,1,1,0.0,551.6,12,0,0.0,551.6,12.0,0.0,55.7,92.2,12,12,551.6,551.6
|
||||
06-06 18:34:44,1,1,0.0,550.7,12,0,0.0,550.7,12.0,0.0,56.5,92.2,12,12,550.7,550.7
|
||||
06-06 18:34:49,1,1,0.0,694.0,16,0,0.0,694.0,16.0,0.0,60.4,91.0,16,16,694.0,694.0
|
||||
06-06 18:34:54,1,1,107.2,643.2,17,0,107.2,643.2,17.0,0.0,61.9,90.9,17,17,643.2,643.2
|
||||
06-06 18:34:59,1,1,0.0,631.9,17,0,0.0,631.9,17.0,0.0,62.8,90.9,17,17,631.9,631.9
|
||||
06-06 18:35:04,1,1,35.9,633.9,17,0,35.9,633.9,17.0,0.0,63.3,90.9,17,17,633.9,633.9
|
||||
06-06 18:35:09,1,1,0.0,634.8,17,0,0.0,634.8,17.0,0.0,64.2,90.9,17,17,634.8,634.8
|
||||
06-06 18:35:14,1,1,0.0,676.4,16,0,0.0,676.4,16.0,0.0,64.3,90.9,16,16,676.4,676.4
|
||||
06-06 18:35:19,1,1,0.0,660.5,14,0,0.0,660.5,14.0,0.0,64.5,90.9,14,14,660.5,660.5
|
||||
06-06 18:35:24,1,1,0.0,581.1,13,0,0.0,581.1,13.0,0.0,64.8,90.9,13,13,581.1,581.1
|
||||
06-06 18:35:29,1,1,0.0,564.7,13,0,0.0,564.7,13.0,0.0,65.6,90.9,13,13,564.7,564.7
|
||||
06-06 18:35:34,1,1,1.4,598.2,14,0,1.4,598.2,14.0,0.0,66.4,91.0,14,14,598.2,598.2
|
||||
06-06 18:35:39,1,1,0.0,610.0,14,0,0.0,610.0,14.0,0.0,67.3,91.0,14,14,610.0,610.0
|
||||
06-06 18:35:44,1,1,0.0,609.7,14,0,0.0,609.7,14.0,0.0,68.1,91.0,14,14,609.7,609.7
|
||||
06-06 18:35:49,1,1,102.1,656.1,16,0,102.1,656.1,16.0,0.0,70.0,91.0,16,16,656.1,656.1
|
||||
06-06 18:35:54,1,1,132.5,655.4,18,0,132.5,655.4,18.0,0.0,72.2,91.0,18,18,655.4,655.4
|
||||
06-06 18:35:59,1,1,134.7,728.3,19,0,134.7,728.3,19.0,0.0,72.9,91.0,19,19,728.3,728.3
|
||||
06-06 18:36:04,1,1,0.0,667.5,17,0,0.0,667.5,17.0,0.0,65.8,91.0,17,17,667.5,667.5
|
||||
06-06 18:36:09,1,1,69.5,634.7,17,0,69.5,634.7,17.0,0.0,66.1,91.0,17,17,634.7,634.7
|
||||
06-06 18:36:14,1,1,0.0,360.3,8,0,0.0,360.3,8.0,0.0,48.0,90.4,8,8,360.3,360.3
|
||||
06-06 18:36:19,1,1,0.0,360.2,8,0,0.0,360.2,8.0,0.0,48.5,90.4,8,8,360.2,360.2
|
||||
06-06 18:36:24,1,1,0.0,358.8,8,0,0.0,358.8,8.0,0.0,49.0,90.4,8,8,358.8,358.8
|
||||
06-06 18:36:29,1,1,0.0,359.6,8,0,0.0,359.6,8.0,0.0,49.5,90.4,8,8,359.6,359.6
|
||||
06-06 18:36:34,1,1,152.3,372.2,9,0,152.3,372.2,9.0,0.0,50.8,90.3,9,9,372.2,372.2
|
||||
06-06 18:36:39,1,1,58.0,380.6,9,0,58.0,380.6,9.0,0.0,31.0,91.5,9,9,380.6,380.6
|
||||
06-06 18:36:44,1,1,0.0,400.6,9,0,0.0,400.6,9.0,0.0,31.6,91.5,9,9,400.6,400.6
|
||||
06-06 18:36:49,1,1,0.0,400.5,9,0,0.0,400.5,9.0,0.0,32.1,91.5,9,9,400.5,400.5
|
||||
06-06 18:36:54,1,1,0.0,369.6,8,0,0.0,369.6,8.0,0.0,32.0,91.5,8,8,369.6,369.6
|
||||
06-06 18:36:59,1,1,67.6,353.4,8,0,67.6,353.4,8.0,0.0,33.8,91.6,8,8,353.4,353.4
|
||||
06-06 18:37:04,1,1,0.0,333.3,7,0,0.0,333.3,7.0,0.0,32.6,91.6,7,7,333.3,333.3
|
||||
06-06 18:37:09,1,1,0.0,688.0,18,0,0.0,688.0,18.0,0.0,67.6,89.0,18,18,688.0,688.0
|
||||
06-06 18:37:14,1,1,194.5,663.9,18,0,194.5,663.9,18.0,0.0,69.7,89.1,18,18,663.9,663.9
|
||||
06-06 18:37:19,1,1,0.0,664.0,18,0,0.0,664.0,18.0,0.0,70.7,89.1,18,18,664.0,664.0
|
||||
06-06 18:37:24,1,1,154.4,455.3,11,0,154.4,455.3,11.0,0.0,36.8,91.5,11,11,455.3,455.3
|
||||
06-06 18:37:29,1,1,184.0,505.9,13,0,184.0,505.9,13.0,0.0,39.5,91.5,13,13,505.9,505.9
|
||||
06-06 18:37:34,1,1,0.4,522.0,12,0,0.4,522.0,12.0,0.0,39.2,91.6,12,12,522.0,522.0
|
||||
06-06 18:37:39,1,1,0.0,507.6,11,0,0.0,507.6,11.0,0.0,39.5,91.6,11,11,507.6,507.6
|
||||
06-06 18:37:44,1,1,21.5,639.9,16,0,21.5,639.9,16.0,0.0,49.5,90.9,16,16,639.9,639.9
|
||||
06-06 18:37:49,1,1,0.0,491.3,11,0,0.0,491.3,11.0,0.0,39.2,91.6,11,11,491.3,491.3
|
||||
06-06 18:37:54,1,1,212.5,471.6,13,0,212.5,471.6,13.0,0.0,40.9,91.5,13,13,471.6,471.6
|
||||
06-06 18:37:59,1,1,33.6,591.3,14,0,33.6,591.3,14.0,0.0,41.9,91.5,14,14,591.3,591.3
|
||||
06-06 18:38:04,1,1,0.0,616.3,13,0,0.0,616.3,13.0,0.0,42.3,91.5,13,13,616.3,616.3
|
||||
06-06 18:38:09,1,1,0.0,576.3,13,0,0.0,576.3,13.0,0.0,43.1,91.5,13,13,576.3,576.3
|
||||
06-06 18:38:14,1,1,0.0,561.7,12,0,0.0,561.7,12.0,0.0,43.2,91.5,12,12,561.7,561.7
|
||||
06-06 18:38:19,1,1,225.5,504.9,12,0,225.5,504.9,12.0,0.0,44.5,91.4,12,12,504.9,504.9
|
||||
06-06 18:38:24,1,1,10.4,535.8,12,0,10.4,535.8,12.0,0.0,44.2,91.5,12,12,535.8,535.8
|
||||
06-06 18:38:30,1,1,54.9,579.8,14,0,54.9,579.8,14.0,0.0,45.5,91.5,14,14,579.8,579.8
|
||||
06-06 18:38:35,1,1,36.2,647.5,15,0,36.2,647.5,15.0,0.0,47.4,91.5,15,15,647.5,647.5
|
||||
06-06 18:38:40,1,1,0.0,665.7,15,0,0.0,665.7,15.0,0.0,48.4,91.5,15,15,665.7,665.7
|
||||
06-06 18:38:45,1,1,171.4,305.5,7,0,171.4,305.5,7.0,0.0,26.3,90.5,7,7,305.5,305.5
|
||||
06-06 18:38:50,1,1,154.1,321.3,8,0,154.1,321.3,8.0,0.0,27.0,90.4,8,8,321.3,321.3
|
||||
06-06 18:38:55,1,1,0.0,355.4,8,0,0.0,355.4,8.0,0.0,27.5,90.4,8,8,355.4,355.4
|
||||
06-06 18:39:00,1,1,0.0,329.4,7,0,0.0,329.4,7.0,0.0,27.6,90.4,7,7,329.4,329.4
|
||||
06-06 18:39:05,1,1,0.0,312.5,7,0,0.0,312.5,7.0,0.0,28.0,90.4,7,7,312.5,312.5
|
||||
06-06 18:39:09,1,1,0.0,672.7,14,0,0.0,672.7,14.0,0.0,58.4,90.9,14,14,672.7,672.7
|
||||
|
8527
training_logs/20260608_210110_vllm_metrics_job_639470.csv
Normal file
8527
training_logs/20260608_210110_vllm_metrics_job_639470.csv
Normal file
File diff suppressed because it is too large
Load Diff
8495
training_logs/20260608_210110_vllm_metrics_job_639471.csv
Normal file
8495
training_logs/20260608_210110_vllm_metrics_job_639471.csv
Normal file
File diff suppressed because it is too large
Load Diff
6376
training_logs/20260608_210110_vllm_metrics_job_639473.csv
Normal file
6376
training_logs/20260608_210110_vllm_metrics_job_639473.csv
Normal file
File diff suppressed because it is too large
Load Diff
7784
training_logs/20260608_210110_vllm_metrics_job_639474.csv
Normal file
7784
training_logs/20260608_210110_vllm_metrics_job_639474.csv
Normal file
File diff suppressed because it is too large
Load Diff
557
training_logs/20260608_210110_vllm_metrics_job_639475.csv
Normal file
557
training_logs/20260608_210110_vllm_metrics_job_639475.csv
Normal file
@@ -0,0 +1,557 @@
|
||||
datetime_str,n_engines_reporting,unique_ips,total_prompt_throughput_tokens_per_sec,total_generation_throughput_tokens_per_sec,total_running_requests,total_waiting_requests,avg_prompt_throughput_per_engine,avg_generation_throughput_per_engine,avg_running_requests_per_engine,avg_waiting_requests_per_engine,avg_gpu_kv_cache_usage_pct,avg_prefix_cache_hit_rate_pct,min_running_requests,max_running_requests,min_generation_throughput,max_generation_throughput
|
||||
06-08 19:49:31,1,1,10.6,0.0,1,0,10.6,0.0,1.0,0.0,0.3,0.0,1,1,0.0,0.0
|
||||
06-08 19:49:36,1,1,0.0,45.7,1,0,0.0,45.7,1.0,0.0,0.3,0.0,1,1,45.7,45.7
|
||||
06-08 19:49:41,1,1,0.0,45.9,1,0,0.0,45.9,1.0,0.0,0.4,0.0,1,1,45.9,45.9
|
||||
06-08 19:49:46,1,1,0.0,45.9,1,0,0.0,45.9,1.0,0.0,0.5,0.0,1,1,45.9,45.9
|
||||
06-08 19:49:51,1,1,0.0,43.7,1,0,0.0,43.7,1.0,0.0,0.5,0.0,1,1,43.7,43.7
|
||||
06-08 19:49:56,1,1,0.0,38.2,1,0,0.0,38.2,1.0,0.0,0.3,0.0,1,1,38.2,38.2
|
||||
06-08 19:50:01,1,1,0.0,47.7,1,0,0.0,47.7,1.0,0.0,0.4,0.0,1,1,47.7,47.7
|
||||
06-08 19:50:06,1,1,0.0,38.7,1,0,0.0,38.7,1.0,0.0,0.4,0.0,1,1,38.7,38.7
|
||||
06-08 19:50:11,1,1,0.0,40.1,1,0,0.0,40.1,1.0,0.0,0.5,0.0,1,1,40.1,40.1
|
||||
06-08 19:50:16,1,1,0.0,38.8,1,0,0.0,38.8,1.0,0.0,0.5,0.0,1,1,38.8,38.8
|
||||
06-08 19:50:21,1,1,0.0,38.1,1,0,0.0,38.1,1.0,0.0,0.6,0.0,1,1,38.1,38.1
|
||||
06-08 19:50:26,1,1,0.0,39.1,1,0,0.0,39.1,1.0,0.0,0.6,0.0,1,1,39.1,39.1
|
||||
06-08 19:50:31,1,1,0.0,39.6,1,0,0.0,39.6,1.0,0.0,0.7,0.0,1,1,39.6,39.6
|
||||
06-08 19:50:36,1,1,0.0,39.6,1,0,0.0,39.6,1.0,0.0,0.7,0.0,1,1,39.6,39.6
|
||||
06-08 19:50:41,1,1,0.0,38.4,1,0,0.0,38.4,1.0,0.0,0.8,0.0,1,1,38.4,38.4
|
||||
06-08 19:50:46,1,1,50.6,43.6,2,0,50.6,43.6,2.0,0.0,0.9,30.2,2,2,43.6,43.6
|
||||
06-08 19:50:51,1,1,0.0,61.1,1,0,0.0,61.1,1.0,0.0,0.3,30.2,1,1,61.1,61.1
|
||||
06-08 19:50:56,1,1,56.1,65.7,2,0,56.1,65.7,2.0,0.0,1.2,67.7,2,2,65.7,65.7
|
||||
06-08 19:51:01,1,1,0.0,91.8,2,0,0.0,91.8,2.0,0.0,1.3,67.7,2,2,91.8,91.8
|
||||
06-08 19:51:06,1,1,116.6,78.9,2,0,116.6,78.9,2.0,0.0,1.6,69.4,2,2,78.9,78.9
|
||||
06-08 19:51:11,1,1,0.0,46.6,1,0,0.0,46.6,1.0,0.0,1.1,69.4,1,1,46.6,46.6
|
||||
06-08 19:51:16,1,1,0.0,44.3,1,0,0.0,44.3,1.0,0.0,1.1,74.1,1,1,44.3,44.3
|
||||
06-08 19:51:22,1,1,0.0,45.9,1,0,0.0,45.9,1.0,0.0,1.7,70.2,1,1,45.9,45.9
|
||||
06-08 19:51:27,1,1,0.0,45.9,1,0,0.0,45.9,1.0,0.0,1.8,70.2,1,1,45.9,45.9
|
||||
06-08 19:51:32,1,1,0.0,45.4,1,0,0.0,45.4,1.0,0.0,1.8,70.2,1,1,45.4,45.4
|
||||
06-08 19:51:37,1,1,0.0,45.1,1,0,0.0,45.1,1.0,0.0,1.9,70.2,1,1,45.1,45.1
|
||||
06-08 19:51:42,1,1,0.0,45.1,1,0,0.0,45.1,1.0,0.0,1.9,70.2,1,1,45.1,45.1
|
||||
06-08 19:51:47,1,1,0.0,45.0,1,0,0.0,45.0,1.0,0.0,2.0,70.2,1,1,45.0,45.0
|
||||
06-08 19:51:52,1,1,0.0,91.4,2,0,0.0,91.4,2.0,0.0,2.5,83.6,2,2,91.4,91.4
|
||||
06-08 19:51:57,1,1,0.0,91.1,2,0,0.0,91.1,2.0,0.0,2.7,83.6,2,2,91.1,91.1
|
||||
06-08 19:52:02,1,1,0.0,92.4,2,0,0.0,92.4,2.0,0.0,2.8,83.6,2,2,92.4,92.4
|
||||
06-08 19:52:07,1,1,0.0,94.4,2,0,0.0,94.4,2.0,0.0,2.9,83.6,2,2,94.4,94.4
|
||||
06-08 19:52:12,1,1,0.0,47.8,1,0,0.0,47.8,1.0,0.0,1.3,80.6,1,1,47.8,47.8
|
||||
06-08 19:52:17,1,1,0.0,147.1,3,0,0.0,147.1,3.0,0.0,3.7,77.3,3,3,147.1,147.1
|
||||
06-08 19:52:22,1,1,0.0,146.8,3,0,0.0,146.8,3.0,0.0,3.9,77.3,3,3,146.8,146.8
|
||||
06-08 19:52:27,1,1,0.0,146.9,3,0,0.0,146.9,3.0,0.0,4.1,77.3,3,3,146.9,146.9
|
||||
06-08 19:52:32,1,1,0.0,147.1,3,0,0.0,147.1,3.0,0.0,4.3,77.3,3,3,147.1,147.1
|
||||
06-08 19:52:37,1,1,0.0,135.6,2,0,0.0,135.6,2.0,0.0,3.9,77.3,2,2,135.6,135.6
|
||||
06-08 19:52:42,1,1,59.0,136.9,3,0,59.0,136.9,3.0,0.0,4.8,79.0,3,3,136.9,136.9
|
||||
06-08 19:52:47,1,1,0.0,147.2,3,0,0.0,147.2,3.0,0.0,5.0,79.0,3,3,147.2,147.2
|
||||
06-08 19:52:52,1,1,0.0,147.3,3,0,0.0,147.3,3.0,0.0,5.2,79.0,3,3,147.3,147.3
|
||||
06-08 19:52:57,1,1,69.3,178.9,3,0,69.3,178.9,3.0,0.0,4.6,78.4,3,3,178.9,178.9
|
||||
06-08 19:53:02,1,1,0.0,147.2,3,0,0.0,147.2,3.0,0.0,4.9,78.4,3,3,147.2,147.2
|
||||
06-08 19:53:07,1,1,0.0,146.8,3,0,0.0,146.8,3.0,0.0,5.1,78.4,3,3,146.8,146.8
|
||||
06-08 19:53:12,1,1,143.3,188.5,4,0,143.3,188.5,4.0,0.0,6.2,78.8,4,4,188.5,188.5
|
||||
06-08 19:53:17,1,1,0.0,195.5,4,0,0.0,195.5,4.0,0.0,6.5,78.8,4,4,195.5,195.5
|
||||
06-08 19:53:22,1,1,0.0,195.0,4,0,0.0,195.0,4.0,0.0,6.7,78.8,4,4,195.0,195.0
|
||||
06-08 19:53:27,1,1,94.0,202.6,5,0,94.0,202.6,5.0,0.0,7.1,78.0,5,5,202.6,202.6
|
||||
06-08 19:53:32,1,1,0.0,211.3,4,0,0.0,211.3,4.0,0.0,6.9,78.0,4,4,211.3,211.3
|
||||
06-08 19:53:37,1,1,139.0,219.5,5,0,139.0,219.5,5.0,0.0,7.5,77.1,5,5,219.5,219.5
|
||||
06-08 19:53:42,1,1,0.0,245.3,5,0,0.0,245.3,5.0,0.0,7.9,77.1,5,5,245.3,245.3
|
||||
06-08 19:53:48,1,1,0.0,132.2,2,0,0.0,132.2,2.0,0.0,5.0,92.4,2,2,132.2,132.2
|
||||
06-08 19:53:53,1,1,119.1,82.4,2,0,119.1,82.4,2.0,0.0,4.9,92.4,2,2,82.4,82.4
|
||||
06-08 19:53:58,1,1,0.0,69.8,1,0,0.0,69.8,1.0,0.0,2.8,92.4,1,1,69.8,69.8
|
||||
06-08 19:54:03,1,1,157.0,90.9,3,0,157.0,90.9,3.0,0.0,7.0,92.7,3,3,90.9,90.9
|
||||
06-08 19:54:08,1,1,0.0,133.7,3,0,0.0,133.7,3.0,0.0,7.2,92.7,3,3,133.7,133.7
|
||||
06-08 19:54:13,1,1,173.8,161.0,4,0,173.8,161.0,4.0,0.0,8.0,92.6,4,4,161.0,161.0
|
||||
06-08 19:54:18,1,1,110.3,195.0,3,0,110.3,195.0,3.0,0.0,5.9,92.7,3,3,195.0,195.0
|
||||
06-08 19:54:23,1,1,0.0,124.4,2,0,0.0,124.4,2.0,0.0,5.8,92.7,2,2,124.4,124.4
|
||||
06-08 19:54:28,1,1,0.0,210.8,4,0,0.0,210.8,4.0,0.0,9.9,78.9,4,4,210.8,210.8
|
||||
06-08 19:54:33,1,1,0.0,88.9,2,0,0.0,88.9,2.0,0.0,6.1,92.7,2,2,88.9,88.9
|
||||
06-08 19:54:38,1,1,120.8,76.8,3,0,120.8,76.8,3.0,0.0,6.0,92.8,3,3,76.8,76.8
|
||||
06-08 19:54:43,1,1,0.0,114.7,2,0,0.0,114.7,2.0,0.0,5.8,92.8,2,2,114.7,114.7
|
||||
06-08 19:54:48,1,1,150.1,93.4,2,0,150.1,93.4,2.0,0.0,6.1,92.7,2,2,93.4,93.4
|
||||
06-08 19:54:53,1,1,148.3,118.4,4,0,148.3,118.4,4.0,0.0,11.6,93.1,4,4,118.4,118.4
|
||||
06-08 19:54:58,1,1,0.0,161.9,3,0,0.0,161.9,3.0,0.0,9.3,93.1,3,3,161.9,161.9
|
||||
06-08 19:55:03,1,1,34.9,167.7,4,0,34.9,167.7,4.0,0.0,10.0,93.1,4,4,167.7,167.7
|
||||
06-08 19:55:08,1,1,0.0,177.7,4,0,0.0,177.7,4.0,0.0,10.3,93.1,4,4,177.7,177.7
|
||||
06-08 19:55:13,1,1,0.0,177.5,4,0,0.0,177.5,4.0,0.0,10.5,93.1,4,4,177.5,177.5
|
||||
06-08 19:55:18,1,1,0.0,136.8,3,0,0.0,136.8,3.0,0.0,10.1,93.1,3,3,136.8,136.8
|
||||
06-08 19:55:23,1,1,0.0,78.3,1,0,0.0,78.3,1.0,0.0,0.4,85.1,1,1,78.3,78.3
|
||||
06-08 19:55:28,1,1,0.0,45.6,1,0,0.0,45.6,1.0,0.0,0.5,85.1,1,1,45.6,45.6
|
||||
06-08 19:55:33,1,1,199.2,69.5,2,0,199.2,69.5,2.0,0.0,2.4,85.2,2,2,69.5,69.5
|
||||
06-08 19:55:38,1,1,0.0,90.4,2,0,0.0,90.4,2.0,0.0,2.6,85.2,2,2,90.4,90.4
|
||||
06-08 19:55:43,1,1,0.0,90.4,2,0,0.0,90.4,2.0,0.0,2.7,85.2,2,2,90.4,90.4
|
||||
06-08 19:55:48,1,1,0.0,90.9,2,0,0.0,90.9,2.0,0.0,2.8,85.2,2,2,90.9,90.9
|
||||
06-08 19:55:53,1,1,0.0,90.6,2,0,0.0,90.6,2.0,0.0,2.9,85.2,2,2,90.6,90.6
|
||||
06-08 19:55:58,1,1,0.0,90.9,2,0,0.0,90.9,2.0,0.0,3.1,85.2,2,2,90.9,90.9
|
||||
06-08 19:56:03,1,1,0.0,47.4,1,0,0.0,47.4,1.0,0.0,0.9,77.3,1,1,47.4,47.4
|
||||
06-08 19:56:08,1,1,24.3,87.8,2,0,24.3,87.8,2.0,0.0,1.4,80.0,2,2,87.8,87.8
|
||||
06-08 19:56:13,1,1,0.0,139.8,4,0,0.0,139.8,4.0,0.0,10.9,86.2,4,4,139.8,139.8
|
||||
06-08 19:56:18,1,1,72.7,186.3,4,0,72.7,186.3,4.0,0.0,11.1,86.2,4,4,186.3,186.3
|
||||
06-08 19:56:23,1,1,81.2,167.6,4,0,81.2,167.6,4.0,0.0,7.1,81.3,4,4,167.6,167.6
|
||||
06-08 19:56:28,1,1,70.9,203.6,5,0,70.9,203.6,5.0,0.0,7.9,81.6,5,5,203.6,203.6
|
||||
06-08 19:56:33,1,1,0.0,195.1,3,0,0.0,195.1,3.0,0.0,7.0,81.6,3,3,195.1,195.1
|
||||
06-08 19:56:38,1,1,0.0,140.1,3,0,0.0,140.1,3.0,0.0,7.2,81.6,3,3,140.1,140.1
|
||||
06-08 19:56:43,1,1,0.0,143.4,3,0,0.0,143.4,3.0,0.0,8.0,92.5,3,3,143.4,143.4
|
||||
06-08 19:56:48,1,1,252.2,220.1,5,0,252.2,220.1,5.0,0.0,12.4,92.5,5,5,220.1,220.1
|
||||
06-08 19:56:54,1,1,0.0,184.2,4,0,0.0,184.2,4.0,0.0,10.5,83.3,4,4,184.2,184.2
|
||||
06-08 19:56:59,1,1,121.9,181.3,4,0,121.9,181.3,4.0,0.0,10.8,83.2,4,4,181.3,181.3
|
||||
06-08 19:57:04,1,1,46.9,180.4,4,0,46.9,180.4,4.0,0.0,11.0,83.6,4,4,180.4,180.4
|
||||
06-08 19:57:09,1,1,333.0,218.5,6,0,333.0,218.5,6.0,0.0,13.9,84.2,6,6,218.5,218.5
|
||||
06-08 19:57:14,1,1,23.0,253.2,5,0,23.0,253.2,5.0,0.0,13.6,84.2,5,5,253.2,253.2
|
||||
06-08 19:57:19,1,1,0.0,188.0,3,0,0.0,188.0,3.0,0.0,10.7,84.2,3,3,188.0,188.0
|
||||
06-08 19:57:24,1,1,160.9,160.9,5,0,160.9,160.9,5.0,0.0,12.6,84.6,5,5,160.9,160.9
|
||||
06-08 19:57:29,1,1,120.6,233.4,5,0,120.6,233.4,5.0,0.0,14.6,85.2,5,5,233.4,233.4
|
||||
06-08 19:57:34,1,1,0.0,202.3,4,0,0.0,202.3,4.0,0.0,13.8,85.2,4,4,202.3,202.3
|
||||
06-08 19:57:39,1,1,0.0,173.1,4,0,0.0,173.1,4.0,0.0,14.0,85.2,4,4,173.1,173.1
|
||||
06-08 19:57:44,1,1,91.3,174.7,5,0,91.3,174.7,5.0,0.0,15.1,85.3,5,5,174.7,174.7
|
||||
06-08 19:57:49,1,1,75.1,230.2,6,0,75.1,230.2,6.0,0.0,16.6,85.6,6,6,230.2,230.2
|
||||
06-08 19:57:54,1,1,62.2,45.9,2,0,62.2,45.9,2.0,0.0,5.5,85.5,2,2,45.9,45.9
|
||||
06-08 19:57:59,1,1,0.0,85.0,2,0,0.0,85.0,2.0,0.0,5.6,85.5,2,2,85.0,85.0
|
||||
06-08 19:58:04,1,1,0.0,72.8,1,0,0.0,72.8,1.0,0.0,5.3,85.5,1,1,72.8,72.8
|
||||
06-08 19:58:09,1,1,0.0,42.9,1,0,0.0,42.9,1.0,0.0,5.4,85.5,1,1,42.9,42.9
|
||||
06-08 19:58:14,1,1,0.0,43.4,1,0,0.0,43.4,1.0,0.0,5.4,85.5,1,1,43.4,43.4
|
||||
06-08 19:58:19,1,1,0.0,155.0,4,0,0.0,155.0,4.0,0.0,15.3,86.0,4,4,155.0,155.0
|
||||
06-08 19:58:24,1,1,0.0,153.2,4,0,0.0,153.2,4.0,0.0,15.5,86.0,4,4,153.2,153.2
|
||||
06-08 19:58:29,1,1,0.0,43.4,1,0,0.0,43.4,1.0,0.0,5.6,85.5,1,1,43.4,43.4
|
||||
06-08 19:58:34,1,1,23.1,135.1,4,0,23.1,135.1,4.0,0.0,15.8,86.7,4,4,135.1,135.1
|
||||
06-08 19:58:39,1,1,0.0,43.6,1,0,0.0,43.6,1.0,0.0,5.7,85.5,1,1,43.6,43.6
|
||||
06-08 19:58:44,1,1,0.0,152.2,4,0,0.0,152.2,4.0,0.0,16.3,86.7,4,4,152.2,152.2
|
||||
06-08 19:58:49,1,1,68.6,81.5,2,0,68.6,81.5,2.0,0.0,6.4,85.5,2,2,81.5,81.5
|
||||
06-08 19:58:54,1,1,0.0,116.3,3,0,0.0,116.3,3.0,0.0,15.0,86.7,3,3,116.3,116.3
|
||||
06-08 19:58:59,1,1,0.0,119.7,3,0,0.0,119.7,3.0,0.0,15.2,86.7,3,3,119.7,119.7
|
||||
06-08 19:59:04,1,1,0.0,116.8,3,0,0.0,116.8,3.0,0.0,15.3,86.7,3,3,116.8,116.8
|
||||
06-08 19:59:09,1,1,0.0,115.9,3,0,0.0,115.9,3.0,0.0,15.5,86.7,3,3,115.9,115.9
|
||||
06-08 19:59:14,1,1,0.0,43.3,1,0,0.0,43.3,1.0,0.0,6.1,85.9,1,1,43.3,43.3
|
||||
06-08 19:59:19,1,1,0.0,117.1,3,0,0.0,117.1,3.0,0.0,15.8,86.7,3,3,117.1,117.1
|
||||
06-08 19:59:24,1,1,0.0,116.1,4,0,0.0,116.1,4.0,0.0,16.1,86.4,4,4,116.1,116.1
|
||||
06-08 19:59:29,1,1,77.8,153.6,4,0,77.8,153.6,4.0,0.0,16.3,86.4,4,4,153.6,153.6
|
||||
06-08 19:59:34,1,1,0.0,156.6,4,0,0.0,156.6,4.0,0.0,16.5,86.4,4,4,156.6,156.6
|
||||
06-08 19:59:39,1,1,0.0,153.2,4,0,0.0,153.2,4.0,0.0,16.7,86.4,4,4,153.2,153.2
|
||||
06-08 19:59:44,1,1,75.8,153.2,4,0,75.8,153.2,4.0,0.0,16.8,86.2,4,4,153.2,153.2
|
||||
06-08 19:59:49,1,1,86.7,193.4,6,0,86.7,193.4,6.0,0.0,17.4,86.2,6,6,193.4,193.4
|
||||
06-08 19:59:54,1,1,61.2,263.6,7,0,61.2,263.6,7.0,0.0,17.9,86.0,7,7,263.6,263.6
|
||||
06-08 19:59:59,1,1,224.0,120.5,4,0,224.0,120.5,4.0,0.0,6.0,86.9,4,4,120.5,120.5
|
||||
06-08 20:00:04,1,1,34.9,135.9,3,0,34.9,135.9,3.0,0.0,6.1,86.9,3,3,135.9,135.9
|
||||
06-08 20:00:09,1,1,255.5,157.3,3,0,255.5,157.3,3.0,0.0,6.3,87.2,3,3,157.3,157.3
|
||||
06-08 20:00:14,1,1,0.0,136.7,2,0,0.0,136.7,2.0,0.0,6.1,87.2,2,2,136.7,136.7
|
||||
06-08 20:00:19,1,1,125.4,151.8,4,0,125.4,151.8,4.0,0.0,10.2,88.0,4,4,151.8,151.8
|
||||
06-08 20:00:24,1,1,74.5,199.5,5,0,74.5,199.5,5.0,0.0,11.0,88.0,5,5,199.5,199.5
|
||||
06-08 20:00:29,1,1,0.0,183.1,3,0,0.0,183.1,3.0,0.0,10.4,88.0,3,3,183.1,183.1
|
||||
06-08 20:00:34,1,1,0.0,284.4,6,0,0.0,284.4,6.0,0.0,13.0,88.9,6,6,284.4,284.4
|
||||
06-08 20:00:39,1,1,0.0,283.8,6,0,0.0,283.8,6.0,0.0,13.4,88.9,6,6,283.8,283.8
|
||||
06-08 20:00:44,1,1,0.0,89.8,2,0,0.0,89.8,2.0,0.0,1.9,79.0,2,2,89.8,89.8
|
||||
06-08 20:00:49,1,1,0.0,89.7,2,0,0.0,89.7,2.0,0.0,2.0,79.0,2,2,89.7,89.7
|
||||
06-08 20:00:54,1,1,0.0,90.2,2,0,0.0,90.2,2.0,0.0,2.2,79.0,2,2,90.2,90.2
|
||||
06-08 20:00:59,1,1,51.6,107.7,3,0,51.6,107.7,3.0,0.0,2.4,78.7,3,3,107.7,107.7
|
||||
06-08 20:01:04,1,1,80.9,166.4,4,0,80.9,166.4,4.0,0.0,2.7,77.8,4,4,166.4,166.4
|
||||
06-08 20:01:09,1,1,39.3,145.9,4,0,39.3,145.9,4.0,0.0,3.0,78.4,4,4,145.9,145.9
|
||||
06-08 20:01:14,1,1,0.0,181.3,4,0,0.0,181.3,4.0,0.0,3.3,78.4,4,4,181.3,181.3
|
||||
06-08 20:01:19,1,1,0.0,156.3,3,0,0.0,156.3,3.0,0.0,3.2,78.4,3,3,156.3,156.3
|
||||
06-08 20:01:24,1,1,73.4,106.2,3,0,73.4,106.2,3.0,0.0,3.4,78.9,3,3,106.2,106.2
|
||||
06-08 20:01:30,1,1,0.0,134.7,2,0,0.0,134.7,2.0,0.0,3.1,78.9,2,2,134.7,134.7
|
||||
06-08 20:01:35,1,1,0.0,90.7,2,0,0.0,90.7,2.0,0.0,3.2,78.9,2,2,90.7,90.7
|
||||
06-08 20:01:40,1,1,46.6,92.0,3,0,46.6,92.0,3.0,0.0,3.9,80.2,3,3,92.0,92.0
|
||||
06-08 20:01:45,1,1,64.4,181.1,4,0,64.4,181.1,4.0,0.0,4.6,80.5,4,4,181.1,181.1
|
||||
06-08 20:01:50,1,1,22.7,176.2,4,0,22.7,176.2,4.0,0.0,7.6,84.9,4,4,176.2,176.2
|
||||
06-08 20:01:55,1,1,97.9,244.6,6,0,97.9,244.6,6.0,0.0,18.8,86.9,6,6,244.6,244.6
|
||||
06-08 20:02:00,1,1,156.9,292.7,6,0,156.9,292.7,6.0,0.0,19.9,87.0,6,6,292.7,292.7
|
||||
06-08 20:02:05,1,1,0.0,251.6,5,0,0.0,251.6,5.0,0.0,18.4,87.0,5,5,251.6,251.6
|
||||
06-08 20:02:10,1,1,52.9,250.9,6,0,52.9,250.9,6.0,0.0,19.9,87.1,6,6,250.9,250.9
|
||||
06-08 20:02:15,1,1,0.0,253.1,5,0,0.0,253.1,5.0,0.0,19.0,87.1,5,5,253.1,253.1
|
||||
06-08 20:02:20,1,1,22.2,257.2,6,0,22.2,257.2,6.0,0.0,20.6,87.4,6,6,257.2,257.2
|
||||
06-08 20:02:25,1,1,0.0,214.8,3,0,0.0,214.8,3.0,0.0,16.8,87.4,3,3,214.8,214.8
|
||||
06-08 20:02:30,1,1,0.0,231.9,5,0,0.0,231.9,5.0,0.0,20.4,86.6,5,5,231.9,231.9
|
||||
06-08 20:02:35,1,1,160.6,251.9,6,0,160.6,251.9,6.0,0.0,22.5,86.6,6,6,251.9,251.9
|
||||
06-08 20:02:40,1,1,61.9,319.5,7,0,61.9,319.5,7.0,0.0,24.6,86.9,7,7,319.5,319.5
|
||||
06-08 20:02:45,1,1,0.0,316.7,7,0,0.0,316.7,7.0,0.0,25.1,86.9,7,7,316.7,316.7
|
||||
06-08 20:02:50,1,1,0.0,311.3,7,0,0.0,311.3,7.0,0.0,25.5,86.9,7,7,311.3,311.3
|
||||
06-08 20:02:55,1,1,133.7,379.3,9,0,133.7,379.3,9.0,0.0,28.2,87.1,9,9,379.3,379.3
|
||||
06-08 20:03:00,1,1,0.0,391.3,8,0,0.0,391.3,8.0,0.0,27.9,87.1,8,8,391.3,391.3
|
||||
06-08 20:03:05,1,1,0.0,246.6,4,0,0.0,246.6,4.0,0.0,12.7,88.8,4,4,246.6,246.6
|
||||
06-08 20:03:10,1,1,150.0,228.0,5,0,150.0,228.0,5.0,0.0,13.3,88.5,5,5,228.0,228.0
|
||||
06-08 20:03:15,1,1,0.0,211.9,4,0,0.0,211.9,4.0,0.0,13.2,88.5,4,4,211.9,211.9
|
||||
06-08 20:03:20,1,1,0.0,189.5,4,0,0.0,189.5,4.0,0.0,13.4,88.5,4,4,189.5,189.5
|
||||
06-08 20:03:25,1,1,56.9,229.6,5,0,56.9,229.6,5.0,0.0,14.3,88.5,5,5,229.6,229.6
|
||||
06-08 20:03:30,1,1,0.0,214.3,4,0,0.0,214.3,4.0,0.0,14.0,88.5,4,4,214.3,214.3
|
||||
06-08 20:03:36,1,1,78.5,207.9,4,0,78.5,207.9,4.0,0.0,4.3,83.8,4,4,207.9,207.9
|
||||
06-08 20:03:41,1,1,27.3,193.5,4,0,27.3,193.5,4.0,0.0,4.6,84.1,4,4,193.5,193.5
|
||||
06-08 20:03:46,1,1,0.0,174.8,3,0,0.0,174.8,3.0,0.0,3.3,84.1,3,3,174.8,174.8
|
||||
06-08 20:03:51,1,1,86.2,107.4,3,0,86.2,107.4,3.0,0.0,4.6,84.8,3,3,107.4,107.4
|
||||
06-08 20:03:56,1,1,23.6,116.6,3,0,23.6,116.6,3.0,0.0,3.6,85.1,3,3,116.6,116.6
|
||||
06-08 20:04:01,1,1,115.7,194.8,4,0,115.7,194.8,4.0,0.0,5.5,85.8,4,4,194.8,194.8
|
||||
06-08 20:04:06,1,1,0.0,159.3,3,0,0.0,159.3,3.0,0.0,5.2,85.8,3,3,159.3,159.3
|
||||
06-08 20:04:11,1,1,35.7,170.2,4,0,35.7,170.2,4.0,0.0,6.0,86.0,4,4,170.2,170.2
|
||||
06-08 20:04:16,1,1,77.1,154.3,4,0,77.1,154.3,4.0,0.0,5.7,85.8,4,4,154.3,154.3
|
||||
06-08 20:04:21,1,1,102.5,186.6,5,0,102.5,186.6,5.0,0.0,6.7,85.8,5,5,186.6,186.6
|
||||
06-08 20:04:26,1,1,0.0,224.6,5,0,0.0,224.6,5.0,0.0,7.1,85.8,5,5,224.6,224.6
|
||||
06-08 20:04:31,1,1,23.5,145.6,4,0,23.5,145.6,4.0,0.0,7.0,86.2,4,4,145.6,145.6
|
||||
06-08 20:04:36,1,1,0.0,137.1,3,0,0.0,137.1,3.0,0.0,6.3,86.2,3,3,137.1,137.1
|
||||
06-08 20:04:41,1,1,0.0,354.4,8,0,0.0,354.4,8.0,0.0,27.8,92.5,8,8,354.4,354.4
|
||||
06-08 20:04:46,1,1,0.0,355.2,8,0,0.0,355.2,8.0,0.0,28.3,92.5,8,8,355.2,355.2
|
||||
06-08 20:04:51,1,1,0.0,356.0,8,0,0.0,356.0,8.0,0.0,28.8,92.5,8,8,356.0,356.0
|
||||
06-08 20:04:56,1,1,0.0,356.9,8,0,0.0,356.9,8.0,0.0,29.3,92.5,8,8,356.9,356.9
|
||||
06-08 20:05:01,1,1,0.0,356.5,8,0,0.0,356.5,8.0,0.0,29.8,92.5,8,8,356.5,356.5
|
||||
06-08 20:05:06,1,1,0.0,532.9,10,0,0.0,532.9,10.0,0.0,36.1,90.6,10,10,532.9,532.9
|
||||
06-08 20:05:11,1,1,0.0,463.7,10,0,0.0,463.7,10.0,0.0,36.7,90.6,10,10,463.7,463.7
|
||||
06-08 20:05:16,1,1,46.1,463.8,11,0,46.1,463.8,11.0,0.0,37.6,90.6,11,11,463.8,463.8
|
||||
06-08 20:05:21,1,1,60.2,541.0,11,0,60.2,541.0,11.0,0.0,38.3,90.8,11,11,541.0,541.0
|
||||
06-08 20:05:26,1,1,0.0,506.6,11,0,0.0,506.6,11.0,0.0,39.1,90.8,11,11,506.6,506.6
|
||||
06-08 20:05:31,1,1,0.0,507.1,11,0,0.0,507.1,11.0,0.0,39.8,90.8,11,11,507.1,507.1
|
||||
06-08 20:05:36,1,1,47.0,540.0,12,0,47.0,540.0,12.0,0.0,43.7,91.0,12,12,540.0,540.0
|
||||
06-08 20:05:41,1,1,39.6,527.1,12,0,39.6,527.1,12.0,0.0,44.5,91.3,12,12,527.1,527.1
|
||||
06-08 20:05:46,1,1,56.6,221.3,5,0,56.6,221.3,5.0,0.0,14.6,90.1,5,5,221.3,221.3
|
||||
06-08 20:05:51,1,1,0.0,221.5,5,0,0.0,221.5,5.0,0.0,14.9,90.1,5,5,221.5,221.5
|
||||
06-08 20:05:56,1,1,75.5,248.6,6,0,75.5,248.6,6.0,0.0,17.2,90.2,6,6,248.6,248.6
|
||||
06-08 20:06:01,1,1,0.0,265.9,6,0,0.0,265.9,6.0,0.0,17.5,90.2,6,6,265.9,265.9
|
||||
06-08 20:06:06,1,1,71.7,276.8,7,0,71.7,276.8,7.0,0.0,18.0,90.1,7,7,276.8,276.8
|
||||
06-08 20:06:11,1,1,0.0,311.2,7,0,0.0,311.2,7.0,0.0,18.4,90.1,7,7,311.2,311.2
|
||||
06-08 20:06:16,1,1,41.2,276.9,7,0,41.2,276.9,7.0,0.0,18.9,90.1,7,7,276.9,276.9
|
||||
06-08 20:06:21,1,1,0.0,310.2,7,0,0.0,310.2,7.0,0.0,19.3,90.1,7,7,310.2,310.2
|
||||
06-08 20:06:26,1,1,0.0,310.4,7,0,0.0,310.4,7.0,0.0,19.8,90.1,7,7,310.4,310.4
|
||||
06-08 20:06:31,1,1,0.0,310.8,7,0,0.0,310.8,7.0,0.0,20.2,90.1,7,7,310.8,310.8
|
||||
06-08 20:06:36,1,1,0.0,275.3,6,0,0.0,275.3,6.0,0.0,20.1,90.1,6,6,275.3,275.3
|
||||
06-08 20:06:41,1,1,0.0,450.7,10,0,0.0,450.7,10.0,0.0,29.8,90.6,10,10,450.7,450.7
|
||||
06-08 20:06:46,1,1,0.0,454.2,10,0,0.0,454.2,10.0,0.0,30.4,90.6,10,10,454.2,454.2
|
||||
06-08 20:06:51,1,1,0.0,415.3,8,0,0.0,415.3,8.0,0.0,28.1,90.6,8,8,415.3,415.3
|
||||
06-08 20:06:56,1,1,0.0,319.6,7,0,0.0,319.6,7.0,0.0,28.1,90.6,7,7,319.6,319.6
|
||||
06-08 20:07:01,1,1,0.0,288.7,6,0,0.0,288.7,6.0,0.0,22.0,90.1,6,6,288.7,288.7
|
||||
06-08 20:07:06,1,1,0.0,313.6,7,0,0.0,313.6,7.0,0.0,29.0,90.6,7,7,313.6,313.6
|
||||
06-08 20:07:12,1,1,93.2,328.8,7,0,93.2,328.8,7.0,0.0,29.3,90.6,7,7,328.8,328.8
|
||||
06-08 20:07:16,1,1,0.0,311.4,7,0,0.0,311.4,7.0,0.0,24.1,90.1,7,7,311.4,311.4
|
||||
06-08 20:07:22,1,1,0.0,406.5,9,0,0.0,406.5,9.0,0.0,33.5,90.6,9,9,406.5,406.5
|
||||
06-08 20:07:27,1,1,0.0,384.8,8,0,0.0,384.8,8.0,0.0,33.5,90.6,8,8,384.8,384.8
|
||||
06-08 20:07:32,1,1,0.0,343.5,7,0,0.0,343.5,7.0,0.0,31.0,90.6,7,7,343.5,343.5
|
||||
06-08 20:07:37,1,1,168.7,327.4,8,0,168.7,327.4,8.0,0.0,31.8,90.5,8,8,327.4,327.4
|
||||
06-08 20:07:42,1,1,289.1,355.8,9,0,289.1,355.8,9.0,0.0,33.2,90.3,9,9,355.8,355.8
|
||||
06-08 20:07:47,1,1,0.0,374.4,8,0,0.0,374.4,8.0,0.0,30.8,90.8,8,8,374.4,374.4
|
||||
06-08 20:07:52,1,1,0.0,328.5,7,0,0.0,328.5,7.0,0.0,30.6,90.8,7,7,328.5,328.5
|
||||
06-08 20:07:57,1,1,54.5,334.9,8,0,54.5,334.9,8.0,0.0,32.3,90.9,8,8,334.9,334.9
|
||||
06-08 20:08:02,1,1,0.0,349.4,7,0,0.0,349.4,7.0,0.0,32.5,90.9,7,7,349.4,349.4
|
||||
06-08 20:08:07,1,1,153.3,360.4,8,0,153.3,360.4,8.0,0.0,33.3,90.7,8,8,360.4,360.4
|
||||
06-08 20:08:12,1,1,81.0,375.6,7,0,81.0,375.6,7.0,0.0,33.4,90.7,7,7,375.6,375.6
|
||||
06-08 20:08:17,1,1,0.0,326.8,7,0,0.0,326.8,7.0,0.0,33.9,90.7,7,7,326.8,326.8
|
||||
06-08 20:08:22,1,1,0.0,174.1,4,0,0.0,174.1,4.0,0.0,15.4,93.7,4,4,174.1,174.1
|
||||
06-08 20:08:27,1,1,34.9,200.5,5,0,34.9,200.5,5.0,0.0,15.9,93.6,5,5,200.5,200.5
|
||||
06-08 20:08:32,1,1,0.0,211.3,4,0,0.0,211.3,4.0,0.0,15.0,93.6,4,4,211.3,211.3
|
||||
06-08 20:08:37,1,1,0.0,174.0,4,0,0.0,174.0,4.0,0.0,15.3,93.6,4,4,174.0,174.0
|
||||
06-08 20:08:42,1,1,0.0,173.6,4,0,0.0,173.6,4.0,0.0,15.5,93.6,4,4,173.6,173.6
|
||||
06-08 20:08:47,1,1,145.7,189.9,5,0,145.7,189.9,5.0,0.0,17.1,93.6,5,5,189.9,189.9
|
||||
06-08 20:08:52,1,1,0.0,216.7,5,0,0.0,216.7,5.0,0.0,17.4,93.6,5,5,216.7,216.7
|
||||
06-08 20:08:57,1,1,0.0,216.8,5,0,0.0,216.8,5.0,0.0,17.7,93.6,5,5,216.8,216.8
|
||||
06-08 20:09:02,1,1,0.0,213.1,4,0,0.0,213.1,4.0,0.0,16.5,93.6,4,4,213.1,213.1
|
||||
06-08 20:09:07,1,1,0.0,173.8,4,0,0.0,173.8,4.0,0.0,16.8,93.6,4,4,173.8,173.8
|
||||
06-08 20:09:12,1,1,0.0,173.7,4,0,0.0,173.7,4.0,0.0,17.0,93.6,4,4,173.7,173.7
|
||||
06-08 20:09:17,1,1,148.6,183.6,5,0,148.6,183.6,5.0,0.0,19.0,93.5,5,5,183.6,183.6
|
||||
06-08 20:09:22,1,1,0.0,181.4,4,0,0.0,181.4,4.0,0.0,18.3,93.5,4,4,181.4,181.4
|
||||
06-08 20:09:27,1,1,0.0,173.0,4,0,0.0,173.0,4.0,0.0,18.6,93.5,4,4,173.0,173.0
|
||||
06-08 20:09:32,1,1,0.0,172.4,4,0,0.0,172.4,4.0,0.0,18.8,93.5,4,4,172.4,172.4
|
||||
06-08 20:09:37,1,1,0.0,173.0,4,0,0.0,173.0,4.0,0.0,19.1,93.5,4,4,173.0,173.0
|
||||
06-08 20:09:42,1,1,14.4,186.6,5,0,14.4,186.6,5.0,0.0,20.2,93.5,5,5,186.6,186.6
|
||||
06-08 20:09:47,1,1,0.0,244.6,5,0,0.0,244.6,5.0,0.0,24.8,87.8,5,5,244.6,244.6
|
||||
06-08 20:09:52,1,1,101.5,283.2,7,0,101.5,283.2,7.0,0.0,27.9,88.2,7,7,283.2,283.2
|
||||
06-08 20:09:57,1,1,0.0,301.3,7,0,0.0,301.3,7.0,0.0,28.3,88.2,7,7,301.3,301.3
|
||||
06-08 20:10:02,1,1,0.0,301.3,7,0,0.0,301.3,7.0,0.0,28.7,88.2,7,7,301.3,301.3
|
||||
06-08 20:10:07,1,1,1.8,282.4,7,0,1.8,282.4,7.0,0.0,26.4,88.2,7,7,282.4,282.4
|
||||
06-08 20:10:13,1,1,252.5,296.6,8,0,252.5,296.6,8.0,0.0,29.7,88.3,8,8,296.6,296.6
|
||||
06-08 20:10:18,1,1,0.0,341.5,7,0,0.0,341.5,7.0,0.0,29.7,88.3,7,7,341.5,341.5
|
||||
06-08 20:10:23,1,1,0.0,271.3,6,0,0.0,271.3,6.0,0.0,27.1,88.3,6,6,271.3,271.3
|
||||
06-08 20:10:28,1,1,0.0,258.8,7,0,0.0,258.8,7.0,0.0,30.5,88.8,7,7,258.8,258.8
|
||||
06-08 20:10:33,1,1,20.5,301.7,7,0,20.5,301.7,7.0,0.0,30.9,88.8,7,7,301.7,301.7
|
||||
06-08 20:10:38,1,1,83.6,282.1,7,0,83.6,282.1,7.0,0.0,28.3,88.6,7,7,282.1,282.1
|
||||
06-08 20:10:43,1,1,42.7,323.8,8,0,42.7,323.8,8.0,0.0,29.3,88.7,8,8,323.8,323.8
|
||||
06-08 20:10:48,1,1,0.0,344.7,8,0,0.0,344.7,8.0,0.0,29.7,88.7,8,8,344.7,344.7
|
||||
06-08 20:10:53,1,1,0.0,322.3,7,0,0.0,322.3,7.0,0.0,29.9,88.7,7,7,322.3,322.3
|
||||
06-08 20:10:58,1,1,0.0,302.6,7,0,0.0,302.6,7.0,0.0,30.3,88.7,7,7,302.6,302.6
|
||||
06-08 20:11:03,1,1,66.5,304.0,8,0,66.5,304.0,8.0,0.0,31.1,88.6,8,8,304.0,304.0
|
||||
06-08 20:11:08,1,1,0.0,338.9,7,0,0.0,338.9,7.0,0.0,31.2,88.6,7,7,338.9,338.9
|
||||
06-08 20:11:13,1,1,0.0,721.5,18,0,0.0,721.5,18.0,0.0,59.5,88.3,18,18,721.5,721.5
|
||||
06-08 20:11:18,1,1,0.0,722.3,18,0,0.0,722.3,18.0,0.0,60.5,88.3,18,18,722.3,722.3
|
||||
06-08 20:11:23,1,1,0.0,327.5,7,0,0.0,327.5,7.0,0.0,32.5,88.7,7,7,327.5,327.5
|
||||
06-08 20:11:28,1,1,0.0,679.6,17,0,0.0,679.6,17.0,0.0,61.9,88.3,17,17,679.6,679.6
|
||||
06-08 20:11:33,1,1,0.0,680.4,17,0,0.0,680.4,17.0,0.0,62.9,88.3,17,17,680.4,680.4
|
||||
06-08 20:11:38,1,1,0.0,730.2,16,0,0.0,730.2,16.0,0.0,61.7,88.3,16,16,730.2,730.2
|
||||
06-08 20:11:43,1,1,0.0,742.7,16,0,0.0,742.7,16.0,0.0,62.7,88.3,16,16,742.7,742.7
|
||||
06-08 20:11:48,1,1,65.5,719.2,16,0,65.5,719.2,16.0,0.0,63.5,88.3,16,16,719.2,719.2
|
||||
06-08 20:11:53,1,1,0.0,708.3,14,0,0.0,708.3,14.0,0.0,54.9,88.3,14,14,708.3,708.3
|
||||
06-08 20:11:58,1,1,0.0,605.1,13,0,0.0,605.1,13.0,0.0,54.6,88.3,13,13,605.1,605.1
|
||||
06-08 20:12:03,1,1,0.0,602.9,13,0,0.0,602.9,13.0,0.0,55.5,88.3,13,13,602.9,602.9
|
||||
06-08 20:12:08,1,1,0.0,394.3,8,0,0.0,394.3,8.0,0.0,35.9,93.1,8,8,394.3,394.3
|
||||
06-08 20:12:13,1,1,0.0,343.5,7,0,0.0,343.5,7.0,0.0,34.3,93.1,7,7,343.5,343.5
|
||||
06-08 20:12:18,1,1,100.7,286.0,5,0,100.7,286.0,5.0,0.0,19.4,89.9,5,5,286.0,286.0
|
||||
06-08 20:12:23,1,1,0.0,315.9,7,0,0.0,315.9,7.0,0.0,35.2,93.1,7,7,315.9,315.9
|
||||
06-08 20:12:28,1,1,137.9,348.1,8,0,137.9,348.1,8.0,0.0,37.9,93.1,8,8,348.1,348.1
|
||||
06-08 20:12:33,1,1,0.0,363.6,8,0,0.0,363.6,8.0,0.0,38.4,93.1,8,8,363.6,363.6
|
||||
06-08 20:12:38,1,1,0.0,364.9,8,0,0.0,364.9,8.0,0.0,38.9,93.1,8,8,364.9,364.9
|
||||
06-08 20:12:43,1,1,233.4,407.5,9,0,233.4,407.5,9.0,0.0,40.6,93.0,9,9,407.5,407.5
|
||||
06-08 20:12:48,1,1,0.0,408.7,9,0,0.0,408.7,9.0,0.0,41.2,93.0,9,9,408.7,408.7
|
||||
06-08 20:12:53,1,1,0.0,395.9,8,0,0.0,395.9,8.0,0.0,40.5,93.0,8,8,395.9,395.9
|
||||
06-08 20:12:58,1,1,92.7,394.1,9,0,92.7,394.1,9.0,0.0,41.1,92.9,9,9,394.1,394.1
|
||||
06-08 20:13:03,1,1,0.0,398.7,8,0,0.0,398.7,8.0,0.0,32.8,92.9,8,8,398.7,398.7
|
||||
06-08 20:13:08,1,1,0.0,363.8,8,0,0.0,363.8,8.0,0.0,33.3,92.9,8,8,363.8,363.8
|
||||
06-08 20:13:13,1,1,0.0,364.2,8,0,0.0,364.2,8.0,0.0,33.8,92.9,8,8,364.2,364.2
|
||||
06-08 20:13:18,1,1,0.0,364.8,8,0,0.0,364.8,8.0,0.0,34.4,92.9,8,8,364.8,364.8
|
||||
06-08 20:13:23,1,1,0.0,364.8,8,0,0.0,364.8,8.0,0.0,34.9,92.9,8,8,364.8,364.8
|
||||
06-08 20:13:28,1,1,0.0,365.0,8,0,0.0,365.0,8.0,0.0,35.4,92.9,8,8,365.0,365.0
|
||||
06-08 20:13:33,1,1,0.0,333.4,7,0,0.0,333.4,7.0,0.0,35.3,92.9,7,7,333.4,333.4
|
||||
06-08 20:13:38,1,1,0.0,319.6,7,0,0.0,319.6,7.0,0.0,35.7,92.9,7,7,319.6,319.6
|
||||
06-08 20:13:44,1,1,0.0,319.4,7,0,0.0,319.4,7.0,0.0,36.2,92.9,7,7,319.4,319.4
|
||||
06-08 20:13:49,1,1,96.1,354.7,8,0,96.1,354.7,8.0,0.0,36.8,92.9,8,8,354.7,354.7
|
||||
06-08 20:13:54,1,1,0.0,320.8,7,0,0.0,320.8,7.0,0.0,33.9,92.9,7,7,320.8,320.8
|
||||
06-08 20:13:59,1,1,360.0,390.2,9,0,360.0,390.2,9.0,0.0,39.2,92.9,9,9,390.2,390.2
|
||||
06-08 20:14:04,1,1,0.0,464.2,11,0,0.0,464.2,11.0,0.0,53.7,90.6,11,11,464.2,464.2
|
||||
06-08 20:14:09,1,1,28.5,487.3,12,0,28.5,487.3,12.0,0.0,56.8,90.7,12,12,487.3,487.3
|
||||
06-08 20:14:14,1,1,0.0,466.8,11,0,0.0,466.8,11.0,0.0,55.0,90.7,11,11,466.8,466.8
|
||||
06-08 20:14:19,1,1,0.0,465.0,11,0,0.0,465.0,11.0,0.0,55.7,90.7,11,11,465.0,465.0
|
||||
06-08 20:14:24,1,1,73.7,471.6,12,0,73.7,471.6,12.0,0.0,58.8,90.8,12,12,471.6,471.6
|
||||
06-08 20:14:29,1,1,105.0,526.7,13,0,105.0,526.7,13.0,0.0,58.0,90.8,13,13,526.7,526.7
|
||||
06-08 20:14:34,1,1,0.0,547.5,13,0,0.0,547.5,13.0,0.0,58.8,90.8,13,13,547.5,547.5
|
||||
06-08 20:14:39,1,1,0.0,514.9,12,0,0.0,514.9,12.0,0.0,59.4,90.8,12,12,514.9,514.9
|
||||
06-08 20:14:44,1,1,0.0,505.3,12,0,0.0,505.3,12.0,0.0,60.1,90.8,12,12,505.3,505.3
|
||||
06-08 20:14:49,1,1,0.0,504.9,12,0,0.0,504.9,12.0,0.0,60.8,90.8,12,12,504.9,504.9
|
||||
06-08 20:14:54,1,1,173.0,606.4,15,0,173.0,606.4,15.0,0.0,64.5,90.8,15,15,606.4,606.4
|
||||
06-08 20:14:59,1,1,195.7,319.9,7,0,195.7,319.9,7.0,0.0,29.4,89.9,7,7,319.9,319.9
|
||||
06-08 20:15:04,1,1,0.0,304.5,6,0,0.0,304.5,6.0,0.0,29.6,89.9,6,6,304.5,304.5
|
||||
06-08 20:15:09,1,1,0.0,245.2,5,0,0.0,245.2,5.0,0.0,29.0,89.9,5,5,245.2,245.2
|
||||
06-08 20:15:14,1,1,143.4,270.1,6,0,143.4,270.1,6.0,0.0,29.7,89.8,6,6,270.1,270.1
|
||||
06-08 20:15:19,1,1,21.8,307.2,6,0,21.8,307.2,6.0,0.0,30.7,89.9,6,6,307.2,307.2
|
||||
06-08 20:15:24,1,1,0.0,288.0,5,0,0.0,288.0,5.0,0.0,30.0,89.9,5,5,288.0,288.0
|
||||
06-08 20:15:29,1,1,74.3,275.3,6,0,74.3,275.3,6.0,0.0,32.0,89.9,6,6,275.3,275.3
|
||||
06-08 20:15:34,1,1,89.7,556.8,13,0,89.7,556.8,13.0,0.0,60.0,89.3,13,13,556.8,556.8
|
||||
06-08 20:15:39,1,1,0.0,565.5,12,0,0.0,565.5,12.0,0.0,60.4,89.3,12,12,565.5,565.5
|
||||
06-08 20:15:44,1,1,0.0,511.6,11,0,0.0,511.6,11.0,0.0,60.9,89.3,11,11,511.6,511.6
|
||||
06-08 20:15:49,1,1,0.0,458.5,10,0,0.0,458.5,10.0,0.0,61.1,89.3,10,10,458.5,458.5
|
||||
06-08 20:15:54,1,1,0.0,269.7,5,0,0.0,269.7,5.0,0.0,23.6,89.9,5,5,269.7,269.7
|
||||
06-08 20:16:00,1,1,0.0,419.8,9,0,0.0,419.8,9.0,0.0,47.9,94.1,9,9,419.8,419.8
|
||||
06-08 20:16:05,1,1,0.0,416.7,9,0,0.0,416.7,9.0,0.0,48.5,94.1,9,9,416.7,416.7
|
||||
06-08 20:16:10,1,1,0.0,418.5,9,0,0.0,418.5,9.0,0.0,49.1,94.1,9,9,418.5,418.5
|
||||
06-08 20:16:15,1,1,0.0,420.7,9,0,0.0,420.7,9.0,0.0,49.7,94.1,9,9,420.7,420.7
|
||||
06-08 20:16:20,1,1,0.0,416.8,9,0,0.0,416.8,9.0,0.0,50.3,94.1,9,9,416.8,416.8
|
||||
06-08 20:16:25,1,1,0.0,419.6,9,0,0.0,419.6,9.0,0.0,50.8,94.1,9,9,419.6,419.6
|
||||
06-08 20:16:30,1,1,0.0,419.8,9,0,0.0,419.8,9.0,0.0,51.4,94.1,9,9,419.8,419.8
|
||||
06-08 20:16:35,1,1,0.0,421.4,9,0,0.0,421.4,9.0,0.0,52.0,94.1,9,9,421.4,421.4
|
||||
06-08 20:16:40,1,1,0.0,419.0,9,0,0.0,419.0,9.0,0.0,52.6,94.1,9,9,419.0,419.0
|
||||
06-08 20:16:45,1,1,0.0,423.5,9,0,0.0,423.5,9.0,0.0,53.2,94.1,9,9,423.5,423.5
|
||||
06-08 20:16:50,1,1,0.0,423.2,9,0,0.0,423.2,9.0,0.0,53.8,94.1,9,9,423.2,423.2
|
||||
06-08 20:16:55,1,1,0.0,418.5,9,0,0.0,418.5,9.0,0.0,54.4,94.1,9,9,418.5,418.5
|
||||
06-08 20:17:00,1,1,0.0,412.8,9,0,0.0,412.8,9.0,0.0,54.9,94.1,9,9,412.8,412.8
|
||||
06-08 20:17:05,1,1,10.6,426.9,10,0,10.6,426.9,10.0,0.0,55.6,94.1,10,10,426.9,426.9
|
||||
06-08 20:17:10,1,1,0.0,464.9,10,0,0.0,464.9,10.0,0.0,56.3,94.1,10,10,464.9,464.9
|
||||
06-08 20:17:15,1,1,0.0,467.3,10,0,0.0,467.3,10.0,0.0,56.9,94.1,10,10,467.3,467.3
|
||||
06-08 20:17:20,1,1,0.0,541.6,12,0,0.0,541.6,12.0,0.0,53.2,88.9,12,12,541.6,541.6
|
||||
06-08 20:17:25,1,1,0.0,496.4,10,0,0.0,496.4,10.0,0.0,48.4,88.9,10,10,496.4,496.4
|
||||
06-08 20:17:30,1,1,0.0,449.9,10,0,0.0,449.9,10.0,0.0,49.0,88.9,10,10,449.9,449.9
|
||||
06-08 20:17:35,1,1,8.8,457.5,10,0,8.8,457.5,10.0,0.0,48.5,88.9,10,10,457.5,457.5
|
||||
06-08 20:17:40,1,1,378.9,467.5,10,0,378.9,467.5,10.0,0.0,50.0,88.5,10,10,467.5,467.5
|
||||
06-08 20:17:45,1,1,0.0,444.9,10,0,0.0,444.9,10.0,0.0,50.6,88.5,10,10,444.9,444.9
|
||||
06-08 20:17:50,1,1,0.0,442.4,10,0,0.0,442.4,10.0,0.0,40.0,87.8,10,10,442.4,442.4
|
||||
06-08 20:17:55,1,1,0.0,441.8,10,0,0.0,441.8,10.0,0.0,40.6,87.8,10,10,441.8,441.8
|
||||
06-08 20:18:00,1,1,81.3,454.2,11,0,81.3,454.2,11.0,0.0,41.4,87.7,11,11,454.2,454.2
|
||||
06-08 20:18:05,1,1,0.0,482.9,11,0,0.0,482.9,11.0,0.0,42.1,87.7,11,11,482.9,482.9
|
||||
06-08 20:18:10,1,1,1516.5,487.1,12,0,1516.5,487.1,12.0,0.0,42.9,86.3,12,12,487.1,487.1
|
||||
06-08 20:18:15,1,1,0.0,464.8,10,0,0.0,464.8,10.0,0.0,42.7,86.3,10,10,464.8,464.8
|
||||
06-08 20:18:20,1,1,0.0,433.7,9,0,0.0,433.7,9.0,0.0,42.7,86.3,9,9,433.7,433.7
|
||||
06-08 20:18:25,1,1,0.0,369.0,8,0,0.0,369.0,8.0,0.0,41.1,86.3,8,8,369.0,369.0
|
||||
06-08 20:18:30,1,1,407.4,413.2,9,0,407.4,413.2,9.0,0.0,42.3,86.3,9,9,413.2,413.2
|
||||
06-08 20:18:35,1,1,0.0,396.5,9,0,0.0,396.5,9.0,0.0,42.9,86.3,9,9,396.5,396.5
|
||||
06-08 20:18:40,1,1,11.2,368.2,9,0,11.2,368.2,9.0,0.0,42.9,86.3,9,9,368.2,368.2
|
||||
06-08 20:18:45,1,1,247.7,509.2,12,0,247.7,509.2,12.0,0.0,38.8,89.3,12,12,509.2,509.2
|
||||
06-08 20:18:50,1,1,430.2,344.5,8,0,430.2,344.5,8.0,0.0,34.9,86.2,8,8,344.5,344.5
|
||||
06-08 20:18:55,1,1,51.3,385.1,9,0,51.3,385.1,9.0,0.0,36.5,86.4,9,9,385.1,385.1
|
||||
06-08 20:19:01,1,1,0.0,354.9,7,0,0.0,354.9,7.0,0.0,33.5,88.6,7,7,354.9,354.9
|
||||
06-08 20:19:06,1,1,192.0,389.2,9,0,192.0,389.2,9.0,0.0,37.5,88.7,9,9,389.2,389.2
|
||||
06-08 20:19:11,1,1,0.0,379.6,7,0,0.0,379.6,7.0,0.0,26.6,88.7,7,7,379.6,379.6
|
||||
06-08 20:19:16,1,1,23.0,364.7,8,0,23.0,364.7,8.0,0.0,27.9,88.7,8,8,364.7,364.7
|
||||
06-08 20:19:21,1,1,0.0,397.1,8,0,0.0,397.1,8.0,0.0,28.4,88.7,8,8,397.1,397.1
|
||||
06-08 20:19:26,1,1,0.0,350.4,6,0,0.0,350.4,6.0,0.0,26.8,88.7,6,6,350.4,350.4
|
||||
06-08 20:19:31,1,1,0.0,297.4,6,0,0.0,297.4,6.0,0.0,27.2,88.7,6,6,297.4,297.4
|
||||
06-08 20:19:36,1,1,0.0,297.1,6,0,0.0,297.1,6.0,0.0,27.6,88.7,6,6,297.1,297.1
|
||||
06-08 20:19:41,1,1,0.0,377.2,8,0,0.0,377.2,8.0,0.0,40.3,90.3,8,8,377.2,377.2
|
||||
06-08 20:19:46,1,1,0.0,338.6,7,0,0.0,338.6,7.0,0.0,37.9,90.3,7,7,338.6,338.6
|
||||
06-08 20:19:51,1,1,117.5,367.2,8,0,117.5,367.2,8.0,0.0,41.4,90.4,8,8,367.2,367.2
|
||||
06-08 20:19:56,1,1,33.1,326.4,7,0,33.1,326.4,7.0,0.0,31.9,88.9,7,7,326.4,326.4
|
||||
06-08 20:20:01,1,1,0.0,377.6,8,0,0.0,377.6,8.0,0.0,42.5,90.4,8,8,377.6,377.6
|
||||
06-08 20:20:06,1,1,0.0,369.7,7,0,0.0,369.7,7.0,0.0,39.7,90.4,7,7,369.7,369.7
|
||||
06-08 20:20:11,1,1,0.0,322.3,6,0,0.0,322.3,6.0,0.0,31.2,90.4,6,6,322.3,322.3
|
||||
06-08 20:20:16,1,1,0.0,282.6,6,0,0.0,282.6,6.0,0.0,31.6,90.4,6,6,282.6,282.6
|
||||
06-08 20:20:21,1,1,44.1,359.5,9,0,44.1,359.5,9.0,0.0,24.4,90.5,9,9,359.5,359.5
|
||||
06-08 20:20:26,1,1,0.0,394.2,8,0,0.0,394.2,8.0,0.0,24.5,90.5,8,8,394.2,394.2
|
||||
06-08 20:20:31,1,1,90.2,414.3,9,0,90.2,414.3,9.0,0.0,25.6,90.5,9,9,414.3,414.3
|
||||
06-08 20:20:36,1,1,0.0,392.3,8,0,0.0,392.3,8.0,0.0,25.5,90.5,8,8,392.3,392.3
|
||||
06-08 20:20:41,1,1,43.9,384.1,9,0,43.9,384.1,9.0,0.0,26.8,90.5,9,9,384.1,384.1
|
||||
06-08 20:20:46,1,1,0.0,415.7,8,0,0.0,415.7,8.0,0.0,26.6,90.5,8,8,415.7,415.7
|
||||
06-08 20:20:51,1,1,76.9,380.7,9,0,76.9,380.7,9.0,0.0,31.9,90.7,9,9,380.7,380.7
|
||||
06-08 20:20:56,1,1,0.0,427.4,8,0,0.0,427.4,8.0,0.0,27.6,90.7,8,8,427.4,427.4
|
||||
06-08 20:21:01,1,1,38.9,378.8,8,0,38.9,378.8,8.0,0.0,28.2,90.7,8,8,378.8,378.8
|
||||
06-08 20:21:06,1,1,0.0,377.9,8,0,0.0,377.9,8.0,0.0,28.8,90.7,8,8,377.9,377.9
|
||||
06-08 20:21:11,1,1,0.0,379.9,8,0,0.0,379.9,8.0,0.0,29.3,90.7,8,8,379.9,379.9
|
||||
06-08 20:21:16,1,1,0.0,338.4,7,0,0.0,338.4,7.0,0.0,28.7,90.7,7,7,338.4,338.4
|
||||
06-08 20:21:21,1,1,0.0,331.8,7,0,0.0,331.8,7.0,0.0,29.2,90.7,7,7,331.8,331.8
|
||||
06-08 20:21:26,1,1,57.1,372.2,8,0,57.1,372.2,8.0,0.0,30.5,90.7,8,8,372.2,372.2
|
||||
06-08 20:21:31,1,1,69.4,366.9,8,0,69.4,366.9,8.0,0.0,29.5,90.7,8,8,366.9,366.9
|
||||
06-08 20:21:36,1,1,0.0,356.7,7,0,0.0,356.7,7.0,0.0,29.9,90.7,7,7,356.7,356.7
|
||||
06-08 20:21:41,1,1,21.1,359.2,8,0,21.1,359.2,8.0,0.0,35.3,90.9,8,8,359.2,359.2
|
||||
06-08 20:21:46,1,1,0.0,453.1,10,0,0.0,453.1,10.0,0.0,35.2,91.1,10,10,453.1,453.1
|
||||
06-08 20:21:51,1,1,197.5,465.1,11,0,197.5,465.1,11.0,0.0,37.8,91.0,11,11,465.1,465.1
|
||||
06-08 20:21:56,1,1,0.0,486.4,11,0,0.0,486.4,11.0,0.0,38.5,91.0,11,11,486.4,486.4
|
||||
06-08 20:22:01,1,1,1.8,470.5,11,0,1.8,470.5,11.0,0.0,38.6,91.1,11,11,470.5,470.5
|
||||
06-08 20:22:06,1,1,0.0,497.9,11,0,0.0,497.9,11.0,0.0,39.3,91.1,11,11,497.9,497.9
|
||||
06-08 20:22:11,1,1,0.0,498.0,11,0,0.0,498.0,11.0,0.0,40.0,91.1,11,11,498.0,498.0
|
||||
06-08 20:22:16,1,1,0.0,498.4,11,0,0.0,498.4,11.0,0.0,40.7,91.1,11,11,498.4,498.4
|
||||
06-08 20:22:21,1,1,0.0,514.3,11,0,0.0,514.3,11.0,0.0,48.5,90.4,11,11,514.3,514.3
|
||||
06-08 20:22:27,1,1,0.0,478.1,11,0,0.0,478.1,11.0,0.0,49.2,90.4,11,11,478.1,478.1
|
||||
06-08 20:22:31,1,1,0.0,354.6,8,0,0.0,354.6,8.0,0.0,44.3,93.2,8,8,354.6,354.6
|
||||
06-08 20:22:37,1,1,0.0,356.4,8,0,0.0,356.4,8.0,0.0,44.8,93.2,8,8,356.4,356.4
|
||||
06-08 20:22:42,1,1,0.0,355.3,8,0,0.0,355.3,8.0,0.0,45.3,93.2,8,8,355.3,355.3
|
||||
06-08 20:22:47,1,1,287.8,372.0,10,0,287.8,372.0,10.0,0.0,50.9,93.2,10,10,372.0,372.0
|
||||
06-08 20:22:52,1,1,0.0,416.2,9,0,0.0,416.2,9.0,0.0,47.3,93.2,9,9,416.2,416.2
|
||||
06-08 20:22:57,1,1,0.0,400.7,9,0,0.0,400.7,9.0,0.0,47.8,93.2,9,9,400.7,400.7
|
||||
06-08 20:23:02,1,1,0.0,371.7,8,0,0.0,371.7,8.0,0.0,47.3,93.2,8,8,371.7,371.7
|
||||
06-08 20:23:07,1,1,0.0,478.3,11,0,0.0,478.3,11.0,0.0,54.5,90.4,11,11,478.3,478.3
|
||||
06-08 20:23:12,1,1,0.0,478.5,11,0,0.0,478.5,11.0,0.0,55.2,90.4,11,11,478.5,478.5
|
||||
06-08 20:23:17,1,1,67.1,470.6,11,0,67.1,470.6,11.0,0.0,47.3,90.3,11,11,470.6,470.6
|
||||
06-08 20:23:22,1,1,0.0,395.5,9,0,0.0,395.5,9.0,0.0,45.6,92.4,9,9,395.5,395.5
|
||||
06-08 20:23:27,1,1,9.8,424.8,10,0,9.8,424.8,10.0,0.0,46.3,92.4,10,10,424.8,424.8
|
||||
06-08 20:23:32,1,1,8.4,447.8,10,0,8.4,447.8,10.0,0.0,46.8,92.4,10,10,447.8,447.8
|
||||
06-08 20:23:37,1,1,0.0,438.9,10,0,0.0,438.9,10.0,0.0,47.4,92.4,10,10,438.9,438.9
|
||||
06-08 20:23:42,1,1,0.0,465.1,10,0,0.0,465.1,10.0,0.0,35.6,94.0,10,10,465.1,465.1
|
||||
06-08 20:23:47,1,1,139.0,512.4,12,0,139.0,512.4,12.0,0.0,38.1,93.9,12,12,512.4,512.4
|
||||
06-08 20:23:52,1,1,83.0,591.6,12,0,83.0,591.6,12.0,0.0,39.5,94.0,12,12,591.6,591.6
|
||||
06-08 20:23:57,1,1,48.3,587.3,13,0,48.3,587.3,13.0,0.0,46.2,94.0,13,13,587.3,587.3
|
||||
06-08 20:24:02,1,1,0.0,602.4,13,0,0.0,602.4,13.0,0.0,47.0,94.0,13,13,602.4,602.4
|
||||
06-08 20:24:07,1,1,0.0,604.3,13,0,0.0,604.3,13.0,0.0,47.9,94.0,13,13,604.3,604.3
|
||||
06-08 20:24:12,1,1,21.0,630.1,14,0,21.0,630.1,14.0,0.0,49.8,94.0,14,14,630.1,630.1
|
||||
06-08 20:24:17,1,1,0.0,586.5,12,0,0.0,586.5,12.0,0.0,48.3,94.0,12,12,586.5,586.5
|
||||
06-08 20:24:22,1,1,9.6,534.5,11,0,9.6,534.5,11.0,0.0,47.8,94.1,11,11,534.5,534.5
|
||||
06-08 20:24:27,1,1,44.8,446.5,10,0,44.8,446.5,10.0,0.0,44.1,91.2,10,10,446.5,446.5
|
||||
06-08 20:24:32,1,1,0.0,465.8,10,0,0.0,465.8,10.0,0.0,44.7,91.2,10,10,465.8,465.8
|
||||
06-08 20:24:37,1,1,0.0,465.3,10,0,0.0,465.3,10.0,0.0,45.4,91.2,10,10,465.3,465.3
|
||||
06-08 20:24:42,1,1,157.0,537.0,13,0,157.0,537.0,13.0,0.0,48.2,91.3,13,13,537.0,537.0
|
||||
06-08 20:24:47,1,1,0.0,521.8,11,0,0.0,521.8,11.0,0.0,47.8,91.3,11,11,521.8,521.8
|
||||
06-08 20:24:52,1,1,0.0,477.2,10,0,0.0,477.2,10.0,0.0,47.3,91.3,10,10,477.2,477.2
|
||||
06-08 20:24:57,1,1,0.0,478.6,10,0,0.0,478.6,10.0,0.0,40.2,92.5,10,10,478.6,478.6
|
||||
06-08 20:25:02,1,1,0.0,459.9,10,0,0.0,459.9,10.0,0.0,50.3,94.0,10,10,459.9,459.9
|
||||
06-08 20:25:07,1,1,0.0,472.4,10,0,0.0,472.4,10.0,0.0,58.5,89.6,10,10,472.4,472.4
|
||||
06-08 20:25:12,1,1,0.0,580.0,13,0,0.0,580.0,13.0,0.0,62.6,89.9,13,13,580.0,580.0
|
||||
06-08 20:25:17,1,1,0.0,580.3,13,0,0.0,580.3,13.0,0.0,63.4,89.9,13,13,580.3,580.3
|
||||
06-08 20:25:22,1,1,0.0,572.4,12,0,0.0,572.4,12.0,0.0,62.4,89.9,12,12,572.4,572.4
|
||||
06-08 20:25:28,1,1,99.1,540.2,13,0,99.1,540.2,13.0,0.0,63.4,89.8,13,13,540.2,540.2
|
||||
06-08 20:25:33,1,1,0.0,580.0,13,0,0.0,580.0,13.0,0.0,64.2,89.8,13,13,580.0,580.0
|
||||
06-08 20:25:37,1,1,0.0,384.5,8,0,0.0,384.5,8.0,0.0,44.3,89.6,8,8,384.5,384.5
|
||||
06-08 20:25:43,1,1,0.6,561.2,13,0,0.6,561.2,13.0,0.0,65.4,89.8,13,13,561.2,561.2
|
||||
06-08 20:25:48,1,1,0.0,579.6,13,0,0.0,579.6,13.0,0.0,66.2,89.8,13,13,579.6,579.6
|
||||
06-08 20:25:53,1,1,100.8,609.2,14,0,100.8,609.2,14.0,0.0,67.2,89.8,14,14,609.2,609.2
|
||||
06-08 20:25:58,1,1,0.0,606.6,13,0,0.0,606.6,13.0,0.0,67.8,89.8,13,13,606.6,606.6
|
||||
06-08 20:26:03,1,1,0.0,578.2,13,0,0.0,578.2,13.0,0.0,68.6,89.8,13,13,578.2,578.2
|
||||
06-08 20:26:08,1,1,0.0,578.9,13,0,0.0,578.9,13.0,0.0,69.4,89.8,13,13,578.9,578.9
|
||||
06-08 20:26:13,1,1,0.0,578.8,13,0,0.0,578.8,13.0,0.0,70.3,89.8,13,13,578.8,578.8
|
||||
06-08 20:26:18,1,1,30.1,595.2,13,0,30.1,595.2,13.0,0.0,63.7,89.8,13,13,595.2,595.2
|
||||
06-08 20:26:23,1,1,0.0,320.7,7,0,0.0,320.7,7.0,0.0,30.3,91.0,7,7,320.7,320.7
|
||||
06-08 20:26:28,1,1,0.0,321.0,7,0,0.0,321.0,7.0,0.0,30.8,91.0,7,7,321.0,321.0
|
||||
06-08 20:26:33,1,1,0.0,321.5,7,0,0.0,321.5,7.0,0.0,31.2,91.0,7,7,321.5,321.5
|
||||
06-08 20:26:38,1,1,0.0,321.4,7,0,0.0,321.4,7.0,0.0,31.7,91.0,7,7,321.4,321.4
|
||||
06-08 20:26:43,1,1,0.0,321.1,7,0,0.0,321.1,7.0,0.0,32.1,91.0,7,7,321.1,321.1
|
||||
06-08 20:26:48,1,1,0.0,321.7,7,0,0.0,321.7,7.0,0.0,32.6,91.0,7,7,321.7,321.7
|
||||
06-08 20:26:53,1,1,0.0,321.0,7,0,0.0,321.0,7.0,0.0,33.0,91.0,7,7,321.0,321.0
|
||||
06-08 20:26:58,1,1,0.0,321.0,7,0,0.0,321.0,7.0,0.0,33.5,91.0,7,7,321.0,321.0
|
||||
06-08 20:27:03,1,1,0.0,321.2,7,0,0.0,321.2,7.0,0.0,33.9,91.0,7,7,321.2,321.2
|
||||
06-08 20:27:08,1,1,1.2,360.7,7,0,1.2,360.7,7.0,0.0,25.6,91.1,7,7,360.7,360.7
|
||||
06-08 20:27:13,1,1,0.0,315.0,6,0,0.0,315.0,6.0,0.0,25.1,91.1,6,6,315.0,315.0
|
||||
06-08 20:27:18,1,1,0.0,276.3,6,0,0.0,276.3,6.0,0.0,25.5,91.1,6,6,276.3,276.3
|
||||
06-08 20:27:23,1,1,0.0,276.6,6,0,0.0,276.6,6.0,0.0,25.9,91.1,6,6,276.6,276.6
|
||||
06-08 20:27:28,1,1,0.0,659.3,16,0,0.0,659.3,16.0,0.0,79.1,93.2,16,16,659.3,659.3
|
||||
06-08 20:27:33,1,1,197.9,633.2,18,0,197.9,633.2,18.0,0.0,73.7,93.2,18,18,633.2,633.2
|
||||
06-08 20:27:38,1,1,0.0,642.2,18,0,0.0,642.2,18.0,0.0,74.6,93.2,18,18,642.2,642.2
|
||||
06-08 20:27:43,1,1,0.0,626.8,17,0,0.0,626.8,17.0,0.0,74.8,93.2,17,17,626.8,626.8
|
||||
06-08 20:27:48,1,1,0.0,409.0,9,0,0.0,409.0,9.0,0.0,30.6,89.1,9,9,409.0,409.0
|
||||
06-08 20:27:53,1,1,0.0,408.2,9,0,0.0,408.2,9.0,0.0,31.1,89.1,9,9,408.2,408.2
|
||||
06-08 20:27:58,1,1,0.0,409.0,9,0,0.0,409.0,9.0,0.0,31.7,89.1,9,9,409.0,409.0
|
||||
06-08 20:28:03,1,1,92.5,406.9,9,0,92.5,406.9,9.0,0.0,32.2,89.1,9,9,406.9,406.9
|
||||
06-08 20:28:08,1,1,0.0,407.7,9,0,0.0,407.7,9.0,0.0,32.7,89.1,9,9,407.7,407.7
|
||||
06-08 20:28:13,1,1,0.0,408.5,9,0,0.0,408.5,9.0,0.0,33.3,89.1,9,9,408.5,408.5
|
||||
06-08 20:28:18,1,1,0.0,409.2,9,0,0.0,409.2,9.0,0.0,33.9,89.1,9,9,409.2,409.2
|
||||
06-08 20:28:23,1,1,145.6,448.8,10,0,145.6,448.8,10.0,0.0,35.8,89.1,10,10,448.8,448.8
|
||||
06-08 20:28:28,1,1,0.0,416.8,8,0,0.0,416.8,8.0,0.0,33.3,89.1,8,8,416.8,416.8
|
||||
06-08 20:28:33,1,1,176.3,442.2,11,0,176.3,442.2,11.0,0.0,35.7,89.2,11,11,442.2,442.2
|
||||
06-08 20:28:38,1,1,0.0,586.2,14,0,0.0,586.2,14.0,0.0,58.5,93.2,14,14,586.2,586.2
|
||||
06-08 20:28:43,1,1,127.8,616.8,15,0,127.8,616.8,15.0,0.0,59.7,93.2,15,15,616.8,616.8
|
||||
06-08 20:28:48,1,1,176.5,628.5,17,0,176.5,628.5,17.0,0.0,91.5,88.4,17,17,628.5,628.5
|
||||
06-08 20:28:53,1,1,80.8,637.6,17,0,80.8,637.6,17.0,0.0,90.5,88.4,17,17,637.6,637.6
|
||||
06-08 20:28:58,1,1,0.0,639.7,17,0,0.0,639.7,17.0,0.0,91.4,88.4,17,17,639.7,639.7
|
||||
06-08 20:29:03,1,1,0.0,483.1,10,0,0.0,483.1,10.0,0.0,48.4,90.4,10,10,483.1,483.1
|
||||
06-08 20:29:08,1,1,0.0,471.2,10,0,0.0,471.2,10.0,0.0,49.1,90.4,10,10,471.2,471.2
|
||||
06-08 20:29:13,1,1,0.0,471.1,10,0,0.0,471.1,10.0,0.0,49.7,90.4,10,10,471.1,471.1
|
||||
06-08 20:29:18,1,1,0.0,390.5,8,0,0.0,390.5,8.0,0.0,48.8,90.4,8,8,390.5,390.5
|
||||
06-08 20:29:23,1,1,0.0,379.0,8,0,0.0,379.0,8.0,0.0,49.3,90.4,8,8,379.0,379.0
|
||||
06-08 20:29:28,1,1,36.9,415.3,9,0,36.9,415.3,9.0,0.0,50.5,90.4,9,9,415.3,415.3
|
||||
06-08 20:29:34,1,1,0.0,404.4,8,0,0.0,404.4,8.0,0.0,42.2,90.4,8,8,404.4,404.4
|
||||
06-08 20:29:39,1,1,296.2,404.8,10,0,296.2,404.8,10.0,0.0,43.6,90.3,10,10,404.8,404.8
|
||||
06-08 20:29:44,1,1,0.0,448.2,9,0,0.0,448.2,9.0,0.0,44.2,90.3,9,9,448.2,448.2
|
||||
06-08 20:29:49,1,1,0.0,429.2,9,0,0.0,429.2,9.0,0.0,44.8,90.3,9,9,429.2,429.2
|
||||
06-08 20:29:54,1,1,0.0,417.4,7,0,0.0,417.4,7.0,0.0,27.5,90.3,7,7,417.4,417.4
|
||||
06-08 20:29:59,1,1,111.4,342.5,8,0,111.4,342.5,8.0,0.0,29.1,90.3,8,8,342.5,342.5
|
||||
06-08 20:30:04,1,1,10.8,409.9,9,0,10.8,409.9,9.0,0.0,29.7,90.3,9,9,409.9,409.9
|
||||
06-08 20:30:09,1,1,0.0,425.1,9,0,0.0,425.1,9.0,0.0,30.3,90.3,9,9,425.1,425.1
|
||||
06-08 20:30:14,1,1,208.9,452.1,10,0,208.9,452.1,10.0,0.0,31.9,90.2,10,10,452.1,452.1
|
||||
06-08 20:30:19,1,1,0.0,635.3,14,0,0.0,635.3,14.0,0.0,77.6,88.2,14,14,635.3,635.3
|
||||
06-08 20:30:24,1,1,277.4,620.9,15,0,277.4,620.9,15.0,0.0,78.8,88.0,15,15,620.9,620.9
|
||||
06-08 20:30:29,1,1,0.0,782.0,20,0,0.0,782.0,20.0,0.0,86.1,45.4,20,20,782.0,782.0
|
||||
06-08 20:30:34,1,1,0.0,781.1,20,0,0.0,781.1,20.0,0.0,87.2,45.4,20,20,781.1,781.1
|
||||
06-08 20:30:39,1,1,738.3,771.9,20,0,738.3,771.9,20.0,0.0,80.8,45.3,20,20,771.9,771.9
|
||||
06-08 20:30:44,1,1,0.0,751.6,19,0,0.0,751.6,19.0,0.0,81.5,45.3,19,19,751.6,751.6
|
||||
06-08 20:30:49,1,1,0.0,747.7,19,0,0.0,747.7,19.0,0.0,82.6,45.3,19,19,747.7,747.7
|
||||
06-08 20:30:54,1,1,0.0,744.0,19,0,0.0,744.0,19.0,0.0,83.6,45.3,19,19,744.0,744.0
|
||||
06-08 20:30:59,1,1,89.4,764.7,20,0,89.4,764.7,20.0,0.0,85.3,45.4,20,20,764.7,764.7
|
||||
06-08 20:31:04,1,1,0.0,751.6,18,0,0.0,751.6,18.0,0.0,84.2,45.4,18,18,751.6,751.6
|
||||
06-08 20:31:09,1,1,0.0,697.7,16,0,0.0,697.7,16.0,0.0,76.0,45.4,16,16,697.7,697.7
|
||||
06-08 20:31:14,1,1,0.0,711.3,16,0,0.0,711.3,16.0,0.0,77.0,45.4,16,16,711.3,711.3
|
||||
06-08 20:31:19,1,1,163.2,687.9,17,0,163.2,687.9,17.0,0.0,78.2,45.4,17,17,687.9,687.9
|
||||
06-08 20:31:24,1,1,91.8,683.3,18,0,91.8,683.3,18.0,0.0,79.9,45.5,18,18,683.3,683.3
|
||||
06-08 20:31:29,1,1,0.0,661.1,17,0,0.0,661.1,17.0,0.0,89.8,88.3,17,17,661.1,661.1
|
||||
06-08 20:31:34,1,1,0.0,387.1,8,0,0.0,387.1,8.0,0.0,37.4,87.6,8,8,387.1,387.1
|
||||
06-08 20:31:39,1,1,0.0,687.3,16,0,0.0,687.3,16.0,0.0,81.6,45.5,16,16,687.3,687.3
|
||||
06-08 20:31:44,1,1,9.0,674.7,17,0,9.0,674.7,17.0,0.0,82.6,45.5,17,17,674.7,674.7
|
||||
06-08 20:31:49,1,1,103.9,699.7,18,0,103.9,699.7,18.0,0.0,84.4,45.6,18,18,699.7,699.7
|
||||
06-08 20:31:54,1,1,154.4,696.4,17,0,154.4,696.4,17.0,0.0,85.2,45.8,17,17,696.4,696.4
|
||||
06-08 20:31:59,1,1,0.0,672.9,17,0,0.0,672.9,17.0,0.0,86.2,45.8,17,17,672.9,672.9
|
||||
06-08 20:32:04,1,1,81.6,391.2,9,0,81.6,391.2,9.0,0.0,35.3,89.2,9,9,391.2,391.2
|
||||
06-08 20:32:09,1,1,0.0,349.3,7,0,0.0,349.3,7.0,0.0,35.0,89.2,7,7,349.3,349.3
|
||||
06-08 20:32:14,1,1,0.0,320.6,7,0,0.0,320.6,7.0,0.0,35.4,89.2,7,7,320.6,320.6
|
||||
06-08 20:32:19,1,1,0.0,320.1,7,0,0.0,320.1,7.0,0.0,35.9,89.2,7,7,320.1,320.1
|
||||
06-08 20:32:24,1,1,0.0,466.0,9,0,0.0,466.0,9.0,0.0,42.7,87.5,9,9,466.0,466.0
|
||||
06-08 20:32:29,1,1,0.0,359.2,7,0,0.0,359.2,7.0,0.0,36.3,89.2,7,7,359.2,359.2
|
||||
06-08 20:32:34,1,1,35.1,341.3,8,0,35.1,341.3,8.0,0.0,38.6,89.3,8,8,341.3,341.3
|
||||
06-08 20:32:39,1,1,0.0,347.7,7,0,0.0,347.7,7.0,0.0,30.2,89.3,7,7,347.7,347.7
|
||||
06-08 20:32:44,1,1,0.0,318.5,7,0,0.0,318.5,7.0,0.0,30.6,89.3,7,7,318.5,318.5
|
||||
06-08 20:32:49,1,1,87.9,323.6,7,0,87.9,323.6,7.0,0.0,31.3,89.3,7,7,323.6,323.6
|
||||
06-08 20:32:54,1,1,1.2,351.1,8,0,1.2,351.1,8.0,0.0,31.8,89.3,8,8,351.1,351.1
|
||||
06-08 20:32:59,1,1,0.0,364.9,8,0,0.0,364.9,8.0,0.0,32.3,89.3,8,8,364.9,364.9
|
||||
06-08 20:33:04,1,1,8.2,347.3,8,0,8.2,347.3,8.0,0.0,30.7,89.3,8,8,347.3,347.3
|
||||
06-08 20:33:09,1,1,0.0,359.4,7,0,0.0,359.4,7.0,0.0,31.0,89.3,7,7,359.4,359.4
|
||||
06-08 20:33:14,1,1,0.0,319.6,7,0,0.0,319.6,7.0,0.0,31.4,89.3,7,7,319.6,319.6
|
||||
06-08 20:33:20,1,1,161.4,364.1,10,0,161.4,364.1,10.0,0.0,35.8,89.5,10,10,364.1,364.1
|
||||
06-08 20:33:25,1,1,9.2,423.3,10,0,9.2,423.3,10.0,0.0,35.3,89.5,10,10,423.3,423.3
|
||||
06-08 20:33:30,1,1,237.4,446.4,11,0,237.4,446.4,11.0,0.0,37.4,88.1,11,11,446.4,446.4
|
||||
06-08 20:33:35,1,1,0.0,521.0,11,0,0.0,521.0,11.0,0.0,38.1,88.1,11,11,521.0,521.0
|
||||
06-08 20:33:40,1,1,189.9,531.7,12,0,189.9,531.7,12.0,0.0,41.5,88.2,12,12,531.7,531.7
|
||||
06-08 20:33:45,1,1,0.0,532.8,11,0,0.0,532.8,11.0,0.0,41.4,88.2,11,11,532.8,532.8
|
||||
06-08 20:33:50,1,1,21.1,532.6,12,0,21.1,532.6,12.0,0.0,43.9,88.2,12,12,532.6,532.6
|
||||
06-08 20:33:55,1,1,112.6,581.5,12,0,112.6,581.5,12.0,0.0,44.3,88.3,12,12,581.5,581.5
|
||||
06-08 20:34:00,1,1,0.0,698.0,16,0,0.0,698.0,16.0,0.0,65.7,91.1,16,16,698.0,698.0
|
||||
06-08 20:34:05,1,1,1.0,647.3,17,0,1.0,647.3,17.0,0.0,66.6,91.1,17,17,647.3,647.3
|
||||
06-08 20:34:10,1,1,0.0,664.6,14,0,0.0,664.6,14.0,0.0,52.3,90.1,14,14,664.6,664.6
|
||||
06-08 20:34:15,1,1,97.4,659.5,16,0,97.4,659.5,16.0,0.0,57.1,90.2,16,16,659.5,659.5
|
||||
06-08 20:34:20,1,1,0.0,646.5,14,0,0.0,646.5,14.0,0.0,55.9,90.2,14,14,646.5,646.5
|
||||
06-08 20:34:25,1,1,0.0,614.2,14,0,0.0,614.2,14.0,0.0,56.8,90.2,14,14,614.2,614.2
|
||||
06-08 20:34:30,1,1,29.7,647.1,15,0,29.7,647.1,15.0,0.0,59.1,90.2,15,15,647.1,647.1
|
||||
06-08 20:34:35,1,1,0.0,661.6,15,0,0.0,661.6,15.0,0.0,60.0,90.2,15,15,661.6,661.6
|
||||
06-08 20:34:40,1,1,167.1,669.7,15,0,167.1,669.7,15.0,0.0,61.5,90.3,15,15,669.7,669.7
|
||||
06-08 20:34:45,1,1,277.0,528.6,12,0,277.0,528.6,12.0,0.0,43.7,90.2,12,12,528.6,528.6
|
||||
06-08 20:34:50,1,1,0.0,478.5,11,0,0.0,478.5,11.0,0.0,43.2,90.2,11,11,478.5,478.5
|
||||
06-08 20:34:55,1,1,0.0,408.4,11,0,0.0,408.4,11.0,0.0,43.8,90.2,11,11,408.4,408.4
|
||||
06-08 20:35:00,1,1,0.0,413.4,11,0,0.0,413.4,11.0,0.0,44.4,90.2,11,11,413.4,413.4
|
||||
06-08 20:35:05,1,1,0.0,398.2,11,0,0.0,398.2,11.0,0.0,44.9,90.2,11,11,398.2,398.2
|
||||
06-08 20:35:10,1,1,57.8,470.7,12,0,57.8,470.7,12.0,0.0,47.3,90.3,12,12,470.7,470.7
|
||||
06-08 20:35:15,1,1,42.6,512.3,11,0,42.6,512.3,11.0,0.0,47.7,90.3,11,11,512.3,512.3
|
||||
06-08 20:35:20,1,1,0.0,437.5,9,0,0.0,437.5,9.0,0.0,45.6,90.3,9,9,437.5,437.5
|
||||
06-08 20:35:25,1,1,0.0,389.2,9,0,0.0,389.2,9.0,0.0,46.1,90.3,9,9,389.2,389.2
|
||||
06-08 20:35:30,1,1,0.0,387.0,10,0,0.0,387.0,10.0,0.0,43.0,90.4,10,10,387.0,387.0
|
||||
06-08 20:35:36,1,1,0.0,385.5,10,0,0.0,385.5,10.0,0.0,43.6,90.4,10,10,385.5,385.5
|
||||
06-08 20:35:41,1,1,0.0,389.0,10,0,0.0,389.0,10.0,0.0,44.1,90.4,10,10,389.0,389.0
|
||||
06-08 20:35:46,1,1,0.0,390.5,10,0,0.0,390.5,10.0,0.0,44.7,90.4,10,10,390.5,390.5
|
||||
06-08 20:35:50,1,1,60.0,456.1,11,0,60.0,456.1,11.0,0.0,41.6,90.2,11,11,456.1,456.1
|
||||
06-08 20:35:56,1,1,0.0,520.8,11,0,0.0,520.8,11.0,0.0,44.2,91.2,11,11,520.8,520.8
|
||||
06-08 20:35:57,1,1,0.0,724.9,16,0,0.0,724.9,16.0,0.0,64.2,87.4,16,16,724.9,724.9
|
||||
|
136
training_logs/20260608_210110_vllm_metrics_job_639476.csv
Normal file
136
training_logs/20260608_210110_vllm_metrics_job_639476.csv
Normal file
@@ -0,0 +1,136 @@
|
||||
datetime_str,n_engines_reporting,unique_ips,total_prompt_throughput_tokens_per_sec,total_generation_throughput_tokens_per_sec,total_running_requests,total_waiting_requests,avg_prompt_throughput_per_engine,avg_generation_throughput_per_engine,avg_running_requests_per_engine,avg_waiting_requests_per_engine,avg_gpu_kv_cache_usage_pct,avg_prefix_cache_hit_rate_pct,min_running_requests,max_running_requests,min_generation_throughput,max_generation_throughput
|
||||
06-08 20:44:42,1,1,8.4,0.0,1,0,8.4,0.0,1.0,0.0,0.3,0.0,1,1,0.0,0.0
|
||||
06-08 20:44:48,2,2,8.8,45.2,2,0,4.4,22.6,1.0,0.0,0.35,0.0,1,1,0.0,45.2
|
||||
06-08 20:44:53,1,1,0.0,47.0,1,0,0.0,47.0,1.0,0.0,0.3,0.0,1,1,47.0,47.0
|
||||
06-08 20:44:58,1,1,0.0,47.3,1,0,0.0,47.3,1.0,0.0,0.4,0.0,1,1,47.3,47.3
|
||||
06-08 20:45:03,1,1,0.0,47.7,1,0,0.0,47.7,1.0,0.0,0.5,0.0,1,1,47.7,47.7
|
||||
06-08 20:45:08,1,1,0.0,47.0,1,0,0.0,47.0,1.0,0.0,0.3,0.0,1,1,47.0,47.0
|
||||
06-08 20:45:13,1,1,8.8,0.0,1,0,8.8,0.0,1.0,0.0,0.3,0.0,1,1,0.0,0.0
|
||||
06-08 20:45:18,1,1,0.0,43.7,1,0,0.0,43.7,1.0,0.0,0.5,0.0,1,1,43.7,43.7
|
||||
06-08 20:45:23,1,1,0.0,43.9,1,0,0.0,43.9,1.0,0.0,0.5,0.0,1,1,43.9,43.9
|
||||
06-08 20:45:28,1,1,0.0,44.0,1,0,0.0,44.0,1.0,0.0,0.6,0.0,1,1,44.0,44.0
|
||||
06-08 20:45:33,1,1,0.0,44.1,1,0,0.0,44.1,1.0,0.0,0.6,0.0,1,1,44.1,44.1
|
||||
06-08 20:45:39,1,1,0.0,44.5,1,0,0.0,44.5,1.0,0.0,0.5,0.0,1,1,44.5,44.5
|
||||
06-08 20:45:45,1,1,0.0,47.7,1,0,0.0,47.7,1.0,0.0,0.6,53.2,1,1,47.7,47.7
|
||||
06-08 20:45:51,1,1,153.9,74.9,2,0,153.9,74.9,2.0,0.0,1.1,56.2,2,2,74.9,74.9
|
||||
06-08 20:45:56,1,1,0.0,86.4,2,0,0.0,86.4,2.0,0.0,1.3,56.2,2,2,86.4,86.4
|
||||
06-08 20:46:01,1,1,0.0,86.5,2,0,0.0,86.5,2.0,0.0,1.4,56.2,2,2,86.5,86.5
|
||||
06-08 20:46:06,1,1,0.0,52.4,1,0,0.0,52.4,1.0,0.0,0.8,56.2,1,1,52.4,52.4
|
||||
06-08 20:46:11,1,1,76.6,55.2,2,0,76.6,55.2,2.0,0.0,1.0,57.2,2,2,55.2,55.2
|
||||
06-08 20:46:15,1,1,24.7,1.0,1,0,24.7,1.0,1.0,0.0,0.3,32.8,1,1,1.0,1.0
|
||||
06-08 20:46:21,1,1,0.0,225.2,4,0,0.0,225.2,4.0,0.0,2.3,66.7,4,4,225.2,225.2
|
||||
06-08 20:46:26,1,1,0.0,191.1,4,0,0.0,191.1,4.0,0.0,2.6,66.7,4,4,191.1,191.1
|
||||
06-08 20:46:31,1,1,0.0,49.0,1,0,0.0,49.0,1.0,0.0,0.5,0.0,1,1,49.0,49.0
|
||||
06-08 20:46:36,1,1,0.0,48.9,1,0,0.0,48.9,1.0,0.0,0.6,0.0,1,1,48.9,48.9
|
||||
06-08 20:46:41,1,1,0.0,48.9,1,0,0.0,48.9,1.0,0.0,0.6,0.0,1,1,48.9,48.9
|
||||
06-08 20:46:46,1,1,0.0,48.0,1,0,0.0,48.0,1.0,0.0,0.7,0.0,1,1,48.0,48.0
|
||||
06-08 20:46:51,1,1,0.0,47.4,1,0,0.0,47.4,1.0,0.0,0.8,0.0,1,1,47.4,47.4
|
||||
06-08 20:46:56,1,1,0.0,48.6,1,0,0.0,48.6,1.0,0.0,0.8,0.0,1,1,48.6,48.6
|
||||
06-08 20:47:01,1,1,0.0,48.2,1,0,0.0,48.2,1.0,0.0,0.9,0.0,1,1,48.2,48.2
|
||||
06-08 20:47:06,1,1,0.0,48.5,1,0,0.0,48.5,1.0,0.0,1.0,0.0,1,1,48.5,48.5
|
||||
06-08 20:47:11,1,1,0.0,48.4,1,0,0.0,48.4,1.0,0.0,1.0,0.0,1,1,48.4,48.4
|
||||
06-08 20:47:16,1,1,0.0,48.3,1,0,0.0,48.3,1.0,0.0,1.1,0.0,1,1,48.3,48.3
|
||||
06-08 20:47:21,1,1,88.4,30.7,1,0,88.4,30.7,1.0,0.0,1.2,76.0,1,1,30.7,30.7
|
||||
06-08 20:47:26,1,1,0.0,188.0,4,0,0.0,188.0,4.0,0.0,2.9,69.0,4,4,188.0,188.0
|
||||
06-08 20:47:31,1,1,0.0,46.5,1,0,0.0,46.5,1.0,0.0,1.4,63.1,1,1,46.5,46.5
|
||||
06-08 20:47:36,1,1,153.0,82.7,2,0,153.0,82.7,2.0,0.0,2.5,67.6,2,2,82.7,82.7
|
||||
06-08 20:47:41,1,1,0.0,92.5,1,0,0.0,92.5,1.0,0.0,1.6,67.6,1,1,92.5,92.5
|
||||
06-08 20:47:46,1,1,75.9,70.1,2,0,75.9,70.1,2.0,0.0,2.3,70.5,2,2,70.1,70.1
|
||||
06-08 20:47:51,1,1,0.0,91.3,2,0,0.0,91.3,2.0,0.0,2.4,70.5,2,2,91.3,91.3
|
||||
06-08 20:47:56,1,1,140.2,108.4,3,0,140.2,108.4,3.0,0.0,3.9,73.9,3,3,108.4,108.4
|
||||
06-08 20:48:01,1,1,0.0,134.0,2,0,0.0,134.0,2.0,0.0,3.2,73.9,2,2,134.0,134.0
|
||||
06-08 20:48:06,1,1,0.0,90.1,2,0,0.0,90.1,2.0,0.0,3.3,73.9,2,2,90.1,90.1
|
||||
06-08 20:48:11,1,1,0.0,90.3,2,0,0.0,90.3,2.0,0.0,3.4,73.9,2,2,90.3,90.3
|
||||
06-08 20:48:16,1,1,0.0,50.8,1,0,0.0,50.8,1.0,0.0,1.2,84.6,1,1,50.8,50.8
|
||||
06-08 20:48:21,1,1,119.5,166.8,3,0,119.5,166.8,3.0,0.0,4.8,75.5,3,3,166.8,166.8
|
||||
06-08 20:48:26,1,1,0.0,136.6,3,0,0.0,136.6,3.0,0.0,5.0,75.5,3,3,136.6,136.6
|
||||
06-08 20:48:31,1,1,0.0,129.7,2,0,0.0,129.7,2.0,0.0,4.0,75.5,2,2,129.7,129.7
|
||||
06-08 20:48:37,1,1,76.6,170.5,4,0,76.6,170.5,4.0,0.0,5.5,78.4,4,4,170.5,170.5
|
||||
06-08 20:48:42,1,1,0.0,183.5,4,0,0.0,183.5,4.0,0.0,5.7,78.4,4,4,183.5,183.5
|
||||
06-08 20:48:47,1,1,0.0,185.4,4,0,0.0,185.4,4.0,0.0,6.0,78.4,4,4,185.4,185.4
|
||||
06-08 20:48:52,1,1,72.9,199.7,5,0,72.9,199.7,5.0,0.0,6.6,78.5,5,5,199.7,199.7
|
||||
06-08 20:48:57,1,1,0.0,185.2,3,0,0.0,185.2,3.0,0.0,5.0,78.5,3,3,185.2,185.2
|
||||
06-08 20:49:02,1,1,0.0,138.2,3,0,0.0,138.2,3.0,0.0,5.2,78.5,3,3,138.2,138.2
|
||||
06-08 20:49:07,1,1,0.0,116.8,2,0,0.0,116.8,2.0,0.0,4.9,78.5,2,2,116.8,116.8
|
||||
06-08 20:49:11,1,1,0.0,138.8,3,0,0.0,138.8,3.0,0.0,3.5,84.1,3,3,138.8,138.8
|
||||
06-08 20:49:17,1,1,67.2,122.2,3,0,67.2,122.2,3.0,0.0,5.6,78.8,3,3,122.2,122.2
|
||||
06-08 20:49:22,1,1,0.0,96.0,1,0,0.0,96.0,1.0,0.0,2.8,78.8,1,1,96.0,96.0
|
||||
06-08 20:49:27,1,1,113.4,63.8,2,0,113.4,63.8,2.0,0.0,3.6,78.9,2,2,63.8,63.8
|
||||
06-08 20:49:32,1,1,33.9,97.4,2,0,33.9,97.4,2.0,0.0,3.6,79.8,2,2,97.4,97.4
|
||||
06-08 20:49:37,1,1,0.0,72.7,1,0,0.0,72.7,1.0,0.0,3.0,79.8,1,1,72.7,72.7
|
||||
06-08 20:49:42,1,1,22.7,50.2,2,0,22.7,50.2,2.0,0.0,3.8,80.8,2,2,50.2,50.2
|
||||
06-08 20:49:47,1,1,210.5,117.7,3,0,210.5,117.7,3.0,0.0,6.6,82.3,3,3,117.7,117.7
|
||||
06-08 20:49:52,1,1,0.0,51.7,1,0,0.0,51.7,1.0,0.0,1.4,75.6,1,1,51.7,51.7
|
||||
06-08 20:49:57,1,1,122.0,50.0,2,0,122.0,50.0,2.0,0.0,2.7,80.2,2,2,50.0,50.0
|
||||
06-08 20:50:02,1,1,0.0,93.9,2,0,0.0,93.9,2.0,0.0,2.8,80.2,2,2,93.9,93.9
|
||||
06-08 20:50:07,1,1,0.0,93.8,2,0,0.0,93.8,2.0,0.0,3.0,80.2,2,2,93.8,93.8
|
||||
06-08 20:50:12,1,1,23.2,77.7,2,0,23.2,77.7,2.0,0.0,3.1,85.4,2,2,77.7,77.7
|
||||
06-08 20:50:17,1,1,0.0,94.2,2,0,0.0,94.2,2.0,0.0,3.2,85.4,2,2,94.2,94.2
|
||||
06-08 20:50:22,1,1,0.0,94.1,2,0,0.0,94.1,2.0,0.0,3.4,85.4,2,2,94.1,94.1
|
||||
06-08 20:50:27,1,1,0.0,93.8,2,0,0.0,93.8,2.0,0.0,3.5,85.4,2,2,93.8,93.8
|
||||
06-08 20:50:32,1,1,0.0,75.1,1,0,0.0,75.1,1.0,0.0,1.9,85.4,1,1,75.1,75.1
|
||||
06-08 20:50:37,1,1,0.0,47.2,1,0,0.0,47.2,1.0,0.0,2.0,85.4,1,1,47.2,47.2
|
||||
06-08 20:50:42,1,1,0.0,47.1,1,0,0.0,47.1,1.0,0.0,2.0,85.4,1,1,47.1,47.1
|
||||
06-08 20:50:47,1,1,0.0,47.1,1,0,0.0,47.1,1.0,0.0,2.1,85.4,1,1,47.1,47.1
|
||||
06-08 20:50:52,1,1,0.0,47.1,1,0,0.0,47.1,1.0,0.0,2.2,85.4,1,1,47.1,47.1
|
||||
06-08 20:50:57,1,1,0.0,123.5,2,0,0.0,123.5,2.0,0.0,8.3,85.8,2,2,123.5,123.5
|
||||
06-08 20:51:02,1,1,0.0,92.4,2,0,0.0,92.4,2.0,0.0,8.5,85.8,2,2,92.4,92.4
|
||||
06-08 20:51:07,1,1,136.6,135.6,3,0,136.6,135.6,3.0,0.0,9.6,85.7,3,3,135.6,135.6
|
||||
06-08 20:51:12,1,1,42.8,164.5,4,0,42.8,164.5,4.0,0.0,10.4,85.9,4,4,164.5,164.5
|
||||
06-08 20:51:17,1,1,99.3,216.3,5,0,99.3,216.3,5.0,0.0,10.9,85.5,5,5,216.3,216.3
|
||||
06-08 20:51:22,1,1,0.0,206.8,4,0,0.0,206.8,4.0,0.0,10.4,85.5,4,4,206.8,206.8
|
||||
06-08 20:51:27,1,1,0.0,184.9,4,0,0.0,184.9,4.0,0.0,10.7,85.5,4,4,184.9,184.9
|
||||
06-08 20:51:32,1,1,92.1,219.6,5,0,92.1,219.6,5.0,0.0,11.1,85.1,5,5,219.6,219.6
|
||||
06-08 20:51:37,1,1,0.0,198.1,4,0,0.0,198.1,4.0,0.0,11.0,85.1,4,4,198.1,198.1
|
||||
06-08 20:51:42,1,1,209.6,201.0,6,0,209.6,201.0,6.0,0.0,12.5,84.9,6,6,201.0,201.0
|
||||
06-08 20:51:48,1,1,0.0,258.2,4,0,0.0,258.2,4.0,0.0,11.5,84.9,4,4,258.2,258.2
|
||||
06-08 20:51:53,1,1,23.6,225.9,5,0,23.6,225.9,5.0,0.0,12.7,85.4,5,5,225.9,225.9
|
||||
06-08 20:51:58,1,1,35.0,236.4,5,0,35.0,236.4,5.0,0.0,12.6,85.6,5,5,236.4,236.4
|
||||
06-08 20:52:03,1,1,0.0,191.5,4,0,0.0,191.5,4.0,0.0,12.3,85.6,4,4,191.5,191.5
|
||||
06-08 20:52:08,1,1,0.0,184.7,4,0,0.0,184.7,4.0,0.0,12.6,85.6,4,4,184.7,184.7
|
||||
06-08 20:52:13,1,1,0.0,155.7,3,0,0.0,155.7,3.0,0.0,12.1,85.6,3,3,155.7,155.7
|
||||
06-08 20:52:18,1,1,235.2,169.3,4,0,235.2,169.3,4.0,0.0,13.2,84.9,4,4,169.3,169.3
|
||||
06-08 20:52:23,1,1,0.0,141.5,3,0,0.0,141.5,3.0,0.0,12.5,84.9,3,3,141.5,141.5
|
||||
06-08 20:52:28,1,1,165.8,164.2,5,0,165.8,164.2,5.0,0.0,14.5,85.3,5,5,164.2,164.2
|
||||
06-08 20:52:33,1,1,27.5,56.6,2,0,27.5,56.6,2.0,0.0,1.5,86.3,2,2,56.6,56.6
|
||||
06-08 20:52:38,1,1,0.0,91.5,2,0,0.0,91.5,2.0,0.0,1.6,86.3,2,2,91.5,91.5
|
||||
06-08 20:52:43,1,1,20.9,91.6,3,0,20.9,91.6,3.0,0.0,2.3,86.6,3,3,91.6,91.6
|
||||
06-08 20:52:48,1,1,13.9,213.6,4,0,13.9,213.6,4.0,0.0,12.4,93.7,4,4,213.6,213.6
|
||||
06-08 20:52:53,1,1,9.6,163.8,3,0,9.6,163.8,3.0,0.0,11.4,93.9,3,3,163.8,163.8
|
||||
06-08 20:52:58,1,1,9.6,148.5,4,0,9.6,148.5,4.0,0.0,13.6,94.1,4,4,148.5,148.5
|
||||
06-08 20:53:03,1,1,0.0,179.0,3,0,0.0,179.0,3.0,0.0,11.8,94.1,3,3,179.0,179.0
|
||||
06-08 20:53:08,1,1,0.0,253.0,6,0,0.0,253.0,6.0,0.0,20.3,87.7,6,6,253.0,253.0
|
||||
06-08 20:53:13,1,1,0.0,252.6,6,0,0.0,252.6,6.0,0.0,20.6,87.7,6,6,252.6,252.6
|
||||
06-08 20:53:18,1,1,0.0,252.6,6,0,0.0,252.6,6.0,0.0,21.0,87.7,6,6,252.6,252.6
|
||||
06-08 20:53:23,1,1,0.0,252.3,6,0,0.0,252.3,6.0,0.0,21.3,87.7,6,6,252.3,252.3
|
||||
06-08 20:53:28,1,1,60.3,165.2,4,0,60.3,165.2,4.0,0.0,14.1,82.7,4,4,165.2,165.2
|
||||
06-08 20:53:33,1,1,63.3,191.4,4,0,63.3,191.4,4.0,0.0,14.0,82.6,4,4,191.4,191.4
|
||||
06-08 20:53:38,1,1,56.1,196.8,5,0,56.1,196.8,5.0,0.0,16.4,83.8,5,5,196.8,196.8
|
||||
06-08 20:53:43,1,1,187.6,233.2,6,0,187.6,233.2,6.0,0.0,17.2,84.5,6,6,233.2,233.2
|
||||
06-08 20:53:48,1,1,0.0,205.9,4,0,0.0,205.9,4.0,0.0,14.7,84.5,4,4,205.9,205.9
|
||||
06-08 20:53:53,1,1,0.0,170.6,4,0,0.0,170.6,4.0,0.0,14.9,84.5,4,4,170.6,170.6
|
||||
06-08 20:53:58,1,1,64.9,191.6,5,0,64.9,191.6,5.0,0.0,15.8,84.5,5,5,191.6,191.6
|
||||
06-08 20:54:03,1,1,0.0,213.4,5,0,0.0,213.4,5.0,0.0,16.1,84.5,5,5,213.4,213.4
|
||||
06-08 20:54:08,1,1,52.8,254.3,6,0,52.8,254.3,6.0,0.0,17.0,84.7,6,6,254.3,254.3
|
||||
06-08 20:54:13,1,1,0.0,217.7,4,0,0.0,217.7,4.0,0.0,16.0,84.7,4,4,217.7,217.7
|
||||
06-08 20:54:18,1,1,106.0,193.1,5,0,106.0,193.1,5.0,0.0,17.0,84.7,5,5,193.1,193.1
|
||||
06-08 20:54:23,1,1,0.0,213.7,5,0,0.0,213.7,5.0,0.0,17.3,84.7,5,5,213.7,213.7
|
||||
06-08 20:54:28,1,1,0.0,213.5,5,0,0.0,213.5,5.0,0.0,17.6,84.7,5,5,213.5,213.5
|
||||
06-08 20:54:33,1,1,0.0,284.3,6,0,0.0,284.3,6.0,0.0,24.4,85.8,6,6,284.3,284.3
|
||||
06-08 20:54:38,1,1,33.5,317.3,7,0,33.5,317.3,7.0,0.0,25.1,85.8,7,7,317.3,317.3
|
||||
06-08 20:54:43,1,1,68.9,367.4,8,0,68.9,367.4,8.0,0.0,25.7,85.7,8,8,367.4,367.4
|
||||
06-08 20:54:48,1,1,0.0,378.3,8,0,0.0,378.3,8.0,0.0,26.2,85.7,8,8,378.3,378.3
|
||||
06-08 20:54:53,1,1,60.8,398.5,9,0,60.8,398.5,9.0,0.0,26.9,85.5,9,9,398.5,398.5
|
||||
06-08 20:54:58,1,1,73.8,305.6,8,0,73.8,305.6,8.0,0.0,20.3,84.5,8,8,305.6,305.6
|
||||
06-08 20:55:03,1,1,0.0,297.3,7,0,0.0,297.3,7.0,0.0,20.5,84.5,7,7,297.3,297.3
|
||||
06-08 20:55:08,1,1,0.0,298.0,7,0,0.0,298.0,7.0,0.0,20.9,84.5,7,7,298.0,298.0
|
||||
06-08 20:55:13,1,1,45.8,305.9,8,0,45.8,305.9,8.0,0.0,21.6,84.5,8,8,305.9,305.9
|
||||
06-08 20:55:18,1,1,0.0,310.2,7,0,0.0,310.2,7.0,0.0,21.8,84.5,7,7,310.2,310.2
|
||||
06-08 20:55:23,1,1,0.0,299.4,7,0,0.0,299.4,7.0,0.0,22.2,84.5,7,7,299.4,299.4
|
||||
06-08 20:55:28,1,1,198.0,404.3,9,0,198.0,404.3,9.0,0.0,30.2,85.2,9,9,404.3,404.3
|
||||
06-08 20:55:33,1,1,240.9,419.3,7,0,240.9,419.3,7.0,0.0,29.6,84.8,7,7,419.3,419.3
|
||||
06-08 20:55:38,1,1,236.9,396.2,9,0,236.9,396.2,9.0,0.0,32.1,84.9,9,9,396.2,396.2
|
||||
06-08 20:55:43,1,1,87.2,425.9,8,0,87.2,425.9,8.0,0.0,31.6,84.7,8,8,425.9,425.9
|
||||
06-08 20:55:48,1,1,0.0,378.0,8,0,0.0,378.0,8.0,0.0,32.1,84.7,8,8,378.0,378.0
|
||||
06-08 20:55:53,1,1,0.0,187.7,4,0,0.0,187.7,4.0,0.0,13.3,87.7,4,4,187.7,187.7
|
||||
06-08 20:55:54,1,1,552.6,360.7,9,0,552.6,360.7,9.0,0.0,20.3,85.9,9,9,360.7,360.7
|
||||
|
33903
training_logs/20260608_210110_vllm_metrics_table.csv
Normal file
33903
training_logs/20260608_210110_vllm_metrics_table.csv
Normal file
File diff suppressed because it is too large
Load Diff
195
training_logs/ablation-pymethods2test-seqnorm_629881.out
Normal file
195
training_logs/ablation-pymethods2test-seqnorm_629881.out
Normal file
File diff suppressed because one or more lines are too long
195
training_logs/ablation-pymethods2test-seqnorm_629882.out
Normal file
195
training_logs/ablation-pymethods2test-seqnorm_629882.out
Normal file
File diff suppressed because one or more lines are too long
195
training_logs/ablation-pymethods2test-seqnorm_629883.out
Normal file
195
training_logs/ablation-pymethods2test-seqnorm_629883.out
Normal file
File diff suppressed because one or more lines are too long
195
training_logs/ablation-pymethods2test-seqnorm_629884.out
Normal file
195
training_logs/ablation-pymethods2test-seqnorm_629884.out
Normal file
File diff suppressed because one or more lines are too long
195
training_logs/ablation-pymethods2test-seqnorm_629885.out
Normal file
195
training_logs/ablation-pymethods2test-seqnorm_629885.out
Normal file
File diff suppressed because one or more lines are too long
195
training_logs/ablation-pymethods2test-seqnorm_629886.out
Normal file
195
training_logs/ablation-pymethods2test-seqnorm_629886.out
Normal file
File diff suppressed because one or more lines are too long
195
training_logs/ablation-pymethods2test-seqnorm_629887.out
Normal file
195
training_logs/ablation-pymethods2test-seqnorm_629887.out
Normal file
File diff suppressed because one or more lines are too long
34464
training_logs/ablation-pymethods2test-seqnorm_632522.out
Normal file
34464
training_logs/ablation-pymethods2test-seqnorm_632522.out
Normal file
File diff suppressed because one or more lines are too long
31562
training_logs/ablation-pymethods2test-seqnorm_632523.out
Normal file
31562
training_logs/ablation-pymethods2test-seqnorm_632523.out
Normal file
File diff suppressed because one or more lines are too long
29753
training_logs/ablation-pymethods2test-seqnorm_632524.out
Normal file
29753
training_logs/ablation-pymethods2test-seqnorm_632524.out
Normal file
File diff suppressed because one or more lines are too long
27052
training_logs/ablation-pymethods2test-seqnorm_632525.out
Normal file
27052
training_logs/ablation-pymethods2test-seqnorm_632525.out
Normal file
File diff suppressed because one or more lines are too long
3
training_logs/ablation-pymethods2test-seqnorm_639470.out
Normal file
3
training_logs/ablation-pymethods2test-seqnorm_639470.out
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:344b0fb95da3cd26cdceae0b7c57286959a5f64b3d590903639b09b125a63a17
|
||||
size 41491386
|
||||
3
training_logs/ablation-pymethods2test-seqnorm_639471.out
Normal file
3
training_logs/ablation-pymethods2test-seqnorm_639471.out
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bd3d99c575ffbca51af3091afbedd03c07b415d664e75c044b5d37d79536045e
|
||||
size 32809309
|
||||
405
training_logs/ablation-pymethods2test-seqnorm_639472.out
Normal file
405
training_logs/ablation-pymethods2test-seqnorm_639472.out
Normal file
@@ -0,0 +1,405 @@
|
||||
|
||||
The following have been reloaded with a version change:
|
||||
1) GCCcore/.14.3.0 => GCCcore/14.3.0
|
||||
|
||||
|
||||
Lmod is automatically replacing "GCC/14.3.0" with
|
||||
"nvidia-compilers/25.9-CUDA-13".
|
||||
|
||||
Deactivating conda environment: /e/scratch/jureap59/feuer1/miniforge3/envs/otagent
|
||||
Activating RL environment: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl
|
||||
Python executable: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python
|
||||
Python path check: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python
|
||||
[ray] RAY_TMPDIR=/tmp/ray/ray_639472
|
||||
[triton_cache] Triton cache: /tmp/triton_cache_feuer1_639472
|
||||
[triton_cache] TorchInductor cache: /tmp/torchinductor_cache_feuer1_639472
|
||||
[proxy] ✓ Found proxychains binary at /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4
|
||||
[proxy] Setting up SSH tunnel to jpbl-s01-01
|
||||
[proxy] SSH key: /e/home/jusers/feuer1/jupiter/.ssh/authorized_keys/id_ed25519_jsc
|
||||
[proxy] Tunnel port: 7003
|
||||
[proxy] Node IP: 10.128.32.49 (workers will connect here)
|
||||
[proxy] ✓ SSH tunnel started successfully
|
||||
[proxy] ✓ Generated proxychains config at /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_639472.conf
|
||||
[proxy] - Internal traffic (10.x.x.x, 172.x.x.x, 169.254.x.x) → DIRECT
|
||||
[proxy] - External traffic (internet) → PROXY via tunnel
|
||||
[proxy] ✓ Daytona timeout settings configured
|
||||
[proxy] Testing proxy connectivity...
|
||||
[proxychains] config file found: /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_639472.conf
|
||||
[proxychains] preloading /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/lib/libproxychains4.so
|
||||
[proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
|
||||
[proxy] ✓ Proxy connectivity test passed (huggingface.co reachable via wrapped binary)
|
||||
[proxy] ⚠ Tunnel not accessible at 10.128.32.49:7003 (workers may fail)
|
||||
[proxy] ✓ Proxy setup complete (using wrapped binary for Ray workers)
|
||||
[container_runtime] Using cloud backend: daytona (no local container setup)
|
||||
=== Universal RL Training Runner ===
|
||||
Config: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/configs/ablation-pymethods2test-seqnorm_rl_config.json
|
||||
Working directory: /e/scratch/jureap59/feuer1/OpenThoughts-Agent
|
||||
Python: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python
|
||||
Python version: Python 3.12.12
|
||||
UV_USE_IO_URING: 0
|
||||
Proxy: DISABLED (direct internet or not configured)
|
||||
========================================
|
||||
=== RLJobRunner: ablation-pymethods2test-seqnorm ===
|
||||
[wandb_utils] Fixing permissions on: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/wandb
|
||||
[wandb_utils] WandB directory ready: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/wandb
|
||||
HF_TOKEN=****pDbg
|
||||
HF_HUB_CACHE=/e/data1/datasets/playground/ot-baf/hf_hub
|
||||
SUPABASE_URL=https://rpzmyuapoqilpghynmza.s... (direct Supabase config)
|
||||
Environment configured:
|
||||
TENSOR_PARALLEL_SIZE=1
|
||||
NUM_INFERENCE_ENGINES=56
|
||||
POLICY_NUM_NODES=14
|
||||
WANDB_DIR=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/wandb
|
||||
Starting Ray cluster with 14 nodes, 4 GPUs/node
|
||||
Cleaning up existing Ray instances...
|
||||
=== Starting Ray Cluster ===
|
||||
Nodes: 14
|
||||
GPUs per node: 4
|
||||
CPUs per node: 288
|
||||
Head node: jpbo-042-01 (10.128.32.49)
|
||||
Ray port: 6379
|
||||
============================
|
||||
Starting Ray head on jpbo-042-01 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_head_jpbo-042-01.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.49 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-01 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --head --node-ip-address=10.128.32.49 --port=6379 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray head on jpbo-042-01
|
||||
Starting Ray worker on jpbo-042-02 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-02.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.50 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-02 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.50 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 1 on jpbo-042-02
|
||||
Starting Ray worker on jpbo-042-03 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-03.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.51 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-03 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.51 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 2 on jpbo-042-03
|
||||
Starting Ray worker on jpbo-042-04 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-04.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.52 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-04 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.52 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 3 on jpbo-042-04
|
||||
Starting Ray worker on jpbo-042-05 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-05.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.53 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-05 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.53 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 4 on jpbo-042-05
|
||||
Starting Ray worker on jpbo-042-06 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-06.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.54 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-06 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.54 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 5 on jpbo-042-06
|
||||
Starting Ray worker on jpbo-042-07 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-07.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.55 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-07 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.55 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 6 on jpbo-042-07
|
||||
Starting Ray worker on jpbo-042-08 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-08.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.56 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-08 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.56 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 7 on jpbo-042-08
|
||||
Starting Ray worker on jpbo-042-09 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-09.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.57 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-09 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.57 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 8 on jpbo-042-09
|
||||
Starting Ray worker on jpbo-042-10 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-10.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.58 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-10 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.58 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 9 on jpbo-042-10
|
||||
Starting Ray worker on jpbo-042-11 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-11.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.59 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-11 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.59 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 10 on jpbo-042-11
|
||||
Starting Ray worker on jpbo-042-12 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-12.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.60 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-12 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.60 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 11 on jpbo-042-12
|
||||
Starting Ray worker on jpbo-042-13 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-13.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.61 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-13 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.61 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 12 on jpbo-042-13
|
||||
Starting Ray worker on jpbo-042-14 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-042-14.log)...
|
||||
Command: srun --export=ALL,VLLM_HOST_IP=10.128.32.62 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-042-14 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.32.49:6379 --node-ip-address=10.128.32.62 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960
|
||||
Started Ray worker 13 on jpbo-042-14
|
||||
Waiting for cluster (56 GPUs, 14 nodes)...
|
||||
Connecting to Ray at 10.128.32.49:6379 (expecting 14 nodes, 56.0 GPUs)
|
||||
[Ray wait] Connection attempt failed:
|
||||
[Ray wait] Retrying in 10s...
|
||||
Ray connection established, polling for resources...
|
||||
[Ray wait] nodes=6/14 GPUs=24.0/56.0 resources={'accelerator_type:GH200': 6.0, 'GPU': 24.0, 'object_store_memory': 257698037760.0, 'memory': 4432041082880.0, 'CPU': 1728.0, 'node:10.128.32.61': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=6/14 GPUs=24.0/56.0 resources={'accelerator_type:GH200': 6.0, 'GPU': 24.0, 'memory': 4432041082880.0, 'object_store_memory': 257698037760.0, 'CPU': 1728.0, 'node:10.128.32.61': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=7/14 GPUs=28.0/56.0 resources={'accelerator_type:GH200': 7.0, 'GPU': 28.0, 'memory': 5174823026688.0, 'object_store_memory': 300647710720.0, 'CPU': 2016.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=9/14 GPUs=36.0/56.0 resources={'accelerator_type:GH200': 9.0, 'node:10.128.32.55': 1.0, 'memory': 6645910339584.0, 'object_store_memory': 386547056640.0, 'GPU': 36.0, 'CPU': 2592.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=9/14 GPUs=36.0/56.0 resources={'accelerator_type:GH200': 9.0, 'node:10.128.32.55': 1.0, 'memory': 6645910339584.0, 'GPU': 36.0, 'object_store_memory': 386547056640.0, 'CPU': 2592.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=9/14 GPUs=36.0/56.0 resources={'accelerator_type:GH200': 9.0, 'node:10.128.32.55': 1.0, 'memory': 6645910339584.0, 'GPU': 36.0, 'object_store_memory': 386547056640.0, 'CPU': 2592.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=9/14 GPUs=36.0/56.0 resources={'accelerator_type:GH200': 9.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 386547056640.0, 'GPU': 36.0, 'memory': 6645910339584.0, 'CPU': 2592.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=9/14 GPUs=36.0/56.0 resources={'accelerator_type:GH200': 9.0, 'GPU': 36.0, 'memory': 6645910339584.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 386547056640.0, 'CPU': 2592.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=9/14 GPUs=36.0/56.0 resources={'accelerator_type:GH200': 9.0, 'GPU': 36.0, 'memory': 6645910339584.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 386547056640.0, 'CPU': 2592.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=9/14 GPUs=36.0/56.0 resources={'accelerator_type:GH200': 9.0, 'GPU': 36.0, 'memory': 6645910339584.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 386547056640.0, 'CPU': 2592.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=9/14 GPUs=36.0/56.0 resources={'accelerator_type:GH200': 9.0, 'GPU': 36.0, 'object_store_memory': 386547056640.0, 'node:10.128.32.55': 1.0, 'memory': 6645910339584.0, 'CPU': 2592.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'node:10.128.32.55': 1.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'node:10.128.32.55': 1.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'node:10.128.32.55': 1.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'node:10.128.32.55': 1.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'object_store_memory': 429496729600.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'GPU': 40.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'node:10.128.32.55': 1.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'memory': 7399658749952.0, 'object_store_memory': 429496729600.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
[Ray wait] nodes=10/14 GPUs=40.0/56.0 resources={'accelerator_type:GH200': 10.0, 'node:10.128.32.55': 1.0, 'object_store_memory': 429496729600.0, 'memory': 7399658749952.0, 'GPU': 40.0, 'CPU': 2880.0, 'node:10.128.32.62': 1.0, 'node:10.128.32.61': 1.0, 'node:10.128.32.57': 1.0, 'node:10.128.32.60': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.32.49': 1.0, 'node:10.128.32.51': 1.0, 'node:10.128.32.59': 1.0, 'node:10.128.32.58': 1.0, 'node:10.128.32.50': 1.0}
|
||||
srun stderr: srun: error: _find_node_record: lookup failure for node "jpbo-123-29"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-30"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-31"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-32"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-33"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-34"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-35"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-36"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-37"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-38"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-39"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-40"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-41"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-42"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-43"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-44"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-45"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-46"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-47"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-123-48"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-01"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-02"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-03"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-04"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-05"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-06"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-07"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-08"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-09"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-10"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-11"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-12"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-13"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-14"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-15"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-16"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-17"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-18"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-19"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-20"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-21"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-22"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-23"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-24"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-25"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-26"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-27"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-28"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-29"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-30"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-31"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-32"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-33"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-34"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-35"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-36"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-37"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-38"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-39"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-40"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-41"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-42"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-43"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-44"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-45"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-46"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-47"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-124-48"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-01"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-02"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-03"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-04"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-05"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-06"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-07"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-08"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-09"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-10"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-11"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-12"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-13"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-14"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-15"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-16"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-17"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-18"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-19"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-20"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-21"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-22"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-23"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-24"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-25"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-26"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-27"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-28"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-29"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-30"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-31"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-32"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-33"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-34"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-35"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-36"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-37"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-38"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-39"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-40"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-41"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-42"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-43"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-44"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-45"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-46"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-47"
|
||||
srun: error: _find_node_record: lookup failure for node "jpbo-125-48"
|
||||
srun: warning: Invalid hostnames in switch configuration: jpbo-123-[29-48],jpbo-124-[01-48],jpbo-125-[01-48]
|
||||
2026-06-07 18:51:57,434 INFO worker.py:1821 -- Connecting to existing Ray cluster at address: 10.128.32.49:6379...
|
||||
[2026-06-07 18:52:02,440 W 356312 356312] rpc_client.h:153: Failed to connect to GCS at address 10.128.32.49:6379 within 5 seconds.
|
||||
[2026-06-07 18:52:32,441 W 356312 356312] gcs_client.cc:205: Failed to get cluster ID from GCS server: TimedOut: Timed out while waiting for GCS to become available.
|
||||
[2026-06-07 18:52:37,527 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:38,527 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:39,528 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:40,529 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:41,529 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:42,530 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:43,531 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:44,531 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:45,532 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:46,533 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:47,533 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:48,534 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:49,535 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:50,535 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:51,536 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:52,537 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:53,537 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:54,538 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:55,538 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:56,539 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:57,540 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:58,540 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:52:59,541 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:00,541 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:01,542 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:02,543 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:03,543 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:04,544 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:05,544 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:06,545 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
2026-06-07 18:53:17,581 INFO worker.py:1821 -- Connecting to existing Ray cluster at address: 10.128.32.49:6379...
|
||||
[2026-06-07 18:53:17,585 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:18,585 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:19,586 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:20,587 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:21,587 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:22,588 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:23,589 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:24,589 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:25,590 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:26,590 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:27,591 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:28,592 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:29,592 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:30,593 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:31,593 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:32,594 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:33,595 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:34,595 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:35,596 W 356312 356312] global_state_accessor.cc:505: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
|
||||
[2026-06-07 18:53:36,597 I 356312 356312] global_state_accessor.cc:487: This node has an IP address of 10.128.32.49, but we cannot find a local Raylet with the same address. This can happen when you connect to the Ray cluster with a different IP address or when connecting to a container.
|
||||
2026-06-07 18:53:36,599 INFO worker.py:2007 -- Connected to Ray cluster.
|
||||
/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
|
||||
warnings.warn(
|
||||
[33m(raylet, ip=10.128.32.61)[0m [2026-06-07 18:54:04,936 E 3809966 3809966] (raylet) main.cc:1032: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
|
||||
[33m(raylet, ip=10.128.32.59)[0m [2026-06-07 18:54:21,624 E 687258 687258] (raylet) main.cc:1032: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14[32m [repeated 2x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)[0m
|
||||
[36m(pid=357163)[0m [2026-06-07 18:54:24,645 E 357163 357912] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
|
||||
[33m(raylet, ip=10.128.32.55)[0m [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
|
||||
[36m(pid=357194)[0m [2026-06-07 18:54:30,665 E 357194 358894] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14[32m [repeated 18x across cluster][0m
|
||||
[33m(raylet, ip=10.128.32.55)[0m [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
|
||||
[33m(raylet, ip=10.128.32.62)[0m [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
|
||||
[2026-06-07 18:54:34,060 E 356312 357159] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
|
||||
[36m(pid=357379)[0m [2026-06-07 18:54:34,002 E 357379 373376] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14[32m [repeated 269x across cluster][0m
|
||||
[33m(raylet, ip=10.128.32.60)[0m [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e[32m [repeated 3x across cluster][0m
|
||||
[33m(raylet, ip=10.128.32.60)[0m [2026-06-07 18:54:51,245 E 3920479 3920479] (raylet) main.cc:1032: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
|
||||
[33m(raylet, ip=10.128.32.62)[0m [2026-06-07 18:55:00,132 E 3804145 3804145] (raylet) main.cc:1032: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14[32m [repeated 2x across cluster][0m
|
||||
[33m(raylet, ip=10.128.32.57)[0m [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
|
||||
[33m(raylet, ip=10.128.32.57)[0m [2026-06-07 18:56:20,331 E 3787540 3787540] (raylet) main.cc:1032: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14
|
||||
[33m(raylet, ip=10.128.32.57)[0m [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e
|
||||
Traceback (most recent call last):
|
||||
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/scripts/ray/wait_for_cluster.py", line 116, in <module>
|
||||
main()
|
||||
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/scripts/ray/wait_for_cluster.py", line 106, in main
|
||||
wait_for_cluster(
|
||||
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/scripts/ray/wait_for_cluster.py", line 94, in wait_for_cluster
|
||||
raise TimeoutError(
|
||||
TimeoutError: Ray cluster did not reach desired resources within 600 seconds (nodes=10, gpus=40.0)
|
||||
srun: error: jpbo-042-01: task 0: Exited with exit code 1
|
||||
|
||||
RL job failed: Ray cluster failed to start within 600s (last error: Command '['srun', '--export=ALL,WANDB_MODE=offline,GLOO_USE_IPV6=0,NCCL_SOCKET_FAMILY=AF_INET,VLLM_FORCE_IPV4=1,VLLM_SKIP_FLAG_DISCOVERY=1,SKYRL_ENABLE_NUMA_AFFINITY=1,DISABLE_AIOHTTP_TRANSPORT=True,VLLM_ALLREDUCE_USE_SYMM_MEM=0,TORCH_CUDNN_SDPA_ENABLED=0,PYTHONFAULTHANDLER=1,TORCH_NCCL_ASYNC_ERROR_HANDLING=1,TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC=1800,TORCH_NCCL_BLOCKING_WAIT_TIMEOUT_MS=1800000,VLLM_MQ_MAX_CHUNKS=240,OT_AGENT_RAY_LOG_DIR=/e/data1/datasets/playground/ot-baf/experiments/_ray_logs,TORCH_NCCL_TRACE_BUFFER_SIZE=10000,TORCH_FR_BUFFER_SIZE=10000,TORCH_NCCL_DESYNC_DEBUG=1,TORCH_NCCL_DEBUG_INFO_TEMP_FILE=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps/nccl_trace,VLLM_PYNCCL_TRACE_BUFFER_SIZE=10000,VLLM_PYNCCL_TRACE_DUMP_DIR=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps/nccl_trace,VLLM_RAY_EXTRA_ENV_VAR_PREFIXES_TO_COPY=TORCH_NCCL_,TORCH_FR_,VLLM_PYNCCL_', '--nodes=1', '--ntasks=1', '--overlap', '--cpu-bind=none', '-w', 'jpbo-042-01', 'bash', '-c', 'unset LD_PRELOAD PROXYCHAINS_CONF_FILE 2>/dev/null; /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python scripts/ray/wait_for_cluster.py --address 10.128.32.49:6379 --expected-gpus 56 --expected-nodes 14 --timeout 600 --poll-interval 10']' returned non-zero exit status 1.)
|
||||
Traceback (most recent call last):
|
||||
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/hpc/ray_utils.py", line 651, in _wait_for_cluster
|
||||
result = subprocess.run(
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/e/scratch/jureap59/feuer1/miniforge3/lib/python3.12/subprocess.py", line 571, in run
|
||||
raise CalledProcessError(retcode, process.args,
|
||||
subprocess.CalledProcessError: Command '['srun', '--export=ALL,WANDB_MODE=offline,GLOO_USE_IPV6=0,NCCL_SOCKET_FAMILY=AF_INET,VLLM_FORCE_IPV4=1,VLLM_SKIP_FLAG_DISCOVERY=1,SKYRL_ENABLE_NUMA_AFFINITY=1,DISABLE_AIOHTTP_TRANSPORT=True,VLLM_ALLREDUCE_USE_SYMM_MEM=0,TORCH_CUDNN_SDPA_ENABLED=0,PYTHONFAULTHANDLER=1,TORCH_NCCL_ASYNC_ERROR_HANDLING=1,TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC=1800,TORCH_NCCL_BLOCKING_WAIT_TIMEOUT_MS=1800000,VLLM_MQ_MAX_CHUNKS=240,OT_AGENT_RAY_LOG_DIR=/e/data1/datasets/playground/ot-baf/experiments/_ray_logs,TORCH_NCCL_TRACE_BUFFER_SIZE=10000,TORCH_FR_BUFFER_SIZE=10000,TORCH_NCCL_DESYNC_DEBUG=1,TORCH_NCCL_DEBUG_INFO_TEMP_FILE=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps/nccl_trace,VLLM_PYNCCL_TRACE_BUFFER_SIZE=10000,VLLM_PYNCCL_TRACE_DUMP_DIR=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps/nccl_trace,VLLM_RAY_EXTRA_ENV_VAR_PREFIXES_TO_COPY=TORCH_NCCL_,TORCH_FR_,VLLM_PYNCCL_', '--nodes=1', '--ntasks=1', '--overlap', '--cpu-bind=none', '-w', 'jpbo-042-01', 'bash', '-c', 'unset LD_PRELOAD PROXYCHAINS_CONF_FILE 2>/dev/null; /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python scripts/ray/wait_for_cluster.py --address 10.128.32.49:6379 --expected-gpus 56 --expected-nodes 14 --timeout 600 --poll-interval 10']' returned non-zero exit status 1.
|
||||
|
||||
The above exception was the direct cause of the following exception:
|
||||
|
||||
Traceback (most recent call last):
|
||||
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/hpc/rl_launch_utils.py", line 1285, in run
|
||||
training_exit_code = self._run_with_ray()
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/hpc/rl_launch_utils.py", line 1532, in _run_with_ray
|
||||
with RayCluster.from_slurm(ray_cfg) as ray_cluster:
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/hpc/ray_utils.py", line 888, in __enter__
|
||||
self.start()
|
||||
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/hpc/ray_utils.py", line 377, in start
|
||||
self._wait_for_cluster()
|
||||
File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/hpc/ray_utils.py", line 679, in _wait_for_cluster
|
||||
raise RuntimeError(
|
||||
RuntimeError: Ray cluster failed to start within 600s (last error: Command '['srun', '--export=ALL,WANDB_MODE=offline,GLOO_USE_IPV6=0,NCCL_SOCKET_FAMILY=AF_INET,VLLM_FORCE_IPV4=1,VLLM_SKIP_FLAG_DISCOVERY=1,SKYRL_ENABLE_NUMA_AFFINITY=1,DISABLE_AIOHTTP_TRANSPORT=True,VLLM_ALLREDUCE_USE_SYMM_MEM=0,TORCH_CUDNN_SDPA_ENABLED=0,PYTHONFAULTHANDLER=1,TORCH_NCCL_ASYNC_ERROR_HANDLING=1,TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC=1800,TORCH_NCCL_BLOCKING_WAIT_TIMEOUT_MS=1800000,VLLM_MQ_MAX_CHUNKS=240,OT_AGENT_RAY_LOG_DIR=/e/data1/datasets/playground/ot-baf/experiments/_ray_logs,TORCH_NCCL_TRACE_BUFFER_SIZE=10000,TORCH_FR_BUFFER_SIZE=10000,TORCH_NCCL_DESYNC_DEBUG=1,TORCH_NCCL_DEBUG_INFO_TEMP_FILE=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps/nccl_trace,VLLM_PYNCCL_TRACE_BUFFER_SIZE=10000,VLLM_PYNCCL_TRACE_DUMP_DIR=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps/nccl_trace,VLLM_RAY_EXTRA_ENV_VAR_PREFIXES_TO_COPY=TORCH_NCCL_,TORCH_FR_,VLLM_PYNCCL_', '--nodes=1', '--ntasks=1', '--overlap', '--cpu-bind=none', '-w', 'jpbo-042-01', 'bash', '-c', 'unset LD_PRELOAD PROXYCHAINS_CONF_FILE 2>/dev/null; /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python scripts/ray/wait_for_cluster.py --address 10.128.32.49:6379 --expected-gpus 56 --expected-nodes 14 --timeout 600 --poll-interval 10']' returned non-zero exit status 1.)
|
||||
[RLJobRunner] Crash detected (exit!=0) — preserving Ray logs to /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm/ray_logs BEFORE trace upload (so a wall-clock kill can't lose crash evidence)...
|
||||
[RLJobRunner] Crash-time Ray log preservation timed out (600s); continuing.
|
||||
[RLJobRunner] Launching trace upload (training exit code: 1):
|
||||
repo_id: DCAgent/ablation-pymethods2test-seqnorm
|
||||
job_dir: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ablation-pymethods2test-seqnorm
|
||||
episodes: last
|
||||
log: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/logs/ablation-pymethods2test-seqnorm_trace_upload.log
|
||||
[RLJobRunner] Waiting for trace upload to complete...
|
||||
[RLJobRunner] Trace upload failed with exit code 1.
|
||||
Preserving Ray logs to /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_3/ray_logs/
|
||||
Collecting Ray logs from worker jpbo-042-02...
|
||||
Collecting Ray logs from worker jpbo-042-03...
|
||||
Collecting Ray logs from worker jpbo-042-04...
|
||||
Collecting Ray logs from worker jpbo-042-05...
|
||||
Collecting Ray logs from worker jpbo-042-06...
|
||||
Collecting Ray logs from worker jpbo-042-07...
|
||||
Collecting Ray logs from worker jpbo-042-08...
|
||||
Collecting Ray logs from worker jpbo-042-09...
|
||||
Collecting Ray logs from worker jpbo-042-10...
|
||||
Collecting Ray logs from worker jpbo-042-11...
|
||||
Collecting Ray logs from worker jpbo-042-12...
|
||||
Collecting Ray logs from worker jpbo-042-13...
|
||||
Collecting Ray logs from worker jpbo-042-14...
|
||||
Ray log preservation complete
|
||||
3
training_logs/ablation-pymethods2test-seqnorm_639473.out
Normal file
3
training_logs/ablation-pymethods2test-seqnorm_639473.out
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:87f7ffbaa62c2a6b35f6d9dcf6ca307c791fade55fc1f969a56a950b2aa6e6a0
|
||||
size 27315985
|
||||
3
training_logs/ablation-pymethods2test-seqnorm_639474.out
Normal file
3
training_logs/ablation-pymethods2test-seqnorm_639474.out
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a3aa1bc3c1ec390f79d8a23ff3acfd863c45e840553dae74c60f804cef0ff984
|
||||
size 30844025
|
||||
33224
training_logs/ablation-pymethods2test-seqnorm_639475.out
Normal file
33224
training_logs/ablation-pymethods2test-seqnorm_639475.out
Normal file
File diff suppressed because one or more lines are too long
8685
training_logs/ablation-pymethods2test-seqnorm_639476.out
Normal file
8685
training_logs/ablation-pymethods2test-seqnorm_639476.out
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user