2026-03-09 14:14:23,124 [INFO] Ollama monitor thread started 2026-03-09 14:14:23,169 [DEBUG] Ollama health ping OK (43.8ms) 2026-03-09 14:14:28,860 [ERROR] HTTP 404 from Telegram: {"ok":false,"error_code":404,"description":"Not Found"} 2026-03-09 14:14:28,860 [INFO] FRANKENSTALLM Ollama Benchmark 2026-03-09 14:14:28,860 [INFO] ============================================================ 2026-03-09 14:14:28,860 [INFO] Models: frankenstallm-3b 2026-03-09 14:14:28,860 [INFO] Tests: 5 2026-03-09 14:14:28,860 [INFO] Time: 2026-03-09 14:14:23 2026-03-09 14:14:28,860 [INFO] ============================================================ 2026-03-09 14:14:28,860 [INFO] ------------------------------------------------------------ 2026-03-09 14:14:28,860 [INFO] Model: frankenstallm-3b 2026-03-09 14:14:28,860 [INFO] ------------------------------------------------------------ 2026-03-09 14:14:28,862 [DEBUG] API request start: model=frankenstallm-3b prompt_len=171 2026-03-09 14:14:29,058 [ERROR] API error (attempt 1/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:14:29,058 [INFO] Retry 1/3 in 2s... 2026-03-09 14:14:31,058 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:14:31,066 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:14:53,171 [DEBUG] Ollama health ping OK (1.8ms) 2026-03-09 14:15:01,068 [INFO] Ollama recovered after restart 2026-03-09 14:15:01,068 [DEBUG] API request start: model=frankenstallm-3b prompt_len=171 2026-03-09 14:15:11,931 [ERROR] API error (attempt 2/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:15:11,931 [INFO] Retry 2/3 in 4s... 2026-03-09 14:15:15,932 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:15:15,940 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:15:28,176 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:15:45,941 [INFO] Ollama recovered after restart 2026-03-09 14:15:45,942 [DEBUG] API request start: model=frankenstallm-3b prompt_len=171 2026-03-09 14:15:56,880 [ERROR] API error (attempt 3/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:15:56,881 [DEBUG] Checkpoint saved: 1 completed pairs 2026-03-09 14:15:56,881 [ERROR] [nlu_01] ERROR: Remote end closed connection without response 2026-03-09 14:15:56,885 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:15:56,892 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:15:58,181 [ERROR] Ollama health ping FAILED: 2026-03-09 14:16:26,894 [INFO] Ollama recovered after restart 2026-03-09 14:16:26,895 [DEBUG] API request start: model=frankenstallm-3b prompt_len=108 2026-03-09 14:16:28,183 [DEBUG] Ollama health ping OK (1.6ms) 2026-03-09 14:16:37,305 [ERROR] API error (attempt 1/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:16:37,306 [INFO] Retry 1/3 in 2s... 2026-03-09 14:16:39,310 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:16:39,318 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:16:59,745 [DEBUG] Ollama health ping OK (1562.3ms) 2026-03-09 14:17:09,319 [INFO] Ollama recovered after restart 2026-03-09 14:17:09,320 [DEBUG] API request start: model=frankenstallm-3b prompt_len=108 2026-03-09 14:17:20,062 [ERROR] API error (attempt 2/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:17:20,062 [INFO] Retry 2/3 in 4s... 2026-03-09 14:17:24,063 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:17:24,070 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:17:34,751 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:17:54,073 [INFO] Ollama recovered after restart 2026-03-09 14:17:54,073 [DEBUG] API request start: model=frankenstallm-3b prompt_len=108 2026-03-09 14:18:04,598 [ERROR] API error (attempt 3/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:18:04,598 [DEBUG] Checkpoint saved: 2 completed pairs 2026-03-09 14:18:04,598 [ERROR] [nlu_02] ERROR: Remote end closed connection without response 2026-03-09 14:18:04,599 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:18:04,606 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:18:04,752 [ERROR] Ollama health ping FAILED: 2026-03-09 14:18:34,608 [INFO] Ollama recovered after restart 2026-03-09 14:18:34,609 [DEBUG] API request start: model=frankenstallm-3b prompt_len=77 2026-03-09 14:18:34,753 [DEBUG] Ollama health ping OK (1.6ms) 2026-03-09 14:18:45,820 [ERROR] API error (attempt 1/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:18:45,820 [INFO] Retry 1/3 in 2s... 2026-03-09 14:18:47,823 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:18:47,831 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:19:09,759 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:19:17,833 [INFO] Ollama recovered after restart 2026-03-09 14:19:17,833 [DEBUG] API request start: model=frankenstallm-3b prompt_len=77 2026-03-09 14:19:28,898 [ERROR] API error (attempt 2/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:19:28,899 [INFO] Retry 2/3 in 4s... 2026-03-09 14:19:32,900 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:19:32,907 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:19:44,765 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:20:02,909 [INFO] Ollama recovered after restart 2026-03-09 14:20:02,909 [DEBUG] API request start: model=frankenstallm-3b prompt_len=77 2026-03-09 14:20:13,486 [ERROR] API error (attempt 3/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:20:13,486 [ERROR] [Benchmark CIRCUIT BREAKER] model=frankenstallm-3b opened after 3 consecutive failures 2026-03-09 14:20:14,766 [ERROR] Ollama health ping FAILED: 2026-03-09 14:20:19,256 [ERROR] HTTP 404 from Telegram: {"ok":false,"error_code":404,"description":"Not Found"} 2026-03-09 14:20:19,256 [DEBUG] Checkpoint saved: 3 completed pairs 2026-03-09 14:20:19,256 [ERROR] [nlu_03] ERROR: Remote end closed connection without response 2026-03-09 14:20:19,256 [WARNING] Circuit breaker OPEN for frankenstallm-3b — skipping remaining 5 tests 2026-03-09 14:20:19,258 [INFO] Checkpoint file deleted (clean completion) 2026-03-09 14:20:19,259 [INFO] ============================================================ 2026-03-09 14:20:19,259 [INFO] SUMMARY 2026-03-09 14:20:19,259 [INFO] ============================================================ 2026-03-09 14:20:19,259 [INFO] frankenstallm-3b auto_avg= 0.0 avg_tps= 0.0 avg_ttft= 0.0ms 2026-03-09 14:20:19,259 [INFO] Results: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/results/ollama_benchmark_results.json 2026-03-09 14:20:19,259 [INFO] Summary: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/results/ollama_benchmark_summary.md 2026-03-09 14:20:24,928 [ERROR] HTTP 404 from Telegram: {"ok":false,"error_code":404,"description":"Not Found"} 2026-03-09 14:21:04,020 [INFO] Ollama monitor thread started 2026-03-09 14:21:04,066 [DEBUG] Ollama health ping OK (44.9ms) 2026-03-09 14:21:09,750 [ERROR] HTTP 404 from Telegram: {"ok":false,"error_code":404,"description":"Not Found"} 2026-03-09 14:21:09,750 [INFO] FRANKENSTALLM Ollama Benchmark 2026-03-09 14:21:09,750 [INFO] ============================================================ 2026-03-09 14:21:09,750 [INFO] Models: frankenstallm-3b 2026-03-09 14:21:09,750 [INFO] Tests: 5 2026-03-09 14:21:09,750 [INFO] Time: 2026-03-09 14:21:04 2026-03-09 14:21:09,750 [INFO] ============================================================ 2026-03-09 14:21:09,750 [INFO] ------------------------------------------------------------ 2026-03-09 14:21:09,750 [INFO] Model: frankenstallm-3b 2026-03-09 14:21:09,750 [INFO] ------------------------------------------------------------ 2026-03-09 14:21:09,752 [DEBUG] API request start: model=frankenstallm-3b prompt_len=171 2026-03-09 14:21:09,948 [ERROR] API error (attempt 1/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:21:09,948 [INFO] Retry 1/3 in 2s... 2026-03-09 14:21:11,948 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:21:11,956 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:21:34,068 [DEBUG] Ollama health ping OK (1.7ms) 2026-03-09 14:21:41,958 [INFO] Ollama recovered after restart 2026-03-09 14:21:41,958 [DEBUG] API request start: model=frankenstallm-3b prompt_len=171 2026-03-09 14:21:52,341 [ERROR] API error (attempt 2/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:21:52,341 [INFO] Retry 2/3 in 4s... 2026-03-09 14:21:56,341 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:21:56,349 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:22:09,073 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:22:26,351 [INFO] Ollama recovered after restart 2026-03-09 14:22:26,351 [DEBUG] API request start: model=frankenstallm-3b prompt_len=171 2026-03-09 14:22:36,939 [ERROR] API error (attempt 3/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:22:36,940 [DEBUG] Checkpoint saved: 1 completed pairs 2026-03-09 14:22:36,940 [ERROR] [nlu_01] ERROR: Remote end closed connection without response 2026-03-09 14:22:36,940 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:22:36,948 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:22:39,074 [ERROR] Ollama health ping FAILED: 2026-03-09 14:23:06,950 [INFO] Ollama recovered after restart 2026-03-09 14:23:06,950 [DEBUG] API request start: model=frankenstallm-3b prompt_len=108 2026-03-09 14:23:09,076 [DEBUG] Ollama health ping OK (1.5ms) 2026-03-09 14:23:17,304 [ERROR] API error (attempt 1/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:23:17,305 [INFO] Retry 1/3 in 2s... 2026-03-09 14:23:19,305 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:23:19,313 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:23:39,078 [DEBUG] Ollama health ping OK (1.9ms) 2026-03-09 14:23:49,315 [INFO] Ollama recovered after restart 2026-03-09 14:23:49,315 [DEBUG] API request start: model=frankenstallm-3b prompt_len=108 2026-03-09 14:23:59,887 [ERROR] API error (attempt 2/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:23:59,887 [INFO] Retry 2/3 in 4s... 2026-03-09 14:24:03,888 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:24:03,895 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:24:14,083 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:24:33,897 [INFO] Ollama recovered after restart 2026-03-09 14:24:33,897 [DEBUG] API request start: model=frankenstallm-3b prompt_len=108 2026-03-09 14:24:44,085 [DEBUG] Ollama health ping OK (1.5ms) 2026-03-09 14:24:44,845 [ERROR] API error (attempt 3/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:24:44,845 [DEBUG] Checkpoint saved: 2 completed pairs 2026-03-09 14:24:44,845 [ERROR] [nlu_02] ERROR: Remote end closed connection without response 2026-03-09 14:24:44,849 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:24:44,857 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:25:14,087 [DEBUG] Ollama health ping OK (1.6ms) 2026-03-09 14:25:14,858 [INFO] Ollama recovered after restart 2026-03-09 14:25:14,859 [DEBUG] API request start: model=frankenstallm-3b prompt_len=77 2026-03-09 14:25:26,320 [ERROR] API error (attempt 1/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:25:26,320 [INFO] Retry 1/3 in 2s... 2026-03-09 14:25:28,321 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:25:28,328 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:25:49,092 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:25:58,330 [INFO] Ollama recovered after restart 2026-03-09 14:25:58,330 [DEBUG] API request start: model=frankenstallm-3b prompt_len=77 2026-03-09 14:26:09,251 [ERROR] API error (attempt 2/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:26:09,252 [INFO] Retry 2/3 in 4s... 2026-03-09 14:26:13,252 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:26:13,260 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:26:24,098 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:26:43,263 [INFO] Ollama recovered after restart 2026-03-09 14:26:43,263 [DEBUG] API request start: model=frankenstallm-3b prompt_len=77 2026-03-09 14:26:54,101 [ERROR] Ollama health ping FAILED: [Errno 104] Connection reset by peer 2026-03-09 14:26:54,101 [ERROR] API error (attempt 3/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:26:54,101 [ERROR] [Benchmark CIRCUIT BREAKER] model=frankenstallm-3b opened after 3 consecutive failures 2026-03-09 14:26:59,876 [ERROR] HTTP 404 from Telegram: {"ok":false,"error_code":404,"description":"Not Found"} 2026-03-09 14:26:59,877 [DEBUG] Checkpoint saved: 3 completed pairs 2026-03-09 14:26:59,877 [ERROR] [nlu_03] ERROR: Remote end closed connection without response 2026-03-09 14:26:59,877 [WARNING] Circuit breaker OPEN for frankenstallm-3b — skipping remaining 5 tests 2026-03-09 14:26:59,877 [INFO] Checkpoint file deleted (clean completion) 2026-03-09 14:26:59,877 [INFO] ============================================================ 2026-03-09 14:26:59,877 [INFO] SUMMARY 2026-03-09 14:26:59,877 [INFO] ============================================================ 2026-03-09 14:26:59,877 [INFO] frankenstallm-3b auto_avg= 0.0 avg_tps= 0.0 avg_ttft= 0.0ms 2026-03-09 14:26:59,877 [INFO] Results: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/results/ollama_benchmark_results.json 2026-03-09 14:26:59,878 [INFO] Summary: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/results/ollama_benchmark_summary.md 2026-03-09 14:27:05,552 [ERROR] HTTP 404 from Telegram: {"ok":false,"error_code":404,"description":"Not Found"} 2026-03-09 14:28:08,822 [INFO] Ollama monitor thread started 2026-03-09 14:28:08,869 [DEBUG] Ollama health ping OK (45.8ms) 2026-03-09 14:28:14,648 [ERROR] HTTP 404 from Telegram: {"ok":false,"error_code":404,"description":"Not Found"} 2026-03-09 14:28:14,649 [INFO] FRANKENSTALLM Ollama Benchmark 2026-03-09 14:28:14,649 [INFO] ============================================================ 2026-03-09 14:28:14,649 [INFO] Models: frankenstallm-3b 2026-03-09 14:28:14,649 [INFO] Tests: 5 2026-03-09 14:28:14,649 [INFO] Time: 2026-03-09 14:28:08 2026-03-09 14:28:14,649 [INFO] ============================================================ 2026-03-09 14:28:14,649 [INFO] ------------------------------------------------------------ 2026-03-09 14:28:14,649 [INFO] Model: frankenstallm-3b 2026-03-09 14:28:14,649 [INFO] ------------------------------------------------------------ 2026-03-09 14:28:14,650 [DEBUG] API request start: model=frankenstallm-3b prompt_len=171 2026-03-09 14:28:14,848 [ERROR] API error (attempt 1/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:28:14,848 [INFO] Retry 1/3 in 2s... 2026-03-09 14:28:16,849 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:28:16,856 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:28:38,871 [DEBUG] Ollama health ping OK (2.2ms) 2026-03-09 14:28:46,858 [INFO] Ollama recovered after restart 2026-03-09 14:28:46,858 [DEBUG] API request start: model=frankenstallm-3b prompt_len=171 2026-03-09 14:28:57,888 [ERROR] API error (attempt 2/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:28:57,888 [INFO] Retry 2/3 in 4s... 2026-03-09 14:29:01,889 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:29:01,897 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:29:13,877 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:29:31,898 [INFO] Ollama recovered after restart 2026-03-09 14:29:31,899 [DEBUG] API request start: model=frankenstallm-3b prompt_len=171 2026-03-09 14:29:43,368 [ERROR] API error (attempt 3/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:29:43,369 [DEBUG] Checkpoint saved: 1 completed pairs 2026-03-09 14:29:43,369 [ERROR] [nlu_01] ERROR: Remote end closed connection without response 2026-03-09 14:29:43,369 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:29:43,377 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:29:43,878 [ERROR] Ollama health ping FAILED: 2026-03-09 14:30:13,378 [INFO] Ollama recovered after restart 2026-03-09 14:30:13,378 [DEBUG] API request start: model=frankenstallm-3b prompt_len=108 2026-03-09 14:30:13,880 [DEBUG] Ollama health ping OK (1.6ms) 2026-03-09 14:30:24,272 [ERROR] API error (attempt 1/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:30:24,272 [INFO] Retry 1/3 in 2s... 2026-03-09 14:30:26,272 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:30:26,280 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:30:46,926 [DEBUG] Ollama health ping OK (3045.7ms) 2026-03-09 14:30:56,282 [INFO] Ollama recovered after restart 2026-03-09 14:30:56,282 [DEBUG] API request start: model=frankenstallm-3b prompt_len=108 2026-03-09 14:31:07,268 [ERROR] API error (attempt 2/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:31:07,268 [INFO] Retry 2/3 in 4s... 2026-03-09 14:31:11,269 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:31:11,276 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:31:21,932 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:31:41,278 [INFO] Ollama recovered after restart 2026-03-09 14:31:41,278 [DEBUG] API request start: model=frankenstallm-3b prompt_len=108 2026-03-09 14:31:51,475 [ERROR] API error (attempt 3/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:31:51,476 [DEBUG] Checkpoint saved: 2 completed pairs 2026-03-09 14:31:51,476 [ERROR] [nlu_02] ERROR: Remote end closed connection without response 2026-03-09 14:31:51,480 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:31:51,488 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:31:51,937 [ERROR] Ollama health ping FAILED: 2026-03-09 14:32:21,489 [INFO] Ollama recovered after restart 2026-03-09 14:32:21,490 [DEBUG] API request start: model=frankenstallm-3b prompt_len=77 2026-03-09 14:32:21,939 [DEBUG] Ollama health ping OK (1.6ms) 2026-03-09 14:32:32,556 [ERROR] API error (attempt 1/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:32:32,557 [INFO] Retry 1/3 in 2s... 2026-03-09 14:32:34,557 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:32:34,565 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:32:53,969 [DEBUG] Ollama health ping OK (2029.6ms) 2026-03-09 14:33:04,567 [INFO] Ollama recovered after restart 2026-03-09 14:33:04,567 [DEBUG] API request start: model=frankenstallm-3b prompt_len=77 2026-03-09 14:33:15,548 [ERROR] API error (attempt 2/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:33:15,548 [INFO] Retry 2/3 in 4s... 2026-03-09 14:33:19,549 [WARNING] Health check failed — attempting Ollama restart via systemctl 2026-03-09 14:33:19,557 [INFO] Waiting 30s after restart attempt... 2026-03-09 14:33:28,974 [ERROR] Ollama health ping FAILED: timed out 2026-03-09 14:33:49,558 [INFO] Ollama recovered after restart 2026-03-09 14:33:49,558 [DEBUG] API request start: model=frankenstallm-3b prompt_len=77 2026-03-09 14:33:58,976 [DEBUG] Ollama health ping OK (1.3ms) 2026-03-09 14:34:00,253 [ERROR] API error (attempt 3/3) model=frankenstallm-3b: Remote end closed connection without response Traceback (most recent call last): File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 553, in query_ollama return _ollama_request(model, prompt, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/ollama_benchmark.py", line 510, in _ollama_request with urllib.request.urlopen(req, timeout=60) as resp: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen return opener.open(url, data, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 515, in open response = self._open(req, data) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 532, in _open result = self._call_chain(self.handle_open, protocol, protocol + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain result = func(*args) ^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1373, in http_open return self.do_open(http.client.HTTPConnection, req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/urllib/request.py", line 1348, in do_open r = h.getresponse() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response 2026-03-09 14:34:00,253 [ERROR] [Benchmark CIRCUIT BREAKER] model=frankenstallm-3b opened after 3 consecutive failures 2026-03-09 14:34:06,039 [ERROR] HTTP 404 from Telegram: {"ok":false,"error_code":404,"description":"Not Found"} 2026-03-09 14:34:06,039 [DEBUG] Checkpoint saved: 3 completed pairs 2026-03-09 14:34:06,039 [ERROR] [nlu_03] ERROR: Remote end closed connection without response 2026-03-09 14:34:06,039 [WARNING] Circuit breaker OPEN for frankenstallm-3b — skipping remaining 5 tests 2026-03-09 14:34:06,040 [INFO] Checkpoint file deleted (clean completion) 2026-03-09 14:34:06,040 [INFO] ============================================================ 2026-03-09 14:34:06,040 [INFO] SUMMARY 2026-03-09 14:34:06,040 [INFO] ============================================================ 2026-03-09 14:34:06,040 [INFO] frankenstallm-3b auto_avg= 0.0 avg_tps= 0.0 avg_ttft= 0.0ms 2026-03-09 14:34:06,040 [INFO] Results: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/results/ollama_benchmark_results.json 2026-03-09 14:34:06,040 [INFO] Summary: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/results/ollama_benchmark_summary.md 2026-03-09 14:34:11,718 [ERROR] HTTP 404 from Telegram: {"ok":false,"error_code":404,"description":"Not Found"}