2026-08-10:09:45:52 WARNING  [config.evaluate_config:287] --limit SHOULD ONLY BE USED FOR TESTING. REAL METRICS SHOULD NOT BE COMPUTED USING LIMIT.
2026-08-10:09:45:59 INFO     [_cli.run:388] Selected Tasks: ['arc_challenge', 'winogrande', 'piqa', 'boolq', 'openbookqa', 'truthfulqa_mc1', 'lambada_openai', 'copa', 'rte', 'wsc', 'mmlu', 'sciq']
2026-08-10:09:45:59 INFO     [evaluator:214] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-08-10:09:45:59 INFO     [evaluator:239] Initializing gguf model, with arguments: {'pretrained': '/tmp/miniart-2.0-q4_k_m.gguf', 'n_ctx': 2048, 'n_threads': 2}
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.15/x64/bin/lm_eval", line 6, in <module>
    sys.exit(cli_evaluate())
             ^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/lm_eval/__main__.py", line 10, in cli_evaluate
    parser.execute(args)
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/lm_eval/_cli/harness.py", line 60, in execute
    args.func(args)
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/lm_eval/_cli/run.py", line 391, in _execute
    results = simple_evaluate(
              ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/lm_eval/utils.py", line 575, in _wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/lm_eval/evaluator.py", line 242, in simple_evaluate
    lm = lm_eval.api.registry.get_model(model).create_from_arg_obj(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/lm_eval/api/model.py", line 169, in create_from_arg_obj
    return cls(**arg_dict, **additional_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/lm_eval/models/gguf.py", line 41, in __init__
    assert self.base_url, "must pass `base_url` to use GGUF LM!"
           ^^^^^^^^^^^^^
AssertionError: must pass `base_url` to use GGUF LM!