Vast.io - help with custom authentication headers (Cloudflare Tunnel, API Tokens)

[Floflobel]

Problem Summary

GuideLLM v0.4.0 fails to authenticate when connecting to backends protected by Cloudflare Tunnel or other custom authentication mechanisms that require specific HTTP headers or non-standard authentication tokens.

I use the vast.io service to try to perform benchmarks to compare GPU models.

Current Behavior

1. Token in URL query string: When using authentication tokens as query parameters (e.g., https://example.com/?token=xyz), GuideLLM appends paths like /health and /v1/chat/completions, which breaks the authentication.

2. Environment variable OPENAI_API_KEY: Setting this variable doesn't pass the token to Cloudflare-protected endpoints, resulting in 401 Unauthorized errors.

3. --backend-kwargs approach: Attempting to pass custom headers via --backend-kwargs '{"extra_headers": {...}}' results in:

   TypeError: OpenAIHTTPBackend.__init__() got an unexpected keyword argument 'extra_headers'
   

Expected Behavior

According to the v0.3.0 release notes, PR #230 added "CLI options for custom headers and SSL verification" with "support for custom headers, SSL verification control, query parameters, and request body modifications" for the OpenAI Backend.[2]

However, the correct CLI syntax for passing custom authentication headers does not appear to be documented in:

• The main README
• guidellm benchmark --help output
• The official documentation

Questions

1. What is the correct CLI syntax to pass custom HTTP headers (e.g., CF-Access-Client-Secret, X-Auth-Token, or Authorization: Bearer)?
2. Is there a --request-headers or --custom-headers flag that should be used?
3. How can users authenticate with Cloudflare Tunnel or similar proxy services that require non-standard authentication?

Examples

guidellm benchmark   --target "https://lift-airlines-bracelet-oven.trycloudflare.com/?token=xxxtokenxxx"   --backend-type openai_http   --model "qwen3-vl:30b-a3b-instruct"   --processor "Qwen/Qwen3-VL-30B-A3B-Instruct"   --rate-type synchronous --max-requests 1  --max-seconds 120  --data "prompt_tokens=256,output_tokens=128" --sample-requests 1

Traceback (most recent call last):                                                                                                                                                       
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/guidellm/backends/openai.py", line 168, in validate                                                                             
    response.raise_for_status()                                                                                                                                                          
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^                                                                                                                                                          
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/httpx/_models.py", line 829, in raise_for_status                                                                                
    raise HTTPStatusError(message, request=request, response=self)                                                                                                                       
httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://lift-airlines-bracelet-oven.trycloudflare.com/?token=xx/health'                                                                                                                                                                             
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401                                                                                                 
                                                                                                                                                                                         
The above exception was the direct cause of the following exception:                                                                                                                     
                                                                                                                                                                                         
Traceback (most recent call last):                                                                                                                                                       
  File "/opt/guidellm/.venv/bin/guidellm", line 10, in <module>                                                                                                                          
    sys.exit(cli())                                                                                                                                                                      
             ~~~^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/guidellm/__main__.py", line 430, in run
    asyncio.run(
    ~~~~~~~~~~~^
        benchmark_generative_text(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
        )
        ^
    )
    ^
  File "/home/adm-fbellencontre/.local/share/uv/python/cpython-3.13.9-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/home/adm-fbellencontre/.local/share/uv/python/cpython-3.13.9-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/guidellm/benchmark/entrypoints.py", line 450, in benchmark_generative_text
    backend, model = await resolve_backend(
                     ^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/guidellm/benchmark/entrypoints.py", line 118, in resolve_backend
    await backend_instance.validate()
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/guidellm/backends/openai.py", line 170, in validate
    raise RuntimeError(
    ...<2 lines>...
    ) from exc
RuntimeError: Backend validation request failed. Could not connect to the server or validate the backend configuration.

---

guidellm benchmark   --target "https://lift-airlines-bracelet-oven.trycloudflare.com/?token=xxxtokenxxx"   --backend-type openai_http   --model "qwen3-vl:30b-a3b-instruct"   --processor "Qwen/Qwen3-VL-30B-A3B-Instruct"   --rate-type synchronous --max-requests 1  --max-seconds 120  --data "prompt_tokens=256,output_tokens=128" --sample-requests 1 --backend-kwargs '{"validate_backend":false}'

              "error": "Client error '401 Unauthorized' for url 'https://lift-airlines-bracelet-oven.trycloudflare.com/?token=xxxx/v1/chat/completions'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401",

---

export OPENAI_API_KEY="xxxx"

guidellm benchmark   --target "https://lift-airlines-bracelet-oven.trycloudflare.com"   --backend-type openai_http   --model "qwen3-vl:30b-a3b-instruct"   --processor "Qwen/Qwen3-VL-30B-A3B-Instruct"   --rate-type synchronous --max-requests 1  --max-seconds 120  --data "prompt_tokens=256,output_tokens=128" --sample-requests 1 --backend-kwargs '{"validate_backend":false}'

              "error": "Client error '401 Unauthorized' for url 'https://lift-airlines-bracelet-oven.trycloudflare.com/v1/chat/completions'\nFor more information check: https://develope
r.mozilla.org/en-US/docs/Web/HTTP/Status/401",

---

guidellm benchmark \
  --target "https://lift-airlines-bracelet-oven.trycloudflare.com" \
  --backend-type openai_http \
  --model "qwen3-vl:30b-a3b-instruct" \
  --processor "Qwen/Qwen3-VL-30B-A3B-Instruct" \
  --rate-type synchronous \
  --max-requests 1 \
  --max-seconds 120 \
  --data "prompt_tokens=256,output_tokens=128" \
  --backend-kwargs '{"extra_headers": {"X-Auth-Token": "xxxtokenxxx"}}'

Traceback (most recent call last):                                                                                                                                                       
  File "/opt/guidellm/.venv/bin/guidellm", line 10, in <module>                
  File "/opt/guidellm/.venv/bin/guidellm", line 10, in <module>
    sys.exit(cli())
             ~~~^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/guidellm/__main__.py", line 430, in run
    asyncio.run(
    ~~~~~~~~~~~^
        benchmark_generative_text(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
        )
        ^
    )
    ^
  File "/home/adm-fbellencontre/.local/share/uv/python/cpython-3.13.9-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/home/adm-fbellencontre/.local/share/uv/python/cpython-3.13.9-linux-x86_64-gnu/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/guidellm/benchmark/entrypoints.py", line 450, in benchmark_generative_text
    backend, model = await resolve_backend(
                     ^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/guidellm/benchmark/entrypoints.py", line 107, in resolve_backend
    Backend.create(backend, target=target, model=model, **(backend_kwargs or {}))
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/guidellm/.venv/lib/python3.13/site-packages/guidellm/backends/backend.py", line 79, in create
    return backend(**kwargs)
TypeError: OpenAIHTTPBackend.__init__() got an unexpected keyword argument 'extra_headers'

---

Environment

• GuideLLM version: 0.4.0
• Python version: 3.13.9
• Backend: OpenAI-compatible server behind Cloudflare Tunnel
• Target: vast.ai instance with Cloudflare tunnel authentication