Running openevolve with a local llm.

[paul-emilemorgades]

Hello,

I am struggling to run openevolve with a local llm.
I am just trying to run the example function_minimization.

Here how I start my local llm using optillm (based on the makefile):

env/bin/optillm --model google/gemma-3-270m-it --port 8000 &
        OPTILLM_PID=$$! && \
        echo $$OPTILLM_PID > /tmp/optillm.pid 

Then here how I run the example:

export OPENAI_API_KEY=optillm; python3 openevolve-run.py examples/function_minimization/initial_program.py examples/function_minimization/evaluator.py --config examples/function_minimization/localConfig.yaml --iterations 50
And finally here the part of my config I have changed from example's config:

# LLM configuration
llm:
  primary_model: "gemma-3-270m-it"
  primary_model_weight: 0.8
  secondary_model: "gemma-3-270m-it"
  api_base: "http://127.0.0.1:8000/v1"
  api_key: "optillm"
  temperature: 0.7
  max_tokens: 16000
  timeout: 120
  

Still I have this kind of error:

2025-12-10 15:21:20,564 - INFO - HTTP Request: POST http://127.0.0.1:8000/v1/chat/completions "HTTP/1.1 500 INTERNAL SERVER ERROR"
2025-12-10 15:21:20,566 - WARNING - Error on attempt 2/4: Error code: 500 - {'error': "litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=gemma-3-270m-it\n Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers"}. Retrying...

Please do you have any idea, what I am doing wrong?
Thank you in advance.

[codelion]

We need to set the OPTILLM_API_KEY to enable the local inference server in OptiLLM, without that it tries to route using LiteLLM.

try

OPTILLM_API_KEY=optillm env/bin/optillm --model google/gemma-3-270m-it --port 8000 &