as we use our own openAI compatible server we need to setup the server base URL. like this one https://github.com/lm-sys/FastChat/blob/main/docs/openai_api.md