Add support for dual-AI provider (Anthropic + OpenAI) with user selection

[Hebx]

Description

Mallory currently uses Anthropic’s SDK (ANTHROPIC_API_KEY) for AI chat.

Add support for OpenAI (OPENAI_API_KEY) and let users choose the provider per chat session (Anthropic ↔ OpenAI). Keep backward compatibility and make the system extensible for future providers.

Project specification & scope

Backend

• Identify where Anthropic is used (likely apps/server/*, chat endpoint).
• Create a provider abstraction layer (e.g. apps/server/lib/ai/):
  • Define a common interface (chat, stream if required).
• Implement providers:
  • openai.ts — implement interface using the openai npm package.
  • anthropic.ts — refactor existing Anthropic usage (@anthropic-ai/sdk) to implement the same interface.
• Add provider registry and selector: getAIClient(providerName: string).
• Update /api/chat:
  • Accept provider: "openai" | "anthropic" in request body (fallback to default env).
  • Route requests through getAIClient(provider).
• Update environment:
  • Add OPENAI_API_KEY to .env.example.
  • Keep ANTHROPIC_API_KEY for backwards compatibility.

Frontend

• Add a UI control (dropdown/toggle) to select provider per session.
• Show active provider in the UI (e.g., badge: Powered by GPT-4o / Powered by Claude).
• Send selected provider to backend in chat request payload.

Testing

• Unit tests for provider modules (with mocked API responses).
• Integration tests for /api/chat routing with providers mocked.
• E2E test verifying frontend provider selection flows to backend and returns responses.
• Error & fallback tests (optional).

Docs

• Update README.md and .env.example:
  • Document OPENAI_API_KEY and ANTHROPIC_API_KEY.
  • Show example payload: { provider: "openai" }.
  • Note model name mapping and provider-specific caveats.

Optional / Extras

• Fallback logic: if selected provider errors/outage, optionally retry with other provider (configurable).
• Instrumentation: log provider used, latency, errors.
• Add more providers later (Mistral, Gemini) using same interface.

Acceptance criteria

• Server has a provider abstraction layer.
• openai provider implemented; OPENAI_API_KEY supported.
• anthropic provider refactored to same interface.
• /api/chat accepts provider and routes requests accordingly.
• Frontend allows user to select provider and sends selection to backend.
• Tests cover both provider code paths.
• README.md and .env.example updated.
• (Optional) Fallback logic implemented and documented.

[lopeselio]

I think this should be a great feature add for testing purposes as not all developers use Anthropic Claude LLM paid features.

[edgarpavlovsky]

i'm always a bit torn on giving users more flexibility, because i don't know how many users actually know the difference between these models. i think if we're going to add provider options, we should add an option for at least one free / open source model developers can use to quickly test mallory - not sure what the best options are here

[Hebx]

Yes i had to test between Claude and GPT so i made a subbranch in my Fork that use openai api, I will experiment with adding both providers into one branch and open a pr for it, for a free tier model, I think Gemini 2.5 will be a good choice.