Skip to content

Conversation

@codeflash-ai
Copy link

@codeflash-ai codeflash-ai bot commented Nov 13, 2025

📄 295% (2.95x) speedup for _model_matches_any_wildcard_pattern_in_list in litellm/proxy/auth/auth_checks.py

⏱️ Runtime : 410 milliseconds 104 milliseconds (best of 55 runs)

📝 Explanation and details

The key optimization is caching compiled regex patterns with an LRU cache to eliminate repeated compilation overhead. The original code was calling re.match() with a new pattern string each time, forcing Python to compile the regex from scratch on every call - this was the bottleneck consuming 95% of execution time.

Key changes:

  • Added @lru_cache(maxsize=8192) decorated _compile_pattern() function that compiles and caches regex patterns
  • Replaced re.match(pattern, model) with _compile_pattern(pattern).match(model) to use cached compiled patterns
  • Simplified the nested any() generator expressions in _model_matches_any_wildcard_pattern_in_list to explicit for-loops with early returns for better readability and slight performance gain

Why this speeds up the code:

  • Regex compilation is expensive - the original line profiler shows re.match() taking 95.1% of total time
  • With caching, identical patterns (like "bedrock/*", "openai/*") are compiled once and reused across calls
  • The cache size of 8192 handles most real-world scenarios where the same wildcard patterns are checked repeatedly

Impact on workloads:
The function is called from _check_model_access_helper, which appears to be in the authentication hot path for model access validation. Since auth checks happen on every API request, this optimization provides significant performance benefits:

  • 295% speedup overall with most test cases showing 100-5000% improvements
  • Particularly effective for repeated pattern matching scenarios (large model lists, repeated auth checks)
  • Best performance gains when the same wildcard patterns are evaluated multiple times, which is typical in production auth workflows

The optimization maintains identical behavior and return values while dramatically reducing computational overhead for this critical auth validation function.

Correctness verification report:

Test Status
⚙️ Existing Unit Tests 🔘 None Found
🌀 Generated Regression Tests 612 Passed
⏪ Replay Tests 🔘 None Found
🔎 Concolic Coverage Tests 🔘 None Found
📊 Tests Coverage 100.0%
🌀 Generated Regression Tests and Runtime
import re

# imports
import pytest
from litellm.proxy.auth.auth_checks import \
    _model_matches_any_wildcard_pattern_in_list

# --- Unit Tests ---

# 1. Basic Test Cases

def test_basic_exact_wildcard_match():
    # Model matches simple wildcard
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/*"]) # 45.2μs -> 2.51μs (1704% faster)
    # Model does not match wildcard
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrockzzzz/us.amazon.nova-micro-v1:0", ["bedrock/*"]) # 172μs -> 160μs (7.52% faster)
    # Model matches pattern with wildcard in middle
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/us.*"]) # 46.2μs -> 1.86μs (2390% faster)
    # Model matches any model with global wildcard
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("anything", ["*"]) # 25.8μs -> 1.10μs (2248% faster)
    # Model matches multiple patterns, first one matches
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", ["bedrock/*", "openai/*"]) # 2.97μs -> 914ns (225% faster)
    # Model matches last pattern in list
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/bar", ["bedrock/*", "openai/*"]) # 32.5μs -> 1.34μs (2323% faster)
    # Model does not match any pattern
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("foo/bar", ["bedrock/*", "openai/*"]) # 236μs -> 222μs (6.13% faster)
    # Model matches with provider logic
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("gpt-4o", ["openai/*"]) # 13.1μs -> 10.2μs (27.5% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("claude-3-5-sonnet-20240620", ["anthropic/*"]) # 51.4μs -> 8.91μs (477% faster)

def test_basic_multiple_patterns():
    # Model matches one of several patterns
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["openai/*", "bedrock/*"]) # 6.79μs -> 3.12μs (118% faster)
    # Model matches none
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("foo/bar", ["openai/*", "bedrock/*"]) # 251μs -> 241μs (4.21% faster)
    # Model matches with wildcard at the end
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("custommodel-xyz", ["custom/*", "openai/*"]) # 267μs -> 206μs (29.4% faster)

# 2. Edge Test Cases

def test_empty_allowed_list():
    # No allowed patterns, must not match
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", []) # 1.38μs -> 512ns (170% faster)
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("", []) # 615ns -> 228ns (170% faster)

def test_empty_model_string():
    # Empty model string, only matches global wildcard
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("", ["*"]) # 5.91μs -> 2.75μs (115% faster)
    # Empty model should not match specific wildcards
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("", ["bedrock/*", "openai/*"]) # 252μs -> 248μs (1.59% faster)

def test_empty_pattern_string():
    # Empty pattern string is not a wildcard
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", [""]) # 1.77μs -> 954ns (86.0% faster)
    # Empty pattern in list with a valid wildcard
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["", "bedrock/*"]) # 4.78μs -> 2.33μs (105% faster)


def test_pattern_with_multiple_wildcards():
    # Multiple wildcards in pattern
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo/bar", ["bedrock/*/bar"]) # 65.3μs -> 3.68μs (1676% faster)
    # Wildcard at start
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("foo/bar", ["*/bar"]) # 34.9μs -> 1.26μs (2657% faster)
    # Wildcard at end
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("foo/bar", ["foo/*"]) # 29.0μs -> 1.11μs (2507% faster)
    # Wildcard only pattern
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("anything", ["*"]) # 3.34μs -> 922ns (262% faster)

def test_provider_logic_edge_cases():
    # Model without provider, should not match provider wildcard
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("randommodel", ["openai/*"]) # 177μs -> 173μs (2.55% faster)
    # Model with provider embedded
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-4o", ["openai/*"]) # 4.00μs -> 1.77μs (126% faster)
    # Model with provider in name but not in pattern
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("openai/gpt-4o", ["bedrock/*"]) # 14.2μs -> 12.6μs (13.0% faster)
    # Custom provider logic
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("custommodel-abc", ["custom/*"]) # 115μs -> 112μs (2.01% faster)
    # Model with unknown provider
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("unknownmodel", ["openai/*", "bedrock/*"]) # 211μs -> 208μs (1.40% faster)

def test_pattern_with_special_characters():
    # Pattern with regex special chars
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/us.amazon.nova-micro-v1:*"]) # 75.7μs -> 2.91μs (2499% faster)
    # Pattern with dot
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/us.amazon.*"]) # 43.4μs -> 1.28μs (3289% faster)
    # Pattern with dash
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/*-micro-*"]) # 42.7μs -> 1.03μs (4048% faster)
    # Pattern with colon
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/*:0"]) # 33.4μs -> 1.13μs (2849% faster)

def test_pattern_with_no_wildcard():
    # Pattern with no wildcard should not match unless model is identical
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/us.amazon.nova-micro-v1"]) # 1.77μs -> 898ns (96.9% faster)
    # Exact match
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/us.amazon.nova-micro-v1:1"]) # 905ns -> 521ns (73.7% faster)
    # Exact match with correct string
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/us.amazon.nova-micro-v1:0"]) # 770ns -> 329ns (134% faster)
    # Only matches if wildcard present
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/us.amazon.nova-micro-v1:*"]) # 4.58μs -> 2.13μs (115% faster)

def test_case_sensitivity():
    # Should be case sensitive
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("Bedrock/foo", ["bedrock/*"]) # 168μs -> 151μs (11.6% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", ["bedrock/*"]) # 3.99μs -> 1.71μs (133% faster)

def test_wildcard_pattern_is_only_star():
    # Pattern is only "*" matches everything
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", ["*"]) # 5.08μs -> 2.24μs (127% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/bar", ["*"]) # 1.94μs -> 875ns (122% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("", ["*"]) # 1.45μs -> 671ns (117% faster)

def test_wildcard_pattern_with_empty_string_in_list():
    # Empty string in allowed list should not match unless "*" is present
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrock/foo", [""]) # 1.76μs -> 848ns (107% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", ["", "*"]) # 4.33μs -> 1.90μs (128% faster)

def test_model_with_slash_and_pattern_without():
    # Model with slash, pattern without slash and wildcard
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrock/foo", ["bedrock*"]) # 57.5μs -> 2.39μs (2305% faster)
    # Model with slash, pattern with slash and wildcard
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", ["bedrock/*"]) # 3.68μs -> 1.25μs (194% faster)

def test_model_with_provider_and_pattern_with_provider_and_wildcard():
    # Model is "openai/gpt-4o", pattern is "openai/*"
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-4o", ["openai/*"]) # 4.76μs -> 2.26μs (111% faster)
    # Model is "anthropic/claude-3-5-sonnet-20240620", pattern is "anthropic/*"
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("anthropic/claude-3-5-sonnet-20240620", ["anthropic/*"]) # 2.70μs -> 1.28μs (110% faster)

def test_model_with_provider_and_pattern_with_wrong_provider():
    # Model is "openai/gpt-4o", pattern is "bedrock/*"
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("openai/gpt-4o", ["bedrock/*"]) # 20.3μs -> 16.4μs (23.6% faster)

# 3. Large Scale Test Cases

def test_large_allowed_model_list_all_match():
    # Large list, all patterns match "*"
    allowed = ["*"] * 1000
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", allowed) # 4.73μs -> 2.21μs (114% faster)

def test_large_allowed_model_list_none_match():
    # Large list, none match
    allowed = [f"provider{i}/*" for i in range(1000)]
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrock/foo", allowed) # 66.2ms -> 4.53ms (1362% faster)

def test_large_allowed_model_list_one_match_at_start():
    # Large list, first pattern matches
    allowed = ["bedrock/*"] + [f"provider{i}/*" for i in range(999)]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", allowed) # 45.9μs -> 2.62μs (1651% faster)

def test_large_allowed_model_list_one_match_at_end():
    # Large list, last pattern matches
    allowed = [f"provider{i}/*" for i in range(999)] + ["bedrock/*"]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", allowed) # 28.7ms -> 271μs (10470% faster)

def test_large_allowed_model_list_one_match_in_middle():
    # Large list, middle pattern matches
    allowed = [f"provider{i}/*" for i in range(500)] + ["bedrock/*"] + [f"provider{i}/*" for i in range(499)]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", allowed) # 14.4ms -> 135μs (10489% faster)

def test_large_allowed_model_list_multiple_matches():
    # Large list, multiple patterns match
    allowed = ["bedrock/*", "openai/*"] + [f"provider{i}/*" for i in range(998)]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/foo", allowed) # 5.82μs -> 2.29μs (154% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/bar", allowed) # 46.6μs -> 1.69μs (2665% faster)

def test_large_varied_models_and_patterns():
    # Large number of models and patterns, only some match
    allowed = [f"provider{i}/*" for i in range(500)] + ["openai/*"] + [f"provider{i}/*" for i in range(499)]
    models = [f"provider{i}/model" for i in range(500)] + ["openai/gpt-4o"] + [f"provider{i}/model" for i in range(499)]
    # All provider models should match their pattern
    for i in range(500):
        codeflash_output = _model_matches_any_wildcard_pattern_in_list(models[i], allowed) # 83.6ms -> 33.9ms (147% faster)
    # openai/gpt-4o should match openai/*
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-4o", allowed) # 325μs -> 126μs (157% faster)
    # Models not in allowed should not match
    codeflash_output = not _model_matches_any_wildcard_pattern_in_list("bedrock/foo", allowed) # 5.44ms -> 4.49ms (21.0% faster)

def test_large_performance():
    # Check performance with large data (not a strict timing test, but should not hang)
    allowed = [f"provider{i}/*" for i in range(1000)]
    model = "provider999/model"
    codeflash_output = _model_matches_any_wildcard_pattern_in_list(model, allowed) # 14.8ms -> 294μs (4916% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
import re

# imports
import pytest
from litellm.proxy.auth.auth_checks import \
    _model_matches_any_wildcard_pattern_in_list

# --- Unit Tests --- #

# 1. Basic Test Cases
@pytest.mark.parametrize(
    "model,allowed_model_list,expected",
    [
        # Exact wildcard match at provider level
        ("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/*"], True),
        # Wildcard at provider/model level
        ("bedrock/us.amazon.nova-micro-v1:0", ["bedrock/us.*"], True),
        # Wildcard does not match
        ("bedrockzzzz/us.amazon.nova-micro-v1:0", ["bedrock/*"], False),
        # Global wildcard matches anything
        ("openai/gpt-3.5-turbo", ["*"], True),
        # Multiple allowed patterns, one matches
        ("openai/gpt-3.5-turbo", ["bedrock/*", "openai/*"], True),
        # Multiple allowed patterns, none match
        ("openai/gpt-3.5-turbo", ["bedrock/*", "vertex_ai/*"], False),
        # Custom LLM provider mapping
        ("gpt-4o", ["openai/*"], True),
        ("claude-3-5-sonnet-20240620", ["anthropic/*"], True),
        ("command-r-plus", ["cohere/*"], True),
        # Custom provider, matches
        ("customprovider/custom-model", ["customprovider/*"], True),
        # Custom provider, does not match
        ("customprovider/custom-model", ["openai/*"], False),
        # Model with provider prefix, matches
        ("openrouter/foo-model", ["openrouter/*"], True),
        # Model with provider prefix, does not match
        ("openrouter/foo-model", ["openai/*"], False),
    ]
)
def test_basic_model_matches_any_wildcard_pattern_in_list(model, allowed_model_list, expected):
    # Basic scenarios for matching and non-matching patterns
    codeflash_output = _model_matches_any_wildcard_pattern_in_list(model, allowed_model_list) # 888μs -> 445μs (99.7% faster)

# 2. Edge Test Cases

def test_empty_allowed_model_list():
    # Edge: Empty allowed model list should never match
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo", []) # 1.36μs -> 443ns (208% faster)

def test_empty_model_string():
    # Edge: Empty model string should not match any pattern except a global wildcard
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("", ["*"]) # 5.73μs -> 2.73μs (110% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("", ["openai/*"]) # 164μs -> 162μs (1.36% faster)

def test_empty_allowed_model_pattern():
    # Edge: Empty pattern in allowed_model_list should not match anything
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo", [""]) # 1.89μs -> 967ns (95.2% faster)

def test_model_with_special_characters():
    # Edge: Model with special characters, pattern should match accordingly
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo@2023", ["openai/gpt-3.5-turbo*"]) # 67.6μs -> 2.80μs (2316% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo@2023", ["openai/gpt-3.5-turbo"]) # 1.63μs -> 656ns (149% faster)

def test_pattern_with_multiple_wildcards():
    # Edge: Pattern with multiple wildcards
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo@2023", ["openai/*@*"]) # 58.7μs -> 2.85μs (1962% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo@2023", ["openai/*-turbo@*"]) # 45.4μs -> 1.37μs (3218% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo@2023", ["openai/gpt-*"]) # 35.8μs -> 1.05μs (3301% faster)

def test_pattern_with_no_wildcard():
    # Edge: Pattern with no wildcard should not match unless model is exactly equal
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo", ["openai/gpt-3.5-turbo"]) # 1.89μs -> 941ns (101% faster)

def test_model_with_no_provider_prefix():
    # Edge: Model with no provider prefix, pattern with provider wildcard
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("gpt-4o", ["openai/*"]) # 23.4μs -> 16.8μs (39.5% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("gpt-4o", ["bedrock/*"]) # 10.3μs -> 7.48μs (37.4% faster)

def test_pattern_with_only_wildcard():
    # Edge: Pattern is just a wildcard
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("anything", ["*"]) # 5.11μs -> 2.28μs (124% faster)

def test_model_and_pattern_with_slashes():
    # Edge: Model and pattern with multiple slashes
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/foo/bar/baz", ["openai/foo/*"]) # 54.9μs -> 2.38μs (2202% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/foo/bar/baz", ["openai/*/baz"]) # 39.1μs -> 1.23μs (3093% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/foo/bar/baz", ["openai/*/*"]) # 39.1μs -> 845ns (4524% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/foo/bar/baz", ["openai/bar/*"]) # 50.3μs -> 13.7μs (267% faster)

def test_model_with_provider_not_in_stub():
    # Edge: Model with unknown provider, should not match any pattern except global wildcard
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("unknownprovider/model", ["openai/*"]) # 172μs -> 163μs (5.15% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("unknownprovider/model", ["*"]) # 4.22μs -> 1.69μs (150% faster)

def test_pattern_with_regex_metacharacters():
    # Edge: Pattern with regex metacharacters, must be treated as literal except for *
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo(2023)", ["openai/gpt-3.5-turbo(*)"]) # 74.2μs -> 2.62μs (2737% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo(2023)", ["openai/gpt-3.5-turbo*"]) # 4.09μs -> 1.44μs (185% faster)

def test_model_with_provider_and_colon():
    # Edge: Model with colon, provider pattern
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/anthropic.claude-3-5-sonnet-20240620", ["bedrock/*"]) # 5.13μs -> 2.31μs (122% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bedrock/anthropic.claude-3-5-sonnet-20240620", ["openai/*"]) # 17.4μs -> 14.4μs (21.1% faster)

def test_model_with_unusual_provider():
    # Edge: Model with provider not in mapping, pattern matches provider
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("bytez/bytez-model", ["bytez/*"]) # 50.9μs -> 2.21μs (2197% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("heroku/heroku-model", ["heroku/*"]) # 34.6μs -> 1.13μs (2961% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("cometapi/comet-model", ["cometapi/*"]) # 31.9μs -> 989ns (3122% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("oci/oci-model", ["oci/*"]) # 28.2μs -> 852ns (3212% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("compactifai/compact-model", ["compactifai/*"]) # 32.4μs -> 939ns (3355% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("ovhcloud/ovh-model", ["ovhcloud/*"]) # 30.5μs -> 732ns (4061% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("clarifai/clarifai-model", ["clarifai/*"]) # 29.8μs -> 699ns (4158% faster)

# 3. Large Scale Test Cases

def test_large_allowed_model_list_one_match():
    # Large allowed model list, only one matches
    allowed_model_list = [f"provider{i}/*" for i in range(999)]
    allowed_model_list.append("openai/*")
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo", allowed_model_list) # 28.8ms -> 275μs (10336% faster)

def test_large_allowed_model_list_none_match():
    # Large allowed model list, none match
    allowed_model_list = [f"provider{i}/*" for i in range(1000)]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo", allowed_model_list) # 65.9ms -> 4.24ms (1454% faster)

def test_large_allowed_model_list_all_wildcards():
    # Large allowed model list, all are global wildcards
    allowed_model_list = ["*"] * 1000
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo", allowed_model_list) # 37.7μs -> 2.40μs (1470% faster)

def test_large_allowed_model_list_first_match():
    # Large allowed model list, first pattern matches
    allowed_model_list = ["openai/*"] + [f"provider{i}/*" for i in range(999)]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo", allowed_model_list) # 49.0μs -> 2.29μs (2039% faster)

def test_large_allowed_model_list_last_match():
    # Large allowed model list, last pattern matches
    allowed_model_list = [f"provider{i}/*" for i in range(999)] + ["openai/*"]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo", allowed_model_list) # 28.8ms -> 276μs (10287% faster)

def test_large_model_name():
    # Large model name, pattern matches
    model = "openai/" + "gpt-" + "x" * 950
    allowed_model_list = ["openai/gpt-*"]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list(model, allowed_model_list) # 49.4μs -> 3.04μs (1524% faster)

def test_large_model_name_no_match():
    # Large model name, pattern does not match
    model = "openai/" + "gpt-" + "x" * 950
    allowed_model_list = ["bedrock/*"]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list(model, allowed_model_list) # 64.9μs -> 17.6μs (268% faster)

def test_large_allowed_model_list_mixed_wildcards():
    # Large allowed model list, mixture of wildcards and non-wildcards
    allowed_model_list = [f"provider{i}/model{i}" for i in range(500)] + [f"provider{i}/*" for i in range(500)]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("provider499/model499", allowed_model_list) # 14.3ms -> 172μs (8228% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("provider999/model999", allowed_model_list) # 50.4ms -> 50.0ms (0.871% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo", allowed_model_list) # 2.78ms -> 2.24ms (24.4% faster)

def test_large_allowed_model_list_with_special_chars():
    # Large allowed model list with special characters
    allowed_model_list = [f"openai/gpt-3.5-turbo@{i}*" for i in range(1000)]
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo@999", allowed_model_list) # 407μs -> 7.64μs (5231% faster)
    codeflash_output = _model_matches_any_wildcard_pattern_in_list("openai/gpt-3.5-turbo@1001", allowed_model_list) # 4.65μs -> 1.38μs (237% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-_model_matches_any_wildcard_pattern_in_list-mhx2el25 and push.

Codeflash Static Badge

The key optimization is **caching compiled regex patterns** with an LRU cache to eliminate repeated compilation overhead. The original code was calling `re.match()` with a new pattern string each time, forcing Python to compile the regex from scratch on every call - this was the bottleneck consuming 95% of execution time.

**Key changes:**
- Added `@lru_cache(maxsize=8192)` decorated `_compile_pattern()` function that compiles and caches regex patterns
- Replaced `re.match(pattern, model)` with `_compile_pattern(pattern).match(model)` to use cached compiled patterns
- Simplified the nested `any()` generator expressions in `_model_matches_any_wildcard_pattern_in_list` to explicit for-loops with early returns for better readability and slight performance gain

**Why this speeds up the code:**
- Regex compilation is expensive - the original line profiler shows `re.match()` taking 95.1% of total time
- With caching, identical patterns (like `"bedrock/*"`, `"openai/*"`) are compiled once and reused across calls
- The cache size of 8192 handles most real-world scenarios where the same wildcard patterns are checked repeatedly

**Impact on workloads:**
The function is called from `_check_model_access_helper`, which appears to be in the authentication hot path for model access validation. Since auth checks happen on every API request, this optimization provides significant performance benefits:
- **295% speedup overall** with most test cases showing 100-5000% improvements
- Particularly effective for repeated pattern matching scenarios (large model lists, repeated auth checks)
- Best performance gains when the same wildcard patterns are evaluated multiple times, which is typical in production auth workflows

The optimization maintains identical behavior and return values while dramatically reducing computational overhead for this critical auth validation function.
@codeflash-ai codeflash-ai bot requested a review from mashraf-222 November 13, 2025 06:45
@codeflash-ai codeflash-ai bot added ⚡️ codeflash Optimization PR opened by Codeflash AI 🎯 Quality: High Optimization Quality according to Codeflash labels Nov 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⚡️ codeflash Optimization PR opened by Codeflash AI 🎯 Quality: High Optimization Quality according to Codeflash

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant