⚡️ Speed up function _model_matches_any_wildcard_pattern_in_list by 295%
#448
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📄 295% (2.95x) speedup for
_model_matches_any_wildcard_pattern_in_listinlitellm/proxy/auth/auth_checks.py⏱️ Runtime :
410 milliseconds→104 milliseconds(best of55runs)📝 Explanation and details
The key optimization is caching compiled regex patterns with an LRU cache to eliminate repeated compilation overhead. The original code was calling
re.match()with a new pattern string each time, forcing Python to compile the regex from scratch on every call - this was the bottleneck consuming 95% of execution time.Key changes:
@lru_cache(maxsize=8192)decorated_compile_pattern()function that compiles and caches regex patternsre.match(pattern, model)with_compile_pattern(pattern).match(model)to use cached compiled patternsany()generator expressions in_model_matches_any_wildcard_pattern_in_listto explicit for-loops with early returns for better readability and slight performance gainWhy this speeds up the code:
re.match()taking 95.1% of total time"bedrock/*","openai/*") are compiled once and reused across callsImpact on workloads:
The function is called from
_check_model_access_helper, which appears to be in the authentication hot path for model access validation. Since auth checks happen on every API request, this optimization provides significant performance benefits:The optimization maintains identical behavior and return values while dramatically reducing computational overhead for this critical auth validation function.
✅ Correctness verification report:
🌀 Generated Regression Tests and Runtime
To edit these changes
git checkout codeflash/optimize-_model_matches_any_wildcard_pattern_in_list-mhx2el25and push.