Add redis cache to e2e and fuzz tests #3041
Conversation
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings. WalkthroughThis PR integrates Redis caching into e2e and fuzz testing environments. It adds Redis 8.0.5-alpine3.21 services to Docker Compose configurations for both environments, updates GitHub Actions workflows with corresponding Redis services, configures environment-specific Redis credentials, removes in-memory cache configurations from Django settings, and eliminates the setup-e2e-environment action while updating Makefiles to orchestrate the cache service alongside other containers. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (5 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (1)
🧰 Additional context used🧠 Learnings (5)📓 Common learnings📚 Learning: 2025-12-26T06:57:19.911ZApplied to files:
📚 Learning: 2025-10-26T12:50:50.512ZApplied to files:
📚 Learning: 2025-12-26T06:08:58.549ZApplied to files:
📚 Learning: 2025-12-26T06:08:58.549ZApplied to files:
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
🔇 Additional comments (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
.github/workflows/run-ci-cd.yamlbackend/.env.e2e.examplebackend/settings/e2e.pydocker-compose/e2e.yaml
💤 Files with no reviewable changes (1)
- backend/settings/e2e.py
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-12-21T19:03:59.068Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 1139
File: .github/workflows/setup-backend-environment/action.yaml:16-27
Timestamp: 2025-12-21T19:03:59.068Z
Learning: Composite actions (runs: using: composite) execute as steps within the calling job's context and can access the job context, including job.services.* properties (e.g., job.services.<service_id>.id, job.services.<service_id>.ports). Service containers must be defined at the job level, but a composite action's steps can reference them via the job context.
Applied to files:
.github/workflows/run-ci-cd.yaml
🪛 dotenv-linter (4.0.0)
backend/.env.e2e.example
[warning] 16-16: [UnorderedKey] The DJANGO_REDIS_HOST key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 17-17: [UnorderedKey] The DJANGO_REDIS_PASSWORD key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run frontend unit tests
- GitHub Check: Run backend tests
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (3)
docker-compose/e2e.yaml (3)
14-15: LGTM!The cache dependency with
service_healthycondition ensures the backend waits for Redis to be fully operational before starting, which is the correct approach.
49-66: Well-configured Redis service for E2E testing.The cache service configuration demonstrates good practices:
- Memory limit (100mb) with
allkeys-lrueviction policy prevents unbounded memory growth- Health check ensures service readiness before dependent services start
- Data volume provides persistence across container restarts
- Network-only access (no exposed ports) maintains isolation
89-89: LGTM!The volume definition correctly supports Redis data persistence.
arkid15r
dismissed
coderabbitai[bot]’s stale review
The merge-base changed after approval.
arkid15r
force-pushed
the
feature/e2e-backend
branch
from
de86aab to
0159b0d
Compare
ahmedxgouda
force-pushed
the
feature/e2e-redis-cache
branch
from
0728fdc to
4521dac
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
.github/workflows/run-ci-cd.yamlbackend/.env.e2e.examplebackend/Makefilebackend/settings/e2e.pydocker-compose/e2e.yamlfrontend/Makefile
💤 Files with no reviewable changes (1)
- backend/settings/e2e.py
🚧 Files skipped from review as they are similar to previous changes (2)
- backend/Makefile
- .github/workflows/run-ci-cd.yaml
🧰 Additional context used
🧠 Learnings (5)
📓 Common learnings
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3041
File: .github/workflows/run-ci-cd.yaml:233-243
Timestamp: 2025-12-26T06:08:58.549Z
Learning: For the OWASP/Nest repository, Redis image versions should remain consistent across all environments (production, staging, local, E2E, and CI/CD E2E tests). When upgrading Redis, update all docker-compose files and CI/CD workflow configurations together to maintain environment parity.
📚 Learning: 2025-12-26T06:08:58.549Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3041
File: .github/workflows/run-ci-cd.yaml:233-243
Timestamp: 2025-12-26T06:08:58.549Z
Learning: Ensure Redis image versions stay in sync across all environments by updating every relevant YAML file together (docker-compose files and CI/CD workflow configurations). When upgrading Redis, bump the image version in all docker-compose files and in all CI workflow YAMLs in one coordinated change to maintain parity across production, staging, local, E2E, and CI tests.
Applied to files:
docker-compose/e2e.yaml
📚 Learning: 2025-10-26T12:50:50.512Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 2429
File: backend/Makefile:30-32
Timestamp: 2025-10-26T12:50:50.512Z
Learning: The `exec-backend-e2e-command` and `exec-db-e2e-command` Makefile targets in the backend/Makefile are intended for local development and debugging only, not for CI/CD execution, so the `-it` flags are appropriate.
Applied to files:
frontend/Makefile
📚 Learning: 2025-12-26T06:57:19.911Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 1139
File: docker-compose/fuzz.yaml:70-83
Timestamp: 2025-12-26T06:57:19.911Z
Learning: In the fuzz testing environment (docker-compose/fuzz.yaml), the data-loader and graphql services are orchestrated in separate phases via the Makefile's test-fuzz target, not run simultaneously. The data is loaded to completion first, then the graphql fuzzer is started separately against the populated database. The docker-compose dependencies reflect runtime health checks rather than orchestration order.
Applied to files:
frontend/Makefile
📚 Learning: 2025-12-26T06:08:58.549Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3041
File: .github/workflows/run-ci-cd.yaml:233-243
Timestamp: 2025-12-26T06:08:58.549Z
Learning: For the OWASP/Nest repository, Redis image versions should remain consistent across all environments (production, staging, local, E2E, and CI/CD E2E tests). When upgrading Redis, update all docker-compose files and CI/CD workflow configurations together to maintain environment parity.
Applied to files:
frontend/Makefilebackend/.env.e2e.example
🪛 dotenv-linter (4.0.0)
backend/.env.e2e.example
[warning] 16-16: [UnorderedKey] The DJANGO_REDIS_HOST key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 17-17: [UnorderedKey] The DJANGO_REDIS_PASSWORD key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: Run frontend unit tests
- GitHub Check: Run fuzz tests
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run backend tests
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (4)
backend/.env.e2e.example (1)
16-17: LGTM! Password mismatch resolved.The Redis configuration correctly points to the cache service and uses the password that matches the CI/CD workflow configuration. The critical password mismatch flagged in the previous review appears to be resolved.
frontend/Makefile (1)
63-63: LGTM! Cache service properly integrated.The cache service is correctly added to both docker-compose commands in the E2E test workflow. This ensures Redis is available during both the data-loading phase and the test-execution phase.
Also applies to: 65-65
docker-compose/e2e.yaml (2)
15-16: LGTM! Backend cache dependency configured correctly.The backend service properly depends on the cache service with a
service_healthycondition, ensuring Redis is ready before the backend starts.
112-112: LGTM! Cache volume properly declared.The
e2e-cache-datavolume declaration correctly supports the cache service's persistent storage requirements.
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docker-compose/fuzz.yaml (1)
2-28: Critical: Backend missing dependency on cache service.The backend service is configured to use Redis (via
DJANGO_REDIS_HOST=cacheinbackend/.env.fuzz.example), but does not declare a dependency on thecacheservice. This creates a race condition where the backend may attempt to connect to Redis before it's ready, causing startup failures.🔎 Proposed fix to add cache dependency
depends_on: db: condition: service_healthy + cache: + condition: service_healthyThis mirrors the correct pattern in
docker-compose/e2e.yaml(lines 15-16).
🧹 Nitpick comments (1)
backend/.env.fuzz.example (1)
16-17: LGTM! Redis configuration aligns with the fuzz cache service.The Redis host and password values correctly match the cache service defined in
docker-compose/fuzz.yaml.
Optional: Consider alphabetical ordering of environment variables.
The static analysis tool suggests placing these Redis variables before
DJANGO_SETTINGS_MODULEto maintain alphabetical order. This is a minor style convention.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
.github/workflows/run-ci-cd.yaml.github/workflows/setup-e2e-environment/action.yamlbackend/.env.fuzz.examplebackend/Makefilebackend/apps/api/rest/v0/__init__.pybackend/settings/fuzz.pydocker-compose/e2e.yamldocker-compose/fuzz.yaml
💤 Files with no reviewable changes (2)
- .github/workflows/setup-e2e-environment/action.yaml
- backend/settings/fuzz.py
🚧 Files skipped from review as they are similar to previous changes (2)
- backend/Makefile
- .github/workflows/run-ci-cd.yaml
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3041
File: .github/workflows/run-ci-cd.yaml:233-243
Timestamp: 2025-12-26T06:08:58.549Z
Learning: For the OWASP/Nest repository, Redis image versions should remain consistent across all environments (production, staging, local, E2E, and CI/CD E2E tests). When upgrading Redis, update all docker-compose files and CI/CD workflow configurations together to maintain environment parity.
📚 Learning: 2025-12-26T06:08:58.549Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3041
File: .github/workflows/run-ci-cd.yaml:233-243
Timestamp: 2025-12-26T06:08:58.549Z
Learning: For the OWASP/Nest repository, Redis image versions should remain consistent across all environments (production, staging, local, E2E, and CI/CD E2E tests). When upgrading Redis, update all docker-compose files and CI/CD workflow configurations together to maintain environment parity.
Applied to files:
backend/.env.fuzz.example
📚 Learning: 2025-12-26T06:08:58.549Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3041
File: .github/workflows/run-ci-cd.yaml:233-243
Timestamp: 2025-12-26T06:08:58.549Z
Learning: Ensure Redis image versions stay in sync across all environments by updating every relevant YAML file together (docker-compose files and CI/CD workflow configurations). When upgrading Redis, bump the image version in all docker-compose files and in all CI workflow YAMLs in one coordinated change to maintain parity across production, staging, local, E2E, and CI tests.
Applied to files:
docker-compose/e2e.yamldocker-compose/fuzz.yaml
📚 Learning: 2025-12-26T06:57:19.911Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 1139
File: docker-compose/fuzz.yaml:70-83
Timestamp: 2025-12-26T06:57:19.911Z
Learning: In docker-compose/fuzz.yaml, document and enforce that the data-loader and graphql services are started in separate phases as defined by the Makefile target test-fuzz: first load data to completion, then start the graphql fuzzer against the populated database. Do not rely on orchestration order from docker-compose dependencies; instead ensure the Makefile orchestrates the phased startup and that health checks reflect runtime readiness rather than enforcing a start order. This guideline is specific to this file and to the fuzz testing setup in the OWASP/Nest repository.
Applied to files:
docker-compose/fuzz.yaml
🪛 dotenv-linter (4.0.0)
backend/.env.fuzz.example
[warning] 16-16: [UnorderedKey] The DJANGO_REDIS_HOST key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
[warning] 17-17: [UnorderedKey] The DJANGO_REDIS_PASSWORD key should go before the DJANGO_SETTINGS_MODULE key
(UnorderedKey)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: Run frontend unit tests
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run backend tests
- GitHub Check: Run fuzz tests
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (4)
docker-compose/e2e.yaml (2)
15-16: LGTM! Backend properly depends on cache availability.The
depends_onwithservice_healthycondition ensures Redis is ready before the backend starts, preventing connection failures.
73-91: LGTM! Cache service configuration is complete and correct.The Redis cache service is properly configured with:
- Consistent Redis version (8.0.5) matching other environments as per learnings
- Password protection with healthcheck authentication
- Memory limits and eviction policy (100MB, allkeys-lru)
- Dedicated volume for persistence
Previous concerns about password mismatches and version inconsistencies have been addressed.
Also applies to: 112-112
docker-compose/fuzz.yaml (1)
71-89: LGTM! Cache service configuration is correct and consistent.The Redis cache service is properly configured with:
- Consistent Redis version (8.0.5) matching the e2e environment
- Password matching
backend/.env.fuzz.example- Memory limits and eviction policy
- Dedicated volume for persistence
The configuration correctly mirrors the e2e environment pattern.
Also applies to: 110-110
backend/apps/api/rest/v0/__init__.py (1)
71-81: LGTM! Fuzz environment API configuration is correct.The fuzz environment configuration appropriately:
- Disables authentication and throttling for automated testing
- Sets a descriptive server URL
- Mirrors the established E2E environment pattern
This implementation is consistent with the overall approach for test environments.
ahmedxgouda
changed the title
|
@coderabbitai The backend system goes down after some fuzz requests. This is new. It didn't happen before the last update to the feature branch to be up to date with the main branch. Could you find the problem and suggest fixes? |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 745 🏁 Script executed: Length of output: 972 🏁 Script executed: Length of output: 5510 🏁 Script executed: Length of output: 3586 🏁 Script executed: Length of output: 2075 🏁 Script executed: Length of output: 740 🏁 Script executed: Length of output: 1822 🏁 Script executed: Length of output: 621 🏁 Script executed: Length of output: 4092 🏁 Script executed: Length of output: 1653 🏁 Script executed: Length of output: 1178 🏁 Script executed: Length of output: 1364 🏁 Script executed: Length of output: 1010 🏁 Script executed: Length of output: 811 @ahmedxgouda I found the problem! The backend is crashing because of a missing dependency in Root CauseAfter merging from main, the mentorship app now includes a signal handler that gets triggered when Program models are saved. This signal calls Here's the issue in your docker-compose configuration: In backend:
depends_on:
db:
condition: service_healthy
# ❌ Missing cache dependency!In backend:
depends_on:
db:
condition: service_healthy
cache:
condition: service_healthy # ✅ Cache dependency presentWhen the data loader runs FixUpdate backend:
container_name: fuzz-nest-backend
command: >
sh -c '
python manage.py migrate &&
gunicorn wsgi:application --bind 0.0.0.0:9500
'
build:
context: ../backend
dockerfile: docker/Dockerfile
depends_on:
db:
condition: service_healthy
cache:
condition: service_healthy # Add this
env_file: ../backend/.env.fuzz.example
networks:
- fuzz-nest-network
ports:
- 9500:9500
healthcheck:
interval: 10s
retries: 10
test: >
sh -c '
wget --spider http://backend:9500/a/
'
timeout: 10s
start_period: 5sThis ensures the backend waits for Redis to be healthy before starting, preventing connection failures when the signal handlers try to access the cache during data loading. 🧠 Learnings used |
|
@coderabbitai It is not related. The backend goes down also in ci/cd, and I added the dependency and it goes down after some fuzz requests with this error: |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 1744 🏁 Script executed: Length of output: 512 🏁 Script executed: Length of output: 273 🏁 Script executed: Length of output: 2783 🏁 Script executed: Length of output: 894 🏁 Script executed: Length of output: 5344 🏁 Script executed: Length of output: 998 🏁 Script executed: Length of output: 155 🏁 Script executed: Length of output: 2550 🏁 Script executed: Length of output: 1827 🏁 Script executed: Length of output: 331 🏁 Script executed: Length of output: 541 🏁 Script executed: Length of output: 1264 🏁 Script executed: Length of output: 886 🏁 Script executed: Length of output: 5158 🏁 Script executed: Length of output: 525 🏁 Script executed: Length of output: 144 🏁 Script executed: Length of output: 546 🏁 Script executed: Length of output: 820 🏁 Script executed: Length of output: 700 🏁 Script executed: Length of output: 854 🏁 Script executed: Length of output: 1047 @ahmedxgouda I've found the root cause of the OOM issue! It's a combination of problems introduced by the recent merge from main: Root Cause AnalysisPerfect! The web search confirms the issue. Here are the three problems causing the OOM:
Complete SolutionHere are the fixes to apply across all environments: 1. Update
|
|
@coderabbitai Still the same error. Also, the clear_index_cache method last update was 4 months ago. |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 669 🏁 Script executed: Length of output: 369 🏁 Script executed: Length of output: 180 🏁 Script executed: Length of output: 513 🏁 Script executed: Length of output: 1354 🏁 Script executed: Length of output: 2246 🏁 Script executed: Length of output: 772 🏁 Script executed: Length of output: 1127 🏁 Script executed: Length of output: 246 🏁 Script executed: Length of output: 315 🏁 Script executed: Length of output: 8813 🏁 Script executed: Length of output: 1218 🏁 Script executed: Length of output: 1782 🏁 Script executed: Length of output: 2577 🏁 Script executed: Length of output: 1108 🏁 Script executed: Length of output: 1382 🏁 Script executed: Length of output: 39 🏁 Script executed: Length of output: 1225 🏁 Script executed: Length of output: 2188 🏁 Script executed: Length of output: 979 🏁 Script executed: Length of output: 898 🏁 Script executed: Length of output: 558 🌐 Web query: 💡 Result: Key django-redis connection-pool settings and how to use them
Sources
References @ahmedxgouda I found it! The real issue is from PR #2528 (GraphQL resolver caching) that was merged from main on December 19, 2025. This introduced the Root CauseThe fuzzer ( The FixAdd connection pool limits to prevent unlimited connection creation: Update
|
|
@coderabbitai still the same error. |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 108 🏁 Script executed: Length of output: 513 🏁 Script executed: Length of output: 1934 🏁 Script executed: Length of output: 6978 🏁 Script executed: Length of output: 53 🏁 Script executed: Length of output: 100 🏁 Script executed: Length of output: 6692 🏁 Script executed: Length of output: 6704 🏁 Script executed: Length of output: 3082 🏁 Script executed: Length of output: 1490 🏁 Script executed: Length of output: 2269 🏁 Script executed: Length of output: 1441 🏁 Script executed: Length of output: 324 @ahmedxgouda I found it! The root cause is unbounded @lru_cache decorators in backend/apps/slack/utils.py causing memory accumulation during fuzzing. Root CauseThe LRU cache retains references to all of the arguments of the decorated function in its cache, and if we invoke such a decorated function with an object as an argument, that object will persist in memory indefinitely, or until the program terminates or the cache is cleared. You have 4 functions in
During 300 seconds of fuzzing, the GraphQL fuzzer generates random queries that access fields like Setting maxsize to None can lead to unbounded cache growth, which, if unchecked, can consume all available memory and lead to performance degradation, and the same problem occurs when cached values are large objects like QuerySets. The Fix1. Update
|
|
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docker-compose/fuzz/compose.yaml (1)
1-8: Add explicitmaxsizelimits to unboundedlru_cachedecorators inbackend/apps/slack/utils.py.Four functions have unbounded
lru_cachedecorators that cause memory accumulation during fuzz testing:get_gsoc_projects,get_news_data,get_staff_data, andget_posts_data. Add explicitmaxsizelimits (e.g.,maxsize=16) to each decorator to prevent OOM issues.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
backend/Makefiledocker-compose/e2e/compose.yamldocker-compose/fuzz/compose.yamlfrontend/Makefile
🚧 Files skipped from review as they are similar to previous changes (1)
- frontend/Makefile
🧰 Additional context used
🧠 Learnings (5)
📓 Common learnings
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 0
File: :0-0
Timestamp: 2025-12-30T15:05:23.146Z
Learning: In the OWASP/Nest repository, four functions in backend/apps/slack/utils.py (get_gsoc_projects, get_news_data, get_staff_data, get_posts_data) have unbounded lru_cache decorators that can cause memory accumulation during fuzz testing. These should have explicit maxsize limits (e.g., maxsize=16) to prevent OOM issues.
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3041
File: .github/workflows/run-ci-cd.yaml:233-243
Timestamp: 2025-12-26T06:08:58.549Z
Learning: For the OWASP/Nest repository, Redis image versions should remain consistent across all environments (production, staging, local, E2E, and CI/CD E2E tests). When upgrading Redis, update all docker-compose files and CI/CD workflow configurations together to maintain environment parity.
📚 Learning: 2025-12-26T06:57:19.911Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 1139
File: docker-compose/fuzz.yaml:70-83
Timestamp: 2025-12-26T06:57:19.911Z
Learning: In the fuzz testing environment (docker-compose/fuzz.yaml), the data-loader and graphql services are orchestrated in separate phases via the Makefile's test-fuzz target, not run simultaneously. The data is loaded to completion first, then the graphql fuzzer is started separately against the populated database. The docker-compose dependencies reflect runtime health checks rather than orchestration order.
Applied to files:
backend/Makefiledocker-compose/fuzz/compose.yaml
📚 Learning: 2025-10-26T12:50:50.512Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 2429
File: backend/Makefile:30-32
Timestamp: 2025-10-26T12:50:50.512Z
Learning: The `exec-backend-e2e-command` and `exec-db-e2e-command` Makefile targets in the backend/Makefile are intended for local development and debugging only, not for CI/CD execution, so the `-it` flags are appropriate.
Applied to files:
backend/Makefile
📚 Learning: 2025-12-26T06:08:58.549Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3041
File: .github/workflows/run-ci-cd.yaml:233-243
Timestamp: 2025-12-26T06:08:58.549Z
Learning: For the OWASP/Nest repository, Redis image versions should remain consistent across all environments (production, staging, local, E2E, and CI/CD E2E tests). When upgrading Redis, update all docker-compose files and CI/CD workflow configurations together to maintain environment parity.
Applied to files:
backend/Makefile
📚 Learning: 2025-12-26T06:08:58.549Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3041
File: .github/workflows/run-ci-cd.yaml:233-243
Timestamp: 2025-12-26T06:08:58.549Z
Learning: Ensure Redis image versions stay in sync across all environments by updating every relevant YAML file together (docker-compose files and CI/CD workflow configurations). When upgrading Redis, bump the image version in all docker-compose files and in all CI workflow YAMLs in one coordinated change to maintain parity across production, staging, local, E2E, and CI tests.
Applied to files:
docker-compose/e2e/compose.yamldocker-compose/fuzz/compose.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: Run fuzz tests
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run backend tests
- GitHub Check: Run frontend unit tests
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (8)
backend/Makefile (2)
133-139: LGTM! Cache service correctly added to e2e and fuzz targets.The
run-backend-e2eandrun-backend-fuzztargets now properly include thecacheservice alongsidebackendanddb, ensuring Redis is available during these test runs. The path update to the new directory structure (docker-compose/e2e/compose.yamlanddocker-compose/fuzz/compose.yaml) is consistent.
167-173: LGTM! Both test-fuzz phases now include the cache service.The cache service is correctly included in both orchestration phases:
- First phase (data loading):
db cache backend data-loader- Second phase (fuzzing):
db cache backend graphqlThis ensures Redis is available during data loading (when
clear_index_cache()may be triggered by signals) and during the actual fuzz testing.docker-compose/e2e/compose.yaml (3)
10-17: LGTM! Backend service correctly depends on cache health.The backend now waits for both
dbandcacheservices to be healthy before starting. This addresses the root cause identified in Issue #3034 whereclear_index_cache()could fail if Redis wasn't ready during data loading.
92-112: LGTM! Path updates and volume declarations are correct.The e2e-tests service paths are correctly updated for the new directory structure, and the
e2e-cache-datavolume is properly declared.
73-91: Cache service configuration is correct and Redis version is consistent across all environments.The cache service is well-configured with:
- Memory limit (
--maxmemory 100mb) to prevent unbounded growth- LRU eviction policy appropriate for caching
- Password authentication
- Proper healthcheck
Redis version
redis:8.0.5-alpine3.21is consistent across all environments:
- Local:
docker-compose/local/compose.yaml- Production:
docker-compose/production/compose.yaml- Staging:
docker-compose/staging/compose.yaml- E2E:
docker-compose/e2e/compose.yaml- Fuzz:
docker-compose/fuzz/compose.yaml- CI/CD:
.github/workflows/run-ci-cd.yamldocker-compose/fuzz/compose.yaml (3)
10-17: LGTM! Backend correctly depends on cache for fuzz environment.The cache dependency ensures Redis is healthy before the backend starts, addressing the crash issue where
clear_index_cache()failed during data loading when Redis wasn't available.
92-112: LGTM! Path updates and volume declaration are correct.The graphql service paths are correctly updated for the new directory structure (
../../backend/dockerand../../backend/fuzzing_results), and thefuzz-cache-datavolume is properly declared.
72-91: Verify Redis version consistency and password alignment across all environments.The cache service configuration appears appropriate with memory limits and LRU eviction. However, confirm: (1) the default password
nest-fuzz-cache-passwordmatchesDJANGO_REDIS_PASSWORDinbackend/.env.fuzz.example, and (2) Redis version8.0.5-alpine3.21is consistent across all docker-compose files (production, staging, local, E2E) and CI/CD workflow configurations to maintain environment parity.
2 participants
Proposed change
Resolves #3034
Add the PR description here.
Checklist
make check-testlocally and all tests passed