Skip to content

Conversation

@loloop
Copy link
Member

@loloop loloop commented Sep 29, 2025

This PR intends to:
• Add a vapor backend starter base to the project
• Remove the SwiftSoup dependency for meetup fetching from the app, delegating that feature to the backend instead

Tasks:

  • Create a common shared package between the apps and the backend (CocoaHeadsCore)
  • Move SwiftSoup and the meetup fetching service to the backend

Next steps

  • Migrate the app to use the backend functions
  • Automate testing and deployment of new versions of the backend
  • Piece by piece move Chapters, Raffles, Q&A and everything that depends on CloudKit to the backend

@claude
Copy link

claude bot commented Sep 29, 2025

Claude finished @loloop's task —— View job


PR Review: Vapor Backend Addition

Reviewing this PR that adds a Vapor backend and creates a shared CocoaHeadsCore package.

Review Checklist:

  • Analyze overall architecture changes
  • Review CocoaHeadsCore shared package structure
  • Examine Vapor backend implementation
  • Check MeetupController security and implementation
  • Review Docker configuration
  • Assess test coverage
  • Check for potential bugs and performance issues
  • Evaluate code quality and best practices

Review Summary

This is a well-structured PR that introduces a Vapor backend to move functionality away from CloudKit dependencies. The architecture changes are thoughtful and the implementation is generally solid.

Strengths

Architecture & Design:

  • Clean separation with CocoaHeadsCore as shared package
  • Proper dependency injection pattern in backend
  • Good use of Swift 6.2 and modern language features
  • Workspace structure properly organized

Backend Implementation:

  • MeetupController follows Vapor best practices with RouteCollection
  • Proper async/await usage throughout
  • Good error handling with custom Error enum and localized descriptions
  • Clean parsing logic with separation of concerns

Infrastructure:

  • Well-configured Dockerfile with multi-stage build and security practices
  • Comprehensive docker-compose.yml for development
  • Proper user management (vapor user) in container

⚠️ Areas for Improvement

Security Concerns:

  1. URL Validation (backend/Sources/backend/Controllers/MeetupController.swift:23-28):

    guard
      let url = URL(string: meetup.url),
      case (let data, _) = try await URLSession.shared.data(from: url),
      let html = String(data: data, encoding: .utf8)
    • No URL scheme validation - could allow file://, ftp://, etc.
    • No domain whitelist - allows requests to any host
    • Recommendation: Add URL validation to only allow HTTPS meetup.com URLs
  2. Database Configuration (backend/Sources/backend/configure.swift:11-20):

    • Uses default credentials in fallback values
    • Recommendation: Consider failing fast if environment variables aren't set in production

Code Quality Issues:

  1. Platform Targets Inconsistency (CocoaHeadsCore/Package.swift:8-12):

    platforms: [
      .iOS("18.2"),
      .visionOS("2.2"), 
      .macOS(.v26)  // This version doesn't exist
    • Bug: .macOS(.v26) is invalid - should be .macOS(.v15) or similar
  2. Test Coverage (backend/Tests/backendTests/backendTests.swift):

    • All tests are commented out
    • No actual test coverage for MeetupController
    • Critical: Missing validation for the core meetup parsing functionality
  3. Error Handling Edge Cases:

    • Title parsing could fail with empty string if no comma found
    • Image parsing has no fallback for missing images
    • Coordinate extraction could be more robust

Performance Considerations:

  1. Blocking Operations (MeetupController.swift:24):

    case (let data, _) = try await URLSession.shared.data(from: url)
    • No timeout configuration
    • No request size limits
    • Recommendation: Add timeout and size limits
  2. String Operations (MeetupController.swift:152-158):

    • Regex compilation on every call in removingFirstOccurrence
    • Recommendation: Pre-compile regex patterns or use simpler string operations

Documentation & Maintenance:

  1. Missing Documentation:
    • No API documentation for MeetupController endpoints
    • No documentation for error responses
    • Missing environment setup instructions

📋 Recommendations

High Priority:

  1. Fix the macOS platform version in CocoaHeadsCore/Package.swift
  2. Add URL validation to whitelist only HTTPS meetup.com URLs
  3. Implement actual tests for MeetupController functionality
  4. Add proper timeout and size limits to URLSession requests

Medium Priority:

  1. Add comprehensive API documentation
  2. Implement request rate limiting for the meetup endpoint
  3. Add structured logging throughout the backend
  4. Consider adding health check endpoints

Low Priority:

  1. Optimize regex compilation in string extensions
  2. Add more robust fallbacks for parsing edge cases
  3. Consider adding response caching for meetup data

🎯 Next Steps Alignment

The PR aligns well with the stated goals:

  • ✅ CocoaHeadsCore shared package created successfully
  • ✅ SwiftSoup moved to backend
  • 🔄 App migration to backend (next step)
  • 🔄 Automation setup (next step)

The foundation is solid for the planned migration of Chapters, Raffles, Q&A, and CloudKit dependencies to the backend.


@claude
Copy link

claude bot commented Sep 30, 2025

Claude finished @loloop's task —— View job


PR Review: Vapor Backend Addition

Reviewing this PR that adds a Vapor backend and creates a shared CocoaHeadsCore package.

Review Checklist:

  • Analyze overall architecture changes
  • Review CocoaHeadsCore shared package structure
  • Examine Vapor backend implementation
  • Check MeetupController security and implementation
  • Review Docker configuration
  • Assess test coverage
  • Check for potential bugs and performance issues
  • Evaluate code quality and best practices

Review Summary

This is a well-structured PR that introduces a Vapor backend to move functionality away from CloudKit dependencies. The architecture changes are thoughtful and the implementation is generally solid.

Strengths

Architecture & Design:

  • Clean separation with CocoaHeadsCore as shared package
  • Proper dependency injection pattern in backend
  • Good use of Swift 6.2 and modern language features
  • Workspace structure properly organized

Backend Implementation:

  • MeetupController follows Vapor best practices with RouteCollection
  • Proper async/await usage throughout
  • Good error handling with custom Error enum and localized descriptions
  • Clean parsing logic with separation of concerns

Infrastructure:

  • Well-configured Dockerfile with multi-stage build and security practices
  • Comprehensive docker-compose.yml for development
  • Proper user management (vapor user) in container

⚠️ Areas for Improvement

Security Concerns:

  1. URL Validation (backend/Sources/backend/Controllers/MeetupController.swift:23-28):

    guard
      let url = URL(string: meetup.url),
      case (let data, _) = try await URLSession.shared.data(from: url),
      let html = String(data: data, encoding: .utf8)
    • No URL scheme validation - could allow file://, ftp://, etc.
    • No domain whitelist - allows requests to any host
    • Recommendation: Add URL validation to only allow HTTPS meetup.com URLs
  2. Database Configuration (backend/Sources/backend/configure.swift:11-20):

    • Uses default credentials in fallback values
    • Recommendation: Consider failing fast if environment variables aren't set in production

Code Quality Issues:

  1. Platform Targets Inconsistency (CocoaHeadsCore/Package.swift:8-12):

    platforms: [
      .iOS("18.2"),
      .visionOS("2.2"), 
      .macOS(.v26)  // This version doesn't exist
    • Bug: .macOS(.v26) is invalid - should be .macOS(.v15) or similar
  2. Backend Platform Target (backend/Package.swift:6-8):

    platforms: [
      .macOS(.v26)  // Same issue here
    • Bug: .macOS(.v26) is invalid
  3. Test Coverage (backend/Tests/backendTests/backendTests.swift):

    • All tests are commented out
    • No actual test coverage for MeetupController
    • Critical: Missing validation for the core meetup parsing functionality
  4. Error Handling Edge Cases:

    • Title parsing could fail with empty string if no comma found
    • Image parsing has no fallback for missing images
    • Coordinate extraction could be more robust

Performance Considerations:

  1. Blocking Operations (MeetupController.swift:24):

    case (let data, _) = try await URLSession.shared.data(from: url)
    • No timeout configuration
    • No request size limits
    • Recommendation: Add timeout and size limits
  2. String Operations (MeetupController.swift:152-158):

    • Regex compilation on every call in removingFirstOccurrence
    • Recommendation: Pre-compile regex patterns or use simpler string operations

Documentation & Maintenance:

  1. Missing Documentation:
    • No API documentation for MeetupController endpoints
    • No documentation for error responses
    • Missing environment setup instructions

📋 Recommendations

High Priority:

  1. Fix the macOS platform version in both CocoaHeadsCore/Package.swift and backend/Package.swift
  2. Add URL validation to whitelist only HTTPS meetup.com URLs
  3. Implement actual tests for MeetupController functionality
  4. Add proper timeout and size limits to URLSession requests

Medium Priority:

  1. Add comprehensive API documentation
  2. Implement request rate limiting for the meetup endpoint
  3. Add structured logging throughout the backend
  4. Consider adding health check endpoints

Low Priority:

  1. Optimize regex compilation in string extensions
  2. Add more robust fallbacks for parsing edge cases
  3. Consider adding response caching for meetup data

🎯 Next Steps Alignment

The PR aligns well with the stated goals:

  • ✅ CocoaHeadsCore shared package created successfully
  • ✅ SwiftSoup moved to backend
  • 🔄 App migration to backend (next step)
  • 🔄 Automation setup (next step)

The foundation is solid for the planned migration of Chapters, Raffles, Q&A, and CloudKit dependencies to the backend.


@loloop loloop marked this pull request as ready for review September 30, 2025 20:11
@claude
Copy link

claude bot commented Sep 30, 2025

Claude finished @loloop's task —— View job


PR Review: Vapor Backend Addition

Reviewing this PR that adds a Vapor backend and creates a shared CocoaHeadsCore package.

Review Checklist:

  • Analyze overall architecture changes
  • Review CocoaHeadsCore shared package structure
  • Examine Vapor backend implementation
  • Check MeetupController security and implementation
  • Review Docker configuration and deployment
  • Assess test coverage
  • Check for potential bugs and performance issues
  • Evaluate code quality and best practices

Review Summary

This is a well-structured PR that introduces a Vapor backend to move functionality away from CloudKit dependencies. The architecture changes are thoughtful and the implementation is generally solid.

Strengths

Architecture & Design:

  • Clean separation with CocoaHeadsCore as shared package
  • Proper dependency injection pattern in backend
  • Good use of Swift 6.2 and modern language features
  • Workspace structure properly organized

Backend Implementation:

  • MeetupController follows Vapor best practices with RouteCollection
  • Proper async/await usage throughout
  • Good error handling with custom Error enum and localized descriptions
  • Clean parsing logic with separation of concerns

Infrastructure:

  • Well-configured Dockerfile with multi-stage build and security practices
  • Comprehensive docker-compose.yml for development
  • Proper user management (vapor user) in container

⚠️ Areas for Improvement

Critical Issues:

  1. Platform Targets Bug (CocoaHeadsCore/Package.swift:11 & backend/Package.swift:7):
    .macOS(.v26)  // This version doesn't exist
    • Bug: .macOS(.v26) is invalid - should be .macOS(.v15) or similar
    • This will cause build failures

Security Concerns:

  1. URL Validation (backend/Sources/backend/Controllers/MeetupController.swift:23-28):

    guard
      let url = URL(string: meetup.url),
      case (let data, _) = try await URLSession.shared.data(from: url),
      let html = String(data: data, encoding: .utf8)
    • No URL scheme validation - could allow file://, ftp://, etc.
    • No domain whitelist - allows requests to any host
    • Recommendation: Add URL validation to only allow HTTPS meetup.com URLs
  2. Database Configuration (backend/Sources/backend/configure.swift:16-18):

    username: Environment.get("DATABASE_USERNAME") ?? "vapor_username",
    password: Environment.get("DATABASE_PASSWORD") ?? "vapor_password",
    • Uses default credentials in fallback values
    • Recommendation: Consider failing fast if environment variables aren't set in production

Code Quality Issues:

  1. Test Coverage (backend/Tests/backendTests/backendTests.swift):

    • All tests are commented out
    • No actual test coverage for MeetupController
    • Critical: Missing validation for the core meetup parsing functionality
  2. Error Handling Edge Cases:

    • Title parsing (MeetupController.swift:49) could fail with empty string if no comma found
    • Image parsing has no fallback for missing images
    • Coordinate extraction could be more robust

Performance Considerations:

  1. HTTP Request Configuration (MeetupController.swift:24):

    case (let data, _) = try await URLSession.shared.data(from: url)
    • No timeout configuration
    • No request size limits
    • Recommendation: Add timeout and size limits
  2. Regex Compilation (MeetupController.swift:154):

    guard let range = range(of: "\\b\(word)\\b", options: [.regularExpression, .caseInsensitive])
    • Regex compilation on every call in removingFirstOccurrence
    • Recommendation: Pre-compile regex patterns or use simpler string operations

Documentation & Maintenance:

  1. Missing Documentation:
    • No API documentation for MeetupController endpoints
    • No documentation for error responses
    • Missing environment setup instructions

📋 Recommendations

High Priority:

  1. Fix macOS platform version in both CocoaHeadsCore/Package.swift and backend/Package.swift
  2. Add URL validation to whitelist only HTTPS meetup.com URLs
  3. Implement actual tests for MeetupController functionality
  4. Add proper timeout and size limits to URLSession requests

Medium Priority:

  1. Add comprehensive API documentation
  2. Implement request rate limiting for the meetup endpoint
  3. Add structured logging throughout the backend
  4. Consider adding health check endpoints

Low Priority:

  1. Optimize regex compilation in string extensions
  2. Add more robust fallbacks for parsing edge cases
  3. Consider adding response caching for meetup data

🎯 Next Steps Alignment

The PR aligns well with the stated goals:

  • ✅ CocoaHeadsCore shared package created successfully
  • ✅ SwiftSoup moved to backend
  • 🔄 App migration to backend (next step)
  • 🔄 Automation setup (next step)

The foundation is solid for the planned migration of Chapters, Raffles, Q&A, and CloudKit dependencies to the backend.


Copy link
Collaborator

@unnamedd unnamedd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@loloop, thanks for the effort here. As I am not working—not even a bit—in the CH projects, I do not see myself in a position to block your work with a "Request changes" from GitHub's Pull Request; however, I’m raising some points here for your appreciation.

Give a check, and if you think they are addressable, apply the comments and, if you want me to give you a second review, just re-request a review and I will do so with great pleasure.

Comment on lines 22 to 28
guard
let url = URL(string: meetup.url),
case (let data, _) = try await URLSession.shared.data(from: url),
let html = String(data: data, encoding: .utf8)
else {
throw Error.urlError
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest you to split into two different guards here. You are not being clear enough when raising errors.
If something goes wrong with the URL, then you will throw Error.urlError, however, in case you get an error while you are requesting the content from that URL using URLSession or get an error while you are converting from the received data into a String, you will also throw Error.urlError. You had to separate at least in two more guards to throw the proper error.

throw Error.urlError
}

guard !url.absoluteString.contains("Entrar") else {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is "Entrar" a safe string to use?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used to be, but meetup has released a completely new design while I was working on moving this code around, so I'll need to do more testing to figure out if everything here still works

case .addressError:
"Erro ao buscar o endereço"
case .coordinateError:
"Erro no parse das coordenadas"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are going to use the error messages in portuguese, perhaps here we should use "extrair" instead of "parse". What do you think?

Suggested change
"Erro no parse das coordenadas"
"Erro ao extrair as coordenadas"

Copy link
Member Author

@loloop loloop Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These errors used to be part of the UI when SwiftSoup was directly integrated in the app. I'm totally down to translate these messages to english now that they're part of the API

case .coordinateError:
"Erro no parse das coordenadas"
case .dateError:
"Não consegui encontrar ou fazer decode da data"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"Não consegui encontrar ou fazer decode da data"
"Não foi possível encontrar ou decodificar a data"


extension String {
fileprivate func removingFirstOccurrence(of word: String) -> String {
guard let range = range(of: "\\b\(word)\\b", options: [.regularExpression, .caseInsensitive]) else {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we have just one use case, but I was wondering if would be a good place to make use of RegexBuilder here

// Created by Mauricio on 5/14/25.
//

import CocoaHeadsCore
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CocoaHeadsKit or CocoaHeadsCore? I see them being mixed here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CocoaHeadsKit is the package that provides all the code for the apps, CocoaHeadsCore is a new package that provides shared code between the apps and the Vapor backend

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps it would be better if they were renamed in order to avoid confusion. To me, as you probably noticed, they were just getting mixed with each other, while in fact, they do separate things, but I am completely out of ideas to suggest something better haha

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think better documenting them would help with this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed, it would

@claude
Copy link

claude bot commented Oct 27, 2025

Claude finished @loloop's task —— View job


PR Review: Vapor Backend Addition

Reviewing this PR that adds a Vapor backend and creates a shared CocoaHeadsCore package.

Review Checklist:

  • Analyze overall architecture changes
  • Review CocoaHeadsCore shared package structure
  • Examine Vapor backend implementation
  • Check for security concerns
  • Review Docker configuration and deployment
  • Assess test coverage
  • Check for potential bugs and performance issues
  • Evaluate code quality and best practices
  • Address previous review feedback

Review Summary

This is a significant architectural improvement that successfully moves from SwiftSoup dependency to a modern Vapor backend with Firecrawl integration. The code quality is generally good, but there are critical issues that must be addressed before merging.

Major Improvements Since Previous Reviews

1. Modernized Event Scraping:

  • ✅ Replaced brittle SwiftSoup parsing with robust Firecrawl API
  • ✅ Better structured data extraction with schema validation
  • ✅ Cleaner error handling with localized messages (now in English)

2. Architecture & Design:

  • ✅ Clean separation with CocoaHeadsCore as shared package
  • ✅ Proper dependency injection pattern in backend
  • ✅ Good use of Swift 6.2 and modern language features
  • ✅ Well-structured workspace organization

3. Infrastructure:

  • ✅ Comprehensive Docker setup with multi-stage builds
  • ✅ Production-ready deployment scripts
  • ✅ GitHub Actions for automated deployment

🚨 Critical Issues

1. Invalid macOS Platform Versions (CocoaHeadsCore/Package.swift:11 & backend/Package.swift:7):

.macOS(.v26)  // This version doesn't exist - will cause build failures
  • Impact: Complete build failure across the project
  • Fix: Change to .macOS(.v15) or similar valid version

2. URL Security Vulnerability (backend/Sources/backend/Controllers/EventScrape/ScrapingController.swift:27):

guard let url = URL(string: meetup.url) else {
  throw MeetupError.urlError
}
  • Risk: Allows arbitrary URLs including file://, ftp://, javascript://
  • Fix: Add URL scheme validation to only allow HTTPS

3. Missing API Key Validation:

  • Firecrawl service properly checks for API key, but no rate limiting or validation
  • Could lead to API quota exhaustion

⚠️ Security Concerns

1. Database Configuration (backend/Sources/backend/configure.swift:16-18):

username: Environment.get("DATABASE_USERNAME") ?? "vapor_username",
password: Environment.get("DATABASE_PASSWORD") ?? "vapor_password",
  • Uses hardcoded fallback credentials
  • Recommendation: Fail fast if environment variables aren't set in production

2. Potential SSRF Attack:

  • No URL domain validation allows requests to internal services
  • Recommendation: Whitelist allowed domains (e.g., meetup.com, luma.com)

📊 Test Coverage Issues

1. Complete Test Absence (backend/Tests/backendTests/backendTests.swift):

  • All tests are commented out
  • Critical: No validation for the core scraping functionality
  • Missing tests for error handling, URL validation, data parsing

2. Missing Integration Tests:

  • No tests for Firecrawl service integration
  • No tests for ScrapingController endpoints

🔧 Code Quality Issues

1. Error Handling Edge Cases:

  • Location defaults to (0.0, 0.0) when coordinates missing (ScrapingController.swift:43-44)
  • No validation for required fields beyond basic guard clauses
  • Date parsing only supports ISO8601, no fallback formats

2. Performance Considerations:

  • Firecrawl polling without exponential backoff (FirecrawlService.swift:51-58)
  • No timeout configuration for HTTP requests
  • Missing response caching for repeated requests

3. Code Consistency:

  • Mix of error message languages (some Portuguese remnants in comments)
  • Inconsistent error types between different controllers

🎯 Positive Architecture Decisions

1. Service Layer Design:

  • ✅ Protocol-based FirecrawlServiceProtocol enables testing
  • ✅ Clean separation between scraping logic and HTTP handling
  • ✅ Proper dependency injection in ScrapingController

2. Data Modeling:

  • ✅ Well-structured MeetupEvent with nested types
  • ✅ Comprehensive Firecrawl schema definition
  • ✅ Proper Codable conformance throughout

3. Deployment:

  • ✅ Production-ready Dockerfile with security best practices
  • ✅ Automated deployment pipeline
  • ✅ Environment-based configuration

📋 Immediate Action Items

Critical (Must Fix Before Merge):

  1. Fix macOS platform versions in both Package.swift files
  2. Add URL validation to only allow HTTPS URLs from trusted domains
  3. Implement basic tests for ScrapingController and error handling

High Priority:

  1. Add environment variable validation (fail fast if missing in production)
  2. Implement request timeout and size limits
  3. Add exponential backoff for Firecrawl polling

Medium Priority:

  1. Add comprehensive API documentation
  2. Implement rate limiting for scraping endpoints
  3. Add structured logging throughout backend
  4. Consider adding health check endpoints

🎯 Next Steps Alignment

The PR successfully achieves its stated goals:

  • ✅ CocoaHeadsCore shared package created and working
  • ✅ SwiftSoup moved to backend with modern Firecrawl replacement
  • 🔄 Ready for app migration to backend APIs (next phase)
  • 🔄 Infrastructure ready for automation (next phase)

This provides a solid foundation for migrating Chapters, Raffles, Q&A, and CloudKit dependencies to the backend.


@claude
Copy link

claude bot commented Oct 29, 2025

Claude finished @loloop's task —— View job


PR Review: Vapor Backend Addition

Reviewing this PR that adds a Vapor backend and creates a shared CocoaHeadsCore package.

Review Checklist:

  • Analyze overall architecture changes
  • Review CocoaHeadsCore shared package structure
  • Examine Vapor backend implementation
  • Check for security concerns
  • Review Docker configuration and deployment
  • Assess test coverage
  • Check for potential bugs and performance issues
  • Evaluate code quality and best practices
  • Address previous review feedback

Review Summary

This is a significant architectural improvement that successfully moves from SwiftSoup dependency to a modern Vapor backend with Firecrawl integration. The code quality is generally good, but there are critical issues that must be addressed before merging.

Major Improvements Since Previous Reviews

1. Modernized Event Scraping:

  • ✅ Replaced brittle SwiftSoup parsing with robust Firecrawl API
  • ✅ Better structured data extraction with schema validation
  • ✅ Cleaner error handling with localized messages (now in English)

2. Architecture & Design:

  • ✅ Clean separation with CocoaHeadsCore as shared package
  • ✅ Proper dependency injection pattern in backend
  • ✅ Good use of Swift 6.2 and modern language features
  • ✅ Well-structured workspace organization

3. Infrastructure:

  • ✅ Comprehensive Docker setup with multi-stage builds
  • ✅ Production-ready deployment scripts
  • ✅ GitHub Actions for automated deployment

🚨 Critical Issues

1. Invalid macOS Platform Versions (CocoaHeadsCore/Package.swift:11 & backend/Package.swift:7):

.macOS(.v26)  // This version doesn't exist - will cause build failures
  • Impact: Complete build failure across the project
  • Fix: Change to .macOS(.v15) or similar valid version

2. URL Security Vulnerability (backend/Sources/backend/Controllers/EventScrape/ScrapingController.swift:27):

guard let url = URL(string: meetup.url) else {
  throw MeetupError.urlError
}
  • Risk: Allows arbitrary URLs including file://, ftp://, javascript://
  • Fix: Add URL scheme validation to only allow HTTPS

3. Missing API Key Validation:

  • Firecrawl service properly checks for API key, but no rate limiting or validation
  • Could lead to API quota exhaustion

⚠️ Security Concerns

1. Database Configuration (backend/Sources/backend/configure.swift:16-18):

username: Environment.get("DATABASE_USERNAME") ?? "vapor_username",
password: Environment.get("DATABASE_PASSWORD") ?? "vapor_password",
  • Uses hardcoded fallback credentials
  • Recommendation: Fail fast if environment variables aren't set in production

2. Potential SSRF Attack:

  • No URL domain validation allows requests to internal services
  • Recommendation: Whitelist allowed domains (e.g., meetup.com, luma.com)

📊 Test Coverage Issues

1. Complete Test Absence (backend/Tests/backendTests/backendTests.swift):

  • All tests are commented out
  • Critical: No validation for the core scraping functionality
  • Missing tests for error handling, URL validation, data parsing

2. Missing Integration Tests:

  • No tests for Firecrawl service integration
  • No tests for ScrapingController endpoints

🔧 Code Quality Issues

1. Error Handling Edge Cases:

  • Location defaults to (0.0, 0.0) when coordinates missing (ScrapingController.swift:43-44)
  • No validation for required fields beyond basic guard clauses
  • Date parsing only supports ISO8601, no fallback formats

2. Performance Considerations:

  • Firecrawl polling without exponential backoff (FirecrawlService.swift:51-58)
  • No timeout configuration for HTTP requests
  • Missing response caching for repeated requests

3. Code Consistency:

  • Mix of error message languages (some Portuguese remnants in comments)
  • Inconsistent error types between different controllers

🎯 Positive Architecture Decisions

1. Service Layer Design:

  • ✅ Protocol-based FirecrawlServiceProtocol enables testing
  • ✅ Clean separation between scraping logic and HTTP handling
  • ✅ Proper dependency injection in ScrapingController

2. Data Modeling:

  • ✅ Well-structured MeetupEvent with nested types
  • ✅ Comprehensive Firecrawl schema definition
  • ✅ Proper Codable conformance throughout

3. Deployment:

  • ✅ Production-ready Dockerfile with security best practices
  • ✅ Automated deployment pipeline
  • ✅ Environment-based configuration

📋 Immediate Action Items

Critical (Must Fix Before Merge):

  1. Fix macOS platform versions in both Package.swift files
  2. Add URL validation to only allow HTTPS URLs from trusted domains
  3. Implement basic tests for ScrapingController and error handling

High Priority:

  1. Add environment variable validation (fail fast if missing in production)
  2. Implement request timeout and size limits
  3. Add exponential backoff for Firecrawl polling

Medium Priority:

  1. Add comprehensive API documentation
  2. Implement rate limiting for scraping endpoints
  3. Add structured logging throughout backend
  4. Consider adding health check endpoints

🎯 Next Steps Alignment

The PR successfully achieves its stated goals:

  • ✅ CocoaHeadsCore shared package created and working
  • ✅ SwiftSoup moved to backend with modern Firecrawl replacement
  • 🔄 Ready for app migration to backend APIs (next phase)
  • 🔄 Infrastructure ready for automation (next phase)

This provides a solid foundation for migrating Chapters, Raffles, Q&A, and CloudKit dependencies to the backend.


@loloop loloop changed the title WIP: Adds Vapor backend to project Adds Vapor backend to project Dec 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants