Skip to content

Conversation

@minoskt
Copy link
Collaborator

@minoskt minoskt commented Jul 23, 2025

  • Add support for downsampling the data collected by Monsoon through the --granularity argument.
  • Add support for Parquet output format using the --format parquet argument for improved performance with large datasets.
  • Remove support for deprecated TLS versions 1.0 and 1.1. (Stop supporting deprecated TLS versions #8)
  • Convert time into context-based constants, at constants.py.
  • Better handle mitmproxy errors, e.g., when the certificate is not found.
  • Better handle device power-off to avoid data corruption (Android only for now).
  • Fix issues with GPIO control in latest update of Raspberry Pi OS.
  • This version supports the performance evaluation of Brave Android, reported at Brave for Android continues to outperform other browsers in speed and performance, based on recent tests.

@minoskt minoskt requested a review from stoletheminerals July 23, 2025 08:55
@github-actions
Copy link

[puLL-Merge] - brave/blade@9

Description

This PR updates BLaDE to version 0.3, introducing significant improvements for performance evaluation of mobile devices. Key changes include support for Parquet output format for better performance with large datasets, data downsampling capabilities through granularity control, removal of deprecated TLS versions, better error handling, and improvements to device power management. The update also includes comprehensive documentation updates and code refactoring to use context-based constants.

Possible Issues

  • GPIO Resource Management: The GPIO library changes in gpiolib.py add line.release() calls which may cause issues if multiple processes try to access the same GPIO pins simultaneously
  • ADB Command Escaping: The URL escaping logic in adb_commands.py has been simplified but may still have edge cases with complex URLs containing special characters
  • Parquet Buffer Management: The new Parquet buffering system could potentially lose data if the process crashes before the buffer is flushed
  • Configuration Removal: Removal of wifi and Lighthouse support from device configurations may break existing workflows depending on these features
  • Hard-coded Paths: The certificate path handling in pageload-server.py uses relative paths which could fail in different execution contexts

Security Hotspots

  • TLS Configuration: While the code enforces TLS 1.2+ in pageload-server.py, the certificate validation and SSL context configuration should be reviewed for production use
  • Shell Command Injection: The open_url_on_activity function in adb_commands.py still constructs shell commands with user input, though URL escaping has been improved
  • Certificate Path Handling: The certificate loading logic uses file paths that could be manipulated if not properly validated
  • Process Control: The async process management in async_calls.py could be vulnerable to process injection if process arguments are not properly sanitized

Privacy Hotspots

  • Device Tracking: The addition of pageVisitId in the pageload injection script creates unique identifiers for page visits that could potentially be used for tracking
  • Performance Data Collection: The enhanced performance monitoring collects detailed timing information and device characteristics that could be used for fingerprinting
  • Network Traffic Analysis: The improved network monitoring capabilities could potentially capture sensitive user data if not properly configured
Changes

Changes

CHANGELOG.md & README.md

  • Updated to version 0.3 with comprehensive feature list
  • Enhanced documentation with better formatting and contributor information
  • Added recent blog posts and research publications

src/setup/setup.sh & src/setup/README.md

  • Added fastparquet dependency installation
  • Updated setup instructions with improved security recommendations
  • Removed wifi configuration from device examples

src/tools/libs/constants.py

  • Complete refactor from simple time constants to comprehensive configuration constants
  • Added constants for all major components (Monsoon, ADB, BT, USB, etc.)
  • Organized constants by functionality with clear naming conventions

src/tools/libs/monsoonlib.py

  • Major enhancement with Parquet format support
  • Added data downsampling through granularity parameter
  • Improved error handling and resource management
  • Added context manager support (__enter__/__exit__)
  • Enhanced voltage validation with proper bounds checking

src/tools/libs/automation/adb_commands.py

  • Improved URL handling and escaping in open_url_on_activity
  • Simplified browser-specific functions (removed hard-coded UI interactions)
  • Better proxy configuration with constants
  • Enhanced device unlock sequence

src/tools/control-monsoon.py

  • Added granularity and format parameters
  • Improved argument validation and help text
  • Better error handling for connection issues

src/tools/libs/gpiolib.py

  • Added proper GPIO line resource management with line.release()
  • Updated consumer naming for better process identification

src/tools/pageload-inject.py & pageload-server.py

  • Enhanced injection script with better duplicate prevention
  • Added unique page visit IDs and improved storage management
  • Enforced TLS 1.2+ minimum version
  • Better error handling and CORS support
sequenceDiagram
    participant User
    participant ControlDevice
    participant DeviceLib
    participant MonsoonLib
    participant USBLib
    participant ADBLib
    
    User->>ControlDevice: Start measurement with granularity
    ControlDevice->>DeviceLib: start_measuring(device, path, granularity)
    DeviceLib->>USBLib: Disable USB for measurement
    DeviceLib->>ADBLib: Enable ADB over WiFi (Android)
    DeviceLib->>MonsoonLib: collect_measurements(file, format, granularity)
    
    loop Data Collection
        MonsoonLib->>MonsoonLib: Collect samples batch
        alt Format is CSV
            MonsoonLib->>MonsoonLib: Write directly to CSV
        else Format is Parquet
            MonsoonLib->>MonsoonLib: Buffer samples
            alt Buffer full
                MonsoonLib->>MonsoonLib: Flush to Parquet file
            end
        end
    end
    
    User->>ControlDevice: Stop measurement
    ControlDevice->>DeviceLib: stop_measuring(device)
    DeviceLib->>MonsoonLib: Stop collection & cleanup
    DeviceLib->>USBLib: Re-enable USB
    DeviceLib->>ADBLib: Disable ADB over WiFi
Loading

def get_device_adb_connection_state(device, port=constants.ADB_OVER_WIFI_DEFAULT_PORT):

# list adb devices and check if given device is listed (either identifier or ip:port)
output = os.popen("adb devices").read()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reported by reviewdog 🐶
[semgrep] Starting a process with a shell; seems safe, but may be changed in the future, consider
rewriting without shell


Source: https://semgrep.dev/r/gitlab.bandit.B605


Cc @thypon @kdenhartog

def get_device_adb_connection_state(device, port=constants.ADB_OVER_WIFI_DEFAULT_PORT):

# list adb devices and check if given device is listed (either identifier or ip:port)
output = os.popen("adb devices").read()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reported by reviewdog 🐶
[semgrep] Starting a process with a shell; seems safe, but may be changed in the future, consider
rewriting without shell


Source: https://semgrep.dev/r/gitlab.bandit.B607


Cc @thypon @kdenhartog

def power_off_device(device, connection):

adb_identifier = __get_adb_identifier(device, connection)
os.system(f"adb -s {adb_identifier} reboot -p")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reported by reviewdog 🐶
[semgrep] Starting a process with a shell; seems safe, but may be changed in the future, consider
rewriting without shell


Source: https://semgrep.dev/r/gitlab.bandit.B605


Cc @thypon @kdenhartog

def reboot_device(device, connection):

adb_identifier = __get_adb_identifier(device, connection)
os.system(f"adb -s {adb_identifier} reboot")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reported by reviewdog 🐶
[semgrep] Starting a process with a shell; seems safe, but may be changed in the future, consider
rewriting without shell


Source: https://semgrep.dev/r/gitlab.bandit.B605


Cc @thypon @kdenhartog

script = os.path.join(__location__, "../bt-connect.py")
process = subprocess.Popen(["sudo", script, "--device", bt_mac_address])
time.sleep(constants.FIVE_SECONDS)
process = subprocess.Popen(["sudo", script, bt_mac_address])

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reported by reviewdog 🐶
[semgrep] Python possesses many mechanisms to invoke an external executable. However,
doing so may present a security issue if appropriate care is not taken to
sanitize any user provided or variable input. This plugin test is part of a
family of tests built to check for process spawning and warn appropriately.
Specifically, this test looks for the spawning of a subprocess without the
use of a command shell. This type of subprocess invocation is not
vulnerable to shell injection attacks, but care should still be taken to
ensure validity of input.


Source: https://semgrep.dev/r/gitlab.bandit.B603


Cc @thypon @kdenhartog

Co-Authored-By: Artem Chaikin <10810135+stoletheminerals@users.noreply.github.com>
Co-Authored-By: Ralph Ankele <117092743+ankeleralph@users.noreply.github.com>
@minoskt minoskt merged commit 1e6c793 into main Jul 23, 2025
6 checks passed
@minoskt minoskt deleted the release/v0.3 branch July 23, 2025 09:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants