Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 46 additions & 20 deletions .github/actions/sync-files/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,6 @@ inputs:
description: 'Comma-separated list of filenames to exclude'
required: false
default: 'README,CHANGELOG,.gitignore,.gitkeep'
sanitize_script:
description: 'Optional script path to run after sync (for sanitization/post-processing)'
required: false
default: ''
outputs:
added:
description: 'Number of files added'
Expand All @@ -46,42 +42,64 @@ runs:
shell: bash
run: |
# For config type, handle special case:
# 1. Copy config.yaml.example to temp location as config.yaml
# 2. Run sanitization on the renamed file
# 3. Use the sanitized file as source for sync

# Copy config.yaml.example to temp location as config.yaml and sanitize

if [[ -f "${{ inputs.source_path }}" ]]; then
# Create temp directory
TEMP_DIR=$(mktemp -d)
echo "TEMP_CONFIG_DIR=$TEMP_DIR" >> $GITHUB_ENV

# Copy and rename config.yaml.example to config.yaml
cp "${{ inputs.source_path }}" "$TEMP_DIR/config.yaml"

# Run sanitization if script is provided
if [[ -n "${{ inputs.sanitize_script }}" ]] && [[ -f "${{ inputs.sanitize_script }}" ]]; then
echo "Sanitizing config file..."
bash "${{ inputs.sanitize_script }}" "$TEMP_DIR/config.yaml"
fi


# Sanitize: replace RPC URLs with placeholders and remove dev section
bash .github/scripts/sanitize-config.sh "$TEMP_DIR/config.yaml"

# Update source path for sync
echo "CONFIG_SOURCE=$TEMP_DIR/config.yaml" >> $GITHUB_ENV
else
echo "Config source file not found: ${{ inputs.source_path }}"
exit 1
fi


- name: Prepare docker-compose file
if: inputs.type == 'docker_compose'
shell: bash
run: |
# For docker_compose type, handle special case:
# Copy and sanitize (remove alloy service and volumes)

if [[ -f "${{ inputs.source_path }}" ]]; then
# Create temp directory
TEMP_DIR=$(mktemp -d)
echo "TEMP_DOCKER_COMPOSE_DIR=$TEMP_DIR" >> $GITHUB_ENV

# Copy docker-compose.yaml
cp "${{ inputs.source_path }}" "$TEMP_DIR/docker-compose.yaml"

# Sanitize: remove alloy service and volumes section using yq
bash .github/scripts/sanitize-docker-compose.sh "$TEMP_DIR/docker-compose.yaml"

# Update source path for sync
echo "DOCKER_COMPOSE_SOURCE=$TEMP_DIR/docker-compose.yaml" >> $GITHUB_ENV
else
echo "Docker compose source file not found: ${{ inputs.source_path }}"
exit 1
fi

Comment on lines +65 to +89
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Update stale comment about docker-compose sanitization implementation

The docker-compose prep step calls .github/scripts/sanitize-docker-compose.sh, which uses sed, but the comment still says “using yq”.

You can align the comment with the implementation:

-        # Sanitize: remove alloy service and volumes section using yq
+        # Sanitize: remove alloy service and volumes section using the sanitize-docker-compose.sh script
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Prepare docker-compose file
if: inputs.type == 'docker_compose'
shell: bash
run: |
# For docker_compose type, handle special case:
# Copy and sanitize (remove alloy service and volumes)
if [[ -f "${{ inputs.source_path }}" ]]; then
# Create temp directory
TEMP_DIR=$(mktemp -d)
echo "TEMP_DOCKER_COMPOSE_DIR=$TEMP_DIR" >> $GITHUB_ENV
# Copy docker-compose.yaml
cp "${{ inputs.source_path }}" "$TEMP_DIR/docker-compose.yaml"
# Sanitize: remove alloy service and volumes section using yq
bash .github/scripts/sanitize-docker-compose.sh "$TEMP_DIR/docker-compose.yaml"
# Update source path for sync
echo "DOCKER_COMPOSE_SOURCE=$TEMP_DIR/docker-compose.yaml" >> $GITHUB_ENV
else
echo "Docker compose source file not found: ${{ inputs.source_path }}"
exit 1
fi
- name: Prepare docker-compose file
if: inputs.type == 'docker_compose'
shell: bash
run: |
# For docker_compose type, handle special case:
# Copy and sanitize (remove alloy service and volumes)
if [[ -f "${{ inputs.source_path }}" ]]; then
# Create temp directory
TEMP_DIR=$(mktemp -d)
echo "TEMP_DOCKER_COMPOSE_DIR=$TEMP_DIR" >> $GITHUB_ENV
# Copy docker-compose.yaml
cp "${{ inputs.source_path }}" "$TEMP_DIR/docker-compose.yaml"
# Sanitize: remove alloy service and volumes section using the sanitize-docker-compose.sh script
bash .github/scripts/sanitize-docker-compose.sh "$TEMP_DIR/docker-compose.yaml"
# Update source path for sync
echo "DOCKER_COMPOSE_SOURCE=$TEMP_DIR/docker-compose.yaml" >> $GITHUB_ENV
else
echo "Docker compose source file not found: ${{ inputs.source_path }}"
exit 1
fi
🤖 Prompt for AI Agents
.github/actions/sync-files/action.yml around lines 62 to 86: the block preparing
the docker-compose file claims it "Sanitize: remove alloy service and volumes
section using yq" but the actual script
.github/scripts/sanitize-docker-compose.sh uses sed; update the comment to
correctly state that the sanitize script uses sed (or a shell stream-edit)
instead of yq, keeping the rest of the logic unchanged so the comment accurately
reflects the implementation.

- name: Sync files
id: sync
shell: bash
run: |
# Use prepared config source if it's a config type, otherwise use original source
# Use prepared source for config/docker_compose types, otherwise use original source
if [[ "${{ inputs.type }}" == "config" ]] && [[ -n "$CONFIG_SOURCE" ]]; then
SOURCE_PATH="$CONFIG_SOURCE"
elif [[ "${{ inputs.type }}" == "docker_compose" ]] && [[ -n "$DOCKER_COMPOSE_SOURCE" ]]; then
SOURCE_PATH="$DOCKER_COMPOSE_SOURCE"
else
SOURCE_PATH="${{ inputs.source_path }}"
fi

${{ github.action_path }}/sync.sh \
"${{ inputs.type }}" \
"${{ inputs.title }}" \
Expand All @@ -97,7 +115,15 @@ runs:
if [[ -n "$TEMP_CONFIG_DIR" ]] && [[ -d "$TEMP_CONFIG_DIR" ]]; then
rm -rf "$TEMP_CONFIG_DIR"
fi


- name: Cleanup docker-compose temp directory
if: inputs.type == 'docker_compose' && always()
shell: bash
run: |
if [[ -n "$TEMP_DOCKER_COMPOSE_DIR" ]] && [[ -d "$TEMP_DOCKER_COMPOSE_DIR" ]]; then
rm -rf "$TEMP_DOCKER_COMPOSE_DIR"
fi

- name: Upload artifacts
uses: actions/upload-artifact@v4
if: always()
Expand Down
11 changes: 10 additions & 1 deletion .github/scripts/doc-generator.sh
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,16 @@ run_doc_generation() {
failed=true
fi
echo "::endgroup::"


echo "::group::Running node-update-docker-compose"
if npm run node-update-docker-compose; then
echo "✅ Updated docker-compose in setup guide"
else
echo "❌ node-update-docker-compose failed"
failed=true
fi
echo "::endgroup::"

echo "::group::Running node-generate-api-docs"
if npm run node-generate-api-docs; then
echo "✅ Generated API documentation"
Expand Down
119 changes: 0 additions & 119 deletions .github/scripts/sanitize-config.py

This file was deleted.

25 changes: 11 additions & 14 deletions .github/scripts/sanitize-config.sh
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
#!/bin/bash
set -e

# Sanitize GenLayer config file for documentation
# Sanitize config.yaml for documentation
# Replaces RPC URLs with placeholders and removes dev section
# Usage: sanitize-config.sh <config_file>

CONFIG_FILE="$1"
Expand All @@ -18,17 +19,13 @@ fi

echo "Sanitizing config file: $CONFIG_FILE"

# Replace URLs with TODO placeholders (only on non-commented lines; preserve indent)
sed -i.bak -E '/^[[:space:]]*#/! s|^([[:space:]]*)zksyncurl:[[:space:]]*".*"|\1zksyncurl: "TODO: Set your GenLayer Chain ZKSync HTTP RPC URL here"|' "$CONFIG_FILE"
sed -i.bak -E '/^[[:space:]]*#/! s|^([[:space:]]*)zksyncwebsocketurl:[[:space:]]*".*"|\1zksyncwebsocketurl: "TODO: Set your GenLayer Chain ZKSync WebSocket RPC URL here"|' "$CONFIG_FILE"
rm -f "${CONFIG_FILE}.bak"
# Use yq to:
# 1. Replace RPC URLs with TODO placeholders
# 2. Delete node.dev section
yq -i '
.rollup.genlayerchainrpcurl = "TODO: Set your GenLayer Chain ZKSync HTTP RPC URL here" |
.rollup.genlayerchainwebsocketurl = "TODO: Set your GenLayer Chain ZKSync WebSocket RPC URL here" |
del(.node.dev)
' "$CONFIG_FILE"

# Remove node.dev sections using Python script
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [[ -f "$SCRIPT_DIR/sanitize-config.py" ]]; then
python3 "$SCRIPT_DIR/sanitize-config.py" "$CONFIG_FILE"
else
echo "Warning: sanitize-config.py not found, skipping Python sanitization"
fi

echo "Config sanitization completed"
echo "Config sanitization completed"
29 changes: 29 additions & 0 deletions .github/scripts/sanitize-docker-compose.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why delete the alloy configuration? I understand that we want telemetry from all users, I wouldn't remove it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it is not shown in the validator guide, here we only store what is shown there. The day we start showing it, we add it, or we remove the removal

Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/bin/bash
set -e

# Sanitize docker-compose.yaml for documentation
# Removes the alloy service and volumes section
# Usage: sanitize-docker-compose.sh <docker_compose_file>

DOCKER_COMPOSE_FILE="$1"

if [[ -z "$DOCKER_COMPOSE_FILE" ]]; then
echo "Usage: $0 <docker_compose_file>" >&2
exit 1
fi

if [[ ! -f "$DOCKER_COMPOSE_FILE" ]]; then
echo "File not found: $DOCKER_COMPOSE_FILE" >&2
exit 1
fi

echo "Sanitizing docker-compose file: $DOCKER_COMPOSE_FILE"

# Use yq to remove alloy service and volumes section
yq -i 'del(.services.alloy) | del(.volumes)' "$DOCKER_COMPOSE_FILE"

# Remove leftover comments about alloy (yq preserves them)
sed -i '/# Grafana Alloy/,/^[^ #]/{ /^[^ #]/!d }' "$DOCKER_COMPOSE_FILE"
sed -i '/^$/N;/^\n$/d' "$DOCKER_COMPOSE_FILE"

echo "Docker-compose sanitization completed"
31 changes: 20 additions & 11 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ This workflow automatically synchronizes documentation from the `genlayerlabs/ge
### What it does

1. **Prepare**: Detects version from input or automatically finds latest tag
2. **Sync Files** (parallel matrix strategy, 5 sync types):
2. **Sync Files** (parallel matrix strategy, 6 sync types):
- **Changelog files** → `content/validators/changelog/`
- **Config file** → `content/validators/config.yaml` (with sanitization)
- **Config file** → `content/validators/config.yaml`
- **Docker Compose file** → `content/validators/docker-compose.yaml`
- **API gen method docs** → `pages/api-references/genlayer-node/gen/` (filtered by regex)
- **API debug method docs** → `pages/api-references/genlayer-node/debug/` (filtered by regex)
- **API ops method docs** → `pages/api-references/genlayer-node/ops/`
Expand Down Expand Up @@ -113,6 +114,8 @@ docs/
configs/
└── node/
└── config.yaml.example # Will be copied to content/validators/config.yaml
release/
└── docker-compose.yaml # Will be copied to content/validators/docker-compose.yaml
```

### Customizing Paths and Filtering
Expand Down Expand Up @@ -141,7 +144,7 @@ The workflow uses 7 main jobs with the following dependency chain:
```
prepare (version detection)
sync-files (matrix: 5 parallel jobs)
sync-files (matrix: 6 parallel jobs)
aggregate-results (merges artifacts)
Expand All @@ -167,16 +170,21 @@ The workflow uses composite actions for code reusability:
- `.github/scripts/sync-artifact-files.sh` - Applies synced files to repository with rsync --delete
- `.github/scripts/aggregate-reports.sh` - Aggregates sync metrics from parallel jobs
- `.github/scripts/git-utils.sh` - Branch creation, commit, and push operations
- `.github/scripts/sanitize-config.sh` - Sanitizes config files (URLs and dev sections)
- `.github/scripts/sanitize-config.py` - Python script to remove node.dev sections
- `.github/scripts/version-utils.sh` - Version detection and validation
- `.github/scripts/doc-generator.sh` - Wrapper for npm documentation generation
- `.github/scripts/sanitize-config.sh` - Sanitizes config file (replaces URLs, removes dev section)
- `.github/scripts/sanitize-docker-compose.sh` - Removes alloy service and volumes from docker-compose

### Config File Sanitization
The config sync process includes automatic sanitization:
1. **URL Replacement**: Real URLs replaced with TODO placeholders
2. **Dev Section Removal**: `node.dev` sections stripped using Python script
3. **Comparison**: Only sanitized content is compared to detect actual changes
### Config Sanitization
The config sync process includes automatic sanitization using `yq`:
- **URL Replacement**: RPC URLs replaced with TODO placeholders
- **Dev Section Removal**: `node.dev` section is removed

### Docker Compose Sanitization
The docker-compose sync process includes automatic sanitization using `yq` and `sed`:
- **Alloy Service Removal**: The `alloy` monitoring service is removed
- **Volumes Removal**: The `volumes` section is removed
- **Comment Cleanup**: Alloy-related comments are removed

### Branch Naming Convention
Sync branches follow the pattern: `docs/node/{version}`
Expand Down Expand Up @@ -228,5 +236,6 @@ The summary job generates a comprehensive report in the GitHub Actions UI:
After syncing files, the workflow runs these npm scripts:
- `npm run node-generate-changelog` - Generates changelog page from synced files
- `npm run node-update-setup-guide` - Updates setup guide with version info
- `npm run node-update-config` - Processes configuration documentation
- `npm run node-update-config` - Updates config.yaml example in setup guide
- `npm run node-update-docker-compose` - Updates docker-compose.yaml example in setup guide
- `npm run node-generate-api-docs` - Generates API reference pages
Loading