Graph-Python-scripts

Scripts and tools for analyzing SingularityNET Ambassador Program meeting summaries, generating graph visualizations, and producing interactive analysis reports.

Overview

This repository provides a comprehensive toolkit for analyzing meeting data from the SingularityNET Ambassador Program. It includes:

• Scripts for fetching meeting summaries and generating graph visualizations
• Analysis tools for computing graph metrics (degree, centrality, clustering, connected components)
• Interactive dashboard with network visualizations and community review system
• Automated workflows for generating reports and deploying to GitHub Pages

All scripts read from a shared public JSON data source and produce various outputs including images, GEXF files for Gephi, Markdown reports, and an interactive HTML dashboard.

Features

• Graph Visualization: Generate network graphs from meeting participation data
• Interactive Dashboard: Web-based dashboard with network visualizations, filtering, and exploration
• Multiple Analysis Methods: Degree analysis, centrality metrics, clustering, and connected components
• Community Review System: Submit and view reviews of analysis results with trust scoring
• Automated Reporting: GitHub Actions workflows automatically generate and deploy reports
• GEXF Export: Export graphs for analysis in Gephi

Quick Start

Setup

1. Create a virtual environment and install dependencies:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -r requirements.txt

Running Scripts

See the sections below for specific scripts organized by category.

Interactive Dashboard

The repository includes an interactive HTML dashboard hosted on GitHub Pages that provides:

• Unified Analysis View: All analysis methods in a single tabbed interface
• Interactive Network Visualization: Co-attendance graph with node filtering and exploration
• Community Reviews: Submit reviews directly from the dashboard
• Audit Statistics: Trust scores and review summaries for each analysis method

Access the Dashboard: https://singularitynet-archive.github.io/Graph-Python-scripts/

The dashboard is automatically generated and deployed when changes are made to analysis scripts or configurations.

Dashboard Features

• Network Visualization: Interactive co-attendance graph built with vis-network
  • Click nodes to filter connections
  • Hover for details
  • Adjustable physics and layout
• Multiple Analysis Tabs:
  • Summary statistics
  • Co-attendance degree analysis
  • Field degree analysis
  • Path structure analysis
  • Centrality metrics
  • Clustering analysis
  • Connected components
  • Audit tab with review statistics
• Review System: Submit reviews with ratings (Correct, Needs Review, Incorrect) and comments

Analysis Scripts

Scripts are organized by category:

Basic Graph Scripts

Simple graph generation and data export utilities.

• Nodes-Edges.py - Build a directed graph from a single meeting entry and save as image

  python Scripts/Nodes-Edges.py

  Output: graph.png

• Nodes-Edges2.py - Build a combined directed graph across all meetings and save as image

  python Scripts/Nodes-Edges2.py

  Output: graph2.png

• GEXF-export.py - Build a comprehensive directed graph and export to GEXF format for Gephi

  python Scripts/GEXF-export.py

  Output: Scripts/all_workgroups_graph_sanitized.gexf

• Import-JSON.py - Generate a text summary of workgroups and meetings

  python Scripts/Import-JSON.py

  Output: Scripts/workgroup_meetings_summary.txt

Graph Analysis Scripts

Advanced analysis tools for computing graph metrics and generating detailed reports.

• Degree Analysis - Co-attendance degree analysis

  python "Graph Analysis/Degree_Analysis/degree_analysis_to_md.py"

  Output: Graph Analysis/Degree_Analysis/degree_analysis_report.md

• Path Analysis - JSON path structure analysis

  python "Graph Analysis/Path_Analysis/path_analysis_report.py"

  Output: reports/path_analysis_report.md

• Centrality Analysis - Field centrality (co-occurrence) metrics

  python "Graph Analysis/Path_Analysis/Centrality_Analysis/json_centrality_analysis.py"

  Output: reports/centrality_analysis_report.md

• Clustering Analysis - Field co-occurrence graph clustering

  python "Graph Analysis/Clustering_Analysis/json_clustering_analysis.py"

  Output: reports/clustering_analysis_report.md

• Connected Components - Connected components analysis

  python "Graph Analysis/Connected_Components/json_connected_components.py"

  Output: reports/connected_components_report.md

Unified Analysis

The main dashboard generator that combines all analyses into a single interactive HTML report.

• unified_analysis.py - Generate comprehensive unified analysis report with HTML dashboard

  python "Graph Analysis/unified_analysis.py" --output reports/unified_analysis_report.md
  python "Graph Analysis/unified_analysis.py" --html --html-output docs/index.html

  Output:
  • reports/unified_analysis_report.md (Markdown report)
  • docs/index.html (Interactive HTML dashboard)

Community Review System

This repository includes a community review system to validate graph analysis results. Reviews can be submitted directly through the interactive dashboard.

How It Works

1. Submit a Review:

   • Visit the dashboard and navigate to any analysis tab
   • Scroll to the review form at the bottom
   • Fill out the rating (Correct, Needs Review, or Incorrect) and comments
   • Click "Submit Review"

2. Automatic Processing:

   • Review is saved locally in your browser (localStorage)
   • Review JSON file is automatically added to the reviews/ folder via GitHub API
   • Review appears immediately on the page underneath the form
   • Summary statistics are updated in the Audit tab

3. View Reviews:

   • All reviews are displayed in their respective analysis tabs
   • The Audit tab provides aggregated statistics:
     • Trust scores per analysis method
     • Rating distribution (Correct, Needs Review, Incorrect)
     • All review comments organized by method

Authentication

The first time you submit a review, you'll be prompted for a GitHub Personal Access Token (starts with ghp_). This token is:

• Verified and stored locally in your browser
• Used only to add review files to the reviews/ folder via GitHub API
• Not shared or stored on any server
• Can be revoked anytime from your GitHub settings

To create a token:

1. Go to GitHub Settings > Developer settings > Personal access tokens
2. Generate a new token (classic) with repo scope
3. Copy the token (starts with ghp_) when prompted

Review Data

Reviews are stored as JSON files in the reviews/ directory with the following structure:

{
  "id": "unique-review-id",
  "method": "coattendance",
  "rating": "correct",
  "comment": "Your review comment here",
  "reviewer": "Your Name",
  "suggestions": "Optional suggestions",
  "file": "docs/index.html",
  "timestamp": "2025-01-15T10:00:00.000Z"
}

For more details on submitting reviews, see CONTRIBUTING.md. For technical details about the audit system, see REVIEWING.md.

Data Source

All scripts read from a shared public JSON data source:

https://raw.githubusercontent.com/SingularityNET-Archive/SingularityNET-Archive/refs/heads/main/Data/Snet-Ambassador-Program/Meeting-Summaries/2025/meeting-summaries-array.json

Outputs

Scripts generate various outputs:

• Images: graph.png, graph2.png - Rendered network graphs
• GEXF Files: Scripts/all_workgroups_graph_sanitized.gexf - For import into Gephi
• Markdown Reports: Various reports in reports/ and Graph Analysis/ directories
• HTML Dashboard: docs/index.html - Interactive dashboard (automatically deployed to GitHub Pages)
• Review Files: JSON files in reviews/ directory

Most outputs are generated automatically when scripts run. The HTML dashboard is automatically generated and deployed by GitHub Actions when changes are made.

Repository Structure

├── Scripts/              # Basic graph generation scripts and utilities
│   ├── Nodes-Edges.py   # Single-meeting graph
│   ├── Nodes-Edges2.py  # Multi-meeting graph
│   ├── GEXF-export.py   # Gephi export
│   └── Import-JSON.py    # Workgroup summary
│
├── Graph Analysis/       # Advanced analysis scripts
│   ├── Degree_Analysis/      # Co-attendance degree analysis
│   ├── Path_Analysis/        # JSON path structure analysis
│   ├── Centrality_Analysis/  # Field centrality metrics
│   ├── Clustering_Analysis/  # Graph clustering
│   ├── Connected_Components/ # Connected components
│   └── unified_analysis.py   # Unified dashboard generator
│
├── reports/              # Generated Markdown reports
│
├── docs/                 # HTML dashboard and assets
│   ├── index.html       # Main interactive dashboard
│   ├── script.js        # Dashboard JavaScript
│   └── style.css        # Dashboard styles
│
├── reviews/              # Community review JSON files
│   └── processed/       # Processed review files
│
└── .github/workflows/    # GitHub Actions automation
    ├── pages.yml        # Dashboard generation and deployment
    ├── static.yml       # GitHub Pages deployment
    └── review_audit.yml # Review audit collection

See Scripts/README.md, Graph Analysis/README.md, and reports/README.md for more details about each directory.

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines on:

• Submitting reviews of analysis results
• Contributing code improvements
• Review best practices

Documentation

• CONTRIBUTING.md - How to submit reviews and contribute
• REVIEWING.md - Technical details about the review audit system
• Scripts/README.md - Details about basic graph scripts
• Graph Analysis/README.md - Details about analysis scripts
• reports/README.md - Generated report descriptions

Automated Workflows

The repository uses GitHub Actions for automation:

• Dashboard Generation (.github/workflows/pages.yml): Automatically generates HTML dashboard when analysis scripts or configurations change
• GitHub Pages Deployment (.github/workflows/static.yml): Automatically deploys dashboard to GitHub Pages
• Review Audit Collection (.github/workflows/review_audit.yml): Collects and aggregates reviews nightly

Notes

• Scripts run headlessly and save files to disk
• Generated images can be opened via your OS default viewer
• The dashboard is automatically updated when changes are pushed to the main branch
• Reviews require a GitHub Personal Access Token with repo scope for automatic file submission