Skip to content

Conversation

@ma-efremoff
Copy link
Collaborator

@ma-efremoff ma-efremoff commented Dec 12, 2025

https://nda.ya.ru/t/69XuJhd37Pivkp

## Summary by Sourcery

Introduce a dedicated Flow page with its own routing, navigation entry, and top row, refactor existing Flow navigation tab components into this new page, and extend Flow-related APIs and types to support computations listing and monitoring integrations.

New Features:

  • Add a standalone Flow page with tabs for graph, computations, workers, specs, and monitoring, including lazy-loaded main and top-row components.
  • Provide a computations table view with filtering, sorting, and detailed attribute inspection for Flow computations.
  • Add navigation from pipeline nodes in the map node table to the new Flow page and display a dedicated Flow icon for pipeline nodes.

Bug Fixes:

  • Fix Flow computation metric definitions by correcting memory usage metrics and switching to 10-minute averages for CPU and memory usage.
  • Correct a string formatting utility to accept arbitrary parameter types and safely stringify them.

Enhancements:

  • Refine Flow graph rendering and canvas components to highlight high CPU and memory usage and adjust group sizing and labeling.
  • Extend Flow message rendering to support markdown content, configurable padding, and automatic expansion when a single message is present.
  • Generalize Flow filters to track pipeline path and computation name filters, and update URL mappings and selectors accordingly.
  • Add generic get endpoint hooks and utilities for fetching Flow attributes and wiring monitoring integrations via UIFactory configuration.
  • Update label and graph color themes to support warning and error variants in new Flow views.

@sourcery-ai
Copy link

sourcery-ai bot commented Dec 12, 2025

Reviewer's Guide

Introduces a dedicated Flow page with routing, URL mapping, and navigation integration; refactors Flow graph/layout components into the new page; extends Flow API types and endpoints; adds computations listing and monitoring integration; improves flow messages rendering and graph canvas highlighting for high resource consumption.

Sequence diagram for navigation FlowRedirect to new Flow page

sequenceDiagram
    actor User
    participant Navigation as NavigationContent
    participant FlowRedirect
    participant History as AppBrowserHistory
    participant Router as ReactRouter
    participant FlowPage as FlowPage
    participant Flow as Flow

    User->>Navigation: click pipeline node row
    Navigation->>FlowRedirect: render Tab.FLOW component
    FlowRedirect->>Navigation: select path from navigation store
    FlowRedirect->>History: replace(makeFlowLink(path))
    FlowRedirect->>History: push(makeFlowLink(path))
    History->>Router: update location to /:cluster/flow/...?
    Router->>FlowPage: match route /:cluster/flow
    FlowPage->>Flow: render Flow
    Flow->>Flow: read pipelinePath from flow.filters
    Flow-->>User: show Flow tabs and content for pipeline
Loading

Sequence diagram for Flow messages loading on Flow page

sequenceDiagram
    actor User
    participant Flow as Flow
    participant FlowMessagesLoaded
    participant useFlowExecuteQuery
    participant flowExecuteEndpoint as flowExecute
    participant ytApiV4Id
    participant Backend as FlowService

    User->>Flow: open Flow page
    Flow->>FlowMessagesLoaded: render in header
    FlowMessagesLoaded->>useFlowExecuteQuery: call<'describe-pipeline'>(status_only=true)
    useFlowExecuteQuery->>flowExecuteEndpoint: invoke with ParamsType and BodyType
    flowExecuteEndpoint->>ytApiV4Id: ytApiV4Id.flowExecute(flowExecuteDescribePipeline)
    ytApiV4Id->>Backend: HTTP describe-pipeline(status_only)
    Backend-->>ytApiV4Id: FlowDescribePipelineData(messages)
    ytApiV4Id-->>flowExecuteEndpoint: data
    flowExecuteEndpoint-->>useFlowExecuteQuery: {data}
    useFlowExecuteQuery-->>FlowMessagesLoaded: hook result
    alt error
        FlowMessagesLoaded-->>Flow: render YTErrorInline
    else success
        FlowMessagesLoaded-->>Flow: render FlowMessages(messages)
    end
    Flow-->>User: messages dialog available
Loading

Class diagram for updated Flow types and filters

classDiagram
    class FlowExecuteTypes {
    }
    class FlowDescribeComputationsData {
        +Array~FlowComputation~ computations
    }
    class FlowDescribePipelineData {
        ... existing fields ...
    }

    class FlowExecuteCommand {
        <<type alias>>
    }

    class FlowMessage {
        +FlowNodeStatus level
        +string text
        +unknown yson
        +YTError error
        +string markdown_text
    }

    class FlowComputationMetrics {
        +number cpu_usage_current
        +number cpu_usage_30s
        +number cpu_usage_10m
        +number memory_usage_current
        +number memory_usage_30s
        +number memory_usage_10m
    }

    class FlowPartitionsStats {
        +number count
        +number healthy
        +number degraded
        +number failed
    }

    class FlowComputation {
        +string id
        +string name
        +string class_name
        +boolean highlight_cpu_usage
        +boolean hightlight_memory_usage
        +FlowComputationMetrics metrics
        +FlowPartitionsStats partitions_stats
        +FlowComputationstatus status
    }

    class FlowNodeBase {
        +string id
        +string name
        +FlowNodeStatus status
    }

    class FlowNodeStatus {
        <<union>>
        idle
        running
        warning
        error
        fatal
        maximum
    }

    class FlowComputationstatus {
        <<union>>
        info
        warning
        error
    }

    class FlowFiltersState {
        +string pipelinePath
        +string computationsNameFilter
    }

    class FlowTab {
        <<const object>>
        +GRAPH
        +GRAPH_DATA
        +COMPUTATIONS
        +WORKERS
        +MONITORING
        +STATIC_SPEC
        +DYNAMIC_SPEC
    }

    class FlowTabType {
        <<type alias>>
    }

    class FiltersSlice {
        +updateFlowFilters(payload Partial~FlowFiltersState~)
    }

    class useFlowComputationsNameFilter {
        +string computationsNameFilter
        +setComputationsNameFilter(value string)
    }

    FlowExecuteCommand --> FlowExecuteTypes : keyof
    FlowExecuteTypes --> FlowDescribePipelineData : ResponseType
    FlowExecuteTypes --> FlowDescribeComputationsData : ResponseType

    FlowComputation --> FlowNodeBase : extends
    FlowComputation --> FlowComputationMetrics : has
    FlowComputation --> FlowPartitionsStats : has
    FlowComputation --> FlowComputationstatus : status

    FlowMessage --> FlowNodeStatus : level

    FlowTabType --> FlowTab : keyof
    FlowFiltersState --> FlowTab : used in routing

    FiltersSlice --> FlowFiltersState : manages
    useFlowComputationsNameFilter --> FlowFiltersState : reads/updates
Loading

File-Level Changes

Change Details Files
Create a standalone Flow page with its own routing, top row, URL mapping, and navigation entry, decoupled from Navigation tab rendering.
  • Add Page.FLOW and register it in global routing (ClusterPage, slideout menu, header pages) and TopRowContent with lazy-loaded Flow page components.
  • Introduce FlowPage and FlowPageTopRow containers plus lazy loaders to host the Flow UI outside Navigation.
  • Replace Navigation Flow tab rendering with FlowRedirect that forwards map-node Flow tab to the new Flow page using makeFlowLink.
packages/ui/src/shared/constants/settings.ts
packages/ui/src/ui/pages.ts
packages/ui/src/ui/constants/slideoutMenu.ts
packages/ui/src/ui/containers/ClusterPage/ClusterPage.js
packages/ui/src/ui/containers/AppNavigation/TopRowContent/TopRowContent.tsx
packages/ui/src/ui/pages/navigation/Navigation/ContentViewer/helpers/getComponentByMode.ts
packages/ui/src/ui/pages/flow/FlowPage/FlowPage.tsx
packages/ui/src/ui/pages/flow/FlowPage/FlowPage.scss
packages/ui/src/ui/pages/flow/FlowPageTopRow/FlowPageTopRow.tsx
packages/ui/src/ui/pages/flow/index.tsx
packages/ui/src/ui/pages/flow/lazy.tsx
packages/ui/src/ui/utils/app-url/index.ts
packages/ui/src/ui/store/location.main.ts
Refactor flow filters, URL mapping, and selectors to support pipeline path and computations name filtering instead of in-tab view mode.
  • Redefine FlowFiltersState to store pipelinePath and computationsNameFilter, with a generic updateFlowFilters reducer.
  • Update URL mapping to map Flow page paths and computations name filters via flowParams and flowComputationsParams, removing navigation map-node specific flow mapping.
  • Expose selectors and hooks for pipelinePath and computationsNameFilter (getFlowPipelinePath, getFlowComputationsNameFilter, useFlowComputationsNameFilter).
packages/ui/src/ui/store/reducers/flow/filters.ts
packages/ui/src/ui/store/reducers/flow/url-mapping.ts
packages/ui/src/ui/store/selectors/flow/filters.ts
packages/ui/src/ui/store/reducers/flow/filters.hooks.ts
packages/ui/src/ui/store/reducers/navigation/url-mapping.ts
packages/ui/src/ui/store/location.main.ts
Extend Flow API typing and client to support multiple flow execute commands and a generic get endpoint for attributes.
  • Replace FlowExecuteParams/FlowExecuteData with FlowExecuteTypes mapping that encodes params, body, and response types per command (describe-pipeline, describe-computations).
  • Adapt flowExecute and useFlowExecuteQuery to use FlowExecuteTypes and optional body, and update rum-wrap-api typing accordingly.
  • Introduce FlowDescribeComputationsData and extend FlowComputation/FlowMessage types with fields used by new UI (class_name, metrics 10m, highlight flags, markdown_text).
  • Add generic get endpoint and useGetQuery hook for fetching flow attributes via YTApiId.flowAttributes.
packages/ui/src/shared/yt-types.d.ts
packages/ui/src/ui/store/api/yt/flow/endpoint.ts
packages/ui/src/ui/store/api/yt/flow/index.ts
packages/ui/src/ui/rum/rum-wrap-api.ts
packages/ui/src/shared/constants/yt-api-id.ts
packages/ui/src/ui/store/api/yt/get/endpoint.ts
packages/ui/src/ui/store/api/yt/get/index.ts
Implement new Flow page layout with tabs (graph, graph data, computations, workers, specs, monitoring) and a state header with status, controls, leader address, and messages.
  • Create Flow component with FlowState header (status label, start/pause/stop buttons, leader_controller_address with clipboard, messages) and FlowTabs built from FlowTab and UIFactory monitoring config.
  • Wire FlowContent routing per FlowTab to FlowGraph, FlowComputations, FlowLayout (workers), dynamic/static spec, and monitoring view, with default redirect to graph tab.
  • Add FlowMessagesLoaded using useFlowExecuteQuery('describe-pipeline', status_only) and YTErrorInline to display flow messages and errors, and integrate FlowMonitoring that either renders a custom component or external link based on UIFactory config.
packages/ui/src/ui/pages/flow/Flow/Flow.tsx
packages/ui/src/ui/pages/flow/Flow/Flow.scss
packages/ui/src/ui/UIFactory/index.tsx
packages/ui/src/ui/store/actions/flow/status.ts (indirectly referenced)
Move Flow graph and layout components from navigation tab to the new Flow page and enhance visualisation for resource consumption and message rendering.
  • Relocate FlowGraph, its renderers (Computation, Stream, Sink, groups, renderer), and FlowLayout from navigation paths to /pages/flow/..., updating imports to new relative roots.
  • Change FlowGraphImpl to use typed useFlowExecuteQuery<'describe-pipeline'> and drop inline messages container; messages are now loaded separately in Flow header.
  • Update Computation and ComputationCanvas to use 10-minute CPU/memory metrics, highlight_* flags, and warn-colored text/labels when high consumption; hide labels if block height is too small and support warning color in YTGraphCanvasBlock drawMetaItem/drawInnerText.
  • Extend FlowMessages to accept paddingTop option and show markdown_text via Markdown component; auto-expand single message in dialog; add TextWithHighConsumption helper with tooltip and icon.
  • Increase computation group block min width to 600 to avoid cramped content and adjust FlowGraphRenderer.scss for optional padding-top.
  • Ensure GRAPH_COLORS includes warning and Label supports theme_error for health status mapping.
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowGraph/FlowGraph.tsx
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowGraph/renderers/Computation.tsx
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowGraph/renderers/ComputationCanvas.ts
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowGraph/renderers/FlowGraphRenderer.tsx
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowGraph/renderers/FlowGraphRenderer.scss
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowGraph/renderers/StreamCanvas.ts
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowGraph/renderers/ComputationGroupCanvas.ts
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowGraph/renderers/SinkCanvas.tsx
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowGraph/utils/FlowGroupBlock.ts
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowLayout/FlowLayout.tsx
packages/ui/src/ui/pages/navigation/tabs/Flow/FlowLayout/FlowLayout.scss
packages/ui/src/ui/components/YTGraph/canvas/YTGrapCanvasBlock.ts
packages/ui/src/ui/components/YTGraph/constants/index.ts
packages/ui/src/ui/components/Label/Label.tsx
packages/ui/src/ui/components/Label/Label.scss
packages/ui/src/ui/components/Markdown/Markdown.tsx
Add Flow computations table view with filtering, sorting, and health labeling.
  • Implement FlowComputations with sticky toolbar hosting a debounced name filter, and FlowComputationsTable using DataTableGravity/tanstack for virtualized, sortable table.
  • Fetch computations via useFlowExecuteQuery<'describe-computations'> and filter by name; expose columns for name, class, partitions count, CPU and memory usage (10m), health, and actions (attributes button).
  • Map computation status to Label theme using STATUS_TO_BG_THEME and localize column headers via i18n keysets.
packages/ui/src/ui/pages/flow/Flow/FlowComputations/FlowComputations.tsx
packages/ui/src/ui/pages/flow/Flow/FlowComputations/FlowComputations.scss
packages/ui/src/ui/pages/flow/Flow/FlowComputations/i18n/index.ts
packages/ui/src/ui/pages/flow/Flow/FlowComputations/i18n/en.json
packages/ui/src/ui/pages/flow/Flow/FlowComputations/i18n/ru.json
Integrate Flow awareness into navigation map-node listing and icons, including pipeline detection and linking to Flow page.
  • Add isPipelineNode helper based on pipeline_format_version and use it in getSupportedTabs and MapNodeIcon to show Flow tab and pipeline icon for pipeline nodes.
  • Extend map-node getList attributes to include pipeline_format_version and render map-node names for pipeline nodes with a right-arrow and routed link to Flow page via makeFlowLink.
  • Refine imports for navigation selectors and map-node table to use selectors and URL helpers consistently.
packages/ui/src/ui/utils/navigation/isPipelineNode.ts
packages/ui/src/ui/store/selectors/navigation/navigation.ts
packages/ui/src/ui/components/MapNodeIcon/MapNodeIcon.tsx
packages/ui/src/ui/store/actions/navigation/content/map-node.js
packages/ui/src/ui/pages/navigation/content/MapNode/MapNodesTable.js
packages/ui/src/ui/utils/app-url/index.ts
Minor shared utility and styling improvements needed by the new Flow functionality.
  • Relax formatByParams to accept Record<string, unknown> and convert values via String() instead of toString() constraint.
  • Allow Label theme_error alongside theme_danger in Label.scss/theme type, and add warning color entry to graph constants for high-consumption highlighting.
  • Make Markdown component return null when text is empty and adjust FlowGraphRenderer.scss messages padding modifier.
  • Introduce new SVG icon asset for pipeline nodes and register flowAttributes API id for flow attribute fetching.
packages/ui/src/ui/utils/format.ts
packages/ui/src/ui/components/Label/Label.scss
packages/ui/src/ui/components/Label/Label.tsx
packages/ui/src/ui/components/YTGraph/constants/index.ts
packages/ui/src/ui/components/Markdown/Markdown.tsx
packages/ui/src/ui/pages/flow/Flow/FlowGraph/renderers/FlowGraphRenderer.scss
packages/ui/src/shared/constants/yt-api-id.ts
packages/ui/src/ui/assets/img/svg/icons/node-pipeline.svg

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions
Copy link
Contributor

Storybook is ready.

@github-actions
Copy link
Contributor

Playwright components report is ready.

@github-actions
Copy link
Contributor

Statoscope report is ready.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 12, 2025

E2E-local report is ready.
E2E-remote report is ready.
E2E-screenshots report is ready.

@ma-efremoff ma-efremoff force-pushed the ma-efremoff/flow branch 3 times, most recently from eb7d606 to aa333b3 Compare December 22, 2025 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants