[WIP] Improve Execution Serialization #51
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces enhancements to execution tracking and output normalization in the retrack engine. The main changes add support for node aliases, improve serialization of execution data, and introduce a utility for DataFrame normalization. These updates make execution outputs more readable and structured, and provide better context through node aliases.
Execution Tracking and Serialization Improvements:
aliasesattribute to theExecutionclass, along with methods to set and store node aliases during execution. This allows each node's output to be associated with a human-readable alias.to_normalized_dictmethod inExecution, which serializes inputs, outputs, child executions, and results into a normalized dictionary format for easier downstream processing and inspection.Node Alias Support:
alias()method to theBaseNodeclass, which returns a node's alias if available, falling back to its name. This ensures consistent alias retrieval for execution tracking.Utility Functions:
to_normalized_dictfunction inretrack/utils/transformers.pyto convert DataFrame columns into a list of dictionaries with names and values, supporting the new serialization logic.New Normalized JSON Example:
{ "inputs": [ { "name": "a", "values": [ "1" ] }, { "name": "b", "values": [ "2" ] }, { "name": "context", "values": [ null ] }, { "name": "c", "values": [ "3" ] }, { "name": "d", "values": [ "4" ] } ], "outputs": [ { "node_id": "c1ec8f0109c209bb", "alias": "Input", "values": [ "1" ] }, { "node_id": "fcc373ce66014558", "alias": "Input", "values": [ "2" ] }, { "node_id": "1b4b55ed6d5c9d46", "alias": "Math", "values": [ 3.0 ] } ], "executions": [], "results": [ { "name": "output", "values": [ 3.0 ] }, { "name": "message", "values": [ null ] } ] }