Skip to content

Conversation

@arnoldwakim
Copy link
Contributor

Rationale for this change

StreamChunksFromReader previously did not observe context cancellation. As a result, if a client disconnected early, the reader could continue producing data indefinitely, potentially blocking on channel sends, leaking RecordBatch objects, leaking the reader, and consuming unbounded memory and CPU (this observation triggered this PR).

This fix ensures that data streaming promptly stops when the client disconnects.

What changes are included in this PR?

  • StreamChunksFromReader now accepts a context.Context.
  • Tiny change was made to DoGet, to ensure it continues to work with the context-aware StreamChunksFromReader.

Are these changes tested?

Are there any user-facing changes?

  • StreamChunksFromReader now accepts a context.Context.

@arnoldwakim arnoldwakim force-pushed the fix/flight-StreamChunksFromReader-cancellation-aware branch from 8f81717 to 34ef151 Compare December 20, 2025 10:27
@arnoldwakim arnoldwakim force-pushed the fix/flight-StreamChunksFromReader-cancellation-aware branch from 34ef151 to 5882ee7 Compare December 20, 2025 21:24
Copy link
Member

@zeroshade zeroshade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reasonable way to add a test for this?

@arnoldwakim
Copy link
Contributor Author

arnoldwakim commented Dec 24, 2025

I have used a callbackRecordReader reader to test that the new behaviour is okay.
Kindly let me know if this is what you expected.

@arnoldwakim arnoldwakim force-pushed the fix/flight-StreamChunksFromReader-cancellation-aware branch from 1a96d8d to 4110b63 Compare December 24, 2025 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants