Skip to content

Conversation

@googs1025
Copy link
Member

What type of PR is this?

/kind feature

What this PR does / why we need it:

Introduce a new intra-flow dispatch policy named "DeadlinePriority" that
schedules requests based on deadline urgency, computed as EnqueueTime +
EffectiveTTL. Requests without a valid TTL are treated as lowest priority
and ordered by FCFS for fairness.

Which issue(s) this PR fixes:

Fixes #1913

Does this PR introduce a user-facing change?:

add DeadlinePriority plugin in intra-flow dispatch policy

@k8s-ci-robot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/feature Categorizes issue or PR as related to a new feature. labels Dec 6, 2025
@k8s-ci-robot k8s-ci-robot requested a review from liu-cong December 6, 2025 02:59
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: googs1025
Once this PR has been reviewed and has the lgtm label, please assign ahg-g for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested a review from shmuelk December 6, 2025 02:59
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 6, 2025
@netlify
Copy link

netlify bot commented Dec 6, 2025

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit a26cf47
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/69339bf632fd5c00088033dc
😎 Deploy Preview https://deploy-preview-1960--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@googs1025
Copy link
Member Author

@LukeAVanDrie
sorry to bother.
This is currently a draft, but could you please take a look and check if the implementation is reasonable? I will then proceed with the next step of writing test cases.

return DeadlinePriorityPolicyName
}

// RequiredQueueCapabilities returns an empty slice, indicating that this policy can operate with any queue.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: likely copy-paste error

@LukeAVanDrie
Copy link
Contributor

LukeAVanDrie commented Dec 6, 2025

Initial pass LGTM, but I will take a deeper look first thing Monday morning. Thanks for contributing!

I want to double check how I default EffectiveTTL if undefined.

Regarding QueueCapability pairings, this will be validated automatically by the Flow Registry. We guarantee the paired queue / policy are compatible, so no need to document this in your go-docs. The RequiredCapabilities function is sufficient.

As you may have noticed, pretty much every IntraFlow impl will just return PeekHead. When I align this with the actual EPP plugin model, I will probably make the official plugin the comparator, not the policy impl (which is just imperative code boilerplate). I can handle migrating your policy then.

There should be a README.md which gives a (slightly outdated) policy overview. It also described how to get auto-test coverage for the black box contracts that the orchestrating engine relies upon. Basically, you'll get some free test coverage simply by registering your policy.

@LukeAVanDrie
Copy link
Contributor

LukeAVanDrie commented Dec 6, 2025

Nit: An alternative policy name here may be Earliest Deadline First (EDF). This is the conventional name and a bit more expressive. If GAIE ever supports TTFT latency targets we can use the strictest deadline (TTL vs target, if defined) as the deadline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Implement Deadline-Based Priority IntraFlow Dispatch Policy

3 participants