Skip to content

Conversation

@Aydin-ab
Copy link
Contributor

@Aydin-ab Aydin-ab commented Sep 9, 2025

Why are these changes needed?

Adding new examples for gpt-oss, later released as an anyscale templates as well

Note:

Using our latest version ray==2.49.2 + vllm 0.10.2 breaks but this seems already fixed in nightly so we can change that requirement once the next release is out (mid october)

Also, our current ray-llm image is using vllm 0.10.0 so we have to build a new image with vllm>=0.10.1 this makes the anyscale service launch pretty long (~5min)

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@Aydin-ab Aydin-ab requested review from a team as code owners September 9, 2025 23:49
@Aydin-ab Aydin-ab marked this pull request as draft September 9, 2025 23:49
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This PR adds a new example for deploying gpt-oss models. The changes are mostly good, but there are a few issues. There's a critical copy-paste error in the Anyscale service configuration for gpt-oss-120b in the example notebook and README. There are also some inconsistencies in the documentation links, Docker image versions, and configuration values in the examples. The Dockerfile could also be improved for better practices. I've provided detailed comments and suggestions to address these points.

Copy link
Contributor

@kunling-anyscale kunling-anyscale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@Aydin-ab Aydin-ab force-pushed the add-gpt-oss-example branch 2 times, most recently from 37d016d to 2c574a5 Compare September 10, 2025 07:59
Copy link
Contributor

@kunling-anyscale kunling-anyscale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add few suggestions and comments

@Aydin-ab
Copy link
Contributor Author

i notice the ray-llm nightly image is using vllm==0.10.2 so we can probably start the engineering review in the next release

@Aydin-ab Aydin-ab marked this pull request as ready for review October 2, 2025 23:59
@Aydin-ab
Copy link
Contributor Author

Aydin-ab commented Oct 3, 2025

i notice the ray-llm nightly image is using vllm==0.10.2 so we can probably start the engineering review in the next release

for context using our latest version ray==2.49.2 + vllm 0.10.2 breaks but this seems already fixed in nightly so we can change that requirement once the next release is out (mid october)

meanwhile we can continue and publish with this version of ray==2.49.0, vllm==0.10.1

@ray-gardener ray-gardener bot added serve Ray Serve Related Issue docs An issue or change related to documentation release-test release test llm labels Oct 3, 2025
Aydin Abiar added 4 commits October 3, 2025 13:57
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
@Aydin-ab Aydin-ab force-pushed the add-gpt-oss-example branch from 695fdda to b25fe07 Compare October 3, 2025 20:57
cursor[bot]

This comment was marked as outdated.

Aydin Abiar added 2 commits October 3, 2025 14:01
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
cursor[bot]

This comment was marked as outdated.

Signed-off-by: Aydin Abiar <aydin@anyscale.com>
@angelinalg angelinalg added the go add ONLY when ready to merge, run all tests label Oct 3, 2025
Aydin Abiar added 3 commits October 3, 2025 19:48
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Aydin-ab and others added 2 commits October 6, 2025 17:39
Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com>
Signed-off-by: Aydin Abiar <62435714+Aydin-ab@users.noreply.github.com>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
@angelinalg angelinalg merged commit fa0dc39 into ray-project:master Oct 7, 2025
6 checks passed
liulehui pushed a commit to liulehui/ray that referenced this pull request Oct 9, 2025
joshkodi pushed a commit to joshkodi/ray that referenced this pull request Oct 13, 2025
Signed-off-by: Josh Kodi <joshkodi@gmail.com>
justinyeh1995 pushed a commit to justinyeh1995/ray that referenced this pull request Oct 20, 2025
xinyuangui2 pushed a commit to xinyuangui2/ray that referenced this pull request Oct 22, 2025
Signed-off-by: xgui <xgui@anyscale.com>
landscapepainter pushed a commit to landscapepainter/ray that referenced this pull request Nov 17, 2025
Aydin-ab added a commit to Aydin-ab/ray-aydin that referenced this pull request Nov 19, 2025
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Future-Outlier pushed a commit to Future-Outlier/ray that referenced this pull request Dec 7, 2025
Signed-off-by: Future-Outlier <eric901201@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs An issue or change related to documentation go add ONLY when ready to merge, run all tests llm release-test release test serve Ray Serve Related Issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants