-
-
Notifications
You must be signed in to change notification settings - Fork 329
Enable retry for POST requests in streaming error handling #523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -88,6 +88,18 @@ | |
| end | ||
| end.to raise_error(expected_error_for(provider)) | ||
| end | ||
|
|
||
| it 'retries the request' do | ||
|
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Inspired by the test that @AlexVPopov suggested in the issue #417 |
||
| stub_error_response(provider, :chunk) | ||
|
|
||
| expect do | ||
| chat.ask('Count from 1 to 3') do |chunk| | ||
| chunks << chunk | ||
| end | ||
| end.to raise_error(expected_error_for(provider)) | ||
|
|
||
| expect(WebMock).to have_requested(:post, expected_url_for(provider)).times(3) | ||
| end | ||
| end | ||
|
|
||
| describe 'Faraday version 2' do # rubocop:disable RSpec/NestedGroups | ||
|
|
@@ -124,6 +136,18 @@ | |
| end | ||
| end.to raise_error(expected_error_for(provider)) | ||
| end | ||
|
|
||
| it 'retries the request' do | ||
| stub_error_response(provider, :chunk) | ||
|
|
||
| expect do | ||
| chat.ask('Count from 1 to 3') do |chunk| | ||
| chunks << chunk | ||
| end | ||
| end.to raise_error(expected_error_for(provider)) | ||
|
|
||
| expect(WebMock).to have_requested(:post, expected_url_for(provider)).times(3) | ||
| end | ||
| end | ||
| end | ||
| end | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -28,10 +28,10 @@ | |
| config.vertexai_location = ENV.fetch('GOOGLE_CLOUD_LOCATION', 'us-central1') | ||
|
|
||
| config.request_timeout = 240 | ||
| config.max_retries = 10 | ||
| config.retry_interval = 1 | ||
| config.retry_backoff_factor = 3 | ||
| config.retry_interval_randomness = 0.5 | ||
| config.max_retries = 2 | ||
|
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Once retry is enabled the tests at spec/ruby_llm/chat_streaming_spec.rb:57 starts to wait between retries and with the current settings it takes a lot of time so I adjusted to something more adequate for test environments. One question is, should the faraday-retry middleware actually wait in the test environment or it could be set to simply skip the waiting and retry right away? |
||
| config.retry_interval = 0.01 | ||
| config.retry_backoff_factor = 0.1 | ||
| config.retry_interval_randomness = 0.01 | ||
|
|
||
| config.model_registry_class = 'Model' | ||
| end | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've done this since those tests running with retry enable were changing the VCR recorded requests and I wasn't sure if it was supposed to happen, need some help here =)