Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion lib/ruby_llm/connection.rb
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,8 @@ def setup_retry(faraday)
interval_randomness: @config.retry_interval_randomness,
backoff_factor: @config.retry_backoff_factor,
exceptions: retry_exceptions,
retry_statuses: [429, 500, 502, 503, 504, 529]
retry_statuses: [429, 500, 502, 503, 504, 529],
methods: %i[delete get head options patch post put]
}
end

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions spec/ruby_llm/chat_error_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@
RSpec.describe RubyLLM::Chat do
include_context 'with configured RubyLLM'

before do
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've done this since those tests running with retry enable were changing the VCR recorded requests and I wasn't sure if it was supposed to happen, need some help here =)

RubyLLM.configure do |config|
config.max_retries = 0
end
end

describe 'error handling' do
CHAT_MODELS.each do |model_info|
model = model_info[:model]
Expand Down
24 changes: 24 additions & 0 deletions spec/ruby_llm/chat_streaming_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,18 @@
end
end.to raise_error(expected_error_for(provider))
end

it 'retries the request' do
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inspired by the test that @AlexVPopov suggested in the issue #417

stub_error_response(provider, :chunk)

expect do
chat.ask('Count from 1 to 3') do |chunk|
chunks << chunk
end
end.to raise_error(expected_error_for(provider))

expect(WebMock).to have_requested(:post, expected_url_for(provider)).times(3)
end
end

describe 'Faraday version 2' do # rubocop:disable RSpec/NestedGroups
Expand Down Expand Up @@ -124,6 +136,18 @@
end
end.to raise_error(expected_error_for(provider))
end

it 'retries the request' do
stub_error_response(provider, :chunk)

expect do
chat.ask('Count from 1 to 3') do |chunk|
chunks << chunk
end
end.to raise_error(expected_error_for(provider))

expect(WebMock).to have_requested(:post, expected_url_for(provider)).times(3)
end
end
end
end
Expand Down
8 changes: 4 additions & 4 deletions spec/support/rubyllm_configuration.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@
config.vertexai_location = ENV.fetch('GOOGLE_CLOUD_LOCATION', 'us-central1')

config.request_timeout = 240
config.max_retries = 10
config.retry_interval = 1
config.retry_backoff_factor = 3
config.retry_interval_randomness = 0.5
config.max_retries = 2
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once retry is enabled the tests at spec/ruby_llm/chat_streaming_spec.rb:57 starts to wait between retries and with the current settings it takes a lot of time so I adjusted to something more adequate for test environments.

One question is, should the faraday-retry middleware actually wait in the test environment or it could be set to simply skip the waiting and retry right away?

config.retry_interval = 0.01
config.retry_backoff_factor = 0.1
config.retry_interval_randomness = 0.01

config.model_registry_class = 'Model'
end
Expand Down
4 changes: 4 additions & 0 deletions spec/support/streaming_error_helpers.rb
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,10 @@ def expected_error_for(provider)
ERROR_HANDLING_CONFIGS[provider][:expected_error]
end

def expected_url_for(provider)
ERROR_HANDLING_CONFIGS[provider][:url]
end

def stub_error_response(provider, type)
config = ERROR_HANDLING_CONFIGS[provider]
return unless config
Expand Down