Skip to content

Introduce Generic Structured Output Handling (CrewAI Inspired) #244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ZhengKai91 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Introduce Generic Structured Output Handling (CrewAI Inspired) #244

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions stagehand/llm/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
import litellm

from stagehand.metrics import get_inference_time_ms, start_inference_timer
from pydantic import BaseModel
from .structed_output_handler import StructuredOutputHandler

if TYPE_CHECKING:
from ..logging import StagehandLogger
Expand Down Expand Up @@ -119,6 +121,7 @@ async def create_response(
category="llm",
)


try:
# Start tracking inference time
start_time = start_inference_timer()
Expand All @@ -134,6 +137,13 @@ async def create_response(
self.metrics_callback(response, inference_time_ms, function_name)

return response
except litellm.BadRequestError as e:
Copy link

@cubic-dev-ai cubic-dev-ai bot Dec 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Catching all BadRequestError is too broad. This fallback to StructuredOutputHandler will fail if the error was unrelated to response_format (e.g., invalid model name, rate limits) or if response_format wasn't in the request. Consider checking if response_format is present in filtered_params before attempting the fallback, or catching a more specific error condition.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At stagehand/llm/client.py, line 140:

<comment>Catching all `BadRequestError` is too broad. This fallback to `StructuredOutputHandler` will fail if the error was unrelated to `response_format` (e.g., invalid model name, rate limits) or if `response_format` wasn&#39;t in the request. Consider checking if `response_format` is present in `filtered_params` before attempting the fallback, or catching a more specific error condition.</comment>

<file context>
@@ -134,6 +137,13 @@ async def create_response(
                 self.metrics_callback(response, inference_time_ms, function_name)
 
             return response
+        except litellm.BadRequestError as e:
+            handler = StructuredOutputHandler(litellm)
+            response = await handler.handle_structured_inference(**filtered_params)
</file context>
Fix with Cubic

handler = StructuredOutputHandler(litellm)
response = await handler.handle_structured_inference(**filtered_params)
inference_time_ms = get_inference_time_ms(start_time)
if self.metrics_callback:
self.metrics_callback(response, inference_time_ms, function_name)
return response

except Exception as e:
self.logger.error(f"Error calling litellm.acompletion: {e}", category="llm")
Expand Down
Loading