Skip to content

Structured output tool has a required response. #1013

@cptspacemanspiff

Description

@cptspacemanspiff

So I started playing around with the agent SDK, and I realized that the structured output will always force an additional user turn, calling the LLM with placeholder data. Since the structured response is generally the final artifact a sdk call, this will incur additional cached writes for the structured output output, alongside the cached reads of the whole history, which are generally unneeded.

I think that it would be nice if the whole dummy-response handshake was skipped/delayed until there is an additional user call, either from some other tool in the same context chain, or the next user message. The whole required response does not really make sense for a server side tool (if my understanding is correct).

Example

A 'say hello', with a structured output response:

Turn 1 Request (boilerplate systemprompt + structured output tool call):

system blocks: 3
  [0] 'x-anthropic-billing-header: cc_version=2.1.131.880; cc_entrypoint=sdk-py; cch...'
  [1] "You are a Claude agent, built on Anthropic's Claude Agent SDK."
  [2] 'Say hi.'
messages: 3
  msg[0](user).content[0] (text): "<system-reminder>\nAs you answer the user's questions, you can use the followi..."
  msg[0](user).content[1] (text): 'hello'
tools: 1 -> ['StructuredOutput']

Turn 1 response (text + structured output)

Turn 1 response:
  msg[1](assistant).content[0] (text): "Hi there! 👋 It's great to meet you! I'm Claude, an AI assistant built on Anth..."
  msg[1](assistant).content[1]: tool_use name='StructuredOutput' input={"greeting": "Hello! I'm Claude, nice to meet you!"}

Ideally we should be done at this point because the structured output has been returned, however we still are forced to send a synthetic tool response.

Turn 2 request

Turn 2 request:
  msg[2](user).content[0]: tool_result id=toolu_0128w2xXViuuTx7A7Ujpu9wh content='Structured output provided successfully'

Turn 2 response

2 output tokens, end of response empty data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions