Introduce support of Responses API #290
Conversation
Some model (incl. `o4-mini-deep-research`) aren't compatible with the chat/completion API. This PR introduces a new `Response` class, which - similarly to `Chat` (and inheriting from the same base `Conversation` class) - allows a user to target the `responses` endpoint.
|
@crmne let me know what you think about the approach, I think this is an important missing part of RubyLLM to actually support all models. If the approach does look good to you, I'll add more tests (incl. streaming ones) and documentation. |
| # chat.ask "What's the best way to learn Ruby?" | ||
| # chat.ask "Can you elaborate on that?" | ||
| class Chat | ||
| include Enumerable |
There was a problem hiding this comment.
I'm moving most of the Chat implementation to a base class Conversation so both Chat and Conversation inherit from the logic
| 'tool_123' => instance_double(RubyLLM::ToolCall, | ||
| id: 'tool_123', | ||
| name: 'test_tool', | ||
| arguments: { 'arg1' => 'value1' }) |
There was a problem hiding this comment.
I think those tests were wrongly relying on the Dummy app ToolCall class VS the RubyLLM one
| end | ||
|
|
||
| RESPONSE_MODELS = [ | ||
| { provider: :openai, model: 'o4-mini-deep-research', params: { tools: [{ type: 'web_search_preview' }] } } |
There was a problem hiding this comment.
the o4-mini-deep-research requires 1 web_search_preview tool.
|
I was working on this other night also. I think what I would prefer is if RubyLLM hid from me which endpoint was necessary for the given model and just handled this automatically. Not sure I love the library consumer having to figure out whether to call chat or response. Excited to see someone working on this! Definitely needed (see #213). |
Yes, I wasn't sure as the tools and some parameters are different.
Gosh, thanks for sharing; I couldn't find anything searching for |
|
Wondering if we can just switch to Responses for everything and keep the old code around for compatibility on other providers. |
2 participants
What this does
Some models (incl.
o4-mini-deep-research) aren't compatible with the chat/completion API. They will fail withThis PR introduces a new
Responseclass, which - similarly toChat(and inheriting from the same baseConversationclass) - allows a user to target theresponsesendpoint.Type of change
Scope check
Quality check
overcommit --installand all hooks passmodels.json,aliases.json)API changes
Related issues
I was surprised nothing was there.