feat: Make timeouts configurable in semantic LLM operations

[YoungVor]

TL;DR

Added configurable request timeout for all semantic operators in Fenic.

What changed?

• Added an optional request_timeout parameter to all semantic operators and extensions (map, extract, predicate, reduce, classify, analyze_sentiment, summarize, parse_pdf, join, sim_join)
• Added verification for request_timeout to be more than 0 and not more than system maximum
• Updated all LLM provider clients to use http clients with large 10 minute timeout
• Modified the inference layer to respect this timeout parameter when making LLM requests
• Updated the model client to use the request-specific timeout, or default to old 120-second value
• Propagated the timeout parameter through all relevant classes in the execution path
• Added tool for testing long llm requests
• Embedding request objects do not have a timeout

How to test?

1. Test with a standard request:

import fenic as fn
df = fn.DataFrame({"text": ["This is a long document..."]})
result = df.select(fn.semantic.map("Summarize this text", text=fn.col("text")))

2. Test with a custom timeout:

# Set a longer timeout for complex operations
result = df.select(fn.semantic.map("Summarize this text", 
                                   text=fn.col("text"), 
                                   request_timeout=300.0))

# Set a shorter timeout for quick operations
result = df.select(fn.semantic.analyze_sentiment(fn.col("text"), 
                                                request_timeout=30.0))

3. Verify timeout behavior by intentionally setting a very short timeout:

# This should trigger a timeout and retry
result = df.select(fn.semantic.extract(fn.col("text"), 
                                      MySchema, 
                                      request_timeout=0.1))

Why make this change?

Different semantic operations may require different timeout thresholds:

• Complex operations like parsing large PDFs or extracting structured data from long documents may need longer timeouts
• Simple operations like sentiment analysis might benefit from shorter timeouts
• Users with specific latency requirements can now customize timeouts to match their needs

This change improves flexibility and user control over request handling, especially for workloads with varying complexity and time sensitivity.

[YoungVor]

• feat: Make timeouts configurable in semantic LLM operations #270 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.