feat: Implement Exponential Backoff Retry Handler for LLM and External Service Calls#44
Open
Yashaswini-ds wants to merge 1 commit into
Conversation
- Add RetryConfig dataclass for clean configuration - Implement with_retry decorator supporting both sync and async functions - Apply exponential backoff: delay = base_delay * (backoff_factor ** attempt) - Add jitter to prevent thundering herd problem - Integrate structured retry logs via app/observability/logging.py - Apply decorator to app/rag/llm.py generate_answer() for Ollama resilience Closes devloperdevesh#29
|
Someone is attempting to deploy a commit to the devloperdevesh's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
Hey @Yashaswini-ds! Saw your work on GSSoC 2026. We are building TermUI, a TypeScript terminal UI framework with React-style hooks and JSX, rendered entirely in the terminal. We have 67 unassigned GSSoC issues open. 19 are marked Karanjot, TermUI maintainer |
Author
|
Hi @Karanjot786, Thank you for reaching out and for the invitation. I'll definitely take a look at TermUI and the open GSSoC issues. The project sounds interesting, especially since it uses TypeScript and React-style concepts. Looking forward to exploring it! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Implements a reusable exponential backoff retry decorator in
app/reliability/retry_handler.pyand applies it to the Ollama LLM client inapp/rag/llm.py.Currently, when the Ollama LLM server is unavailable or times out, the application immediately returns an error with no retry logic. This PR adds production-grade resilience to handle transient failures.
Type of Change
Related Issue
Closes #29
What Was Implemented
app/reliability/retry_handler.py(was empty stub)RetryConfigdataclass for clean, type-safe configurationwith_retrydecorator supporting both sync and async functionsdelay = min(base_delay * (backoff_factor ** attempt), max_delay)app/observability/logging.pyapp/rag/llm.py@with_retrydecorator togenerate_answer()with max_retries=3, base_delay=1.0s, max_delay=8.0sChecklist