fix(otlp): avoid panic in grpc retry without tokio runtime#3472
Open
Wassbdr wants to merge 4 commits intoopen-telemetry:mainfrom
Open
fix(otlp): avoid panic in grpc retry without tokio runtime#3472Wassbdr wants to merge 4 commits intoopen-telemetry:mainfrom
Wassbdr wants to merge 4 commits intoopen-telemetry:mainfrom
Conversation
Use runtime-aware tonic retry fallback (Tokio -> NoAsync) and add regression test for no-runtime context.\n\nFixes open-telemetry#3432
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3472 +/- ##
=======================================
+ Coverage 83.3% 83.4% +0.1%
=======================================
Files 128 128
Lines 25225 25280 +55
=======================================
+ Hits 21026 21103 +77
+ Misses 4199 4177 -22 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…case Introduce new tests for the tonic retry wrapper to ensure correct behavior when the Tokio runtime is available and when the retry feature is disabled. These tests validate the retry logic and the number of attempts made during the operation.
…ion checks Refactor metric tests to include a retry mechanism for flushing metrics and waiting for the gRPC collector to be reachable. This improves the reliability of the tests by ensuring that metrics are properly emitted and validated under varying conditions.
Refactor the panic message in the force_flush function to improve readability by removing unnecessary line breaks. This change enhances the clarity of error reporting during metric flushing attempts.
Member
|
Per comment on #3432 , let's hold off on this one until we know what we are doing with the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix panic when
experimental-grpc-retryis enabled and exports run from a thread without an active Tokio runtime (e.g. thread-basedBatchSpanProcessor).Fixes #3432.
Root cause
The tonic retry wrapper used
Tokioruntime delays unconditionally in retry-enabled builds. The thread-based processors run exports on plain threads without a Tokio reactor, so retry backoff triggered panic (there is no reactor running).Changes
Tokiowhen a Tokio runtime is activeopentelemetry_sdk::runtime::NoAsyncotherwiseopentelemetry-otlp/CHANGELOG.mdValidation
Ran locally:
All passed (
1/1+25/25).