fix(openfeature): block initialize() until RC config arrives (#16650)

## Motivation

`DataDogProvider.initialize()` returns immediately without waiting for Remote Config data. The OpenFeature SDK then emits `PROVIDER_READY` (per spec: "READY when initialize() terminates normally"), so consumers believe the provider is ready. Flag evaluations in this window silently return default values with `reason: DEFAULT` — there is no error, no indication that config hasn't loaded yet.

This was reported by a customer running a Python script (not a long-running server). On servers the bug is masked because RC config typically arrives during startup before any evaluations happen. In scripts and short-lived processes, `set_provider()` returns in 0.00s and the very next evaluation gets defaults.

Every other Datadog OpenFeature provider blocks inside `initialize()` until config arrives:
- **Java**: `CountDownLatch.await(timeout, unit)` — default 30s
- **Go**: `sync.Cond.Wait()` inside a loop — default 30s
- **Node.js**: `await initController.wait()` (deferred Promise) — default 30s

Fixes: FFL-1843

## Changes

- `DataDogProvider.__init__()` now creates a `threading.Event` (`_config_event`) used to block `initialize()` until config arrives.
- `initialize()` checks if config already exists (fast path), otherwise calls `_config_event.wait(timeout)`. If the timeout expires without config, it raises `ProviderNotReadyError` (the SDK then dispatches `PROVIDER_ERROR`).
- `on_configuration_received()` calls `_config_event.set()` to unblock `initialize()` when the first RC payload arrives. If init already timed out, it emits `PROVIDER_READY` for late recovery.
- `shutdown()` clears the event for clean re-initialization.
- New env var `DD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MS` (default 30000) controls the timeout. Also configurable via constructor: `DataDogProvider(init_timeout=10.0)`.

## Decisions

- **Blocking by default** (30s timeout) matches Java/Go/Node.js. The OpenFeature Python SDK only has `set_provider()` (no `set_provider_and_wait()` yet), and it calls `initialize()` synchronously on the caller's thread. So blocking here means `set_provider()` itself blocks — which is the correct default behavior for most users.
- **Timeout raises `ProviderNotReadyError`** rather than returning silently. This puts the provider in `ERROR` state (not premature `READY`), which is the same pattern Java and Node.js use on timeout.
- **Late recovery supported**: if config arrives after the timeout, `on_configuration_received()` emits `PROVIDER_READY` and the provider transitions from `ERROR` to `READY`.
- **No `init_timeout=0` async mode**: rather than adding a special non-blocking mode to the provider, async customers can wrap `set_provider()` in a background thread and listen for `PROVIDER_READY` events. A proper `set_provider_and_wait()` is being contributed upstream to the OpenFeature Python SDK ([open-feature/python-sdk#567](open-feature/python-sdk#567)).

## Testing

Verified locally using system-tests parametric tests against a patched build:

| Test | Before fix | After fix |
|---|---|---|
| `test_ffe_evaluation_immediately_after_start_without_config` | **FAILED** — `ffe_start()` returned in 0.00s, eval returned defaults | **PASSED** — blocked 30s, timed out correctly |
| `test_ffe_init_blocks_until_config_received` | PASSED | PASSED |
| `test_ffe_init_returns_real_values_not_defaults` | PASSED | PASSED |

Server log confirms blocking: `Waiting up to 30.0s for initial FFE configuration from Remote Config`

Existing FFE parametric tests: 13 passed, 0 failed (remaining errors were container resource exhaustion, not code-related).

Co-authored-by: dd-oleksii <oleksii.shmalko@datadoghq.com>

Author: 2 people

ddtrace/internal/openfeature/_provider.py

@@ -8,6 +8,7 @@
 from collections import OrderedDict
 from collections.abc import MutableMapping
 from importlib.metadata import version
+import threading
 import typing
 
 from openfeature.evaluation_context import EvaluationContext
@@ -90,11 +91,20 @@ class DataDogProvider(AbstractProvider):
     Feature Flags and Experimentation (FFE) product.
     """
 
-    def __init__(self, *args: typing.Any, **kwargs: typing.Any):
+    def __init__(self, *args: typing.Any, initialization_timeout: typing.Optional[float] = None, **kwargs: typing.Any):
         super().__init__(*args, **kwargs)
         self._metadata = Metadata(name="Datadog")
         self._status = ProviderStatus.NOT_READY
-        self._config_received = False
+
+        # Initialization timeout: constructor arg takes priority, then env var (default 30s)
+        if initialization_timeout is not None:
+            self._initialization_timeout = initialization_timeout
+        else:
+            self._initialization_timeout = ffe_config.initialization_timeout_ms / 1000.0
+
+        # Event used to block initialize() until config arrives.
+        # Also serves as the "config received" flag via is_set().
+        self._config_received = threading.Event()
 
         # Cache for reported exposures to prevent duplicates
         # Stores mapping of (flag_key, subject_id) -> (allocation_key, variant_key)
@@ -119,9 +129,6 @@ def __init__(self, *args: typing.Any, **kwargs: typing.Any):
             self._flag_eval_metrics = FlagEvalMetrics()
             self._flag_eval_hook = FlagEvalHook(self._flag_eval_metrics)
 
-        # Register this provider instance for status updates
-        _register_provider(self)
-
     def get_metadata(self) -> Metadata:
         """Returns provider metadata."""
         return self._metadata
@@ -142,32 +149,52 @@ def initialize(self, evaluation_context: EvaluationContext) -> None:
         """
         Initialize the provider.
 
-        Called by the OpenFeature SDK when the provider is set.
-        Provider Creation → NOT_READY
-                                 ↓
-                   First Remote Config Payload
-                                 ↓
-                            READY (emits PROVIDER_READY event)
-                                 ↓
-                           Shutdown
-                                 ↓
-                          NOT_READY
+        Blocks until Remote Config delivers the first FFE configuration or
+        the initialization timeout expires.
+
+        The timeout is configurable via:
+        - Constructor: DataDogProvider(initialization_timeout=10.0)  # seconds
+        - Env var: DD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MS=10000
+
+        Provider lifecycle:
+            NOT_READY -> initialize() blocks -> config arrives -> READY
+            NOT_READY -> initialize() blocks -> timeout -> raises ProviderNotReadyError
         """
         if not self._enabled:
             return
 
+        # Register for RC config callbacks (in initialize, not __init__, so
+        # re-initialization after shutdown re-registers the provider)
+        _register_provider(self)
+
         try:
             # Start the exposure writer for reporting
             start_exposure_writer()
         except ServiceStatusError:
             logger.debug("Exposure writer is already running", exc_info=True)
 
-        # If configuration was already received before initialization, emit ready now
+        # Fast path: config already available (RC delivered before set_provider)
         config = _get_ffe_config()
-        if config is not None and not self._config_received:
-            self._config_received = True
+        if config is not None:
+            logger.debug("FFE configuration already available, provider is READY")
+            self._config_received.set()
             self._status = ProviderStatus.READY
-            self._emit_ready_event()
+            return  # SDK will dispatch PROVIDER_READY
+
+        # Block until config arrives or timeout expires
+        logger.debug(
+            "Waiting up to %.1fs for initial FFE configuration from Remote Config", self._initialization_timeout
+        )
+        if not self._config_received.wait(timeout=self._initialization_timeout):
+            # Timeout expired without receiving config
+            from openfeature.exception import ProviderNotReadyError
+
+            raise ProviderNotReadyError(
+                f"Provider timed out after {self._initialization_timeout:.1f}s waiting for "
+                "initial configuration from Remote Config"
+            )
+
+        # Config received during wait -- on_configuration_received() already set status
 
     def shutdown(self) -> None:
         """
@@ -196,7 +223,7 @@ def shutdown(self) -> None:
         # Unregister provider
         _unregister_provider(self)
         self._status = ProviderStatus.NOT_READY
-        self._config_received = False
+        self._config_received.clear()
 
     def resolve_boolean_details(
         self,
@@ -463,14 +490,18 @@ def on_configuration_received(self) -> None:
         """
         Called when a Remote Configuration payload is received and processed.
 
-        Emits PROVIDER_READY event on first configuration.
+        Updates status first, then signals the event to unblock initialize().
+        Emits PROVIDER_READY for late arrivals (config received after initialize() timed out).
         """
-        if not self._config_received:
-            self._config_received = True
+        if not self._config_received.is_set():
             self._status = ProviderStatus.READY
             logger.debug("First FFE configuration received, provider is now READY")
+            # Emit READY for late recovery: config arrived after init timed out
             self._emit_ready_event()
 
+        # Signal the event last to unblock initialize() after status is updated
+        self._config_received.set()
+
     def _emit_ready_event(self) -> None:
         """
         Safely emit PROVIDER_READY event.

ddtrace/internal/settings/openfeature.py

@@ -30,10 +30,20 @@ class OpenFeatureConfig(DDConfig):
         default=1.0,
     )
 
+    # Provider initialization timeout in milliseconds.
+    # Controls how long initialize() blocks waiting for the first Remote Config payload.
+    # Default is 30000ms (30 seconds), matching Java, Go, and Node.js SDKs.
+    initialization_timeout_ms = DDConfig.var(
+        int,
+        "DD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MS",
+        default=30000,
+    )
+
     _openfeature_config_keys = [
         "experimental_flagging_provider_enabled",
         "ffe_intake_enabled",
         "ffe_intake_heartbeat_interval",
+        "initialization_timeout_ms",
     ]
 
 

releasenotes/notes/fix-openfeature-init-blocking-70c8d5a99287cc49.yaml

@@ -0,0 +1,14 @@
+---
+fixes:
+  - |
+    openfeature: This fix resolves an issue where ``DataDogProvider.initialize()`` returned before
+    configuration was received, causing the OpenFeature SDK to mark the provider as ready to serve
+    evaluations too early and flag evaluations to silently return default values. The provider now
+    waits for configuration before returning.
+features:
+  - |
+    openfeature: This introduces a configurable initialization timeout for ``DataDogProvider``.
+    The timeout controls how long ``initialize()`` waits for configuration before returning,
+    and defaults to 30 seconds. Set it via the
+    ``DD_EXPERIMENTAL_FLAGGING_PROVIDER_INITIALIZATION_TIMEOUT_MS`` environment variable or the
+    ``init_timeout`` constructor parameter.

tests/openfeature/conftest.py

@@ -0,0 +1,22 @@
+"""
+Shared fixtures for openfeature tests.
+"""
+
+import pytest
+
+from tests.utils import override_global_config
+
+
+@pytest.fixture(autouse=True)
+def fast_initialization_timeout():
+    """
+    Override the provider initialization timeout to 100ms for all tests.
+
+    The production default is 30s (matching other SDKs), but that makes any
+    test that calls api.set_provider() without pre-loaded config hang for 30s.
+    Tests that need to verify timeout/blocking behaviour set their own explicit
+    initialization_timeout= on DataDogProvider() directly, which takes priority
+    over the config value.
+    """
+    with override_global_config({"initialization_timeout_ms": 100}):
+        yield

tests/openfeature/test_provider.py

@@ -55,6 +55,9 @@ class TestProviderInitializationShutdown:
 
     def test_provider_initialization(self, provider, evaluation_context):
         """Provider should initialize without errors."""
+        # Pre-load config so initialize() takes the fast path (no blocking wait)
+        config = create_config(create_boolean_flag("test-flag", enabled=True, default_value=True))
+        process_ffe_configuration(config)
         # Should not raise
         provider.initialize(evaluation_context)
 

tests/openfeature/test_provider_status.py

@@ -5,8 +5,12 @@
 - NOT_READY by default
 - READY when first Remote Config payload is received
 - Event emission on status change
+- Blocking initialization until config arrives or timeout
 """
 
+import threading
+import time
+
 from openfeature import api
 from openfeature.provider import ProviderStatus
 import pytest
@@ -43,7 +47,7 @@ def test_provider_starts_not_ready(self):
             provider = DataDogProvider()
 
             assert provider._status == ProviderStatus.NOT_READY
-            assert provider._config_received is False
+            assert not provider._config_received.is_set()
 
     def test_provider_becomes_ready_after_first_config(self):
         """Test that provider becomes READY after receiving first configuration."""
@@ -61,7 +65,7 @@ def test_provider_becomes_ready_after_first_config(self):
 
                 # Verify becomes READY
                 assert provider._status == ProviderStatus.READY
-                assert provider._config_received is True
+                assert provider._config_received.is_set()
             finally:
                 api.clear_providers()
 
@@ -73,14 +77,14 @@ def test_provider_ready_event_emitted(self):
 
             try:
                 # Provider should not have received config yet
-                assert not provider._config_received
+                assert not provider._config_received.is_set()
 
                 # Process a configuration
                 config = create_config(create_boolean_flag("test-flag", enabled=True))
                 process_ffe_configuration(config)
 
                 # Provider should now have received config and be READY
-                assert provider._config_received
+                assert provider._config_received.is_set()
                 assert provider._status == ProviderStatus.READY
             finally:
                 api.clear_providers()
@@ -140,7 +144,7 @@ def test_provider_status_after_shutdown(self):
 
                 # Verify back to NOT_READY
                 assert provider._status == ProviderStatus.NOT_READY
-                assert provider._config_received is False
+                assert not provider._config_received.is_set()
             finally:
                 api.clear_providers()
 
@@ -194,3 +198,98 @@ def on_provider_ready(event_details):
             finally:
                 api.remove_handler(ProviderEvent.PROVIDER_READY, on_provider_ready)
                 api.clear_providers()
+
+
+class TestProviderInitializationBlocking:
+    """Test that initialize() blocks until config arrives or timeout expires."""
+
+    def test_initialize_blocks_until_config_arrives(self):
+        """initialize() should block and return once config is delivered mid-wait."""
+        with override_global_config({"experimental_flagging_provider_enabled": True}):
+            provider = DataDogProvider(initialization_timeout=5.0)
+
+            # Deliver config from a background thread after 0.5s
+            def deliver_config():
+                time.sleep(0.5)
+                config = create_config(create_boolean_flag("test-flag", enabled=True))
+                process_ffe_configuration(config)
+
+            timer = threading.Thread(target=deliver_config, daemon=True)
+            timer.start()
+
+            try:
+                start = time.monotonic()
+                api.set_provider(provider)
+                elapsed = time.monotonic() - start
+
+                # Should have blocked for ~0.5s (not instant, not full timeout)
+                assert elapsed >= 0.3, f"initialize() returned too fast ({elapsed:.2f}s)"
+                assert elapsed < 4.0, f"initialize() took too long ({elapsed:.2f}s), should have unblocked at ~0.5s"
+                assert provider._status == ProviderStatus.READY
+                assert provider._config_received.is_set()
+            finally:
+                api.clear_providers()
+
+    def test_initialize_fast_path_when_config_exists(self):
+        """initialize() should return immediately if config already exists."""
+        with override_global_config({"experimental_flagging_provider_enabled": True}):
+            # Deliver config BEFORE creating provider
+            config = create_config(create_boolean_flag("test-flag", enabled=True))
+            process_ffe_configuration(config)
+
+            provider = DataDogProvider(initialization_timeout=5.0)
+
+            try:
+                start = time.monotonic()
+                api.set_provider(provider)
+                elapsed = time.monotonic() - start
+
+                # Should be near-instant (config already available)
+                assert elapsed < 1.0, f"initialize() took {elapsed:.2f}s, should be instant with pre-loaded config"
+                assert provider._status == ProviderStatus.READY
+            finally:
+                api.clear_providers()
+
+    def test_initialize_timeout_raises(self):
+        """initialize() should raise ProviderNotReadyError after timeout expires."""
+        with override_global_config({"experimental_flagging_provider_enabled": True}):
+            provider = DataDogProvider(initialization_timeout=0.5)
+
+            try:
+                start = time.monotonic()
+                # set_provider catches the exception and dispatches PROVIDER_ERROR
+                api.set_provider(provider)
+                elapsed = time.monotonic() - start
+
+                # Should have blocked for ~0.5s (the timeout)
+                assert elapsed >= 0.3, f"initialize() returned too fast ({elapsed:.2f}s)"
+                assert elapsed < 2.0, f"initialize() took too long ({elapsed:.2f}s)"
+
+                # Provider should be in ERROR state (SDK caught ProviderNotReadyError)
+                client = api.get_client()
+                assert client.get_provider_status() == ProviderStatus.ERROR
+            finally:
+                api.clear_providers()
+
+    def test_late_recovery_after_timeout(self):
+        """Config arriving after timeout should transition provider to READY."""
+        with override_global_config({"experimental_flagging_provider_enabled": True}):
+            provider = DataDogProvider(initialization_timeout=0.5)
+
+            try:
+                # Let it timeout
+                api.set_provider(provider)
+
+                # Provider should be in ERROR state
+                client = api.get_client()
+                assert client.get_provider_status() == ProviderStatus.ERROR
+
+                # Now deliver config (late recovery)
+                config = create_config(create_boolean_flag("test-flag", enabled=True))
+                process_ffe_configuration(config)
+
+                # Provider should recover to READY
+                assert provider._status == ProviderStatus.READY
+                assert provider._config_received.is_set()
+            finally:
+                api.clear_providers()