Skip to content

Commit 56c8a42

Browse files
jeongyoonleeclaude
andcommitted
Add AIPW docstring warning to get_toc() and rate_score() (#888)
The observed-outcome mode uses naive difference-in-means, which is valid for RCTs but biased for observational data. Add docstring notes advising users to pass AIPW pseudo-outcomes for observational settings, per Yadlowsky et al. (2021) Section 4. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 1c555cf commit 56c8a42

1 file changed

Lines changed: 15 additions & 3 deletions

File tree

causalml/metrics/rate.py

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,16 @@ def get_toc(
3131
to calculate TOC. Otherwise, it's estimated as the difference between the mean outcomes
3232
of the treatment and control groups in each quantile band.
3333
34-
Note: when using observed outcomes, if a quantile band contains only treated or only
35-
control units, the code falls back to TOC(q) = 0 for that band (i.e., subset ATE is
36-
set to the overall ATE). This is a conservative approximation and is logged as a warning.
34+
Note: when using observed outcomes (i.e. without ``treatment_effect_col``), the subset
35+
ATE is estimated via a naive difference-in-means. This is valid for randomized
36+
experiments (RCTs) but may be biased for observational data due to confounding within
37+
quantile bands. For observational settings, compute doubly-robust (AIPW) pseudo-outcomes
38+
externally and pass them as ``treatment_effect_col``. See Yadlowsky et al. (2021),
39+
Section 4 for details.
40+
41+
If a quantile band contains only treated or only control units, the code falls back to
42+
TOC(q) = 0 for that band (i.e., subset ATE is set to the overall ATE). This is a
43+
conservative approximation and is logged as a warning.
3744
3845
For details, see Yadlowsky et al. (2021), `Evaluating Treatment Prioritization Rules
3946
via Rank-Weighted Average Treatment Effects`. https://arxiv.org/abs/2111.07966
@@ -175,6 +182,11 @@ def rate_score(
175182
so the absolute scale matches the TOC values but may differ slightly from the paper's
176183
continuous integral definition. Model rankings are preserved.
177184
185+
When using observed outcomes (without ``treatment_effect_col``), the underlying TOC
186+
estimates the subset ATE via naive difference-in-means, which is valid for RCTs but
187+
biased for observational data. For observational settings, pass AIPW pseudo-outcomes
188+
as ``treatment_effect_col``. See the ``get_toc()`` docstring for details.
189+
178190
For details, see Yadlowsky et al. (2021), `Evaluating Treatment Prioritization Rules
179191
via Rank-Weighted Average Treatment Effects`. https://arxiv.org/abs/2111.07966
180192

0 commit comments

Comments
 (0)