[BUG] after the fix in this commit 6c69155, does the result in the paper still hold for Multivariate inference

[FerociousCentaur]

Bug report checklist

• I provided code that demonstrates a minimal reproducible example.
• I confirmed bug exists on the latest mainline of Chronos via source install.

Describe the bug

The bug fixed in the commit says the future and past covariates were not separate because of the way they were calculated. When I personally tried it, I found that even when i don't provide any future covariates, the past covariates keys were used as future covariates as well, so basically there was a data leak issue, which was giving the move look ahead bias. So I just want to confirm, that are the results in the paper still hold true ? Or if there are new results after the fix, can you please share them.

Expected behavior

Model shouldn't be able to see future data if it is not specified under future covariates and is specified as past-only covariate.

To reproduce

Use the model checkpoint/ version before this commit and put a breakpoint on line number 848 in _build_batch function in dataset.py. Do not pass future covarites, just pass past covariates.

Environment description
Operating system: ubuntu
Python version: 3.12
CUDA version: 13.0
PyTorch version: 2.9.1
HuggingFace transformers version:
HuggingFace accelerate version:
Pandas version:

[abdulfatir]

@FerociousCentaur Since this is not really a current bug, but more of a question, please use discussions for this. The results in the tech report are not affected by the issue because it primarily affected fine-tuning, as noted in the PR discussion.

Could you clarify what you mean by the following and provide an MWE?

When I personally tried it, I found that even when i don't provide any future covariates, the past covariates keys were used as future covariates as well, so basically there was a data leak issue, which was giving the move look ahead bias.

Using past_covariates as future is not the correct and expected behavior. However, even if it were, this is not really lookahead bias because past_covariates are only supposed to have data before the prediction start date. If it contains future data, then the data formatting is inaccurate.

[FerociousCentaur]

Hey @abdulfatir , thanks for your quick response.

Clarification:
I mean when we pass the data for fine tuning, we pass the whole data out of which chunks are prepared internally based on prediction length and context length ( by randomly choose a index during train ). And, future covariates is a subset of past covariates. So, when you say "past_covariates are only supposed to have data before the prediction start date. If it contains future data, then the data formatting is inaccurate", is it correct even during fine tuning when the index is chosen randomly and then using that index we decide context and prediction labels ?

So, lets say I fed in data from time t=0 to t=X. and context length is C and prediction length is P ( C+P<=X ). And for fine tuning, i provided the whole dataset as input, just specified what is past covariates, and whats target. Now, internally, a random index is decided between [ C : X-P ], say idx, then context is [idx-C:idx] and target is [idx:idx+P] . But , if i have not specified any future_covariates, and only past_covariates, then I expect only past_covariates data to be used during inference, but before the fix, the same past_only_covariates columns' data were leaking as future_covariates of length [idx:idx+P].

[abdulfatir]

@FerociousCentaur as noted in the PR, fine-tuning indeed had a bug. Please use the latest version of the package for correct fine-tuning. In the Chronos-2 tech report, we do not report fine-tuning results, only zero-shot.