Question about Model Source Code

[1127484067]

Hello, thank you to your team for the Chronos-2 model. It has provided us with new insights into the future development of time series forecasting foundation models. Regarding the code you provided, I would like to ask if there is source code available for fine-tuning the Chronos-2 model on specific domain data?

[shchur]

Hi @1127484067, you can use the fit method to fine-tune the Chronos2Pipeline:

chronos-forecasting/src/chronos/chronos2/pipeline.py

Lines 96 to 175 in 3211108

	def fit(
	self,
	inputs: TensorOrArray
	| Sequence[TensorOrArray]
	| Sequence[Mapping[str, TensorOrArray | Mapping[str, TensorOrArray | None]]],
	prediction_length: int,
	validation_inputs: TensorOrArray
	| Sequence[TensorOrArray]
	| Sequence[Mapping[str, TensorOrArray | Mapping[str, TensorOrArray | None]]]
	| None = None,
	finetune_mode: Literal["full", "lora"] = "full",
	lora_config: "LoraConfig | dict | None" = None,
	context_length: int | None = None,
	learning_rate: float = 1e-6,
	num_steps: int = 1000,
	batch_size: int = 256,
	output_dir: Path | str | None = None,
	min_past: int | None = None,
	finetuned_ckpt_name: str = "finetuned-ckpt",
	callbacks: list["TrainerCallback"] | None = None,
	remove_printer_callback: bool = False,
	disable_data_parallel: bool = True,
	convert_inputs: bool = True,
	**extra_trainer_kwargs,
	) -> "Chronos2Pipeline":
	"""
	Fine-tune a copy of the current Chronos-2 model on the given inputs and return a new pipeline.
	Parameters
	----------
	inputs
	The time series on which the model will be fine-tuned. The allowed formats of inputs are the same as `Chronos2Pipeline.predict()`.
	Note: when `inputs` is a list of dicts, the values inside `future_covariates` are not technically used for training the model;
	however, this key is used to infer which covariates are known into the future. Therefore, if your task contains known future covariates,
	make sure that this key exists in `inputs`. The values of individual future covariates may be set to `None` or an empty array.
	prediction_length
	The prediction horizon for which the model will be fine-tuned
	validation_inputs
	The time series used for validation and model selection. The format of `validation_inputs` is exactly the same as `inputs`, by default None which
	means that no validation is performed. Note that enabling validation may slow down fine-tuning for large datasets.
	finetune_mode
	One of "full" (performs full fine-tuning) or "lora" (performs Low Rank Adaptation (LoRA) fine-tuning), by default "full"
	lora_config
	The configuration to use for LoRA fine-tuning when finetune_mode="lora". Can be a `LoraConfig` object or a dict which is used to initialize `LoraConfig`.
	When unspecified and finetune_mode="lora", a default configuration is used
	context_length
	The maximum context length used during fine-tuning, by default set to the model's default context length
	learning_rate
	The learning rate for the optimizer, by default 1e-6
	When finetune_mode="lora", we recommend using a higher value of the learning rate, such as 1e-5
	num_steps
	The number of steps to fine-tune for, by default 1000
	batch_size
	The batch size used for fine-tuning. Note that the batch size here means the number of time series, including target(s) and covariates,
	which are input into the model. If your data has multiple target and/or covariates, the effective number of time series tasks in a batch
	will be lower than this value, by default 256
	output_dir
	The directory in which outputs from the `Trainer` will be saved, by default set to `chronos-2-finetuned/{%Y-%m-%d_%H-%M-%S}`
	min_past
	The minimum number of time steps the context must have during fine-tuning. All time series shorter than `min_past + prediction_length`
	are filtered out, by default set equal to prediction_length
	finetuned_ckpt_name
	The name of the directory inside `output_dir` in which the final fine-tuned checkpoint will be saved, by default "finetuned-ckpt"
	callbacks
	A list of `TrainerCallback`s which will be forwarded to the HuggingFace `Trainer`
	remove_printer_callback
	If True, all instances of `PrinterCallback` are removed from callbacks
	disable_data_parallel
	If True, ensures that DataParallel is disabled and training happens on a single GPU
	convert_inputs
	If True (default), preprocess raw inputs (convert tensors, encode categoricals, validate).
	If False, inputs are expected to be already preprocessed using `chronos.chronos2.dataset.prepare_inputs`.
	This allows for efficient training on large datasets that don't fit in memory.
	**extra_trainer_kwargs
	Extra kwargs are directly forwarded to `TrainingArguments`
	Returns
	-------
	A new `Chronos2Pipeline` with the fine-tuned model
	"""

Alternatively, you can use the fine-tuning functionality via AutoGluon: https://auto.gluon.ai/dev/tutorials/timeseries/forecasting-chronos.html