python - 当我在 Optuna 研究期间动态添加/删除参数时会发生什么？

Question

Optuna 的常见问题解答有 clear answer当涉及到在研究过程中动态调整参数范围时:这不会造成任何问题，因为每个采样器都是单独定义的。

但是添加和/或删除参数又如何呢？ Optuna 能够处理此类调整吗？

在执行此操作时我注意到的一件事是，在结果数据框中，这些参数会获取其他试验的 nan 条目。能够将这些 nan 设置为未采样时的(默认)值会有什么好处吗？考虑到所有这些未知的值，这项研究仍然合理吗？

Answer 1

问题已得到解答here :

Thanks for the question. Optuna internally supports two types of sampling: optuna.samplers.BaseSampler.sample_independent and optuna.samplers.BaseSampler.sample_relative.

The former optuna.samplers.BaseSampler.sample_independent is a method that samples independently on each parameter, and is not affected by the addition or removal of parameters. The added parameters are taken into account from the timing when they are added.

The latter optuna.samplers.BaseSampler.sample_relative is a method that samples by considering the correlation of parameters and is affected by the addition or removal of parameters. Optuna's default search space for correlation is the product set of the domains of the parameters that exist from the beginning of the hyperparameter tuning to the present. Developers who implement samplers can implement their own search space calculation method optuna.samplers.BaseSampler.infer_relative_search_space. This may allow correlations to be considered for hyperparameters that have been added or removed, but this depends on the sampling algorithm, so there is no API for normal users to modify.