function - 我如何正确设置衰减学习率回调并将其传递给 xgboost 中的自定义函数？

Question

我有这个函数来设置递减的学习率:

def learning_rate_005_decay_power_099(current_iter):
    base_learning_rate = 0.05
    lr = base_learning_rate  * np.power(.99, current_iter)
    return lr if lr > 1e-3 else 1e-3

现在我想将它传递给xgboost的train API中的callback参数。我正在尝试这个:

watchlist = [(xg_train, 'train'), (xg_test, 'test')]

num_round = params['num_round']
xgclassifier = xgb.train(params, xg_train, num_round, watchlist, early_stopping_rounds=early_stopping, callbacks = [xgb.callback.reset_learning_rate(learning_rate_005_decay_power_099)]);

但我收到此错误:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-142-398cd3e1ab95> in <module>
     82 
     83             num_round = params['num_round']
---> 84             xgclassifier = xgb.train(params, xg_train, num_round, watchlist, early_stopping_rounds=early_stopping, callbacks = [xgb.callback.reset_learning_rate(learning_rate_005_decay_power_099)]);
     85             xgb_rounds.append(xgclassifier.best_iteration)
     86 

~/.local/lib/python3.6/site-packages/xgboost/training.py in train(params, dtrain, num_boost_round, evals, obj, feval, maximize, early_stopping_rounds, evals_result, verbose_eval, xgb_model, callbacks, learning_rates)
    214                            evals=evals,
    215                            obj=obj, feval=feval,
--> 216                            xgb_model=xgb_model, callbacks=callbacks)
    217 
    218 

~/.local/lib/python3.6/site-packages/xgboost/training.py in _train_internal(params, dtrain, num_boost_round, evals, obj, feval, xgb_model, callbacks)
     68                            end_iteration=num_boost_round,
     69                            rank=rank,
---> 70                            evaluation_result_list=None))
     71         # Distributed code: need to resume to this point.
     72         # Skip the first update if it is a recovery step.

~/.local/lib/python3.6/site-packages/xgboost/callback.py in callback(env)
    137         if context == 'train':
    138             bst, i, n = env.model, env.iteration, env.end_iteration
--> 139             bst.set_param('learning_rate', get_learning_rate(i, n, learning_rates))
    140         elif context == 'cv':
    141             i, n = env.iteration, env.end_iteration

~/.local/lib/python3.6/site-packages/xgboost/callback.py in get_learning_rate(i, n, learning_rates)
    128             new_learning_rate = learning_rates[i]
    129         else:
--> 130             new_learning_rate = learning_rates(i, n)
    131         return new_learning_rate
    132 

TypeError: learning_rate_005_decay_power_099() takes 1 positional argument but 2 were given

如果我使用完全相同的工作例程使用 lightgbm 尝试此操作，效果会很好:

xg_train = lgb.Dataset(X_train, label=y_train, silent = True)
            xg_test = lgb.Dataset(X_test, label=y_test, silent = True)

            watchlist = [xg_train, xg_test]
            names = ['train', 'test']

            xgclassifier = lgb.train(params, xg_train, num_round, valid_sets = watchlist, valid_names = names, early_stopping_rounds= early_stopping,
                                     callbacks=[lgb.reset_parameter(learning_rate=learning_rate_005_decay_power_099)]);

请问如何正确设置此操作？另外，xgboost 和 lightgbm 中的 2 个调用有何不同？

提前非常感谢

Answer 1

我认为它需要一个带有 2 个参数的函数:

f(boosting_round, num_boost_round)

您只为参数 boosting_round 提供了值。

https://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.callback.reset_learning_rate