我正在尝试使用以下代码对神经网络的多个参数执行网格搜索:
def create_network(optimizer='rmsprop'):
# Start Artificial Neural Network
network = Sequential()
# Adding the input layer and the first hidden layer
# units = neurons
network.add(Dense(units = 16,
activation = tf.keras.layers.LeakyReLU(alpha=0.3)))
# Adding the second hidden layer
network.add(Dense(units = 16,
activation = tf.keras.layers.LeakyReLU(alpha=0.3)))
# Adding the third hidden layer
network.add(Dense(units = 16,
activation = tf.keras.layers.LeakyReLU(alpha=0.3)))
# Adding the output layer
network.add(Dense(units = 1))
# Compile NN
network.compile(optimizer = optimizer,
loss = 'mean_squared_error',
metrics=['mae', tf.keras.metrics.RootMeanSquaredError()])
# Return compiled network
return network
# Wrap Keras model so it can be used by scikit-learn
ann = KerasRegressor(build_fn=create_network, verbose=0)
# Create hyperparameter space
epoch_values = [10, 25, 50, 100, 150, 200]
batches = [10, 20, 30, 40, 50, 100, 1000]
optimizers = ['rmsprop', 'adam', 'SGD']
neurons = [16, 32, 64, 128, 256]
lr_values = [0.001, 0.01, 0.1, 0.2, 0.3]
# Create hyperparameter options
hyperparameters = dict(optimizer=optimizers, epochs=epoch_values, batch_size=batches, units=neurons,learning_rate=lr_values)
# Create grid search
# cv=5 is the default 5-fold
grid = GridSearchCV(estimator=ann, cv=5, param_grid=hyperparameters)
# Fit grid search
grid_result = grid.fit(X, y)
但是我得到了错误:
learning_rate is not a legal parameter
只有优化器、epochs 和 batch_size 起作用...搜索中无法识别其他参数。
我该如何解决这个问题?
如果有相关知识,我还想为网格搜索添加更多参数。
最佳答案
目前,您没有指示网络使用学习率,因此 scikit-learn
网格搜索不知道如何更改它。明确告诉优化器如何更改 create_network
函数中的学习率(同样适用于 neurons
或任何其他参数)。这样的事情应该有效:
def create_network(optimizer='rmsprop', neurons=16, learning_rate=0.001):
# Start Artificial Neural Network
network = Sequential()
# Adding the input layer and the first hidden layer
network.add(Dense(units = neurons,
activation = tf.keras.layers.LeakyReLU(alpha=0.3)))
# Adding the second hidden layer
network.add(Dense(units = neurons,
activation = tf.keras.layers.LeakyReLU(alpha=0.3)))
# Adding the third hidden layer
network.add(Dense(units = neurons,
activation = tf.keras.layers.LeakyReLU(alpha=0.3)))
# Adding the output layer
network.add(Dense(units = 1))
###############################################
# Add optimizer with learning rate
if optimizer == 'rmsprop':
opt = tf.keras.optimizers.RMSprop(learning_rate=learning_rate)
elif optimizer == 'adam':
opt = tf.keras.optimizers.Adam(learning_rate=learning_rate)
elif optimizer == 'SGD':
opt = tf.keras.optimizers.SGD(learning_rate=learning_rate)
else:
raise ValueError('optimizer {} unrecognized'.format(optimizer))
##############################################
# Compile NN
network.compile(optimizer = opt,
loss = 'mean_squared_error',
metrics=['mae', tf.keras.metrics.RootMeanSquaredError()])
# Return compiled network
return network
# Wrap Keras model so it can be used by scikit-learn
ann = KerasRegressor(build_fn=create_network, verbose=0)
# Create hyperparameter space
epoch_values = [10, 25, 50, 100, 150, 200]
batches = [10, 20, 30, 40, 50, 100, 1000]
optimizers = ['rmsprop', 'adam', 'SGD']
neuron_list = [16, 32, 64, 128, 256]
lr_values = [0.001, 0.01, 0.1, 0.2, 0.3]
# Create hyperparameter options
hyperparameters = dict(
epochs=epoch_values,
batch_size=batches,
optimizer=optimizers,
neurons=neuron_list,
learning_rate=lr_values)
# Create grid search
# cv=5 is the default 5-fold
grid = GridSearchCV(estimator=ann, cv=5, param_grid=hyperparameters)
# Fit grid search
grid_result = grid.fit(X, y)
也可以对神经元
以及与网络结构相关的任何其他参数进行类似的修改。确保将 create_network
的参数名称与 hyperparameters
中的键相匹配。
关于Python:为神经网络定义网格搜索参数的问题,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/68884087/