在 sklearn.model_selection.cross_val_score
中使用 n_jobs = -1
作为参数时出现错误。我是深度学习和 ANN 的初学者,根据 this 中的讲师在 k 折交叉验证中,使用 n_jobs = -1
来使用 CPU 的所有处理器以减少时间,但在我的情况下它会引发错误。
错误-
BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
可以找到完整的堆栈跟踪 here .
import keras
from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import cross_val_score
def build_classifier():
classifier = Sequential()
classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu', input_dim = 11))
classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu'))
classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
return classifier
classifier = KerasClassifier(build_fn = build_classifier, batch_size = 10, nb_epoch = 100)
accuracies = cross_val_score(estimator = classifier, X = X_train, y = Y_train, cv = 10, n_jobs = -1)
最佳答案
尝试在外部文件中创建您的 build_classifier
函数并导入它。例如:
在文件 classifier_builder.py
中:
import keras
def build_classifier():
classifier = Sequential()
classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu', input_dim = 11))
classifier.add(Dense(units = 6, kernel_initializer = 'uniform', activation = 'relu'))
classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
return classifier
然后在你的笔记本上:
import classifier_builder
classifier = KerasClassifier(build_fn = build_classifier, batch_size = 10, nb_epoch = 100)
accuracies = cross_val_score(estimator = classifier, X = X_train, y = Y_train, cv = 10, n_jobs = -1)
这解决了我的问题。显然,内联函数不可 picklable。
关于python - BrokenProcessPool 关于在 cross_val_score 中使用 n_jobs 参数,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/53135494/