python - 使用 sklearn 进行网格搜索的显式(预定义)验证集

Question

我有一个数据集，之前已分为 3 组:训练、验证和测试。为了比较不同算法的性能，必须按照给定的方式使用这些集合。

我现在想使用验证集优化我的 SVM 的参数。但是，我找不到如何将验证集显式输入 sklearn.grid_search.GridSearchCV()。下面是我之前用于在训练集上进行 K 折交叉验证的一些代码。但是，对于这个问题，我需要使用给定的验证集。我该怎么做？

from sklearn import svm, cross_validation
from sklearn.grid_search import GridSearchCV

# (some code left out to simplify things)

skf = cross_validation.StratifiedKFold(y_train, n_folds=5, shuffle = True)
clf = GridSearchCV(svm.SVC(tol=0.005, cache_size=6000,
                             class_weight=penalty_weights),
                     param_grid=tuned_parameters,
                     n_jobs=2,
                     pre_dispatch="n_jobs",
                     cv=skf,
                     scoring=scorer)
clf.fit(X_train, y_train)

Answer 1

使用 PredefinedSplit

ps = PredefinedSplit(test_fold=your_test_fold)

然后在GridSearchCV

中设置cv=ps

test_fold : “array-like, shape (n_samples,)

test_fold[i] gives the test set fold of sample i. A value of -1 indicates that the corresponding sample is not part of any test set folds, but will instead always be put into the training fold.

另见 here

when using a validation set, set the test_fold to 0 for all samples that are part of the validation set, and to -1 for all other samples.