python - 如何对没有predict_proba或decision_function的模型使用CalibrateClassifierCV

Question

我正在尝试使用 CaliberatedClassifierCV() 校准我的模型输出，以创建更好的拟合校准曲线。据我了解，对于基于树的模型、神经网络，必须使用这种方法来校准输出以获得最佳性能。但是，当我尝试这样做时，它会引发错误。

from sklearn.calibration import CalibratedClassifierCV
from sklearn.model_selection import RandomizedSearchCV

pipe_dtr = Pipeline(steps=[('preprocessor', preprocessor),
                           ('clf', DecisionTreeRegressor(random_state=62))])
params_dtr = {
    'clf__max_depth' : np.arange(1,100,5),
    'clf__min_samples_leaf' : [0.01, 0.1, 1]
}
gs_dtr = RandomizedSearchCV(estimator=pipe_dtr, 
                    param_distributions=params_dtr,
                    n_iter=25,
                    scoring='roc_auc',
                    cv=5)

gs_dtr.fit(X_train, y_train)

calib_pipe_dtr = Pipeline(steps=[('preprocessor', preprocessor), 
                                ('calibrator', CalibratedClassifierCV(gs_dtr.best_estimator_, cv='prefit'))])
calib_pipe_dtr.fit(X_train,y_train)

这会引发以下错误

RuntimeError: classifier has no decision_function or predict_proba method.

我该如何解决这个问题..请发表意见。谢谢

Answer 1

回归模型应该用于CalibrateClassifierCV。如果您正在解决分类问题，请使用 DecisionTreeClassifier。

工作示例:

from sklearn.datasets import load_iris
import numpy as np
from sklearn.tree import DecisionTreeClassifier
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.calibration import CalibratedClassifierCV
from sklearn.model_selection import RandomizedSearchCV
from sklearn.model_selection import train_test_split

X, y= load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2, stratify=y)
pipe_dtr = Pipeline(steps=[('preprocessor', StandardScaler()),
                           ('clf', DecisionTreeClassifier(random_state=62))])
params_dtr = {
    'clf__max_depth' : np.arange(1,100,5),
    'clf__min_samples_leaf' : [0.01, 0.1, 1]
}
gs_dtr = RandomizedSearchCV(estimator=pipe_dtr, 
                    param_distributions=params_dtr,
                    n_iter=25,
                    scoring='accuracy',
                    cv=5)

gs_dtr.fit(X_train, y_train)

calib_pipe_dtr = Pipeline(steps=[('preprocessor', StandardScaler()), 
                                ('calibrator', CalibratedClassifierCV(gs_dtr.best_estimator_, cv='prefit'))])
calib_pipe_dtr.fit(X_train, y_train)