python - 使用 BaggingClassifier 时打印决策树和 feature_importance

Question

在 scikit learn 中使用 DecisionTreeClassifier 时，可以轻松获取决策树和重要特征。但是，如果我使用装袋函数(例如 BaggingClassifier)，我将无法获得它们中的任何一个。

由于我们需要使用 BaggingClassifier 来拟合模型，因此我无法返回与 DecisionTreeClassifier 相关的结果(打印树(图)、feature_importances_...)。

这是我的脚本:

seed = 7
n_iterations = 199
DTC = DecisionTreeClassifier(random_state=seed,
                                                 max_depth=None,
                                                 min_impurity_split= 0.2,
                                                 min_samples_leaf=6,
                                                 max_features=None, #If None, then max_features=n_features.
                                                 max_leaf_nodes=20,
                                                 criterion='gini',
                                                 splitter='best',
                                                 )

#parametersDTC = {'max_depth':range(3,10), 'max_leaf_nodes':range(10, 30)}
parameters = {'max_features':range(1,200)}
dt = RandomizedSearchCV(BaggingClassifier(base_estimator=DTC,
                              #max_samples=1,
                              n_estimators=100,
                              #max_features=1,
                              bootstrap = False,
                              bootstrap_features = True, random_state=seed),
                        parameters, n_iter=n_iterations, n_jobs=14, cv=kfold,
                        error_score='raise', random_state=seed, refit=True) #min_samples_leaf=10

# Fit the model

fit_dt= dt.fit(X_train, Y_train)
print(dir(fit_dt))
tree_model = dt.best_estimator_

# Print the important features (NOT WORKING)

features = tree_model.feature_importances_
print(features)

rank = np.argsort(features)[::-1]
print(rank[:12])
print(sorted(list(zip(features))))

# Importing the image (NOT WORKING)
from sklearn.externals.six import StringIO

tree.export_graphviz(dt.best_estimator_, out_file='tree.dot') # necessary to plot the graph

dot_data = StringIO() # need to understand but it probably relates to read of strings
tree.export_graphviz(dt.best_estimator_, out_file=dot_data, filled=True, class_names= target_names, rounded=True, special_characters=True)
graph = pydotplus.graph_from_dot_data(dot_data.getvalue())

img = Image(graph.create_png())
print(dir(img)) # with dir we can check what are the possibilities in graph.create_png

with open("my_tree.png", "wb") as png:
    png.write(img.data)

我收到如下错误:“BaggingClassifier”对象没有属性“tree_”，“BaggingClassifier”对象没有属性“feature_importances”。有谁知道我怎样才能获得它们？谢谢。

Answer 1

基于the documentation ，BaggingClassifier 对象确实没有属性“feature_importances”。您仍然可以按照此问题的答案中所述自行计算:Feature importances - Bagging, scikit-learn

您可以使用属性 estimators_ 访问在 BaggingClassifier 拟合期间生成的树，如下例所示:

from sklearn import svm, datasets
from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import BaggingClassifier


iris = datasets.load_iris()
clf = BaggingClassifier(n_estimators=3)
clf.fit(iris.data, iris.target)
clf.estimators_

clf.estimators_ 是 3 个拟合决策树的列表:

[DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,
             max_features=None, max_leaf_nodes=None,
             min_impurity_split=1e-07, min_samples_leaf=1,
             min_samples_split=2, min_weight_fraction_leaf=0.0,
             presort=False, random_state=1422640898, splitter='best'),
 DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,
             max_features=None, max_leaf_nodes=None,
             min_impurity_split=1e-07, min_samples_leaf=1,
             min_samples_split=2, min_weight_fraction_leaf=0.0,
             presort=False, random_state=1968165419, splitter='best'),
 DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,
             max_features=None, max_leaf_nodes=None,
             min_impurity_split=1e-07, min_samples_leaf=1,
             min_samples_split=2, min_weight_fraction_leaf=0.0,
             presort=False, random_state=2103976874, splitter='best')]

因此您可以迭代列表并访问每一棵树。