python - 一次随机将 Pandas 数据框分成几组以进行 x 折交叉验证

Question

假设我有一个包含 500 行的数据框。我想执行 10 折交叉验证。因此，我需要将这些数据分成 10 组，每组包含 50 行。我也想同时随机将整个数据分成 10 组。

有没有办法使用 pandas、numpy 等任何库来做到这一点？

Answer 1

您可以使用 sklearn 的 KFold :

import numpy as np
import pandas as pd
from sklearn.model_selection import KFold 

# create dummy dataframe with 500 rows
features = np.random.randint(1, 100, 500)
labels = np.random.randint(1, 100, 500)
df = pd.DataFrame(data = {"X": features, "Y": labels})

kf = KFold(n_splits=10, random_state=42, shuffle=True) # Define the split - into 10 folds 
kf.get_n_splits(df) # returns the number of splitting iterations in the cross-validator
print(kf) 

for train_index, test_index in kf.split(df):
    print("TRAIN:", train_index) 
    print("TEST:", test_index)
    X_train, X_test = df.loc[train_index, "X"], df.loc[test_index, "X"]
    y_train, y_test = df.loc[train_index, "Y"], df.loc[test_index, "Y"]

示例 taken from here .