r - 在 R 中引导逻辑回归后如何做混淆矩阵？

Question

我将我的数据分成训练集和测试集。我引导了我的训练集，我需要对我的测试集进行验证测试。我如何实现这一目标？我是否通过混淆矩阵进行比较？如果是的话，请大家指教？

这是访问数据集的共享链接: https://drive.google.com/open?id=11LzPjH8RQraOI0eOYJRVRwgnRGL6Bnic

library(tidyverse)

library(caret)

mydata <- read.csv("C:/Users/User/Desktop/FYP/FYP2/data.csv")

# create training data
mydata_ones <- mydata[which(mydata$INJ_FAT == 1), ]
mydata_zeros <- mydata[which(mydata$INJ_FAT == 0), ]
set.seed(100) #for repeatability of samples

mydata_ones_training_rows <- sample(1:nrow(mydata_ones), 0.8*nrow(mydata_ones))
mydata_zeros_training_rows <- sample(1:nrow(mydata_zeros),0.8*nrow(mydata_zeros))

training_ones <- mydata_ones[mydata_ones_training_rows, ]
training_zeros <- mydata_zeros[mydata_zeros_training_rows, ]
train.data <- rbind(training_ones, training_zeros) # row bind the 1's and 0's
#print(trainingData)

# create test data
test_ones <- mydata_ones[-mydata_ones_training_rows, ]
test_zeros <- mydata_zeros[-mydata_zeros_training_rows, ]
test.data <- rbind(test_ones, test_zeros)

library(boot) 

x <- model.matrix(~., train.data)
logit.bootstrap <- function(data, indices) {

  d <- data[indices, ]
  fit <- glm(INJ_FAT~., data = d, family = "binomial")

  return(coef(fit))
}

set.seed(12345)
logit.boot <- boot(data=as.data.frame(x), statistic=logit.bootstrap, R=3500)
logit.boot

Answer 1

在您的启动函数中，您只保留回归系数，因此要进行任何类型的验证，您需要取回预测概率。首先，我在下面运行 10 个 Bootstrap ，请注意，您要么使用模型矩阵，要么使用公式和 data.frame，但不能同时使用两者，在您的代码中，您将以 2 个截距结束:

library(tidyverse)
library(caret)

set.seed(100)
mydata <- read.csv("data.csv")
idx = createDataPartition(mydata$INJ_FAT,p=0.8)

train.data <- mydata[idx$Resample1,]
test.data <- mydata[-idx$Resample1,]

library(boot) 
set.seed(12345)
logit.boot <- boot(data=train.data, statistic=logit.bootstrap, R=10)

你的系数存储在这里，每个 bootstrap 1 行，每列是 1 个系数:

head(logit.boot$t)
          [,1]      [,2]       [,3]      [,4]      [,5]     [,6]       [,7]
[1,] -4.271000 1.1001241 -1.4136104 -1.621620 -2.584495 5.374047  -2.691607
[2,] -5.048106 1.6833989 -0.2461192 -2.053468 -1.937496 5.608855  -2.415466
[3,] -8.152342 0.9078029 -1.2023567 -1.102740 -2.585418 5.462476  -2.304434
[4,] -6.254665 1.1466750 -0.5599730 -2.132731 -3.401947 4.939235 -17.332697

对于 1 个 bootstrap，要获得预测概率，您可以:

logodds_to_pred = function(pred,levels){
ifelse(exp(pred)/(1+exp(pred))>0.5,levels[2],levels[1])
}

predictions_b1 = model.matrix(INJ_FAT~.,data=test.data) %*% logit.boot$t[1,]
# convert to 0/1, if prob > 0.5 it's 1 else 0
predictions_b1 = logodds_to_pred(predictions_b1,c(0,1))

confusionMatrix(table(predictions_b1,test.data$INJ_FAT))
Confusion Matrix and Statistics


predictions_b1   0   1
             0 544  27
             1  10  91

               Accuracy : 0.9449          
                 95% CI : (0.9249, 0.9609)
    No Information Rate : 0.8244          
    P-Value [Acc > NIR] : < 2.2e-16       

                  Kappa : 0.7984          

 Mcnemar's Test P-Value : 0.008529  

要为所有 Bootstrap 收集它，我们对所有 Bootstrap 进行矩阵乘法:

logodds = model.matrix(INJ_FAT~.,data=test.data) %*% t(logit.boot$t)
predictions = apply(logodds,2,logodds_to_pred,level=c(0,1))

对于每个 bootstrap(列)，我们做混淆矩阵并得到一个总结:

results = lapply(1:ncol(predictions),function(i){
confusionMatrix(table(test.data$INJ_FAT,predictions[,i]))$overall
})

results[[1]]
      Accuracy          Kappa  AccuracyLower  AccuracyUpper   AccuracyNull 
  9.449405e-01   7.983978e-01   9.249033e-01   9.609404e-01   8.497024e-01 
AccuracyPValue  McnemarPValue 
  6.879023e-15   8.528852e-03

不太确定您将如何通过许多 Bootstrap 整合您的结果，但我想您可以继续进行上述操作..