I want to create a vector of accuracy measures from decision trees created by repeating holdout samples (same size). I am trying this in CARET.
ctrl <- trainControl(method = "LGOCV",
repeats = 60, p=0.66)
mod1 <- train(Species ~ ., data = iris,
method = "rpart",
trControl = ctrl)
My goal is now to get the vector of accuracy measures from each of the 60 repeated trials. But not sure what to do next.
With one trial, I would use the confusionMatrix(). But not sure what to do in this case.
Best How To :
method = "LGOCV", use
number = 60 not
There is a sub-object called
> ctrl <- trainControl(method = "LGOCV",
+ number = 60, p=0.66)
> mod1 <- train(Species ~ ., data = iris,
+ method = "rpart",
+ trControl = ctrl)
'data.frame': 60 obs. of 3 variables:
$ Accuracy: num 0.902 0.961 0.922 0.922 0.941 ...
$ Kappa : num 0.853 0.941 0.882 0.882 0.912 ...
$ Resample: chr "Resample14" "Resample13" "Resample15" "Resample11" ...
See the option
trainControl. By default this saves the results for the optimal model. You can get them for each tuning parameter too.