Runs a resampling (possibly in parallel).

resample(task, learner, resampling, measures = NULL, ctrl = list())

Arguments

task

(Task): Object of type Task. Instead if a Task object, it is also possible to provide a key to retrieve a task from the mlr_tasks dictionary.

learner

(Learner): Object of type Learner. Instead if a Learner object, it is also possible to provide a key to retrieve a task from the mlr_learners dictionary.

resampling

(Resampling): Object of type Resampling. Instead if a Resampling object, it is also possible to provide a key to retrieve a task from the mlr_resamplings dictionary.

measures

(list of Measure): List of performance measures to calculate. Defaults to the measures specified in the Task task.

ctrl

(named list(), e.g. as returned by mlr_control()): Object to control various parts of the execution. See mlr_control().

Value

ResampleResult.

Examples

task = mlr_tasks$get("iris") learner = mlr_learners$get("classif.rpart") resampling = mlr_resamplings$get("cv") # explicitly instantiate the resampling for this task for reproduciblity set.seed(123) resampling$instantiate(task) rr = resample(task, learner, resampling) print(rr, digits = 2)
#> <ResampleResult> of learner 'iris' on task 'classif.rpart' with 10 iterations #> Measure Min. 1st Qu. Median Mean 3rd Qu. Max. Sd #> classif.ce 0 0 0.033 0.06 0.067 0.2 0.08
# retrieve performance rr$performance("classif.ce")
#> [1] 0.00000000 0.20000000 0.20000000 0.06666667 0.06666667 0.00000000 #> [7] 0.00000000 0.00000000 0.06666667 0.00000000
rr$aggregated
#> classif.ce #> 0.06
# merged prediction object for all experiments pred = rr$prediction pred$confusion
#> truth #> response setosa versicolor virginica #> setosa 50 0 0 #> versicolor 0 46 5 #> virginica 0 4 45
# Repeat resampling with featureless learner rr.featureless = resample(task, "classif.featureless", resampling) # Combine the ResampleResults into a BenchmarkResult bmr = rr$combine(rr.featureless) bmr$aggregated(objects = FALSE)
#> hash resampling_id task_id learner_id classif.ce #> 1: 46087c25867c730b cv iris classif.rpart 0.06 #> 2: 19650135edd3897f cv iris classif.featureless 0.80