Runs a resampling (possibly in parallel).

resample(task, learner, resampling, ctrl = list())

## Arguments

task :: Task. :: Learner. :: Resampling. :: named list() Object to control learner execution. See mlr_control() for details.

## Note

The fitted models are discarded after the predictions have been scored in order to reduce memory consumption. If you need access to the models for later analysis, set store_models to TRUE via mlr_control().

## Syntactic Sugar

The mlr3 package provides some shortcuts to ease the creation of its objects.

First, instead of the objects themselves, it is possible to pass a character() vector which is used to lookup the provided keys in a mlr3misc::Dictonary:

Additionally, each task has an associated default measure (stored in mlr_reflections) which is used as a fallback if no other measure is provided. Classification tasks default to the classification error in "classif.ce", regression tasks to the mean squared error in "regr.mse".

## Parallelization

This function can be parallelized with the future package. One job is one resampling iteration, and all jobs are send to an apply function from future.apply in a single batch. To select a parallel backend, use future::plan().

## Examples

task = mlr_tasks$get("iris") learner = mlr_learners$get("classif.rpart")
resampling = mlr_resamplings$get("cv") # explicitly instantiate the resampling for this task for reproduciblity set.seed(123) resampling$instantiate(task)

rr = resample(task, learner, resampling)
print(rr)#> <ResampleResult> of 10 iterations
#> Task: iris
#> Learner: classif.rpart
# retrieve performance
rr$performance("classif.ce")#> task task_id learner learner_id resampling #> 1: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> 2: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> 3: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> 4: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> 5: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> 6: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> 7: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> 8: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> 9: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> 10: <TaskClassif> iris <LearnerClassifRpart> classif.rpart <ResamplingCV> #> resampling_id iteration prediction classif.ce #> 1: cv 1 <PredictionClassif> 0.00000000 #> 2: cv 2 <PredictionClassif> 0.20000000 #> 3: cv 3 <PredictionClassif> 0.20000000 #> 4: cv 4 <PredictionClassif> 0.06666667 #> 5: cv 5 <PredictionClassif> 0.06666667 #> 6: cv 6 <PredictionClassif> 0.00000000 #> 7: cv 7 <PredictionClassif> 0.00000000 #> 8: cv 8 <PredictionClassif> 0.00000000 #> 9: cv 9 <PredictionClassif> 0.06666667 #> 10: cv 10 <PredictionClassif> 0.00000000rr$aggregate("classif.ce")#> classif.ce
#>       0.06
# merged prediction objects of all resampling iterations
pred = rr$prediction pred$confusion#>             truth
#> response     setosa versicolor virginica
#>   setosa         50          0         0
#>   versicolor      0         46         5
#>   virginica       0          4        45
# Repeat resampling with featureless learner
rr.featureless = resample(task, "classif.featureless", resampling)

# Combine the ResampleResults into a BenchmarkResult
bmr = rr\$combine(rr.featureless)
print(bmr)#> <BenchmarkResult> of 20 rows with 2 resampling runs