Leave-One-Out Cross-Validation

Splits data using leave-one-observation-out. This is identical to cross-validation with the number of folds set to the number of observations.

If this resampling is combined with the grouping features of tasks, it is possible to create custom splits based on an arbitrary factor variable, see the examples.

Dictionary

This Resampling can be instantiated via the dictionary mlr_resamplings or with the associated sugar function rsmp():

mlr_resamplings$get("loo")
rsmp("loo")

References

Bischl B, Mersmann O, Trautmann H, Weihs C (2012). “Resampling Methods for Meta-Model Validation with Recommendations for Evolutionary Computation.” Evolutionary Computation, 20(2), 249–275. doi:10.1162/evco_a_00069 .

See also

• Chapter in the mlr3book: https://mlr3book.mlr-org.com/chapters/chapter3/evaluation_and_benchmarking.html#sec-resampling

• Package mlr3spatiotempcv for spatio-temporal resamplings.

• Dictionary of Resamplings: mlr_resamplings

• as.data.table(mlr_resamplings) for a table of available Resamplings in the running session (depending on the loaded packages).

• mlr3spatiotempcv for additional Resamplings for spatio-temporal tasks.

Other Resampling: Resampling, mlr_resamplings, mlr_resamplings_bootstrap, mlr_resamplings_custom, mlr_resamplings_custom_cv, mlr_resamplings_cv, mlr_resamplings_holdout, mlr_resamplings_insample, mlr_resamplings_repeated_cv, mlr_resamplings_subsampling

Super class

mlr3::Resampling -> ResamplingLOO

Active bindings

iters

(integer(1))
Returns the number of resampling iterations which is the number of rows of the task provided to instantiate. Is NA if the resampling has not been instantiated.

Methods

Public methods

• ResamplingLOO$new()

• ResamplingLOO$clone()

Inherited methods

• mlr3::Resampling$format()
• mlr3::Resampling$help()
• mlr3::Resampling$instantiate()
• mlr3::Resampling$print()
• mlr3::Resampling$test_set()
• mlr3::Resampling$train_set()

---

Method new()

Creates a new instance of this R6 class.

Usage

ResamplingLOO$new()

---

Method clone()

The objects of this class are cloneable with this method.

Usage

ResamplingLOO$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

# Create a task with 10 observations
task = tsk("penguins")
task$filter(1:10)

# Instantiate Resampling
loo = rsmp("loo")
loo$instantiate(task)

# Individual sets:
loo$train_set(1)
#> [1] 1 6 7 4 3 9 8 5 2
loo$test_set(1)
#> [1] 10

# Disjunct sets:
intersect(loo$train_set(1), loo$test_set(1))
#> integer(0)

# Internal storage:
loo$instance # vector
#>  [1] 10  1  6  7  4  3  9  8  5  2

# Combine with group feature of tasks:
task = tsk("penguins")
task$set_col_roles("island", add_to = "group")
loo$instantiate(task)
loo$iters # one fold for each level of "island"
#> [1] 3