This is the result container object returned by resample().

Note that all stored objects are accessed by reference. Do not modify any object without cloning it first.

Format

R6::R6Class object.

Construction

rr = ResampleResult$new(data, uhash = NULL)

Fields

  • data :: data.table::data.table()
    Internal data storage. We discourage users to directly work with this field.

  • task :: Task

    The task resample() operated on.

  • learners :: list of Learner
    List of trained learners, sorted by resampling iteration.

  • resampling :: Resampling
    Instantiated Resampling object which stores the splits into training and test.

  • warnings :: data.table::data.table()
    Returns a table with all warning messages. Column names are "iteration" and "msg". Note that there can be multiple rows per resampling iteration if multiple warnings have been recorded.

  • errors :: data.table::data.table()
    Returns a table with all error messages. Column names are "iteration" and "msg". Note that there can be multiple rows per resampling iteration if multiple errors have been recorded.

  • uhash :: character(1)
    Unique hash for this object.

Methods

  • predictions(predict_sets = "test")
    character() -> list of Prediction
    List of prediction objects, sorted by resampling iteration. If multiple sets are given, these are combined to a single one for each iteration.

  • prediction(predict_sets = "test")
    character() -> Prediction
    Combined Prediction of all individual resampling iterations, and all provided predict sets. Note that performance measures do not operate on this object, but instead on each prediction object separately and then combine the performance scores with the aggregate function of the respective Measure.

  • score(measures = NULL, ids = TRUE)
    (list of Measure, logical(1)) -> data.table::data.table()
    Returns a table with one row for each resampling iteration, including all involved objects: Task, Learner, Resampling, iteration number (integer(1)), and Prediction. A column with the individual (per resampling iteration) performance is added for each Measure, named with the id of the respective measure. If ids is TRUE, extra columns with the ids of objects ("task_id", "learner_id", "resampling_id") are binded to the table to allow a more convenient subsetting. If measures is NULL, measures defaults to the return value of default_measures().

  • aggregate(measures = NULL)
    list of Measure -> named numeric()
    Calculates and aggregates performance values for all provided measures, according to the respective aggregation function in Measure. If measures is NULL, measures defaults to the return value of default_measures().

  • help()
    () -> NULL
    Opens the help page for this object.

S3 Methods

Examples

task = tsk("iris") learner = lrn("classif.rpart") resampling = rsmp("cv", folds = 3) rr = resample(task, learner, resampling) print(rr)
#> <ResampleResult> of 3 iterations #> * Task: iris #> * Learner: classif.rpart #> * Warnings: 0 in 0 iterations #> * Errors: 0 in 0 iterations
rr$aggregate(msr("classif.acc"))
#> classif.acc #> 0.94
rr$prediction()
#> <PredictionClassif> for 150 observations: #> row_id truth response #> 1 setosa setosa #> 2 setosa setosa #> 6 setosa setosa #> --- #> 140 virginica virginica #> 146 virginica virginica #> 149 virginica virginica
rr$prediction()$confusion
#> truth #> response setosa versicolor virginica #> setosa 50 0 0 #> versicolor 0 45 4 #> virginica 0 5 46
rr$warnings
#> Empty data.table (0 rows and 2 cols): iteration,msg
rr$errors
#> Empty data.table (0 rows and 2 cols): iteration,msg