This is the result container object returned by resample().

Note that all stored objects are accessed by reference. Do not modify any object without cloning it first.

S3 Methods

Public fields

data

(data.table::data.table())
Internal data storage. We discourage users to directly work with this field. Use as.table.table(ResampleResult) instead.

task

(Task)
The task resample() operated on.

learner

(Learner)
Untrained learner resample() operated on.

resampling

(Resampling)
Instantiated Resampling object which stores the splits into training and test.

uhash

(character(1))
Unique hash for this object.

Active bindings

learners

(list of Learner)
List of trained learners, sorted by resampling iteration.

warnings

(data.table::data.table())
A table with all warning messages. Column names are "iteration" and "msg". Note that there can be multiple rows per resampling iteration if multiple warnings have been recorded.

errors

(data.table::data.table())
A table with all error messages. Column names are "iteration" and "msg". Note that there can be multiple rows per resampling iteration if multiple errors have been recorded.

Methods

Public methods


Method new()

Creates a new instance of this R6 class. An alternative construction method is provided by as_resample_result().

Usage

ResampleResult$new(
  task,
  learner,
  states,
  resampling,
  iterations,
  predictions,
  uhash = NULL
)

Arguments

task

(Task)
Single task all learners are trained on.

learner

(Learner)
Learner used to fit the individual models.

states

(list())
List of learner states (this includes the fitted models).

resampling

(Resampling)
Instantiated resampling. Number of iterations (resampling$iters) must match the number of states.

iterations

(integer())
Iteration numbers.

predictions

(list of Prediction)
List of prediction objects.

uhash

(character(1))
Unique hash for this ResampleResult. If NULL, a new unique hash is generated. This unique hash is primarily needed to group information in BenchmarkResults.


Method format()

Helper for print outputs.

Usage

ResampleResult$format()


Method print()

Printer.

Usage

ResampleResult$print()

Arguments

...

(ignored).


Method help()

Opens the corresponding help page referenced by field $man.

Usage

ResampleResult$help()


Method prediction()

Combined Prediction of all individual resampling iterations, and all provided predict sets. Note that performance measures do not operate on this object, but instead on each prediction object separately and then combine the performance scores with the aggregate function of the respective Measure.

Usage

ResampleResult$prediction(predict_sets = "test")

Arguments

predict_sets

(character())
Subset of {"train", "test"}.

Returns

Prediction.


Method predictions()

List of prediction objects, sorted by resampling iteration. If multiple sets are given, these are combined to a single one for each iteration.

Usage

ResampleResult$predictions(predict_sets = "test")

Arguments

predict_sets

(character())
Subset of {"train", "test"}.

Returns

List of Prediction objects, one per element in predict_sets.


Method score()

Returns a table with one row for each resampling iteration, including all involved objects: Task, Learner, Resampling, iteration number (integer(1)), and Prediction. Additionally, a column with the individual (per resampling iteration) performance is added for each Measure in measures, named with the id of the respective measure id. If measures is NULL, measures defaults to the return value of default_measures().

Usage

ResampleResult$score(measures = NULL, ids = TRUE, predict_sets = "test")

Arguments

measures

(Measure | list of Measure)
Measure(s) to calculate.

ids

(logical(1))
If ids is TRUE, extra columns with the ids of objects ("task_id", "learner_id", "resampling_id") are added to the returned table. These allow to subset more conveniently.

predict_sets

(character())
Vector of predict sets ({"train", "test"}) to construct the Prediction objects from. Default is "test".

Returns

data.table::data.table().


Method aggregate()

Calculates and aggregates performance values for all provided measures, according to the respective aggregation function in Measure. If measures is NULL, measures defaults to the return value of default_measures().

Usage

ResampleResult$aggregate(measures = NULL)

Arguments

measures

(Measure | list of Measure)
Measure(s) to calculate.

Returns

Named numeric().


Method filter()

Subsets the ResampleResult, reducing it to only keep the iterations specified in iters.

Usage

ResampleResult$filter(iters)

Arguments

iters

(integer())
Resampling iterations to keep.

Returns

Returns the object itself, but modified by reference. You need to explicitly $clone() the object beforehand if you want to keeps the object in its previous state.


Method clone()

The objects of this class are cloneable with this method.

Usage

ResampleResult$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

task = tsk("iris") learner = lrn("classif.rpart") resampling = rsmp("cv", folds = 3) rr = resample(task, learner, resampling) print(rr)
#> <ResampleResult> of 3 iterations #> * Task: iris #> * Learner: classif.rpart #> * Warnings: 0 in 0 iterations #> * Errors: 0 in 0 iterations
rr$aggregate(msr("classif.acc"))
#> classif.acc #> 0.9466667
rr$prediction()
#> <PredictionClassif> for 150 observations: #> row_id truth response #> 6 setosa setosa #> 9 setosa setosa #> 10 setosa setosa #> --- #> 146 virginica virginica #> 147 virginica virginica #> 149 virginica virginica
rr$prediction()$confusion
#> truth #> response setosa versicolor virginica #> setosa 50 0 0 #> versicolor 0 46 4 #> virginica 0 4 46
rr$warnings
#> Empty data.table (0 rows and 2 cols): iteration,msg
rr$errors
#> Empty data.table (0 rows and 2 cols): iteration,msg