This is the result container object returned by benchmark(). A BenchmarkResult consists of the data row-binded data of multiple ResampleResults, which can easily be re-constructed.

Note that all stored objects are accessed by reference. Do not modify any object without cloning it first.

## Public fields

data

(ResultData)
Internal data storage object of type ResultData. We discourage users to directly work with this field. Use as.table.table(BenchmarkResult) instead.

## Active bindings

(character(1))
Task type of objects in the BenchmarkResult. All stored objects (Task, Learner, Prediction) in a single BenchmarkResult are required to have the same task type, e.g., "classif" or "regr". This is NA for empty BenchmarkResults.

(data.table::data.table())
Table of included Tasks with three columns:

learners

(data.table::data.table())
Table of included Learners with three columns:

#### Arguments

data

(ResultData)
An object of type ResultData, either extracted from another ResampleResult, another BenchmarkResult, or manually constructed with as_result_data().

### Method help()

Opens the help page for this object.

### Method print()

Printer.

#### Arguments

bmr

(BenchmarkResult)
A second BenchmarkResult object.

#### Arguments

measures

(Measure | list of Measure)
Measure(s) to calculate.

ids

(logical(1))
Adds object ids ("task_id", "learner_id", "resampling_id") as extra character columns for convenient subsetting.

predict_sets

(character())
Vector of predict sets ({"train", "test"}) to construct the Prediction objects from. Default is "test".

### Method aggregate()

Returns a result table where resampling iterations are combined into ResampleResults. A column with the aggregated performance score is added for each Measure, named with the id of the respective measure.

For convenience, different flags can be set to extract more information from the returned ResampleResult:

BenchmarkResult$aggregate( measures = NULL, ids = TRUE, uhashes = FALSE, params = FALSE, conditions = FALSE ) #### Arguments measures (Measure | list of Measure) Measure(s) to calculate. ids (logical(1)) Adds object ids ("task_id", "learner_id", "resampling_id") as extra character columns for convenient subsetting. uhashes (logical(1)) Adds the uhash values of the ResampleResult as extra character column "uhash". params (logical(1)) Adds the hyperparameter values as extra list column "params". You can unnest them with mlr3misc::unnest(). conditions (logical(1)) Adds the number of resampling iterations with at least one warning as extra integer column "warnings", and the number of resampling iterations with errors as extra integer column "errors". #### Returns ### Method filter() Subsets the benchmark result. If task_ids is not NULL, keeps all tasks with provided task ids and discards all others tasks. Same procedure for learner_ids and resampling_ids. #### Usage BenchmarkResult$filter(
learner_ids = NULL,
learner_hashes = NULL,
resampling_ids = NULL,
resampling_hashes = NULL
)

#### Arguments

(character())

(character())

learner_ids

(character())
Ids of Learners to keep.

learner_hashes

(character())
Hashes of Learners to keep.

resampling_ids

(character())
Ids of Resamplings to keep.

resampling_hashes

(character())
Hashes of Resamplings to keep.

#### Arguments

i

(integer(1))
The iteration value to filter for.

uhash

(logical(1))
The ushash value to filter for.

### Method clone()

The objects of this class are cloneable with this method.

BenchmarkResult$clone(deep = FALSE) #### Arguments deep Whether to make a deep clone. ## Examples set.seed(123) learners = list( lrn("classif.featureless", predict_type = "prob"), lrn("classif.rpart", predict_type = "prob") ) design = benchmark_grid( tasks = list(tsk("sonar"), tsk("spam")), learners = learners, resamplings = rsmp("cv", folds = 3) ) print(design) #> task learner resampling #> 1: <TaskClassif[45]> <LearnerClassifFeatureless[33]> <ResamplingCV[19]> #> 2: <TaskClassif[45]> <LearnerClassifRpart[33]> <ResamplingCV[19]> #> 3: <TaskClassif[45]> <LearnerClassifFeatureless[33]> <ResamplingCV[19]> #> 4: <TaskClassif[45]> <LearnerClassifRpart[33]> <ResamplingCV[19]> bmr = benchmark(design) print(bmr) #> <BenchmarkResult> of 12 rows with 4 resampling runs #> nr task_id learner_id resampling_id iters warnings errors #> 1 sonar classif.featureless cv 3 0 0 #> 2 sonar classif.rpart cv 3 0 0 #> 3 spam classif.featureless cv 3 0 0 #> 4 spam classif.rpart cv 3 0 0 bmr$tasks
bmr$learners #> learner_hash learner_id learner #> 1: 3bbabd1058707305 classif.featureless <LearnerClassifFeatureless[33]> #> 2: fc402e71eadd46bb classif.rpart <LearnerClassifRpart[33]> # first 5 resampling iterations head(as.data.table(bmr, measures = c("classif.acc", "classif.auc")), 5) #> uhash task #> 1: dda21070-039c-4fe7-acbd-8ad8b243b488 <TaskClassif[45]> #> 2: dda21070-039c-4fe7-acbd-8ad8b243b488 <TaskClassif[45]> #> 3: dda21070-039c-4fe7-acbd-8ad8b243b488 <TaskClassif[45]> #> 4: 525b8784-6490-4ab9-980b-70ea2c1a0dcc <TaskClassif[45]> #> 5: 525b8784-6490-4ab9-980b-70ea2c1a0dcc <TaskClassif[45]> #> learner resampling iteration #> 1: <LearnerClassifFeatureless[33]> <ResamplingCV[19]> 1 #> 2: <LearnerClassifFeatureless[33]> <ResamplingCV[19]> 2 #> 3: <LearnerClassifFeatureless[33]> <ResamplingCV[19]> 3 #> 4: <LearnerClassifRpart[33]> <ResamplingCV[19]> 1 #> 5: <LearnerClassifRpart[33]> <ResamplingCV[19]> 2 #> prediction #> 1: <PredictionClassif[19]> #> 2: <PredictionClassif[19]> #> 3: <PredictionClassif[19]> #> 4: <PredictionClassif[19]> #> 5: <PredictionClassif[19]> # aggregate results bmr$aggregate()
# aggregate results with hyperparameters as separate columns mlr3misc::unnest(bmr$aggregate(params = TRUE), "params") #> nr resample_result task_id learner_id resampling_id iters #> 1: 1 <ResampleResult[21]> sonar classif.featureless cv 3 #> 2: 2 <ResampleResult[21]> sonar classif.rpart cv 3 #> 3: 3 <ResampleResult[21]> spam classif.featureless cv 3 #> 4: 4 <ResampleResult[21]> spam classif.rpart cv 3 #> classif.ce method xval #> 1: 0.4660455 mode NA #> 2: 0.2739130 <NA> 0 #> 3: 0.3940399 mode NA #> 4: 0.1086721 <NA> 0 # extract resample result for classif.rpart rr = bmr$aggregate()[learner_id == "classif.rpart", resample_result][[1]] print(rr)
# access the confusion matrix of the first resampling iteration rr$predictions()[[1]]$confusion