This object stores the predictions returned by a learner of class LearnerClassif. The field task_type is set to "classif".

If probabilities are provided via construction and response is missing, the response is calculated from the probabilities: the class label with the highest probability is chosen. In case of ties, a label is selected randomly.

It is possible to set the probability threshold if probabilities are stored:

  • For binary problems only a single threshold value can be set. If the probability exceeds the threshold, the positive class is predicted. If the probability equals the threshold, the label is selected randomly.

  • For binary and multi-class problems, a named numeric vector of thresholds can be set. The length and names must correspond to the number of classes and class names, respectively. To determine the class label, the probabilities are divided by the threshold. This results in a ratio > 1 if the probability exceeds the threshold, and a ratio < 1 otherwise. Note that it is possible that either none or multiple ratios are greater than 1 at the same time. Anyway, the class label with maximum ratio is determined. In case of ties in the ratio, one of the tied class labels is selected randomly.

Format

R6::R6Class object inheriting from Prediction.

Construction

p = PredictionClassif$new(task = NULL, response = NULL, prob = NULL)
  • task :: TaskClassif
    Task for which the predictions are made. Used to extract the row ids and the true labels. Must be subsetted to test set.

  • response :: factor()
    Vector of predicted class labels. One element for each observation in the test set.

  • prob :: matrix()
    Numeric matrix of class probabilities with one column for each class and one row for each observation in the test set.

  • threshold :: numeric(1)
    Probability threshold between 0 and 1. Assigning a value to this field modifies the stored responses.

  • confusion :: matrix()
    Confusion matrix resulting from the comparison of truth and response. Truth is in columns, predicted response in rows.

Note that it is allowed to initialize this object without any arguments in order to allow to manually construct Prediction objects in a piecemeal fashion. Required are "row_ids", "truth", and "predict_type". Depending on the value of "predict_types", also "response" and "prob" must be set.

Fields

  • row_ids :: (integer() | character())
    Vector of row ids for which predictions are stored.

  • truth :: any
    Vector of true labels.

  • response :: any
    Vector of predicted labels.

  • task_type :: character(1)
    Stores the type of the Task.

  • predict_types :: character()
    Vector of predict types this object stores.

See also

Other Prediction: PredictionRegr, Prediction

Examples

task = mlr_tasks$get("iris") learner = mlr_learners$get("classif.rpart") learner$predict_type = "prob" e = Experiment$new(task, learner)$train()$predict()
#> INFO [mlr3] Training learner 'classif.rpart' on task 'iris' ... #> INFO [mlr3] Predicting with model of learner 'classif.rpart' on task 'iris' ...
p = e$prediction p$predict_types
#> [1] "response" "prob"
#> row_id response truth prob.setosa prob.versicolor prob.virginica #> 1: 1 setosa setosa 1 0 0 #> 2: 2 setosa setosa 1 0 0 #> 3: 3 setosa setosa 1 0 0 #> 4: 4 setosa setosa 1 0 0 #> 5: 5 setosa setosa 1 0 0 #> 6: 6 setosa setosa 1 0 0
# confusion matrix p$confusion
#> truth #> response setosa versicolor virginica #> setosa 50 0 0 #> versicolor 0 49 5 #> virginica 0 1 45
# change threshold p$threshold = mlr3misc::set_names(c(0.05, 0.9, 0.05), task$class_names) p$confusion
#> truth #> response setosa versicolor virginica #> setosa 50 0 0 #> versicolor 0 0 0 #> virginica 0 50 50