Skip to contents

Multiclass Brier Score

Source: R/MeasureSimple.R
mlr_measures_classif.mbrier.Rd

Measure to compare true observed labels with predicted probabilities in multiclass classification tasks.

Details

Brier score for multi-class classification problems with \(k\) labels defined as $$ \frac{1}{n} \sum_{i=1}^n \sum_{j=1}^k (I_{ij} - p_{ij})^2. $$ \(I_{ij}\) is 1 if observation \(x_i\) has true label \(j\), and 0 otherwise. \(p_{ij}\) is the probability that observation \(x_i\) belongs to class \(j\).

Note that there also is the more common definition of the Brier score for binary classification problems in bbrier().

Note

The score function calls mlr3measures::mbrier() from package mlr3measures.

If the measure is undefined for the input, NaN is returned. This can be customized by setting the field na_value.

Dictionary

This Measure can be instantiated via the dictionary mlr_measures or with the associated sugar function msr():

mlr_measures$get("classif.mbrier")
msr("classif.mbrier")

Parameters

Empty ParamSet

Meta Information

  • Type: "classif"

  • Range: \([0, 2]\)

  • Minimize: TRUE

  • Required prediction: prob

See also

Dictionary of Measures: mlr_measures

as.data.table(mlr_measures) for a complete table of all (also dynamically created) Measure implementations.

Other classification measures: mlr_measures_classif.acc, mlr_measures_classif.auc, mlr_measures_classif.bacc, mlr_measures_classif.bbrier, mlr_measures_classif.ce, mlr_measures_classif.costs, mlr_measures_classif.dor, mlr_measures_classif.fbeta, mlr_measures_classif.fdr, mlr_measures_classif.fn, mlr_measures_classif.fnr, mlr_measures_classif.fomr, mlr_measures_classif.fp, mlr_measures_classif.fpr, mlr_measures_classif.logloss, mlr_measures_classif.mauc_au1p, mlr_measures_classif.mauc_au1u, mlr_measures_classif.mauc_aunp, mlr_measures_classif.mauc_aunu, mlr_measures_classif.mauc_mu, mlr_measures_classif.mcc, mlr_measures_classif.npv, mlr_measures_classif.ppv, mlr_measures_classif.prauc, mlr_measures_classif.precision, mlr_measures_classif.recall, mlr_measures_classif.sensitivity, mlr_measures_classif.specificity, mlr_measures_classif.tn, mlr_measures_classif.tnr, mlr_measures_classif.tp, mlr_measures_classif.tpr

Other multiclass classification measures: mlr_measures_classif.acc, mlr_measures_classif.bacc, mlr_measures_classif.ce, mlr_measures_classif.costs, mlr_measures_classif.logloss, mlr_measures_classif.mauc_au1p, mlr_measures_classif.mauc_au1u, mlr_measures_classif.mauc_aunp, mlr_measures_classif.mauc_aunu, mlr_measures_classif.mauc_mu, mlr_measures_classif.mcc