Measure to compare true observed response with predicted quantiles in regression tasks.
Details
\(R^1(\alpha)\) is defined as $$ 1 - \frac{\sum_{i=1}^n \rho_\alpha \left( t_i - r_i(\alpha) \right)}{\sum_{i=1}^n \rho_\alpha \left( t_i - q_{\alpha} \right)}, $$ where for a quantile \(\alpha\), \(\rho_\alpha\) is the pinball function, \(r_i(\alpha)\) are the predictions for the quantile and \(q_{\alpha}\) is the empirical \(\alpha\)-quantile of the test or training data.
\(R^1(\alpha)\) is analogous to \(R^2\) for regression tasks. It compares the pinball function of the predictions relative to a naive model predicting the empirical quantile.
This measure is undefined for constant \(t\).
Dictionary
This Measure can be instantiated via the dictionary mlr_measures or with the associated sugar function msr():
Meta Information
Task type: “regr”
Range: \((-\infty, 1]\)
Minimize: FALSE
Average: macro
Required Prediction: “quantiles”
Required Packages: mlr3
References
Koenker, Roger, Machado, F. JA (1999). “Goodness of Fit and Related Inference Processes for Quantile Regression.” Journal of the American Statistical Association, 94(448), 1296–1310. doi:10.1080/01621459.1999.10473882 .
See also
Chapter in the mlr3book: https://mlr3book.mlr-org.com/chapters/chapter2/data_and_basic_modeling.html#sec-eval
Package mlr3measures for the scoring functions. Dictionary of Measures: mlr_measures
as.data.table(mlr_measures)for a table of available Measures in the running session (depending on the loaded packages).Extension packages for additional task types:
mlr3proba for probabilistic supervised regression and survival analysis.
mlr3cluster for unsupervised clustering.
Other Measure:
Measure,
MeasureClassif,
MeasureRegr,
MeasureSimilarity,
mlr_measures,
mlr_measures_aic,
mlr_measures_bic,
mlr_measures_classif.costs,
mlr_measures_debug_classif,
mlr_measures_elapsed_time,
mlr_measures_internal_valid_score,
mlr_measures_oob_error,
mlr_measures_regr.pinball,
mlr_measures_regr.rsq,
mlr_measures_selected_features
Super classes
mlr3::Measure -> mlr3::MeasureRegr -> MeasureRQR
Methods
Method new()
Creates a new instance of this R6 class.
Usage
MeasureRegrRQR$new(alpha = 0.5, pred_set_mean = TRUE)Arguments
alphanumeric(1)
The quantile for which to compute the measure. Must be one of the quantiles that the Learner was trained on.pred_set_meanlogical(1)
IfTRUE, the mean of the true values is calculated on the prediction set. IfFALSE, the mean of the true values is calculated on the training set.