CRAN release: 2023-06-17
data.table()is now re-exported.
- Fixed a test which randomly failed.
- Improved documentation.
- Add encapsulation mode
"try", which works similar to
"none"but captures errors
CRAN release: 2023-05-05
- Added argument
benchmark_grid()function, which can be used to create a benchmark design, where resamplings have been instantiated on tasks.
- Added S3 method for
- Added S3 method for
- The featureless classification learner now returns proper probabilities (#918).
CRAN release: 2023-03-17
- Many returned tables are now assigned a class for a
- Fixed some typos
CRAN release: 2022-11-02
- Removed dependency on package
- Fixed reassembling of
- Fixed bug where the measured elapsed time was 0: https://stackoverflow.com/questions/73797845/mlr3-benchmarking-with-elapsed-time-measure
- Improved the error message when predict type of fallback learner does not match the predict type of the learner (mlr-org/mlr3extralearners#241).
- The test set is now available to the
Learnerduring train for early stopping.
CRAN release: 2022-08-11
- Added multiclass measures:
classif.costsdoes not require a
- New converter:
- Refactored the task types in
CRAN release: 2022-07-21
- Added new options for parallelization (
"mlr3.exec_chunk_size"). These options are passed down to the respective map functions in package
- Fixed runtime measures depending on specific predict types (#832).
- Improved printing of multiple objects.
CRAN release: 2022-03-01
- Most objects now have a new (optional) field
as.data.table()methods for objects of class
Dictonaryhave been extended with additional columns.
as_task_regr.formula()now remove additional atrributes attached to the data which caused some some learners to break.
- Packages are now loaded prior to calling the
$predict()methods of a
Learner. This ensures that package loading errors are properly propagated and not affected by encapsulation (#771).
CRAN release: 2022-02-14
- Setting a fallback learner for a learner with encapsulation in its default settings now automatically sets encapsulation to
as_task_regr()now support the construction of tasks using the formula interface, e.g.
as_task_regr(mpg ~ ., data = mtcars)(#761).
default_values()function to extract parameter default values from
- The row role
"validation"has been renamed to
"holdout". In the next release,
mlr3will start switching to the now more common terms
"test"for the sets created during resampling.
CRAN release: 2022-01-19
- Improved performance for many operations on
benchmark()got a new argument
cloneto control which objects to clone before performing computations.
- Tasks are checked for infinite values during the conversion from
as_task_regr(). A warning is signaled if any column contains infinite values.
CRAN release: 2021-11-16
- Learners which are capable of resuming/continuing (e.g., learner
nroundsupdated) can now optionally store a stack of trained learners to be used to hotstart their training. Note that this feature is still somewhat experimental. See
- New measures to score similarity of selected feature sets:
sim.jaccard(Jaccard Index) and
sim.phi(Phi coefficient) (#690).
predict_newdata()now also supports
- New function
install_pkgs()to install required packages. This generic works for all objects with a
packagesfield as well as
- New learner
$set_levels()to control how data with factor columns is returned, independent of the used
- Measures now return
NAif prerequisite are not met (#699). This allows to conveniently score your experiments with multiple measures having different requirements.
- Feature names may no longer contain the special character
CRAN release: 2021-08-05
- New method to assign labels to columns in tasks:
Task$label(). These will be used in visualizations in the future.
- New method to add stratification variables:
- New helper function
partition()to split a task into a training and test set.
- New standardized getter
- New measures
"bic"to compute the Akaike Information Criterion or the Bayesian Information Criterion, respectively.
- New Resampling method:
ResamplingCustomCV. Creates a custom resampling split based on the levels of a user-provided factor variable.
- New argument
benchmark()to conveniently enable encapsulation and also set the fallback learner to the featureless learner. This is simply for convenience, configuring each learner individually is still possible and allows a more fine-grained control (#634, #642).
- New field
Learnerto enable parallel predictions via the future backend. This currently is only enabled while calling the
$predict_newdatamethods and is disabled during
benchmark()where you have other means to parallelize.
- Deprecated public (and already documented as internal) field
BenchmarkResultto simplify the API and avoid confusion. The converter
as.data.table()can be used instead to access the internal data.
- Measures now have formal hyperparameters. A popular example where this is required is the F1 score, now implemented with customizable
- Changed default of argument
- Fixed getter
- Fixed hashing of some measures.
- Removed experimental column role
uri. This role be split up into multiple roles by the
- Update paramtest to error on extra parameters
CRAN release: 2021-03-05
- Added a
- Renamed column
- Added converters
as_prediction_regr()to reverse the operation of
- Specifying a weight column during
learner$predict_newdata()is not mandatory anymore (#563).
Task$data()defaults to return only active rows and columns, instead of asserting to only return rows and columns. As a result, the
$data()method can now also be used to query inactive rows and cols from the
- New (experimental) column role
uriwhich is intended to point to external resources, e.g. images on the file system.
- New helper
set_threads()to control the number of threads during calls to external packages. All objects will be migrated to have threading disabled in their defaults to avoid conflicting parallelization techniques (#605).
- New option
mlr3.debug: avoid calls to
benchmark()to improve the readability of tracebacks.
- New experimental option
mlr3.allow_utf8_names: allow non-ascii characters in column names in tasks.
CRAN release: 2021-01-21
- Result containers
BenchmarkResultnow optionally remove the DataBackend of the Tasks in order to reduce file size and memory footprint after serialization. To remove the backends from the containers, set
benchmark(), respectively. Note that this behavior will eventually will be the default for future releases.
- Prediction objects generated by
Learner$predict_newdata()now have row ids starting from 1 instead auto incremented row ids of the training task.
as.data.table.DictionaryTasksnow returns an additional column
- Added flag
BenchmarkResult$score()to allow to work with failing learners more conveniently.
CRAN release: 2020-12-06
- New methods for
$set_row_rolesas a replacement for the deprecated and less flexible
- Learners can now have a timeout (#556).
- Removed S3 method
friedman.test.BenchmarkResult()in favor of the new
CRAN release: 2020-10-21
MeasureOOBErrornow has set property
- New learner property
"featureless"to tag learners which can operate on featureless tasks.
- Fixed [ResampleResult] ignoring argument
predict_setsfor returned [Prediction] objects.
- Compatibility with new version of
CRAN release: 2020-10-07
- Updated properties of featureless learners to apply it on all feature types (did not work on POSIXct columns).
- Fixed measures being calculated as
BenchmarkResultfor resamplings with a single iteration (#551).
- Fixed a bug where a broken heuristic disabled nested parallelization via package
BenchmarkResultnow share a common interface to store the experiment results. Manual construction is still possible with helper function
- Fixed deep cloning of
- New measure
classif.prauc(area under precision-recall curve).
- Removed dependency on orphaned package
CRAN release: 2020-09-13
- Compact in-memory representation of R6 objects to save space when saving objects via
- Objects in containers like
BenchmarkResultare now de-duplicated for an optimized serialization.
- Fixed data set
breast_cancer: all factor features are now correctly stored as ordered factors.
- Added a new utility function
CRAN release: 2020-08-07
- Added classification task
ResamplingLOOfor leave-one-out resampling.
- Regression now supports predict type
ResamplingBootstrapin combination with grouping (#514).
- Fixed plot method of
- Added hyperparameter
CRAN release: 2020-07-22
CRAN release: 2020-06-02
future.applyis now imported (instead of suggested). This is necessary to ensure reproducibility: This way exactly the same result is calculated, independent of the parallel backend.
- Fixed a bug where prediction on new data for a task with blocking information raised an exception (#496).
- New binding:
CRAN release: 2020-04-17
- Some handy cheat sheets can now be downloaded from the project homepage.
- Added new measures
classif.bbrier(binary Brier score) and
classif.mbrier(multi-class Brier score).
- Added new Resampling:
- Added base class for unsupervised tasks:
CRAN release: 2020-03-09
CRAN release: 2020-02-23
Switched to new
roxygen2documentation format for R6 classes.
Row ids now must be numeric. It was previously allowed to have character row ids, but this lead to confusion and unnecessary code bloat. Row identifiers (e.g., to be used in plots) can still be part of the task, with row role
Row names can now be queried with
DataBackendMatrixnow supports to store an optional (numeric) dense part.
Added new method
ResampleResults to a subset of iterations.
character()-> object converters.
Empty test sets are now handled separately by learners (#421). An empty prediction object is returned for all learners.
The internal train and predict function of
Learnernow should be implemented as private method: instead of public methods
predict_internal, private methods
.predictare now encouraged.
It is now encouraged to move some internal methods from public to private:
Learner$train_internalshould now be private method
Learner$predict_internalshould now be private method
Measure$score_internalshould now be private method
$.score. The public methods will be deprecated in a future release.
Removed arguments from the constructor of measures
classif.costs. These can be set directly by
CRAN release: 2019-12-19
We have published an article about mlr3 in the Journal of Open Source Software: https://joss.theoj.org/papers/10.21105/joss.01903. See
citation("mlr3")for the citation info.
Learners returned by
BenchmarkResult$learnersare reset to encourage the safer alternative
BenchmarkResult$score()to access trained models.
Fix ordering of levels in
PredictionClassif$set_threshold()(triggered an assertion).
CRAN release: 2019-12-10
Switched from package
Measures can now calculate all scores using micro or macro averaging (#400).
Measures can now be configured to return a customizable performance score (instead of
NA) in case the score cannot be calculated.
Character columns are now treated differently from factor columns. In the long term,
character()columns are supposed to store text.
Fixed a bug triggered by integer grouping variables in
benchmark_grid()now accepts instantiated resamplings under certain conditions.
CRAN release: 2019-10-28
Task$set_row_roles()are now deprecated. Instead it is recommended for now to work with the lists
Learner$predict_newdata()now works without argument
taskif the learner has been fitted with
Names of column roles have been unified (
"groups"have been renamed).
Stratification is now controlled via a task column role (was a parameter of class
Added a S3
predict()method for class
Learnerto increase interoperability with other packages.
Many objects now come with a
$help()which opens the respective manual page.
CRAN release: 2019-09-18
It is now possible to predict and score results on the training set or on both training and test set. Learners can be instructed to predict on multiple sets by setting
"test"). Measures operate on all sets specified in their field
ResampleResult$predictions()are now methods instead of fields, and allow to extract predictions for different predict sets.
ResampleResult$performance()has been renamed to
BenchmarkResult$performance()has been renamed to
Changed API for (internal) constructors accepting
paradox::ParamSet(). Instead of passing the initial values separately, the initial values must now be set directly in the
CRAN release: 2019-08-25
Deprecated support of automatically creating objects from strings. Instead,
mlr3provides the following helper functions intended to ease the creation of objects stored in dictionaries:
BenchmarkResultnow ensures that the stored
ResampleResults are in a persistent order. Thus,
ResampleResults can now be addressed by their position instead of their hash.
New S3 generic
Removed the control object