Helpers to be used alongside
metric_summarizer() when creating new metrics. See Custom performance metrics for more
get_weights(data, estimator) finalize_estimator(x, estimator = NULL, metric_class = "default") finalize_estimator_internal(metric_dispatcher, x, estimator) dots_to_estimate(data, ...) validate_estimator(estimator, estimator_override = NULL)
A table with truth values as columns and predicted values as rows.
NULLfor auto-selection, or a single character for the type of estimator to use.
The column used to autoselect the estimator. This is generally the
truthcolumn, but can also be a table if your metric has table methods.
A single character of the name of the metric to autoselect the estimator for. This should match the method name created for
A simple dummy object with the class provided to
metric_class. This is created and passed along for you.
A set of unquoted column names or one or more
dplyrselector functions to choose which variables contain the class probabilities. If
truthis binary, only 1 column should be selected. Otherwise, there should be as many columns as factor levels of
A character vector overriding the default allowed estimator list of
c("binary", "macro", "micro", "macro_weighted"). Set this if your classification estimator does not support all of these methods.
get_weights() accepts a confusion matrix and an
estimator of type
"macro_weighted" and returns the correct weights.
It is useful when creating multiclass metrics.
finalize_estimator() is the engine for auto-selection of
on the type of
x is the
truth column. This function
is called from the vector method of your metric.
finalize_estimator_internal() is an S3 generic that you should extend for
your metric if it does not implement only the following estimator types:
If your metric does support all of these, the default version of
finalize_estimator_internal() will autoselect
If you need to create a method, it should take the form:
finalize_estimator_internal.metric_name. Your method for
finalize_estimator_internal() should do two things:
NULL, autoselect the
estimatorbased on the type of
xand return a single character for the
NULL, validate that it is an allowed
estimatorfor your metric and return it.
If you are using the default for
estimator is selected using the following heuristics:
NULL, it is validated and returned immediately as no auto-selection is needed.
"binary"is returned if it has 2 levels, otherwise
"binary"is returned if it has 2 columns, otherwise
"macro"is returned. This is useful if you have
Dots -> Estimate
dots_to_estimate() is useful with class probability metrics that take
... rather than
estimate as an argument. It constructs either a single
name if 1 input is provided to
... or it constructs a quosure where the
expression constructs a matrix of as many columns as are provided to
These are eventually evaluated in the
summarise() call in
metric_summarizer() and evaluate to either a vector or a matrix for further
use in the underlying vector functions.
validate_estimator() is called from your metric specific method of
finalize_estimator_internal() and ensures that a user provided estimator
is of the right format and is one of the allowed values.