These functions are appropriate for cases where the model outcome is a
numeric. The ratio of performance to deviation
rpd()) and the ratio of performance to inter-quartile (
are both measures of consistency/correlation between observed
and predicted values (and not of accuracy).
rpiq(data, ...) # S3 method for data.frame rpiq(data, truth, estimate, na_rm = TRUE, ...) rpiq_vec(truth, estimate, na_rm = TRUE, ...)
Not currently used.
The column identifier for the true results
The column identifier for the predicted
results (that is also
tibble with columns
.estimate and 1 row of values.
For grouped data frames, the number of rows returned will be the same as the number of groups.
rpd_vec(), a single
numeric value (or
In the field of spectroscopy in particular, the ratio of performance to deviation (RPD) has been used as the standard way to report the quality of a model. It is the ratio between the standard deviation of a variable and the standard error of prediction of that variable by a given model. However, its systematic use has been criticized by several authors, since using the standard deviation to represent the spread of a variable can be misleading on skewed dataset. The ratio of performance to inter-quartile has been introduced by Bellon-Maurel et al. (2010) to address some of these issues, and generalise the RPD to non-normally distributed variables.
Williams, P.C. (1987) Variables affecting near-infrared reflectance spectroscopic analysis. In: Near Infrared Technology in the Agriculture and Food Industries. 1st Ed. P.Williams and K.Norris, Eds. Am. Cereal Assoc. Cereal Chem., St. Paul, MN.
Bellon-Maurel, V., Fernandez-Ahumada, E., Palagos, B., Roger, J.M. and McBratney, A., (2010). Critical review of chemometric indicators commonly used for assessing the quality of the prediction of soil attributes by NIR spectroscopy. TrAC Trends in Analytical Chemistry, 29(9), pp.1073-1081.
The closely related deviation metric:
# Supply truth and predictions as bare column names rpd(solubility_test, solubility, prediction)#> # A tibble: 1 x 3 #> .metric .estimator .estimate #> <chr> <chr> <dbl> #> 1 rpd standard 2.88library(dplyr) set.seed(1234) size <- 100 times <- 10 # create 10 resamples solubility_resampled <- bind_rows( replicate( n = times, expr = sample_n(solubility_test, size, replace = TRUE), simplify = FALSE ), .id = "resample" ) # Compute the metric by group metric_results <- solubility_resampled %>% group_by(resample) %>% rpd(solubility, prediction) metric_results#> # A tibble: 10 x 4 #> resample .metric .estimator .estimate #> <chr> <chr> <chr> <dbl> #> 1 1 rpd standard 2.78 #> 2 10 rpd standard 2.87 #> 3 2 rpd standard 3.04 #> 4 3 rpd standard 3.41 #> 5 4 rpd standard 3.02 #> 6 5 rpd standard 2.66 #> 7 6 rpd standard 2.81 #> 8 7 rpd standard 2.61 #> 9 8 rpd standard 3.45 #> 10 9 rpd standard 2.93#> # A tibble: 1 x 1 #> avg_estimate #> <dbl> #> 1 2.96