Machine learning models have successfully been applied to challenges in applied in biology, medicine, finance, physics, and other fields. With modern software it is easy to train even a complex model that fits the training data and results in high accuracy on test set. However, problems often arise when models are confronted with the real-world data. This paper describes methodology and tools for model-agnostic auditing. It provides functinos for assessing and comparing the goodness of fit and performance of models. In addition, the package may be used for analysis of the similarity of residuals and for identification of outliers and influential observations. The examination is carried out by diagnostic scores and visual verification. The code presented in this paper are implemented in the auditor package. Its flexible and consistent grammar facilitates the validation models of a large class of models.
Supplementary materials are available in addition to this article. It can be downloaded at RJ-2019-036.zip
Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
For attribution, please cite this work as
Gosiewska & Biecek, "auditor: an R Package for Model-Agnostic Visual Validation and Diagnostics", The R Journal, 2019
BibTeX citation
@article{RJ-2019-036,
author = {Gosiewska, Alicja and Biecek, Przemysław},
title = {auditor: an R Package for Model-Agnostic Visual Validation and Diagnostics},
journal = {The R Journal},
year = {2019},
note = {https://doi.org/10.32614/RJ-2019-036},
doi = {10.32614/RJ-2019-036},
volume = {11},
issue = {2},
issn = {2073-4859},
pages = {85-98}
}