RobustCalibration: Robust Calibration of Computer Models in R

Two fundamental research tasks in science and engineering are forward predictions and data inversion. This article introduces a new R package RobustCalibration for Bayesian data inversion and model calibration using experiments and field observations. Mathematical models for forward predictions are often written in computer code, and they can be computationally expensive to run. To overcome the computational bottleneck from the simulator, we implemented a statistical emulator from the RobustGaSP package for emulating both scalar-valued or vector-valued computer model outputs. Both posterior sampling and maximum likelihood approach are implemented in the RobustCalibration package for parameter estimation. For imperfect computer models, we implement the Gaussian stochastic process and scaled Gaussian stochastic process for modeling the discrepancy function between the reality and mathematical model. This package is applicable to various other types of field observations and models, such as repeated experiments, multiple sources of measurements and correlated measurement bias. We discuss numerical examples of calibrating mathematical models that have closed-form expressions, and differential equations solved by numerical methods.

Mengyang Gu (University of California, Santa Barbara)

P. Agram and M. Simons. A noise model for InSAR time series. Journal of Geophysical Research: Solid Earth, 120(4): 2752–2771, 2015.
K. R. Anderson, I. A. Johanson, M. R. Patrick, M. Gu, P. Segall, M. P. Poland, E. K. Montgomery-Brown and A. Miklius. Magma reservoir failure and the onset of caldera collapse at Kı̄lauea volcano in 2018. Science, 366(6470): 2019.
P. D. Arendt, D. W. Apley and W. Chen. Quantification of model uncertainty: Calibration, model discrepancy, and identifiability. Journal of Mechanical Design, 134(10): 100908, 2012.
M. J. Bayarri, J. O. Berger, R. Paulo, J. Sacks, J. A. Cafeo, J. Cavendish, C.-H. Lin and J. Tu. A framework for validation of computer models. Technometrics, 49(2): 138–154, 2007a.
M. Bayarri, J. Berger, J. Cafeo, G. Garcia-Donato, F. Liu, J. Palomo, R. Parthasarathy, R. Paulo, J. Sacks and D. Walsh. Computer model validation with functional output. The Annals of Statistics, 35(5): 1874–1906, 2007b.
G. Box and G. Coutie. Application of digital computers in the exploration of functional relationships. Proceedings of the IEE-Part B: Radio and Electronic Engineering, 103(1S): 100–107, 1956.
J. Brajard, A. Carrassi, M. Bocquet and L. Bertino. Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: A case study with the Lorenz 96 model. Journal of Computational Science, 44: 101171, 2020.
M. Carmassi, P. Barbillon, M. Chiodetti, M. Keller and E. Parent. CaliCo: A R package for Bayesian calibration. arXiv preprint arXiv:1808.01932, 2018.
W. Chang, M. Haran, P. Applegate and D. Pollard. Calibrating an ice sheet model using high-dimensional binary spatial data. Journal of the American Statistical Association, 111(513): 57–72, 2016.
W. Chang, B. A. Konomi, G. Karagiannis, Y. Guan and M. Haran. Ice model calibration using semicontinuous spatial data. The Annals of Applied Statistics, 16(3): 1937–1961, 2022.
M. Gu. Jointly robust prior for Gaussian stochastic process in emulation, calibration and variable selection. Bayesian Analysis, 14(1): 2019.
M. Gu, K. Anderson and E. McPhillips. Calibration of imperfect geophysical models by multiple satellite interferograms with measurement bias. Technometrics, In Press, 2023. DOI 10.1080/00401706.2023.2182365.
M. Gu, J. Palomo and J. O. Berger. RobustGaSP: Robust Gaussian Stochastic Process Emulation in R. The R Journal, 11(1): 112–136, 2019. DOI 10.32614/RJ-2019-011.
M. Gu and L. Wang. Scaled Gaussian stochastic process for computer model calibration and prediction. SIAM/ASA Journal on Uncertainty Quantification, 6(4): 1555–1583, 2018.
M. Gu, F. Xie and L. Wang. A theoretical framework of the scaled Gaussian stochastic process in prediction and calibration. SIAM/ASA Journal on Uncertainty Quantification, 10(4): 1435–1460, 2022.
R. K. Hankin. Introducing BACCO, an R bundle for Bayesian analysis of computer code output. Journal of Statistical Software, 14: 1–21, 2005.
D. Higdon, J. Gattiker, B. Williams and M. Rightley. Computer model calibration using high-dimensional output. Journal of the American Statistical Association, 103(482): 570–583, 2008.
M. C. Kennedy and A. O’Hagan. Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(3): 425–464, 2001.
D. C. Liu and J. Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical programming, 45(1-3): 503–528, 1989.
F. Liu, M. Bayarri and J. Berger. Modularization in Bayesian analysis, with emphasis on analysis of computer models. Bayesian Analysis, 4(1): 119–150, 2009.
E. N. Lorenz. Predictability: A problem partly solved. In Proc. Seminar on predictability, 1996.
B. MacDonald, P. Ranjan, H. Chipman, et al. GPfit: An R package for fitting a Gaussian process model to deterministic simulator outputs. Journal of Statistical Software, 64(i12): 2015.
J. Maclean and E. T. Spiller. A surrogate-based approach to nonlinear, non-Gaussian joint state-parameter data assimilation. arXiv preprint arXiv:2012.04793, 2020.
J. Palomo, R. Paulo, G. Garcı́a-Donato, et al. SAVE: An R package for the statistical analysis of computer models. Journal of Statistical Software, 64(13): 1–23, 2015.
R. Paulo, G. Garcı́a-Donato and J. Palomo. Calibration of computer models with multivariate output. Computational Statistics and Data Analysis, 56(12): 3959–3974, 2012.
M. Plumlee. Bayesian calibration of inexact computer models. Journal of the American Statistical Association, 112(519): 1274–1285, 2017.
O. Roustant, D. Ginsbourger and Y. Deville. DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization. Journal of statistical software, 51: 1–55, 2012.
J. Sacks, W. J. Welch, T. J. Mitchell, H. P. Wynn, et al. Design and analysis of computer experiments. Statistical science, 4(4): 409–423, 1989.
T. J. Santner, B. J. Williams and W. I. Notz. The design and analysis of computer experiments. Springer Science & Business Media, 2003.
N. A. Simakov, R. L. Jones-Ivey, A. Akhavan-Safaei, H. Aghakhani, M. D. Jones and A. K. Patra. Modernizing Titan2D, a parallel AMR geophysical flow code to support multiple rheologies and extendability. In International conference on high performance computing, pages. 101–112 2019. Springer.
K. Soetaert, T. Petzoldt and R. W. Setzer. Solving differential equations in R: Package deSolve. Journal of statistical software, 33: 1–25, 2010.
R. Tuo and C. J. Wu. Efficient calibration for imperfect computer models. The Annals of Statistics, 43(6): 2331–2352, 2015.
H. Wickham. ggplot2. Wiley Interdisciplinary Reviews: Computational Statistics, 3(2): 180–185, 2011.
D. Williamson, M. Goldstein, L. Allison, A. Blaker, P. Challenor, L. Jackson and K. Yamazaki. History matching for exploring and reducing climate model parameter space using observations and a large perturbed physics ensemble. Climate dynamics, 41(7-8): 1703–1729, 2013.
R. K. Wong, C. B. Storlie and T. Lee. A frequentist approach to computer model calibration. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79: 635–648, 2017.
J. Ypma. Nloptr: R interface to NLopt. 2014. URL R package version 1.0.4.
H. A. Zebker, P. A. Rosen and S. Hensley. Atmospheric effects in interferometric synthetic aperture radar surface deformation and topographic maps. Journal of Geophysical Research: Solid Earth, 102(B4): 7547–7563, 1997. URL



Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".


For attribution, please cite this work as

Gu, "RobustCalibration: Robust Calibration of Computer Models in R", The R Journal, 2024

BibTeX citation

  author = {Gu, Mengyang},
  title = {RobustCalibration: Robust Calibration of Computer Models in R},
  journal = {The R Journal},
  year = {2024},
  note = {},
  doi = {10.32614/RJ-2023-085},
  volume = {15},
  issue = {4},
  issn = {2073-4859},
  pages = {84-105}