Financial risk managers routinely use non–linear time series models to predict the downside risk of the capital under management. They also need to evaluate the adequacy of their model using so–called backtesting procedures. The latter involve hypothesis testing and evaluation of loss functions. This paper shows how the R package GAS can be used for both the dynamic prediction and the evaluation of downside risk. Emphasis is given to the two key financial downside risk measures: Value-at-Risk (VaR) and Expected Shortfall (ES). High-level functions for: (i) prediction, (ii) backtesting, and (iii) model comparison are discussed, and code examples are provided. An illustration using the series of log–returns of the Dow Jones Industrial Average constituents is reported.
The GAS package of Catania et al. (2016) provides a complete framework for modeling, estimating and predicting time series processes for which the time variation in the parameters is driven by the score of the conditional density function. This increasingly popular class of score-driven models has been introduced by Creal et al. (2013) and Harvey (2013). Ardia et al. (2019) describe the general functionality implemented in the GAS package, but do not cover the functionality useful for the estimation and backtesting of Value-at-Risk (VaR) and Expected Shortfall (ES), which are the two leading risk measures used in finance. The aim of this paper is to show how the functions available in the GAS package can be used for VaR and ES evaluation, prediction, and backtesting.
The economic relevance of this topic follows partly from the Basel
Accords (currently the Basel III Accords), which impose that banks and
financial institutions have to meet capital requirements, and must rely
on state-of-the-art risk systems. In particular, they must assess the
uncertainty about the future values of their portfolios and estimate the
extent and the likelihood of potential losses using a risk measure.
Nowadays, VaR and ES risk measures are the standards (Jorion 1997). For
an asset (or portfolio) return, the VaR at a given time horizon equals
the return such that lower returns only occur with a given probability
level
The estimation of the VaR and ES thus requires first to accurately
estimate the conditional distribution of the future portfolios’ or
assets’ returns. Formally, assuming a continuous cumulative density
function (cdf) with time-varying parameters
Formally, in GAS models the vector of time-varying parameters,
We focus on the three major steps practitioners involved in risk management face during their job: (i) prediction of future downside risk, (ii) backtesting, and (iii) comparison with alternative models. The empirical part of the article deals with these three points from an applied perspective while the computational part details the GAS functionalities devoted to downside risk.
Financial returns exhibit several stylized facts that need to be taken into consideration to produce reliable risk forecasts. Empirically, the distribution of returns is (left) skewed and fat tailed, and its variance is time varying (i.e., returns exhibit the so-called volatility clustering); see, for example, McNeil et al. (2015).
To account for these features, we consider a very flexible
specification, in which we assume that the log return at time
(See the Appendix for the score in these cases.) Given a series of
The recursive method of forecasting is usually employed to backtest the adequacy of a statistical model, as well as to perform models comparisons in terms of VaR and ES predictions (Marcellino et al. 2006). The objective of a backtesting analysis is to verify the precision of the prediction by separating the estimation window and the evaluation period. The objective of a model comparison analysis is usually to order models according to a loss function.
To this end, the full sample of
Once a series of VaR predictions is available, forecasts adequacy is
assessed through backtesting procedures. VaR backtesting procedures
usually check the correct coverage of the unconditional and conditional
left-tail of the log-returns distribution. Correct unconditional
coverage (UC) was first considered by Kupiec (1995), while correct
conditional coverage (CC) by Christoffersen (1998). The main difference
between UC and CC concerns the distribution we are focusing on. For
instance, UC considers correct coverage of the left-tail of the
unconditional log-return distribution,
The DQ test by Engle and Manganelli (2004) assesses the joint hypothesis that
Real world applications consider several models for VaR prediction. If
correct unconditional/conditional coverage is achieved by more than one
model, the practitioner faces the problem of not being able to choose
between different alternatives. In this situation, model comparison
techniques are used to choose the best performing model. Model ranking
is achieved thanks to the definition of a loss function. Among several
available loss functions for quantile prediction
(McAleer and Da Veiga 2008), the Quantile Loss (QL) used for quantile
regressions (Koenker and Bassett 1978) is one of the most frequent choices
in the VaR context; see González-Rivera et al. (2004). Formally, given a
VaR prediction at risk level
The quantile loss function in (2) for VaR assessment is
appropriate since quantiles are elicited by it, that is, when the
conditional distribution is static over the sample, the VaR can be
estimated by minimizing the average quantile loss function.
Unfortunately, there is no loss function available for which the ES risk
measure is elicitable; see, for instance, Bellini and Bignozzi (2015) and Ziegel (2016).
However, it has been recently shown by Fissler and Ziegel (2016) (FZ) that the
couple (VaR, ES) is jointly elicitable, as the values of
Summarizing, given a set of available models, a typical downside risk forecasting exercise consists of the following three major steps:
The next section is devoted to detailing the implementation of each of these steps with the GAS package.
We first briefly review how to make predictions with GAS models using
the GAS package. The main illustration is for the last dji30ret
, which is a dataframe consisting of the thirty
Dow Jones Industrial Average constituents:
> library("GAS")
> data("dji30ret", package = "GAS")
> dji30ret <- tail(dji30ret, 2500)
The corresponding returns are shown as gray points in Figure 1. We see the time-variation in the volatility of the daily return series, which we model next using the GAS specification with skewed Student-t innovations, as previously described. Thanks to the GAS package, it is straightforward to estimate the GAS model on rolling windows of the available data and to make one-step ahead rolling forecasts.
Specifically, the user needs to specify the model through the
UniGASSpec()
function, and then perform rolling predictions with the
UniGASRoll()
function. In the GAS package, models are specified
through the definition of the conditional distribution assumed for the
data, Dist
, and the list of time-varying parameters, GASpar
.
Dist
is a character equal to the label of the distribution. For
instance, "sstd"
, "std"
, and "norm"
; see Table 1 of Ardia et al. (2019)
for the list of distributions and associated labels available in the
GAS package.
GASPar
is a list
with named boolean
elements. Entries name are:
location
, scale
, skewness
, and shape
. These indicate whether the
associated distribution parameters are time varying or not. By default
we have
GASPar = list(location = FALSE, scale = TRUE, skewness = FALSE, shape = FALSE)
,
that is, only volatility is time varying. For instance, in order to
specify the three GAS models: GAS–
> GASSpec_N <- UniGASSpec(Dist = "norm", GASPar = list(scale = TRUE))
> GASSpec_ST <- UniGASSpec(Dist = "std", GASPar = list(scale = TRUE))
> GASSpec_SKST <- UniGASSpec(Dist = "sstd", GASPar = list(scale = TRUE))
UniGASSpec()
delivers an object of the class "uGASSpec"
which comes
with several methods; see help("UniGASSpec")
.
The UniGASRoll()
function accepts an object of the class "uGASSpec"
,
"GASSpec"
, a numeric
vector for the series of returns, data
, and
other arguments, such as:
ForecastLength
;RefitWindow
;RefitEvery
,among others; see help("UniGASRoll")
. ForecastLength
and
RefitEvery
are numeric
elements while RefitWindow
is a character
equal to "moving"
(the default) for a rolling window scheme or
"recursive"
for an expanding window. As previously mentioned, in this
paper we consider the case RefitWindow = "moving"
. We fix the length
of the out-of-sample period to
> library("parallel")
> cluster <- makeCluster(2)
> H <- 1000
> Roll_N <- UniGASRoll(dji30ret[, "GE"], GASSpec_N, RefitEvery = 5,
cluster = cluster, ForecastLength = H)
We have also made use of parallel processing (with 2 cores) through the
definition of a cluster
object exploiting the parallel package
included in R since version 2.14.0.
The output of UniGASRoll()
is an object of the class "uGASRoll"
which comes with several methods; see help("UniGASRoll")
.
VaR and ES one-step ahead rolling forecasts at the risk level
Roll_N
using the quantile
and
ES
methods, respectively:
> alpha <- 0.01
> VaR_N <- quantile(Roll_N, probs = alpha)
> ES_N <- ES(Roll_N, probs = alpha)
VaR_N
and ES_N
are matrices of dimension probs
argument in quantile
and ES
can also be a
numeric
vector of VaR_N
and ES_N
would be a matrix
.quantile
and
ES
methods. To compute multi-step ahead predictions an object of class
"uGASFor"
delivered by the function UniGASFor
has to be provided;
see help("UniGASFor")
.
Predictions for the GAS–GASSpec_ST
and GASSpec_SKST
model
definitions are computed analogously. Figure 1 reports 1%
VaR predictions delivered by the GAS–
Let us now show how the accuracy of the VaR forecasts can be evaluated
using the BacktestVaR()
function in the GAS package. This function
accepts the following arguments:
data
, numeric
containing the out-of-sample data;VaR
, numeric
containing the series of VaR forecasts;alpha
, the VaR risk level Lags
, the number of lags used in the DQ test, by default
Lags = 4
; see Engle and Manganelli (2004).The function returns a list
with named entries:
LRuc
, the test statistic and associated LRcc
, the test statistic and associated DQ
, the test statistic and associated Loss
, the quantile loss (QL), as defined in ((2)),
together with the average QL used by González-Rivera et al. (2004);AD
, the mean and max VaR Absolute Deviation (AD) used by
McAleer and Da Veiga (2008);AE
, the Actual over Expected ratio.For instance, in order to compute the VaR backtest measures defined
above on the forecast series VaR_N
, we use:
> VaRBacktest_N <- BacktestVaR(data = tail(dji30ret[, "GE"], H), VaR = VaR_N, alpha = alpha)
Then, the DQ test statistic and its associated
> VaRBacktest_N$DQ
$stat
[,1]
[1,] 52.47578
$pvalue
[,1]
[1,] 4.7043e-09
which, in this case, is against the null of correct model specification for the 1% VaR level.
Now, if we evaluate VaR forecasts using the GAS–VaRBacktest_ST
object, then the DQ test
reports:
> VaRBacktest_ST$DQ
$stat
[,1]
[1,] 8.763418
$pvalue
[,1]
[1,] 0.270091
Asset | GAS– |
GAS– |
GAS– |
GAS– |
GAS– |
GAS– |
AA | 0.00 | 0.17 | 0.16 | 0.00 | 0.00 | 0.00 |
AIG | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
AXP | 0.09 | 0.99 | 0.25 | 0.19 | 0.13 | 0.15 |
BA | 0.29 | 0.98 | 0.30 | 0.13 | 0.55 | 0.54 |
BAC | 0.00 | 0.00 | 0.06 | 0.00 | 0.03 | 0.06 |
C | 0.00 | 0.00 | 0.00 | 0.01 | 0.00 | 0.00 |
CAT | 0.00 | 0.22 | 0.22 | 0.00 | 0.38 | 0.51 |
CVX | 0.00 | 0.02 | 0.00 | 0.02 | 0.03 | 0.01 |
DD | 0.03 | 0.41 | 0.34 | 0.35 | 0.31 | 0.13 |
DIS | 0.00 | 0.05 | 0.08 | 0.00 | 0.35 | 0.15 |
GE | 0.00 | 0.04 | 0.00 | 0.16 | 0.30 | 0.03 |
GM | 0.00 | 0.06 | 0.02 | 0.02 | 0.29 | 0.17 |
HD | 0.01 | 0.01 | 0.10 | 0.09 | 0.63 | 0.57 |
HPQ | 0.00 | 0.01 | 0.03 | 0.00 | 0.11 | 0.35 |
IBM | 0.06 | 0.03 | 0.03 | 0.00 | 0.04 | 0.05 |
INTC | 0.04 | 0.07 | 0.33 | 0.06 | 0.10 | 0.06 |
JNJ | 0.95 | 1.00 | 0.35 | 0.00 | 0.00 | 0.00 |
JPM | 0.18 | 0.95 | 0.21 | 0.27 | 0.42 | 0.17 |
KO | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
MCD | 0.11 | 0.68 | 0.73 | 0.10 | 0.17 | 0.40 |
MMM | 0.00 | 0.34 | 0.00 | 0.00 | 0.18 | 0.33 |
MRK | 0.00 | 0.05 | 0.06 | 0.00 | 0.11 | 0.05 |
MSFT | 0.11 | 0.28 | 0.42 | 0.05 | 0.43 | 0.52 |
PFE | 0.15 | 0.37 | 0.38 | 0.13 | 0.81 | 0.75 |
PG | 0.00 | 0.18 | 0.07 | 0.13 | 0.11 | 0.06 |
T | 0.35 | 0.37 | 0.42 | 0.02 | 0.01 | 0.06 |
UTX | 0.38 | 0.41 | 0.07 | 0.09 | 0.90 | 0.93 |
VZ | 0.04 | 0.99 | 0.96 | 0.88 | 0.70 | 0.83 |
WMT | 0.00 | 0.00 | 0.01 | 0.93 | 0.54 | 0.26 |
XOM | 0.00 | 0.00 | 0.00 | 0.07 | 0.04 | 0.17 |
The large
We reproduce this analysis for each of the thirty daily stock return
series in the dji30ret
data set previously detailed. The out-of-sample
period starts on February 14, 2005, and includes the recent Global
Financial Crisis of 2007-2008. We consider two VaR risk levels:
Table 1 reports the
The goal of VaR model comparison is to rank the models in terms of
accuracy of their VaR forecasts. This can be done using the average
value of the quantile loss in (2), as available in the output
from the BacktestVaR()
function.
In our application on the daily stock returns of General Electric, we
find that the model comparison in terms of average QL also favors
GAS–
> round(VaRBacktest_ST$Loss$Loss / VaRBacktest_N$Loss$Loss, 2)
[1] 0.94
This indicates that GAS–
The left part of Table 2 (QL ratios) reports the results of
repeating this model comparison analysis for all thirty daily stock
return series in the dji30ret
dataset.
QL ratios | FZL ratios | |||||||
Asset | GAS– |
GAS– |
GAS– |
GAS– |
GAS– |
GAS– |
GAS– |
GAS– |
AA | 0.93 | 0.93 | 0.98 | 0.98 | 0.95 | 0.95 | 0.98 | 0.98 |
AIG | 1.00 | 1.01 | 1.02 | 1.03 | 0.86 | 0.87 | 0.93 | 0.94 |
AXP | 0.93 | 0.94 | 0.98 | 0.99 | 0.94 | 0.96 | 0.98 | 1.00 |
BA | 0.99 | 1.00 | 0.99 | 0.99 | 0.98 | 0.99 | 0.99 | 0.99 |
BAC | 0.83 | 0.85 | 1.01 | 1.01 | 0.88 | 0.88 | 0.98 | 0.98 |
C | 0.97 | 1.00 | 1.02 | 1.03 | 0.94 | 0.95 | 0.98 | 0.99 |
CAT | 0.82 | 0.83 | 0.91 | 0.91 | 0.83 | 0.84 | 0.91 | 0.91 |
CVX | 1.00 | 1.18 | 1.01 | 1.06 | 0.99 | 1.20 | 1.01 | 1.09 |
DD | 0.94 | 0.94 | 0.98 | 0.98 | 0.95 | 0.97 | 0.98 | 0.98 |
DIS | 0.98 | 0.99 | 1.00 | 1.00 | 0.95 | 0.96 | 0.99 | 1.00 |
GE | 0.95 | 0.96 | 0.97 | 0.98 | 0.96 | 0.99 | 0.98 | 0.98 |
GM | 0.87 | 0.88 | 0.93 | 0.94 | 0.91 | 0.93 | 0.96 | 0.97 |
HD | 1.02 | 1.00 | 1.00 | 1.00 | 1.01 | 1.00 | 1.00 | 1.00 |
HPQ | 0.95 | 0.95 | 0.94 | 0.94 | 0.95 | 0.95 | 0.95 | 0.95 |
IBM | 1.00 | 1.00 | 0.94 | 0.94 | 0.92 | 0.93 | 0.93 | 0.93 |
INTC | 0.97 | 0.97 | 0.96 | 0.96 | 0.94 | 0.94 | 0.96 | 0.96 |
JNJ | 1.03 | 1.01 | 1.02 | 1.00 | 1.00 | 0.99 | 1.01 | 0.99 |
JPM | 0.92 | 0.92 | 0.98 | 0.99 | 0.94 | 0.95 | 0.99 | 0.99 |
KO | 0.97 | 0.96 | 0.96 | 0.96 | 0.97 | 0.96 | 0.93 | 0.92 |
MCD | 1.00 | 0.99 | 0.99 | 0.99 | 0.99 | 0.99 | 0.99 | 0.99 |
MMM | 0.82 | 0.83 | 0.90 | 0.90 | 0.80 | 0.82 | 0.89 | 0.89 |
MRK | 0.83 | 0.84 | 0.89 | 0.89 | 0.81 | 0.82 | 0.88 | 0.89 |
MSFT | 0.86 | 0.87 | 0.90 | 0.90 | 0.87 | 0.90 | 0.91 | 0.91 |
PFE | 0.83 | 0.84 | 0.91 | 0.91 | 0.83 | 0.83 | 0.90 | 0.91 |
PG | 0.86 | 0.87 | 0.95 | 0.95 | 0.82 | 0.85 | 0.92 | 0.93 |
T | 0.95 | 0.97 | 1.00 | 1.00 | 0.97 | 0.98 | 0.99 | 0.99 |
UTX | 0.93 | 0.94 | 0.94 | 0.94 | 0.93 | 0.94 | 0.92 | 0.92 |
VZ | 0.91 | 0.91 | 0.98 | 0.98 | 0.93 | 0.94 | 0.97 | 0.97 |
WMT | 0.97 | 0.97 | 0.99 | 1.00 | 0.96 | 0.97 | 0.99 | 1.00 |
XOM | 1.02 | 0.96 | 1.02 | 1.01 | 0.97 | 0.92 | 1.00 | 0.99 |
The FZLoss
function implements the FZ loss and accepts the following
arguments:
data
: vector of observations;VaR
: vector of VaR predictions;ES
: vector of ES predictions;alpha
: the This function returns a numeric
vector of the same size of data
with
the FZ losses computed at each point in time. For instance, using the
VaR_N
and ES_N
vectors previously computed we can evaluate the
associated FZ loss as:
> FZL <- FZLoss(data = tail(dji30ret[, "GE"], H), VaR = VaR_N, ES = ES_N,
alpha = alpha)
where FZL
is a numeric
vector of length H
.
The right part of Table 2 (FZL ratios) reports the results
of performing model comparison analysis for all thirty daily stock
return series in the dji30ret
dataset according to the FZ loss
reported in (3). It shows, for both
Under the regulation of the Basel Accords, risk managers of financial institutions need to rely on state-of-the-art methodologies for predicting and evaluating their downside risk (Board of Governors of the Federal Reserve Systems 2012). This article illustrates the usefulness of the R package GAS in putting the theory of modern downside risk management into practice. The recommended strategy consists of four steps: (i) model specification, (ii) downside risk predictions, (iii) backtesting, and (iv) model comparison. We illustrate this proposed implementation in R using the package GAS applied to the Value-at-Risk and Expected Shortfall estimation for the daily returns of the thirty Dow Jones Industrial Average constituents.
The results in this paper were obtained using R 3.5.0 with the package
GAS version 0.2.8 available on CRAN
at
https://cran.r-project.org/package=GAS. Computations were performed on
Windows 7 x64 (build 7601) Service Pack 1, x86_64-w64-mingw32/x64
(64-bit) with Intel(R) Xeon(R) CPU E5-2560 v3 2.30 GHz.
The authors thank the Editor-in-Chief, John Verzani, two Executive Editors, Roger Bivand and Norman Matloff, the anonymous reviewer, Alexios Ghalanos, participants at the R/Finance conference 2017 in Chicago, and participants at the session "Financial econometrics with R" at the 11th International Conference on Computational and Financial Econometrics 2017 in London. The authors acknowledge Google for financial support via the Google Summer of Code 2016 and 2017 project "GAS".
To obtain the score of the
NumericalMathematics, TimeSeries
This article is converted from a Legacy LaTeX article using the texor package. The pdf version is the official version. To report a problem with the html, refer to CONTRIBUTE on the R Journal homepage.
probs
argument in quantile
and ES
can also be a
numeric
vector of VaR_N
and ES_N
would be a matrix
.[↩]Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
For attribution, please cite this work as
Ardia, et al., "Downside Risk Evaluation with the R Package GAS", The R Journal, 2018
BibTeX citation
@article{RJ-2018-064, author = {Ardia, David and Boudt, Kris and Catania, Leopoldo}, title = {Downside Risk Evaluation with the R Package GAS}, journal = {The R Journal}, year = {2018}, note = {https://rjournal.github.io/}, volume = {10}, issue = {2}, issn = {2073-4859}, pages = {410-421} }