GREENeR: An R Package to Estimate and Visualize Nutrients Pressures on Surface Waters

Nutrient pollution affects fresh and coastal waters around the globe. Planning mitigating actions requires tools to assess fluxes of nutrient emissions to waters and expected restoration impacts. Conceptual river basin models take advantage of data on nutrient emissions and concentrations at monitoring stations, providing a physical interpretation of monitored conditions, and enabling scenario analysis. The GREENeR package streamlines water quality model in a region of interest, considering nutrient pathways and the hydrological structure of the river network. The package merges data sources, analyzes local conditions, calibrate the model, and assesses yearly nutrient levels along the river network, determining contributions of load in freshwaters from diffuse and point sources. The package is enriched with functions to perform thorough parameter sensitivity analysis and for mapping nutrient sources and fluxes. The functionalities of the package are demonstrated using datasets from the Vistula river basin.

Angel Udías (European Commission, Joint Research Centre (JRC)) , Bruna Grizzetti (European Commission, Joint Research Centre (JRC)) , Olga Vigiak (European Commission, Joint Research Centre (JRC)) , Alberto Aloe (European Commission, Joint Research Centre (JRC)) , Cesar Alfaro (Departament of Computer Science and Statistics, Rey Juan Carlos University) , Javier Gomez (Departament of Computer Science and Statistics, Rey Juan Carlos University)
01-03-2024

1 Introduction

Nitrogen and phosphorus are key nutrients that heavily impact aquatic ecosystems. Detecting primary sources of nutrient pollution and their downstream spread, alongside assessing achievable reductions through restoration policies, are crucial for effective natural resource management. They aid in identifying priority intervention areas and planning actions to restore the ecological balance of receiving waters.

Modelling tools can be useful to assess the impacts of future scenarios, policy measures, and climate changes at the regional and continental scale (Seitzinger et al. 2005; Ludwig et al. 2010; Arheimer et al. 2012; Bouraoui et al. 2014; Bartosova et al. 2019; Beusen et al. 2022), and to check the coherence of different policy targets, for instance between water and agricultural policies. The assessment of policy scenarios requires a flexible and spatially detailed analysis to account for climatic, hydrological, and socio-economic gradients (Grizzetti et al. 2021).

Several types of models can be applied to predict the transport of nutrients in river basins Fu et al. (2019). Among them, statistical or conceptual models have the advantage of being readily applied in large watersheds. These model rely on calibration of few parameters for establishing links between emissions at sources and fate in the stream network. They take full advantage of nutrient emissions and concentrations data at monitoring stations, which are now accessible with increasing spatial and temporal resolution.

A classic example of river basin conceptual model is SPARROW (Smith et al. 1997; Schwarz et al. 2006), which is widely applied in the U.S. to assess water quality over large regions. SPARROW has inspired the Geospatial Regression Equation for European Nutrient (GREEN) losses model (Grizzetti et al. 2005; Grizzetti et al. 2012, 2021) which is adapted to European conditions. GREEN has been used for assessing the nutrient loads to the European seas (Grizzetti et al. 2012, 2021), nitrogen retention in European freshwaters (Grizzetti et al. 2012; La Notte et al. 2017), and for policy scenario analysis (Bouraoui et al. 2014; Leip et al. 2015; La Notte et al. 2017; Malagó et al. 2019; Grizzetti et al. 2021). Despite their usefulness, this type of river basin models are seldom compiled as dedicated R packages, for example SPARROW has some scripts available to process information for and analysis of models results in the R environment.

The GREEN model application comprises several key steps, including data extraction and organization, data harmonization and integration, examination and validation of input data sets, model calibration and parameter selection, model run, and result visualization. All of these features are now integrated into an R package (GREENeR) that comprises functions to streamline the process, evaluate and visualize all the steps, thus strengthening the robustness of model application.

1.1 About water surface nutrients estimation with GREEN

GREEN (Grizzetti et al. 2005, 2008; Grizzetti et al. 2012, 2021) is a conceptual model to assess total nitrogen TN and total phosphorous TP from a region of interest (usually a river basin), accounting for both diffuse and point sources. The package allows the analysis of different scenarios of nutrient input in the region of interest, where “scenario” indicates a combination of annual time-series of inputs, such as nitrogen or phosphorus and the topological structure of the region. The model comprises several nutrient sources and pathways:

  1. agricultural diffuse sources include nutrients from mineral fertilisers and manure application, nitrogen crop and soil fixation. These sources undergo retention in the land phase (basin retention that account for e.g. crop uptake and volatilization losses) before reaching the stream;
  2. other diffuse emissions, such as from scattered dwellings (i.e., isolated houses and small agglomerations that are not connected to sewerage systems), and atmospheric nitrogen deposition (for nitrogen module) or background losses (for phosphorus module), are also reduced, e.g. due to soil processes, before reaching the stream network;
  3. point sources consist of urban and industrial wastewater discharges that are discharged into surface waters directly.

Once in the river network, all nutrient loads are reduced by in-stream retention in rivers and lakes.

Basin retention of agriculture sources is a decay function proportional to the inverse of the total annual precipitation in the catchment. Conversely, river retention is a decay function proportional to the river length, considered as a proxy for water residence time. Finally, lake retention is simulated as a function of the lakes residence time and average depth.

The basin is divided into spatial subunits (called catchments), with a given area, a river reach, an inlet node, and an outlet node. The catchments are topologically connected from the headwaters to the outlet in a cascading sequence. The sequence of nutrient load accumulation through the stream network is defined by Shreve’s order (Shreve 1966). Nutrient input from the different sources, basin and river retention are simulated in each catchment and routed through the river network. For each catchment \(i\) in the basin, the GREEN nutrient load \(L_i\) is estimated by the general equation:

\[\begin{equation} L_{i,y} = (1-Lret_i) \cdot (DSA_{i,y} \cdot (1-Bret_{i,y} )+ DSB_{i,y} + PS_{i,y} +U_{i,y} ) \cdot (1-Rret_i) \tag{1} \end{equation}\]

where \(L_{i,y}\) is the nutrient load at the catchment outlet-node (ton/yr) in \(y\) year; and the other variables represent different sources and sinks of nutrients.

Sources of nutrients are:

Sinks of nutrients are:

\[\begin{equation} Lret = \max\left(0.1, 1 - \frac{1}{1 + (dref / z) \cdot RT} \right) \tag{2} \end{equation}\]

where \(z\) represents the average lake depth (m); \(RT\) is the hydraulic residence time (yr); and \(dref\) denotes a nutrient-related coefficient (\(dref = 7.3\) for TN and \(dref = 26\) for TP).

\[\begin{equation} Bret_{i,y} = 1 - \exp(-alpha_P \cdot NrmInvRain_{i,y}) \tag{3} \end{equation}\]

where \(NrmInvRain_{i,y}\) is the inverse of annual precipitation (mm) of the \(i\) catchment in \(y\) year, normalized by its maximum (Frank and Todeschini 1994). To keep basin retention coefficients comparable across regions, the minimum precipitation \(minPrec\) is set to \(50\) mm/y, and thus \(NrmInvRain_{i,y}\) is defined as

\[\begin{equation*} NrmInvRain_{i,y} = \frac{1 / \max(50, precipitation_{i,y})}{(1 / minPrec)} \end{equation*}\] \[\begin{equation} Rret_i = 1 - \exp(-alpha_L \cdot NrmLengthKm_i) \tag{4} \end{equation}\]

where \(NrmLengthKm_i\) is the length (km) of the catchment reach, normalized by the maximum in the dataset:

\[\begin{equation*} NrmLengthKm_i = \frac{\textrm{catchment length } i \textrm{ reach, in km}}{\max(\textrm{Reach length in the region, in km})} \end{equation*}\]

Since the maximum reach length depends on the region and its subdivision of network reaches, the calibrated coefficient \(alpha_L\) cannot be compared across regions that adopt different discretization. Note that basin retention varies from year to year according to annual precipitation, whereas the fraction of river and lake retention for a given catchment is constant in time.

Equation (1) is applied sequentially from the most upstream nodes to the basin outlet. The model parameters are:

  1. the basin retention coefficient (\(alpha_P\)), which together with annual precipitation regulates nutrient retention of diffuse agricultural sources (Equation (3));
  2. the river retention coefficient (\(alpha_L\): Equation (4)), which regulates the river retention per river km.
  3. the fraction of domestic diffuse sources that reaches the stream network (\(sd_{coeff}\)).

2 About the GREENeR package

2.1 Package organization

This work presents a new efficient and enhanced implementation of the model GREEN developed as an R package named GREENeR. GREENeR is developed in the R statistical software and provides tools and methods for applying GREEN to an area of interest and assessing annual time series of nutrient loads in a river network and at the basin outlet, plus contributions of nutrient sources to these loads. Some of the key features of the package comprise: a) functions for the creation of scenarios from data sources; b) computational efficiency; c) parallel-capable ; d) an extended suite of fine tuning options to control model calibration; e) built-in parameter sensitivity analysis; and f) functions to perform customised post-process analyses.

The functions of GREENeR package are arranged in three groups: i) functions to perform graphical summaries of model inputs; ii) functions to perform model parameters calibration and sensitivity analysis; iii) functions to compute, analyze and visualize through graphs and maps the model outputs, i.e. total loads and contributions by source. The package supports parallel processing, which helps reduce the computational load in handling large basins.

The time-series can represent either historic (current or past) conditions that can be associated to observations for model calibration, or hypothetical conditions foreseen under theoretical changes (e.g. to forecast results of nutrient management plans, or under climatic change).

A scenario contains:

  1. The geospatial geometry of the catchments of the region, which currently is defined according to the Catchment Characterisation and Modelling Database for European Rivers v2 (CCM2) (De Jager and Vogt 2007).

  2. The annual time series (1990-2018) of nutrient inputs per catchment.

  3. Additional catchment information (annual precipitation, annual forest fraction, lake retention, reach length).

  4. Observed total load (TN or TP) from monitoring stations.

Two historical scenarios, one for TN and one for TP, of the Lay basin (France) are provided with the GREENeR package. The Lay basin has an area of \(1971\) km\(^2\), and is divided into \(189\) CCM2 catchments. The mean catchment area is \(10.4\) km\(^2\). Nutrient observations comprised \(22\) TN entries from six monitoring stations and \(58\) TP data entries from eight monitoring stations (data from WISE Waterbase (EEA 2021)).

Further, in this article we present examples drawn for the TN historic scenario of the transboundary Vistula (Wisla) river basin, one of the largest river basins of Europe. The Vistula scenario comprises \(15465\) CCM2 catchments that compose the \(193.894\) km\(^2\) river basin, TN inputs for 1990-2018, as well as 1364 observed TN loads from 412 monitoring stations. The TN input dataset is part of the larger dataset created to assess nutrient concentration in the European freshwaters (Vigiak et al. 2023).

2.2 Key functions in the GREENeR package

A brief overview of the package and its main functions is given in Figure 1.

Schematic diagram of the procedure of GREENeR, including functions, data inputs and outputs, to estimate the nutrient loads. The green boxes represent data objects and the blue boxes represent the functions.

Figure 1: Schematic diagram of the procedure of GREENeR, including functions, data inputs and outputs, to estimate the nutrient loads. The green boxes represent data objects and the blue boxes represent the functions.

The key functions included in the package are:

2.3 Input data requirements

GREENeR requires information on the nutrient inputs, the topology, and the geospatial geometry of the region of interest. The spatio-temporal input data must include all nutrient source fields, and differs in the two nutrient scenarios (TN or TP). In the case of TN (e.g. Figure 2), fields are (Equation (5) in Appendix 1):

Maps showing the average TN inputs in the Vistula river basin in 1990-2018. The figure was generated with GREENeR input\_maps() function. Tot.Diff is the sum of diffuse inputs.

Figure 2: Maps showing the mean annual TN inputs in the Vistula river basin in 1990-2018. The figure was generated with GREENeR input_maps() function. TOT.Diff = sum of diffuse inputs.

In the case of TP, fields are (Equation (6) in Appendix 1):

European datasets for 1990-2018 generated to assess historic and current nutrient fluxes (Vigiak et al. 2023) are available upon request. The model and the package are compatible with an external dataset and can be customized with local information as long as the data structure is respected.

In both nutrient cases, GREENeR needs additional annual catchment information:

The catchment topology outlines the hydrological network configuration within the region. Each catchment must have a unique numerical identifier (HydroID)), to establish the network structure via a source HydroID and destination HydroID table. It is important to note that any outlet of the basin will be given as destination HydroID identifier “-1”. Complementing the topology of the hydrologic network, the length of catchment reach should also be included (Equation (4)).

Finally, in order to calibrate the model, it is necessary to have some observed nutrient loads from monitoring stations associated to any catchment of the region. Ultimately, the quality, size, and spatial distribution of observed loads determine the robustness of the calibration process. In Europe, a large dataset of annual concentrations is publicly available (WISE Waterbase, EEA 2021), but annual flow must be derived from other sources (Grizzetti et al. 2021; Vigiak et al. 2023).

2.4 Performance and memory use

The GREENeR package performance and memory usage depends on the size of the region (number of catchments) and the number of years of the simulation. As a reference, the Danube basin (i.e. the largest European basin, with \(138013\) catchments) TN dataset for 1990-2018 requires approximately \(389\) Mb of memory to store the annual data. Table 1 shows the memory occupied by the scenarios for 6 European basins. In addition, it shows the computation times required in the execution of some key functions of the package. All executions were conducted on two computer configurations:

The time required for the execution of the different functions increases linearly with the number of catchments (or Shreve order) of the region (Table 1). The calib_green() function is the most computationally demanding. However, the parallel implementation of this function drastically reduces the time required for this process. Calibration can be carried out in about 6 hours for practical implementation in large basins.

Table 1: Computational requirements to run some GREENeR functions in six European basins scenarios of 29 years. The columns Shreve order, Area and Number of catchments characterize the size of each basin. Mem is the amount of memory occupied by the GREEN model scenario generated by GREENeR package. The green(), green_shares() and calib_green() columns show the computation time required to execute the corresponding GREENeR functions for each scenarios under two computer configurations, CPU1, CPU2 (explanation in the text). All runs of calib_green() have been performed with 200 iterations. The reported time are the average of 5 runs of each process.
green()
green_shares()
calib_green()
Basin name Shreve order Area km\(^2\) Number of catchments Mem MB CPU1 sec CPU2 sec CPU1 sec CPU2 sec CPU1 sec CPU2 sec
Lay 95 1971 189 0.5 6 4 15 12 316 31
Miño 2803 16985 5572 15.7 57 45 147 90 4110 394
Seine 2902 75989 5793 16.3 77 55 177 106 6320 452
Ebro 9351 85611 18568 52.3 160 123 425 224 10876 995
Vistula 7757 193894 15465 43.6 132 100 314 187 9004 814
Danube 69505 802032 138013 388.6 1120 766 2855 1108 94682 5638

3 Estimating nutrient loads using the GREENeR package

The entire procedure is summarized on Figure 1. Once the input data have been uploaded, the region scenario (Nutrient data and Catch data) is generated. The calib_green() function explores the parameter set ranges with LHC scheme, calculating GoF of parameter sets. Sensitivity analysis of its results is conducted to determine the best parameter set. Finally, the green_shares() function is used to estimate the nutrient loads and source apportionment (i.e. contribution to loads by source), per year and per catchment.

3.1 Scenario preparation

Assembling input data for running the GREEN model is time consuming. To facilitate the process, the read_NSdata() function assembles and organises annual information to generate a list of two objects: Nutrient Time Series Data Object and Catch Data Object of GREEN scenario. It needs four CSV files with specific data, namely:

The spatial identifiers (HydroID) and temporal (year) units must be coherent in all the files. Besides organizing the input data, the read_NSdata() function also calculates the Shreve order of each catchment, normalises the precipitation (calculating \(NrmInvRain_{i,y}\)) and the reach length (\(NrmLengthKm_i\)).

csv_path <- "data/csv/"
scen <- read_NSdata(csv_path,"nutrients.csv","monitoring.csv", "forestFr.csv",
                    "precipitation.csv","topology.csv", "lakeProperties.csv",
                    "length.csv")
nutrient <- scen[[1]]
catch    <- scen[[2]]

The shreve() function can calculate the Shreve order independently based on the topology of the basin:

shreve_order <- shreve(catch) 

Finally, the geospatial geometry of the catchments should be uploaded to enable the visualisation functionalities in map form. The read_geometry() function loads the geometry information file, i.e. a geospatial vector with the geometry of the catchment region:

geometry <- read_geometry(file = "data/shapes/Wisla.shp")

The geo-reference information, geometry and attributes of the spatial entities must be in a shapefile format (.shp), editable with ArcGIS or similar software. The identifiers of each catchment (HydroID) must be consistent with those in the CSV files of the scenario dataset.

Once the scenario has been generated, the library includes functions to examine the nutrient sources in a basin, and explore their distribution over time or in space. The input_plot() function provides annual average nutrient loads for the whole period and density plots of nutrient loads. The input_Tserie() function allows to examine the temporal evolution of the inputs, whereas the input_maps() function generates maps of nutrient inputs distribution in the basin (Figure 2).

input_maps(nutient, catch, geometry, "Wisla", "gr1")

3.2 Calibration procedure and sensitivity analysis

The calibration process is essential to find a suitable set of parameters for the model. In GREENeR package, the core function calib_green() enables to run a parameter sensitivity analysis and concurrently assessing the model performance according to several GoF metrics.

Sensitivity Analysis (SA) investigates how the variation in the output of a numerical model can be attributed to variations of its input factors. SA aims at identifying the most influential inputs or parameters, and quantifying how much they contribute to the variability/uncertainty of the model outputs. SA provides information on how much of the output variance is controlled by each parameter, or combination of parameters. SA is increasingly being used in environmental modelling for a variety of purposes, including uncertainty assessment, model calibration and diagnostic evaluation, dominant control analysis, and robust decision-making (Saltelli et al. 2011; Butler et al. 2014; Norton 2015; Pianosi et al. 2016; Mannina et al. 2018). This is achieved by running the model for many samples of the parameter space to determine their impact on the model outputs. SA allows identification of the parameters and input variables that strongly influence the model response (model output). Conversely, it may be of interest to the modeller to verify that although some model parameters may not be very well established they do not significantly contribute to output uncertainty. Saltelli et al. (2000) single out three main classes of SA methods: screening, local, and global methods. Local sensitivity analysis methods focus on assessing the impact of small variations in the input values of a model on the output results. Global sensitivity methods seek to understand how variations in the full range of input values affect the results. Puy et al. (2022) provide a comprehensive overview of several R packages for performing global sensitivity analysis.

Screening methods are economical and qualitative methods. Only screening methods are included in this implementation of the library as they provide a quick assessment of the relative importance of variables. This is particularly useful in exploratory studies and in cases where an initial subset of relevant variables needs to be identified.

Adequate sampling of parameter space is very important in model calibration. Several studies of uncertainty analysis in water resources problems (Melching and Bauwens 2001) concluded that Monte Carlo simulation and Latin Hypercube Sampling (LHS) (McKay et al. 1979; Carnell 2012) methods are very powerful, robust, and flexible. Other approaches are possible (Mauricio Zambrano-Bigiarini 2014), but the LHS has the advantage that it is easily parallelisable, it explores the full range of parameter sets, it does not direct the search depending on the values of previous iterations like other optimisation methods (gradients, genetic algorithms, etc.), and is therefore independent of the GoF metric. This allows performing a posteriori sensitivity analysis for several metrics without having to repeat the process.

The calib_green() function computes 15 GoF metrics (Mauricio Zambrano-Bigiarini 2014; Althoff and Rodrigues 2021) (listed in Appendix 2), and returns an object with all parameter sets generated by the LHS and the corresponding GoF scores. Running calib_green() requires the following settings:

  1. The expected ranges for each parameter. The ranges are defined by two vectors of three values, one for the lower limits and one for the upper limits of the three parameters. The values correspond to each model parameter in sequence: \(alpha_P\) (Equation (3)), \(alpha_L\) (Equation (4)), and \(sd_{coeff}\) (Equations (5) and (6)).

  2. The number of iterations to be performed during the calibration process. The higher the number of iterations, the more likely it is to achieve good results, but the longer the computation time. Depending on the basin and parameter range width, it is recommended to run at least 200 iterations to have enough information to continue the calibration process.

  3. The years to be included in the calibration process.

n_iter <- 2000  
low <- c(10, 0.000, 0.1)   
upp <- c(50, 0.1, 0.9)    
years <- c(1990:2018)

calibration <- calib_green(nutrient, catch, n_iter, low, upp, years)

The calibration process is automatically parallelised by the package and uses all computer cores except one. The computation time depends on the computer, the number of catchments in the basin, and the number of iterations (Table 1). The calib_green() function returns a data frame with a row for each iteration with parameter values and the resulting 15 GoF metric scores. An example of four parameter sets is shown below (cut to the first five GoF metrics).

    alpha_P    alpha_L  sd_coeff    NSE   mNSE   rNSE    KGE PBIAS %
s1 39.31203 0.07628558 0.6325056 0.5623 0.6452 0.9595 0.2512   -44.1
s2 35.06049 0.05510985 0.1577470 0.6733 0.6948 0.9563 0.3763   -36.3
s3 28.34947 0.04418381 0.7209566 0.8690 0.7737 0.8998 0.7926    -1.7
s4 40.73533 0.01351723 0.6195734 0.8370 0.7734 0.9607 0.6292   -22.6

The selection of the appropriate GoF metric(s) for calibration and evaluation of hydrological models can be challenging even for very experienced hydrologists. Choosing the right GoF metric for model calibration largely depends on the overall study scope, which defines the main interest (e.g. high or low load, upper or lower catchment area), and on the available observation dataset (size and quality) (Kumarasamy and Belmont 2018). Automated calibration typically relies, often exclusively, on a single GoF metric, with the Nash-Suttclife efficiency (NSE) been the most frequently used metric in hydrological models (Gupta et al. 2009; Westerberg et al. 2011; Wöhling et al. 2013). As described in Singh and Frevert (2002), a single criterion is inherently predisposed to bias towards certain components of the data time series. Automated procedures may be improved by using more than one criteria as discussed by Gupta et al. (1998). Although automation can help the calibration process become more objective, efficient and practical, it is not a substitute for expert hydrologic intuition and understanding. Whether automated or manual calibration is used, a common approach is to adjust the parameters that display the highest sensitivity (Madsen 2003; Doherty and Skahill 2006).

GREENeR includes several functions to assist in selecting the best parameter set. The calib_boxplot() function shows (Figure 3) relationships between best parameter sets chosen according to one GoF parameter (title of each boxplot) in relation to six most frequently used metrics. Additionally, in the lower panel, the figure shows the distribution of model parameters in the most performing subset of parameter sets. The best parameter set according to each GoF metrics is marked as a red dot in each boxplot (Figure 3).

calib_boxplot(calibration, rate_bs = 5)
Calibration results for the Vistula river basin (TN Scenario). The figure displays the output of the calib\_boxplot() function, and compare several GoF metrics (bR2, KGE, mNSE, NSE, PBIAS, R2, rNSE, VE) with each other and with the model parameters distributions. The top two rows show boxplots of each metric (specified in the title) with others (in the x labels). The lowest row shows the parameter value distributions for the top 5\% (or another threshold indicated in rate\_bs parameter of the function) parameter sets ranked according to the GoF metrics in the x label. The red dots represent the best parameter values for each boxplot.

Figure 3: Calibration results for the Vistula river basin (TN Scenario). The figure displays the output of the calib_boxplot() function, and compare several GoF metrics (bR2, KGE, mNSE, NSE, PBIAS, R2, rNSE, VE) with each other and with the model parameters distributions. The top two rows show boxplots of each metric (specified in the title) with others (in the x labels). The lowest row shows the parameter value distributions for the top 5% (or another threshold indicated in rate_bs parameter of the function) parameter sets ranked according to the GoF metrics in the x label. The red dots represent the best parameter values for each boxplot.

The select_params() function extracts the parameter set that scored the best GoF metric of choice from the calibration result object.

best_params <- select_params(calibration, param = "NSE")

alpha_P  <- best_params$alpha_P
alpha_L  <- best_params$alpha_L 
sd_coeff <- best_params$sd_coeff

It is not recommended to use the parameters extracted by the select_params() function without having performed an analysis of all the calibration results. Instead, alternative parameter sets (according to different GoF) should be compared before making the final selection.

Screening and SA of model parameters can be done via the scatter_plot() and calib_dot() functions of GREENeR package. The scatter_plot() function shows all parameter realizations in the LHS dataset against a selected GoF metric to visualize the influence of each parameter on the metric scores. The result depends on the GoF metric of choice (any metric calculated with calib_green() can be selected) (Figure 4).

scatter_plot(calibration, param = "R2")
Scatter plots of model parameters against R2 metric generated by the scatter\_plot() function for the Vistula river basin (TN).

Figure 4: Scatter plots of model parameters against R2 metric generated by the scatter_plot() function for the Vistula river basin (TN).

Figure 4 example shows that the highest R2 are achieved for values of the parameter \(alpha_P\) around \(33.3\) and for the parameter \(alpha_L\) around \(0.025\), whereas the model is insensitive to the \(sd_{coef}\) parameter. Further, scatter_plot() function helps check if the parameter ranges defined in the search were suitable, or if the number of iterations was sufficient. In Figure 4, the best values of the parameters \(alpha_P\) and \(alpha_L\) are in the central part of the plots, thus the search parameter ranges were appropriate, and the distribution of dots is sufficient to visualize the effect of each parameter, indicating that the number of iterations was adequate.

The calib_dot() function shows the distribution of parameters in relation to each other for a chosen GoF metric, and highlights potential correlations between parameters (Figure 5).

calib_dot(calibration, param = "KGE") 
Dot plots of parameter pairs for the Kling-Gupta efficiency (KGE) generated by the calib\_dot() function for the Vistula river basin (TN).

Figure 5: Dot plots of parameter pairs for the Kling-Gupta efficiency (KGE) generated by the calib_dot() function for the Vistula river basin (TN).

In Figure 5, the Kling-Gupta Efficiency metric (KGE, Althoff and Rodrigues 2021) shows interactions between \(alpha_P\) and \(alpha_L\), i.e. a higher \(alpha_P\) should be associated to a lower \(alpha_L\) for achieving similar KGE. The best \(alpha_P\) values identified for both R2 and KGE metrics are similar, whereas \(alpha_L\) values vary considerably (see also Figure 3).

The GREENeR package includes two useful model calibration functions: compare_calib() and simobs_annual_plot(). compare_calib() shows a scatter plot that compares observed data points with corresponding values predicted by two parameter sets. Figure 6 displays a plot generated by the compare_calib() for the Vistula (TN) basin scenario when using the best parameter sets based on NSE and rNSE metrics. The plot reveals that when rNSE parameter set result in an underestimation of loads in the upper range.

metrics <- c("NSE","rNSE")
compare_calib(nutrient, catch, alpha_p1 = 31, alpha_l1 = 0.02, sd_coef1 = 0.6,
              alpha_p2 = 50, alpha_l2 = 0.01, sd_coef2 = 0.6, years = c(1990:2018), 
              name_basin = "Wisla", metrics)
Result of compare\_calib() for the Vistula river basin (TN), comparing the best set of parameters selected according to the NSE and rNSE metrics.

Figure 6: Result of compare_calib() for the Vistula river basin (TN), comparing the best set of parameters selected according to the NSE and rNSE metrics.

The simobs_annual_plot() function compares observed and predicted values over the years (Figure 7), which identifies erroneous model performance or data distribucion problems over time.

year_range <-  c(1994, 1996:2001, 2006:2009, 2012)      
simobs_annual_plot(nutrient, catch, alpha_P, alpha_L, sd_coeff, year_range, 
                   name_basin = "Wisla", max_value = 10000)
Result of the simobs\_annual\_plot() for the Vistula river basin (TN), with the parameter set selected for highest NSE and limiting the plots to the interval between 0 and 10000.

Figure 7: Result of the simobs_annual_plot() for the Vistula river basin (TN), with the parameter set selected for highest NSE and limiting the plots to the interval between 0 and 10000

3.3 Estimation of catchment nutrient loads and contribution in the basin

Once the most appropriate parameter set has been selected, it is possible to run the model to estimate nutrient loads in the region. The green_shares() function runs GREEN with the selected parameter set and returns catchment nutrient loads and the contributions by source in the simulation period.

The nutrient_tserie() function shows the temporal evolution of the total load at the basin outlet in the simulation period with contributions by sources (Figure 8). Other options of the function show nutrient loads in different zones of the basin (upper, central and lower part).

nutrient_load <- green_shares(nutrient, catch, alpha_P, alpha_L, sd_coeff, years)
nutrient_tserie(nutrient_load, geometry, "Wisla basin", "gr3")
Total nitrogen load at the Vistula river basin outlet from 1990 to 2018. Colors indicate the contribution of different sources (Min = mineral, Man = manure, Atm = atmospheric deposition, Fix = crop fixation, Soil = soil fixation, Sd = scattered dwellings and Ps = point sources).

Figure 8: TN load at the Vistula river basin outlet from 1990 to 2018. Colors indicate the contribution of different sources (Min = mineral, Man = manure, Atm = atmospheric deposition, Fix = crop fixation, Soil = soil fixation, Sd = scattered dwellings and Ps = point sources).

The nutrient_maps() function uses the object returned by green_shares() to generate maps of the distribution of nutrient loads by source for a given year or as the mean for several years (Figure 9). The results are shown in logarithmic scale to improve the visualization as nutrient loads in a region may vary over several orders of magnitude. Alternative options of the function show the total load at the outlets of the catchment or the specific loads (i.e. load per catchment area; in kt/km\(^2\)/y).

map_title <- "Output Loads for the Vistula basin"
nutrient_maps(nutrient_load, geometry, map_title, "gr1", legend_position = 1)
Maps of TN loads in the Vistula river basin by different sources and in total (the sum of all other maps, bottom right), in logarithm scale as generated with nutrient\_maps() function.

Figure 9: Maps of TN loads in the Vistula river basin by different sources and in total (the sum of all other maps, bottom right), in logarithm scale as generated with nutrient_maps() function.

3.4 Estimation of nutrient fluxes and sources contribution in the basin

The region_nut_balance() function runs GREEN with the selected parameter set, and returns the nutrient mass balance and fluxes of the region (mean value for the simulation period). The results of this function can be visualized in a Sankey diagram with the function N4_sankey() (Figure 10) .

nut_bal <- region_nut_balance(nutrient, catch, alpha_P, alpha_L, sd_coeff, 
                              years)
sank <- N4_sankey(nut_bal)
Sankey plot for the Vistula river basin for TN scenario, average for period 1990-2018. The plot represents nitrogen input sources on the left (Min=mineral, Man=manure, Atm=atmospheric deposition, Fix=crop fixation, Soil=soil fixation, Sd=scattered dwellings and Ps=point sources), and nitrogen sinks (land, river and lake retention) and outlet discharge (load to outlet) on the right. The bars in the middle visualize nitrogen agricultural diffuse sources and loads to the stream network.

Figure 10: Sankey plot for the Vistula river basin for TN scenario, average for period 1990-2018. The plot represents nitrogen input sources on the left (Min=mineral, Man=manure, Atm=atmospheric deposition, Fix=crop fixation, Soil=soil fixation, Sd=scattered dwellings and Ps=point sources), and nitrogen sinks (land, river and lake retention) and outlet discharge (load to outlet) on the right. The bars in the middle visualize nitrogen agricultural diffuse sources and loads to the stream network.

4 Conclusion

The GREENeR package provides several functions to assess nutrient pressures in fresh and coastal waters based on the GREEN model (Grizzetti et al. 2012, 2021). It allows assessing annual nutrient loads and concentrations throughout the river network and at river outlets, as well as the contributions of diffuse and point sources to the total load. GREENeR includes functions to route nutrient sources in a region, considering different pathways and the hydrological structure of the river network. The package is enriched by functions that aid selecting the set of parameters that best suits the study scope. Several functions assist in the preparation of scenarios by assembling input data from the appropriate sources, and in visualising inputs and nutrient fluxes in space and time. The version of the GREEN model implemented in the package is computationally efficient and includes parallel running capabilities for the calibration process, greatly reducing the time required for large basins or regional applications, e.g. at continental scale as in (Vigiak et al. 2023).

5 Acknowledgements

This work has been partially supported by grant XMIDAS, ref. PID2021-122640OB-I00, funded by the Spanish Ministry of Science and Innovation.

6 Appendix

6.1 Appendix 1

Diffuse and point sources are defined differently for each nutrient module, i.e. nitrogen or phosphorus. Equation (5) formulates the general GREEN function stated in equation (1) in the case of nitrogen. In GREEN nitrogen model, the total nitrogen load \(L_i\) of catchment \(i\) is estimated as:

\[\begin{equation} \begin{split} L_i = (1 - Lret_i) \cdot ((MinN_i + ManN_i + FixN_i + SoilN_i + (1 - FF_i) \cdot AtmN_i) \cdot \\ (1 - Bret_i) + 0.38 \cdot FF_i \cdot AtmN_i + sd_{coeff} \cdot SdN_i + PsN_i + U_i) \cdot (1 - Rret_i) \end{split} \tag{5} \end{equation}\]

where \(MinN_i\) is the annual amount of nitrogen from mineral fertilisers (ton/yr); \(ManN_i\) is the annual amount of nitrogen in manure fertilisers (ton/yr); \(FixN_i\) is the annual amount of nitrogen fixation by leguminous crops and fodder (ton/yr); \(SoilN_i\) is the annual amount of nitrogen fixation by bacteria in soils (ton/yr); \(AtmN_i\) is the annual nitrogen deposition from atmosphere (ton/yr); \(FF_i\) is the non-agricultural land cover in the catchment (fraction); \(SdN_i\) is the nitrogen input from scattered dwellings (ton/yr); \(PsN_i\) is the nitrogen input from point sources (ton/yr); and, finally, \(U_i\) is the nitrogen load from upstream catchments (ton/yr).

Note that nitrogen atmospheric deposition losses are split into two parts, i.e. inputs to agricultural land, which undergo the basin retention (such as crop uptake) that depends on annual precipitation, while in forest areas they are reduced by a fixed rate, before entering into the stream. The contribution of atmospheric nitrogen deposition in non-agricultural land is thus estimated as \(0.38 \cdot FF \cdot AtmN\). For an atmospheric deposition of \(10\) kgN/ha this corresponds to a background of \(3.8\) kgN/ha (in line with the values reported by HELCOM (2003)).

Phosphorous background losses are split into two parts, with the inputs to agricultural land undergoing basin retention, while in forest areas they are considered entering into the stream. Background losses for phosphorus are estimated at 0.15 kgP/ha (in line with the values reported by (HELCOM 2003).

In GREEN phosphorus model, the total annual phosphorus load \(L_i\) of catchment \(i\) the equation estimates:

\[\begin{equation} \begin{split} L_i = (1 - Lret_i) \cdot ((MinP_i + ManP_i + (1 - FF_i) \cdot BgP_i) \cdot (1 - Bret_i) + \\ FF_i \cdot BgP_i + sd_{coeff} \cdot SdP_i + PsP_i + Ui_i) \cdot (1 - Rret_i) \end{split} \tag{6} \end{equation}\]

where \(MinP_i\) is the annual amount of phosphorus mineral fertilisers (ton/yr); \(ManP_i\) is the annual amount of phosphorus in manure fertilisers (ton/yr); \(FF_i\) is the non-agricultural land cover in the catchment (fraction);\(BgP_i\) is the annual amount of phosphorus background losses (ton/yr); \(SdP_i\) is the phosphorus input from scattered dwellings (ton/yr); \(PsP_i\) is the phosphorus input from point sources (ton/yr); and \(Ui_i\) is the phosphorus load from upstream catchments (ton/yr).

6.2 Appendix 2

The calib_green() function applies the following GoF metrics (Mauricio Zambrano-Bigiarini 2014; Althoff and Rodrigues 2021) (NSE, rNSE, mNSE, R2, PBIAS), where:

See (Nash and Sutcliffe 1970; Kitanidis and Bras 1980; Yapo et al. 1996; Gupta et al. 1998; Krause et al. 2005; Criss and Winston 2008; Mauricio Zambrano-Bigiarini 2014; Papacharalampous et al. 2019) for further details.

6.3 Supplementary materials

Supplementary materials are available in addition to this article. It can be downloaded at RJ-2023-065.zip

6.4 CRAN packages used

GREENeR

6.5 CRAN Task Views implied by cited packages

D. Althoff and L. N. Rodrigues. Goodness-of-fit criteria for hydrological models: Model calibration and performance assessment. Journal of Hydrology, 600: 126674, 2021. URL https://www.sciencedirect.com/science/article/pii/S0022169421007228.
B. Arheimer, J. Dahné and C. Donnelly. Climate change impact on riverine nutrient load and land-based remedial measures of the baltic sea action plan. Ambio, 41(6): 600–612, 2012. URL https://doi.org/10.1007/s13280-012-0323-0.
A. Bartosova, R. Capell, J. E. Olesen, M. Jabloun, J. C. Refsgaard, C. Donnelly, K. Hyytiäinen, S. Pihlainen, M. Zandersen and B. Arheimer. Future socioeconomic conditions may have a larger impact than climate change on nutrient loads to the baltic sea. Ambio, 48(11): 1325–1336, 2019. URL https://doi.org/10.1007/s13280-019-01243-5.
A. H. W. Beusen, J. C. Doelman, L. P. H. Van Beek, P. J. T. M. Van Puijenbroek, J. M. Mogollón, H. J. M. Van Grinsven, E. Stehfest, D. P. Van Vuuren and A. F. Bouwman. Exploring river nitrogen and phosphorus loading and export to global coastal waters in the shared socio-economic pathways. Global Environmental Change, 72: 102426, 2022. URL https://www.sciencedirect.com/science/article/pii/S0959378021002053.
F. Bouraoui, V. Thieu, B. Grizzetti, W. Britz and G. Bidoglio. Scenario analysis for nutrient emission reduction in the european inland waters. Environmental Research Letters, 9(12): 125007, 2014. URL https://doi.org/10.1088/1748-9326/9/12/125007.
M. P. Butler, P. M. Reed, K. Fisher-Vanden, K. Keller and T. Wagener. Identifying parametric controls and dependencies in integrated assessment models using global sensitivity analysis. Environmental modelling & software, 59: 10–29, 2014. URL https://www.sciencedirect.com/science/article/pii/S1364815214001327.
R. Carnell. Lhs: Latin hypercube samples. 2012. URL https://CRAN.R-project.org/package=lhs. R package version 0.10.
R. E. Criss and W. E. Winston. Do nash values have value? Discussion and alternate proposals. Hydrological Processes: An International Journal, 22(14): 2723–2725, 2008. URL https://doi.org/10.1002/hyp.7072.
A. De Jager and J. Vogt. Rivers and catchments of europe-catchment characterisation model (CCM). European Commission, Joint Research Centre (JRC), Ispra, Italy, 2007.
J. Doherty and B. E. Skahill. An advanced regularization methodology for use in watershed model calibration. Journal of Hydrology, 327(3-4): 564–577, 2006. URL https://www.sciencedirect.com/science/article/pii/S0022169405006542.
EEA. European Environment Agency (EEA). WISE Water Framework Directive Database, 2021. URL https://www.eea.europa.eu/data-and-maps/data/wise-wfd-4. Accessed: 2022-07-10.
I. E. Frank and R. Todeschini. The data analysis handbook. Elsevier, 1994.
B. Fu, W. S. Merritt, B. F. Croke, T. R. Weber and A. J. Jakeman. A review of catchment-scale water quality and erosion models and a synthesis of future prospects. Environmental modelling & software, 114: 75–97, 2019. URL https://www.sciencedirect.com/science/article/pii/S1364815218307461.
B. Grizzetti, F. Bouraoui and A. Aloe. Changes of nitrogen and phosphorus loads to european seas. Global Change Biology, 18(2): 769–782, 2012. URL https://doi.org/10.1111/j.1365-2486.2011.02576.x.
B. Grizzetti, F. Bouraoui and G. De Marsily. Assessing nitrogen pressures on european surface water. Global Biogeochemical Cycles, 22(4): 2008. URL https://doi.org/10.1029/2007GB003085.
B. Grizzetti, F. Bouraoui, G. De Marsily and G. Bidoglio. A statistical method for source apportionment of riverine nitrogen loads. Journal of Hydrology, 304(1-4): 302–315, 2005. URL https://www.sciencedirect.com/science/article/pii/S0022169404005013.
B. Grizzetti, O. Vigiak, A. Udias, A. Aloe, M. Zanni, F. Bouraoui, A. Pistocchi, C. Dorati, R. Friedland, A. De Roo, et al. How EU policies could reduce nutrient pollution in european inland and coastal waters. Global Environmental Change, 69: 102281, 2021. URL https://www.sciencedirect.com/science/article/pii/S0959378021000601.
H. V. Gupta, H. Kling, K. K. Yilmaz and G. F. Martinez. Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. Journal of hydrology, 377(1-2): 80–91, 2009. URL https://www.sciencedirect.com/science/article/pii/S0022169409004843.
H. V. Gupta, S. Sorooshian and P. O. Yapo. Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information. Water Resources Research, 34(4): 751–763, 1998. URL https://doi.org/10.1029/97WR03495.
H. C. HELCOM. Fourth baltic sea load compilation (PLC-4). 93, 189 pages. 2003.
P. K. Kitanidis and R. L. Bras. Real-time forecasting with a conceptual hydrologic model: 2. Applications and results. Water Resources Research, 16(6): 1034–1044, 1980. URL https://doi.org/10.1029/WR016i006p01034.
P. Krause, D. Boyle and F. Bäse. Comparison of different efficiency criteria for hydrological model assessment. Advances in geosciences, 5: 89–97, 2005. URL https://adgeo.copernicus.org/articles/5/89/2005/.
B. Kronvang, J. Hezlar, P. Boers, J. P. Jensen, H. Behrendt, T. Anderson, B. Arheimer, M. Venohr, C. C. Hoffmann and C. Nielsen. Nutrient retention handbook. Software Manual for EUROHARP-NUTRET and Scientific review on nutrient retention. EUROHARP report, 9–2004, 2004.
K. Kumarasamy and P. Belmont. Calibration parameter selection and watershed hydrology model evaluation in time and frequency domains. Water, 10(6): 710, 2018. URL https://doi.org/10.3390/w10060710.
A. La Notte, J. Maes, S. Dalmazzone, N. D. Crossman, B. Grizzetti and G. Bidoglio. Physical and monetary ecosystem service accounts for europe: A case study for in-stream nitrogen retention. Ecosystem services, 23: 18–29, 2017. URL https://www.sciencedirect.com/science/article/pii/S2212041616304545.
A. Leip, G. Billen, J. Garnier, B. Grizzetti, L. Lassaletta, S. Reis, D. Simpson, M. A. Sutton, W. De Vries, F. Weiss, et al. Impacts of european livestock production: Nitrogen, sulphur, phosphorus and greenhouse gas emissions, land-use, water eutrophication and biodiversity. Environmental Research Letters, 10(11): 115004, 2015. URL https://doi.org/10.1088/1748-9326/10/11/115004.
W. Ludwig, A. Bouwman, E. Dumont and F. Lespinas. Water and nutrient fluxes from major mediterranean and black sea rivers: Past and future trends and their implications for the basin-scale budgets. Global biogeochemical cycles, 24(4): 2010. URL https://doi.org/10.1029/2009GB003594.
H. Madsen. Parameter estimation in distributed hydrological catchment modelling using automatic calibration with multiple objectives. Advances in water resources, 26(2): 205–216, 2003. URL https://www.sciencedirect.com/science/article/pii/S0309170802000921.
A. Malagó, F. Bouraoui, B. Grizzetti and A. De Roo. Modelling nutrient fluxes into the mediterranean sea. Journal of Hydrology: Regional Studies, 22: 100592, 2019. URL https://www.sciencedirect.com/science/article/pii/S221458181830226X.
G. Manache and C. S. Melching. Sensitivity analysis of a water-quality model using latin hypercube sampling. Journal of Water Resources Planning and Management, 130(3): 232–242, 2004. URL https://doi.org/10.1061/(ASCE)0733-9496(2004)130:3(232).
G. Mannina, A. Cosenza, M. Neumann and P. Vanrolleghem. Global sensitivity analysis in wastewater treatment modelling. Advances in Wastewater Treatment, 1: 32, 2018.
Mauricio Zambrano-Bigiarini. hydroGOF: Goodness-of-fit functions for comparison of simulated and observed hydrological time series. 2014. URL https://github.com/hzambran/hydroGOF. R package version 0.3-8.
M. McKay, R. Beckman and W. Conover. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21(2): 239–245, 1979. URL https://doi.org/10.2307/1268522.
C. S. Melching and W. Bauwens. Uncertainty in coupled nonpoint source and stream water-quality models. Journal of Water Resources Planning and Management, 127(6): 403–413, 2001. URL https://doi.org/10.1061/(ASCE)0733-9496(2001)127:6(403).
M. L. Messager, B. Lehner, G. Grill, I. Nedeva and O. Schmitt. Estimating the volume and age of water stored in global lakes using a geo-statistical approach. Nature communications, 7(1): 1–11, 2016. URL https://doi.org/10.1038/ncomms13603.
J. E. Nash and J. V. Sutcliffe. River flow forecasting through conceptual models part i-a discussion of principles. Journal of hydrology, 10(3): 282–290, 1970. URL https://www.sciencedirect.com/science/article/pii/0022169470902556.
J. Norton. An introduction to sensitivity assessment of simulation models. Environmental Modelling & Software, 69: 166–174, 2015. URL https://www.sciencedirect.com/science/article/pii/S1364815215001085.
G. Papacharalampous, H. Tyralis and D. Koutsoyiannis. Comparison of stochastic and machine learning methods for multi-step ahead forecasting of hydrological processes. Stochastic environmental research and risk assessment, 33(2): 481–514, 2019. URL https://doi.org/10.1007/s00477-018-1638-6.
F. Pianosi, K. Beven, J. Freer, J. W. Hall, J. Rougier, D. B. Stephenson and T. Wagener. Sensitivity analysis of environmental models: A systematic review with practical workflow. Environmental Modelling & Software, 79: 214–232, 2016. URL https://www.sciencedirect.com/science/article/pii/S1364815216300287.
A. Puy, S. Lo Piano, A. Saltelli and S. A. Levin. Sensobol: An r package to compute variance-based sensitivity indices. Journal of Statistical Software, 102(5): 1–37, 2022. URL https://www.jstatsoft.org/index.php/jss/article/view/v102i05.
J. C. Refsgaard and H. J. Henriksen. Modelling guidelines—-terminology and guiding principles. Advances in Water Resources, 27(1): 71–82, 2004. URL https://www.sciencedirect.com/science/article/pii/S0309170803001489.
A. Saltelli, P. Annoni, et al. Sensitivity analysis. 2011.
A. Saltelli, S. Tarantola and F. Campolongo. Sensitivity analysis as an ingredient of modeling. Statistical Science, 377–395, 2000. URL http://www.jstor.org/stable/2676831.
G. E. Schwarz, A. B. Hoos, R. Alexander and R. Smith. The SPARROW surface water-quality model: Theory, application and user documentation. 2006.
S. Seitzinger, J. Harrison, E. Dumont, A. H. Beusen and A. Bouwman. Sources and delivery of carbon, nitrogen, and phosphorus to the coastal zone: An overview of global nutrient export from watersheds (NEWS) models and their application. Global biogeochemical cycles, 19(4): 2005. URL https://doi.org/10.1029/2005GB002606.
R. L. Shreve. Statistical law of stream numbers. The Journal of Geology, 74(1): 17–37, 1966. URL http://www.jstor.org/stable/30075174.
V. P. Singh and D. K. Frevert. Mathematical models of large watershed hydrology. Water Resources Publication, 2002.
R. A. Smith, G. E. Schwarz and R. B. Alexander. Regional interpretation of water-quality monitoring data. Water resources research, 33(12): 2781–2798, 1997. URL https://doi.org/10.1029/97WR02171.
A. N. Strahler. Quantitative analysis of watershed geomorphology. Eos, Transactions American Geophysical Union, 38(6): 913–920, 1957. URL https://doi.org/10.1029/TR038i006p00913.
O. Vigiak, A. Udías, B. Grizzetti, M. Zanni, A. Aloe, F. Weiss, J. Hristov, B. Bisselink, A. de Roo and A. Pistocchi. Recent regional changes in nutrient fluxes of european surface waters. Science of The Total Environment, 858: 160063, 2023. URL https://data.jrc.ec.europa.eu/collection/wpi.
I. Westerberg, J.-L. Guerrero, P. Younger, K. Beven, J. Seibert, S. Halldin, J. Freer and C.-Y. Xu. Calibration of hydrological models using flow-duration curves. Hydrology and Earth System Sciences, 15(7): 2205–2227, 2011. URL https://doi.org/10.5194/hess-15-2205-2011.
T. Wöhling, L. Samaniego and R. Kumar. Evaluating multiple performance criteria to calibrate the distributed hydrological model of the upper neckar catchment. Environmental earth sciences, 69(2): 453–468, 2013. URL 10.1007/s12665-013-2306-2.
P. O. Yapo, H. V. Gupta and S. Sorooshian. Automatic calibration of conceptual rainfall-runoff models: Sensitivity to calibration data. Journal of hydrology, 181(1-4): 23–48, 1996. URL https://www.sciencedirect.com/science/article/pii/0022169495029184.

References

Reuse

Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".

Citation

For attribution, please cite this work as

Udías, et al., "GREENeR: An R Package to Estimate and Visualize Nutrients Pressures on Surface Waters", The R Journal, 2024

BibTeX citation

@article{RJ-2023-065,
  author = {Udías, Angel and Grizzetti, Bruna and Vigiak, Olga and Aloe, Alberto and Alfaro, Cesar and Gomez, Javier},
  title = {GREENeR: An R Package to Estimate and Visualize Nutrients Pressures on Surface Waters},
  journal = {The R Journal},
  year = {2024},
  note = {https://doi.org/10.32614/RJ-2023-065},
  doi = {10.32614/RJ-2023-065},
  volume = {15},
  issue = {3},
  issn = {2073-4859},
  pages = {119-137}
}