PracTools is an R package with functions that compute sample sizes for various types of finite population sampling designs when totals or means are estimated. One-, two-, and three-stage designs are covered as well as allocations for stratified sampling and probability proportional to size sampling. Sample allocations can be computed that minimize the variance of an estimator subject to a budget constraint or that minimize cost subject to a precision constraint. The package also contains some specialized functions for estimating variance components and design effects. Several finite populations are included that are useful for classroom instruction.
Samples from finite populations are one of the mainstays of research in demographics, economics, and public health. In the U.S., for example, the Consumer Price Index is based on samples of business establishments and households (Bureau of Labor Statistics 2013 17); the unemployment rate is estimated from the Current Population Survey, which is a sample of households (Bureau of Labor Statistics 2013 1); and various health characteristics of the population are estimated from the National Health Interview Survey (Center for Disease Control and Prevention 2013b) and the National Health and Nutrition Examination Survey (Center for Disease Control and Prevention 2013a) both of which are household surveys. Smaller scale academic and marketing research surveys are also typically done using finite population samples.
Standard techniques used in sample design are stratification, clustering, and selection with varying probabilities. Depending on the units to be surveyed (e.g., persons, schools, businesses, institutions) and the method of data collection (e.g., telephone, personal interview, mail survey), the samples may be selected in one or several stages. There are several packages in R that can select samples and analyze survey data. Among the packages for sample selection are pps (Gambino 2012), sampling (Tillé and Matei 2013), samplingbook (Manitz 2013), and simFrame (Alfons et al. 2010). The survey package (Lumley 2010, 2014) has an extensive set of features for creating weights, generating descriptive statistics, and fitting models to survey data.
A basic issue in sample design is how many units should be selected at each stage in order to efficiently estimate population values. If strata are used, the number of units to allocate to each stratum must be determined. In this article, we review some basic techniques for sample size determination in complex samples and the package PracTools (Valliant et al. 2015) that contains specialized routines to facilitate the calculations, most of which are not found in the packages noted above. We briefly summarize some of selection methods and associated formulas used in designing samples and describe the capabilities of PracTools. The penultimate section presents a few examples using the PracTools functions and the final section is a conclusion.
Complex samples can involve any or all of stratification, clustering, multistage sampling, and sampling with varying probabilities. This section discusses these techniques, why they are used, and formulas that are needed for determining sample allocations. Many texts cover these topics, including Cochran (1977), Lohr (1999), Särndal et al. (1992), and Valliant et al. (2013).
Simple random sampling without replacement (srswor) is a method of
probability sampling in which all samples of a given size sample
in the base
package in R can be used to select simple random samples either with or
without replacement. One way of determining an srswor sample size is
to specify that a population value
where nCont
in PracTools will make the computation in
(1). In a real application, the population values in
(1) must be estimated from a sample or guessed based
on prior knowledge.
Another way of determining a sample size is to set a tolerance for how
close the estimate should be to the population value. If the tolerance
(sometimes called the margin of error) is
One could also require that the relative absolute error,
nContMoe
in PracTools. When estimating a
proportion, there are options other than a normal approximation
confidence interval on
Another estimand in a survey might be the difference in means or
proportions. The difference could be between two disjoint groups or
between the estimates for the same group at two different time periods.
The standard approach in such a case would be to find a sample size that
will yield a specified power for detecting a particular size of the
difference. The functions power.t.test
and power.prop.test
in the R
package stats will do this for independent simple random samples.
The case of partially overlapping samples can also be handled (e.g., see
Woodward (1992)). For example, persons may be surveyed at some baseline
date and then followed-up at a later time. An estimate of the difference
in population means may be desired, but the samples do not overlap
completely because of dropouts, planned sample rotation, or nonresponse.
Such non-overlap would be common in panel surveys. Suppose that
where
The value of nDep2sam
in PracTools will perform this calculation.
nProp2sam
will do a similar calculation for testing the difference in
proportions with overlapping samples.
Probability proportional to size (pps) sampling can be very efficient if the measure of size (mos) used for sampling is correlated with the quantities measured in a survey. For example, enrollment in an elementary school may be related to the number of children receiving a government assistance program. In an establishment survey, the total number of employees is often correlated with other employment counts in the establishment, like the number who participate in a retirement plan. In a hospital survey, the number of inpatient beds is usually related to numbers of patients discharged in a month’s time. Household samples are often selected using several stages, the first of which is a sample of geographic areas. An effective mos is typically the number of persons or housing units in each geographic area.
If a variable follows a certain regression structure in the population,
then an optimal measure of size can be estimated. The key finding is due
to (Godambe and Joshi 1965). Isaki and Fuller (1982) extended this to a linear model
A model that may fit some establishment or institutional populations
reasonably well has a variance with the form,
gammaFit
in PracTools
returns an estimate of
The variance formulas for pps without replacement sampling are difficult or impossible to use for sample size determination because they involve joint selection probabilities of units and the sample size is not readily accessible. One practical approach is to use a variance formula appropriate for pps with replacement (ppswr) sampling. The simplest estimator of the mean that is usually studied with ppswr sampling is called “p-expanded with replacement” (pwr) (Särndal et al. 1992 2) and is defined as
where
where
We later give an example of how (7) may be evaluated using PracTools.
Stratified sampling is a useful way of restricting the dispersion of a sample across groups in a population. It can also lead to improvements in precision of overall estimates if an efficient allocation to the strata is used. For example, establishments can be stratified by type of business (retail, wholesale, manufacturing, etc.). Other methods of creating strata are provided by the R package stratification (Baillargeon and Rivest 2014). Given that strata have been created, there are various ways of efficiently allocating a sample to strata: (i) minimize the variance of an estimator given a fixed total sample size (Neyman allocation), (ii) minimize the variance of an estimator for a fixed total budget, (iii) minimize the total cost for a target CV or variance of an estimator, or (iv) allocate the sample subject to several CV or cost criteria subject to a set of constraints on stratum sample sizes or other desiderata. The last is referred to as multicriteria optimization.
The standard texts noted earlier give closed form solutions for (i) and
(ii). For example, suppose a mean is estimated, an srswor is to be
selected within each stratum (
where strAlloc
in PracTools.
Multicriteria optimization and allocations with constraints are more
realistic for multipurpose surveys. In some cases, solutions to
particular allocation problems are available as in Gabler et al. (2012). More
generally, the alabama
(Varadhan 2015) and Rsolnp
(Ghalanos and Theussl 2014) packages will solve nonlinear optimization problems
with constraints and can be very useful for complicated sample
allocations. Among the constraints that are used in practical work are
ones on minimum and maximum stratum sample sizes and relvariances of
overall and individual stratum estimates. Theussl and Borchers (2015) present a CRAN
Task View on optimization and mathematical programming in R. An R
package that will form strata for multipurpose samples is
SamplingStrata
(Barcaroli 2014). Also, the Solver
add-on to Microsoft
Two- and three-stage sampling is commonplace in household surveys but can be also used in other situations. For example, the U.S. National Compensation Survey selects a three-stage sample—geographic areas, establishments, and occupations—to collect compensation data (Bureau of Labor Statistics 2013 8). Allocating the sample efficiently requires estimates of the contribution to the variance of an estimate by each stage of sampling.
As an example, consider a two-stage design in which the primary sampling
units (PSUs) are selected with varying probabilities and with
replacement and elements are selected at the second-stage by srswor.
As noted earlier, determining sample sizes as if the PSUs are selected
with-replacement is a standard workaround in applied sampling to deal
with the fact that without-replacement variance formulas for pps
samples are too complex to use for finding allocations. (Other selection
methods along with three-stage designs are reviewed in Valliant et al. (2013 9).) Let
with
A simple cost function for two-stage sampling assumes that there is a cost per sample PSU and a cost per sample element of collecting and processing data. We model the total cost as
where
The optimal number of units to select per PSU, i.e., the number that minimizes the approximate relvariance, is
Alternatively, to find the optimal m for a fixed relvariance,
clusOpt2
in PracTools will do either of
these calculations.
For three-stage sampling, suppose that
where
The terms
To arrive at an optimal allocation, costs need to be considered. A cost function for three-stage sampling, analogous to the one for two-stage sampling, is
The term
If a target relvariance is set at clusOpt3
will do these computations for
three-stage samples.
In finite population sampling, a distinction is drawn between multistage sampling and multiphase sampling. In a multiphase sample, an initial sample is selected, some characteristics of the units are observed, and a decision is made about how to select a subsample from the initial sample based on what has been observed. In multistage sampling, the same design is used in later stages regardless of what was found in the first-stage units. There is a more technical definition of the difference between multistage and multiphase, but it is unimportant for this discussion. An example of two-phase sampling is to select a subsample of nonrespondents to the initial phase to attempt to get them to cooperate. This is known as a nonresponse follow-up study (NRFU).
Another type of two-phase design is double sampling for stratification. In this design, information is collected in the first phase which is then used to stratify elements for second phase sampling. For example, researchers working to develop a case definition for undiagnosed medical symptoms in U.S. personnel serving in the 1991 Persian Gulf War surveyed a stratified simple random sample of Gulf War-era veterans (Iannacchione et al. 2011). Based on survey responses in the first phase, respondents were classified as likely having or not having a certain type of illness. Blood specimens were requested from randomly sampled phase-1 respondents within the illness strata and analyzed using expensive tests.
As an illustration, take the case of double sampling for stratification.
Cochran (1977) and Neyman (1938) give the two-phase variance of an estimated
mean or proportion when phase-1 is a simple random sample of
where
where
The function dub
in PracTools will calculate the optimal
second-phase sampling fraction and the phase-1 and phase-2 sample sizes
that will minimize the variance in Equation (17). The
function also computes the size of an srs that would cost the same as
the two-phase sample and the ratio of the two-phase stratified variance
to the srs variance.
PracTools will calculate sample sizes for estimators whose variance
can be written in one of the forms given in Section 2.
This directly covers linear estimators of means and totals. A nonlinear
estimator that is a differentiable function of a vector of estimated
totals is also covered. But, a user must do some work to linearize the
estimator and determine the inputs that are required for a PracTools
function. Suppose that the estimator is
where
where
In a two-stage sample where
Nonetheless, there are types of estimators that our package does not cover. Quantile estimators require a special approximation and variance formula (Francisco and Fuller 1991) that does not come from the standard linearization approach. The Gini coefficient, used as a measure of income inequality (e.g., Deaton 1997), is another example of an estimator that is too complicated to linearize using the methods above.
PracTools is a collection of specialized functions written in R along with several example finite populations that can be used for teaching. The package is available from the Comprehensive R Archive Network (CRAN) at http://CRAN.R-project.org/package=PracTools. Because the function code is visible, the routines can be modified (and improved) by any user. A brief description of the functions is given in Table 1. The use of the functions, their input parameters, and the values they return are described in the help files. The package also contains nine example populations of different types. These are listed in Table 2.
Function | Purpose |
---|---|
BW2stagePPS , BW2stageSRS |
Variance components in two-stage samples from a population frame |
BW2stagePPSe |
Estimated variance components in two-stage samples from a sample |
BW3stagePPS |
Variance components in three-stage samples from a population frame |
BW3stagePPSe |
Estimated variance components in three-stage samples from a sample |
clusOpt2 |
Optimal allocation in a two-stage sample |
clusOpt2fixedPSU |
Optimal second-stage sample size in a two-stage sample when the PSU sample is fixed |
clusOpt3 |
Optimal allocation in a three-stage sample |
clusOpt3fixedPSU |
Optimal second and third-stage sample sizes in a three-stage sample when the PSU sample is fixed |
CVcalc2, CVcalc3 |
Compute the coefficient of variation of an estimated total in two- and three-stage designs |
deffH, deffK, deffS |
Henry, Kish, and Spencer design effects |
dub |
Allocation of double sample for stratification |
gammaFit |
Estimate variance power in a linear model |
nCont, nContMoe |
Sample size to meet CV, variance, or margin of error targets (continuous variable) |
nDep2sam |
Sample sizes for two-sample comparison of means with overlapping samples (continuous variable) |
nLogOdds |
Sample size calculation for a proportion using log-odds method |
nProp |
Sample size calculation for a proportion using target CV or variance |
nProp2sam |
Sample sizes for two-sample comparison of proportions with overlapping samples (continuous variable) |
nPropMOE |
Sample size calculation for a proportion using margin of error target |
NRFUopt |
Sample sizes for two-phase nonresponse follow-up study |
nWilson |
Sample size calculation for a proportion using Wilson method |
pclass |
Form nonresponse adjustment classes based on propensity scores |
strAlloc |
Sample allocation in stratified samples |
Population | Description |
---|---|
HMT |
Generate population that follows the model in Hansen et al. (1983) |
hospital |
Population of 393 short-stay hospitals with fewer than 1000 beds |
labor |
Clustered population of 478 persons extracted from Sept. 1976 Current Population Survey |
MDarea.pop |
Artificial population of of 403,997 persons arrayed in census tracts and block groups |
nhis, nhispart |
Datasets of persons with demographic and socioeconomic variables |
nhis.large |
21,588 persons with 18 demographic and health-related variables |
smho.N874 |
874 mental health organizations with 6 financial variables |
smho98 |
875 mental health organizations with 8 financial and patient-count variables |
This section gives some examples for computing sample sizes for estimating proportions and differences of proportions, an allocation to strata, and the optimal numbers of PSUs, secondary units, and elements in a three-stage sample.
The function nProp
will return the sample size required for estimating
a proportion with a specified CV or variance. For a CV target, Equation
(1) is used; the formula for a variance target is
similar. The function takes the following parameters:
CV0 |
target value of coefficient of variation of the estimated proportion |
V0 |
target value of variance of the estimated proportion |
pU |
population proportion |
N |
number of units in finite population; default is Inf |
A single numeric value, the sample size, is returned. An advance guess
is needed for the value of the population proportion,
> nProp(CV0 = 0.05, N = Inf, pU = 0.1)
[1] 3600
If the population has only 500 elements, then the necessary sample size is much smaller:
> nProp(CV0 = 0.05, N = 500, pU = 0.1)
[1] 439.1315
In this function and others in the package, sample sizes are not rounded
in case the exact value is of interest to a user. To obtain sample sizes
for two overlapping groups, nDep2sam
is the appropriate function,
which uses Equation (4). The function takes these inputs:
S2x |
unit variance of analysis variable x in sample 1 |
S2y |
unit variance of analysis variable y in sample 2 |
g |
proportion of sample 1 that is in the overlap with sample 2 |
r |
ratio of the size of sample 1 to that of sample 2 |
rho |
unit-level correlation between x and y |
alt |
should the test be 1-sided or 2-sided; allowed values are "one.sided" or "two.sided" |
del |
size of the difference between the means to be detected |
sig.level |
significance level of the hypothesis test |
pow |
desired power of the test |
Among other things, the user must specify the unit (or population)
standard deviations in the two populations from which the samples are
selected, the proportion of the first sample that is in the second
(i.e., a measure of overlap), and the unit-level correlation between the
variables being measured in the two samples. If there is no overlap in
the samples, it would be natural to set rho = 0
. The size of the
difference in the means that is to be detected and the power of the test
on the difference in means must also be declared. The code below
computes the sample size needed to detect a difference of 5 in the means
with a power of 0.8 when the unit variances in both groups are 200, 75
percent of the first sample is in the second, the samples from the two
groups are to be the same size, and the unit-level correlation is 0.9.
This function and several others in the package use the class
‘power.htest
’ as a convenient way of returning the output.
> nDep2sam(S2x = 200, S2y = 200, g = 0.75, r = 1, rho = 0.9,
+ alt = "one.sided", del = 5, sig.level = 0.05, pow = 0.80)
Two-sample comparison of means
Sample size calculation for overlapping samples
n1 = 33
n2 = 33
S2x.S2y = 200, 200
delta = 5
gamma = 0.75
r = 1
rho = 0.9
alt = one.sided
sig.level = 0.05
power = 0.8
nDep2sam
will also accept a vector of differences as input, e.g.,
del <- seq(2, 10, 0.001)
. This makes it easy to generate a plot like
in Figure 1, where the sample size in each group is
plotted against the difference in means for several levels of power.
This is a useful way to present options to users.
The function gammaFit
will estimate the variance parameter in the
model hospital
population bundled with PracTools to
estimate the variance power in the model,
UPrandomsystematic
from the package sampling, a sample of size 30 is
then selected with probability proportional to
> data("hospital")
> x <- hospital$x
> y <- hospital$y
> X <- cbind(sqrt(x), x)
> (res <- gammaFit(X = X, x = x, y = y, maxiter = 100, tol = 0.001))
Convergence attained in 47 steps.
g.hat = 1.612009
$g.hat
X
1.612009
> require(sampling)
> n <- 30
> pik <- n * sqrt(x^res$g.hat) / sum(sqrt(x^res$g.hat))
> sam <- UPrandomsystematic(pik)
> hosp.sam <- hospital[sam == 1, ]
To determine a sample size for ppswr sampling, the function nCont
can be used. This function is designed to evaluate Equation
(1) for simple random samples. However, Equation
(7) has the same form if we equate
nCont
is
> nCont(CV0 = 0.05, N = Inf, CVpop = sqrt(2))
[1] 800
A sample can be allocated to strata using strAlloc
. The function takes
a number of parameters which are described in the help file. A standard
problem in applied sampling is to find an allocation that will minimize
the variance of an estimated mean subject to a fixed total budget. To
solve this problem, the stratum population sizes, standard deviations,
stratum per-unit costs, total budget, and the type of allocation
(alloc = "totcost"
) are specified; partial output is:
> Nh <- c(215, 65, 252, 50, 149, 144)
> Sh <- c(267, 106, 69, 110, 98, 445)
> ch <- c(1400, 200, 300, 600, 450, 1000)
> strAlloc(Nh = Nh, Sh = Sh, cost = 100000, ch = ch, alloc = "totcost")
allocation = totcost
Nh = 215, 65, 252, 50, 149, 144
Sh = 267, 106, 69, 110, 98, 445
nh = 30.578027, 9.710196, 20.008418, 4.475183, 13.719233, 40.387433
nh/n = 0.25722085, 0.08168169, 0.16830983, 0.03764502, 0.11540551, 0.33973710
The function returns a list with the type of allocation, population
stratum counts and standard deviations, sample sizes for each stratum,
the proportions of the sample allocated to each stratum, and the
anticipated standard error of the mean. Other options for the allocation
types are proportional to stratum population sizes ("prop"
), Neyman
("neyman"
), and minimization of the total cost subject to a specified
variance or CV target ("totvar"
). The nh
component of the list can
then be used in, e.g., the strata
function in sampling, to select
the sample. Either the round
or ceiling
function could be applied to
nh
to create integer sample sizes. (If non-integers are supplied to
strata
, they will be truncated to the integer floor.)
When designing multistage samples, decisions must be made about how many
units to select at each stage. To illustrate this, we consider a
three-stage sample. A considerable amount of data is needed to estimate
realistic ingredients required for clusOpt3
. The function,
BW3stagePPSe
, will estimate BW2stagePPSe
does similar calculations for two-stage sampling.
Variance component estimation is, of course, a difficult area where a
number of alternatives have been developed in the model-based
literature. The forms used in BW2stagePPSe
and BW3stagePPSe
are
fairly simple ANOVA-type estimates. These estimates have known defects,
like occasionally being negative.
The following example computes a three-stage allocation that minimizes
the variance of the pwr-estimator assuming that the budget for
variable cost is 100,000; the PSU, SSU, and per-element costs are 500,
100, and 120, respectively; cal.sw = 1
specifies that the optima be found for a fixed
total budget. The full description of the input parameters can be found
in the help file for clusOpt3
.
> clusOpt3(unit.cost = c(500, 100, 120), delta1 = 0.01, delta2 = 0.10, unit.rv = 1,
+ k1 = 1, k2 = 1, tot.cost = 100000, cal.sw = 1)
C1 = 500
C2 = 100
C3 = 120
delta1 = 0.01
delta2 = 0.1
unit relvar = 1
k1 = 1
k2 = 1
cost = 1e+05
m.opt = 28.3
n.opt = 7.1
q.opt = 2.7
CV = 0.0499
Along with the inputs, the output includes the optimal sample size for
each stage and the CV that is anticipated for the pwr-estimator given
that design. The sample sizes above can be rounded and then used in the
sampling package to select units at each of the stages. For example,
suppose that 28 PSUs and 7 SSUs will be selected with probabilities
proportional to a mos. Within each sample SSU, a sample of 3 elements
will be selected via srswor. The PSUs can be selected using the
function cluster
and the data extracted. Then, a cluster sample of 7
SSUs can be selected from each of those 28 units in a loop, again using
cluster
and the data for those sample SSUs extracted. The sample of
SSUs would then be treated as strata and the strata
function used to
select 3 elements from each SSU using srswor.
The function dub
will compute the allocation to strata for a double
sampling design in which phase-1 is used to assign units to strata. The
function takes these input parameters:
c1 |
cost per unit in phase-1 |
c2 |
cost per unit in phase-2 |
Ctot |
total variable cost |
Nh |
vector of stratum population counts or proportions |
Sh |
vector of stratum population standard deviations |
Yh.bar |
vector of stratum population means |
The inputs,
> Wh <- rep(0.25, 4)
> Ph <- c(0.02, 0.12, 0.37, 0.54)
> Sh <- sqrt(Ph * (1 - Ph))
> c1 <- 10; c2 <- 50; Ctot <- 20000
> dub(c1, c2, Ctot, Nh = Wh, Sh, Yh.bar = Ph)
V1 = 0.04191875
V2 = 0.1307118
n1 = 404.1584
n2 = 319.1683
n2/n1 = 0.789711
ney.alloc = 30.89801, 71.71903, 106.55494, 109.99634
Vopt = 0.0005132573
nsrs = 400
Vsrs = 0.0004839844
Vratio = 1.06
The function also computes the size of an srs, nsrs
, that would cost
the same total amount, assuming that the per-unit cost is nsrs
; and the ratio of the two variances. Often, the
two-phase design has very little gain, and sometimes a loss as in this
example, compared to simple random sampling. However, double sampling
for stratification is usually undertaken to control the sample sizes in
the strata whose members are not known in advance.
To illustrate a calculation for a nonlinear estimator, consider the
proportion of Hispanics with insurance coverage in the MDarea.pop
,
which is part of the package. Define BW2stageSRS
can be
used to compute
> # recode Hispanic to be 1 = Hispanic, 0 if not
> y2 <- abs(MDarea.pop$Hispanic - 2)
> y1 <- y2 * MDarea.pop$ins.cov
> # proportion of Hispanics with insurance
> p <- sum(y1) / sum(y2)
> # linear sub
> z <- y1 - p * y2
> BW2stageSRS(z, psuID = MDarea.pop$TRACT)
The result is BW2stageSRS
this way:
> BW2stageSRS(y1, psuID = MDarea.pop$TRACT)
which returns
Finite population sampling is one of the more important areas in statistics since many key economic and social measures are derived from surveys. R through its packages is gradually accumulating capabilities for selecting and analyzing samples from finite populations. Pieces that have been missing are sample size computations for the kinds of complex designs that are used in practice. PracTools contributes to filling that gap by providing a suite of sample size calculation routines for one-, two-, and three-stage samples. We also include features for stratified allocations, for probability proportional to size sampling, and for incorporating costs into the computations. Several realistic example populations, that should be useful for classroom instruction, are also part of the package.
The package is limited in the sense that it covers only some of the sample selection schemes that we have found are most useful and prevalent in the practice of survey sampling. There are many other selection algorithms that have their own, specialized variance formulas. Tillé (2006) covers many of these. PracTools also does not select samples, but there are a number of other R packages, mentioned in this paper that do. One of the great advantages of R is that users can readily access different packages for specialized tasks like sample size calculation and sample selection.
pps, sampling, samplingbook, simFrame, survey, PracTools, stratification, alabama, Rsolnp, SamplingStrata
MissingData, OfficialStatistics, Optimization, Survival
This article is converted from a Legacy LaTeX article using the texor package. The pdf version is the official version. To report a problem with the html, refer to CONTRIBUTE on the R Journal homepage.
Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
For attribution, please cite this work as
Valliant, et al., "PracTools: Computations for Design of Finite Population Samples", The R Journal, 2015
BibTeX citation
@article{RJ-2015-028, author = {Valliant, Richard and Dever, Jill A. and Kreuter, Frauke}, title = {PracTools: Computations for Design of Finite Population Samples}, journal = {The R Journal}, year = {2015}, note = {https://rjournal.github.io/}, volume = {7}, issue = {2}, issn = {2073-4859}, pages = {163-176} }