Comparing two proportions through the difference is a basic problem in statistics and has applications in many fields. More than twenty confidence intervals (Newcombe 1998a; Newcombe 1998b) have been proposed. Most of them are approximate intervals with an asymptotic infimum coverage probability much less than the nominal level. In addition, large sample may be costly in practice. So exact optimal confidence intervals become critical for drawing valid statistical inference with accuracy and precision. Recently, (Wang 2010, 2012) derived the exact smallest (optimal) one-sided
The comparison of two proportions through the difference is one of the basic statistical problems. One-sided confidence intervals are of interest if the goal of a study is to show superiority (or inferiority), e.g., that a treatment is better than the control. If both limits are of interest, then two-sided intervals are needed.
In practice, most available intervals, see
(Newcombe 1998a; Newcombe 1998b), are approximate ones,
i.e., the probability that the interval includes the difference of two
proportions, the so-called coverage probability, is not always at least
the nominal level although the interval aims at it. Also, even with a
large sample size, the infimum coverage probability may still be much
less than the nominal level and does not converge to this quantity. In
fact, the Wald type interval has an infimum coverage probability zero
for any sample sizes and any nominal level
Exact intervals which assure an infimum coverage probability of at least
PairedCI()
and BinomCI()
, where PairedCI()
is
for calculating lower one-sided, upper one-sided and two-sided
confidence intervals for the difference of two paired proportions and
BinomCI()
is for the difference of two independent proportions when
the sample size is small to medium. Results from ExactCIdiff are
compared with those from the function ci.pd()
in the R package
Epi (Carstensen et al. 2013), and the
PROC FREQ
procedure in the software SAS (SAS Institute Inc. 2011).
Depending on how the data are collected, one group of three intervals is
needed for the difference of two paired proportions and another group
for the difference of two independent proportions. Pointed out by
(Mehrotra et al. 2003), an exact inference procedure may result in
poor powerful analysis if an impropriate statistic is employed. Wang’s
one-sided intervals (Wang 2010, 2012),
obtained through a carefully inductive construction on an order, are
optimal in the sense that they are a subset of any other one-sided
Although R provides exact confidence intervals for one proportion, e.g.,
the function exactci()
in the package
PropCIs (Scherer 2013), the
function binom.exact()
in the package
exactci
(Fay 2010; Fay 2012) and the function binom.test()
in the package
stats (Version 2.15.2), there is no exact confidence interval
available in R, to the best of our knowledge, for the difference of two
proportions, which is widely used in practice. ExactCIdiff is the
first available R package to serve this purpose. The R package
ExactNumCI (Sun and Park 2013) claims that its function pdiffCI()
generates an exact confidence interval for the difference of two
independent proportions, however, pointed out by a referee, the coverage
probability of a 95% confidence interval, when the numbers of trials in
two independent binomial experiments are 3 and 4, respectively, is equal
to 0.8734 when the two true proportions are equal to 0.3 and 0.5,
respectively.
In the rest of the article, we discuss how to compute intervals for the
difference of two paired proportions
Suppose there are
The parameter of interest is the difference of
To make interval construction simpler, let
Suppose a lower one-sided
and
Following (Wang 2012), the construction of the smallest
as shown in the diagram below:
Therefore,
Step 1: Point
…
Step k: For
Step k+1: Now we determine
For each point
To simplify the construction on
from which
For each point
Let
Then define
Since
There are three issues to compute the rank function
compute the infimum in
determine the smallest solution of equation ((4));
repeat this process on all points in
These have to be done numerically.
Regarding i), use a two-step approach to search for the infimum when
Regarding ii), the smallest solution is found by the bisection method
with different initial upper search points and a fixed initial lower
search point
Regarding iii), use unique()
to eliminate the repeated points in which()
to search for
For any given order on a sample space the smallest one-sided
that are similar to ((4)) and ((5)).
Two facts are worth mentioning. a) Among all one-sided
Suppose we observe two independent binomial random variables
is the parameter of interest. The sample space
Following (Wang 2010), a rank function
as shown in the diagram below:
Repeating the process in the previous section, we can derive this new
rank function
We illustrate the usage of the PairedCI()
function to calculate the
exact smallest lower one-sided confidence interval
Function PairedCI()
has the following arguments:
PairedCI(n12, t, n21, conf.level = 0.95, CItype = "Lower", precision = 0.00001,
grid.one = 30, grid.two = 20)
The arguments n12
, t
, and n21
are the observations from the
experiment. The value of conf.level
is the confidence coefficient of
the interval, CItype
to obtain either
an upper one-sided or a two-sided interval. The precision of the
confidence interval with a default value 0.00001 is rounded to 5
decimals. The values of grid.one
and grid.two
are the number of grid
points in the two-step approach to search the infimum. The higher the
values of grid.one
and grid.two
, the more accurate is the solution
but the longer is also the computing time. Based on our extensive
numerical study, we find that grid.one = 30
and grid.two = 20
are
sufficient enough for the problem.
In the data by (Karacan et al. 1976), the researchers wish to see how much
more help the marijuana use provides for sleeping by using a lower
one-sided 95
> library(ExactCIdiff)
> lciall <- PairedCI(9, 20, 3, conf.level = 0.95) # store relevant quantities
> lciall # print lciall
$conf.level
[1] 0.95 # confidence level
$CItype
[1] "Lower" # lower one-sided interval
$estimate
[1] 0.1875 # the mle of p1 - p2
$ExactCI
[1] 0.00613 1.00000 # the lower one-sided 95% interval
> lci <- lciall$ExactCI # extracting the lower one-sided 95% interval
> lci # print lci
[1] 0.00613 1.00000
The use of marijuana helps sleeping because the interval
The upper one-sided
> uci <- PairedCI(9, 20, 3, conf.level = 0.95, CItype = "Upper")$ExactCI
> uci # the upper one-sided 95% interval
[1] -1.00000 0.36234
> u975 <- PairedCI(9, 20, 3, conf.level = 0.975, CItype = "Upper")$ExactCI
> u975 # the upper one-sided 97.5% interval
[1] -1.00000 0.39521
> l975 <- PairedCI(9, 20, 3, conf.level = 0.975, CItype = "Lower")$ExactCI
> l975 # the lower one-sided 97.5% interval
[1] -0.03564 1.00000
> ci95 <- PairedCI(9, 20, 3, conf.level = 0.95)$ExactCI
> ci95
[1] -0.03564 0.39521 # the two-sided 95% interval
# it is equal to the intersection of two one-sided intervals
In summary, three
The second data set is from a two-arm randomized clinical trial for
testing the effect of tobacco smoking on mice (Essenberg 1952).
In the treatment (smoking) group, the number of mice is BinomCI()
computes exact
confidence intervals for
Function BinomCI()
has the following arguments:
BinomCI(n1, n2, x, y, conf.level = 0.05, CItype = "Lower", precision = 0.00001,
grid.one = 30, grid.two = 20)
The arguments n1
, n2
, x
and y
are the observations from the
experiment. The rest of the arguments are the same as in function
PairedCI()
.
In this clinical trial, the maximum likelihood estimate for the
difference between two tumor rates
> lciall <- BinomCI(23, 32, 21, 19, CItype = "Lower")
> lciall # print lciall
$conf.level
[1] 0.95 # confidence level
$CItype
[1] "Lower"
$estimate
[1] 0.319293 # the mle of p1 - p2
$ExactCI
[1] 0.133 1.00000 # the lower one-sided 95% interval
> lci <- lciall$ExactCI # extracting the lower one-sided 95% interval
> lci
[1] 0.133 1.00000
The lower one-sided 95% confidence interval for
The following code is for the upper one-sided and two-sided 95% confidence intervals.
> uci <- BinomCI(23, 32, 21, 19, conf.level = 0.95, CItype = "Upper")$ExactCI
> uci # the upper one-sided 95% interval
[1] -1.00000 0.48595
> u975 <- BinomCI(23, 32, 21, 19, conf.level = 0.975, CItype = "Upper")$ExactCI
> u975 # the upper one-sided 97.5% interval
[1] -1.00000 0.51259
> l975 <- BinomCI(23, 32, 21, 19, conf.level = 0.975, CItype = "Lower")$ExactCI
> l975 # the lower one-sided 97.5% interval
[1] 0.09468 1.00000
> ci95 <- BinomCI(23, 32, 21, 19)$ExactCI
> ci95
[1] 0.09468 0.51259 # the two-sided 95% interval
# it is equal to the intersection of two one-sided intervals
They are equal to
Our smallest exact one-sided confidence interval
Exact method | Asymptotic method |
|
|
The coverage for the exact upper 95% confidence interval BinomCI()
and the function ci.pd()
in the package
Epi. The left plot of Figure 1 shows the coverage
against
In light of the unsatisfied coverage for the asymptotic approaches, we
next compare our exact intervals to the exact intervals by the
PROC FREQ
procedure in the software SAS. First revisit Example 2,
where SAS provides a wider exact two-sided 95EXACT RISKDIFF
statement
within PROC FREQ
. This is the SAS default. The other exact 95%
interval in SAS using METHOD = FMSCORE
is
Two exact upper intervals produced by BinomCI()
in the R package
ExactCIdiff and the PROC FREQ
procedure in SAS are shown in
Figure 2. The smaller upper confidence interval is
preferred due to the higher precision. Almost all the points in the
figure are below the diagonal line, which confirms a better performance
of the interval by BinomCI()
. The average lengths of the two-sided
interval for
BinomCI()
and PROC FREQ
when
A group of three exact confidence intervals (lower one-sided, upper
one-sided, and two-sided) are computed efficiently with the R package
ExactCIdiff for each of the differences of two proportions:
A practical issue for ExactCIdiff is the computation time that depends
on the sample size PairedCI()
(BinomCI()
) and the location of observations BinomCI()
), e.g.,
PairedCI(30, 40, 30) = [-0.15916, 0.15916]
, with a sample size of 100,
takes about a hour to complete on an HP laptop with Intel(R) Core(TM)
i5=2520M CPU@2.50 GHz and 8 GB RAM, and
PairedCI(300, 10, 10, CItype = "Lower") = [0.86563, 1.00000]
, with a
sample size of 320, takes less than one minute. Our exact interval is
constructed by an inductive method. By nature, when there are many
sample points, i.e., the sample size is large, deriving an order on all
sample points is very time consuming. Thus the confidence limit on a
sample point, which is located at the beginning (ending) part of the
order, needs a short (long) time to calculate. Roughly speaking, when
the sample size is more than 100, one would expect a long computation
time for a two-sided interval. More details may be found at:
http://www.wright.edu/~weizhen.wang/software/ExactTwoProp/examples.pdf.
Wang’s research is partially supported by NSF grant DMS-0906858. The authors are grateful to three anonymous referees and the Editor for their constructive suggestions.
ExactCIdiff, Epi, PropCIs, exactci
This article is converted from a Legacy LaTeX article using the texor package. The pdf version is the official version. To report a problem with the html, refer to CONTRIBUTE on the R Journal homepage.
Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
For attribution, please cite this work as
Shan & Wang, "ExactCIdiff: An R Package for Computing Exact Confidence Intervals for the Difference of Two Proportions", The R Journal, 2013
BibTeX citation
@article{RJ-2013-026, author = {Shan, Guogen and Wang, Weizhen}, title = {ExactCIdiff: An R Package for Computing Exact Confidence Intervals for the Difference of Two Proportions}, journal = {The R Journal}, year = {2013}, note = {https://rjournal.github.io/}, volume = {5}, issue = {2}, issn = {2073-4859}, pages = {62-70} }