The R Journal: article published in 2010, volume 2:1

Two-sided Exact Tests and Matching Confidence Intervals for Discrete Data
Michael P. Fay , The R Journal (2010) 2:1, pages 53-58.

Abstract There is an inherent relationship between two-sided hypothesis tests and confidence intervals. A series of two-sided hypothesis tests may be inverted to obtain the matching 100(1-α)% confidence interval defined as the smallest interval that contains all point null parameter values that would not be rejected at the α level. Unfortunately, for discrete data there are several different ways of defining two-sided exact tests and the most commonly used two-sided exact tests are defined one way, while the most commonly used exact confidence intervals are inversions of tests defined another way. This can lead to inconsistencies where the exact test rejects but the exact confidence interval contains the null parameter value. The packages exactci and exact2x2 provide several exact tests with the matching confidence intervals avoiding these inconsistencies as much as possible. Examples are given for binomial and Poisson parameters and both paired and unpaired 2 × 2 tables. Applied statisticians are increasingly being encouraged to report confidence intervals (CI) and parameter estimates along with p-values from hypothesis tests. The htest class of the stats package is ideally suited to these kinds of analyses, because all the related statistics may be presented when the results are printed. For exact two-sided tests applied to discrete data, a test-CI inconsistency may occur: the p-value may indicate a significant result at level α while the associated 100(1-α)% confidence interval may cover the null value of the parameter. Ideally, we would like to present a unified report (Hirji, 2006), whereby the p-value and the confidence interval match as much as possible.

CRAN packages: exactci, exact2x2, exactci, exact2x2, exactci, exact2x2, PropCIs, rateratio.test, coin, perm
CRAN Task Views implied by cited CRAN packages: ClinicalTrials, Survival

@article{RJ-2010-008,
  author = {Michael P. Fay},
  title = {{Two-sided Exact Tests and Matching Confidence Intervals for
          Discrete Data}},
  year = {2010},
  journal = {{The R Journal}},
  doi = {10.32614/RJ-2010-008},
  url = {https://doi.org/10.32614/RJ-2010-008},
  pages = {53--58},
  volume = {2},
  number = {1}
}