Calibrate p-values under a robust Bayesian perspective so that they can be directly interpreted as lower bounds on Bayes factors in favor of point null hypotheses.
bcal(p)
| p | A numeric vector with values in the [0,1] interval. |
|---|
bcal returns a numeric vector with the same
length as p. A warning message is thrown if there are
NA or NaN values in p.
bcal is a vectorized implementation of the calibration of
p-values into lower bounds for Bayes factors developed by
Sellke et al. (2001)
. The calibration is:
$$B(p) = -e \, p \, log(p)$$ for \(p < 1/e\),
where \(p\) is a p-value on a classical test statistic, and
\(B(p) = 1\) otherwise. \(B(p)\) should be interpreted as an
approximation to the lower bound of the Bayes factor (in favor of the null
hypothesis) that is found by changing the prior distribution of the parameter
of interest (under the alternative hypothesis) over wide classes of
distributions.
Sellke et al. (2001) noted that a scenario in which they definitely recommend this calibration is when investigating fit to the null model/hypothesis with no explicit alternative in mind. Pericchi and Torres (2011) warn that despite the usefulness and appropriateness of this p-value calibration it does not depend on sample size and hence the lower bounds obtained with large samples may be conservative.
Pericchi L, Torres D (2011).
“Quick anomaly detection by the Newcomb—Benford law, with applications to electoral processes data from the USA, Puerto Rico and Venezuela.”
Statistical Science, 26(4), 502--516.
Sellke T, Bayarri MJ, Berger JO (2001).
“Calibration of p values for testing precise null hypotheses.”
The American Statistician, 55(1), 62--71.
bfactor_interpret for the interpretation of Bayes
factors.
bfactor_to_prob to turn Bayes factors into posterior
probabilities.
pcal for a p-value calibration that returns lower
bounds for the posterior probabilities of point null hypotheses.
# Calibration of a typical "threshold" p-value: # ---------------------------------------------------------------- bcal(.05)#> [1] 0.4071622# Calibration of typical "threshold" p-values: # ---------------------------------------------------------------- bcal(c(.1, .05, .01, .005, .001))#> [1] 0.62590752 0.40716223 0.12518150 0.07201160 0.01877723# Application: chi-squared goodness-of-fit test, # lower bound on the Bayes factor in favor of the null hypothesis: # ---------------------------------------------------------------- data <- matrix(c(18, 12, 10, 12, 10, 23), ncol = 2) bcal(chisq.test(data)[["p.value"]])#> [1] 0.3757126