Barnard's Exact Test: Formula, Calculator, Interpretation, Python, R, SPSS, SAS and Excel Guide
Barnard's Exact Test is an unconditional exact method for comparing two independent binomial proportions in a 2 × 2 table. This complete guide explains the hypotheses, formula, nuisance parameter, Fisher and Boschloo comparisons, calculator logic, APA reporting, seven cross-software Python and R charts, SPSS, SAS and Excel workflows, code and downloadable reports through a worked analysis of 649 students.
649 observations
Risk difference −5.74 points
Pooled-score p ≈ .0487
Barnard's Exact Test Model Overview
Barnard's Exact Test is an unconditional exact test for comparing two independent binomial proportions in a 2 × 2 table. It keeps the two group sample sizes fixed, allows the total number of outcomes to vary, and calculates an exact tail probability without conditioning on both margins.
What Question Does It Answer?
The method tests whether the probability of a binary outcome is equal in two independent groups. In this worked example, it asks whether the probability of G3 ≥ 10 differs between female and male students.
How the Exact Test Is Built
The observed difference is standardized with a pooled score statistic. Barnard's Exact Test then evaluates every possible pair of binomial counts that is at least as extreme as the observed table. Because the common probability under the null is unknown, the test reports the largest exact tail probability across all permissible null probabilities.
How It Differs from Fisher's Exact Test
Fisher's exact test conditions on the observed margins. Barnard's Exact Test does not condition on the combined outcome total. The appropriate choice depends on the sampling design: Fisher is naturally conditional, whereas Barnard is naturally suited to two fixed binomial sample sizes with a nonfixed outcome total.
Worked Example at a Glance
The analysis contains 383 female students and 266 male students. G3 ≥ 10 occurred for 333 females and 216 males, giving observed proportions of 86.95% and 81.20%. The male-minus-female difference is −5.74 percentage points. The pooled-score Barnard statistic is approximately 1.993, with a two-sided exact p-value near .049.
Quick Answer: Barnard's Exact Test Result
The analysis compares two independent proportions: the proportion of female students with G3 ≥ 10 and the corresponding proportion of male students. The pooled-score Barnard's Exact Test is used as the primary unconditional exact procedure.
Test Summary
- Procedure: two-sided pooled-score Barnard's Exact Test
- Statistic: 1.9927
- Exact p-value: approximately 0.0487
- Decision at α = .05: reject narrowly
- Evidence description: borderline rather than strong
Substantive Meaning
- Female: 333 of 383 reached G3 ≥ 10.
- Male: 216 of 266 reached G3 ≥ 10.
- Absolute difference: 5.74 percentage points.
- Risk ratio, male/female: 0.934.
- Cramer's V: 0.078, a small association.
Table of Contents
- Research question, hypotheses and data design
- When to use Barnard's Exact Test
- Barnard's Exact Test formula and calculation
- Variables and data dictionary
- Worked 2 × 2 table and complete results
- Barnard vs Fisher, Boschloo and chi-square
- Barnard's Exact Test calculator steps
- Seven Python and R chart interpretations
- Python, R, SPSS, SAS and Excel workflows
- Expandable software code
- Assumptions, limitations and advanced interpretation
- APA reporting for Barnard's Exact Test
- Common mistakes and how to correct them
- Reports and worked Excel download
- Related Salar Cafe guides
- Frequently asked questions
- Conclusion
Research Question, Hypotheses and Data Design
Research Question
Does the probability of reaching G3 ≥ 10 differ between female and male students in the analysed sample?
Null and Alternative Hypotheses
| Hypothesis | Statistical statement | Meaning |
|---|---|---|
| Null hypothesis | H0: pF = pM | The probability of G3 ≥ 10 is equal in the two groups. |
| Two-sided alternative | H1: pF ≠ pM | The probability differs in either direction. |
| Possible directional alternative | H1: pF > pM | Female students have a higher outcome probability; use only when prespecified. |
Sampling Design Behind the Test
Barnard's Exact Test treats the female and male totals as the sizes of two independent binomial samples. The total number reaching G3 ≥ 10 is allowed to vary under repeated samples. This design distinction is the central reason to consider an unconditional exact test rather than automatically conditioning on both margins.
Why the Direction Must Be Fixed Before Testing
A two-sided test is appropriate when both higher and lower differences would be meaningful. A one-sided test can have greater power in its chosen direction but should not be selected after viewing the group proportions. See Null and Alternative Hypothesis and Type I and Type II Error for the consequences of changing hypotheses after inspecting results.
When to Use Barnard's Exact Test
Use Barnard's Exact Test When
- The table is exactly 2 × 2.
- The two groups are independent.
- The outcome is binary.
- The group sample sizes are fixed or treated as fixed.
- The total number of successes is not fixed.
- An unconditional exact p-value is desired.
- Small, sparse or unbalanced samples make asymptotic methods unattractive.
- A more powerful design-matched alternative to Fisher's conditional test is needed.
Choose Another Method When
- The same people are measured twice or observations are matched.
- There are more than two groups or outcome categories.
- Observations are clustered within schools, hospitals or households.
- Covariates must be adjusted.
- The outcome is quantitative and should not be dichotomized.
- Both margins were fixed by the experimental design.
- Multiple 2 × 2 strata require a common adjusted association.
- The research question concerns prediction rather than a simple unadjusted comparison.
Simple Test-Selection Logic
If no, use a method designed for larger tables.
If no, use a paired or clustered method.
Use Barnard for two binomial samples; use Fisher when both margins are fixed.
Small Samples Are Not the Only Use Case
Barnard's Exact Test is frequently introduced as a small-sample method, but exactness is not restricted to small data. It can be used whenever the two-binomial sampling design is appropriate. With large samples, Barnard, Fisher and chi-square results often become similar, while effect estimation becomes more important than the precise choice among near-equivalent p-values.
Barnard's Exact Test and Dichotomization
The current analysis converts G3 from a 0–19 quantitative score to G3 ≥ 10 versus G3 < 10. This threshold may be useful for a pass-style question, but it discards grade distance. A complete project can report the binary analysis and separately examine the original quantitative G3 distribution.
Barnard's Exact Test Formula and Calculation
The calculation has two layers. First, a statistic measures the observed difference between the two proportions. Second, exact binomial probabilities are summed over all possible tables that are at least as extreme, and the largest tail probability over the nuisance parameter is reported.
Step 1: Calculate the Group Proportions
Step 2: Calculate the Pooled Proportion
Step 3: Calculate the Pooled Score Statistic
The sign depends on group order. Reversing the two groups changes the sign but does not change the two-sided decision.
Step 4: Enumerate Every Possible 2 × 2 Table
Female successes can range from 0 to 383, and male successes can range from 0 to 266. The method considers 384 × 267 = 102,528 possible success-count pairs. For each candidate table, it calculates the chosen statistic and identifies whether the table belongs to the two-sided rejection region.
Step 5: Calculate Joint Binomial Probabilities
The same common probability π is used for both groups under the equality null. The probability of each possible table is the product of the two independent binomial probabilities.
Step 6: Sum the Exact Tail
Step 7: Maximize Over the Nuisance Parameter
The nuisance-parameter search is not optional. It converts a family of null tail probabilities into one valid unconditional exact p-value. The current pooled two-binomial calculation gives a maximum near 0.0487.
Pooled Versus Unpooled Barnard Statistics
| Specification | Variance denominator | Interpretation |
|---|---|---|
| Pooled score statistic | Uses the common pooled proportion under H0 | Primary specification used in this guide. |
| Unpooled Wald statistic | Uses separate observed group variances | Creates a different ordering of possible tables and can return a different exact p-value. |
Variables and Data Dictionary
| Variable | Role | Coding | N | Counts | Meaning in the analysis |
|---|---|---|---|---|---|
| sex | Independent grouping variable | F = female; M = male | 649 | F = 383; M = 266 | Defines the two independent binomial samples. |
| G3 | Original quantitative outcome | Final grade from 0 to 19 | 649 | Not analysed on its original scale in this test | Source variable used to create the binary threshold. |
| G3 ≥ 10 | Binary event | 1 when G3 is at least 10 | 649 | 549 events | Success or outcome-present category. |
| G3 < 10 | Binary non-event | 0 when G3 is below 10 | 649 | 100 non-events | Failure or outcome-absent category. |
Observed 2 × 2 Table
| Group | G3 ≥ 10 | G3 < 10 | Total | G3 ≥ 10 proportion |
|---|---|---|---|---|
| Female | 333 | 50 | 383 | 0.8694517 |
| Male | 216 | 50 | 266 | 0.8120301 |
| Total | 549 | 100 | 649 | 0.8459168 |
Expected Counts Under Independence
| Cell | Observed | Expected | Observed − expected |
|---|---|---|---|
| Female, G3 ≥ 10 | 333 | 323.986 | +9.014 |
| Female, G3 < 10 | 50 | 59.014 | −9.014 |
| Male, G3 ≥ 10 | 216 | 225.014 | −9.014 |
| Male, G3 < 10 | 50 | 40.986 | +9.014 |
Worked 2 × 2 Table and Complete Results
333 of 383
216 of 266
Male minus female
Pooled score
Two-sided unconditional
Small association
Primary Barnard's Exact Test Result
| Test | Statistic | Alternative | P-value | Decision at α = .05 |
|---|---|---|---|---|
| Barnard's Exact Test, pooled score | 1.9927455 | Two-sided | ≈ 0.0487033 | Reject H0 narrowly |
Effect Estimates
| Measure | Estimate | 95% lower | 95% upper | Interpretation |
|---|---|---|---|---|
| Female outcome proportion | 0.8694517 | 0.8320101 | 0.8995558 | Estimated probability of G3 ≥ 10 among female students. |
| Male outcome proportion | 0.8120301 | 0.7607620 | 0.8544140 | Estimated probability of G3 ≥ 10 among male students. |
| Risk difference, M − F | −0.0574216 | −0.1152394 | 0.0003962 | The observed male probability is 5.74 percentage points lower. |
| Risk ratio, M/F | 0.9339565 | 0.8711326 | 1.0013111 | The male probability is about 93.4% of the female probability. |
| Odds ratio, M/F | 0.6486486 | 0.4228947 | 0.9949170 | The observed male success odds are lower than the female success odds. |
| Cramer's V | 0.0782221 | — | — | The unadjusted association is small. |
Statistical Interpretation
The pooled-score Barnard's Exact Test gives a p-value slightly below .05. At the conventional 5% level, the equality null is rejected, but the evidence is close to the threshold. At a 1% level, the null is not rejected.
Practical Interpretation
The group difference is 5.74 percentage points, and Cramer's V is 0.078. The result is therefore statistically borderline and practically small in association-strength terms. Whether a 5.74-point difference is educationally important depends on the research context, consequences and measurement quality.
Confidence-Interval Interpretation
The reported risk-difference interval nearly includes zero, and the risk-ratio interval nearly includes 1. These approximate intervals are not obtained by inverting the same Barnard procedure, so a slight boundary difference between the p-value and interval is possible. The interval method should be named in formal work.
Barnard's Exact Test Versus Fisher, Boschloo and Chi-Square
Unconditional exact test using a score or Wald statistic and nuisance-parameter maximization.
Conditional exact test based on the hypergeometric distribution with fixed margins.
Unconditional exact test that orders tables through Fisher exact p-values.
Asymptotic association test based on observed and expected cell counts.
Barnard's Exact Test Numerical Comparison
| Method | Statistic | P-value | Decision at .05 | Interpretive role |
|---|---|---|---|---|
| Barnard pooled-score exact | 1.9927455 | ≈ 0.0487033 | Reject narrowly | Primary unconditional exact result. |
| Fisher's exact test | OR(F/M) ≈ 1.5417 | 0.0597971 | Fail to reject | Conditional exact comparison. |
| Pearson chi-square | χ²(1) = 3.9710 | 0.0462893 | Reject | Uncorrected asymptotic comparison. |
| Yates-corrected chi-square | χ²(1) = 3.5427 | 0.0598078 | Fail to reject | Continuity-corrected asymptotic comparison. |
| Feature | Barnard | Fisher | Boschloo | Pearson chi-square |
|---|---|---|---|---|
| Reference distribution | Two binomial samples | Hypergeometric | Two binomial samples with Fisher ordering | Asymptotic chi-square |
| Conditions on success total | No | Yes | No | No exact conditioning |
| Exact for finite samples | Yes, for the stated procedure | Yes, conditionally | Yes | No; approximation |
| Nuisance parameter | Maximized | Removed by conditioning | Maximized | Not used in the same way |
| Typical strength | Good power and design match for two binomials | Familiar and straightforward | Strong exact power relative to Fisher | Fast and familiar for large samples |
| Main limitation | Implementation details must be stated | Can be conservative outside fixed-margin designs | Less familiar and computationally intensive | Can be inaccurate with sparse expected counts |
Fisher Exact Test Barnard Decision Rule
The search phrase fisher exact test barnard usually reflects a test-selection question. The decision should come from the sampling design, not from which method returns the smaller p-value. Use Fisher's exact test when both margins are fixed by design. Use Barnard's Exact Test when the two sample sizes are fixed but the outcome totals are free to vary.
Is Barnard's Exact Test Always More Powerful?
Barnard-type unconditional tests generally avoid the loss of information caused by conditioning on the success margin and often have higher power. However, Barnard's Exact Test is a family of tests, and a statement about power depends on the chosen statistic and rejection rule. Boschloo's procedure is specifically constructed as an unconditional improvement over Fisher's test.
Should Every Method Be Reported?
No. A primary method should be selected in advance. A comparison table can be useful for teaching or sensitivity analysis, but the final conclusion should not switch methods after results are known. When several methods lie close to .05, emphasize effect estimates and uncertainty rather than treating one threshold crossing as conclusive.
Barnard's Exact Test Calculator: Step-by-Step Workflow
A reliable Barnard's Exact Test calculator must do more than accept four counts. It must identify the two independent samples, define the event category, select a pooled or unpooled statistic, choose the alternative and search the nuisance parameter over its full range.
Calculator Inputs for the Worked Example
| Input | Value | Meaning |
|---|---|---|
| Group 1 successes | 333 | Female students with G3 ≥ 10. |
| Group 1 non-successes | 50 | Female students with G3 < 10. |
| Group 2 successes | 216 | Male students with G3 ≥ 10. |
| Group 2 non-successes | 50 | Male students with G3 < 10. |
| Alternative | Two-sided | Either group can have the higher probability. |
| Statistic | Pooled score | Uses a common variance estimate under H0. |
| Alpha | 0.05 | Decision threshold selected before analysis. |
Calculator Step 1: Confirm the Sample Totals
The two independent sample sizes must be 383 and 266. If a calculator reports 549 and 100 as the binomial sample sizes, it is treating outcome totals as samples rather than female and male groups.
Calculator Step 2: Confirm the Observed Statistic
The pooled-score statistic should have an absolute value close to 1.9927455. A different value can indicate different coding, group direction, event definition or pooled setting.
Calculator Step 3: Inspect the Nuisance Search
The search should cover π from 0 to 1 and return the maximum two-sided tail probability. A finer grid or larger sampling parameter improves precision but requires more computation.
Calculator Step 4: Read Counts and Effect Sizes Before the P-Value
Report the two proportions, risk difference, risk ratio and odds ratio beside the exact test. A calculator that returns only “significant” or “not significant” is incomplete for scientific reporting.
Calculator Step 5: Interpret the Result
Read the exact p-value together with the two observed proportions, the 5.74-percentage-point risk difference and the small association measure. A result close to .05 should be described as borderline rather than decisive.
Seven Python and R Barnard's Exact Test Chart Interpretations
The Python and R workflows use the same seven Barnard's Exact Test figures. Each visual is shown once and followed by one complete interpretation that applies across both software environments.
Chart 1: Barnard's Exact Test - 2 × 2 Contingency HeatmapPython + R
The largest cell is female students with G3 ≥ 10. Both groups contain 50 students below 10, but the group totals are different.
Female: 333 outcome-present and 50 outcome-absent, n = 383. Male: 216 outcome-present and 50 outcome-absent, n = 266.
Equal non-event counts do not mean equal non-event rates. The female non-event rate is 13.05%, whereas the male non-event rate is 18.80%.
The heatmap anchors every later proportion, effect size and exact probability in the four observed cells.
Chart 2: Barnard's Exact Test - Within-Group Outcome PercentagesPython + R
The female outcome-present segment is larger, and the male outcome-absent segment is larger.
Female G3 ≥ 10: 86.945%. Male G3 ≥ 10: 81.203%. Difference: 5.742 percentage points.
The difference is easier to interpret after removing the unequal group-size effect from the bar heights.
Barnard's Exact Test evaluates whether a standardized difference this large is unusual under equal probabilities.
Chart 3: Barnard's Exact Test - Outcome Proportions with 95% Confidence IntervalsPython + R
The female point estimate is higher, while the two group intervals overlap in the low-to-mid .80 range.
Female: 0.86945, 95% CI [0.83201, 0.89956]. Male: 0.81203, 95% CI [0.76076, 0.85441].
Separate one-proportion intervals quantify each group but do not directly test the between-group difference.
Visual interval overlap is not a formal significance rule, especially when the final exact p-value is close to .05.
Chart 4: Barnard's Exact Test - Risk Ratio and Odds RatioPython + R
Both point estimates are below 1 when male students are compared with female students.
Risk ratio: 0.93396, 95% CI [0.87113, 1.00131]. Odds ratio: 0.64865, 95% CI [0.42289, 0.99492].
The risk ratio describes relative probability. The odds ratio describes relative odds and appears farther from 1 because the outcome is common.
Confusing odds with probability can exaggerate the practical impression of the association.
Chart 5: Barnard's Exact Test - Comparison-Test P-ValuesPython + R
The p-values cluster near .05, so the binary decision changes across methods and Barnard implementations.
Displayed chart: Barnard ≈ .067, Fisher ≈ .060, Pearson ≈ .046 and Yates ≈ .060. Pooled two-binomial Barnard result used in the article: ≈ .049.
The figure illustrates that exact-test specification and conditioning rules matter when evidence is borderline.
Method selection should follow study design and a declared implementation, not whichever bar crosses .05.
Chart 6: Barnard's Exact Test - Nuisance-Parameter ProfilePython + R
The tail probability rises sharply away from the boundaries and reaches an off-centre maximum near .049.
Maximum tail: approximately 0.0487. The profile is nearly symmetric across complementary null probabilities.
The Barnard p-value is the highest point on this profile, not the tail probability at an arbitrary single π.
Nuisance maximization is the defining step that makes the procedure unconditional and exact for the stated rejection rule.
Chart 7: Barnard's Exact Test - Observed and Expected CountsPython + R
Female success and male non-success are above expectation; the opposite cells are below expectation.
Expected counts: 323.986, 59.014, 225.014 and 40.986. Each observed deviation has magnitude about 9.014.
The deviations produce Pearson χ²(1) = 3.9710, p = .0463.
All expected counts exceed 40, so disagreement among methods is not caused by sparse expected cells; it reflects conditioning and continuity rules near .05.
Barnard's Exact Test in Python, R, SPSS, SAS and Excel
Barnard's Exact Test in Python
SciPy provides a dedicated Barnard exact function. Its columns represent the two binomial experiments, so the female and male groups must be placed in columns.
- Use a 2 × 2 integer array.
- Choose two-sided, less or greater.
- Set pooled=True for the pooled-score specification.
- Increase the sampling parameter to check numerical stability.
- Calculate effect sizes and intervals separately.
Worked target: table [[333, 216], [50, 50]], statistic ≈ 1.9927 and p ≈ .0487.
Barnard's Exact Test in R
The Barnard package implements pooled score and unpooled Wald versions and lets the analyst control nuisance-parameter resolution through dp.
- Arrange the two independent samples in columns.
- Use
pooled=TRUEfor the score statistic. - Use a sufficiently fine
dpvalue. - Inspect the nuisance matrix when the result is close to alpha.
- Report the package and version.
Barnard's Exact Test in SPSS
SPSS CROSSTABS is useful for the observed table, row percentages, expected counts, chi-square, Fisher's exact test, risk estimates and phi/Cramer's V. A Barnard calculation can be added through validated Python integration or an extension workflow.
- Verify the event and group coding in CROSSTABS.
- Save counts and expected counts.
- Do not relabel Fisher's exact result as Barnard's Exact Test.
- Keep the Python integration table convention consistent with the SPSS table.
- Export the SPSS output for the comparison statistics.
Barnard's Exact Test in SAS
SAS PROC FREQ supports Barnard's unconditional exact test for the risk difference in a 2 × 2 table through the EXACT BARNARD request.
- Create a weighted 2 × 2 table or analyse row-level data.
- Request risk-difference output and the Barnard exact test.
- Confirm the event order and group order.
- Use ODS output for a reproducible table.
- Report the SAS version and exact-test options.
Barnard's Exact Test in Excel
Excel can reproduce counts, proportions, score statistic, expected counts and effect estimates. A complete unconditional exact p-value requires enumeration of all possible count pairs and maximization over π.
- Use worksheet formulas for descriptive values.
- Use dynamic arrays, VBA or Office Script for enumeration.
- Do not substitute CHISQ.TEST and call it Barnard's Exact Test.
- Document the nuisance grid and tie rule.
- Cross-check the final p-value in validated statistical software.
Software Reporting Standard
Every software section should report the same design information: two independent samples, event coding, alternative, pooled or unpooled score, nuisance resolution, statistic and p-value. This is more informative than claiming that different software packages “agree” without naming their settings.
- Counts: 333, 50, 216, 50
- Sample sizes: 383 and 266
- Alternative: two-sided
- Primary statistic: pooled score
- Primary p-value: approximately .0487
Expandable Code for Barnard's Exact Test
Python: SciPy pooled-score Barnard's Exact Test
import numpy as np
from scipy.stats import barnard_exact, fisher_exact, chi2_contingency
# Rows are outcome categories; columns are independent samples.
# Column 1 = Female (n=383), Column 2 = Male (n=266)
table_barnard = np.array([
[333, 216], # G3 >= 10
[ 50, 50], # G3 < 10
], dtype=int)
barnard = barnard_exact(
table_barnard,
alternative="two-sided",
pooled=True,
n=4096,
)
print("Barnard statistic:", barnard.statistic)
print("Barnard p-value:", barnard.pvalue)
# Conventional group-by-outcome table for descriptive/comparison tests
table_groups = np.array([
[333, 50], # Female
[216, 50], # Male
], dtype=int)
fisher_or, fisher_p = fisher_exact(table_groups, alternative="two-sided")
pearson = chi2_contingency(table_groups, correction=False)
yates = chi2_contingency(table_groups, correction=True)
print("Fisher OR:", fisher_or)
print("Fisher p:", fisher_p)
print("Pearson chi-square:", pearson.statistic, pearson.pvalue)
print("Yates chi-square:", yates.statistic, yates.pvalue)Python: effect sizes and Wilson intervals
import math
from statsmodels.stats.proportion import proportion_confint
female_success, female_total = 333, 383
male_success, male_total = 216, 266
p_f = female_success / female_total
p_m = male_success / male_total
risk_difference = p_m - p_f
risk_ratio = p_m / p_f
odds_ratio = ((male_success / (male_total - male_success)) /
(female_success / (female_total - female_success)))
female_ci = proportion_confint(
female_success, female_total, alpha=0.05, method="wilson"
)
male_ci = proportion_confint(
male_success, male_total, alpha=0.05, method="wilson"
)
print("Female proportion:", p_f, female_ci)
print("Male proportion:", p_m, male_ci)
print("Risk difference M-F:", risk_difference)
print("Risk ratio M/F:", risk_ratio)
print("Odds ratio M/F:", odds_ratio)R: Barnard package pooled-score workflow
# install.packages("Barnard")
library(Barnard)
# X = [n1, n2; n3, n4]
# Columns are the independent samples:
# Female = 333 success, 50 non-success
# Male = 216 success, 50 non-success
result <- barnard.test(
n1 = 333,
n2 = 216,
n3 = 50,
n4 = 50,
dp = 0.0001,
pooled = TRUE
)
print(result$contingency.matrix)
print(result$statistic)
print(result$nuisance.parameter)
print(result$p.value)
# Plot the two-sided nuisance profile
plot(
result$nuisance.matrix[, c(1, 3)],
type = "l",
xlab = "Common null probability",
ylab = "Two-sided exact tail probability"
)SPSS: crosstab and embedded-Python preparation
* sex_group: 0 = Female, 1 = Male.
* g3_event: 0 = G3 below 10, 1 = G3 at least 10.
CROSSTABS
/TABLES=sex_group BY g3_event
/FORMAT=AVALUE TABLES
/STATISTICS=CHISQ RISK PHI
/CELLS=COUNT ROW COLUMN EXPECTED RESID SRESID
/COUNT ROUND CELL.
* CROSSTABS supplies the descriptive table,
* Pearson/Yates chi-square, Fisher exact,
* risk estimates, phi and Cramer's V.
*
* Run Barnard's Exact Test through a validated
* Python integration or extension, using:
* Female sample: 333 event, 50 non-event.
* Male sample: 216 event, 50 non-event.
* Alternative: two-sided.
* Statistic: pooled score.SAS: PROC FREQ Barnard exact test
data barnard_example;
input sex $ outcome $ count;
datalines;
Female G3_GE_10 333
Female G3_LT_10 50
Male G3_GE_10 216
Male G3_LT_10 50
;
run;
proc freq data=barnard_example order=data;
tables sex*outcome / chisq riskdiff relrisk;
weight count;
exact barnard fisher;
run;Check the category order in the SAS output because risk differences and directional alternatives depend on which category is treated as the event.
Excel: descriptive and pooled-score formulas
Suggested cells
B3 = 333 Female G3 >= 10
C3 = 50 Female G3 < 10
B4 = 216 Male G3 >= 10
C4 = 50 Male G3 < 10
D3 = SUM(B3:C3) Female total
D4 = SUM(B4:C4) Male total
B5 = SUM(B3:B4) Outcome total
C5 = SUM(C3:C4) Non-outcome total
D5 = SUM(D3:D4) Grand total
E3 = B3/D3 Female proportion
E4 = B4/D4 Male proportion
E5 = E4-E3 Risk difference M-F
E6 = E4/E3 Risk ratio M/F
E7 = (B4/C4)/(B3/C3) Odds ratio M/F
F3 = D3*B5/$D$5 Expected F outcome
G3 = D3*C5/$D$5 Expected F non-outcome
F4 = D4*B5/$D$5 Expected M outcome
G4 = D4*C5/$D$5 Expected M non-outcome
Pooled proportion:
=B5/D5
Pooled score:
=(E3-E4)/SQRT((B5/D5)*(1-B5/D5)*(1/D3+1/D4))
Important:
The exact Barnard p-value requires enumeration of
all possible success-count pairs and maximization
over the common null probability.Assumptions, Limitations and Advanced Interpretation
Core Assumptions
Data Conditions
- Two independent groups
- Binary outcome
- One record per observational unit
- Independent Bernoulli trials
- Correct event and group coding
- Fixed group sample sizes
Analysis Conditions
- Prespecified alternative
- Declared pooled or unpooled statistic
- Defined two-sided ordering
- Global nuisance search
- Adequate numerical resolution
- Reproducible software specification
Is Barnard's Exact Test conservative?
The phrase Barnard's Exact Test conservative has two possible meanings. Maximizing over the nuisance parameter deliberately uses the largest null tail probability, protecting the type I error rate. In addition, discreteness means the actual rejection probability can fall below the nominal alpha for some parameter values. Barnard's Exact Test generally avoids the extra conditioning associated with Fisher's test, but no exact discrete test has a perfectly continuous p-value scale.
Why “exact” does not mean assumption-free
Exact refers to the probability calculation under the stated model. Independence, binary coding, sampling design, table arrangement and test specification remain assumptions. An exact calculation applied to dependent or wrongly coded data is not a valid answer to the intended research question.
Why table orientation matters
Some programs treat rows as samples, while others treat columns as samples. In SciPy and the R Barnard package documentation used here, the columns represent the two binomial experiments. A transposed matrix can preserve the absolute score in special tables while changing the exact sample space and p-value.
Why two-sided exact tests can differ
Discrete tables require an ordering rule for “as extreme or more extreme.” Absolute score ordering, probability ordering and doubled one-sided procedures can define different rejection regions. A full report names the chosen implementation.
Why the nuisance profile may contain several peaks
The exact rejection region is discrete, and the binomial weights assigned to its tables change with π. The resulting tail function does not have to be a simple single-peaked curve. Global search or a sufficiently fine grid is required.
Effect size versus statistical significance
The current p-value lies near .05, while Cramer's V is only 0.078. A small association can cross a significance threshold in a moderately large sample. Report practical magnitude before describing the decision.
Approximate confidence intervals versus exact-test inversion
The displayed Wilson, risk-difference, log-risk-ratio and log-odds-ratio intervals use different construction methods. They are not all exact inversions of the pooled-score Barnard test, so their boundary decisions can differ slightly. State each interval method.
Unadjusted association and confounding
The 2 × 2 table does not adjust for prior grades, study time, failures, absences, school or other variables. It describes an unadjusted association. A regression model is needed for a conditional comparison, and causal interpretation requires a suitable design.
Clustered observations
If students are clustered within schools or classes, independence can be violated. The ordinary two-binomial probability model then understates the complexity of the sampling process. A cluster-aware model or design-based method may be necessary.
Dichotomization sensitivity
Changing the threshold from G3 ≥ 10 to another cutoff changes every cell and can change the conclusion. Testing many thresholds and reporting the most favourable one inflates false-positive risk. The threshold should have substantive justification.
Zero cells and relative effects
Barnard's Exact Test can still analyse a table containing a zero cell. However, ordinary odds ratios and risk ratios may become zero or infinite, and log confidence intervals may fail. A stated correction or exact interval method is then needed.
Power and sample-size planning
Power depends on group sizes, baseline outcome probability, target difference, chosen statistic and alpha. See Statistical Power and Effect Size before designing a study around a binary outcome.
APA Reporting for Barnard's Exact Test
APA reporting is one of the most searched parts of an applied statistics guide because readers need wording they can place in assignments, theses, manuscripts and reports. A complete Barnard's Exact Test write-up includes the design, event coding, group counts or proportions, test statistic, exact p-value, effect estimate, confidence interval method, decision and a noncausal interpretation.
What to Include in an APA-Style Report
Method Information
- Two independent groups
- Binary outcome and event definition
- Two-sided or one-sided alternative
- Pooled score or unpooled Wald specification
- Software and procedure
- Significance level
Results Information
- Four observed counts
- Both group proportions
- Barnard statistic and exact p-value
- Risk difference and direction
- Relative effect where useful
- Confidence interval and practical interpretation
APA-Style Methods Sentence
“A two-sided pooled-score Barnard's unconditional exact test was conducted to compare the probability of G3 ≥ 10 between female and male students. The analysis treated the two group totals as fixed binomial sample sizes and did not condition on the combined number of students reaching the threshold.”
Full APA-Style Results Example
“The G3 ≥ 10 outcome occurred for 333 of 383 female students (86.95%) and 216 of 266 male students (81.20%). A two-sided pooled-score Barnard's Exact Test indicated borderline evidence of a difference between the two proportions, T = 1.99, p = .049. The male-minus-female risk difference was −.057, 95% CI [−.115, .000], and the male-to-female risk ratio was 0.934, 95% CI [0.871, 1.001]. The association was small, Cramer's V = .078. These results describe an unadjusted association and do not establish a causal effect of sex on final-grade attainment.”
Concise APA Results Sentence
“The proportion reaching G3 ≥ 10 was 86.95% for females and 81.20% for males; Barnard's pooled-score exact test gave T = 1.99, p = .049, with a small association, Cramer's V = .078.”
Nonsignificant Barnard's Exact Test
A two-sided Barnard's Exact Test compared Group 1 (x₁/n₁, p₁%) with Group 2 (x₂/n₂, p₂%).
The result was not statistically significant, T = statistic, p = p-value.
The risk difference was estimate, 95% CI [lower, upper]. The analysis did not provide sufficient evidence of a difference; it did not prove that the probabilities were identical.
Use this wording when the prespecified Barnard's Exact Test p-value is at or above the selected alpha level.
Significant Barnard's Exact Test
A two-sided pooled-score Barnard's Exact Test indicated that the outcome proportion differed between Group 1 (x₁/n₁, p₁%) and Group 2 (x₂/n₂, p₂%).
The Barnard's Exact Test result was T = statistic, p = p-value.
The absolute proportion difference was estimate percentage points, 95% CI [lower, upper], indicating a contextual effect description.
Describe magnitude and uncertainty; do not use the Barnard's Exact Test p-value as the effect-size description.
One-Sided Barnard's Exact Test
A one-sided pooled-score Barnard's Exact Test evaluated the prespecified hypothesis that Group 1 had a higher outcome probability than Group 2.
The Barnard's Exact Test result was statistically significant / not statistically significant, T = statistic, p = p-value.
The observed proportions were p₁% and p₂%, respectively, with a directional risk difference of estimate.
Use a one-sided Barnard's Exact Test only when the direction was specified before the data were examined.
Reusable APA Templates for Barnard's Exact Test
APA Comparison with Fisher's Exact Test
“Barnard's unconditional exact test was selected as the primary analysis because the study consisted of two fixed binomial sample sizes and a nonfixed outcome total. Fisher's conditional exact test was reported as a sensitivity comparison, p = .060.”
APA-Style Table for the Main Results
| Group or statistic | Value | 95% CI |
|---|---|---|
| Female G3 ≥ 10, n/N (%) | 333/383 (86.95%) | [83.20%, 89.96%] |
| Male G3 ≥ 10, n/N (%) | 216/266 (81.20%) | [76.08%, 85.44%] |
| Risk difference, male − female | −0.057 | [−0.115, 0.000] |
| Risk ratio, male/female | 0.934 | [0.871, 1.001] |
| Odds ratio, male/female | 0.649 | [0.423, 0.995] |
| Barnard pooled-score statistic | 1.993 | — |
| Two-sided exact p-value | .049 | — |
| Cramer's V | .078 | — |
Suggested table note: “Barnard's p-value is from a two-sided pooled-score unconditional exact test. Group proportion intervals are Wilson intervals; the risk-difference, risk-ratio and odds-ratio intervals use their stated estimation methods. M = male; F = female.”
APA Rounding Rules
- Report the test statistic to two or three decimals.
- Report p = .049 rather than p = .05 when the exact value is .0487.
- Use p < .001 only when the value is actually below .001.
- Give proportions as percentages with one or two decimals.
- Give effect estimates and confidence limits with consistent precision.
- Do not report p = .000.
APA Interpretation Words to Use
| Avoid | Use instead | Reason |
|---|---|---|
| “The test proved a difference.” | “The test provided evidence of a difference.” | Hypothesis tests quantify evidence; they do not prove a population claim. |
| “The groups were the same.” | “The test did not detect sufficient evidence of a difference.” | Failure to reject is not evidence of exact equality. |
| “Males were 35% less likely.” | “Male odds were about 35% lower; the risk ratio was 0.934.” | Odds and probability are not interchangeable. |
| “The effect was important because p < .05.” | “The result was borderline and the association was small.” | Statistical and practical significance are different. |
| “Barnard's Exact Test is nonparametric.” | “Barnard's Exact Test is an exact unconditional test for two binomial proportions.” | The exact model description is more precise. |
Common Barnard's Exact Test Mistakes and How to Correct Them
| Common mistake | Why it is a problem | Correct approach |
|---|---|---|
| Using Barnard's Exact Test for paired data | The two-binomial independence assumption is violated. | Use a paired binary method for matched or repeated observations. |
| Passing the table in the wrong software orientation | The program can treat outcome totals as sample sizes. | Verify that the two binomial sample totals are 383 and 266. |
| Reporting only “exact test” | Readers cannot identify the statistic or conditioning rule. | Name Barnard, pooled or unpooled, alternative and software. |
| Choosing Barnard only because its p-value is smaller | This is outcome-driven test selection. | Select the method from the sampling design before viewing results. |
| Ignoring the nuisance-parameter search | The result may not be the required maximum tail probability. | Use a global search or fine grid and check stability. |
| Calling Fisher's exact output Barnard's Exact Test | The two tests use different sample spaces. | Label every exact procedure correctly. |
| Confusing the odds ratio with the risk ratio | The outcome is common, so the measures differ substantially. | Report the absolute difference and risk ratio before the odds ratio. |
| Interpreting separate interval overlap as a formal test | Overlap is not equivalent to testing the difference. | Use a direct exact test and a difference interval. |
| Reporting p = .000 | A p-value is not exactly zero. | Use p < .001 when appropriate. |
| Claiming causation from a 2 × 2 association | The table is unadjusted and observational. | Use association wording and discuss confounding. |
| Ignoring the original quantitative outcome | Dichotomization discards grade information. | Report the threshold analysis and consider a full-scale analysis. |
| Calling a small effect large because it is significant | Sample size affects significance. | Interpret the 5.74-point difference and Cramer's V = .078 directly. |
Barnard's Exact Test Reports and Worked Excel Download
R Barnard's Exact Test ReportR exact-test output, effect estimates, nuisance information and chart pages.
SPSS Output PDFCrosstabulation, expected counts, chi-square, Fisher, risk estimates and association measures.
Worked Excel WorkbookEditable 2 × 2 calculations, proportions, expected counts and effect-size formulas.
Frequently Asked Questions About Barnard's Exact Test
The Barnard's Exact Test questions below address calculation, assumptions, software, effect sizes and reporting without repeating the full explanations already provided above.
What is Barnard's Exact Test?
Barnard's Exact Test is an unconditional exact test for comparing two independent binomial proportions in a 2 × 2 table. It fixes the two sample sizes, allows the combined outcome total to vary and maximizes the exact tail probability over an unknown common null probability.
What is the null hypothesis?
The usual null hypothesis is equality of the two outcome probabilities, p1 = p2. A two-sided alternative states that the probabilities differ in either direction.
When should I use Barnard's Exact Test?
Use it for a 2 × 2 table with two independent groups, one binary outcome, fixed group sample sizes and a success total that was not fixed by design.
When should I use Fisher's exact test instead?
Fisher's exact test directly matches a design in which both margins are fixed. Barnard's Exact Test better matches two independent binomial samples whose group totals are fixed but whose outcome total can vary.
Is Barnard's Exact Test more powerful than Fisher's exact test?
Unconditional exact procedures often gain power by avoiding conditioning on the outcome margin. The precise comparison depends on the Barnard statistic and rejection rule.
Is Barnard's Exact Test conservative?
It controls size by taking the largest null tail over the nuisance parameter, and discreteness can make actual rejection probabilities lower than alpha at some parameter values. It is generally less affected by fixed-margin conditioning than Fisher's test.
Does Barnard's Exact Test require expected counts above 5?
No. It uses exact binomial probabilities. Expected counts are relevant to the quality of chi-square approximations, not to the validity of Barnard's Exact Test.
Can Barnard's Exact Test be used with unequal sample sizes?
Yes. The current group sizes are 383 and 266, and the exact calculation incorporates both denominators.
Can it be used for paired data?
No. Paired or matched binary outcomes require a paired method because the independence assumption is not satisfied.
What is the Barnard statistic in this example?
The pooled-score statistic is approximately 1.9927455 when female and male are treated as the two independent binomial samples.
What is the p-value in this example?
For the two-sided pooled-score specification used in the main analysis, the unconditional p-value is approximately 0.0487033.
Why can software return different Barnard p-values?
Programs can differ in table orientation, pooled versus unpooled variance, two-sided ordering, nuisance search resolution and numerical optimization. Report the exact procedure and settings.
What is the nuisance parameter?
It is the unknown common outcome probability under the equality null. Barnard's Exact Test maximizes the exact tail probability over all allowed values of this parameter.
What is pooled=True in Python?
It requests the pooled score statistic, which uses a common variance estimate under the null. pooled=False uses an unpooled Wald denominator and can change the rejection region.
What does the Python n option do?
It controls the number of nuisance sampling points used by the numerical search. More points increase computation and help verify precision.
What does dp mean in R?
In the Barnard R package, dp controls the resolution of the nuisance-parameter search. A smaller dp gives a finer grid.
Can SPSS calculate Barnard's Exact Test?
SPSS can create the table and comparison statistics. Barnard's Exact Test can be added through validated Python integration or an extension.
Can SAS calculate Barnard's Exact Test?
Yes. SAS can request Barnard's exact test in PROC FREQ. The category order and event definition should be verified before interpretation.
Can Excel calculate Barnard's Exact Test?
Excel can calculate descriptive quantities and the observed score. A complete exact p-value requires enumeration and nuisance maximization through dynamic arrays, VBA, Office Script or external software.
What effect size should be reported?
Report the risk difference first, followed by a risk ratio or odds ratio when useful and an association measure such as Cramer's V.
What is the effect size here?
The male-minus-female risk difference is −0.0574, the risk ratio is 0.934, the odds ratio is 0.649 and Cramer's V is 0.078.
Why does the odds ratio look stronger than the risk ratio?
The outcome is common in both groups. Odds move more sharply than probabilities when event rates are high.
Does p = .049 mean the effect is important?
No. The p-value is borderline, and Cramer's V indicates a small association. Practical importance depends on context and consequences.
Can Barnard's Exact Test prove causation?
No. It tests an unadjusted association. Causal conclusions require an appropriate design and control of confounding and dependence.
What should an APA report include?
Include the group counts or proportions, test name and specification, statistic, exact p-value, risk difference, confidence interval method, effect size and a careful interpretation.
How should p = .048703 be rounded?
Report p = .049 in ordinary APA-style prose and retain more digits in a technical table when useful.
Should I report Fisher's exact test too?
It can be reported as a prespecified sensitivity comparison, but the primary method should be selected from the design rather than from the smallest p-value.
What if the outcome threshold changes?
The four table counts and the conclusion can change. Thresholds should be substantively justified and preferably selected before analysis.
Can Barnard's Exact Test adjust for covariates?
No. Use an appropriate binary regression model when adjustment, interactions, clustering or prediction are required.
What is the main conclusion from this worked example?
Female students had a 5.74-percentage-point higher observed G3 ≥ 10 rate. The pooled-score Barnard result was borderline at p ≈ .049, and the association was small.
Barnard's Exact Test Conclusion
Barnard's Exact Test is an exact unconditional method for comparing two independent binomial proportions. It is most useful when the two sample sizes are fixed but the total number of outcomes is not fixed, allowing the analysis to avoid conditioning on both margins.
In the worked example, 333 of 383 female students and 216 of 266 male students had G3 ≥ 10. The observed proportions were 86.95% and 81.20%, producing a male-minus-female risk difference of −5.74 percentage points. The two-sided pooled-score Barnard statistic was approximately 1.993, with p ≈ .049.
The evidence is borderline and the association is small, Cramer's V = .078. A strong report therefore includes the four counts, both proportions, exact-test specification, risk difference, confidence intervals, comparison-test context and a noncausal interpretation.
