Power Analysis Guide

Calculate appropriate sample sizes for your study using power analysis, understand effect sizes, and avoid underpowered or wastefully overpowered designs.

Core Concepts

The Four Parameters of Power Analysis

Every power analysis involves four interrelated quantities. Fix any three to solve for the fourth:

Parameter	Symbol	Definition	Typical Value
Effect size	d, r, f, etc.	Magnitude of the phenomenon you expect to detect	Varies by field
Significance level (alpha)	alpha	Probability of Type I error (false positive)	0.05
Statistical power (1 - beta)	1 - beta	Probability of detecting a true effect	0.80 or 0.90
Sample size	N	Number of observations needed	Solve for this

Error Types

	H0 is true (no effect)	H0 is false (effect exists)
Reject H0	Type I error (alpha)	Correct (power = 1 - beta)
Fail to reject H0	Correct (1 - alpha)	Type II error (beta)

Effect Size Conventions

Cohen's d (Two-Group Comparison)

d = (M1 - M2) / SD_pooled

Size	Cohen's d	Interpretation
Small	0.2	Subtle, may need large N to detect
Medium	0.5	Noticeable, typical in social sciences
Large	0.8	Obvious, often visible without statistics

Correlation (r)

Size	r	r-squared
Small	0.1	1% variance explained
Medium	0.3	9% variance explained
Large	0.5	25% variance explained

Cohen's f (ANOVA)

Size	f	Equivalent eta-squared
Small	0.10	0.01
Medium	0.25	0.06
Large	0.40	0.14

Odds Ratio (Logistic Regression)

Size	OR
Small	1.5
Medium	2.5
Large	4.0

Power Analysis in Python (statsmodels)

Two-Sample t-Test

from statsmodels.stats.power import TTestIndPower

analysis = TTestIndPower()

# Solve for sample size
n = analysis.solve_power(
    effect_size=0.5,    # Cohen's d = medium
    alpha=0.05,         # Significance level
    power=0.80,         # 80% power
    ratio=1.0,          # Equal group sizes
    alternative='two-sided'
)
print(f"Required N per group: {int(n) + 1}")  # Output: 64

# Solve for power (given N)
power = analysis.solve_power(
    effect_size=0.5,
    alpha=0.05,
    nobs1=50,
    ratio=1.0,
    alternative='two-sided'
)
print(f"Power with N=50 per group: {power:.3f}")  # Output: 0.697

Paired t-Test

from statsmodels.stats.power import TTestPower

analysis = TTestPower()
n = analysis.solve_power(
    effect_size=0.3,    # Small-medium effect
    alpha=0.05,
    power=0.80,
    alternative='two-sided'
)
print(f"Required N (paired): {int(n) + 1}")  # Output: 90

One-Way ANOVA

from statsmodels.stats.power import FTestAnovaPower

analysis = FTestAnovaPower()
n = analysis.solve_power(
    effect_size=0.25,   # Cohen's f = medium
    alpha=0.05,
    power=0.80,
    k_groups=4          # Number of groups
)
print(f"Required N per group: {int(n) + 1}")  # Output: 45

Chi-Square Test

from statsmodels.stats.power import GofChisquarePower

analysis = GofChisquarePower()
n = analysis.solve_power(
    effect_size=0.3,    # Cohen's w = medium
    alpha=0.05,
    power=0.80,
    n_bins=4            # Degrees of freedom + 1
)
print(f"Required total N: {int(n) + 1}")

Multiple Regression

from statsmodels.stats.power import FTestPower

analysis = FTestPower()
# For R-squared: convert to f2 = R2 / (1 - R2)
r_squared = 0.10  # Expected R-squared for the model
f2 = r_squared / (1 - r_squared)  # f2 = 0.111

n = analysis.solve_power(
    effect_size=f2,
    alpha=0.05,
    power=0.80,
    df_num=5            # Number of predictors
)
# n returned is df_denom; total N = n + df_num + 1
total_n = int(n) + 5 + 1
print(f"Required total N: {total_n}")

Power Analysis in R (pwr Package)

library(pwr)

# Two-sample t-test
result <- pwr.t.test(d = 0.5, sig.level = 0.05, power = 0.80,
                     type = "two.sample", alternative = "two.sided")
cat("N per group:", ceiling(result$n), "\n")

# Correlation test
result <- pwr.r.test(r = 0.3, sig.level = 0.05, power = 0.80,
                     alternative = "two.sided")
cat("Total N:", ceiling(result$n), "\n")

# One-way ANOVA (4 groups)
result <- pwr.anova.test(k = 4, f = 0.25, sig.level = 0.05, power = 0.80)
cat("N per group:", ceiling(result$n), "\n")

# Chi-square test
result <- pwr.chisq.test(w = 0.3, df = 3, sig.level = 0.05, power = 0.80)
cat("Total N:", ceiling(result$N), "\n")

# Plot power curve
result <- pwr.t.test(d = 0.5, sig.level = 0.05, power = NULL,
                     n = seq(10, 200, by = 5))
plot(result)

Using G*Power (Desktop Application)

G*Power (gpower.hhu.de) is a free, widely-used GUI application for power analysis:

1. Select test family: t-tests, F-tests, chi-square, z-tests, exact tests
2. Select statistical test: e.g., "Means: Difference between two independent means (two groups)"
3. Select type of analysis: A priori (compute N), Post hoc (compute power), Sensitivity (compute detectable effect)
4. Input parameters: Effect size, alpha, power, allocation ratio
5. Calculate: Click "Calculate" to get the result
6. Plot: Use "X-Y plot for a range of values" to visualize power curves

Practical Recommendations

Choosing Effect Sizes

Do NOT blindly use Cohen's conventions. Instead:

1. Literature review: Find effect sizes reported in similar studies
2. Pilot data: Run a small pilot study to estimate the effect
3. Smallest effect of interest (SESOI): What is the smallest effect that would be practically meaningful?
4. Meta-analyses: Use pooled effect sizes from meta-analyses in your area

Common Mistakes

Mistake	Problem	Solution
Post hoc power analysis	Circular and uninformative after data collection	Only do a priori power analysis
Using Cohen's "medium" by default	May be unrealistic for your field	Base on literature or SESOI
Ignoring attrition	Actual N may be lower than planned	Inflate N by 10-20% for expected dropout
Forgetting multiple comparisons	Bonferroni corrections reduce power	Adjust alpha for the number of tests
Not reporting power analysis	Reviewers cannot evaluate adequacy	Always report in Methods section

Reporting Template

A priori power analysis was conducted using [G*Power 3.1 / statsmodels / R pwr].
For a [test name] with an expected effect size of [d/r/f = X] (based on
[source: previous study / meta-analysis / pilot data]), alpha = .05, and
power = .80, the required sample size was [N per group / total N]. To account
for an estimated [X]% attrition rate, we recruited [final N] participants.