The idea
A hypothesis test asks: is this result too unusual to be chance? You assume a statement is true (the null hypothesis), then check whether the data contradicts it strongly enough to reject it.
The steps
- State the hypotheses.
- Identify the test statistic and its distribution.
- Choose the significance level (α).
- State the decision rule (the critical value).
- Collect data and calculate the statistic.
- Decide: reject or fail to reject H0.
Null and alternative
- H0 (null): the claim you test, which always contains the equality, such as μ = 0 or μ ≤ 0.
- Ha (alternative): what you conclude if you reject H0.
A two-tailed test (H0: μ = 0 vs Ha: μ ≠ 0) rejects in either tail. A one-tailed test (H0: μ ≤ 0 vs Ha: μ > 0) rejects in one tail. To show something is true, put it in Ha.
The test statistic
Test statistic = (sample statistic − hypothesized value) / standard error
Example. 64 monthly excess returns average 1.2% with s = 4%. Is the true mean different from zero at 5%?
- Standard error = 4 / √64 = 0.5%.
- t = (1.2 − 0) / 0.5 = 2.4.
- Two-tailed 5% critical value ≈ 1.96 (with 63 degrees of freedom, t is close to z).
- 2.4 > 1.96, so reject H0: the mean excess return is significantly different from zero.
Note the wording: you fail to reject H0, you never “accept” it.
Errors and power
| H0 true | H0 false | |
|---|---|---|
| Reject H0 | Type I error (probability α) | Correct (power) |
| Fail to reject H0 | Correct | Type II error (probability β) |
Lowering α reduces Type I errors but raises Type II errors, for a given sample size. A larger sample reduces both. Power = 1 − β, the chance of correctly rejecting a false null.
The p-value
The p-value is the smallest significance level at which you would reject H0. If p = 0.02, you reject at 5% but not at 1%. Rule: reject when p < α.
Which test?
| Question | Test | Distribution |
|---|---|---|
| Is a mean equal to a value? | t-test (z if variance known) | t, df = n − 1 |
| Are two means equal (independent samples)? | Difference-in-means t-test | t |
| Are two means equal (paired, dependent samples)? | Paired comparisons t-test on the differences | t |
| Is a variance equal to a value? | Chi-square test: (n − 1)s² / σ0² | Chi-square, df = n − 1 |
| Are two variances equal? | F-test: s1² / s2² (larger on top) | F |
Use the paired test when the samples are linked, such as two strategies measured over the same months.
Statistical versus economic significance
A result can be statistically significant but too small to matter after trading costs, taxes or risk. Always ask whether a significant result is also economically meaningful.
Exam focus
Expect to set up hypotheses, compute a t-statistic, compare it with a given critical value, interpret a p-value, identify Type I and II errors, and choose the right test for a situation.