Welch's t-test

Two groups with unequal spread.

When should I use Welch's t-test?

Welch's t-test is what Chavery runs when Independent t-test does not hold: the groups being compared are about equally spread out. A group that varies far more than the others makes the comparison of their averages unreliable. When the check fails, the parametric result would overstate what the data supports, so the engine hands over to this test rather than running the other one anyway.

What your data needs to look like

Two sets of measurements.

How researchers arrive at Welch's t-test

Most people reach this test by way of another one. Chavery routes here when an assumption of the parametric test does not hold:

• Independent t-test — when equal variance fails (Levene's test).

That decision is made from your data, not from a preference. It is the same rule every time, which is why two people with the same numbers get the same test.

What Chavery does with it

It checks the assumptions first and says whether each one held. When an assumption fails it names the alternative test rather than running this one anyway.

Then it reports the statistic, the p-value, a confidence interval and an effect size, and writes the sentence you would put in your results chapter. The same numbers always give the same answer; nothing is generated by a language model.

Reporting it

A p-value on its own is not a result. Chavery reports the effect size beside it, because a difference can be statistically significant and too small to matter, and an examiner will ask which yours is. More on effect size.

Not sure this is the test you need? Work through the question Chavery asks and it will name the test your design calls for.

Related tests

• Independent t-test — Do two separate groups differ on average?
• Kruskal-Wallis — Three or more groups, without assuming normality.
• Mann-Whitney U — Two groups, without assuming normality.
• One-sample t-test — Is this group's mean different from a known value?
• One-way ANOVA — Three or more groups.
• Paired t-test — Before and after, in the same people.
• Wilcoxon signed-rank — Before and after, without assuming normality.