On this page
A statistical test gets chosen the same way in a surprising number of studies: whichever option is easiest to find in SPSS, or whatever test a colleague used in a similar-looking paper. The results come back, a p-value appears, and the paper moves forward — until a reviewer asks why a parametric test was used on a sample of 14, or why chi-square was applied to a table with expected cell counts under 5.
At that point, the honest answer is often "we didn't check," and the fix means re-running the analysis under revision deadline pressure — sometimes with a result that no longer supports the original conclusion. The test should be chosen at the protocol stage, based on two things: the type of data and the study design. This article walks through both.
Why This Happens
Statistics is taught in isolation from real datasets
Most medical curricula cover statistical concepts once, in the abstract, disconnected from the messy reality of an actual clinical dataset with missing values, small subgroups, and skewed distributions.
The decision defaults to whoever has the software
In many teams, "who runs the statistics" is decided by software access, not statistical training — so the test chosen is often the first one that produces a p-value, rather than the one the data actually supports.
Significance gets confused with correctness
A p-value under 0.05 feels like validation that the right test was used. It isn't. The wrong test can produce a significant — or non-significant — result that doesn't reflect what the data actually shows.
Common Mistakes We See
- Skipping the normality check. Applying a t-test or ANOVA without first testing whether continuous data is normally distributed (Shapiro-Wilk, or visual inspection for larger samples).
- Treating ordinal data as continuous. Pain scores, Likert scales, and similar ordinal variables usually call for non-parametric tests, not a t-test or Pearson correlation.
- Ignoring paired or repeated-measures data. Before/after measurements on the same patients need a paired test — using an independent-samples test here inflates the sample size artificially and invalidates the result.
- Using chi-square with small expected cell counts. When more than 20% of cells have an expected count under 5, Fisher's exact test is the correct choice, not chi-square.
- Running multiple comparisons without correction. Testing many subgroups without a Bonferroni or similar adjustment inflates the risk of a false-positive finding.
Practical Recommendations
The test depends on two questions: what type of data are you comparing, and how many groups are involved?
| Data type | Two groups | More than two groups |
|---|---|---|
| Continuous, normally distributed | Independent-samples t-test | One-way ANOVA |
| Continuous, not normally distributed | Mann-Whitney U test | Kruskal-Wallis test |
| Paired / repeated continuous | Paired t-test | Repeated-measures ANOVA |
| Categorical | Chi-square (or Fisher's exact if cells are small) | Chi-square |
| Survival / time-to-event | Kaplan-Meier + log-rank test | Cox proportional hazards regression |
| Predicting a binary outcome | Logistic regression | |
| Predicting a continuous outcome | Linear regression | |
Quick Checklist
- Data type is identified for every variable (continuous, ordinal, categorical, time-to-event)
- Normality is tested before choosing between parametric and non-parametric methods
- Paired vs. independent samples is confirmed before selecting the test
- Expected cell counts are checked before using chi-square
- A correction is applied if multiple comparisons are being run
- The statistical plan is written into the protocol before data collection begins
- A statistician has reviewed the plan, even briefly, before analysis starts
Need Expert Guidance?
Book a free call with PureMed and let our research specialists help you save valuable time, avoid costly research mistakes, and move confidently toward publication.
Frequently Asked Questions
Is SPSS enough, or do I need R?
SPSS is sufficient for the vast majority of clinical studies using standard tests. R becomes valuable for more advanced modeling — for example complex mixed-effects models or certain meta-analysis methods — but it isn't required for most single-center studies.
What sample size justifies using a parametric test?
There's no fixed cutoff, but as a practical guide, samples above roughly 30 per group are often reasonably robust to moderate departures from normality due to the central limit theorem. Below that, formally testing normality matters more.
How do reviewers usually catch the wrong test?
Most often through the reported sample size next to a parametric test, through a chi-square table with visibly small cell counts, or through paired data analyzed as if it were independent — all of which are identifiable without re-running the analysis.
Can I change the statistical test after data collection is finished?
Only if the original choice was genuinely incorrect for the data type — not simply because a different test would produce a more favorable p-value. Any change should be documented and justified in the methods section.