Statistical Test Example

Now that you are familiar with what questions need to be answered in order to select a statistical test, let's practice selecting the right test.

The table below illustrates how you might go about selecting a statistical test. The table below also shows that based on the study design and the distribution of the data the statistical test that is best for analysis can change.

Study Design

Groups

Time points

Data Distribution

Variable type

Test

Pretest/Posttest: Studying an eight-week tutoring component at an after-school program. Assessing student satisfaction of 40 participants. Comparing the same students’ satisfaction using a pretest before the tutoring begins and a posttest after the tutoring component ends.

One

Two

Normal

Ordinal: 1=Very satisfied, 2=Satisfied, 3=Not at all satisfied

Paired T-test

Same as above.

One

Two

Non-normal

Same as above

Wilcoxon sign-rank test

Pretest, Posttest, and Control Group: Comparing the satisfaction of two groups of students in different after-school programs. Each group has 25 participants. Comparing the satisfaction scores using a pretest before the intervention and a posttest after the intervention.

Two

Two

Non-Normal

Same as above

Wilcoxon rank-sum test

Pretest, Posttest: Assessing weight loss after a nutrition intervention among the one group of 50 students who receive the intervention. Would like to determine if there is a relationship between participation in the intervention and weight loss. Weight is measured before and after the intervention.

One

Two

Normal

Continuous (ratio): weight in pounds

Paired T-test

 

 

These statistics can tell us if the results/relationships we observe are real or just due to chance. Associated with each statistic is a p-value that tells us if something is statistically significant. If someone says the test was statistically significant they mean the results are likely not due to random chance.

For many statistical tests the results are considered significant if the p-value is .05 or less. This is based on a level of 95% confidence. This cut-off or criteria of .05 was set during the early development of statistical methods and is somewhat arbitrary. This is why sometimes a result may be statistically significant without necessarily being meaningful.

It is important to understand also that a result of statistical significance does not mean the effect is meaningful. Sometimes small effects can be statistically significant simply because a very large number of people have participated in the research. In order to account for this, the effect size is often calculated in research.

The effect size is a measure of magnitude or strength (versus the p-value of statistical tests which tell us if a relationship is due to chance or not) and it lets researchers know if the results are meaningful or just due to the large number of people who participated in the research.

 

 

    Test your knowledge of Types of Analysis                          Explore Additional Resources