E-valuate

In an impact evaluation, a test is undertaken to see whether a change measured in an intervention group is greater or less than that in a comparison group, and whether the change is statistically significant. The intervention group undertakes a ‘treatment’ of some kind, which the researcher would like to test in terms of its effectiveness in changing a variable of interest. The comparison group does not receive the treatment and therefore serves as a control for whether the treatment has an impact. The selection of individuals into treatment and comparison groups could be random as with a randomised control trial (RCT), but they could be matched through other means, often known as quasi-experimental methods. The same calculations apply for these different sampling methods in terms of sample size, power and calculating the minimum detectable effect size.

The bigger a change is measured in the variable of interest, the more likely it is to be statistically significant for a given sample size. Other variables are also vital, such as the power of the evaluation, where a larger sample size would be needed for 90% statistical power than for 80% for example. In addition, researchers and evaluators need to consider whether clustering needs to be applied. E-valuate provides all the key variables required to make these calculations.

The key variables involved and statistical terms associated are explained in the instructions below. Follow the instructions alongside your calculation to ensure you determine the most robust and accurate results for your evaluation.

The three calculators and what they do....

E-valuate has three calculators depending on which variables are known or set. Click on the links below to find the instructions for each calculator.

The Sample Size Calculator allows you to find out the sample size required for your evaluation for a given level of statistical significance and power, as well as the minimum detectable effect size you would like to measure.

The Power Calculator provides a measure of the statistical significance of your evaluation for a given sample size and minimum detectable effect size.

The Effect Size Calculator allows you to measure what your minimum detectable effect size will be for a given sample size and measure of statistical significance.