Choosing sample size before doing a study is important.

“You might ask: Why bother with calculating sample size before the study starts? Why not do the analyses as you collect data? If you don’t have a significant result, then collect some more data, and reanalyze. If you do obtain a statistically significant result, stop the study.

The problem with this approach is that you’ll keep going if you don’t like the result, but stop if you do like the result. The consequence is that the chance of obtaining a "significant" result if the null hypothesis were true is a lot higher than 5%.”

If you want to understand why, please read the original article below.

http://www.graphpad.com/library/BiostatsSpecial/article_151.htm

 

Here is how you can choose the sample size.

For surveys, you can use the following tool from surveysystem.com.

http://www.surveysystem.com/sscalc.htm

Their calculation is based on the following formula (http://www.surveysystem.com/sample-size-formula.htm).

image

Where

Z = Z value (e.g. 1.96 for 95% confidence level)
p = percentage picking a choice, expressed as decimal
(.5 used for sample size needed)
c = confidence interval, expressed as decimal
(e.g., .04 = ±4)

Correction for finite population

image

where pop = population

 

For controlled experiment, the calculation can be more complicated.

Below is a simplified formula provided by http://www.graphpad.com/faq/viewfaq.cfm?faq=1038.

  • Step 1 is to state the smallest effect you want to detect expressed as the difference in one group minus the difference in the other, with the results normalized to the expected SD.
  • Step 2 is to divide the result in step 1 by 2.00 to get the standardized effect size ES.
  • Step 3 is to look in the table below to look up the needed sample size (per group, and total for the entire experiment). This table is abridged from Table 9-26 of  by Bausell and Li, who unfortunately do not adequately explain how it is computed.

image

 

More information about power analysis and effect size can be found in http://www.graphpad.com/index.cfm?cmd=library.page&pageID=19&categoryID=4.

Written by Shengdong Zhao

Shen is an Associate Professor in the Computer Science Department, National University of Singapore (NUS). He is the founding director of the NUS-HCI Lab, specializing in research and innovation in the area of human computer interaction.