Even though I learned power analysis in elementary statistics class, I totally forget it because I never use it. As we have to put power analysis in our grant, I am forced to relearn this topic.
Definition: type II error:
- null hypothesis should be rejected but we don’t
- or, false negative (there is an effect but fail to detect it)
(Null hypothesis is usually boring, e.g. mean=0, or no difference between two samples etc. So null hypothesis usually means no effect, or negative. Rejecting null hypothesis is “finding positive effect” and thus usually “exciting”. )
Definition: the probability of type II error is called beta
Definition: power is 1 – beta
How to calculate beta (or power)?
You need (1) null hypothesis (2) alpha, type I error cut off, usually 0.05 (3) standard deviation (or distribution) of samples
Step:
- calculate the decision point (to reject or accept null hypo) based on your null hypo and alpha
- calculate beta based on the decision point above and (3)
Example:
You null hypothesis is “mean of A is 0”. You measured A 100 times. The mean is 1 and standard deviation is 10 (and thus the standard deviation of mean is 10/sqrt(100)=1). You are deciding if the null hypothesis should be rejected and what is beta if alpha is 0.05. You calculate and find the p value is 0.1587, which is bigger than 0.05 so you accept the null hypothsis.
Then what is the probability that this decision is indeed wrong (you accept null hypo but indeed null hypo is false)? or beta?
As you make decisions based on alpha = 0.05, that means your decision point of mean of sample is 1.65: if the mean of sample is bigger than 1.65, you reject null; if smaller than 1.65, you accept it (like in our case where mean is 1). So beta equals to the probability that you find mean of sample is <1.65, while the true mean is 1. And this probability is 0.74. And power is thus 0.26.
To increase power, you usually need to increase the sample size. In our case, as the standard deviation of sample is very high, we need a lot more samples. If our N = 400, then our decision point will become 0.83 and the power will increase to 0.63.
This is very nice but I don’t understand where you got 1.65 as the decision point of sample mean. Thank you!
Hi,
I came through your article while i was searching on how to answer some statistic questions posted to me. I hope you are generous enough probably to help me out with the questions as statistics are totally beyond my field.
The questions are: Calculate the coeeficient of variance and use it to determine the power of the study.
Here is the scenario:
We have 1 group of 22 subjects which was divided into 2 groups. Each group will be given TEST product or REFERENCE product and blood samples will be taken. The group will be switched to receive the other treatment after 1 week wash out period, hence yielding 22 readings each for TEST & REF.
For Cmax, Mean TESTcmax = 11.43 ng/ml ; REFcmax = 10.27 ng/ml
Standard deviation Cmax: TESTstdv = 5.88 ; REFstdv= 7.26
You can find the Anova table here:
http://answers.yahoo.com/question/index?qid=20140225204032AA9t1nP
Hope you have your reply soon.
TQ