Blue Week Special Offer | Brighten your week!
days
hours
minutes
seconds

# Unable to understand Statistical Significance

If a new banner ad on a website caused a drop in user engagement:

Null hypothesis: Users who saw the banner ad spent the same amount of time on the website as those who didn’t.
Alternative hypothesis: Users who saw the banner ad spent less time on the website than those who didn’t.

For above H1 can’t we use the statement that:
Alternative hypothesis: Users who saw the banner ad spent more time on the website than those who didn’t.

Why can’t we use the above-mentioned H1 in our hypothesis statement with “more” instead of “less time”?

@joshi.ananya.joshi1

In significant testing, there is one-tailed test (left or right tailed) or two-tailed test (both-tailed).

This particular hypothesis is one-tailed / left-tailed.

Yes, studied those. But can you help me out the understanding of the concept of power (beta) of hypothesis testing and how to compute beta?

1 Like

There is an indirect relationship between alpha and beta.

For example, if alpha is presently 10% and you reduce to say 1%,
you accept / do not reject the null hypothesis 99% of the time.

This means some results that should be rejected were accepted. Therefore, Type II error / beta increases.

As an illustration, if a judge wants to be very certain that he / she is sending the right person to jail / does not want to send an innocent person to jail. The judge demands incontrovertible evidence. If someone commits a crime and the evidence is not hard enough, the judge acquits the person.

Decreasing Type I error increases Type II error.

The power of a test is 1 - beta.

1 Like

How can we compute power i.e. (1-beta) in order to determine the power of the hypothesis test?
Process/method of calculation?

Kindly see here:

In this example they give us 2 events that occurred and are asking us to use statistics to see if those events could be correlated. The events are:

1. The website added a new banner.
2. The website experienced a drop in user engagement.

Since these are the only two events they gave us we only really can formulate two hypothesises… hypothosi…? whatever.

1. the new banner did not cause the drop of engagement - this is the “null hypothesis”
2. the new banner did cause the drop of engagement - this is the alternative to that null hypothesis

You suggested an alternate alternative hypothesis - “Users who saw the banner ad spent more time on the website that those who didnt” and asked why that isnt a valid alternative hypothesis. The answer is that they are telling us that user engagement has dropped, NOT raised, and we are trying to make a guess at why that drop occurred. The actual situation could very well be that people that saw the add spent more time on the site, but that situation would mean that the null hypothesis is true - the banner did not cause the drop of engagement.

This is not the only possible Hypothesis though, We could also want to know if the drop of engagment was caused by the fact that it happened on a Wednesday, or that the site went offline, or that the site was for drug trafficking and there was a big bust and all their clients are in jail… etc. But with the given facts that we have, these are the possible null and alternative hypothesessesss.