Simulating Difference in Means

Lecture 14?

Dr. Elijah Meyer

NC State University
ST 511 - Fall 2024

2024-10-07

Checklist

– Are you keeping up with Slack?

– Quiz No quiz this week

– Difference in mean (theory) walkthrough posted

– Exam-1: October 9th (in-class)

– Exam-1: Assigned October 9th; Due 11:59pm October 15th

Announcements

– All videos are live

– All solutions are posted

– Exam equation sheet is posted

Warm Up: Review

Hypothesis testing: What is…

– Null hypothesis

– Alternative hypothesis

– Sample statistic

– Test statistic

– p-value

– alpha

– Decision

– Conclusion

Warm Up: Review

Hypothesis testing: What is…

– Null hypothesis - assumption about the population

– Alternative hypothesis - our research question

– Sample statistic - our calculated statistic from our data

– Test statistic - also called standardized statistic (Z or t)

– p-value (probability of observing our stat, or something more extreme, given the null is true)

– alpha - “type-1 error” or threshold to compare vs p-value

– Decision - Reject or fail to reject the null

– Conclusion - Strong or weak evidence to conclude the alternative

Sampling distribution under the null

Here is the approximated null distribution. And we can calculate the p-value straight from here!

Z = \(\frac{\hat{p} - \pi_o}{SE}\)

Z = \(\frac{.37 - .5}{0.05}\) = -2.61

These two are equivalent because as soon as we “know \(\pi\)”, we also know the variance of our sampling distribution … meaning with know that the our statistic \(\hat{p}\) is ~ normal.

This is not true for hypothesis tests for the mean. We can’t say that \(\bar{x}\) ~ t.

So when we do hypothesis testing for the mean, you will always standardize so our t-stat ~ \(t_\text{df}\) (or do a simulation test).

Warm Up: Review

Confidence intervals: What is…

– The purpose

– The equation

– Confidence level

– Interpretation

– Meaning of confidence

Warm Up: Review

Confidence intervals: What is…

– The purpose - to estimate!

– The equation \(stat \pm \text{margin of error}\)

– Confidence level - The percentage of the distribution in between your upper and lower bound

– Interpretation - We are ___% confident that our true parameter is within (lower, upper). Note: we need to think about direction when looking at a difference!

– Meaning of confidence - In the long run… if we make many 95% confidence intervals, we would expect 95% of all confidence intervals to cover the true parameter.

Context

We are going to use a familiar data set (penguins) to demonstrate these theory based procedures. In this example, we are interested in exploring if the species of the penguin impacts the bill length (mm) of the penguin on the Palmer island. We will be looking at the Chinstrap and Gentoo species of penguin. We are interested in researching if there is a difference in bill length between the Gentoo penguins the Chinstrap penguins.

– Variables?

– Null and alternative hypothesis?

Set up

– explanatory: species - categorical

– response: bill length (mm) - quantitative

difference in means scenario

Set up

\(H_o: \mu_g - \mu_c = 0\)

\(H_a: \mu_g - \mu_c \neq 0\)

The data

penguins |>
  filter(species %in% c("Chinstrap", "Gentoo"),
         (!is.na(sex))) |>
  group_by(species) |>
  summarise(means = mean(bill_length_mm),
            count = n())
# A tibble: 2 × 3
  species   means count
  <fct>     <dbl> <int>
1 Chinstrap  48.8    68
2 Gentoo     47.6   119

The data

penguins |>
  filter(species %in% c("Chinstrap", "Gentoo"),
         (!is.na(sex))) |>
  group_by(species) |>
  summarise(means = mean(bill_length_mm),
            count = n())
# A tibble: 2 × 3
  species   means count
  <fct>     <dbl> <int>
1 Chinstrap  48.8    68
2 Gentoo     47.6   119

\(\bar{x}_g - \bar{x}_c = -1.2\)

Assumptions

What are they?

Assumptions

– Independence: does one observation influence the other

  • between groups

  • within groups

Assumptions

Normality: Gentoo

Assumptions

Normality: Chinstrap

Let’s simulate

Steps

  • Mix the two groups together, because we assume that species and bill length are independent

  • Shuffle observations into two new groups of same size n1 and n2

  • calculate new sample means

  • subtract!

  • do this many many times!

R code

set.seed(12345)

null_dist <- penguins |>
  filter(species %in% c("Gentoo", "Chinstrap")) |>
  specify(response = bill_length_mm, explanatory = species) |>
  hypothesize(null = "independence") |>
  generate(reps = 1000, type = "permute") |>
  calculate(stat = "diff in means", order = c("Gentoo", "Chinstrap"))

R code

# A tibble: 1 × 1
  p_value
    <dbl>
1   0.016

Confidence Interval

Now we want to estimate what \(\mu_g - \mu_c\) really is…

How does the simulation steps change?

Simulation

Steps

This is a bootstrap resample

  • Sample with replacement within each group n1 and n2 times

  • Calculate our new sample means

  • Subtract

  • Do this process many many times!

R code

set.seed(12345)

boot_df <- penguins |>
  filter(species %in% c("Gentoo", "Chinstrap")) |>
  specify(response = bill_length_mm, explanatory = species) |>
  generate(reps = 1000, type = "bootstrap") |>
  calculate(stat = "diff in means" , order = c("Gentoo", "Chinstrap"))

R code

R code

boot_df |>
  summarize(
    lower = quantile(stat, 0.025),
    upper = quantile(stat, 0.975)
  )
# A tibble: 1 × 2
  lower  upper
  <dbl>  <dbl>
1 -2.25 -0.338

Review

🔗 class website