Computing effect sizes in models with clustered standard errors and/or fixed effects

[hauselin]

Are there any guidelines/recommendations on how to compute effect sizes for models with clustered standard errors and/or fixed effects? See examples below.

library(fixest)
mod_clustered_se <- feols(mpg ~ vs, mtcars, cluster = ~am)
mod_fixed_effects <- feols(mpg ~ vs | am, mtcars)

Would doing this d = estimate / (sqrt(n) * std_error_of_estimate) be accurate?

See here for relevant discussions

• https://stats.stackexchange.com/questions/580917/does-using-robust-standard-errors-change-effect-size

[strengejacke]

If I understand the argumentation right, it doesn't matter.

For an effect size like Cohen's d, it's the residual standard deviation that's important, not the standard errors of the regression coefficient estimates. Even if the latter are changed with CR-SEs, the former isn't.

[hauselin]

Clustered SEs are often used in within-subject/repeated-measures designs (where errors are not IID), such as when each subject has multiple observations. In such cases (e.g., paired t-test), Cohen's d is calculated differently than in an independent t-test. If so, not accounting for the clustered nature of the data when computing Cohen's d from models with clustered SEs seems wrong? What do you think?

[mattansb]

Although it is true that effect sizes (often) use pooled variance in their estimation, and standard t-tests also use pooled variance in the computation of std. error, this does not imply that using some other form of adjusted / robust std. error should affect the estimation of the effect size.

Cohen's d for within-subject designs typically is different from Cohen's d for between-subject designs, but (1) these are effect sizes quantifying the same effect through different lenses, and (2) it doesn't' have to be (see repeated_measures_d()).

[hauselin]

Thanks for both your helpful insights!