Effect size for Dunnett's test and Steels test

Question

I did Dunnett's test and Steels test using R, but I cannot know how to calculate the effect size. Could you show me how to calculate the effect sizes for Dunnett's test and Steels test.

## https://zenn.dev/masahiro_kondo/articles/7a0250c97e41f6
#data generation
set.seed(1234)
n <- 200
th1 <- 
  data.frame(ID = c(1:n)) %>%
  mutate(Group = case_when(
    ID <= n/4 ~ "A",
    ID <= 2*n/4 +10 ~ "B",
    ID <= 3*n/4 -10 ~ "C",
    ID <= n ~ "D"))

th1$Group <- as.factor(th1$Group)

y <- c(
  round(rnorm(n=50, mean=0, sd=1),2),
  round(rnorm(n=50, mean=0.2, sd=1),2),
  round(rnorm(n=50, mean=0.4, sd=1),2),
  round(rnorm(n=50, mean=-0.2, sd=1),2)
)

th2 <- cbind(th1, y)

###Dunnett test
library(multcomp)

fx=factor(th1$Group)
summary(glht(aov(y~fx),linfct=mcp(fx="Dunnett")))

###Steel test

Steel <-nparcomp(y ~ Group, data=th2, asy.method = "mult.t",
                 type = "Dunnett", control = "A",alternative = "two.sided",info = FALSE)
summary(Steel)

Answer 1

In order to calculate the effect size, you can use Cohen's d. To do this in R you can use multiple packages, including lsr and effsize.

So using package effsize and your data (th2) we first need to extract groups in order to calculate the effect site.

# Extract groups
groupA <- th2$y[th2$Group == "A"]
groupB <- th2$y[th2$Group == "B"]
groupC <- th2$y[th2$Group == "C"]
groupD <- th2$y[th2$Group == "D"]

# Calculating Cohen's d for A vs B
d.AB <- cohen.d(groupA, groupB)
# A vs C
d.AC <- cohen.d(groupA, groupC)
# A vs D
d.AD <- cohen.d(groupA, groupD)

And the output for AB

> d.AB

Cohen's d

d estimate: -0.8367544 (large)
95 percent confidence interval:
     lower      upper 
-1.2324411 -0.4410677

This might also help: https://www.statology.org/cohens-d-in-r/

But for the Steels test, you could try Rank-Biserial Correlation as an effect size (below is example for group A and B)

# Mann-Whitney U test for group A and B
MWU <- wilcox.test(groupA, groupB)

# Extract U statistic and sample sizes
U <- MWU$statistic
n.A <- length(groupA)
n.B <- length(groupB)

# Calculate rank-biserial correlation
rank.b = 1 - (2 * U) / (n.A * n.B)

See also here: https://www.researchgate.net/post/What_is_the_most_appropriate_effect_size_type_for_mann_whitney_u_analysis

https://www.rdocumentation.org/packages/effectsize/versions/0.7.0/topics/rank_biserial