Example data for borrowing from control arms of external (historical) trials

[chstock]

As recently discussed in the team, borrowing from control arms of external (historical) trials is what the package may be used for, and an example dataset to run such a borrowing analysis as an illustration would be helpful. I put together a dataset with results from trials identified by one of the discussed systematic reviews: hist_pbo_dat.csv.

I used a subset of the included studies (those conducted and published after 2010). Only control arms with pill placebo were chosen. I mainly used the systematic review to identifiy the studies. I extracted data from clinicaltrials.gov where possible, and from journal publications otherwise. Data for other time points than the primary assessment in the trial and some baseline characteristics were extracted from figures in the papers (using one the tools available for this purpose).

To obtain priors for an MMRM, the data still needs to be synthesized, which seems to be a little challenge on its. To me, it looks like we could, for example, use a Bayesian multivariate meta-analysis model. There might also be simpler alternatives. An individual patient-level dataset to be combined with the prior would have to be simulated. This could either be an active arm only, or both an active and a control arm (to illustrate borrowing for a hybrid control design). Despite the simulation, the example may already be quite a realistic one.

[wlandau]

Excellent work, @chstock! Thanks so much for compiling this dataset.

To help the discussion along, here are some simple visualizations of the point estimates, 50%, and 95% independent normal confidence intervals on change from baseline.

library(tidyverse)
q50 <- qnorm(p = 0.75)
q95 <- qnorm(p = 0.975)
data <- "hist_pbo_dat.csv" |>
  read_csv(col_types = cols()) |>
  mutate(
    week = ordered(week, levels = sort(unique(week))),
    name = paste(first_author, year, sep = "_"),
    base_50_lower = base_est - q50 * base_sd / sqrt(base_n),
    base_50_upper = base_est + q50 * base_sd / sqrt(base_n),
    base_95_lower = base_est - q95 * base_sd / sqrt(base_n),
    base_95_upper = base_est + q95 * base_sd / sqrt(base_n),
    cfb_50_lower = cfb_est - q50 * cfb_se,
    cfb_50_upper = cfb_est + q50 * cfb_se,
    cfb_95_lower = cfb_est - q95 * cfb_se,
    cfb_95_upper = cfb_est + q95 * cfb_se
  )

ggplot(data) +
  geom_point(
    aes(
      x = week,
      y = cfb_est,
      color = name
    ),
    position = position_dodge(width = 0.5),
    size = 3,
    shape = 3
  ) +
  geom_linerange(
    aes(
      x = week,
      ymin = cfb_50_lower,
      ymax = cfb_50_upper,
      color = name
    ),
    position = position_dodge(width = 0.5),
    linewidth = 2
  ) +
  geom_errorbar(
    aes(
      x = week,
      ymin = cfb_95_lower,
      ymax = cfb_95_upper,
      color = name
    ),
    position = position_dodge(width = 0.5),
    linewidth = 0.75
  )

This is interesting because discrepancies in weeks 1, 2, and 4 could help determine how borrowing impacts the later time points. And of course, we might be interested in week 10, but only 2 studies are observed there.

Now for baseline:

ggplot(data) +
  geom_point(
    aes(
      x = week,
      y = base_est,
      color = name
    ),
    position = position_dodge(width = 0.5),
    size = 3,
    shape = 3
  ) +
  geom_linerange(
    aes(
      x = week,
      ymin = base_50_lower,
      ymax = base_50_upper,
      color = name
    ),
    position = position_dodge(width = 0.5),
    linewidth = 2
  ) +
  geom_errorbar(
    aes(
      x = week,
      ymin = base_95_lower,
      ymax = base_95_upper,
      color = name
    ),
    position = position_dodge(width = 0.5),
    linewidth = 0.75
  ) +
  theme_gray(16)

I wonder why Atkinson et al. (2014) looks so different from the others.

To obtain priors for an MMRM, the data still needs to be synthesized, which seems to be a little challenge on its. To me, it looks like we could, for example, use a Bayesian multivariate meta-analysis model. There might also be simpler alternatives.

Possibly related: maybe your reference mentions this, but it would be great to fit that model on summary-level data. Longitudinal hierarchical models on full patient-level data are extremely computationally expensive, so even if we had such patient-level data available, I am not sure it would be desirable use in place of summaries. I wonder if we would have more uncertainty if we can't estimate correlations among within-patient residuals.

An individual patient-level dataset to be combined with the prior would have to be simulated. This could either be an active arm only, or both an active and a control arm (to illustrate borrowing for a hybrid control design). Despite the simulation, the example may already be quite a realistic one.

Yes, and this data gives us a great basis to create such simulations.

[chstock]

Many thanks, @wlandau, I agree. You were faster with plotting, which I had in mind too but then didn't manage. :) I will take a closer look at the meta-analysis models and will try to provide a fit. They are for summary-level data and should work in principle. I believe they are typically fitted to data with a very small number of outcomes and wonder how/ if they work if each visit would be considered an outcome.

[chstock]

Related to this discussion: the meta-analysis model by Wei and Higgins (2013) was considered in the group meeting (on 18 July 2024) as a potential model to synthesize historical MMRM data: Wei Y, Higgins JP. Bayesian multivariate meta-analysis with multiple outcomes. Stat Med. 2013 Jul 30;32(17):2911-34.