Decision curves and the MCID

Exact net benefit for the baseline model, thresholds 1 to 30%

Published

November 11, 2026

Code 
source("_setup.R")
source("R/decision_curve.R")
fit   <- read.csv("output/dca_fitted.csv")
band  <- read.csv("output/dca_ensemble_band.csv")
summ  <- read.csv("output/dca_summary.csv")
ens   <- read.csv("output/dca_ensemble.csv")
get_band <- function(m) band[band$measure == m, c("threshold", "lo", "hi")]

What “exact” means here

Net benefit at threshold \(t\) is the true-positive rate minus the false-positive rate weighted by \(t/(1-t)\). The baseline model has seven binary predictors, so there are only 128 predictor profiles, each with a known risk. Given the probability of each profile, net benefit at every threshold is an exact calculation: sort the profiles by risk, flag those at or above \(t\), and add up. No patient-level sample is drawn.

The profile probabilities come from numerical integration of the fitted 7-dimensional normal, not from simulating patients. The integrator (mvtnorm) uses a randomised quasi-Monte Carlo rule internally, with the error target set to \(10^{-7}\) per profile. Repeat runs with different seeds agree to about \(10^{-6}\), which is below any difference that matters here, and the test suite checks this.

The same code takes separate “decision” and “outcome” risks, so once genetic terms enter the data-generating model it can score a clinical-only rule against the true risk without change.

ImportantThe curve and bands below come from the fitted copula

Since decisions D9 and D10 (18 October 2026), the baseline cohort for the genetic layer is the observed distribution of the clinical profiles in the 4,078 unique admissions of boscore.xlsx (the 49 non-event twin rows removed), with patients resampled in the bootstrap, and the baseline model is a refit on that cohort. The curve and bands below are the copula’s with the published equation, a sensitivity analysis. The MCID table at the end of this page shows the adopted refit values next to the earlier ones.

The baseline decision curve

Code 
plot_dca(fit, get_band("nb"), band_fill = jama_cols[["steel"]], mark_col = jama_cols[["grey"]],
         line_cols = c("Baseline model" = jama_cols[["navy"]], "Treat all" = jama_cols[["steel"]], "Treat none" = jama_cols[["grey"]])) +
  theme_jama(base_size = 12)
Figure 1: Net benefit of the fixed baseline model on the fitted cohort. The shaded band is the 90% range across 150 near-optimal correlation matrices. Dashed lines mark the 5% (primary) and 20% (secondary) thresholds.
Code 
summ |>
  transmute(
    Threshold = threshold,
    `Net benefit` = nb,
    `Ensemble range` = sprintf("%.4f to %.4f", nb_lo, nb_hi),
    `Treat all` = nb_all,
    `True positives per 1,000` = tp_per_1000,
    `False positives per 1,000` = fp_per_1000,
    Flagged = flagged
  ) |>
  gt() |>
  fmt_percent(c(Threshold, Flagged), decimals = 1) |>
  fmt_number(c(`Net benefit`, `Treat all`), decimals = 4) |>
  fmt_number(c(`True positives per 1,000`, `False positives per 1,000`), decimals = 1)
Table 1: Baseline net benefit at the pre-specified thresholds. Ranges are the 5th to 95th percentile across the correlation ensemble.
Threshold Net benefit Ensemble range Treat all True positives per 1,000 False positives per 1,000 Flagged
5.0% 0.0148 0.0137 to 0.0160 −0.0187 21.4 125.7 14.7%
10.0% 0.0088 0.0080 to 0.0101 −0.0753 13.9 46.1 6.0%
20.0% 0.0040 0.0032 to 0.0048 −0.2097 7.0 12.0 1.9%
Code 
plot_dca_rates(fit, line_col = jama_cols[["navy"]], mark_col = jama_cols[["grey"]]) + theme_jama(base_size = 12)
Figure 2: True and false positives per 1,000 admissions as the threshold rises.

The curves are step functions because the 128 profiles carry a limited number of distinct risks. Each step is a group of profiles dropping below the threshold. Adding genetic terms will split these groups and smooth the curves.

The minimum clinically important difference

The pre-specified MCID (reading b) is 20% more true cases of invasive mould disease at no increase in false positives, which in net-benefit terms is 0.2 times the baseline true-positive rate at the threshold. The 5% threshold is primary, because it is the cut-off the Stanzani paper used to define high risk. The 20% threshold is secondary.

Code 
plot_mcid(fit, get_band("mcid_b"), band_fill = jama_cols[["steel"]], line_col = jama_cols[["navy"]], mark_col = jama_cols[["grey"]]) + theme_jama(base_size = 12)
Figure 3: Net-benefit gain the genetic layer would have to deliver, by threshold. Shaded band as in the decision-curve figure.
Code 
ref <- read.csv("data/baseline_net_benefit_reference.csv")
q_v1 <- read.csv("output/v1/profile_probabilities.csv")
risk <- plogis(profile_logit(pars, all_profiles()))
v1 <- dca_exact(q_v1$q, risk, thresholds = c(0.05, 0.20))
emp <- read.csv("output/dca_empirical_summary.csv")
base <- read.csv("output/dca_baseline_summary.csv")
refit <- read.csv("output/dca_baseline_refit_summary.csv")
sel <- function(d, col) d[[col]][d$threshold %in% c(0.05, 0.20)]
tibble(
  Threshold = c("5% (primary)", "20% (secondary)"),
  `Indicative (rho = 0.30)` = ref$mcid_b_20pc_more_tp[ref$threshold %in% c(0.05, 0.20)],
  `Copula v1` = v1$mcid_b,
  `Copula v2 (at the lock)` = sel(summ, "mcid_b"),
  `v2 ensemble 5%` = sel(summ, "mcid_lo"),
  `v2 ensemble 95%` = sel(summ, "mcid_hi"),
  `4,127 rows (D8)` = sel(emp, "mcid_b"),
  `Published equation, base cohort (D9)` = sel(base, "mcid_b"),
  `Refit baseline (adopted)` = sel(refit, "mcid_b"),
  `Refit bootstrap 5%` = sel(refit, "mcid_lo"),
  `Refit bootstrap 95%` = sel(refit, "mcid_hi"),
  `Observed outcomes` = sel(base, "observed_mcid")
) |>
  gt() |>
  fmt_number(where(is.numeric), decimals = 5)
Table 2: MCID (reading b) on three cohorts.
Threshold Indicative (rho = 0.30) Copula v1 Copula v2 (at the lock) v2 ensemble 5% v2 ensemble 95% 4,127 rows (D8) Published equation, base cohort (D9) Refit baseline (adopted) Refit bootstrap 5% Refit bootstrap 95% Observed outcomes
5% (primary) 0.00420 0.00439 0.00428 0.00397 0.00460 0.00445 0.00404 0.00458 0.00387 0.00566 0.00441
20% (secondary) 0.00191 0.00141 0.00140 0.00117 0.00167 0.00142 0.00113 0.00160 0.00122 0.00301 0.00142
Code 
i5 <- which(abs(fit$threshold - 0.05) < 1e-9)
ok <- abs(fit$mcid_b / fit$mcid_b[i5] - 1) < 0.05
lo <- i5; while (lo > 1 && ok[lo - 1]) lo <- lo - 1
hi <- i5; while (hi < length(ok) && ok[hi + 1]) hi <- hi + 1
plateau <- fit$threshold[c(lo, hi)]

The primary value is stable across the four cohorts (within about 6% of the indicative figure), while the secondary value falls by about a quarter. The adopted refit-baseline values are +0.00458 at 5% and +0.00160 at 20%. They are higher than the published-equation values (+0.00404 and +0.00113) because the published equation under-predicts slightly on the corrected cohort (3.06% predicted against 3.26% observed), so adopting the refit makes the primary test harder for the genetic layer. Results are also reported against the published-equation and 4,127-row values. The 5% value sits on a plateau: the MCID stays within 5% of its value at 5% for thresholds from about 4.2% to 7.8%. The spread across the ensemble, about plus or minus 7% at 5% and plus or minus 18% at 20%, is the part that rests on the unpublished correlation structure.

How much does the MCID depend on the outcome risk?

The published equation was fitted with the 49 twin rows in, and on the corrected cohort it under-predicts slightly (3.06% predicted against 3.26% observed). So the MCID can be computed under different readings of the “true” outcome risk. The decision rule is the published equation’s, except in scenario C. Patients are resampled, with the refit repeated inside every resample.

Code 
sens <- read.csv("output/mcid_outcome_risk_sensitivity.csv")
sens |>
  transmute(Scenario = scenario, Threshold = threshold, MCID = mcid_b, `5%` = mcid_lo, `95%` = mcid_hi,
            `Relative to A` = mcid_relative_to_A, `True positives per 1,000` = tp_per_1000) |>
  gt(groupname_col = "Threshold") |>
  fmt_percent(`Relative to A`, decimals = 0) |>
  fmt_number(c(MCID, `5%`, `95%`), decimals = 5) |>
  fmt_number(`True positives per 1,000`, decimals = 1)
Table 3: MCID under four readings of the outcome risk, with 90% patient-bootstrap intervals.
Scenario MCID 5% 95% Relative to A True positives per 1,000
0.05
A: published truth 0.00404 0.00368 0.00439 100% 20.2
B: refit truth, published rule 0.00452 0.00380 0.00529 112% 22.6
C: refit truth and rule 0.00458 0.00386 0.00564 113% 22.9
D: observed outcomes 0.00441 0.00367 0.00519 109% 22.1
0.1
A: published truth 0.00239 0.00208 0.00271 100% 11.9
B: refit truth, published rule 0.00277 0.00222 0.00333 116% 13.9
C: refit truth and rule 0.00285 0.00239 0.00454 119% 14.3
D: observed outcomes 0.00265 0.00205 0.00328 111% 13.2
0.2
A: published truth 0.00113 0.00090 0.00138 100% 5.7
B: refit truth, published rule 0.00137 0.00100 0.00175 121% 6.8
C: refit truth and rule 0.00160 0.00121 0.00301 141% 8.0
D: observed outcomes 0.00142 0.00100 0.00188 125% 7.1

The adopted value (A, published equation as truth) is the lowest at every threshold. The alternatives raise the primary MCID by 9 to 13% and the secondary by 21 to 41%, which makes the adopted reading the most permissive one for the primary test. The intervals overlap, so the data cannot separate the readings, and the published coefficient lies inside the refit’s interval for every term:

Code 
read.csv("output/refit_coefficients.csv") |>
  transmute(Term = gsub("_", " ", term), Published = published, Refit = refit, `5%` = lo, `95%` = hi) |>
  gt() |>
  fmt_number(2:5, decimals = 2)
Table 4: Published coefficients against a refit on the 4,078 unique admissions (90% patient-bootstrap interval).
Term Published Refit 5% 95%
(Intercept) −5.45 −5.59 −6.14 −5.18
uncontrolled malignancy 0.68 0.76 0.44 1.11
high risk chemo 0.79 0.86 0.48 1.30
high dose steroids 0.80 0.80 0.38 1.22
severe lymphopenia 0.89 0.93 0.56 1.30
cmv 1.14 1.42 0.85 1.95
prolonged neutropenia 1.52 1.58 1.12 2.12
prior mould disease 1.64 1.99 1.50 2.43

The 4,127-row MCID (+0.00445 at 5%, +0.00142 at 20%) sits close to the refit-truth and observed-outcome readings, so reporting against it, as decision D9 commits to, brackets them.

Using the machinery for the genetic layer

# Cells: clinical profile x genetic class. Decisions use the clinical-only risk;
# outcomes use the true risk including genetics.
base  <- dca_exact(w, risk_true, risk_pred = risk_clinical)
with  <- dca_exact(w, risk_true, risk_pred = risk_with_genetics)
dca_compare(base, with)   # delta_nb, the baseline MCID, and whether it is met

dca_compare() takes the MCID from the baseline curve, so the test is fixed by the baseline and cannot drift with the candidate model.

Limitations

  • Results are for the baseline alone. No genetic term has been added.
  • The ensemble spread reflects only the correlation structure. It does not include uncertainty from admission clustering or from using the apparent aROC of 0.85.
  • Net benefit is per admission. Patients with several admissions contribute correlated events, which matters for uncertainty but not for the exact point values here.