Comparing prediction performance of 10-year CVD risk w/o metabolites
| Prediction performance | Women (95% CI*) | Men (95% CI) |
| Recalibrated QRISK3 | ||
| Harrell’s C-index† | 0.750 (0.739 to 0.763) | 0.706 (0.696 to 0.716) |
| Adding metabolites associated with CVD independently from QRISK3 score | ||
| C-statistics | 0.759 (0.747 to 0.772) | 0.710 (0.701 to 0.720) |
| IDI‡ (%) | 0.30 (0.17 to 0.41) | 0.20 (0.12 to 0.28) |
| Continuous NRI§ (%) | 12.4 (6.7 to 16.6) | 6.8 (2.7 to11.6) |
| Events | 6.5 (1.0 to 10.8) | 4.0 (0.0 to 8.3) |
| Non-events | 5.9 (5.0 to 6.8) | 2.8 (1.8 to 3.9) |
| Categorical NRI (%) | 0.3 (−1.8 to 0.9) | 0.9 (−0.2 to 2.0) |
| Events | 0.4 (−1.2 to 1.5) | 0.4 (−0.7 to 1.4) |
| Non-events | 0.7 (−0.8 to 0.5) | 0.5 (0.3 to 0.8) |
| Adding metabolites with regularisation (using Elastic-net) | ||
| Harrell’s C-index | 0.759 (0.746 to 0.770) | 0.710 (0.700 to 0.719) |
| IDI (%) | 0.16 (0.03 to 0.26) | 0.16 (0.04 to 0.25) |
| Continuous NRI (%) | 4.4 (−0.7 to 9.6) | 7.4 (3.3 to 11.0) |
| Events | 4.7 (−0.3 to 9.9) | 5.2 (1.4 to 8.8) |
| Non-events | 0.3 (−1.3 to 0.7) | 2.2 (1.2 to 3.3) |
| Categorical NRI (%) | 0.3 (−1.6 to 1.1) | 0.7 (−0.8 to 1.8) |
| Events | 0.2 (−1.2 to 1.5) | 0.3 (−1.1 to 1.5) |
| Non-events | 0.4 (−0.5 to 0.3) | 0.4 (0.1 to 0.7) |
| Adding metabolites selected by BorutaSHAP from XGBoost | ||
| Harrell’s C-index | 0.759 (0.748 to 0.771) | 0.710 (0.701 to 0.719) |
| IDI (%) | 0.26 (0.11 to 0.38) | 0.13 (0.03 to 0.20) |
| Continuous NRI (%) | 14.7 (9.2 to 19.7) | 5.5 (1.7 to 9.5) |
| Events | 2.7 (−2.9 to 7.7) | 0.1 (−4.0 to 3.4) |
| Non-events | 12.0 (11.0 to 12.9) | 5.9 (4.9 to 6.9) |
| Categorical NRI (%) | 0.0 (−1.6 to 1.3) | 0.7 (−0.5 to 1.8) |
| Events | 0.6 (−0.9 to 1.9) | 0.3 (−0.9 to 1.2) |
| Non-events | 0.6 (−0.7 to 0.5) | 0.5 (0.2 to 0.7) |
Comparing prediction performance of 10-year CVD risk w/o metabolites. In all models, metabolites are added to recalibrated QRISK3 using Cox proportional-hazards regression. Hyper-parameters of each model are in appendix.
*Bootstrap percentile CI, bootstrap for 500 times.
†Harrell’s C-index, measuring the probability that a randomly selected subject with shorter time-to-event will have a higher predicted probability of event than a randomly selected subject with longer time-to-event.
‡Integrated discrimination improvement, summarising the extent a new model increases risk in events and decreases risk in non-event compared with the old model.
§Net reclassification improvement, quantifying the appropriateness of the change in predicted probabilities or categorised risk group when changing from old to new model; categorical NRI is based on a 10% risk threshold.
CVD, cardiovascular disease; NRI, net reclassification improvement.