Background Natural experiments are used to evaluate population health interventions when exposure to the intervention has not been manipulated by the researchers. As an example, the interrupted time series (ITS) is a strong natural experimental design. However, the weakness of ITS is the inability to determine whether effects are attributable to something other than the intervention which affects the outcome and occurs at the same time. We used the ITS technique to evaluate the effectiveness of the Health in Pregnancy (HiP) grant, a universal unconditional cash transfer of £190 It was introduced in the UK in April 2009 and withdrawn in April 2011. We compared outcomes before the introduction of HiP in Scotland and immediately after its withdrawal with those during the period for which it existed, and by varying the timing of the intervention period, ascertain if observed effects could be attributed to HiP or other co-existing policies.

Methods All singleton births with a gestational age 26–44 weeks were extracted from routinely collected birth data between January 2004 and December 2013. The outcomes of interest were birthweight and contacting midwife before 25 weeks. Interrupted time series models, adjusting for maternal, birth characteristics, seasonality and temporal trends, were used. The intervention period (April 2009 to April 2011) and post-intervention period (May 2011 to December 2013) were compared to the pre-intervention period (January 2004 to March 2009). Four different start dates (6 & 12 months before and after April 2009) and three duration periods (18, 24, 30 months) were chosen.

Results There were 272,935 births pre-intervention, 109,775 during the intervention and 144,663 post-intervention. Compared to pre-intervention, birthweight remained unchanged in both the intervention period -2.6 g (95% CI -6.9; 1.7) and post-intervention -5.0 g (-11.1; 1.0). During the intervention period women were more likely to contact the midwife OR 1.10 95%CI (1.02; 1.18). For birthweight, the duration of the intervention did not alter the effect of HiP. The effect on contacting a midwife was similar across start dates and duration periods.

Conclusion The strongest intervention effects did not coincide with the dates during which the HiP grant was in place, and there was little dilution of the effects, meaning any effects found could be due to policies and events other than the HiP grant. The lack of random allocation to intervention means the groups were exposed to different macro-level environments, which may impact on birth outcomes during the intervention period.