Matt Glover is a health economist with nearly 10 years’ experience in both academia and consultancy. Following the completion of a BA (Hons) in economics at University of Durham and an MSc in health economics at University of York, Matt joined the Health Economics Research Group at Brunel University London, first as a Research Assistant, later becoming a Research Fellow. He still maintains ties with Brunel through a lecturing position and holds a part-time role in a London based health economics consultancy.
Areas of specialism
Matt’s research interests are largely in the methods and applications of health technology assessment, with a focus on decision analytic modelling. He has also been involved in numerous randomised controlled trials assessing the cost-effectiveness of a range of interventions, including health service delivery, in diverse therapeutic areas.
Courses I teach on
CPD and Short courses
Implementation of the National Health Service abdominal aortic aneurysm (AAA) screening programme (NAAASP) for men aged 65 years began in England in 2009. An important element of the evidence base supporting its introduction was the economic modelling of the long?term cost?effectiveness of screening, which was based mainly on 4?year follow?up data from the Multicentre Aneurysm Screening Study (MASS) randomized trial. Concern has been expressed about whether this conclusion of cost?effectiveness still holds, given the early performance parameters, particularly the lower prevalence of AAA observed in NAAASP.
The existing published model was adjusted and updated to reflect the current best evidence. It was recalibrated to mirror the 10?year follow?up data from MASS; the main cost parameters were re?estimated to reflect current practice; and more robust estimates of AAA growth and rupture rates from recent meta?analyses were incorporated, as were key parameters as observed in NAAASP (attendance rates, AAA prevalence and size distributions).
The revised and updated model produced estimates of the long?term incremental cost?effectiveness of £5758 (95 per cent confidence interval £4285 to £7410) per life?year gained, or £7370 (£5467 to £9443) per quality?adjusted life?year (QALY) gained.
Although the updated parameters, particularly the increased costs and lower AAA prevalence, have increased the cost per QALY, the latest modelling provides evidence that AAA screening as now being implemented in England is still highly cost?effective.
Disease-modifying biological agents and other drug regimens have substantially improved control of disease activity and joint damage in people with rheumatoid arthritis of the hand. However, commensurate changes in function and quality of life are not always noted. Tailored hand exercises might provide additional improvements, but evidence is lacking. We estimated the effectiveness and cost-effectiveness of tailored hand exercises in addition to usual care during 12 months.
In this pragmatic, multicentre, parallel-group trial, at 17 National Health Service sites across the UK we randomly assigned 490 adults with rheumatoid arthritis who had pain and dysfunction of the hands and had been on a stable drug regimen for at least 3 months, to either usual care or usual care plus a tailored strengthening and stretching hand exercise programme. Participants were randomly assigned with stratification by centre. Allocation was computer generated and unmasked to participants and therapists delivering treatment after randomisation. Outcome assessors and all investigators were masked to allocation. Physiotherapists or occupational therapists gave the treatments. The primary outcome was the Michigan Hand Outcomes Questionnaire overall hand function score at 12 months. The analysis was by intention to treat. We calculated cost per quality-adjusted life-year. This trial is registered as ISRCTN 89936343.
Between Oct 5, 2009, and May 10, 2011, we screened 1606 people, of whom 490 were randomly assigned to usual care (n=244) or tailored exercises (n=246). 438 of 490 participants (89%) provided 12 month follow-up data. Improvements in overall hand function were 3·6 points (95% CI 1·5?5·7) in the usual care group and 7·9 points (6·0?9·9) in the exercise group (mean difference between groups 4·3, 95% CI 1·5?7·1; p=0·0028). Pain, drug regimens, and health-care resource use were stable for 12 months, with no difference between the groups. No serious adverse events associated with the treatment were recorded. The cost of tailored hand exercise was £156 per person; cost per quality-adjusted life-year was £9549 with the EQ-5D (£17?941 with imputation for missing data).
We have shown that a tailored hand exercise programme is a worthwhile, low-cost intervention to provide as an adjunct to various drug regimens. Maximisation of the benefits of biological and DMARD regimens in terms of function, disability, and health-related quality of life should be an important treatment aim.
Abdominal aortic aneurysm (AAA) screening programmes have been established for men in the UK to reduce deaths from AAA rupture. Whether or not screening should be extended to women is uncertain.
To evaluate the cost-effectiveness of population screening for AAAs in women and compare a range of screening options.
A discrete event simulation (DES) model was developed to provide a clinically realistic model of screening, surveillance, and elective and emergency AAA repair operations. Input parameters specifically for women were employed. The model was run for 10 million women, with parameter uncertainty addressed by probabilistic and deterministic sensitivity analyses.
Population screening in the UK.
Women aged e 65 years, followed up to the age of 95 years.
Invitation to ultrasound screening, followed by surveillance for small AAAs and elective surgical repair for large AAAs.
Main outcome measures
Number of operations undertaken, AAA-related mortality, quality-adjusted life-years (QALYs), NHS costs and cost-effectiveness with annual discounting.
AAA surveillance data, National Vascular Registry, Hospital Episode Statistics, trials of elective and emergency AAA surgery, and the NHS Abdominal Aortic Aneurysm Screening Programme (NAAASP).
Systematic reviews of AAA prevalence and, for elective operations, suitability for endovascular aneurysm repair, non-intervention rates, operative mortality and literature reviews for other parameters.
The prevalence of AAAs (aortic diameter of e 3.0 cm) was estimated as 0.43% in women aged 65 years and 1.15% at age 75 years. The corresponding attendance rates following invitation to screening were estimated as 73% and 62%, respectively. The base-case model adopted the same age at screening (65 years), definition of an AAA (diameter of e 3.0 cm), surveillance intervals (1 year for AAAs with diameter of 3.0?4.4 cm, 3 months for AAAs with diameter of 4.5?5.4 cm) and AAA diameter for consideration of surgery (5.5 cm) as in NAAASP for men. Per woman invited to screening, the estimated gain in QALYs was 0.00110, and the incremental cost was £33.99. This gave an incremental cost-effectiveness ratio (ICER) of £31,000 per QALY gained. The corresponding incremental net monetary benefit at a threshold of £20,000 per QALY gained was ?£12.03 (95% uncertainty interval ?£27.88 to £22.12). Almost no sensitivity analyses brought the ICER below £20,000 per QALY gained; an exception was doubling the AAA prevalence to 0.86%, which resulted in an ICER of £13,000. Alternative screening options (increasing the screening age to 70 years, lowering the threshold for considering surgery to diameters of 5.0 cm or 4.5 cm, lowering the diameter defining an AAA in women to 2.5 cm and lengthening the surveillance intervals for the smallest AAAs) did not bring the ICER below £20,000 per QALY gained when considered either singly or in combination.
The model for women was not directly validated against empirical data. Some parameters were poorly estimated, potentially lacking relevance or unavailable for women.
The accepted criteria for a population-based AAA screening programme in women are not currently met.
A large-scale study is needed of the exact aortic size distribution for women screened at relevant ages. The DES model can be adapted to evaluate screening options in men.
shown to be effective at reducing AAA-related mortality and
is cost-effective. There are, however, significant variations
in AAA screening protocols between healthcare systems and numerous challenges to be overcome. Further
evidence of cost- and clinical effectiveness is likely to be
required before there is widespread international adoption
of AAA screening.
A third of deaths in the UK from ruptured abdominal aortic aneurysm (AAA) are in women. In men, national screening programmes reduce deaths from AAA and are cost-effective. The benefits, harms, and cost-effectiveness in offering a similar programme to women have not been formally assessed, and this was the aim of this study.
We developed a decision model to assess predefined outcomes of death caused by AAA, life years, quality-adjusted life years, costs, and the incremental cost-effectiveness ratio for a population of women invited to AAA screening versus a population who were not invited to screening. A discrete event simulation model was set up for AAA screening, surveillance, and intervention. Relevant women-specific parameters were obtained from sources including systematic literature reviews, national registry or administrative databases, major AAA surgery trials, and UK National Health Service reference costs.
AAA screening for women, as currently offered to UK men (at age 65 years, with an AAA diagnosis at an aortic diameter of e3·0 cm, and elective repair considered at e5·5cm) gave, over 30 years, an estimated incremental cost-effectiveness ratio of £30?000 (95% CI 12?000?87?000) per quality-adjusted life year gained, with 3900 invitations to screening required to prevent one AAA-related death and an overdiagnosis rate of 33%. A modified option for women (screening at age 70 years, diagnosis at 2·5 cm and repair at 5·0 cm) was estimated to have an incremental cost-effectiveness ratio of £23?000 (9500?71?000) per quality-adjusted life year and 1800 invitations to screening required to prevent one AAA-death, but an overdiagnosis rate of 55%. There was considerable uncertainty in the cost-effectiveness ratio, largely driven by uncertainty about AAA prevalence, the distribution of aortic sizes for women at different ages, and the effect of screening on quality of life.
By UK standards, an AAA screening programme for women, designed to be similar to that used to screen men, is unlikely to be cost-effective. Further research on the aortic diameter distribution in women and potential quality of life decrements associated with screening are needed to assess the full benefits and harms of modified options.
We sought to analyse the impacts found, and the methods used, in a series of assessments of programmes and portfolios of health research consisting of multiple projects.
We analysed a sample of 36 impact studies of multi-project research programmes, selected from a wider sample of impact studies included in two narrative systematic reviews published in 2007 and 2016. We included impact studies in which the individual projects in a programme had been assessed for wider impact, especially on policy or practice, and where findings had been described in such a way that allowed them to be collated and compared.
Included programmes were highly diverse in terms of location (11 different countries plus two multi-country ones), number of component projects (8 to 178), nature of the programme, research field, mode of funding, time between completion and impact assessment, methods used to assess impact, and level of impact identified.
Thirty-one studies reported on policy impact, 17 on clinician behaviour or informing clinical practice, three on a combined category such as policy and clinician impact, and 12 on wider elements of impact (health gain, patient benefit, improved care or other benefits to the healthcare system). In those multi-programme projects that assessed the respective categories, the percentage of projects that reported some impact was policy 35% (range 5?100%), practice 32% (10?69%), combined category 64% (60?67%), and health gain/health services 27% (6?48%).
Variations in levels of impact achieved partly reflected differences in the types of programme, levels of collaboration with users, and methods and timing of impact assessment. Most commonly, principal investigators were surveyed; some studies involved desk research and some interviews with investigators and/or stakeholders. Most studies used a conceptual framework such as the Payback Framework. One study attempted to assess the monetary value of a research programme?s health gain.
The widespread impact reported for some multi-project programmes, including needs-led and collaborative ones, could potentially be used to promote further research funding. Moves towards greater standardisation of assessment methods could address existing inconsistencies and better inform strategic decisions about research investment; however, unresolved issues about such moves remain.
Obstructive sleep apnoea-hypopnoea (OSAH) causes excessive daytime sleepiness, impairs quality-of-life, and increases cardiovascular disease and road traffic accident risks. Continuous positive airway pressure (CPAP) treatment and mandibular advancement devices (MAD) have been shown to be effective in individual trials but their effectiveness particularly relative to disease severity is unclear.
A MEDLINE, Embase and Science Citation Index search updating two systematic reviews to August 2013 identified 77 RCTs in adult OSAH patients comparing: MAD with conservative management (CM); MAD with CPAP; or CPAP with CM. Overall MAD and CPAP significantly improved apnoea-hypopnoea index (AHI) (MAD ?9.3/hr (p Â 0.001), CPAP ?25.4 (p Â 0.001)). In direct comparisons mean AHI and Epworth sleepiness scale score were lower (7.0/hr (p Â 0.001) and 0.67 (p = 0.093) respectively) for CPAP. There were no CPAP vs. MAD trials in mild OSAH but in comparisons with CM, MAD and CPAP reduced ESS similarly (MAD 2.01 (p Â 0.001); CPAP 1.23 (p = 0.012).
Both MAD and CPAP are clinically effective in the treatment of OSAH. Although CPAP has a greater treatment effect, MAD is an appropriate treatment for patients who are intolerant of CPAP and may be comparable to CPAP in mild disease.
Building on an approach applied to cardiovascular and cancer research, we estimated the economic returns from United Kingdom public- and charitable-funded musculoskeletal disease (MSD) research that arise from the net value of the improved health outcomes in the United Kingdom.
To calculate the economic returns from MSD-related research in the United Kingdom, we estimated (1) the public and charitable expenditure on MSD-related research in the United Kingdom between 1970 and 2013; (2) the net monetary benefit (NMB), derived from the health benefit in quality adjusted life years (QALYs) valued in monetary terms (using a base-case value of a QALY of £25,000) minus the cost of delivering that benefit, for a prioritised list of interventions from 1994 to 2013; (3) the proportion of NMB attributable to United Kingdom research; and (4) the elapsed time between research funding and health gain. The data collected from these four key elements were used to estimate the internal rate of return (IRR) from MSD-related research investments on health benefits. We analysed the uncertainties in the IRR estimate using a one-way sensitivity analysis.
Expressed in 2013 prices, total expenditure on MSD-related research from 1970 to 2013 was £3.5 billion, and for the period used to estimate the rate of return, 1978-1997, was £1.4 billion. Over the period 1994?2013 the key interventions analysed produced 871,000 QALYs with a NMB of £16 billion, allowing for the net NHS costs resulting from them and valuing a QALY at £25,000. The proportion of benefit attributable to United Kingdom research was 30% and the elapsed time between funding and impact of MSD treatments was 16 years. Our best estimate of the IRR from MSD-related research was 7%, which is similar to the 9% for CVD and 10% for cancer research.
Our estimate of the IRR from the net health gain to public and charitable funding of MSD-related research in the United Kingdom is substantial, and justifies the research investments made between 1978 and 1997. We also demonstrated the applicability of the approach previously used in assessing the returns from cardiovascular and cancer research. Inevitably, with a study of this kind, there are a number of important assumptions and caveats that we highlight, and these can inform future research.
Small abdominal aortic aneurysms (AAAs; 3.0-5.4 cm in diameter) are usually asymptomatic and managed by regular ultrasound surveillance until they grow to a diameter threshold (commonly 5.5 cm) at which surgical intervention is considered. The choice of appropriate surveillance intervals is governed by the growth and rupture rates of small AAAs, as well as their relative cost-effectiveness.
The aim of this series of studies was to inform the evidence base for small AAA surveillance strategies. This was achieved by literature review, collation and analysis of individual patient data, a focus group and health economic modelling.
We undertook systematic literature reviews of growth rates and rupture rates of small AAAs. The databases MEDLINE, EMBASE on OvidSP, Cochrane Central Register of Controlled Trials 2009 Issue 4, ClinicalTrials.gov, and controlled-trials.com were searched from inception up until the end of 2009. We also obtained individual data on 15,475 patients from 18 surveillance studies.
Systematic reviews of publications identified 15 studies providing small AAA growth rates, and 14 studies with small AAA rupture rates, up to December 2009 (later updated to September 2012). We developed statistical methods to analyse individual surveillance data, including the effects of patient characteristics, to inform the choice of surveillance intervals and provide inputs for health economic modelling. We updated an existing health economic model of AAA screening to address the cost-effectiveness of different surveillance intervals.
In the literature reviews, the mean growth rate was 2.3 mm/year and the reported rupture rates varied between 0 and 1.6 ruptures per 100 person-years. Growth rates increased markedly with aneurysm diameter, but insufficient detail was available to guide surveillance intervals. Based on individual surveillance data, for each 0.5-cm increase in AAA diameter, growth rates increased by about 0.5 mm/year and rupture rates doubled. To control the risk of exceeding 5.5 cm to below 10% in men, on average a 7-year surveillance interval is sufficient for a 3.0-cm aneurysm, whereas an 8-month interval is necessary for a 5.0-cm aneurysm. To control the risk of rupture to below 1%, the corresponding estimated surveillance intervals are 9 years and 17 months. Average growth rates were higher in smokers (by 0.35 mm/year) and lower in patients with diabetes (by 0.51 mm/year). Rupture rates were almost fourfold higher in women than men, doubled in current smokers and increased with higher blood pressure. Increasing the surveillance interval from 1 to 2 years for the smallest aneurysms (3.0-4.4 cm) decreased costs and led to a positive net benefit. For the larger aneurysms (4.5-5.4 cm), increasing surveillance intervals from 3 to 6 months led to equivalent cost-effectiveness.
There were no clear reasons why the growth rates varied substantially between studies. Uniform diagnostic criteria for rupture were not available. The long-term cost-effectiveness results may be susceptible to the modelling assumptions made.
Surveillance intervals of several years are clinically acceptable for men with AAAs in the range 3.0-4.0 cm. Intervals of around 1 year are suitable for 4.0-4.9-cm AAAs, whereas intervals of 6 months would be acceptable for 5.0-5.4-cm AAAs. These intervals are longer than those currently employed in the UK AAA screening programmes. Lengthening surveillance intervals for the smallest aneurysms was also shown to be cost-effective. Future work should focus on optimising surveillance intervals for women, studying whether or not the threshold for surgery should depend on patient characteristics, evaluating the usefulness of surveillance for those with aortic diameters of 2.5-2.9 cm, and
The effectiveness of exercise for improving hand and wrist function in people with rheumatoid arthritis (RA) is uncertain.
The study aims were (1) to estimate the clinical effectiveness and cost-effectiveness of adding an optimised exercise programme for hands and upper limbs to standard care for patients with RA; and (2) to qualitatively describe the experience of participants in the trial with a particular emphasis on acceptability of the intervention, exercise behaviours and reasons for adherence/non-adherence.
A pragmatic, multicentred, individually randomised controlled trial with an embedded qualitative study. Outcome assessors were blind to group assignment and independent of treatment delivery.
Seventeen NHS trusts in England comprising 21 rheumatology and therapy departments.
Adults with RA who had pain and dysfunction of the hands and/or wrists and had been on stable medication for at least 3 months. Patients were excluded if they were under 18 years old, had undergone upper limb surgery/fracture in the last 6 months, were on a waiting list for upper limb surgery or were pregnant.
Usual care or usual care plus an individualised exercise programme. Usual care consisted of joint protection education, general exercise advice and functional splinting if required. The exercise programme consisted of six sessions of strengthening and stretching exercises with a hand therapist, daily home exercises and strategies to maximise adherence.
Main outcome measures
The primary outcome was the Michigan Hand Outcome Questionnaire (MHQ) overall hand function subscale score at 12 months. Secondary outcome measures included the full MHQ, pain, health-related quality of life (Short Form questionnaire-12 items), impairment (grip strength, dexterity and range of motion) and self-efficacy. European Quality of Life-5 Dimensions, medication and health-care use were collected for the health economics evaluation. Follow-up was at 4 and 12 months post randomisation. Analysis was performed on an intention-to-treat basis.
We randomised 490 patients (244 to usual care, 246 to exercise programme). Compliance with the treatments was very good (93% of usual care participants and 75% of exercise programme participants completed treatment). Outcomes were obtained for 89% of participants at 12 months (222 for usual care, 216 for exercise programme). There was a statistically significant difference in favour of the exercise programme for the primary outcome at 4 and 12 months [mean difference 4.6 points, 95% confidence interval (CI) 2.2 to 7.0 points; and mean difference 4.4 points, 95% CI 1.6 to 7.1 points, respectively]. There were no significant differences in pain scores or adverse events. The estimated difference in mean quality-adjusted life-years (QALYs) accrued over 12 months was 0.01 greater (95% CI ?0.03 to 0.05) in the exercise programme group. Imputed analysis produced incremental cost-effectiveness ratio estimates of £17,941 (0.59 probability of cost-effectiveness at willingness-to-pay threshold of £30,000 per QALY). The qualitative study found the exercise programme to be acceptable and highlighted the importance of the therapist in enabling patients to establish a routine and incorporate the exercises into their lives.
The results of the Strengthening And stretching for Rheumatoid Arthritis of the Hand trial suggest that the addition of an exercise programme for RA hands/wrists to usual care is clinically effective and cost-effective when compared with usual care alone. No adverse effects were associated with the exercise programme. The economic analysis suggests that the intervention is likely to be cost-effective.
Building on an approach developed to assess the economic returns to cardiovascular research, we estimated the economic returns from UK public and charitable funded cancer-related research that arise from the net value of the improved health outcomes.
To assess these economic returns from cancer-related research in the UK we estimated: 1) public and charitable expenditure on cancer-related research in the UK from 1970 to 2009; 2) net monetary benefit (NMB), that is, the health benefit measured in quality adjusted life years (QALYs) valued in monetary terms (using a base-case value of a QALY of GB£25,000) minus the cost of delivering that benefit, for a prioritised list of interventions from 1991 to 2010; 3) the proportion of NMB attributable to UK research; 4) the elapsed time between research funding and health gain; and 5) the internal rate of return (IRR) from cancer-related research investments on health benefits. We analysed the uncertainties in the IRR estimate using sensitivity analyses to illustrate the effect of some key parameters.
In 2011/12 prices, total expenditure on cancer-related research from 1970 to 2009 was £15 billion. The NMB of the 5.9 million QALYs gained from the prioritised interventions from 1991 to 2010 was £124 billion. Calculation of the IRR incorporated an estimated elapsed time of 15 years. We related 17% of the annual NMB estimated to be attributable to UK research (for each of the 20 years 1991 to 2010) to 20 years of research investment 15 years earlier (that is, for 1976 to 1995). This produced a best-estimate IRR of 10%, compared with 9% previously estimated for cardiovascular disease research. The sensitivity analysis demonstrated the importance of smoking reduction as a major source of improved cancer-related health outcomes.
We have demonstrated a substantive IRR from net health gain to public and charitable funding of cancer-related research in the UK, and further validated the approach that we originally used in assessing the returns from cardiovascular research. In doing so, we have highlighted a number of weaknesses and key assumptions that need strengthening in further investigations. Nevertheless, these cautious estimates demonstrate that the returns from past cancer research have been substantial, and justify the investments made during the period 1976 to 1995.
Mandibular advancement devices (MADs) are used to treat obstructive sleep apnoea-hypopnoea syndrome (OSAHS) but evidence is lacking regarding their clinical and cost-effectiveness in less severe disease.
To compare clinical- and cost-effectiveness of a range of MADs against no treatment in mild to moderate OSAHS.
Measurements and methods
This open-label, randomised, controlled, crossover trial was undertaken at a UK sleep centre. Adults with Apnoea-Hypopnoea Index (AHI) 5?Â30/h and Epworth Sleepiness Scale (ESS) score e9 underwent 6?weeks of treatment with three non-adjustable MADs: self-moulded (SleepPro 1; SP1); semi-bespoke (SleepPro 2; SP2); fully-bespoke MAD (bMAD); and 4?weeks no treatment. Primary outcome was AHI scored by a polysomnographer blinded to treatment. Secondary outcomes included ESS, quality of life, resource use and cost.
90 patients were randomised and 83 were analysed. All devices reduced AHI compared with no treatment by 26% (95% CI 11% to 38%, p=0.001) for SP1, 33% (95% CI 24% to 41%) for SP2 and 36% (95% CI 24% to 45%, pÂ0.001) for bMAD. ESS was 1.51 (95% CI 0.73 to 2.29, pÂ0.001, SP1) to 2.37 (95% CI 1.53 to 3.22, pÂ0.001, bMAD) lower than no treatment (pÂ0.001 for all). Compliance was lower for SP1, which was the least preferred treatment at trial exit. All devices were cost-effective compared with no treatment at a £20?000/quality-adjusted life year (QALY) threshold. SP2 was the most cost-effective up to £39?800/QALY.
Non-adjustable MADs achieve clinically important improvements in mild to moderate OSAHS and are cost-effective. Of those trialled, the semi-bespoke MAD is an appropriate first choice.
This report reviews approaches and tools for measuring the impact of research programmes, building on, and extending, a 2007 review.
(1) To identify the range of theoretical models and empirical approaches for measuring the impact of health research programmes; (2) to develop a taxonomy of models and approaches; (3) to summarise the evidence on the application and use of these models; and (4) to evaluate the different options for the Health Technology Assessment (HTA) programme.
We searched databases including Ovid MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature and The Cochrane Library from January 2005 to August 2014.
This narrative systematic literature review comprised an update, extension and analysis/discussion. We systematically searched eight databases, supplemented by personal knowledge, in August 2014 through to March 2015.
The literature on impact assessment has much expanded. The Payback Framework, with adaptations, remains the most widely used approach. It draws on different philosophical traditions, enhancing an underlying logic model with an interpretative case study element and attention to context. Besides the logic model, other ideal type approaches included constructionist, realist, critical and performative. Most models in practice drew pragmatically on elements of several ideal types. Monetisation of impact, an increasingly popular approach, shows a high return from research but relies heavily on assumptions about the extent to which health gains depend on research. Despite usually requiring systematic reviews before funding trials, the HTA programme does not routinely examine the impact of those trials on subsequent systematic reviews. The York/Patient-Centered Outcomes Research Institute and the Grading of Recommendations Assessment, Development and Evaluation toolkits provide ways of assessing such impact, but need to be evaluated. The literature, as reviewed here, provides very few instances of a randomised trial playing a major role in stopping the use of a new technology. The few trials funded by the HTA programme that may have played such a role were outliers.
The findings of this review support the continued use of the Payback Framework by the HTA programme. Changes in the structure of the NHS, the development of NHS England and changes in the National Institute for Health and Care Excellence?s remit pose new challenges for identifying and meeting current and future research needs. Future assessments of the impact of the HTA programme will have to take account of wider changes, especially as the Research Excellence Framework (REF), which assesses the quality of universities? research, seems likely to continue to rely on case studies to measure impact. The HTA programme should consider how the format and selection of case studies might be improved to aid more systematic assessment. The selection of case studies, such as in the REF, but also more generally, tends to be biased towards high-impact rather than low-impact stories. Experience for other industries indicate that much can be learnt from the latter. The adoption of researchfish® (researchfish Ltd, Cambridge, UK) by most major UK research funders has implications for future assessments of impact. Although the routine capture of indexed research publications has merit, the degree to which researchfish will succeed in collecting other, non-indexed outputs and activities remains to be established.
There were limitations in how far we could address challenges that faced us as we extended the focus beyond that of the 2007 review, and well beyond a narrow focus just on the HTA programme.
Research funders can benefit from continuing to monitor and evaluate the impac
Obstructive sleep apnoea?hypopnoea (OSAH) causes excessive daytime sleepiness (EDS), impairs quality of life (QoL) and increases cardiovascular disease and road traffic accident risks. Continuous positive airway pressure (CPAP) treatment is clinically effective but undermined by intolerance, and its cost-effectiveness is borderline in milder cases. Mandibular advancement devices (MADs) are another option, but evidence is lacking regarding their clinical effectiveness and cost-effectiveness in milder disease.
(1) Conduct a randomised controlled trial (RCT) examining the clinical effectiveness and cost-effectiveness of MADs against no treatment in mild to moderate OSAH. (2) Update systematic reviews and an existing health economic decision model with data from the Trial of Oral Mandibular Advancement Devices for Obstructive sleep apnoea?hypopnoea (TOMADO) and newly published results to better inform long-term clinical effectiveness and cost-effectiveness of MADs and CPAP in mild to moderate OSAH.
A crossover RCT comparing clinical effectiveness and cost-effectiveness of three MADs: self-moulded [SleepPro 1" (SP1); Meditas Ltd, Winchester, UK]; semibespoke [SleepPro 2" (SP2); Meditas Ltd, Winchester, UK]; and fully bespoke [bespoke MAD (bMAD); NHS Oral-Maxillofacial Laboratory, Addenbrooke?s Hospital, Cambridge, UK] against no treatment, in 90 adults with mild to moderate OSAH. All devices improved primary outcome [apnoea?hypopnoea index (AHI)] compared with no treatment: relative risk 0.74 [95% confidence interval (CI) 0.62 to 0.89] for SP1; relative risk 0.67 (95% CI 0.59 to 0.76) for SP2; and relative risk 0.64 (95% CI 0.55 to 0.76) for bMAD (p Â 0.001). Differences between MADs were not significant. Sleepiness [as measured by the Epworth Sleepiness Scale (ESS)] was scored 1.51 [95% CI 0.73 to 2.29 (SP1)] to 2.37 [95% CI 1.53 to 3.22 (bMAD)] lower than no treatment (p Â 0.001), with SP2 and bMAD significantly better than SP1. All MADs improved disease-specific QoL. Compliance was lower for SP1, which was unpopular at trial exit. At 4 weeks, all devices were cost-effective at £20,000/quality-adjusted life-year (QALY), with SP2 the best value below £39,800/QALY.
A MEDLINE, EMBASE and Science Citation Index search updating two existing systematic reviews (one from November 2006 and the other from June 2008) to August 2013 identified 77 RCTs in adult OSAH patients comparing MAD with conservative management (CM), MADs with CPAP or CPAP with CM. MADs and CPAP significantly improved AHI [MAD ?9.3/hour (p Â 0.001); CPAP ?25.4/hour (p Â 0.001)]. Effect difference between CPAP and MADs was 7.0/hour (p Â 0.001), favouring CPAP. No trials compared CPAP with MADs in mild OSAH. MAD and CPAP reduced the ESS score similarly [MAD 1.6 (p Â 0.001); CPAP 1.6 (p Â 0.001)].
An existing model assessed lifetime cost?utility of MAD and CPAP in mild to moderate OSAH, using the revised meta-analysis to update input values. The TOMADO provided utility estimates, mapping ESS score to European Quality of Life-5 Dimensions three-level version for device cost?utility. Using SP2 as the standard device, MADs produced higher mean costs and mean QALYs than CM [incremental cost-effectiveness ratio (ICER) £6687/QALY]. From a willingness to pay (WTP) of £15,367/QALY, CPAP is cost-effective, although the likelihood of MADs (p = 0.48) and CPAP (p = 0.49) being cost-effective is very similar. Both were better than CM, but there was much uncertainty in the choice between CPAP and MAD (at a WTP £20,000/QALY, the probability of being the most cost-effective was 47% for MAD and 52% for CPAP). When SP2 lifespan increased to 18 months, the ICER for CPAP compared with MAD became £44,066. The ICER for SP1 compared with CM was £1552, and for bMAD compared with CM the ICER was £13,836. The ICER for CPAP