Around 75 percent of forecasts about the impacts of new, potentially high-cost healthcare technologies still in research and development, are accurate.
This is according to the ECRI Institute which reviewed its own forecasts on major technologies and found that three-quarters of early predictions were still sound when the technologies reached routine clinical care.
ECRI Institute’s case study, published in the February 2015 “Biomedical Innovations” themed issue of Health Affairs, compared very early predictions with updated ones made after the technologies entered clinical care.
ECRI Institute’s article addresses three questions central to forecasting’s usefulness: are early forecasts sufficiently accurate to help providers acquire the most promising technology and payers to set effective coverage policies; what variables contribute to inaccurate forecasts; and how can forecasters manage the variables to improve accuracy?
The article analyzed and compared forecasts published between 2007 and 2010 by ECRI Institute on four technologies, and updates published in late 2013 and 2014.
Topics included single-room proton beam radiation therapy for various cancers; digital breast tomosynthesis imaging technology for breast cancer screening; transcatheter aortic valve replacement for serious heart valve disease; and minimally invasive robot-assisted surgery for various cancers.
“We examined revised ECRI forecasts to identify inaccuracies in the earlier forecasts and explore why they occurred,” said Lerner. “We found that frequent revision of forecasts could improve accuracy, especially for complex technologies whose eventual use signals a paradigm shift in clinical care for a disease.
“We were surprised that so few forecasters examine their own work, and we hope this initial self evaluation leads to increasingly robust, independent efforts.”
IT and Data Security, ECRI Institute, Risk Management, US
