We’ve been quietly making predictions each week and submitting to the CDC’s Flu challenge. Here’s a brief update of some of the lessons learned.
First, the data are continually updated (at least the previous week, and occasionally further back). These can be modest changes or significant, but they can’t be anticipated (both the direction of the change and magnitude vary even in the same region). This should be factored into the predictions, either in the form of weighting the most recent data less, or performing sensitivity studies with some representative confidence intervals.
Second, the profiles this year are generally considerably more complicated than previous recent years. This may be due, in part, to El Niño. If so, our reliance on historical values for specific humidity cannot accurately account for humidity variations.
Third, incorporation of priors into the forecast appears to produce more accurate forecasts, although not in all cases. This changes as we move through the season, so that initially, the prior forecasts are consistently better, but, by the end of the season, they will have little value, and solutions not using priors will perform better. (This week, three of the regions performed better without priors).
Fourth, we also incorporated a timing delay to account for the fact that there is a delay between the time that an individual becomes infectious and presents themselves to a clinic. The shape of the prediction was not measurably altered, but the likelihood was significantly improved, due to the shifting of the profile. A more robust approach would be to include an “Exposed” category to the compartmental model, which is currently being done.
Fifth, we found that school holidays is often an important driver of transmission, although because of the limited number of parameters allowed by the model, it can sometimes result in strong future predictions that may not occur. In particular, to account for the decrease over the Christmas period, the model introduces a significant decrease in R0. This same magnitude is then applied in late March during the spring break resulting in a dramatic decrease in %ILI (see Regions 4 and 7 in the accompanying Figure). Although this may yet occur, it is quite dramatic, and is more likely the result of the model over-fitting smaller fluctuations during the early rise portion of the season.
For almost all regions, we predict that the peak has already occurred. For any remaining ones, it will likely occur this week or next.