What AP learned about using online surveys to measure support for presidential candidates

It’s almost two months until Election Day but voters already have started mailing in their ballots for president, a reminder that more than a third of all votes this year will be cast before Nov. 8. The huge increase in advance voting over the past decade is forcing the AP and other media organizations to look for more accurate and less expensive ways to estimate which candidates all those early voters are supporting. The AP earlier this year commissioned an experiment to see if an online, probability-based survey could estimate candidate support among both advance and Election Day voters more accurately than the combined exit poll of Election Day voters and telephone poll of advance voters. That’s the dual, and increasingly costly, methodology the AP and its network partners in the National Election Pool now use. The experiment in the Florida presidential primary -- reported in this blog last May -- found that the average error in the estimates for each of the five Republican and two Democratic candidates who were still in the race at that time was 1 percentage point. In contrast, the average error on the combined exit poll and telephone poll estimates of the same seven candidates was 2.2 percentage points. Now that Florida election officials have compiled the official vote broken down by advance and Election Day voters, we can update the earlier findings. Among advance voters, the online survey estimates for the two Democratic candidates were off an average of 1.1 percentage points, while the telephone poll estimates had an average error of 8.4 percentage points. Among the five Republican candidates, the average online error was 0.7 percentage points, compared to an average of 0.5 percentage points for the telephone poll. The NEP exit poll’s estimates from Election Day voters were more accurate than the online survey estimates. The average error for all candidates from both parties was 0.7 percentage points for the exit poll and 2.7 percentage points for the online survey. The AP experiment was conducted by GfK Custom Research and paid for by a grant from the Knight Foundation. GfK used the Florida residents on its probability-based, 55,000-member national online panel as participants in the survey.They added in a larger non-probability-based sample of Florida residents, using known differences between the two groups to blend them into one sample for each party. The experiment also compared online estimates of the demographics and attitudes of each party’s primary electorate with the exit poll/telephone poll findings. Question-by-question comparisons between the online results with normal geographic weightings and the exit poll results at 5 pm election day can be found here; comparisons after both online and exit poll/telephone poll samples were weighted to the final vote count can be found here. GfK’s full report on the Florida experiment, updated to include the advance and Election Day breakdowns, can be found here.