Cost-Effectiveness
Among programs that meet our other criteria, we give priority to those that deliver the greatest number of votes per dollar spent. The cost to generate a single vote can vary dramatically—some programs may require several thousand dollars to move just one vote, while others can generate 15 or more votes with the same investment. This variation is why cost-effectiveness is a central factor in our recommendations.
We measure this impact differently depending on the type of program:
For non-partisan programs, effectiveness may be assessed by:
The number of additional ballots cast by individuals who likely would not have voted otherwise.
The improvement in voters’ understanding of key issues and current events.
For partisan programs, we focus on:
The number of net Democratic votes gained per $1,000 invested.
By grounding our support in these metrics, we aim to ensure that every dollar we direct contributes as effectively as possible to strengthening democracy.
Our Cost Effectiveness Approach
To estimate the cost-effectiveness of a program, we perform our own analysis.
Whenever possible, the cost-effectiveness estimates are based on randomized controlled trials (RCTs), the gold standard in determining causal inference. In most cases, the organizations we consider have conducted RCTs of their own programs. When an organization under consideration has not conducted its own RCT, we review other organizations’ RCTs showing the effects of similar tactics.
We rely on organizations’ providing us with their study design, methodology, raw data, cost of implementing the program, and results from randomized control trials (RCTs). Our evaluation team reviews the methodology and assumptions, and in most cases, we perform our own analysis of the raw data to verify results.
From this analysis, we estimate the cost per net Democratic vote for each program under consideration and we recommend the most cost-effective programs that can achieve scale in our key competitive races.
Our cost-effectiveness estimates typically assess a program’s cost per vote for a single election or net Democratic vote over multiple election cycles. We also quantify the number of races, up and down the ballot, on which a program may have an effect.
When the differences in estimated cost-effectiveness are not significant, we also consider factors such as confidence in an organization's track record, capability of their leadership, and the strength of the evidence supporting a particular election program.
After each election cycle, organizations provide us with data on the results of their RCTs, and we compare our projections to the actual cost of the program per net vote.
The Value of Testing
Testing not only provides us with confidence that programs will produce net votes, it also helps organizations improve their programs.
Some organizations have the ability to test within the election cycle to determine which messages and tactics persuade voters to turn out for a particular candidate.
By elevating the best-performing tactics and dropping the worst, their programs maximize the impact of donations.
Our Metrics Compared to Figures Many Organizations Report
Focus for Democracy reports on the unique and discrete contributions of each program we recommend.
Some organizations overstate a program’s impact, perhaps unintentionally, because of errors like these:
They may assume they were responsible for all the votes gained in the areas where they worked, failing to consider the work of other groups or other factors. For example, an organization might claim, “Because of our work, voter turnout increased by 10% in the county.” However, they might neglect to mention other organizations that also encouraged voters to get to the polls in the county. Alternatively, they may have overlooked other factors, such as heightened public interest in the election, which could have accounted for some of the gains.
If they’ve run a randomized controlled trial, that tells us how much of the vote gain resulted from their work alone.
Any organization that produced more votes for the winning candidate than the margin by which the race was decided can —and should — recognize that the outcome would have been different if its work had been absent.
Even then, it’s essential to recognize that the efforts of other organizations and the candidate's campaign made the race close enough that the votes they secured could put the candidate over the line.
They may measure the wrong thing.
Organizations often report metrics that reveal little or nothing about their actual impact. For example, they might report the outputs of their program, such as the number of mailers sent, digital ads produced, doors knocked on, phone calls made, and texts sent to persuade voters to cast a ballot.
Although many of these outputs are necessary, Focus for Democracy’s analysis goes a step further, measuring the outcomes — the impact those tactics have on the election.
For nonpartisan efforts, that means the number of people who voted who would not have done so in the absence of the program. For partisan organizations or evaluators, this means the net number of Democratic votes gained as a result of the program.
When can an organization accurately say its work changed the election outcome?
There has probably never been an election in which a single organization determined the outcome. Both candidates, their campaigns, and numerous organizations typically contribute to the outcome. But let’s say a House candidate wins the election by a 10,000-vote margin (as often happens). In that case, it is accurate to say that for any organization that has produced more than 10,000 votes, the results would have been different without its work.
That’s different than saying the organization “won the election.” No one organization did that. However, it does mean that each organization that received more than 10,000 votes (and the donors who made that possible) should know that the outcome would have been different without them. In close elections, there will usually be more than one group that produced more votes than the margin by which the race was decided. But the absence of any one of them would have resulted in a different outcome.
Learn More About Our Work:
Focus for Democracy Action is a 501(c)(4) nonprofit dedicated to strengthening democracy through evidence-based donor advising. A minority of its work involves lawful coordination with candidates and campaigns, as permitted under IRS regulations for 501(c)(4) organizations. All recommendations are vetted to ensure compliance with IRS regulations.