Study: Widely Used Nonprofit Efficiency Tool Doesn’t Work

A recent study from North Carolina State University finds that the tool most often used to assess the efficiency of nonprofit organizations isn’t just inaccurate – it can actually be negatively correlated with efficiency.

At issue is something called the overhead ratio, which is the amount of money a nonprofit organization spends on overhead – such as infrastructure, executive compensation and day-to-day management – relative to overall spending. Overhead spending does not include funding spent on implementing programs, such as program staff salaries.

The overhead ratio is important because it is used by many nonprofit rating sites, donors and nonprofit scholars to assess a nonprofit’s efficiency at using its resources to accomplish its organizational goals.

“But the overhead ratio doesn’t actually measure efficiency, for two reasons,” says Jason Coupet, an assistant professor of public administration at NC State and lead author of a paper on the study. “First, the overhead ratio doesn’t account for what organizations are actually doing with their resources. And second, the ratio doesn’t account for what organizations are accomplishing with their non-overhead spending.

“Some nonprofit researchers have raised concerns about the accuracy of overhead ratios as a means of assessing efficiency, but our work is the first to approach it using efficiency theory – and we were able to demonstrate the problem using real-world data,” Coupet says.

For this study, the researchers collected data on the overhead, programmatic and fundraising expenditures of 666 Habitat for Humanity affiliates around the country, as well as how many houses they were able to offer low-income home buyers.

To better understand the value of overhead ratio as a means of measuring efficiency, the researchers turned to two well-established tools for assessing efficiency: Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA). The tools rely on different techniques, but have been proven repeatedly to provide valid and reliable assessments of efficiency across multiple types of organizations.

The researchers used the Habitat for Humanity data to calculate the efficiency of each affiliate using DEA and SFA, as well as calculating each affiliate’s overhead ratio.

The researchers found that DEA and SFA efficiency scores were highly correlated. In other words, if one tool found that an affiliate was efficient, the other tool usually did too.

However, the overhead ratio was negatively correlated with both tools.

“In short, this demonstrates that not only is the overhead ratio bad at assessing efficiency, but also that using it to assess efficiency may actively mislead donors,” Coupet says.

“We’re hoping to work with the nonprofit sector to adopt more accurate efficiency assessment tools,” Coupet says. “That may be tricky, since many nonprofits have organizational goals that are difficult to quantify, but we’re optimistic that better tools can offer richer insights for many organizations. Ultimately, we want to help nonprofit organizations get the most out of their resources, and to identify those nonprofits that are actually operating efficiently.

“If nothing else, we hope people stop using overhead ratios as proxies for efficiency.”

The paper, “Toward a valid approach to nonprofit efficiency measurement,” is published in the journal Nonprofit Management & Leadership. The paper was co-authored by Jessica Haynie, a Ph.D. student at NC State. The researchers would like to thank Habitat for Humanity for providing the data that made the analyses possible.

-shipman-

Note to Editors: The study abstract follows.

“Toward a valid approach to nonprofit efficiency measurement”

Authors: Jason Coupet and Jessica Haynie, North Carolina State University

Published: Sept. 12, Nonprofit Management & Leadership

DOI: 10.1002/nml.21336

Abstract: Past literature in nonprofit management uses the overhead ratio of nonprofits as a measure of efficiency. While the overhead ratio might measure top-heaviness, we argue that it does not measure nonprofit efficiency. To investigate this, we use financial and operational data to rank the efficiency of Habitat for Humanity affiliates with the overhead ratio, as well as Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA), two of the most popular efficiency measures. While the DEA and SFA rankings are statistically correlated, overhead ratio rankings are negatively correlated with both SFA and DEA rankings. We argue that nonprofit scholars, managers, and foundations should move away from concepts and measures of efficiency based on financial ratios, and toward ones that embrace maximizing what nonprofits are able to make and do.