nonprofit performance measurement
In this month’s Social Velocity blog interview, I’m talking with Phil Buchanan. Phil is president of The Center for Effective Philanthropy (CEP) and was the first chief executive of the organization. Under his leadership, the organization has grown into the leading provider of comparative performance data to large foundations and other grantmaking institutions. Phil also serves on the board of Great Nonprofits and is a columnist for The Chronicle of Philanthropy.
You can read past interviews in our Social Innovation Interview Series here.
Nell: At the Center for Effective Philanthropy you work to make philanthropists more effective at creating social change, but a large part of philanthropy is driven by emotion and passion as opposed to results and data. How do you reconcile a push towards more reasoned philanthropy with the emotional aspect that will always be present?
Phil: I understand that some people feel this tension, but to me, it’s hard to understand because I think emotion and passion and results and data can – and should – cohabitate very happily. The passionate, emotional desire to make change is what inspires the commitment to get results. If you believe deeply in helping people in need, but do it in a way that doesn’t help, what kind of emotional satisfaction do you get from that?
Fay Twersky of the Hewlett Foundation articulated this very well in an essay in Alliance Magazine. She says impact should be pursued with “a warm heart and a hard head.” I like this way of thinking about it.
Nell: One of the the things you promote at CEP is a move from evaluating nonprofits based on overhead spending to evaluating them based on achievement of results. But sadly most funders haven’t yet embraced this distinction. What will it take for funders and the general public to recognize that overhead percentages are meaningless and destructive to the nonprofit sector?
Phil: I think the adoption of better nonprofit performance assessment practices is part of the answer. The more data nonprofits can point to that can show what they achieved with their total budgets, the less relevant how that budget was divided will feel to donors.
Look, I think people tend to gravitate toward that which is available, quantifiable, and comparative. Overhead percentages are all of those things, so they become the default performance measure even those they don’t tell you anything about performance. Caroline Fiennes of the U.K. has a great new book called It Ain’t What You Give, It’s the Way You Give It, and one of the best parts is that she really slays the argument for looking at administrative costs, while also providing guidance on how to approach performance measurement.
The rub is that the only way we’ll get better overall nonprofit performance assessment practices is if funders support that work. In our research, we have seen that, contrary to the stereotypes, nonprofits care about assessment and are working on it. But they want and need much more support – financial and non-financial – from their funders. I hope that funders embrace this and support better assessment practices in service of better outcomes.
I think Mario Morino has been a powerful voice on this topic and I recommend his book, Leap of Reason, to everyone I can. I hope people are listening to Mario because measuring effectiveness isn’t some academic issue. People who work at nonprofits deeply want to be effective. Foundations want to be effective. The people we help desperately need us to be effective. So we should – and we must – figure it out and get beyond empty measures. And many have. There are some fantastic exemplars when it comes to nonprofit performance assessment. But there are not enough.
Nell: In addition to leading CEP, you also serve on the board of GreatNonprofits, which allows individuals (clients, donors, volunteers) to review nonprofits. How does the idea of individual consumer reviews of nonprofits fit into the larger movement to evaluate nonprofits based on outcomes when the average person doesn’t yet understand or embrace the idea of nonprofit performance measurement?
Phil: In some ways I think it’s very easy for anyone to grasp. You’re trying to help someone; shouldn’t you ask whether they feel they have been helped? GreatNonprofits can provide that read on whether individuals served by a nonprofit feel they’ve been helped. I think GreatNonprofits, which Perla Ni founded and leads, is really important and I also think we need other kinds of efforts to collect and analyze beneficiary perception data. We need both the kind of open, web-based opportunity GreatNonprofits offers as well as rigorous, survey-based efforts such as the Center for Effective Philanthropy’s YouthTruth initiative, which helps schools, districts, and funders hear from middle school and high school students. We’re debating school reform in this country yet many of those with power and resources don’t understand the students’ experiences. We know that those experiences correlate to outcomes, so this kind of perceptual data could be a vitally important “leading indicator” of progress.
Nell: Philanthropy tends to be fairly risk averse and focused on program funding, as opposed to the organization-building capital investments (money to build organizations rather than buy services) the nonprofit sector so desperately needs. What do you think it will take to get more philanthropists to make riskier, longer-term, organization-building investments?
Phil: I think there needs to be a greater recognition that we count on organizations to get the work done. Sounds obvious, I know, but I think funders sometimes forget.
It is stunning, and sobering, that despite the valiant advocacy of Paul Brest, Paul Shoemaker, GEO, NCRP, and others, there has been no increase in the provision of general operating support over recent years. But we also need to be careful not to pretend operating support alone is the answer. Our research demonstrates that what really matters to grantees is operating support that is multi-year and a decent chunk of change – six figures or up in annual support, ideally. So the problem isn’t just one of grant type, it’s also one of grant size.
This comes back to assessment, too, in my view. If, as a funder, you know what you’re going after, and there is an organization that is focused on the same goal and can show that it’s delivering results, why would you not provide significant, long-term, unrestricted support? And, if you can’t find organizations delivering results toward your shared goal, why wouldn’t you fund in a way that would allow them to build that capacity?
Nell: You recently wrote a fairly scathing critique of Dan Pallotta’s new book, Charity Case because you thought his approach to advocating for the nonprofit sector was misguided. Yet the nonprofit sector is largely underfunded, undervalued, and dismissed in the broader regulatory and political environment. What do you think it will take to change that reality?
Phil: Pallotta’s book doesn’t advocate for the nonprofit sector that I know – or for one that I would ever hope to see. He wants the sector to become something entirely different, something a lot more like business, something that ultimately might not be discernible at all as a distinct sector. His take on the sector is both ahistorical (he demonstrates almost no understanding of the sector’s past contributions) and ideological (he has written that “the free market is a self-correcting system” that supports our “natural desire to help each other” and “only stops working when it is interfered with”). He is infatuated with free market analogies, believes financial incentives are the key to motivating people despite research demonstrating that they are not, insists that public trust in charities is lower than in other sectors when all credible research shows the opposite, and does not seem to understand that many nonprofits work to address the problems that exist as a result of market failures. His book is a disservice to the nonprofit sector.
So, then, what do we need to do to increase the appreciation of public and government officials for the nonprofit sector?
We need to start by standing up and asserting our value as a sector separate and distinct from business and government. We need to stop buying into the fiction that being effective means being “like a business,” whatever that even means. We need to stop praising the “blurring of the boundaries” and start articulating why we need organizations that pursue mission alone rather than profit for their shareholders. We need to explain why the sector is good for our society, good for business, good for government, good for citizens: we all need the nonprofit sector to be its best for us to be our best. And we need to re-learn our history – Olivier Zunz’s recent book on U.S. philanthropy would be a good place to start.
Yes, of course there is much work to do to improve the sector, but that doesn’t mean we need to tear it down. I wrote a series of blog posts for Duke University’s Center for Strategic Philanthropy and Civil Society a few years ago and argued that just as it is possible to walk and chew gum at the same time, it is possible to believe both that the nonprofit sector is and has been a defining strength of this country and that it must dramatically improve its effectiveness. It is possible to both celebrate the diversity of the sector and its various organizations and push for greater clarity of organizational goals, strategies, and performance indicators. It is possible both to applaud initiatives fostering “social innovation” and the government’s embrace of this push and also recognize what has worked in the past.
We need not tear down the sector to improve it. We need not disparage all that has come before in order to chart a better future.
I’m excited to announce something a little different on the Social Velocity blog: a contest! Mario Morino, author of one of my favorite new books, Leap of Reason: Managing to Outcomes in an Era of Scarcity, has generously offered to give away a Leap of Reason board package to three lucky nonprofit readers of the Social Velocity blog. You can read my past review of the book and why I like it so much here and my past interview with Mario here.
The Leap of Reason board package will include a copy of the book for each board member and other supplemental materials to get the board discussing how to manage toward outcomes. If you would like to be entered into the contest, simply respond in the comments with a brief (1-3 sentence) description of why you think your nonprofit is ready to start managing toward outcomes.
In Leap of Reason, Morino, co-founder of Venture Philanthropy Partners, argues that every nonprofit MUST, if it wants to survive in this new environment of “brutal austerity,” create a culture of performance. Many nonprofit organizations simply exist to “do good work.” But that is just not enough anymore. It’s not enough for those that fund the work, and it’s not enough for those who receive the services. Nonprofits must determine what they exist to change and whether they are actually creating those changes. Mario is ever-mindful, however, that large scale evaluation projects are simply unrealistic for the vast majority of nonprofits. They don’t have the money or time to devote to such projects. He and other experts in the book provide key initial steps and case studies to encourage nonprofits to develop their own ways to manage to outcomes.
So, if you think your nonprofit is ready to start managing to outcomes, and you’d like the Leap of Reason board package to help you along, respond in the comments with a short (1-3 sentence) explanation of why you think your nonprofit is ready.
I will then pick three winners. Each nonprofit winner will receive:
- A Leap of Reason book for each board member
- A Leap of Reason User Guide for each board member
- A Leap of Reason Supplemental Reading Packet for each board member
- A Leap of Reason Board Package Overview (a how-to guide for the executive director or board chair leading this process)
Update on February 21, 2012: The response to this contest was so great that Mario Morino graciously agreed to increase the number of winners to 20. Those 20 nonprofit winners have all been notified with the specifics about how to claim their Leap of Reason board packets. Thank you so much to everyone who participated! And I look forward to future Social Velocity blog contests!
In this month’s Social Velocity blog interview, we’re talking with David Henderson. David is the founder of Idealistics Inc., a social sector consulting firm that helps organizations increase outcomes, demonstrate results, and organize information. He has worked in the social sector for the last decade providing direct services to low-income and unhoused adults and families, operating a non-profit organization, and consulting with various social sector organizations. David’s professional focus is on improving the way social sector organizations use information to address poverty.
You can read past interviews in our Social Innovation Interview Series here.
Nell: On your blog, Full Contact Philanthropy, you write a lot about making program evaluation accessible to all nonprofits, even small and under-resourced ones, which is something that a lot of those pushing for evaluation neglect to address. Evaluation can be expensive, time-consuming and poorly executed. What is the essence of good evaluation, and, at a minimum, what should all nonprofits be doing to evaluate their work?
David: Whatever the price tag, a good evaluation helps you make better decisions, a bad evaluation does not. If an organization is not open to changing its course of action regardless of what the data suggest, then evaluation has no meaning. Therefore, the most important step in any evaluation is knowing what you want to evaluate and why.
While some evaluations are expensive, they don’t all have to be. Evaluation does not mean just one thing. There is no one right way to do evaluation. Instead, there are a number of ways organizations can use outcomes metrics to inform their work, ranging from randomized control trials (most accurate and most expensive) to simply monitoring whether a few key indicators are getting better or worse.
More important than the certitude of any one evaluation is the regularity with which an organization uses metrics in decision making. It’s not terribly costly to start every staff meeting with an update on how the people you are helping are doing. But this discipline helps create cultural commitment to using outcomes data in decision making, which is really at the core of any good evaluation strategy.
Nell: Is everything in the social change arena measurable? Are their some public good efforts that are so complex or have so many variables that we cannot measure them, yet they still need to happen?
David: When we think about measurement, we tend to imagine a numeric, linear scale with start and end points. Not everything is quantifiable, but that doesn’t mean it’s not measurable.
Organizations collect information all the time. Some of that data is quantifiable and gets stored in spreadsheets and databases. But we also get a lot of important information through visual observations and conversations.
All of this information, quantitative and qualitative, objective and subjective, helps inform decision making. Taking the information we have and establishing evaluative frameworks that help us make systematic program decisions is the real challenge.
Nell: How does government fit into the effort for social change? Can and is government changing quickly enough to keep up and to have a relevant place?
David: Ideally, the non-profit sector would innovate and test social interventions, and governments would take the best innovations to scale. But successful social innovation requires cultural commitment to both evaluation and failure. And in the current funding environment, failure is not an option. That’s a big problem.
With so much pressure on organizations to show evidence of impact, instead of investing in innovating new social solutions, non-profits are hiring marketing consultants shrouded as evaluation experts to help them tell their stories.
If the government is to invest in and scale what works, as the federal Social Innovation Fund purports to do, organizations have to be free to report what does and what does not work. So long as our focus is on story telling instead of truth telling, it’ll be difficult for non-profits to have the latitude to experiment and evaluate freely, leaving the government precious little worth scaling.
Nell: Your particular interest is social change efforts to alleviate poverty. But since poverty is the result of some very serious failures in America’s infrastructure (inadequate education system, broken health care system, etc) is it possible to fix the results of those inadequacies without addressing those much larger structural deficiencies? Or can social entrepreneurs do both?
David: Poverty eradication has to be the goal, but alleviation is pretty darn important to the 43.6 million Americans and billions more worldwide living in poverty today. Social entrepreneurs as well as a myriad of government efforts address both structural causes and the many harms resulting from poverty.
Regardless of a particular intervention’s focus, every effort is more likely to succeed when informed by regular outcomes assessments. Since my firm’s focus is helping organizations use client metrics to make higher impact program decisions, we work with all types of organizations across the anti-poverty spectrum.
Nell: How does your company Idealistics fit into the solution to poverty?
David: Our practice is about helping organizations make smart, high impact decisions that increase social outcomes. Everything we do is underscored by a vision of a social sector that uses evidence in the crafting, implementation, and iterative evaluation of its interventions.
Probably the most important thing we do toward that end is helping organizations establish decision frameworks. A decision framework converts an agency’s theory of change into a tool, or a mathematical model as we think about it, that organizations can test, update, and use in the design and execution of their interventions.
With a solid decision framework in place, we provide analytically oriented consulting and technology systems that help organizations establish data collection pipelines to make sense of their information.
While a lot of our customers hire us so they can better prove to their funders that they’re making a difference, that isn’t our objective. But the fact is our customers do very well with their funders.
Our clients are able to uniquely demonstrate an analytical approach to their work, and have the evidence they need to back their claims of progress, which makes them very competitive in the evidence-deficient social sector landscape. However, for me and my team, the real gratification is not that our customers impress their funders, but that they are better positioned to change the lives of the people they serve.
Since I was on vacation for a couple of weeks in August and pretty much unplugged, I’m probably not qualified to list the 10 greatest reads in social innovation for the month of August, but I’m still going to give it a shot. As always, please add what I missed to the comments.
You can also read the lists of Great Reads from previous months here.
- Guest blogger on the Tactical Philanthropy blog, Jed Emerson, a pioneer in the impact investing arena, argues that impact investing is at risk of missing a key opportunity to move the field forward.
- Strategic finance is one of the hardest things for many nonprofit leaders to master, but also one of the most critical. Nonprofit Finance Fund explains how to approach it.
- Sea Change Capital Partners and Lodestar Foundation are partnering to create a new fund to pay for nonprofit collaboration and mergers. A pool of merger money is a great new addition to what is a pretty big hole in the nonprofit capital market.
- From the Harvard Business Review blog comes the argument that sometimes it can be good for business to fire some customers. This concept should apply to nonprofits’ donors as well.
- One of the biggest hurdles to nonprofit performance measurement is a lack of money to make it happen. On the Social Currency blog, Angela Francis explains how nonprofits can find the money for evaluation through capacity capital.
- The biggest news in August was nonprofit Jumo’s merger with for-profit GOOD. Antony Bugg-Levine (who was just announced as the new CEO of the Nonprofit Finance Fund yesterday) explains how this merger is just the beginning of a real blurring of sector lines to come.
- On August 24th, US Secretary of Education @arneduncan held a Twitter Town Hall to answer questions about America’s public education system and his ideas for reform. You can see the Tweets at #askarne or read the highlights here. He plans to hold another Twitter Town Hall soon.
- The Future Generations blog offers a great framework and examples of that often touted, but rarely understood, concept: “scale.”
- In the wake of Steve Jobs’ resignation from Apple, Cliff Kuang offers a reflection on Jobs as a supreme innovator and great user of technology.
- From the tech blog, A Smart Bear, comes a lesson for entrepreneurs (and social entrepreneurs too) when being an expert is harmful.
Photo Credit: afunkydamsel