Let’s Take a Step Back in the Outcomes Debate

There is a growing discussion among social impact organizations and those who fund them about how to measure impact.  It is indeed a very slippery endeavor.

Mario Marino, Chairman of Venture Philanthropy Partners (a venture philanthropy fund in Washington D.C. that makes growth capital investments in nonprofits) has been encouraging nonprofits to measure outcomes for years.  Indeed one of the fundamental characteristics of venture philanthropy is a reliance on metrics and outcomes for investment to happen.  He recently wrote a post arguing that he is “increasingly worried that the vast majority of funders and nonprofits are achieving, at best, marginal benefit from their efforts to implement outcomes thinking.”  He argues that in an zealous pursuit of metrics we have left common sense and “softer” impact behind and encouraged nonprofits to move away from the impact they were working towards.

To add further confusion to the outcome measurement discussion, the Gates Foundation’s Melinda Tuan studied 8 approaches to measuring cost vs. social impact, or the value that nonprofit organizations create versus the cost of their activities.  The results of the study were disheartening; none of the approaches they studied was a magic bullet, all had significant drawbacks, which led them to conclude: “Integrated cost approaches to measuring and/or estimating social value are still in the nascent stages of development due to the lack of maturity in the field of social program evaluation.”

And there are other camps working towards outcome measurement, like those debating about whether randomized control trials (a research methodology where a random group of program participants is tracked and compared to a random group of cohorts who did not participate in the program) are feasible for nonprofits. And on the social business side, the GIIN (Global Impact Investing Network) is developing standards for measuring and communicating the social impact of investments known as The Impact Reporting and Investment Standards (IRIS).  And that’s just a start.

This whole social impact measurement endeavor is incredibly important because if we can figure out a way to measure which social change efforts work, and which don’t, we can allocate resources accordingly and, in theory, get closer to solutions to social problems.

But I think we need to first take a step back.  As is so often the case in efforts to build nonprofit capacity, effectiveness and infrastructure (including, in this case, the ability of nonprofits to evaluate their work) the focus is on the largest, most resourced nonprofit organizations.  Let’s remember that more than 80% of nonprofit organizations have budgets under $1 million (see the Nonprofit Almanac).  Budgets that small leave very little room for funds to support randomized control trials or other kinds of outcome measurements.

But an even bigger roadblock is the fact that many nonprofit organizations have not articulated their theory of change, or their logic model.  Many nonprofit organizations are doing good work, but they don’t necessarily have an articulated strategy around that good work.  A logic model helps an organization understand and articulate how they believe that they translate resources (inputs) into social impact, or change in a community.  This understanding allows the organization to better articulate (to potential funders, volunteers, supporters, partners), and create strategy around, their work.  A potential logic model for an English as a Second Language after-school program could be as follows:

One of the first steps Social Velocity undertakes with clients who want to increase organization capacity, sustainability, revenue, growth, or really any kind of progress, is to create a logic model with the organization.  The majority of nonprofits that I encounter don’t have an articulated logic model or theory of change.  It may seem like an academic exercise, but I would argue that it is absolutely critical to just about anything a nonprofit does.  In order to understand their place in the community, the value that their work adds, how additional inputs (like funding) can increase impact, and their strategy for delivering services, they need to articulate this process.

But the larger debate about outcome measurement ignores the fact that the majority of nonprofit organizations have not completed step 1 in outcome measurement: articulating a strategy for using resources to create outcomes.  Once this is articulated, we can talk about how to measure whether that strategy is actually coming to fruition.

Tags: Gates Foundation, GIIN, IRIS, logic model, Mario Marino, Melinda Tuan, Nonprofit Almanac, outcomes, randomized control trials, social impact, Social Velocity, theory of change, Venture Philanthropy, Venture Philanthropy Partners

2010 and the Future of the Social Sector

Lucy Bernholz, head of Blueprint Research and Design, a philanthropy consulting firm, and thought-leader on trends in philanthropy is preparing a monograph on what 2010 will hold for the social sector. As a true adopter of social media, she is asking others to contribute, in essence crowd-sourcing answers, this year to her annual “what will next year bring” treatise. Last week, she asked her blog readers, Twitter followers, and all others the question: “What trend, change, entity, or idea will matter most to the social sector in 2010?”

She’s gotten a great set of responses, in blog, email, Tweet, and other forms, which she and others are collecting. It’s kind of an interesting experiment to ask a broad question to the universe and see what you get back, and whether it is intelligible and adds anything to what she may have already been planning to write. It is also interesting to navigate the very fine line between future-telling and wishful thinking. I probably tend to fall into the latter category, but if we don’t envision the future we want to see, we probably won’t get there.

I submitted my thoughts to Lucy via Twitter, but it is difficult to distill broad ideas into 140 characters, so I will elaborate on my thoughts here.

There are three things that I think will matter most to the social sector in 2010:

1. Increased Philanthropic Dollars Will Go to Organization Building. Donors will increasingly realize that they can achieve a greater social return on their investment (more social impact) when they invest in the capacity, or growth of a successful nonprofit.  That is to say that donors will increasingly realize the power of BUILDING organizations rather than BUYING services.  I don’t think donors will move away from buying services, there will still be a majority of that.  But I think donors will start to understand the difference between a “donation” where they are simply supporting an organization’s current program, versus an “investment” that makes the organization stronger, healthier, better positioned to address the social problem head on.




2. Nonprofits Will Move From Outputs to Outcomes. And in order to meet this trend of donors wanting to invest rather than donate, nonprofits will begin to understand that they will attract more capital if they can demonstrate a social return on investment, or a change in outcomes, not just outputs.  Outputs have been a favorite of the nonprofit sector, i.e. 500 kids went through our after-school program, 1,000 meals were served in our kitchen. But outputs don’t demonstrate social impact, or a change to a problem.  Outcomes do, which is what investors increasingly will want to see.  Outcomes are about changed lives, changed trajectories.  It is so much more powerful and compelling to be able to say that the 500 kids that went through our after-school program stayed in school and increased their academic achievement which was a marked difference from their cohorts that didn’t attend our program.  Then, if you can continue to track those children and demonstrate that they continued to stay in school at a higher rate than their contemporaries, you have a compelling change to a trajectory.  You begin to show how your organization is an intermediary between donors who want to invest in social change and a change you are making in the community.  I believe that philanthropic capital will begin to flow more readily to those nonprofit organizations that can demonstrate outcomes as opposed to outputs, and those nonprofits that can comply will be more successful at attracting capital.




3. The Social Capital Market Will Increasingly Include Philanthropic Capital. The social capital market to date has focused mostly on investing in social businesses that provide both a social and financial return. Philanthropy and nonprofit organizations have been somewhat left behind. But this will change with a growing recognition of the benefits of broadening the definition of social capital markets to include nonprofits and philanthropy.  There is much to be gained when ALL organizations working towards social impact and ALL investors interested in social return can pool resources and work towards closer collaboration, creation of new financial vehicles, sharing of ideas and information.

Perhaps 2010 is too early for all three of these trends to really take hold, but I think the beginnings are there. It will be interesting to see what Lucy comes up with, and what actually starts evolving in a few short months when the new year begins.

But in the meantime, what are your thoughts?  Where do you see the social sector going in the coming year?

Tags: 2010, Blueprint Research and Design, Lucy Bernholz, nonprofit, outcomes, philanthropic capital, Philanthropy, social capital market, social impact, trends