Follow Social Velocity on Google Plus Follow Social Velocity on Facebook Follow Nell Edgington on Twitter Follow SocialVelocity on Linked In View the Social Velocity YouTube Channel Get the Social Velocity RSS Feed

Download a free Financing Not Fundraising e-book when you sign up for email updates from Social Velocity.

evaluation

The Social Innovation Fund Six Years On

social innovation fundThere is an interesting report out today on the effectiveness of the Social Innovation Fund (SIF). Authored by the Social Innovation Research Center (SIRC), a nonpartisan nonprofit research organization, the new report details what has worked and what hasn’t in the six year history of the SIF.

Launched by the Obama administration in 2009, the SIF — a program within the Corporation for National and Community Service — provides significant funding to foundations that follow a venture philanthropy model by regranting that growth capital, along with technical assistance, to evidence-based nonprofits in “youth development, economic opportunity, and healthy futures” areas. In 2014, SIF expanded its efforts to include a portfolio of Pay for Success (social impact bond) grantees.

Now, 6 years on it is interesting to take a look back to understand what, if any, effect SIF has had on the nonprofit sector. The effect of the SIF is also critical given that, as of right now, the House and Senate have both defunded SIF in their respective funding bills.

To date, the SIF portfolio is made up of $241 million of federal investments and $516 million in private matching funds, which was invested in 35 intermediary grantees and 189 subgrantee nonprofits working in 37 states and D.C.

The SIRC report focuses on the current progress of SIF grants made during the first three years of the program (2010-2012). The report finds two clear positive results for the SIF so far. The SIF has:

  1. Added to the nonprofit sector’s evidence base about which programs work, and
  2. Built the capacity of nonprofit subgrantees, especially in the areas of “performance management systems, evaluations, financial management, regulatory compliance systems, and experience with replicating evidence-based models.”

On the negative side, however, the report finds that the SIF put real burdens on funders and nonprofits with its fundraising match requirements and the federal regulatory requirements. The report also finds that the SIF has had little effect on the sector as a whole because the SIF has not very broadly communicated their learnings so far.

To me, of course, most interesting are the report’s finding about capacity building at nonprofit subgrantees. There is such a need for nonprofit capacity building in the sector, and this was a clear goal of the SIF.

The SIF is one of few funders that do more than pay lip service to performance management by actually investing in building the capacity of nonprofits to do it. However, the SIF has been criticized for mostly selecting nonprofits that already had strong capacity. And indeed, the SIRC report finds that the SIF was most successful among those nonprofits that already had high capacity (in performance management, fundraising function, etc.) prior to SIF funding. Indeed, the report found that “poorly-resourced intermediaries working with less well-resourced community based organizations have been at a disadvantage.”

One SIF grantee in particular, The Foundation for a Healthy Kentucky, really struggled to build the capacity of their subgrantees whose starting capacity was so low. As they put it:

During the course of participation, it became clear that…[SIF] was really better suited for replicating existing programs or, at a minimum, investing in well-established programs that had some level of sophistication around organization systems and evaluation.

This mirrors earlier criticism of the SIF that it was set up to grow only those nonprofits that were already doing well, while those nonprofits that struggled with basic capacity issues were left out. The SIF has struggled to determine whether it is funding innovation (new solutions with limited capacity), or proven solutions (with a long track record and the corresponding capacity). It seems the two are mutually exclusive.

What the SIF is trying to do is such tricky business. To identify, fund and and scale solutions that work is really the holy grail in the social change sector. Certainly there are hurdles and missteps, but I think it’s exciting when government gets in the social change game in a big way. Six years is really too soon to tell. So I hope that this brief SIF experiment is allowed to continue, and we can see what a social change public/private partnership of this scale can really do.

To read the full SIRC report go here.

Photo Credit: Obama signs the Serve America Act in 2009, Corporation for National and Community Service

Tags: , , , , , , , , , ,

A Nonprofit Culture of Measurement: An Interview with Mary Winkler

Mary WinklerIn today’s Social Velocity interview I’m talking with Mary Kopczynski Winkler, senior research associate with the Center on Nonprofits and Philanthropy at the Urban Institute. Mary is a nationally recognized expert in the field of performance measurement and management.   She is a founding member of the Leap of Reason Ambassadors Community, a private community of nonprofit thought leaders and practitioners committed to increasing the expectation and adoption of high performance in the social sector and who released the Performance Imperative earlier this year.

You can read past interviews in the Social Velocity interview series here.

Nell: PerformWell is an effort among Urban Institute, Child Trends and Social Solutions to offer tools and strategies for human services nonprofits to measure their work. How successful has this effort been and what are your plans for continuing to grow the capacity of nonprofits to measure their work?

Mary: PerformWell is a free, interactive, web-based resource designed to help human services nonprofits gain knowledge about performance management, access tools and resources they need to better service clients and meet outcomes, and obtain strategies for effective, efficient service delivery. Launched in March 2012, the demand for PerformWell has exceeded our expectations with more than 400,000 visitors (from all 50 states and more than 200 countries); 25,000 individuals have registered for our webinars; and more than 140,000 assessment tools have been downloaded from our site. Webinar survey results are routinely high, but we are working to put additional systems in place to track how nonprofits are using various aspects of PerformWell and to what end.

In 2013, the PeformWell partners engaged in a business planning process with Root Cause. Market research confirmed our views about a large unmet need for performance measurement knowledge and high interest in the resources offered through PerformWell, but that additional products and services are also desired, such as webinar training series, regional user conferences, and customized engagements with nonprofits. Users wanted a more interactive web-experience.

Our short- to medium-term goals include substantial updates to the website to improve the user experience (we also plan to solicit user feedback during and after these changes are implemented); development of additional products and services better aligned with the feedback obtained from the market research undertaken by Root Cause; and exploration of partnerships and sponsorships with nonprofits, consultants and funders to generate additional revenue and resources to expand the content, reach and use of PerformWell to improve the adoption and application of performance measurement and management practice across the nonprofit sector.

Nell: Some believe that measurement is perhaps more straightforward for human services nonprofits — you can measure change to an individual’s behavior or life circumstances — but measurement is more difficult for arts organizations or advocacy groups. What are your thoughts on that?

Mary: Sometimes I think this argument serves as a convenient excuse for organizations to avoid putting even the most basic systems in place to track progress or otherwise hold themselves accountable to their constituents. In 2007, with support from the Hewlett Foundation, the Urban Institute and the Center for What Works, we published a series of simple frameworks, as part of our Outcome Indicators Project, to help nonprofits in 14 program areas engage in performance measurement. Two of these areas are advocacy and performing arts. The Urban Institute also provided research support to the Performing Arts Research Coalition (PARC) to develop standardized surveys to help performing arts organizations across the country obtain more routine and better data from audience members, subscribers, and the community.

Establishing a causal link between advocacy or arts interventions and impact is, in my view, more challenging than for human service organizations. In the case of advocacy organizations, it can be very difficult to isolate the contributions of a particular campaign or even organization to a policy or legislative outcome.

It is, however, possible to devise strategies for capturing information on earlier stage outcomes, such as increased awareness.

I recently participated on a panel at the annual OPERA America conference – on “internal metrics for civic impact.” As much as measurement activities have evolved from the days of the PARC coalition, I observed that most of the metrics and data points were still very internally focused on measures of participation and attendance and fall well-short of anything approximating community or civic impact. I encouraged those present to consider stepping away from a focus on the impact of an individual opera company’s contribution to civic impact, and recommended instead more of a collective impact approach in collaboration with other arts, civic, and education organizations in a community.

In this case, I even hesitated to use the word “impact,” and suggested the group consider distinguishing between collective contribution toward a modest set of civic outcomes (e.g., performing arts promote understanding of other cultures or are a source of pride for those in the community) and the more traditional causal attribution usually reserved for the term “impact.”

Nell: Caroline Fiennes, among others, has argued that individual nonprofits should actually do less evaluation and rather rely on larger research studies to prove their theories of change. What do you make of that argument and the difference between evaluation and measurement? 

Mary: I agree with some of what Caroline puts forth here – particularly her observations about “withholding (unflattering research) and publication bias” – an issue that University of Wisconsin-Madison professor Donald Moynihan has termed “performance perversity.” I also agree both with her suggestion that evaluations be done by a third-party to reduce any tendencies toward subjective reporting or bias and her endorsement of a greater consideration of shared metrics.

I am troubled, however, by the fact that only 7% of UK social-purpose organizations are interested in improving services, and her somewhat cavalier suggestion that monitoring and evaluation “wastes time and money.” Although she is not alone in this second argument (see for example Bill Shambra’s “take-down” of Charity Navigator’s efforts to encourage greater use of performance metrics in “Charity Navigator 3.0: The Empirical Empire’s Death Star?”), such sweeping generalizations undermine the legitimate and courageous attempts of many nonprofits to use data for program improvement efforts.

I agree with Phil Buchanan in that there is a “moral imperative” to make an honest attempt to understand if resources are being used effectively and certainly to guard against the possibility that programs could be doing more harm than good as organizations like Latin American Youth Center and Harlem Children’s Zone have discovered and since corrected.

I see measurement as a necessary practice for every nonprofit. But measurement is different from evaluation. Nonprofits need to start by developing a measurement infrastructure that makes sense for their organization – one that supports their mission and commitment to serve and improve the lives of their clients or constituents – not one that is reactionary and responsive to funders. It is precisely this kind of infrastructure that can lay the groundwork for a more rigorous evaluation, at a time that is right and appropriate for the organization’s stage in development.

I see measurement and evaluation along a continuum of inquiry that should be designed to support the learning objectives of an organization. Measurement helps organizations to take the day-to-day or month-to-month pulse of various activities and program results – these snapshots in time or scorecards help managers and service providers understand trends and provide an opportunity to correct, modify or otherwise adapt operations.

Evaluation is, by definition, more rigorous, more expensive, and takes considerably more time to see results. Evaluation serves a very important role as organizations make decisions about whether to continue, grow, scale or otherwise expand services, but it needs to occur at the right time – and certainly not as an organization is just getting off the ground.

Nell: It is difficult for most nonprofits to find funding for measurement work. For example, in the most recent Nonprofit Finance Fund State of the Sector survey, 69% of nonprofit respondents said their funders rarely or never cover the costs of measurement. How do we change that, or can we?

Mary: Although I am sympathetic to this argument and argue frequently that foundations have a unique and critical role to play in helping to build the capacity of nonprofits to better engage in measurement and evaluation, I think we need to change the conversation to one that focuses on the shared responsibility between nonprofits and funders for making the necessary investments in measurement and evaluation.

If nonprofits are truly ready to embrace a culture of measurement and high performance, then they need to reorganize operations in ways that embed measurement practice at every level of the organization, and change expectations from front-line workers all the way to the board of directors.

This means things like: defining expectations about data collection in job descriptions; setting aside a small percentage of funding for evaluation as a line-item in every grant request; and using data in meaningful ways in everyday discourse. Likewise, funders need to work more collaboratively with grantees to understand the data needs and capacity of nonprofits, consider funding longer-term grants that build in support for measurement and evaluation, and stop asking for data or reports that aren’t part of the conversation about continuous improvement and learning. Funders, too, can support field-building efforts to develop additional tools and resources in support of the measurement work nonprofits seek to accomplish.

There are a number of exemplary efforts already underway including Edna McConnell Clark Foundation’s Propel Next and the World Bank Group’s support of Measure4Change and the East of the River Initiative. Each of these efforts feature: targeted grants to build measurement and evaluation capacity of participating nonprofits; access to technical assistance resources; and a community of practice to help grantees learn from each other, share successes or failures, and reduce what is all too often a sense of isolation among measurement and evaluation practitioners.

Photo Credit: Urban Institute

Tags: , , , , , ,

10 Great Social Innovation Reads: September 2014

social innovationThere were some pretty exciting things happening in the world of social innovation last month. From a new fund to make philanthropy more effective, to a new blog series written by funders making the case for investing in nonprofit leadership, to some ideas for making performance measurement more accessible to small nonprofits and arts and culture organizations, to some interesting partnerships between philanthropy and city government.

It all made for a great month of reads. Below is my pick of the 10 best reads in social innovation in September. As always, add what I  missed to the comments. And if you want a longer list, follow me on Twitter, Facebook, Google+ or LinkedIn.

You can read past months’ 10 Great Social Innovation Reads lists here.

  1. The Fund for Shared Insight, a collaboration among seven major foundations, launched in September. The group plans to “pool financial and other resources to make grants to improve philanthropy…to encourage and incorporate feedback from the people we seek to help; understand the connection between feedback and better results; foster more openness between and among foundations and grantees; and share what we learn.” They plan to be very transparent with this entire experiment. I can’t wait to see what develops.

  2. Another development in the realm of improving philanthropy was the launch of the Stanford Social Innovation Review blog series where foundation leaders discuss why and how they have invested in nonprofit leadership development. As I mentioned earlier, Ira Hirschfield from the Haas Foundation kicked off the series, and Surina Khan from the Women’s Foundation of California was next up. To have such an open dialogue about nonprofit capacity investments, particularly around leadership development, is amazing. Let’s hope it encourages similar conversations outside the blogosphere.

  3. And the third piece from the world of philanthropic enlightenment, Daniel Stid of the Hewlett Foundation wrote a great post about ending the nonprofit starvation cycle. As he put it, “Effective leaders need to be willing to take the risk of saying something that a funder might not want to hear when their organization’s long run effectiveness is at stake. If they are not, then shame on them. Funders, for our part, should fund the full cost of the work we are asking our grantees to undertake in a way that leaves their overall organization and its finances whole; if we don’t, then shame on us.” Amen!

  4. There is further evidence that philanthropy as we know it is changing – a new report by The Economist takes a hard look at how Generations X and Y (those born between 1966 and 1994) are transforming philanthropy, particularly around “a strong desire to have a measurable, enduring impact.” This is exciting because if donors increasingly invest based on results, we can shift more money to social change. As the authors of the report put it, “The young generation of givers is focused on data, measurement and demonstrable results. More than any other generation, they want to check facts, know all the information ahead of time and ensure that they are well-informed at every stage of the process.”

  5. And there was lots to say about measuring performance this month. The Foundation Center and WINGS, a global network of 90 support organizations serving philanthropy in 35 countries, announced the creation of The Global Philanthropy Data Charter to gather and share philanthropy data for public benefit.

  6. Measuring impact is complex and costly, but Carly Pippin from Measuring Success, offers 4 steps for how small nonprofits can assess impact affordably.

  7. Measurement is particularly challenging in the arts and culture arena because, as Natasha Bloor of The Old Vic Theatre explains, “There is an understandable reticence within the cultural and creative industries when it comes to proving the social value of art. For many, the arts have an intrinsic worth that cannot be mapped or measured, with the primary benefit found in creative self-expression itself, rather than the longer-term effects experienced afterwards.” But she offers a new approach that they have found very effective.

  8. And for a completely free way to assess the social value of building low-cost housing, child-care centers, and health clinics there is the Social Impact Calculator, developed by the Low Income Investment Fund. They developed the tool to measure the effect of their own work and then decided to share it.

  9. Stephanie Jacobs of the Nonprofits Assistance Fund offers some tips to turn your board into the financial leaders they need to be.

  10. And finally, there were some interesting examples of partnerships between local government and philanthropy aimed at strengthening cities. Rona Jackson from Living Cities described 5 ways philanthropy and local government can work together. And the Kalamazoo Promise, a partnership between local philanthropists and city schools that pays tuition at a Michigan college for any student who graduates from a Kalamazoo school, shows these ideas in action.

Photo Credit: Valerie Everett

Tags: , , , , , , , ,

10 Great Social Innovation Reads: August 2014

social innovation readsBetween my own time away from social media in August, the general end of summer quiet, and of course, the glut of posts about the Ice Bucket challenge (of which I have already said my piece), my list of great reads in August is admittedly slim.

But there was some interesting debate, most notably about “strategic philanthropy” and about ratings agency Philanthropedia. Also, calls for more nonprofit leadership development and for nonprofit leaders to get out of their own way by taking the Overhead Pledge. Throw in a little Mark Twain, some sharing economy, and a dash of Millennial analysis and you have a pretty good month in the world of social change.

So below is my pick of the 10 best reads in the world of social innovation in August. For an expanded list you can follow me on TwitterFacebookLinkedIn or Google+. And you can see past months’ 10 Great Reads lists here.

  1. Leadership development is a woefully underfunded need in the nonprofit sector. Indeed from 1992-2011 only $3.5 billion of the nearly $287 billion dollars granted by foundations went to support leadership. In order to get more foundations investing in leadership development, Rusty Stahl offers case studies of 9 foundations who already do.

  2. In the summer issue of Stanford Social Innovation Review, the lead article “Strategic Philanthropy for a Complex World” caused quite a stir in the philanthropy world with many arguing that there is not much new there. In August, Alliance Magazine ran a series of editorials by philanthropy leaders as counterpoints. Most interesting among them was Avila Kilmurray’s, former director of the Community Foundation of Northern Ireland, response, in which she said “Can we not just recognize that when any funder sets her/himself the task of addressing complex issues…there needs to be provision for continuous consultation, practice, reflection and change?”

  3. An interesting article in the New York Times paints the Millennial generation as a very communal-minded one, where “the highest value isn’t self-promotion, but its opposite, empathy — an open-minded and -hearted connection to others.” From working, to eating, to shopping it seems Millennials bake social into everything they do. How will the world be different if that holds true as they age?

  4. Writing in Forbes, Tom Watson asks whether nonprofits should participate in GivingTuesday. As he puts it, “Is #GivingTuesday a well-meaning marketing promotion – or is it a real, organic movement for change?…[Does it] seek to increase U.S. giving from 2% of GDP (where it’s been stuck for two generations) to some higher point?” Amen to that!

  5. Rating nonprofit effectiveness is such a tricky challenge. Philanthropedia, one rating system that is driven by crowdsourced feedback from experts, comes under fire from the clean water space for being just “a popularity contest.” But others claim it’s an improvement over previous evaluations.

  6. Writing in the Chronicle of Philanthropy Nicole Wallace shows the value of sharing data by profiling Crisis Text Line, which gives other nonprofits, researchers and government agencies access to their data of 60,000 counseling sessions with teens in crisis to use in their own programs. It begs the question whether other social change data could be shared and how we make that easier to do.

  7. Sue Dorsey from Water for People was among a group of nonprofit leaders at the InsideNGO conference who took the Overhead Pledge in August, vowing to fully disclose the true costs of their nonprofits. And she encourages other nonprofit leaders to follow suit. This is exciting because it is not enough for funders to get over the overhead myth, nonprofit leaders must as well.

  8. I am always a sucker for connecting literature and/or history to social change, and even better both, so David Bonbright’s post about how Mark Twain would have viewed recent trends in business is fascinating. Bonbright argues that Twain wanted American business to fully integrate profit and community. And we are beginning to witness this trend again where companies are “embracing the full implications of what they are – what they mean for the environment, for communities, for the most marginalized people affected by their supply chains…[because] this is best way to remain competitive and successful over time.” Let’s hope!

  9. The new “sharing” economy is not all good, but not all bad either, as Daniel Ben-Horin argues that “there are enormous opportunities for the social sector to engage with the values-driven segment of the sharing economy.”

  10. Finally, some guidance on making your nonprofit email marketing more mobile friendly and your website better able to connect people to your cause. It’s all about responsive, engaging design.

Photo Credit: Seth Anderson

Tags: , , , , , , , , , ,

Guest Post: Intellectual Curiosity is the Cornerstone of Social Progress

NotCuriositye: Third in my list of guest bloggers this summer is David Henderson. David’s professional focus is on improving the way social sector organizations use information to address poverty. Here is his guest post:

I was recently turned down for a position at a startup-up big-data company focused on the philanthropic sector because I’m “too pessimistic”. This company initially sought me out since they don’t have any social sector expertise on staff, a likely requisite to make successful nonprofit software. Our courtship turned sour when I expressed my view that we have a lot more social sector initiatives than evidence that those interventions actually work.

My skepticism that social sector initiatives by and large work was wrongly misconstrued as pessimism that social progress is possible. Skepticism is a critical driver of intellectual curiosity. I spent a pretty penny on two degrees that essentially taught me how to critically assess the divide between rhetoric and results. Indeed, the null hypothesis in a statistical model assumes the intended effect is not present. I guess statisticians are just a bunch of pessimists.

Fundamentally, I believe the company I interviewed with was mirroring the widespread lack of intellectual curiosity that plagues the social sector and impedes real progress. Too many nonprofits are terrified of having their claims of social impact investigated, lest their effects are discovered to be more modest than claimed. And I don’t blame them. The funding community’s emphasis on investing in “what works” has resulted in a proliferation of noise as every nonprofit steadfastly argues their interventions cure everything. It’s no wonder evaluators are seen as Angels of Death.

I generally don’t favor taking cues from the for-profit world, but venture capital and angel investors’ practice of investing in people and teams over ideas is far more conducive to intellectual honesty in product (and social intervention) development. The basic premise of this investment strategy is that initial product ideas are generally wrong, but smart people will investigate, iterate, and innovate.

Compare that philosophy to the social sector, where the expectation is that nonprofits already have the answers, they just need money to scale them up. This assumption is largely incorrect, but by making funding contingent on the perception of effectiveness, the nonprofit sector is incentivized to not question the efficacy of its own work. In this model, continued funding depends on a lack of intellectual curiosity at best, and intellectual dishonesty at worst.

A better alternative is for nonprofits to embrace intellectual curiosity, and to be the first to question their own results. Under this model, nonprofits would invest in their capacity to intelligently probe the effectiveness of their own interventions, by staffing those with the capacity to sift through outcomes data and investing in the growing list of tools that are democratizing evaluation. Of course, this would require a shift in the funding community away from “investing in what works” to more humbly “investigating what works”.

A shift toward intellectual curiosity would create more space for the sector to solicit beneficiary feedback in the design of social interventions, as organizations would no longer be incentivized to defensively “prove” existing approaches work, and instead would be rewarded for proactively evolving practices to achieve better results. It is this very intellectual curiosity that led organizations like GiveDirectly and the Family Independence Initiative to invest in the poor directly, a departure from long-standing anti-poverty practices that the evidence suggests might actually work. It’s a shame that organizations imbued with a mission of experimentation deviate so far from the norm.

I don’t consider it pessimistic to question whether the sector is achieving its intended social impact. To the contrary, it’s rather cynical to set aside what should be the critical question for any nonprofit organization in the name of self-preservation. In order to achieve social progress, the sector needs to expel anti-intellectual policies and actors in favor of a healthy skepticism that questions everything, and is willing to try anything.

Photo Credit: NASA

Tags: , , , , ,

The Power of a Theory of Change

Last week I spoke at the Women’s Collective Giving Network (WCGN) national conference about the power of a theory of change. The WCGN is an affiliation of women’s philanthropic groups across the country. The members pool their money and give grants to nonprofit organizations. It is critically important that they, along with every nonprofit organization, understand and embrace the power of a theory of change.

With increasing competition for social change dollars it is absolutely crucial that nonprofit organizations develop their own theory of change. A theory of change is basically an argument for why a nonprofit organization exists. It describes how an organization uses community resources (money, volunteers, clients) to perform a set of activities which result in changes to the clients’ lives (outcomes) and changes to broader communities, institutions, or systems (impact).

Essentially a theory of change describes how a nonprofit creates social change.

So a very basic theory of change for a literacy program would look like this:

 

It used to be enough for a nonprofit to talk about what they produced (or outputs), such as meals served in a soup kitchen, hours spent reading to a child, beds provided in a homeless shelter, but that just doesn’t cut it anymore. In a world where there are fewer and fewer dollars and more and more nonprofits fighting for those dollars, philanthropists, government funders and others are increasingly asking the question “To What End?” So what if you created outputs, did anything really change because of your work? Did the lives of those in your program change and did our community change?

That’s where a theory of change comes in. If you can articulate what change you hope your organization is creating, then with that fundamental building block in place you can:

  • Chart a strategic direction
  • Prove your results
  • Secure more support for your organization

And ultimately achieve the holy grail of the nonprofit sector: sustainable community change.

A theory of change is the fundamental building block that makes all of this happen, like this:

A theory of change is so fundamental because you cannot chart a strategic direction if you don’t know what you are trying to change. And you can’t prove that you’ve changed something unless you have articulated what it is that you want to change. And you certainly can’t get funders, volunteers, and key decision makers to support you if you can’t tell them what you are trying to change and whether you are actually doing it. So the only way to truly create long-term social change is to start with a theory of change. Which is why I encourage every nonprofit organization to go through the exercise.

If you need help crafting your own theory of change, check out our Creating a Theory of Change step-by-step guide, or if you need customized help to chart a strategic direction, check out our Strategic Planning consulting services.

Photo Credit: Kristymp

 

Tags: , , , , , , , , , , ,


Share




Popular Posts


Search the Social Velocity Blog