In today’s Social Velocity interview, I’m talking with Isaac Castillo, Director of Outcomes, Assessment, and Learning at Venture Philanthropy Partners, where he leads VPP’s approach to data collection, data reporting, and outcome measurement.
Prior to coming to VPP, Isaac served as the Deputy Director for the DC Promise Neighborhood Initiative (DCPNI). At DCPNI, Isaac led efforts to improve outcomes in the Kenilworth-Parkside community in Ward 7 of the District of Columbia through the strategic coordination of programmatic solutions and research-based strategies. Prior to his time at DCPNI, Isaac served as a Senior Research Scientist at Child Trends where he worked with nonprofits throughout the United States on the development and modification of performance management systems and evaluation designs. In addition, Isaac was also the Director of Learning and Evaluation for the Latin American Youth Center (LAYC) where he led the organization’s evaluation and performance management work.
You can read interviews with other social change leaders here.
Nell: You have spent your career using data to improve the performance of the nonprofits for which you worked. Why do you think performance management is so important for nonprofits? Do you think all nonprofits should pursue performance management? When does it make sense and when doesn’t it?
Isaac: I believe that every nonprofit should pursue some form of performance management because they owe it to the clients they serve. Most nonprofits will assume that they are making a positive difference in people’s lives, but in the vast majority of cases they are just guessing. Using some form of performance management will allow every nonprofit organization to confirm this thinking and to identify areas that can and should be improved so that the next cohort of participants can get better services than the last.
Unfortunately, one of the greatest challenges preventing a nonprofit from implementing some form of performance management isn’t a lack of resources, expertise, or time. It is fear. The fear that they will find out that their work isn’t having a positive effect. This fear is what nonprofit leaders need to overcome, not for the benefit of themselves or their organization, but because they owe it to the clients they serve today and the clients they will serve in the future. I believe that every nonprofit should strive to serve tomorrow’s clients better than today’s clients, and one of the only ways to ensure that this happens is the sustained use of performance management.
The type of performance management that each nonprofit should pursue should vary by the size and scope of their work. At a minimum, small nonprofits should be tracking basic demographic and attendance information on their participants, and hopefully at least one meaningful output or outcome. Whether this occurs in a computerized system or in a spiral paper notebook is up to the nonprofit. But it doesn’t have to be costly, and it doesn’t take expertise. It only takes the will and desire to improve as a nonprofit.
Nell: In the nonprofits in which you’ve worked how have you been able to secure resources to fund performance management? What is the case you and your colleagues have made to funders and what do you think it will take to get more funders investing in performance management?
Isaac: Raising funding for performance management work usually takes a mix of several different strategies and approaches for potential and existing funders.
First, I strongly encourage nonprofits to include some percentage (1 to 5 percent – possibly more) of funding in each grant submission or proposal dedicated to supporting performance management and outcome measurement work. By placing this small percentage into each proposal, a nonprofit can begin to raise funds for internal evaluation and performance management activities. It may not seem like a lot, but it can add up, and eventually generate enough funds for a half-time or full-time position to support in-house performance management work.
Second, I also strongly encourage nonprofits to engage in regular ‘funder education’ – where a nonprofit proactively meets with their funders to have ongoing conversations about outcome measurement and evaluation. This allows both the funder and the nonprofit to come to agreement on measurement expectations and to ensure that both groups are focused on the same concepts. I often suggest that the first of these types of meetings focuses on each group’s definitions of three commonly misunderstood terms: outputs, outcomes, and impact.
Finally, I would recommend that the nonprofit and funder have an honest discussion regarding expectations of results and the funding necessary to support the related evaluation work. If a funder is expecting an random control trial (RCT) to be completed to determine ‘impact,’ then the nonprofit should be willing to push the funder to support a large investment to pay for a high quality evaluation. If the funder is only willing to support a small amount for outcome measurement, then the nonprofit should clearly articulate what is possible.
Nell: Ken Berger and Caroline Fiennes recently argued that we may have gone too far by asking nonprofits to produce research about their own outcomes. What’s your response to that argument?
Isaac: I fully support Ken and Caroline in their argument that most nonprofits should stay away from trying to produce impact research. The desire for ‘impact’ is something that has been (and continues to be) pushed unfairly (and without financial support) by the funding community.
I honestly think a lot of confusion in this space comes from inconsistent use and understanding of the term ‘impact’. The term ‘impact’ has a precise definition among researchers but is often used in a much broader context among funders, nonprofits, and the general public. In the research and evaluation world, impact is used to describe the effectiveness of a program while eliminating as many potential confounding factors as possible. That is why the use of random control trials (RCTs) is usually the cornerstone of impact research – RCTs are the easiest way to control for and eliminate confounding factors.
When most non-researchers use the term ‘impact’ however, they are usually just asking if the program or organization works and if it is making a difference for its intended service population. That is a much lower bar to set, and yet it is a critical distinction in this discussion. If you are thinking about ‘impact’ as a researcher, you will need a large amount of resources and expertise to determine ‘impact,’ which usually means completing one or more formal evaluations. If you are thinking about ‘impact’ in the more general sense and less strict way, then pursuing some form of performance management system will allow a nonprofit to determine if their efforts have been successful.
I do think every nonprofit should pursue some form of performance management to ensure that their work is having a positive effect as a complement to existing research that others have done. Relying only on the use of others’ research does not guarantee that a nonprofit will provide effective services and achieve positive outcomes. This type of research is a like a recipe – it shows what has worked in the past and provides a guide for the nonprofit – but a recipe can still be ruined with poor implementation or planning.
Every nonprofit has an obligation to the people they serve (and not to their funders) to ensure that their programming is having a positive effect (or at the very least not causing harm). Without some form of performance management system in place (even one that just uses paper and pencil), a nonprofit will never know if they have strayed too far from the recipe provided by previous research.
I also think there are a growing number of very sophisticated nonprofits that should be using AND producing research on effective programs. Every year, I see more and more nonprofits that hire talented and unbiased researchers dedicated to internal evaluation and outcome measurement work. These individuals are just as talented and unbiased as their colleagues working in traditional research and evaluation organizations. They can, and should, produce original research that can help inform the nonprofit field. The real challenge comes in nonprofit organizations finding the resources to support the hiring and retention of these individuals. Not every nonprofit will have the resources or capacity to hire one or more of these individuals – but those that do should absolutely be trying to produce original outcome and impact research to provide ‘recipes’ for effective programming that nonprofits with fewer resources can use in the future.
Nell: Your former organization, DC Promise Neighborhoods, is part of the national Promise Neighborhoods Initiative launched by the US Department of Education in 2010 and modeled after the famous Harlem Children’s Zone. How successful has this national replication of a successful local model been? Have you been able to replicate outcomes? And what hurdles, if any, have you and other replication sites found?
Isaac: I think that there has been some initial success among the Promise Neighborhoods. Part of the challenge that all the Promise Neighborhoods face is that the Harlem Children’s Zone did not achieve their success overnight. They have been working in Harlem for decades, so it would be unrealistic to believe that the Promise Neighborhoods would be able to create large scale change in a matter of a few years.
However, there are signs of progress across all of the Promise Neighborhoods. Each of the Promise Neighborhoods started to address a few outcomes with the initial round of funding, and these outcomes varied. Some focused on math and reading proficiency for students, some focused on obtaining medical homes for young children, and others sought to increase the amount of healthy food consumed by residents. In DC, we focused on improving school attendance.
I do think that most of the 12 Promise Neighborhood Implementation grantees were able to make progress on the outcomes they identified as initial focus areas. However, the very nature of the work (creating community level change) doesn’t lend itself to the rapid accomplishment of multiple outcomes in a short period of time. Each of the Promise Neighborhoods had to prioritize certain outcomes for their respective communities, and only several years later are they able to claim success and begin to identify the next set of outcomes to be addressed. So while certain outcomes haven’t necessarily been replicated across all the Promise Neighborhoods, that is due to the differences in priorities and community conditions rather than any problem with the model itself.
Photo Credit: Venture Philanthropy Partners
April was another busy month in the world of social change writing. From Google’s shift to mobile, to the Baltimore protests, to using sitcoms to change public opinion, to the pace of social change, to teens and social media, to a new way to measure a country’s performance, there was much to read and digest.
Below are my 10 picks of the best in the world of social change in April, but please add to the list in the comments. And to see what else I found beyond these 10, follow me on Twitter, Facebook, Google+, or LinkedIn.
And you can read past months’ 10 Great Reads lists here.
- There was much analysis about what went wrong in Baltimore, but I found the most insightful to be Dan Diamond’s Forbes piece about how it is fundamentally a “tale of two cities” and the persistent inequality between two very different Baltimores.
- As is Google’s way, they made a huge change to their search algorithm in late April that will affect us all. Google is now favoring websites that are mobile friendly. But fear not, Beth Kanter offers some advice for upgrading your nonprofit’s website.
- For those in the trenches, the pace of social change can seem glacial. But this great graphic from Bloomberg demonstrates that for many issues (prohibition, interracial marriage, women’s suffrage, same-sex marriage) there was a tipping point at which America very quickly changed its mind. Fascinating.
- Civic Tech, or using technology to make citizens more engaged and government more effective, is a huge investment opportunity, says Stacy Donohue from the Omidyar Network. With venture capitalists, the federal government and nonprofit and for-profit solutions all poised to make change, Donohue sees civic tech as a “very real, very now investment opportunity.” Let’s hope that new ideas and (most importantly) lots of new money can turn our struggling democracy around.
- Social change can happen in many different ways, including by altering popular culture. Former Daily Show correspondent Aasif Mandvi is attempting this kind of shift with his new web sitcom that takes a “Cosby Show” approach to portraying American Muslims in order to combat Islamophobia.
- Writing in Slate, Krista Langlois takes a hard look at her fellow environmental journalists and whether they have failed to adequately describe the environmental challenges facing our planet since American concern about climate change has actually declined in the last 20 years.
- One of the most common hurdles to nonprofits raising capacity dollars is the challenge of articulating to funders the potential impact of a capacity investment. Grantmakers for Effective Organizations (GEO) have put together some tools to help funders understand the importance of and return on capacity investments. Share these with your funders.
- In April, MIT and the Social Progress Imperative launched the Social Progress Index, an effort to create a complement to the Gross Domestic Product that measures a nation’s social and environmental performance. The Social Progress Index looks at 52 indicators of a country’s social and environmental performance (like child mortality rate, adult literacy rate, greenhouse gas emissions). As Michael Porter, one of the chief architects behind it puts it, “Measuring social progress offers citizens and leaders a more complete picture of how their country is developing. And that will help societies make better choices, create stronger communities, and enable people to lead more fulfilling lives.”
- Writing on the Huffington Post Politics blog, Robert Reich describes a worrying trend where nonprofits are silencing themselves for fear of losing their big donors. As he writes, “Our democracy is directly threatened when the rich buy off politicians. But no less dangerous is the quieter and more insidious buy-off of institutions democracy depends on to research, investigate, expose, and mobilize action against what is occurring.”
- And finally, if you want to understand where social media is going, Pew Research Center released their most recent findings about teens use of social media and technology.
Photo Credit: Patrick Neil
I have to be honest. I am so sick of hearing about the ice bucket challenge that I am loathe to write about it. But I wonder if many in the nonprofit and philanthropic sector are falling victim, yet again, to shiny object syndrome, so I feel compelled to say something.
To me the ice bucket challenge is yet another example of what happens so often in the world of fundraising. Nonprofit board members and staff hate fundraising, so they desperately search for a magic bullet to make it all go away.
Sometimes that magic bullet is “an endowment,” sometimes its “earned income,” more recently it has been “crowdfunding.” This month it’s a form of crowdfunding taken to the extreme, the ice bucket challenge. Some have been so swept up in the hype that they have gone as far to say that the challenge is “rewriting the charity model.”
The reality is that if you want to create social change you need to develop a sustainable financial model that aligns with your long-term goals. It’s not sexy, it’s not easy, and I’m probably one of the few people on this planet who thinks it’s fun. But there it is.
While many nonprofits are scrambling to figure out how to create their own ice bucket challenge, and some thought leaders are offering tips along the way, maybe we should all just take a step back.
Let’s be very clear. ALS’s close to $100 million windfall is not a revenue stream. It is a one-time infusion of money. Yes, ALS may try to replicate the ice bucket challenge on a regular basis, but the stars will never align in quite the same way, people will move on to the next shiny object, and the money will eventually fade.
Because this pile of money is not a revenue stream, ALS can’t and shouldn’t add long-term staffing or programming because the money won’t be there next year. At the same time, they probably can’t create an endowment because the donors’ intent was not for the money to sit in a bank account. Regranting the money is also tricky, again because donor intent was for it to go specifically to ALS. In all of this ALS will be under the microscope, because as Ken Berger of CharityNavigator cautioned, a year from now everyone will be asking where the money went.
One of the few paths that I see for ALS is to treat the money like capacity capital. This could be an opportunity to invest some of the windfall in building a stronger organization by investing in technology, infrastructure, and systems. And they could do the same for their affiliates. They could require capacity building plans and budgets and invest in those plans accordingly. They could, in essence, create a $100 million capacity capital investment fund for the ALS system.
But the point is that far from being a great thing that all nonprofits should strive to emulate, the ice bucket challenge creates a complex and potentially damaging problem.
So instead of spending board and staff time trying to dream up the next ice bucket challenge, please, please, please spend that time and those resources building your financial model, by creating a long-term financial strategy, raising capacity capital to build your revenue-generating function, developing a compelling strategic plan in which people will want to invest, and growing and educating your board.
These are the ingredients for a robust, sustainable financial model. Not a bucket of water, a video camera, and a social media stream.
Photo Credit: StoiKNA
A big topic of conversation lately has been whether donors really care about impact, or whether they simply just give based on less scientific things like their emotions, or their friends recommendations. Which is why I’m excited to announce that I’ll be participating in a Google Hangout April 30th about using data to attract donors.
Writing in the Stanford Social Innovation Review, Tim Ogden claims that donors have never really been interested in impact. And Ken Berger from Charity Navigator and William Schambra of the Hudson Institute debate (here and here) whether moving the nonprofit sector toward performance management helps or hurts social change efforts.
To add to this conversation, David Henderson and I are hosting a Google Hangout, “How to Use Real Performance Data to Raise More Money,” on Tuesday, April 30th at 2pm Eastern. David is a super smart guy who runs Idealistics, a consultancy that helps nonprofits learn from their outcomes data, increase impact, and demonstrate results to funders and stakeholders. David’s professional focus is on improving the way social sector organizations use information to implement higher impact poverty interventions. He has been quoted in the Chronicle of Philanthropy and has written for Change.org and the Huffington Post. You can read my interview with him from a year and a half ago here.
David and I thought it would be interesting to host a conversation with nonprofit leaders about how nonprofits can use real performance data to raise more money. We’ll kick off the hour-long conversation with a couple of points and a case study or two of nonprofits that are using data to raise more money, but then we’ll open it up to you for questions. You can send us your questions ahead of time (via email to email@example.com or firstname.lastname@example.org) or simply post them to the Google Hangout here as you watch.
I hope you’ll join us!
How to Use Real Performance Data to Raise More Money
A Google Hangout with David Henderson and Nell Edgington
Tuesday, April 30th, 2013
Can nonprofits that use real performance data to raise more money? Are donor increasingly interested in impact data? How can nonprofits communicate their program data to donors? And how should nonprofits respond to questionable performance claims by other organizations? Join David Henderson from Idealistics and Nell Edgington from Social Velocity in a Google Hangout on Tuesday, April 30th at 2pm Eastern to discuss these and many more questions about how nonprofits can use real data to raise more money. We’d love to have you participate in the discussion, so send your questions ahead of time to Nell or David, or leave a comment at the Google Hangout here.
Photo Credit: 401(K) 2013
Day 2 of SoCap was by far my favorite. It started with an interesting keynote from Julie Sunderland of the Gates Foundation. She offered a perhaps more realistic, bordering on the pessimistic, view of the social capital market space. She said that Gates struggles to find entities that can absorb the size investments they want to make. They get excited about the idea of bringing together foundation, government and private dollars in stacked deals, but that the work is complicated and hard and they have yet to craft one of these deals simply because it is extremely difficult to determine the terms. All of this underlines what I’ve said in a previous post: in the nonprofit, philanthropic and government worlds there is still much work to be done to unlock capital.
The first session of the day for me was “Lessons of Behavioral Finance: Understanding and Overcoming Barriers to Impact Investing” with Hope Neighbor and her ground-breaking research, Money for Good, released earlier this year calculating a $120 billion pool of potential impact investing money that is sitting on the sidelines. Hope said that despite our desires to the contrary, people still very much think of their charitable giving as separate from their impact investing, “the reality is that people compartmentalize their money.” And only 3% of the population uses data to compare the organizations they give to.
My favorite session of the day, by far, was “Deep Dive Into the Evergreen Cooperative Initiative.” This session was exactly what I was hoping to see more of at SoCap this year. A group of leaders in Cleveland realized that the heart of their city was quickly deteriorating and no one was doing anything about it. They formed a coalition of the anchor institutions in Cleveland (Case Western Reserve University, Cleveland Clinic, etc), foundations, city leaders and others to create the Evergreen Cooperatives that brings career-track jobs and green, employee-owned businesses to the inner city, transforming a city that has lost 50% of its population in the last 50 years. Beyond the fascinating coalition, business model and results this project is achieving, lies its impressive financing. A combination of bonds, foundation grants, loans, HUD money and others launched this project and financed the 3 businesses they currently operate (a green laundry, an organic greenhouse, and a solar power company). According to Evergreen leaders, “Cleveland wants to be where the world is going, not where the world is.”
To scale this project to create 5,000 jobs (the area needs 46,000 jobs), which will be the impetus to truly transform the inner city economy, they are creating a CDFI and looking to use PRIs and MRIs. What excites me so much about this project is not the spirit of collaboration and tremendous results, but how they are bringing public, private and philanthropic money together in a truly innovative convergence. THIS is the kind of social capital market I’m talking about. Impact investing is great, but it is only ONE piece of the puzzle. I would love to see more examples like Evergreen at SoCap.
The last breakout session I attended for the day was “Nonprofit Analysis: Beyond Metrics,” which gave a great overview of the growing nonprofit evaluators market through the lens of rating one nonprofit, DC Central Kitchen. It was interesting to see how Charity Navigator, the most well-known nonprofit evaluator, has evolved from a system driven purely by IRS 990 form overhead ratios to a three-pronged review including transparency and impact evaluations.
The end of the session gave me serious pause, however, when a member of the audience asked whether any of the evaluators might use the GIIRS system coming out of the impact investing world to rate nonprofit impact. Ken Berger admitted he wasn’t familiar with GIIRS and Tim Ogden of GiveWell said he was skeptical of social return on investment (SROI) calculations in general. Again, my point that the philanthropic and impact investing worlds aren’t communicating and collaborating becomes apparent. Wouldn’t that be amazing if impact in both the philanthropic and impact investing worlds could be measured in a comparable way? That would be truly innovative!
So, although Day 2 of SoCap provided much more conversation and examples of how the philanthropic and government capital markets are evolving, there is still much work to be done to bring both capital fully into the social capital market. Perhaps at SoCap 2011?
Photo Credit: Markets for Good