Not
e: Third in my list of guest bloggers this summer is David Henderson. David’s professional focus is on improving the way social sector organizations use information to address poverty. Here is his guest post:
I was recently turned down for a position at a startup-up big-data company focused on the philanthropic sector because I’m “too pessimistic”. This company initially sought me out since they don’t have any social sector expertise on staff, a likely requisite to make successful nonprofit software. Our courtship turned sour when I expressed my view that we have a lot more social sector initiatives than evidence that those interventions actually work.
My skepticism that social sector initiatives by and large work was wrongly misconstrued as pessimism that social progress is possible. Skepticism is a critical driver of intellectual curiosity. I spent a pretty penny on two degrees that essentially taught me how to critically assess the divide between rhetoric and results. Indeed, the null hypothesis in a statistical model assumes the intended effect is not present. I guess statisticians are just a bunch of pessimists.
Fundamentally, I believe the company I interviewed with was mirroring the widespread lack of intellectual curiosity that plagues the social sector and impedes real progress. Too many nonprofits are terrified of having their claims of social impact investigated, lest their effects are discovered to be more modest than claimed. And I don’t blame them. The funding community’s emphasis on investing in “what works” has resulted in a proliferation of noise as every nonprofit steadfastly argues their interventions cure everything. It’s no wonder evaluators are seen as Angels of Death.
I generally don’t favor taking cues from the for-profit world, but venture capital and angel investors’ practice of investing in people and teams over ideas is far more conducive to intellectual honesty in product (and social intervention) development. The basic premise of this investment strategy is that initial product ideas are generally wrong, but smart people will investigate, iterate, and innovate.
Compare that philosophy to the social sector, where the expectation is that nonprofits already have the answers, they just need money to scale them up. This assumption is largely incorrect, but by making funding contingent on the perception of effectiveness, the nonprofit sector is incentivized to not question the efficacy of its own work. In this model, continued funding depends on a lack of intellectual curiosity at best, and intellectual dishonesty at worst.
A better alternative is for nonprofits to embrace intellectual curiosity, and to be the first to question their own results. Under this model, nonprofits would invest in their capacity to intelligently probe the effectiveness of their own interventions, by staffing those with the capacity to sift through outcomes data and investing in the growing list of tools that are democratizing evaluation. Of course, this would require a shift in the funding community away from “investing in what works” to more humbly “investigating what works”.
A shift toward intellectual curiosity would create more space for the sector to solicit beneficiary feedback in the design of social interventions, as organizations would no longer be incentivized to defensively “prove” existing approaches work, and instead would be rewarded for proactively evolving practices to achieve better results. It is this very intellectual curiosity that led organizations like GiveDirectly and the Family Independence Initiative to invest in the poor directly, a departure from long-standing anti-poverty practices that the evidence suggests might actually work. It’s a shame that organizations imbued with a mission of experimentation deviate so far from the norm.
I don’t consider it pessimistic to question whether the sector is achieving its intended social impact. To the contrary, it’s rather cynical to set aside what should be the critical question for any nonprofit organization in the name of self-preservation. In order to achieve social progress, the sector needs to expel anti-intellectual policies and actors in favor of a healthy skepticism that questions everything, and is willing to try anything.
Photo Credit: NASA
I couldn’t agree more and have said this many times (including on this very blog): Critical thinking (or skepticism) isn’t the same as “pessimism.” It’s essential to driving innovation, working more strategically, and having long-term impact — all those things that are championed as important. It’s ironic that some of the same people who promote data- and evidence-based intervention and strategy are sometimes either unaware of or eschew a fundamental component of rigorous research methodology, which you nicely point out: That the null hypothesis assumes the intended effect isn’t present. Of course, that assumes that there’s an interest in conducting a rigorous study in the first place. I once heard a senior foundation official say that there was no need to use academics for research when all they had to do was call up a consulting firm and have them do a quick survey for them. It was then that I realized why it’s important for foundations to have people on staff who have some exposure to social science research (with all its flaws, it’s still the best we’ve got at this point) and an understanding that most of the problems nonprofits/foundations are trying to resolve are pretty complex and, thus, the methods used to evaluate them are going to be equally so. That’s not pessimistic; it’s smart thinking, which we need more, not less, of.
Thanks for the note Cynthia, I too think a base level of social science background of all sector staff would go a long way to propelling our collective work forward. When I first started working in the social sector I was surprised to discover that social scientific thinking was not especially prevalent throughout philanthropy. To me that seems like a kitchen full of people without culinary training.
Of course, the bigger issue as you point is the lack of interest (and in your anecdote total disregard) for social scientific inquiry. As all things in life, a move toward social scientific inquiry will have to be driven by capital. But if the deployers of capital themselves, such as the senior foundation official you cite, don’t see the value in such inquiry, it’s no wonder nonprofits don’t think critically about their own work.
Interesting topic. I think that non-profits often get stuck in a cycle. Donators have invested in a specific idea, and are expecting that idea to happen. Non-profits also receive a lot of criticism on overhead costs and staffing, so changing up a program, especially one that invests more in staff, can be a huge risk for the organisation.
However, I completely agree with you. I think if we want to see non-profits really making a difference and adapting quickly to various needs, we need to invest more in the people and the overhead, to see a greater impact in the field. Oh, and not be afraid to take a little risk :).