Comments on: Nonprofits Publishing Impact Analysis

By: Holden Karnofsky

Holden Karnofsky — Thu, 07 Feb 2008 05:34:41 +0000

I think these nonprofit websites are standouts in terms of empirical evidence of impact:

Teach for America - links to rigorous Mathematica study

Nurse Family Partnership - links to several academic studies on the program’s effects

The HOPE Program - provides thorough in-house data on client outcomes

All of these websites provide qualitative/emotional information as well, though I’ve linked to the parts that focus on empirical evaluation.

Holden Karnofsky
http://www.givewell.net

By: Sean Stannard-Stockton

Sean Stannard-Stockton — Tue, 05 Feb 2008 05:19:12 +0000

Paul, thanks for your input and for joining the conversation.

Kate,
I’m not an expert in nonprofit analysis. But my take is that being on the right track has less to do with which particular reports you post and more to do with if after reviewing your material can a potential donor express in their own words what your organization attempts to achieve, why your approach is effective, why your approach is superior or at least differentiated from other organizations in the space and cite at least a couple of examples of how your approach has played out historically.

As an investor in the stock market I can answer that set of questions to make a case for why Costco is a better investment than Target (or vice versa). They have differentiated business models and different issues that make them more or less compelling to certain investors with certain outlooks.

(Nothing in this comment should be taken as investment advice)

By: Kate Kromann

Kate Kromann — Mon, 04 Feb 2008 14:41:14 +0000

Great question. We’ve been trying to publish our impact more obviously, concretely and understandably. While we have our annual report posted under “Who we are,” we’ve also added sidebars on the left side of our Web page to give visitors concrete examples of how their dollars help.

What do you think? Is this a good effort, on the right track or still not quite there?

By: Paul Verrette

Paul Verrette — Fri, 01 Feb 2008 14:39:03 +0000

In the more than 300 nonprofits I have examined at the Charities Review Council in Minnesota, what has really jumped out at me is not the quality of the various evaluation models or the elegance of someone’s theory of change. Where I have seen nonprofits of all sizes excel is straightforward accounting of whether or not both outputs and long-term outcomes are being met or not met. In other words, I think the best organizations have public materials that you can see advances or impediments from year to year. You can read their annual reports and web sites like tree rings to see the how they did in various conditions. I believe the open accounting of success and challenge is what builds the long-term committment of donors, volunteers, boards, and staff alike. I am not sure that I have seen much difference from one part of the sector to the other. I will say that many advocacy organizations are working very hard at publicly demonstrating effectiveness and doing some promising work. I think that the National Committee for Responsive Philanthropy may be looking at the ways that advocacy organizations measure their work. Another evaluation project that is very promising is the Pennsylvania Cultural Data Project, this is a Pew Foundation effort looking at the benefit of arts organizations.

By: Heather McLeod Grant

Heather McLeod Grant — Thu, 31 Jan 2008 07:02:47 +0000

This is a very interesting question, and I’m eager to see what others have to say. Of the 12 organizations we studied for our book, Forces for Good, some were quite good about publishing accounts of their impact in annual reports, or in obvious places on their website (About Us, etc.). For others, we often had to dig for it - it was buried on websites, etc., or only available on internal documents. Our research was done a few years ago, so I can’t comment on what’s available now, or how accessible.

A quick glance at the websites of groups like America’s Second Harvest and Environmental Defense quickly reveals that even GREAT organizations don’t have this information pulled together in one place and easily accessible under a simple heading like “Impact”. Their messages/ navigation tend to focus on what they do (programs), their approach, values, who they are, etc…how to get involved. So unless you dig for the annual report, download it, read it - hard to find. I think most nonprofits could do a better job of talking about what they’ve accomplished and their results in a more concise and easy-to-access manner, almost like “facts at a glance”. I’m hoping this debate will encourage them in this direction…

Of course, there are still all the questions and challenges surrounding “impact” - how to measure (which methodology, formative v. summative evaluation, etc.)…relativism (what you measure field to field is different)…time frame (social change takes a long time to accomplish)…but that’s a whole other conversation!

By: Tom Durso

Tom Durso — Thu, 31 Jan 2008 02:25:38 +0000

Come on, Sean, are you going to keep us in suspense like that? Who’s asking you?

Seriously, it’s an interesting query, given the current debate over metrics. I’ll be interested in seeing what you send them.

By: Tidy Sum

Tidy Sum — Wed, 30 Jan 2008 17:56:10 +0000

Great question. My guess is that some sectors are better than others.

Environmental advocacy organizations, for example, that produce the usual newsletter, annual report, and press releases do a great job of documenting how their work influenced public policy.

My in-box today has thoughtful case narratives from Oceana, World Wildlife Fund, E-Law, and EarthRights.

They all document strategic objectives, tactics, challenges, partners, inputs, and outputs.

Most importantly, they celebrate capacity building, leadership development, and concrete policy changes.

I suppose if I had money to burn I could invest in a more boring and technocratic analysis of their specific interventions, but my hunch is that the face validity of their public communications is very high and provide a more accessible and useful resource for donors and other advocacy organizations.

Judging by their growing lists of supporters, I am probably not alone.