What to Measure and Why in Philanthropy

Part Two

I hope her comments help simulate the nonprofit industry to develop one or more solid measures and to include a nonprofit’s questions, clarifications, and comments in the process.

When better and more reliable qualitative and quantitative analysis is available — and not before — the industry may be prepared for “ratings” and “top ten lists.”

Others reactions?

Any useful analysis of organizational performance requires both a qualitative and quantitative component. One possible approach to the nonprofit evaluation challenge might be to marry the quantitative analyses with various sources of qualitative detail, such as permitting the rated organization to comment and explain its own ratings, integrating with a social networking site dedicated to unfettered donor, nonprofit community, and nonprofit client feedback on programs and organizations. Then, these qualitative responses could be put through technological or human filters (I see Google foundation resources involved here) to deliver semi-quanitative reports and qualitative summaries to complement the numbers. Still not an ideal alternative to on-the-ground, firsthand interaction and investigation, but maybe a step better than what’s currently available in aggregate.

Just thoughts…

to: Sean raises an excellent point. Let’s presume that all nonprofits ranked at 1 star are questionable.

Could I cite specific cases where Charity Navigator has under rated a nonprofit? Sure, but the problem is not the specific failures as much as the combined flaws of a) the rating system itself, and b) the hype giving to ranking nonprofits by those ratings.

Sean raises an excellent point. Let’s presume that all nonprofits ranked at 1 star are questionable.

Note: Even these ratings could be in error because CN makes no affirmative effort to have the rating reviewed, commented upon, or ignored by the nonprofit itself.

If Charity Navigator’s rating and rankings are good measures of nonprofits’ success, you could trust them enough to wisely invest in those institutions (within your area of interest) that receive the greatest number of stars.

Even if you are willing to discount all the 1 star ranked organizations, the lack of uniform reporting on 990’s will make it almost impossible to discern the differences between 2 stars and 3 … or between 3 stars and 4.

Until Charity Navigator and others develop more comprehensive and diverse view of nonprofits than the 990, perhaps individual donors should take it upon themselves to ask nonprofits (in their areas of interest) why they received less than 4 star CN ranking.

http://charitynavigator.org/index.cfm?bay=topten.detail&listid=28

If any of those are actually responsible, effective charities meeting their mission with a passion, it would be a bit of a scandal. I am confused why, in all the criticism of CN’s ratings, specific examples of incorrect ratings are not discussed.

As a donor, the two main sources I am aware of for broad information on charities is Guidestar and Charity Navigator. The 990 may not be an ideal or even great source, but it is better than no source, and donors and non profits would be more effective if they were widely used. I think Trent Stamp’s blog is great, and empowers donors to be comfortable asking tough questions. Obviously, CN advises people to go beyond its star rating.

Finding a worthwhile charity to support is time consuming. If there is a better way to do it, hopefully the industry will hop to it in the next decade or two or three…

Of course not. However, can you point to one, two, or a couple dozen examples, of undeserved poor ratings?

Thanks!

I appreciate “a Fundraiser” passing along the Nonprofiteer post (great!) and comments on a critical moment in CN’s evolution. Perhaps CN turned to more aggressive marketing to avoid bankruptcy.

However, every organization benefits from asking if its marketing brings it closer to its mission? CN appears to have decided it did. I’m not so sure.

Do funders want a critical review of nonprofits so badly that they are failing to critically review the methods and results of organizations promising those nonprofit ratings / rankings?

While I continue to believe CN and all others who seek to evaluate nonprofit success will add depth and a diversity of content sources, I am reminded of the marketing maxim that, “Nothing kills a bad product faster than great advertising.”

http://nonprofiteer.typepad.com/the_nonprofiteer/2007/11/re-the-990.html

But, I also wanted to point out a post I put up in October of last year about the critical moment in CN’s lifecycle.

http://donttellthedonor.blogspot.com/2006/10/is-charity-navigator-going-bankrupt.html

At the time, it seemed to me that CN was going to go bankrupt. Since then, it seems they have reached out to secure more individual donations… I wonder if that is one of the reasons they have needed to use more attention grabbing headlines…

Also, because Intelligent Giving attempted a much more nuanced and sensitive approach here in the UK.

Personally I believe that CN has every good intention. They have done a huge service by shinning a spotlight on the concept of rating charities. They have the brand recognition that if they worked on transforming their rating system to something that made more sense they could continue to dominate the field and I for one would celebrate their move.

I think it would be great if Trent Stamp wanted to comment on this post. The atmosphere might seem a little harsh right now. But I for one would welcome his comments. We’ve all heard the arguments against CN. What are the arguments for their system?

Of course, CN and others’ rating systems are flawed at their base — 990’s contain substantial amounts of “interpretation” as Sean points out.

I too would applaud hearing from Trent Stamp / CN regarding their reactions to these posts and if / when / what they may be considering to improve their overall rating / ranking system.

Clearly CN and others are responding to a huge need / desire to better understand nonprofit performance and separate the wheat from the chaff.

Interestingly, one of the other non-profit blogs I read most often is Trent Stamp’s, the CEO of Charity Navigator. I think he does a good job on this blog highlighting some of the most egregious violators of public trust. These organizations are easily spotted through their 990s, with their grossly oversized fundraising expenses. I hope he sees this post and takes the time to respond to your criticisms