My point about IMDB is that a good review isn’t about degrees or credentials. But in charity, unlike movies, it IS a heck of a lot of hard work. Social networking isn’t the miracle cure.

In my analysis I use film critics as an example. Certainly they evaluate films very extensively, but they do not try and return metrics to the reader. Instead they use language to qualitatively review the film.

http://www.vanstokkom.nl/indepers/MightyMeasurablesAlliance0906.pdf

Best regards,
VanStokkom.

On the other hand, if you can’t measure in a way the good you’re doing in the world, I would question what good you’re actually creating. This doesn’t mean sum it up into one tidy little figure – evaluation is a difficult science, but far under-used. Many in the nonprofit sector have a tendency to operate based on an instinct that they’re doing good, and these instincts can definitely be wrong. I can’t find it right now, but there was a great study a little while back about some sort of child development program (mentoring, perhaps?) which sounded great, but they showed to actually, looking across many outcomes, to be *decreasing* the kid’s chances for the future (my recollection is the hypothesis was that because the kids were identified for the program, they were given the feeling that they were “really far gone,” which contributed to them, well, becoming even further gone).

My point is that it’s tempting to rely on a “gut check” for how useful something is, but guts can be wrong, and they’re certainly drawn to flashier programs rather than things that are just quietly working (one of my favorite quotes, from Esther Dyson “Millions of people get hungry every day, even though it’s not very interesting.”). However, going full circle, solid evaluation tends to be very expensive – yet another administrative expense – and has to be weighed in the larger scheme of things that an organization should spend money on.

What if movies were encrypted such that no one (and I mean *no one*) could actually see them? What if the only way to get any of idea of what happens in a movie were to do an extremely difficult, extremely costly study that in the end could never be conclusive, and what if the expense and difficulty of these studies meant that they were rarely carried out, and that when they were, they were carried out by people concerned about “turning people off from Hollywood,” who therefore filled the studies with gushing optimism beyond what they’d actually studied? What if most people guessed at how good the movies were using only the reputation of the director (a reputation that itself was built on movies that no one had seen) and 2-sentence plot summaries?

Then, Hollywood would pretty much suck … but the analogy to charities would be much more accurate.

There is a part of your post that I agree with. Not everything has to be quantified. If you can show me that $1000 will save a child, I can check my gut to see if that’s a good deal, just like I can check my gut to see if I enjoyed a movie (once I’ve actually seen it). But without ANY formal analysis, I simply have nothing to evaluate - no experience, no evidence, nothing except that dang Form 990.

The fundamental difference between charities and consumer products such as movies and restaurants is that the mere fact of being a charity’s customer doesn’t mean you know ANYTHING about it. If we want to separate the good charities from the bad, I simply see no way around doing difficult, costly, painful, formal evaluations.