Category Archives: Impact Measurement

United Way Responds to Google Finance Question

Last month I posted a question on the Google Finance profile page of the United Way of America. I got an email today from Meg Plantz, director, Impact Design and Learning for United Way. You can catch up on the Google Finance for nonprofits discussion here.

United Way of America’s approach to program outcome measurement, reflected in its manual, Measuring Program Outcomes: A Practical Approach (United Way of America, 1996), encourages health and human service agencies to develop ways to measure outcomes quantitatively.  This often is a challenge because many human-service outcomes seem at first glance to be un-quantifiable, and many agencies are used to describing their successes with narrative vignettes rather than with numbers.  When they have used numbers, the numbers often have reflected outputs (e.g., number of classes offered, number of clients served) rather than outcomes (e.g., improvement in parenting skills).  However, for purposes of creating and tracking program improvement over time and demonstrating results to increasingly data-oriented funders, donors, and publics, numerical indications of program performance are important.

Our approach consciously responds to the challenges that quantitative measurement presents.  For example, in our approach, intended outcomes do not have to meet the “measurable” test.  In fact, when agencies are identifying what their outcomes are, we encourage them not to worry about how they will measure them, but instead to focus on what results are meaningful to the program and its clients.

Once the agency is comfortable that it has described the appropriate outcomes for its activities, then we advise them to identify measurable indicators of those outcomes.  Questions such as “What will tell you if clients achieve the intended outcome (e.g., improvement in parenting skills)?  What will you be able to see, count, or measure?” help agencies identify critical aspects of their outcomes (e.g., using age-appropriate discipline methods) and think about ways to quantify them (e.g., record observations of parents in role plays and tally entries that parents make in a journal).  Agencies are encouraged to pick indicators that will provide useful data – data that will help them with program improvement, in communications, and in other management tasks.

Qualitative information complements and can help with quantitative measurement.  For example, agencies can use their narrative success stories to identify intended outcomes and then use quantitative measurement as a way to learn whether the stories are unique or are representative of other clients.  Agencies can use their narrative stories to illustrate their outcome data rather than offering the stories as evidence of outcomes.  Qualitative information such as focus groups discussions of quantitative data can help agencies understand the meaning of the data and make appropriate program improvements.

Meg Plantz
Director, Impact Design and Learning for United Way

Nonprofits Publishing Impact Analysis

A nationally known nonprofit has asked me for examples of organizations that are “doing it right” in terms of publishing their impact analysis. In other words, which nonprofits are self publishing information that is useful to donors who are trying to examine if the organization is effective?

Any ideas? Leave a comment or send me an email.

Efficient Markets in Philanthropy

In response to my post yesterday in which I discussed the value of information to philanthropy and why donors should desire efficient philanthropic markets, Phil Cubeta writes:

The logic here can become relentless and destructive. What this tends towards a lists, like league tables in a sport, with the best at the top. It leads then to managing a nonprofit by the numbers, to get the rating, and it leads to shutting down those that don’t rank high. We then have the tyranny of the metrics, however much arbitrariness is built into them…

The world you want - are you sitting in corner office reading a spreadsheet?

So are the philanthropic capital markets I envision boring and lifeless with endless spreadsheets and numbers to crunch? Not in the least.

Economics is often called the “dismal science”. I know that many people think that finance is boring. But the vision of financial markets as nothing but numbers and spreadsheets does not capture the reality. Do investors buy stock in Apple because they spent hours and hours processing spreadsheet calculations? No. While at the end of the day, buyers of Apple stock believe that the return on capital being generated by the company will make for a profitable investment, the information they use to determine that are not just numbers. The way in which Apple has captured the imagination of the consumer, (an intangible piece of data that cannot be added to a spreadsheet) is by far the most valuable asset that Apple has and it is a major reason why investors have flocked to the stock.

Have you ever watched CNBC, the news channel of the financial markets? It is far from some kind of spreadsheet crunching lecture. Every day, investors or all types come on the show and make passionate arguments for why certain companies are good investments. While numbers and calculations underlie much of their thinking, it is the story, the human story of the companies they discuss that take center stage.

Warren Buffet is widely considered the best for-profit investor of his generation. Does he sit in a corner office reading a spreadsheet the way that Phil suggests? The quote below is from noted investor Whitney Tilson (Tilson is a huge fan of Buffet and a fellow columnist of mine at the Financial Times):

If the future were predictable with any degree of precision, then valuation would be easy. But the future is inherently unpredictable, so valuation is hard — and it’s ambiguous. Good thinking about valuation is less about plugging numbers into a spreadsheet than weighing many competing factors and determining probabilities. It’s neither art nor science — it’s roughly equal amounts of both.

The lack of precision around valuation makes a lot of people uncomfortable. To deal with this discomfort, some people wrap themselves in the security blanket of complex discounted cash flow analyses. My view of these things is best summarized by this brief exchange at the 1996 Berkshire Hathaway annual meeting:

Charlie Munger (Berkshire Hathaway’s vice chairman) said, “Warren talks about these discounted cash flows. I’ve never seen him do one.”

“It’s true,” replied Buffett. “If (the value of a company) doesn’t just scream out at you, it’s too close.”

Taking liberties with Tilson’s quote, I would argue that donors should not “wrap themselves in the security blanket of metrics” because “the lack of precision around measuring the impact that nonprofits achieve makes them uncomfortable.”

World-class investors do not sit in their office crunching spreadsheets all day. Neither should world-class donors. But the underlying logic of both should be that of achieving the highest return on investment.

Recently Phil commented to Perla Ni regarding her site Great Nonprofits (which offers reviews of nonprofits written by volunteers, donors and the people served by the nonprofit):

Thank you so much, Perla, for setting the record straight. In fact, your site is the exact opposite of a metrics driven exercise. You are bringing together the voices of those who have been touched by a nonprofit. I finally “got” what you are doing.

An efficient philanthropic capital market does not only view numbers as valuable inputs to the decision making process. Sites like Great Nonprofits offer extremely valuable information to donors. This sort of qualitative information is critical to both donors and for-profit investors. Great Nonprofits is not the opposite of a metrics driven exercise. They are both part of the same process of determining where donors and investors should direct their capital.

Disclaimer: Nothing in this blog should be construed as investment, tax or legal advice. This blog is for informational use only.

Information Sharing in Philanthropy

I wrote a post a while ago called Paul Brest Needs a Blog (Paul is the head of the Hewlett Foundation). I’ve been an advocate for more people in philanthropy to start blogging in general. In the above mentioned post I wrote:

So why should foundations blog? It seems to me that the imperative is not for them to embrace technology so much as it is for foundations to join and begin to drive the online philanthropy conversation. [But] it is the two-way flow of information that blogs encourage that is important, not blogs themselves.

Even so I’ve noted recently that some people feel that I’ve pushed blogging rather than information sharing. As the conversation we’re all having unfolds I think it is important to take a step back and make sure we haven’t missed the forest for the trees. I wish I had expressed my thoughts with more clarity.
When Phil Cubeta recently asked why nonprofits should blog, astute reader Michele Moon asked:

I’m not entirely sure why it’s blogging, in particular, that’s the focus of discussion, especially because it’s now considered a little bit old-hat, Web 1.5. What is it about the format that makes it so essential to transparency and its tyrant? Is it actually blogging you want to see - personal, real-time updates and editorials, followed (if you’re lucky) by people who read, comment, and sometimes stick around to converse?… Why should it be blogging that we aim to do, or is that shorthand for more complicated online interactivity?

I’m guilty of using “blogging” as short hand for information sharing. I’ll stop making that mistake.

When economists speak about efficient markets they are talking about a situation where money flows to the organizations that can put it to the best use. Widely available, robust information is a critical factor for a functioning efficient market. Recently, in a conversation with Phil Buchanan and other readers on this issue I wrote the following (you can find the full thread here. The Chronicle of Philanthropy recently highlighted the conversation):

In an efficient market, investing is a zero sum game. Maximum returns are generated globally so the only question is matching an investor’s risk/return preferences. In inefficient markets, higher returns accrue to more “effective/smarter” investors. In a public benefit market, since all returns accrue to everyone, investors should desire an efficient market within which they could align their social investments with their personal values/goals.

The philanthropic capital markets are highly inefficient. Far more inefficient than any for-profit marketplace.

Therefore, it seems to me that making the philanthropic capital markets more efficient should be the number one priority of large funders who desire to be effective…

I’m not arguing that the public will make better decisions than the “experts”. I’m saying that efficient markets will produce better outcomes than inefficient markets. In the for-profit world, inefficient markets are great for “expert” investors because they can exploit superior information to generate outperformance of investment returns. But these inefficient markets reduce the total returns in the market by preventing capital from flowing to the best performing investments.

What I’m saying is that unlike in the for-profit market, “expert” philanthropist enjoy no advantage from superior information. The returns they generate accrue to the public, and so no “outperformance” is possible. Instead, they should be interested in the total market functioning at a higher level, since that is the only way to increase the social return on investment that accrues to everyone.

This is the challenge we face as a field. How can we ensure that the $300 billion that is given to charity each year is flowing to the organizations that can put the money to its best use? The key will be our ability to supply market participants with widely available, robust information. Blogs are one tool in this work. There are many others.

Should Foundations Fund Philanthropic Information?

An interesting conversation is beginning to unfold in the comments to Phil Buchanan’s podcast. The point I’m making is not that foundations have some sort of obligation to fund nonprofit information for public use, but that doing so is in their best interest. This conversation ties in directly to the conversation we’ve been having about Google Finance and Google.org.

If a foundation can give $1 that creates $2 of social benefit, or give $1 that spurs the public to give $10, which creates $20 of social benefit, which one should they choose? This ability to give $1 and get $10 of social benefit instead of $2 is the “leverage” that so many philanthropist and foundations say they want to employ.

Here’s the big leverage opportunity of this decade: Provide the individual donors (who every year give seven times more than all the foundations in the country combined) the information they need to make better donation decisions.

Join the conversation with Phil Buchanan and let’s work this problem out!

PS: As background it might be useful for readers to note the essay by Paul Brest, the president of the Hewlett Foundation, in which he discusses “the advantages of good information” in philanthropy. In the essay he mentions Great Nonprofits, whose founder Perla Ni is participating in the conversation with Phil Buchanan. Hewlett is, to my knowledge, the most forward thinking foundation on these issues. Hewlett is also considering funding GiveWell.

PhilanTech

Dahna Goldstein, the founder of PhilanTech, sent me the following email. She didn’t post it as a comment because she didn’t want to appear to be plugging her company, but I asked her for permission to share it.

I’ve just caught up on the discussion on your blog about Google Finance and nonprofits, and wanted to share my $.02.  Google is potentially in a unique position — as are you, by virtue of your thought leadership and initiative on this front — to positively affect how information is shared with the sector at large, with donors, and with other interested parties.

The absence of standardized information about nonprofits makes it difficult to suggest a set of metrics or a pre-defined combination of quantitative and qualitative analyses, as a number of your readers have pointed out.  And asking for new types of reporting from nonprofits risks placing an additional burden on already-burdened nonprofits.

PhilanTech has taken a step towards addressing this issue.  We created the PhilanTrack online grants management system to centralize and streamline the grants management process — creating centralized reporting about nonprofit organizations, activities, outcomes, and finances.  Our vision is for a centralized reporting for all donors (institutional and individual) and other interested parties (researchers, the nonprofits themselves, etc.) to obtain information to inform funding decisions — without creating additional hoops through which the nonprofits must jump.  I’d be happy to tell you more about how it is structured, but in a nutshell, PhilanTrack helps foundations request the information they want to receive from nonprofits (to help them evaluate their effectiveness as grantmakers and to help evaluate potential grants) while helping nonprofits avoid reinventing the wheel each time they report to a different funder.  The types of information that the system manages (activities, outcomes, finances, lessons learned, etc.), I believe, are the types of information that Google should consider posting about nonprofits.

While expense ratios, as you have pointed out, have significant shortcomings, there is still a lot that can be learned about the financial health and stability of an organization (if not its effectiveness) by looking at its finances.  It requires looking beyond CharityNavigator and beyond 990 data in ways that are not familiar to many (both individuals and institutions) who are considering gifts to nonprofits.  At PhilanTech, we have addressed this issue by developing a financial analysis tool that uses basic financial statements to provide six analyses (financial mix, efficiency, debt servicing ability, liquidity, long-term viability, profitability) with a number of different metrics and explanations of why each metric is important and useful in evaluating the financial health of a nonprofit.

Google Finance pages (or Google Knol) should, in my view, combine these financial analyses with the following:  quantitative information about outcomes (where available and qualitative where quantitative info isn’t available); qualitative information about activities, programs/projects, mission, people, sustainability, replication (where relevant), lessons learned, challenges faced and overcome; related organizations (including any partnerships/collaborations); news; funders; discussion (like the type of discussion you prompted about the Red Cross), and perhaps something like the 360 degree views GreatNonprofits is working to create.  And there are ways it could be done without placing too great a burden on nonprofits by leveraging some of the reporting nonprofits are already doing.

United Way on Google Finance

After reading a United Way blogger’s reaction to my prediction that UW would develop an industry standard, narrative outcome measurement form and my discussion of Google Finance, I’ve started a discussion thread on the Google Finance page for United Way.

The United Way’s focus on Outcome Measurement is wonderful. I wish  there were more resources like the ones you list at http://www.unitedway.org/Outcomes/  elsewhere on the web. I was wondering if someone at United Way could  explain to me a little bit about how your organization thinks about  qualitative vs. quantitative evaluation of nonprofits. It seems to me  that quantitative metrics are probably easier to measure, but less  valuable than more difficult to measure qualitative outcomes.

Any help you can provide in thinking through this issue would be  wonderful. Thanks!

FYI: While the Red Cross has stopped posting to the thread I started on their page, I understand from sources that they are taking the questions about their effectiveness quite seriously.

I wonder how the United Way will respond?

Measuring YOUR Nonprofit

During the conversations about what to measure in philanthropy, a dominate theme has been that no universal metric will ever work (although some participants do not agree). This idea is validated by measurement practices in the for-profit markets where different metrics are believed to be important for different companies.

So how should an individual nonprofit think about measurement?

I got the following email from a reader recently:

Your blog. I read it every day. It’s great. But frustrating.

How do WE measure success? We’re trying to implement a program like [deleted to protect privacy]. It will be difficult to quantify success, especially short term. We could have 5 students and really change their lives now–or maybe not be able to point to the impact for years. We could have 50 students and not connect at all. When we discuss this among the staff and with well-meaning supporters, everyone says to just make something up. That really grates on me. And we can’t be the only program with the same problem.

This was my answer:

For a minute, don’t think about numbers. Just tell me what you think your organization would look like in five years if it were successful. For instance, if you raised and spent $1 million and during that 5 years worked with 5 students. Would that be a success? What about 500 students? Or 5,000? If you had a choice between working with 500 students and feeling like you exposed them all to music, but didn’t really change their lives would that be better or worse than working with just 5 students and feeling that you totally changed all of their lives for the better?

After you have an idea of what success would look like, then we can think about ways to measure it.

Rebooting Nonprofit Evaluation Debate

A lively debate about nonprofit evaluation and metrics has been raging in response to my request for input on my meeting later this week with Google.org. However, the conversation has splintered into a debate over whether a systematic, “metric” driven process of scientific measurement is needed, or whether the frame of scientific measurement is “an epistemologically impoverished frame” through which to understand nonprofit evaluation.

I personally believe evaluating nonprofits is mostly about evaluating their output (the social good they produce). Since it is difficult (impossible?) to quantify this output, I think the focus on metrics as a framework for evaluation is misplaced. Metrics can be used, but they should be designed on a case-by-case basis for each situation. That being said, I think the conversation has fallen into the trap of being constrained by historical frames of reference.

I want to have a different conversation.

I’m interested in what information is available to donors who want to evaluate a nonprofit and which of this information is useful. Google.com is mostly a resource that points to information; they don’t tend to create a lot of their own content. So if we imagine a future version of the nonprofit data inside of Google Finance, I don’t imagine it will be some new metric that we design. Instead, it will point to existing information on the web. When I first wrote about nonprofit info in Google Finance, I said I hoped they would not display Charity Navigator ratings (although I would support them noting if a nonprofit had a zero or one star rating since I do believe that a Charity Navigator rating at this level is a significant red flag)

So the conversation I want to have is what information do readers think that donors should consider when evaluating a nonprofit? Then secondly, where or how can this information be captured online so that it can be displayed in Google Finance?

Open Invitation to Foundation Employees

I realize that if you work at a foundation, you may not want to jump into a conversation that involves telling another foundation what to do. However, the conversation we’re having here is really important and would not be complete without the input of the army of program officers (ie. Nonprofit evaluators) that read this blog. So please consider commenting anonymously (just let us know you’re a program officer) or comment publicly and realize that we’re having a broad conversation about nonprofit evaluation that goes beyond Google.org and Google Finance

Open Invitation to Nonprofit Employees

A conversation about nonprofit evaluation would not be complete without the input of the nonprofits being evaluated. What information do you, as nonprofits, what donors looking at when they evaluate you? It could be that someday the Google Finance website about your organization becomes the top ranked search result on google for your nonprofit. What do you want on that page?

Philanthropy Conversation Wants You!

Rather than post today, I’m going to point you back to this post and encourage you to join the growing conversation in the comments section. I think the topic of this conversation is the most important issue facing philanthropy today. The fact that this conversation is centered around Google adds time sensitive relevance to the subject, but the subject matter at hand is far bigger than Google. The issue is how can we improve the available information about nonprofits so that the $300 billion+ donated to charity each year can flow to the best nonprofits. Improving the flow of philanthropic capital will completely transform the nonprofit sector and you won’t believe what we as a sector will be able to accomplish.

So click here and add your voice to the mix. Philanthropy needs you.

What to Measure and Why in Philanthropy

I’m meeting with someone from Google.org next week to talk about what kind of information I think they should make available about nonprofits in Google Finance and other ways that Google.com’s mission statement to “organizing the world’s information” can be directed at the Third Sector.

In preparation, I’d like to spend some time speaking as a community about this issue. I encourage you to leave comments or email me your thoughts.

In response to the thread I started on the Google Finance Red Cross board about how effective they are, I got a comment from Leyla Farah of Cause + Effect public relations:

One item I’d offer: a measurement of “average cost of impact” - in other words, the organization’s total budget divided by the total number of people (or animals, or acres of land) it’s benefited within a specific time period. That metric would (1) force each organization to provide a definition of how it helps people (etc.) - and (2) force it to account for all the costs associated with providing that help.

While Phil Cubeta of Gift Hub scolded me for focusing on metrics:

Paradise Lost versus Gone with the Wind. What metrics do we use to determine which is better? Some subject matter requires judgment, taste, discernment, even wisdom. We have movie critics, book critics, educators to help us make more discriminating judgments. Before we cry ourselves hoarse over metrics, we have to ask whether philanthropy is more like art or more like business. The call for metrics can be a bullying move by the half educated to impose their MBA logic on a sector whose reason for being is that it stands in contrast to both government and business. As the old saying goes, “Do not attempt to cure what you do not understand.” Stressing metrics, Sean, is in terrible taste. You paint yourself as Barbarian.

Personally, I’d like to state that I don’t intend to stress metrics as being valuable unto themselves. However, I do think that all things in life can be judged, at least in each person’s personal view, as being bad, good, better and best (I’m sure there are some exceptions, but you get the point). I think it is critical that we find ways to judge nonprofits so that philanthropic dollars can flow to the organizations that do the most good in the world. To me, funding the best of what is available is far more important than trying to invent the next big thing. I think that information about nonprofits is what is needed and this is why I care about nonprofits being in the Google Finance portal.

As a professional investor in for-profit companies, I can tell you that there are very few (none) golden metrics that allow you to comprehensively judge one for-profit against others. Even very widely used metrics like “price to earnings ratios”, “dividend yields”, “profit margins”, and “earning growth rates”, have been show in practice to be very useful, but not in any way adequate to judging the superiority of one investment choice vs. another on their own.

In my Philanthropy Predictions for 2008 that I wrote for the Chronicle of Philanthropy, I made one reference to measurement:

A United Way-authored outcome-measurement template will be adopted by the sector as the standard format for nonprofit organizations to report on their effectiveness. The narrative-driven form will soon be available for download from the home pages of many nonprofits.

Note that I suggest a “narrative-driven form”. If you read analyst reports on for-profit investments, you’ll see a lot of numbers and metrics, but the heart of the report is a narrative about the company.

This brings me to an excellent comment from the thread mentioned above from an anonymous “young staffer”:

If I may carry the Paradise Lost vs. Gone with the Wind analogy a little further, I think it raises some interesting points.

The first is that there are plenty of potentially relevant metrics with which one could back up one’s a claim for each work’s superiority: their longevity in years, the number of universities that include them in introductory freshmen humanities courses (as a proxy measure of their centrality to our cultural canon), a RottenTomatoes.com-style survey of critics. I can even imagine poor grad students counting allusions to them in last year’s bestsellers.

Relying solely on any one of these potentially valid measures, however, would obviously leave you wide open to criticism for the flaws of your methodology and the limits of the analysis. To construct a strong argument for your preferred choice, one could use both the metrics and qualitative measures. Same goes for nonprofits - the measures are neither perfect nor complete, but that is not the same as nonexistent.

I think the other point is the difficulty of comparing apples and oranges. Let me reframe the question as “Paradise Lost” work of literature vs. “Gone with the Wind” work of film. Both are widely-considered seminal works in their mediums. It’s not hard to imagine metrics, like those above, that could easily distinguish each as a leader within its respective medium. It is much harder, however, to compare them very convincingly across mediums. An author and a film buff might reach very different conclusions about which one matters more in today’s culture. Their distinctive values and tastes will influence that decision.

The same, I think, is true for nonprofits. Too universal a measure like “average cost of impact” might not be helpful for identifying whether a great afterschool program in New York or a great community health program in Uganda is better. The costs and the measures of impact are on different scales. But metrics certainly might help you identify each within its field as the seminal nonprofit. From there, one’s values and tastes might be expected to guide your choice.

So there you have it, a good beginning to an important conversation. If there was a single webpage, like this one for the Red Cross, or this one for Cisco Systems, that contained all the information you would like to see when you wanted to examine a nonprofit for the first time and decide if you might want to support them, what information would you like there to be on the site?

Google.org owes me nothing and anything I tell them might be ignored. But on the other hand, I will deliver the message that we co-create over the next week in this discussion. Someone from one of the largest (and oldest) foundations has already asked me to pass on their offer of help to Google.org after reading my posts on the subject. I do think that any effort that you the reader put into this discussion will be heard by the powers that be at Google.org, even if they do not take action.

More Google Finance for Nonprofits

Someone at Google.org read my posts (here and here) about nonprofit information being available within Google Finance and invited me to meet with them early next month. If you have any thoughts you’d like me to share with them, shoot me an email or leave a comment.

Checking back on the Red Cross discussion I started on the Google Finance discussion board, I found a new reply. I commend the Red Cross employees who have taken a shot at my question of how they know if they are effective, but I’m a little shocked that so far the organization has be unable to provide even minor information related to whether they do a good job. “Hard Nose Philanthropy” is on the rise, nonprofits need to be able to answer a simple question like “Tell me how you know if you’re being effective”.

Does anyone know of any other discussions on Google Finance nonprofit discussion boards? Here’s the new reply from the Red Cross:

From: ike.pig@gmail.com - view profile Date: Wed, Dec 26 2007 7:36 am

Hello — this is Ike, and I am a regional communicator for the Red
Cross.  I stumbled across this over the holiday break.

I understand what you are talking about, with regards to our internal
measure of “effectiveness.”  Unfortunately, you’re asking us the
equivalent of choosing a favorite child.

Such a metric would be arbitrary, and could be easily fashioned to
highlight whichever line of service we wished to justify.  In doing
so, number-crunchers would ask the question “Why in heck is Red Cross
involved in things that AREN’T as high-payoff as _______?”  Just look
at the numbers.  Why be involved in disaster relief when blood
provides the higher “impact?”  Or vice versa?

We’re dealing with two different dynamics here.  As a large multi-
purpose humanitarian organization, we’ve got a tradition being
involved in a number of different activities.  Disaster, blood,
service to armed forces, preparedness, first aid/safety, and some of
the international initiatives Maura described.  Whether we like it or
not, there is a significant slice of America that expects the Red
Cross to play a role in each of those arenas.  Public expectation
drives part of our mission.  In some circumstances, we have made a
promise to be there (like immediate disaster relief).  In others, we
end up getting involved because people think that’s what we’re
supposed to do, and no one else is stepping up (like the Safe and Well
website partnership.)

The second dynamic is our volunteers.  Some only have an interest in
disaster.  Some only want to teach first aid classes.  Some want to
volunteer to drive needed units of blood from the storage centers to
the hospitals.  As a volunteer-led group, we’d alienate so many people
who are truly volunteering their time to make it all work.

Are you really asking us to pick the one most effective line of
service, and do that to the exclusion of the rest?  Because applying a
universal metric to all the lines of service is an invitation to
starting feeding some and starving others.  That would be akin to
comparing the costs of helping 10 families in an apartment fire versus
10 single-home families spread out on different nights.  Yes, one is
more “cost-effective.”  That doesn’t mean it’s time to abandon the
rest.

I think the key element you are dancing around here is the way we
handle donations.  If someone wants to donate just to local fires in
their local chapter jurisdiction, we can assure that happens.  If
someone wants to donate just to Services to Armed Forces, their wishes
are respected and followed through.  We look at the business model of
each of those lines differently, asking first “Are we meeting this
mission?” and “Can we meet it more efficiently another way?”

From: sstannard-stock@ensemblecapital.com - view profile Date: Thurs, Dec 27 2007 8:28 am

Thanks so much for jumping into the conversation. I’m not asking you
to choose anything. I’m just asking how the Red Cross tracks whether
you’re doing a good job.

For example, at my firm, Ensemble Capital Management we look at hard
numbers like revenue growth, assets under management and assets per
client. We also look at softer measures like visibility in the media
and online, depth of relationships with referral sources, and client
satisfaction. You can put good numbers on the first set, but not on
the second.

All I’m asking the Red Cross is how do you know if you are doing a
good job? What do you track? And how do you compare yourself? For
instance, what if I asked you why my money could do more good by
donating it to you than donating it to another similar organization or
even to FEMA? If an investor or prospective client asked me why I
thought that Ensemble was a better investment or firm to hire than our
competitors, I could speak to the issue for hours, citing both hard
data and soft qualities. I’m just asking the Red Cross the same
question.

Why GiveWell Matters

GiveWell and its co-founders Holden Karnofsky and Elie Hassenfeld are suddenly media darlings. Not many people are written up in both the New York Times and the Wall Street Journal on the same day.

Stephanie Strom at the New York Times writes up a great profile in “2 Young Hedge-Fund Veterans Stir Up the World of Philanthropy”:

Their efforts are shaking up the field of philanthropy, generating the kind of buzz more typically devoted to Bill Gates and Warren E. Buffett, as charities ponder what, if anything, their rigorous approach to evaluation means for the future…

Read the full article here.

Sally Beatty and Rachel Emma Silverman at the Wall Street Journal discuss GiveWell in their article, “Doing Due Diligence On Your Donations”:

Donors can readily compare charities from a financial perspective: how much an organization spends on administrative costs or fund raising, for instance. But givers, especially younger, business-minded ones, now tend to want more information on how successful a charity’s programs are in addressing the issues the charity sets out to resolve, from feeding the homeless to securing employment for the disabled. That’s especially important as the number of charities continues to grow, with about 1.4 million to choose from…

… And there are a growing number of groups whose aim is to make charity-effectiveness evaluations open to the public. GiveWell, for instance, was started this year by two former hedge-fund researchers who were frustrated by the lack of available information on charities’ results and impact. They research and grant money to organizations in specific topic areas that the group deems effective and post the results on their Web site. For example, when researching job-training charities in New York, GiveWell asked groups to provide data on how many people took advantage of the programs, what skills they were taught, what percentage of clients found jobs, what kind of jobs they found, and how long workers kept their jobs, says 26-year-old co-founder Elie Hassenfeld.

The article ends with this advice,

It’s also smart to see if the charity’s progress has ever been evaluated by a third party, rather than just the charity itself. Check the charity’s Web site or annual report for specific details on how it gauges its results. If the information isn’t there, call the charity and ask. Be wary about giving, however, if a charity doesn’t answer your questions or provide annual reports or other filings.

When the Wall Street Journal tells donors to be suspicious of nonprofits who won’t provide details of how they gauge their results, you know there’s a sea change coming.

When I first wrote about GiveWell in February and said, “Why are the young members of the GiveWell project doing more to improve our shared knowledge base than The Ford Foundation?” and when I wrote in April that, “Fringe players like Holden (Karnofsky) are actually the real change agents (in philanthropy).” I never thought that by the end of the year, the New York Times would be quoting a GiveWell team member saying:

“There are huge foundations out there whose job it is to find great organizations doing great things,” said Robert Elliott, a club member who is now the Clear Fund’s chairman, “but when you call them and say you’d like to leverage the information they’ve already collected to make a smart donation, it’s a closed book.”

The IRS is focusing more and more on accountability and efficiency in the philanthropic sector. But with GiveWell being featured in the New York Times, Wall Street Journal, Chronicle of Philanthropy and Chicago Tribune in the last week, you have to start thinking about the cultural norms that these reports are creating.

When the LA Times wrote about the Gates Foundation investment policy earlier this year, the article created more movement on “aligned investing” in the foundation world than the IRS would ever accomplish through years of committee meetings.

Will the next LA Times exposé question why foundations are not sharing their philanthropic knowledge with the public and why two 26-year-olds with no philanthropic experience and a tiny budget seem to be doing the most to help donors?

GiveWell in the Chicago Tribune

The Chicago Tribune has a long article about GiveWell on the front page of their business section today. I’m amazed. It is really wonderful to see an innovative startup like GiveWell get written up in the mainstream press.

Red Cross Replies

The Red Cross took less than 7 hours to respond to my question about their effectiveness on the new Google Finance for Nonprofits portal. You can read our back and forth here. I believe that this is the first discussion occurring on a Google Nonprofit page. Personally I’m glad the discussion is about effectiveness. The Red Cross gives a good reply that most donors will be happy with. I was impressed. But I’m sure that no foundation or someone like Holden Karnofsky would find the answer sufficient. No links to impact data. But that’s OK. All of that is coming a short way down the road. I think that open discussions between donors, nonprofits and others in a hosted forum like Google will only hasten the move towards transparency and demonstrated impact.