Category Archives: Evaluation

The Tyranny of Metrics

April 30, 2008 – 2:53 pm

Reader Jeane Goforth writes:

Evaluation. My co-founder and I spent 2 hours standing on a street corner ‘after’ work discussing how to measure and convey the profound experiences we have daily. My stomach churns and my shoulders ache when our expert non-profit adviser talks about metrics. I struggle to add one more thing to my to-do list and I know metrics don’t say enough about why what we’re doing works and why it is important.

To me, this comment sums up everything that is wrong with metrics, both how they are dismissed by detractors and how they are misapplied by advocates.

Bad metrics are “one more things to my to-do list”. If the metrics don’t help the nonprofit run their organization better, than their relevance should be questioned. At Ensemble Capital we track a number of metrics about the performance of the firm. We don’t do this to show them to anyone else, we do it because they help us understand our organization better. But we also know that the metrics that are trackable do not capture everything (or even most) about our business.

One of the most important events in Ensemble’s history (in my mind) was when I was making a presentation to the CEO of a public company about why he should consider opening a private foundation. Half way through the conversation he cut me off and said, “Let’s do it. This makes all the sense in the world. A couple of years ago I asked my CPA if I should have a foundation and he gave me one answer. Then I asked my lawyer and he gave me another answer. But clearly you understand the reasons that I personally need a foundation, so let’s get one started today.” This individual had access to some of the best advisors around. Yet they failed him when it came to philanthropy. That was when I knew Ensemble was going to be a success. But that statement doesn’t show up in any metrics that we track.

Every nonprofit should be searching for relevant metrics to track that can help them run their business. If you don’t realize that total donations are going up only because two large donors have significantly increased their giving, then you will not see the increasing risk to your budget of losing those two donors. If you are working with high school students to help them get into college and the rate at which students are going to college is not budging, you have some examination of your program to do (although it doesn’t mean by itself that you’re failing).

But metrics that do not help the nonprofit are probably useless to donors. If a donor asks for a metric that you do not think is relevant, it is either because you are mismanaging your organization, or far more likely, the donor does not really understand what you do.

Quantifying “doing good” is tough if not impossible. But the idea that nonprofits don’t have time on their to do list to think about whether they are doing a good job is poison. Anyone running a for-profit or nonprofit organization should be thinking everyday about how they can do better and what tools can help them understand their organization. If “tracking metrics” is something that is just busy work, then it is useless work.

By Sean Stannard-Stockton | Also posted in Impact Measurement, Philanthropy | Comments (3)

Evaluation 2.0

April 25, 2008 – 10:21 am

I believe that in many aspects of life, a kind of pendulum effect exists. This effect describes the way in which people’s opinions tend to swing back and forth around reality. Rather than reflecting reality, people’s views vacillate in an arc around true reality. This creates a kind of boom/bust scenario that is very evident in the stock market (dot com stocks will make me rich! Ahh! Dot com stocks are poison!), but also shows up in politics, educations, pop culture, etc. I ran across a well put description this morning on Yahoo Answers:

I think the idea is that: somebody gets a good idea, and then a whole lot of people 1/2 understand it, and make it absolute, taking it to an extreme. Until somebody discovers “another” great idea, (the way things were originally done), and everybody jumps on that runaway train to hell.

The lesson: Moderation. A little common sense goes a long way. Don’t just “swing with the pendulum” of fashion in teaching/learning methods.

I think that there is a strong pendulum effect in philanthropy. I see it at work when we talk about metrics, evaluation, “philanthrocapitalism”, venture philanthropy, etc. Today I want to share with you an excellent article in the Financial Times by Gara LaMarche, the president of The Atlantic Philanthropies. I think LaMarche describes well the way in which approaches to evaluation have “swung too far” and his recommendations for a middle ground makes a ton of sense.

The philanthropic world, poked and prodded by a wave of new donors fresh from success in the business world, is grappling with the issue of evaluation. How do we know that grants – or, as they are now often called, reflecting the influence of the profit-making sector, “investments” – are making an impact?…

…Evaluation is a learning tool for the organisation and the funder, not a stick with which to beat grantees.

…Doing this correctly takes money… Funders should recognise and support their grantees in their efforts to learn what works.

…Evaluation should measure only what is important. Data should never be collected for the sake of it. The “metrics” obsession that has overtaken some funders has not always recognised this. Funders should never make grantees jump through hoops, distracting them from their core mission and costing valuable staff time, for reporting on trivial things. And there is nothing more demoralising, from the grantee’s perspective, than doing all this paperwork only to have it ignored.

…Both funders and the organisations they support need more humility about cause and effect. Organisations working for social or policy change should understand that no significant change was brought about by one organisation working alone.

…Finally, the most important thing: start with what you believe. If you have a passion about ending the death penalty or the isolation of older people – whatever it is – find a way to advance it first and worry about how to measure it second.

You can read the full, excellent article here.

By Sean Stannard-Stockton | Also posted in Cross-Disciplinary Conversations, Impact Measurement, Philanthropy | Comments (5)

Short-term vs. Long-term Focus in Philanthropy

April 4, 2008 – 8:03 am

In the Summer 2007 edition of the Stanford Social Innovation Review, Charles Conn, a senior advisor to the Gordon and Betty Moore Foundation and a high tech executive, wrote about the short-term focus of most foundations in an article titled Robbing the Grandchildren:

If future generations could vote on how foundations invest their money today, would they choose the current allocation? Byron Swift, chair and executive director of the World Land Trust, suggested this thought experiment to me, and I am disturbed to find that my answer is no……U.S. charitable foundations are better positioned than companies, governments, and universities to address these long-term, potentially catastrophic problems. One of the few sources of long-term risk capital, they control more than $500 billion in assets, generating funding that with other charitable giving totals almost 2 percent of GDP. With Warren Buffett’s gift, the Gates Foundation alone will control more than $60 billion in assets and $3 billion to $5 billion in annual spending. Other foundations closely associated with the digital revolution (such as Dell, Ellison, Packard, Hewlett, Moore, Omidyar, Page and Brin, Yang) could account for at least $50 billion to $70 billion more.

Perversely, though, many of these new tech entrepreneurs are worsening foundations’ shortsightedness by implementing businesslike metrics and controls in a way that reinforces short-term thinking and behavior. Other questionable management practices, such as low payout rates and lack of coordination with other organizations, further aggravate foundations’ myopia…

…A recent movement, sometimes called philanthrocapitalism or venture philanthropy, seeks to avoid complacency and lack of focus in foundation management by introducing rigorous success metrics and accountability practices. Many of these new-style foundations limit their scope to a few problem areas and, like corporations, intensely monitor outcome metrics, often with tight windows for review. To those of us who came to foundation work after a career in business, this sounds eminently sensible; after all, the foundation world is littered with fragmented, unfocused, and failed programs…

This short-term, metric-focused approach likewise hampers grantees. Foundations take the passionate and committed people in these institutions and harness them to near-term indices of progress. Grantees, in turn, stop playing the long-term game in order to keep the money flowing. They aim lower, too.

I agree completely with Conn’s thesis, but I want to elaborate because Conn fuses “short-term”, “metric-focus” and “businesslike” as if they automatically go hand in hand.

In the stock market, most people have become more and more short-term oriented. In the 1950’s, investors held stocks for an average of 7 years. Today the average is 11 months. Investors have gained access to vast amounts of information they never use to see and yet in many cases have used this information to change their mind more frequently rather than to gain more conviction in their decisions.

Almost all great investors make financial decisions based on a long-term outlook, not a prediction of what will happen in the next three months. At my firm Ensemble Capital Management, we talk about “arbitraging other investors’ time horizons”. In other words, we try to identify situations where short-term bad news about a company causes other investors to sell the stock so that we can buy it at lower prices. We use short-term good news that causes a stock to move higher to sell stock in companies whose longer-term outlook we think is deteriorating. Warren Buffett or most any great investor will tell you that Wall Street’s obsession with quarterly earnings reports is misplaced. Some companies (such as Coca-Cola, a company Buffett has owned a long time) have stopped issuing guidance to investors regarding what their next quarterly earnings might be.

It is human nature to want results as quickly as possible. But to achieve success, we must match our investment decisions to our time horizon. If we want to fix a local school because our child will be attending starting next year, then it might make sense to focus on short-term solutions. But most donors fund issues because they want to have a sustained impact on a situation. The techniques that might reduce crime in a bad neighborhood the most over the next month are unlikely to be the techniques that will have the largest, permanent impact on reducing crime rates over the next couple of decades.

Financial market participants are often short-term focused. They often focus on metrics which describe short-term conditions, but do little to illuminate long-term trends. But great investors and great philanthropists must focus on the information that matters to the long-term success of their projects.

Disclosure: Nothing in this post should be considered investment advice.

By Sean Stannard-Stockton | Also posted in Effective Giving, Grantmaking, Impact Measurement, Philanthropic Capital Markets | Comments (0)

Albert Ruesga on Metrics Mania

March 26, 2008 – 9:20 am

Albert Ruesga, who blogs at White Courtesy Telephone, was a panel member at the recent Metrics Mania debate at the Bradley Center for Philanthropy & Civic Renewal. Albert posted the text of his comments today and they are outstanding.

At a time when we are seeing a growing backlashing against “philanthrocapitalism”, it is interesting to look at what is being grouped under that term. For many people, “metrics” and the push for more “evaluation” of philanthropy is an unwelcome element of a “business-like” approach to giving. I believe that evaluating nonprofits and philanthropy in general is necessary for a the Third Sector to become a high-performing, high-impact driver of social good. But as I wrote last week, I think that much “evaluation” takes a scientific approach to measurement that is borrowed from the hard sciences, while the lessons of the liberal arts (under which investing and financial markets should be categorized) are more appropriate.

Albert, I think, would agree. He writes:

Measurement and evaluation, when done properly, are not just a bit of value-added for philanthropic or nonprofit work, they’re absolutely essential. Only a fool would disagree with that proposition.

But here I mean not just the kinds of formal evaluations described by Gary Walker in his essay, but informal evaluation as well: the kinds of course corrections we naturally make when we embark on a project, take a false step, and adjust what we do accordingly. Evaluation is not and should not be the sole province of the highly compensated consultant. We evaluate all the time; our own eyes and ears notice things the most astute consultant will never notice; and we’ll often be our own worst critics.

Now here’s where the metrics schmetrics comes in, perhaps: More nonsense has been spoken and written about evaluation than about any other subject in philanthropy. The number of people practicing evaluation without a license and without a proper scientific and philosophical grounding in the subject is, in my view, a scandal. Worries about evaluation, engendered in part by logic models the length of whale intestines, have become the math anxiety of the philanthropic world…

…I want to make clear that I’m not in the least anti-evaluation. As I’ve written elsewhere, I’m concerned that we tend to seek a kind of scientific or moral certainty from a formal evaluation where none exists. The questions that funders most often bring to an evaluator—Was this program worth our $25,000 investment? Should we continue funding it?—are questions only they, the funders, can answer. Say we measure a 25 percent drop in the truancy rate for a hundred kids in some program, and a 25 percent increase in their test scores. Is that worth $25,000 to you? Each donor needs to answer that question for him- or herself. As donors we will never be absolved of our responsibility to use our good judgment.

“Evaluation… the math anxiety of the philanthropic world.” What a great line. Evaluating nonprofits and social good in general is not about math. When I advocate for a financial markets-type approach to evaluation, I am not calling for number crunching. I wrote about this concept in January and will repost my comments from then rather than repeating myself:

Economics is often called the “dismal science”. I know that many people think that finance is boring. But the vision of financial markets as nothing but numbers and spreadsheets does not capture the reality. Do investors buy stock in Apple because they spent hours and hours processing spreadsheet calculations? No. While at the end of the day, buyers of Apple stock believe that the return on capital being generated by the company will make for a profitable investment, the information they use to determine that are not just numbers. The way in which Apple has captured the imagination of the consumer, (an intangible piece of data that cannot be added to a spreadsheet) is by far the most valuable asset that Apple has and it is a major reason why investors have flocked to the stock.

Have you ever watched CNBC, the news channel of the financial markets? It is far from some kind of spreadsheet crunching lecture. Every day, investors or all types come on the show and make passionate arguments for why certain companies are good investments. While numbers and calculations underlie much of their thinking, it is the story, the human story of the companies they discuss that take center stage.

Warren Buffet is widely considered the best for-profit investor of his generation. Does he sit in a corner office reading a spreadsheet the way that Phil suggests? The quote below is from noted investor Whitney Tilson (Tilson is a huge fan of Buffet and a fellow columnist of mine at the Financial Times):

If the future were predictable with any degree of precision, then valuation would be easy. But the future is inherently unpredictable, so valuation is hard — and it’s ambiguous. Good thinking about valuation is less about plugging numbers into a spreadsheet than weighing many competing factors and determining probabilities. It’s neither art nor science — it’s roughly equal amounts of both.

The lack of precision around valuation makes a lot of people uncomfortable. To deal with this discomfort, some people wrap themselves in the security blanket of complex discounted cash flow analyses. My view of these things is best summarized by this brief exchange at the 1996 Berkshire Hathaway annual meeting:

Charlie Munger (Berkshire Hathaway’s vice chairman) said, “Warren talks about these discounted cash flows. I’ve never seen him do one.”

“It’s true,” replied Buffett. “If (the value of a company) doesn’t just scream out at you, it’s too close.”

Taking liberties with Tilson’s quote, I would argue that donors should not “wrap themselves in the security blanket of metrics” because “the lack of precision around measuring the impact that nonprofits achieve makes them uncomfortable.”

World-class investors do not sit in their office crunching spreadsheets all day. Neither should world-class donors. But the underlying logic of both should be that of achieving the highest return on investment.

By Sean Stannard-Stockton | Also posted in Cross-Disciplinary Conversations, Effective Giving, Impact Measurement | Comments (6)

Do Nonprofits Want Funders to Be Critical?

March 5, 2008 – 9:25 am

A comment from an anonymous “director of development” was posted today on my Donors vs Investors III post (check out the growing conversation in the comment section of this thread):

Just to bring another perspective to this line of questions, here’s a fundraiser’s take. I’m sure there are lots of forward thinking, transparent non-profits out there who can speak candidly with anyone about mistakes and areas to improve, but my sense is that the vast majority are like my employer: they would never let any  information that might even suggest something less than sparkling about them be publicly revealed.

We have one foundation funder who is openly critical of us, and funds us with a contract and a set of concrete tasks the organization accomplish. I would call this funder a proactive investor. They didn’t just evaluate us, they made their findings known, and better yet continued to offer us money if we made an effort to clean up our act. Many staff are grateful for this funder, and believe our organization has improved with its participation.

So I guess my point is, perhaps a non-profit is best served by funders who can own their criticism, stand by it and use it as a tool. The many many non-profits out there who are less interested in critically evaluating themselves can benefit from proactive investors like the one I have described. And at least when I am in the room with this funder, I am more or less confident that what they say about us at conferences is what they say to my face.

Wow. I might be advocating for a more public dialog, but I’m surprised as anyone to hear a nonprofit employee say their organization has benefited from a major funder being openly critical.

By Sean Stannard-Stockton | Also posted in Cross-Disciplinary Conversations, Effective Giving, New Philanthropy, Philanthropic Capital Markets, Philanthropy, Transparency, nonprofits | Comments (7)

Investors vs. Donors

February 27, 2008 – 8:30 am

I have some questions and would love your feedback.

  1. Why do investors take credit for picking great investments (”look how smart I am, I bought XYZ stock!”), while philanthropists, especially foundations, claim that the credit goes to the nonprofits they fund (”the grantee did all the work”).
  2. Why is it acceptable for investors to talk about investments they think are bad (”Don’t buy ABC stock, their management is terrible!”), while philanthropists never badmouth nonprofits, even if they think they are ineffective?
  3. Related to #2: Why do public companies generally ignore all the talking heads who say negative things about them, while nonprofits find it intolerable to have a prominent person speak negatively about them in public?

If you’ve read this blog for awhile, you probably think these are leading questions and that I have a firm bias about which approach is better. But I’m truly asking these questions with an open mind. Recently I met with the director of philanthropy-focused grantmaking at a large foundation. I brought up the idea that publicly talking negatively about ineffective nonprofits (especially those that the foundation thought were not “fixable”, say because management was incompetent) would produce a positive social impact by directing other donors’ funds away from the bad nonprofit and towards more effective competitors. She told me that it was a primary value of the foundation to not harm grantees.

I think that is a very compelling counter argument and I’m interested in how readers view my three questions above and the idea that not harming grantees means never saying anything negative about a nonprofit.

By Sean Stannard-Stockton | Also posted in Effective Giving, Foundations, Grantmaking, Philanthropic Capital Markets, Philanthropy | Comments (9)

The Foundation Review

February 14, 2008 – 9:19 am

Tactical Philanthropy reader Teri Behrens (director of evaluation at W.K. Kellogg Foundation) is the inaugural editor of The Foundation Review, launching later this year:

I am very excited about this new venture because I believe the time is ripe for a peer-reviewed publication that will provide practical information about what works — and doesn’t work — in foundation-funded programs.  The target audience for the journal is foundation staff and boards, and those who work with them to implement and evaluate programs.

Anyone who works in philanthropy can point to stacks of technical reports about the field of philanthropy, and about the results accomplished by philanthropy — often published by foundations themselves.  However, there is no central place in which to publish these reports, making retrieval and use of the information challenging at best, and typically there has been no external review.

The Foundation Review will seek to fill this niche.  We are calling for articles written in an accessible style, emphasizing what was done and why, and describing the results and recommendations.  Articles should look more like the report a consultant would submit to a foundation or non-profit, rather than the typical heavy-on-theory academic discipline-based article. The topic for the first issue is comprehensive community change initiatives — see the attached for more information.

Please consider submitting an abstract by March 1, 2008

You can find a complete description of The Foundation Review with instructions for submitting papers here.

By Sean Stannard-Stockton | Also posted in Effective Giving, Philanthropy | Comments (0)

More on CalHospitalCompare.org

February 11, 2008 – 9:00 am

Last week I mentioned CalHospitalCompare.org, a website that shows comparative rankings for the effectiveness of California hospitals.

A reader emails today with some thoughts on problems with the measurements used by the site. He points out that the vaunted Cedars Sinai of Beverly Hills has only an “average” ICU mortality rate according to the site. The site assumes that a hospital should strive for a lower mortality rate (less patients dying is good right?):

Why are we pedantically told “lower is better”?  Why is lower better?  Is this not exactly, specifically, the dead-nuts center of the “metrics” conundrum?

If the Cedars ICU mortality rate was 50%, wouldn’t that mean Cedars must be where everyone in the know knows to go when intensively sick??????  Included among them, obviously, are the intensively sick at death’s door — thus inflating the ICU mortality rate with folks whose only hope is for more time in ICU — not escape from it.  “Mortality” measures nothing meaningful about the care and skill and hand-holding services provided AT DEATH’S DOOR in the Cedars ICU!

That 50% ICU croak rate might just mean Cedars is the place to go to be kept ticking long enough for all my family stragglers and long-losts to fly in and come kiss me goodbye?  I’d still add to the croak rate, but…  Wow!  What service!

What the hell kind of “metric” measures that PRICELESS service???!!!

What about the exact opposite “metric” finding:  Suppose Cedars ICU mortality rate was 1%.  This means, of course, if the “average” is 13%, that Cedars is either magical — or lying.  Let’s say they’re just wonderful.  And…  so…  via “metrics” getting into the hands of salesmen…  everyone discovers how wonderful Cedars ICU can be.  So business goes up on the word — logically, in the fullness of time.  Now…  More and more people at death’s door come to them for those precious last few hours or days…  But.  In the end, folks croak.  So?  Little by little, Cedars ICU mortality rate gets back in line with “average” and, like all intended measurements for goodness, is rendered meaningless.

Why, why, why, why so much time on this obsession with measuring The Good?

The emailer then takes me to task for focusing on measuring nonprofit outcomes. I found his argument above excellent. This is a great example of how important it is to measure the right things. We cannot depend too heavily on any one metric. There is no magic, simple way to determine how effective a nonprofit is.

Right now, any decent economist could give you a long list of statistics that show we are in a recession… and a list of statistics that suggest we are not. That doesn’t mean that metrics are worthless. It does not mean we should not strive to seek knowledge. It means that understanding our world is hard. But through understanding our world, we can create a better one.

By Sean Stannard-Stockton | Also posted in Cross-Disciplinary Conversations, Effective Giving, Impact Measurement, Philanthropy | Comments (0)

Do Nonprofits Trust People?

February 11, 2008 – 8:00 am

A new blog I’ve been following, LifeYears, asks an interesting question today:

Non-profits seem sceptical towards outcome-based indicators of how effective different initiatives and projects are. Is this because they are afraid of what “ordinary people” would think and do if they had this information? And if so, could such a worry be rational?

The author, Ole Rogeberg, poses the question very seriously without passing judgment:

Personally, I hope (and want to believe) that outcome-based indicators would make the connection between giving and accomplishing stronger, that people would not just give in order to “do a good deed” but also start giving to “have an impact on malnutrition” or “reduce malaria” etc. But other people might believe that the consequences would be different. But other people might believe that the consequences would be different. Two possible worries that immediately come to mind:
  • Myopic funding: The more concrete, short-term and certain a program was according to its outcome indicator, the more funding it might receive. This might make it easy to get funding for vaccines and malaria nets, but hard to get funding for investments and development in health infrastructure, education, human rights work with women, etc.
  • Over-focused funding: Some measures may catch the public’s interest far more than others (”go viral”), and this might lead to large shares of funds going to these areas - not because the need is the largest or the impact the greatest, but simply because the indicator is catchy in some way.

Ole ends:

Even if you personally don’t see the problem - how can we answer this fear and develop indicators that don’t have excessive (largely unintended) negative consequences?

Important questions. Change is difficult and often has unintended consequences. Many, many people are resistant to my calls for more measurement and analysis of nonprofit outcomes. How can we handle their concerns? Or might their concerns be right?

Powered by ScribeFire.

By Sean Stannard-Stockton | Also posted in Grantmaking, Impact Measurement, Philanthropy, Transparency | Comments (1)

United Way Responds to Google Finance Question

January 31, 2008 – 7:29 pm

Last month I posted a question on the Google Finance profile page of the United Way of America. I got an email today from Meg Plantz, director, Impact Design and Learning for United Way. You can catch up on the Google Finance for nonprofits discussion here.

United Way of America’s approach to program outcome measurement, reflected in its manual, Measuring Program Outcomes: A Practical Approach (United Way of America, 1996), encourages health and human service agencies to develop ways to measure outcomes quantitatively.  This often is a challenge because many human-service outcomes seem at first glance to be un-quantifiable, and many agencies are used to describing their successes with narrative vignettes rather than with numbers.  When they have used numbers, the numbers often have reflected outputs (e.g., number of classes offered, number of clients served) rather than outcomes (e.g., improvement in parenting skills).  However, for purposes of creating and tracking program improvement over time and demonstrating results to increasingly data-oriented funders, donors, and publics, numerical indications of program performance are important.

Our approach consciously responds to the challenges that quantitative measurement presents.  For example, in our approach, intended outcomes do not have to meet the “measurable” test.  In fact, when agencies are identifying what their outcomes are, we encourage them not to worry about how they will measure them, but instead to focus on what results are meaningful to the program and its clients.

Once the agency is comfortable that it has described the appropriate outcomes for its activities, then we advise them to identify measurable indicators of those outcomes.  Questions such as “What will tell you if clients achieve the intended outcome (e.g., improvement in parenting skills)?  What will you be able to see, count, or measure?” help agencies identify critical aspects of their outcomes (e.g., using age-appropriate discipline methods) and think about ways to quantify them (e.g., record observations of parents in role plays and tally entries that parents make in a journal).  Agencies are encouraged to pick indicators that will provide useful data – data that will help them with program improvement, in communications, and in other management tasks.

Qualitative information complements and can help with quantitative measurement.  For example, agencies can use their narrative success stories to identify intended outcomes and then use quantitative measurement as a way to learn whether the stories are unique or are representative of other clients.  Agencies can use their narrative stories to illustrate their outcome data rather than offering the stories as evidence of outcomes.  Qualitative information such as focus groups discussions of quantitative data can help agencies understand the meaning of the data and make appropriate program improvements.

Meg Plantz
Director, Impact Design and Learning for United Way

By Sean Stannard-Stockton | Also posted in Effective Giving, Impact Measurement, Philanthropic Technology, Philanthropy, Transparency, nptech | Comments (0)

Nonprofits Publishing Impact Analysis

January 29, 2008 – 10:14 pm

A nationally known nonprofit has asked me for examples of organizations that are “doing it right” in terms of publishing their impact analysis. In other words, which nonprofits are self publishing information that is useful to donors who are trying to examine if the organization is effective?

Any ideas? Leave a comment or send me an email.

By Sean Stannard-Stockton | Also posted in Impact Measurement, Philanthropy, nonprofits | Comments (7)

Efficient Markets in Philanthropy

January 24, 2008 – 9:25 am

In response to my post yesterday in which I discussed the value of information to philanthropy and why donors should desire efficient philanthropic markets, Phil Cubeta writes:

The logic here can become relentless and destructive. What this tends towards a lists, like league tables in a sport, with the best at the top. It leads then to managing a nonprofit by the numbers, to get the rating, and it leads to shutting down those that don’t rank high. We then have the tyranny of the metrics, however much arbitrariness is built into them…

The world you want - are you sitting in corner office reading a spreadsheet?

So are the philanthropic capital markets I envision boring and lifeless with endless spreadsheets and numbers to crunch? Not in the least.

Economics is often called the “dismal science”. I know that many people think that finance is boring. But the vision of financial markets as nothing but numbers and spreadsheets does not capture the reality. Do investors buy stock in Apple because they spent hours and hours processing spreadsheet calculations? No. While at the end of the day, buyers of Apple stock believe that the return on capital being generated by the company will make for a profitable investment, the information they use to determine that are not just numbers. The way in which Apple has captured the imagination of the consumer, (an intangible piece of data that cannot be added to a spreadsheet) is by far the most valuable asset that Apple has and it is a major reason why investors have flocked to the stock.

Have you ever watched CNBC, the news channel of the financial markets? It is far from some kind of spreadsheet crunching lecture. Every day, investors or all types come on the show and make passionate arguments for why certain companies are good investments. While numbers and calculations underlie much of their thinking, it is the story, the human story of the companies they discuss that take center stage.

Warren Buffet is widely considered the best for-profit investor of his generation. Does he sit in a corner office reading a spreadsheet the way that Phil suggests? The quote below is from noted investor Whitney Tilson (Tilson is a huge fan of Buffet and a fellow columnist of mine at the Financial Times):

If the future were predictable with any degree of precision, then valuation would be easy. But the future is inherently unpredictable, so valuation is hard — and it’s ambiguous. Good thinking about valuation is less about plugging numbers into a spreadsheet than weighing many competing factors and determining probabilities. It’s neither art nor science — it’s roughly equal amounts of both.

The lack of precision around valuation makes a lot of people uncomfortable. To deal with this discomfort, some people wrap themselves in the security blanket of complex discounted cash flow analyses. My view of these things is best summarized by this brief exchange at the 1996 Berkshire Hathaway annual meeting:

Charlie Munger (Berkshire Hathaway’s vice chairman) said, “Warren talks about these discounted cash flows. I’ve never seen him do one.”

“It’s true,” replied Buffett. “If (the value of a company) doesn’t just scream out at you, it’s too close.”

Taking liberties with Tilson’s quote, I would argue that donors should not “wrap themselves in the security blanket of metrics” because “the lack of precision around measuring the impact that nonprofits achieve makes them uncomfortable.”

World-class investors do not sit in their office crunching spreadsheets all day. Neither should world-class donors. But the underlying logic of both should be that of achieving the highest return on investment.

Recently Phil commented to Perla Ni regarding her site Great Nonprofits (which offers reviews of nonprofits written by volunteers, donors and the people served by the nonprofit):

Thank you so much, Perla, for setting the record straight. In fact, your site is the exact opposite of a metrics driven exercise. You are bringing together the voices of those who have been touched by a nonprofit. I finally “got” what you are doing.

An efficient philanthropic capital market does not only view numbers as valuable inputs to the decision making process. Sites like Great Nonprofits offer extremely valuable information to donors. This sort of qualitative information is critical to both donors and for-profit investors. Great Nonprofits is not the opposite of a metrics driven exercise. They are both part of the same process of determining where donors and investors should direct their capital.

Disclaimer: Nothing in this blog should be construed as investment, tax or legal advice. This blog is for informational use only.

By Sean Stannard-Stockton | Also posted in Effective Giving, Grantmaking, Impact Measurement, New Philanthropy, Philanthropic Capital Markets, Philanthropy, Transparency | Comments (7)

Information Sharing in Philanthropy

January 23, 2008 – 11:45 am

I wrote a post a while ago called Paul Brest Needs a Blog (Paul is the head of the Hewlett Foundation). I’ve been an advocate for more people in philanthropy to start blogging in general. In the above mentioned post I wrote:

So why should foundations blog? It seems to me that the imperative is not for them to embrace technology so much as it is for foundations to join and begin to drive the online philanthropy conversation. [But] it is the two-way flow of information that blogs encourage that is important, not blogs themselves.

Even so I’ve noted recently that some people feel that I’ve pushed blogging rather than information sharing. As the conversation we’re all having unfolds I think it is important to take a step back and make sure we haven’t missed the forest for the trees. I wish I had expressed my thoughts with more clarity.
When Phil Cubeta recently asked why nonprofits should blog, astute reader Michele Moon asked:

I’m not entirely sure why it’s blogging, in particular, that’s the focus of discussion, especially because it’s now considered a little bit old-hat, Web 1.5. What is it about the format that makes it so essential to transparency and its tyrant? Is it actually blogging you want to see - personal, real-time updates and editorials, followed (if you’re lucky) by people who read, comment, and sometimes stick around to converse?… Why should it be blogging that we aim to do, or is that shorthand for more complicated online interactivity?

I’m guilty of using “blogging” as short hand for information sharing. I’ll stop making that mistake.

When economists speak about efficient markets they are talking about a situation where money flows to the organizations that can put it to the best use. Widely available, robust information is a critical factor for a functioning efficient market. Recently, in a conversation with Phil Buchanan and other readers on this issue I wrote the following (you can find the full thread here. The Chronicle of Philanthropy recently highlighted the conversation):

In an efficient market, investing is a zero sum game. Maximum returns are generated globally so the only question is matching an investor’s risk/return preferences. In inefficient markets, higher returns accrue to more “effective/smarter” investors. In a public benefit market, since all returns accrue to everyone, investors should desire an efficient market within which they could align their social investments with their personal values/goals.

The philanthropic capital markets are highly inefficient. Far more inefficient than any for-profit marketplace.

Therefore, it seems to me that making the philanthropic capital markets more efficient should be the number one priority of large funders who desire to be effective…

I’m not arguing that the public will make better decisions than the “experts”. I’m saying that efficient markets will produce better outcomes than inefficient markets. In the for-profit world, inefficient markets are great for “expert” investors because they can exploit superior information to generate outperformance of investment returns. But these inefficient markets reduce the total returns in the market by preventing capital from flowing to the best performing investments.

What I’m saying is that unlike in the for-profit market, “expert” philanthropist enjoy no advantage from superior information. The returns they generate accrue to the public, and so no “outperformance” is possible. Instead, they should be interested in the total market functioning at a higher level, since that is the only way to increase the social return on investment that accrues to everyone.

This is the challenge we face as a field. How can we ensure that the $300 billion that is given to charity each year is flowing to the organizations that can put the money to its best use? The key will be our ability to supply market participants with widely available, robust information. Blogs are one tool in this work. There are many others.

By Sean Stannard-Stockton | Also posted in Cross-Disciplinary Conversations, Effective Giving, Foundations, Giving Blogs, Grantmaking, Impact Measurement, New Philanthropy, Open Source Philanthropy, Philanthropic Capital Markets, Philanthropic Technology, Philanthropy, Transparency | Comments (18)

Should Foundations Fund Philanthropic Information?

January 16, 2008 – 1:49 pm

An interesting conversation is beginning to unfold in the comments to Phil Buchanan’s podcast. The point I’m making is not that foundations have some sort of obligation to fund nonprofit information for public use, but that doing so is in their best interest. This conversation ties in directly to the conversation we’ve been having about Google Finance and Google.org.

If a foundation can give $1 that creates $2 of social benefit, or give $1 that spurs the public to give $10, which creates $20 of social benefit, which one should they choose? This ability to give $1 and get $10 of social benefit instead of $2 is the “leverage” that so many philanthropist and foundations say they want to employ.

Here’s the big leverage opportunity of this decade: Provide the individual donors (who every year give seven times more than all the foundations in the country combined) the information they need to make better donation decisions.

Join the conversation with Phil Buchanan and let’s work this problem out!

PS: As background it might be useful for readers to note the essay by Paul Brest, the president of the Hewlett Foundation, in which he discusses “the advantages of good information” in philanthropy. In the essay he mentions Great Nonprofits, whose founder Perla Ni is participating in the conversation with Phil Buchanan. Hewlett is, to my knowledge, the most forward thinking foundation on these issues. Hewlett is also considering funding GiveWell.

By Sean Stannard-Stockton | Also posted in Effective Giving, Foundations, Grantmaking, Impact Measurement, New Philanthropy, Philanthropic Capital Markets, Philanthropic Technology, Philanthropy, Transparency, nptech | Comments (0)

PhilanTech

January 14, 2008 – 9:11 am

Dahna Goldstein, the founder of PhilanTech, sent me the following email. She didn’t post it as a comment because she didn’t want to appear to be plugging her company, but I asked her for permission to share it.

I’ve just caught up on the discussion on your blog about Google Finance and nonprofits, and wanted to share my $.02.  Google is potentially in a unique position — as are you, by virtue of your thought leadership and initiative on this front — to positively affect how information is shared with the sector at large, with donors, and with other interested parties.

The absence of standardized information about nonprofits makes it difficult to suggest a set of metrics or a pre-defined combination of quantitative and qualitative analyses, as a number of your readers have pointed out.  And asking for new types of reporting from nonprofits risks placing an additional burden on already-burdened nonprofits.

PhilanTech has taken a step towards addressing this issue.  We created the PhilanTrack online grants management system to centralize and streamline the grants management process — creating centralized reporting about nonprofit organizations, activities, outcomes, and finances.  Our vision is for a centralized reporting for all donors (institutional and individual) and other interested parties (researchers, the nonprofits themselves, etc.) to obtain information to inform funding decisions — without creating additional hoops through which the nonprofits must jump.  I’d be happy to tell you more about how it is structured, but in a nutshell, PhilanTrack helps foundations request the information they want to receive from nonprofits (to help them evaluate their effectiveness as grantmakers and to help evaluate potential grants) while helping nonprofits avoid reinventing the wheel each time they report to a different funder.  The types of information that the system manages (activities, outcomes, finances, lessons learned, etc.), I believe, are the types of information that Google should consider posting about nonprofits.

While expense ratios, as you have pointed out, have significant shortcomings, there is still a lot that can be learned about the financial health and stability of an organization (if not its effectiveness) by looking at its finances.  It requires looking beyond CharityNavigator and beyond 990 data in ways that are not familiar to many (both individuals and institutions) who are considering gifts to nonprofits.  At PhilanTech, we have addressed this issue by developing a financial analysis tool that uses basic financial statements to provide six analyses (financial mix, efficiency, debt servicing ability, liquidity, long-term viability, profitability) with a number of different metrics and explanations of why each metric is important and useful in evaluating the financial health of a nonprofit.

Google Finance pages (or Google Knol) should, in my view, combine these financial analyses with the following:  quantitative information about outcomes (where available and qualitative where quantitative info isn’t available); qualitative information about activities, programs/projects, mission, people, sustainability, replication (where relevant), lessons learned, challenges faced and overcome; related organizations (including any partnerships/collaborations); news; funders; discussion (like the type of discussion you prompted about the Red Cross), and perhaps something like the 360 degree views GreatNonprofits is working to create.  And there are ways it could be done without placing too great a burden on nonprofits by leveraging some of the reporting nonprofits are already doing.

By Sean Stannard-Stockton | Also posted in Effective Giving, Impact Measurement, Philanthropic Technology, Philanthropy, nptech | Comments (0)