The GuideStar Blog retired September 9, 2019. We invite you to visit its replacement, the Candid Blog. You’re also welcome to browse or search the GuideStar Blog archives. Onward!

GuideStar Blog

Measuring Nonprofit Impact, Part I: Spotlight on Philanthropedia

measuring-nonprofit-impact-philanthropediaWith GuideStar's recent acquisition of Philanthropedia, some of you may be wondering what exactly Philanthropedia does. We'd like to take this opportunity to introduce ourselves and tell you more about our work.

Philanthropedia was founded as a nonprofit in 2009 with the mission to improve nonprofit effectiveness by directing money to and facilitating discussion about expert-recommended, high-impact nonprofits. Our vision was to inspire everyone to give with impact. In our perfect world, all experts would openly share their knowledge about nonprofits, donors would give to high-impact nonprofits in a social cause they care about, and all nonprofits would improve their practices because of increased transparency, accountability, and discussion.

The idea for Philanthropedia came to one of our colleagues, Howard Bornstein, when he was working on a consulting project at the Gates Foundation. A friend asked him to recommend a great education nonprofit to which she could make a donation—after all, the Gates Foundation was well known for its education grantmaking. Howard realized that he had unique access to some of the world's leading experts on the education nonprofit sector and wondered how he could share that resource with others.

One of the hardest things to measure in the nonprofit sector is the impact an organization is having. So to date, most nonprofits haven't focused on measuring it. Finances (or numbers), on the other hand, are easy to measure and have, for a long time, been the go-to measurement and assessment of nonprofit success. Because financial ratios and fundraising efficiencies cannot be correlated with impact or outcomes, however, we believe they are simply the wrong assessment tool if you, a donor, are actually interested in making your donation go the farthest. So we asked ourselves, how can we help nonprofits measure the impact their work is having? We wondered, who knows about the great work nonprofits are doing? How can we leverage that expertise to help inform individual donors, like us, on the outside?

Our solution was to turn to professionals working in the sector who we believe are incredibly knowledgeable about the work that's going on in their respective fields. These experts have access to private data and information that's not publically available. We believed that by asking them a series of targeted questions, we could unlock some of that previously private knowledge and share it with the rest of the world.

The concept of consulting with a topical expert is not new—journalists regularly interview experts to learn more about the fields they're researching. So here and there, experts are able to share their advice. What we hadn't seen, however, was the wide-scale solicitation of experts' perspectives. So, most experts will go about their lives keeping this knowledge locked inside their heads, because most aren't asked to share their thoughts. Therefore, using the modern concept of crowd-sourcing, we devised a methodology to tap into the knowledge of a diverse, representative, and independent sample of experts.

With our crowd-sourcing methodology, we survey hundreds of nonprofit sector experts to find out which nonprofits in their field they think are having the most impact. These experts are funders, academics, researchers, nonprofit leaders, policy makers, advocates, consultants, government officials, and more. In an open-ended format, we ask them to recommend up to four nonprofits that they think have had the most impact in their specific fields. We also require a justification for each recommendation. We ask the experts to provide the evidence of impact on which they're basing their recommendations. We ask what other organizational strengths (such as leadership, management, finances, etc.) each of the recommended nonprofits has. And last, we ask what each organization could do to improve further.

We have found that no matter what the cause (education, homelessness, climate change, etc.), our results look the same. A small (10-20) number of nonprofits are recommended a lot, and a lot (100-200) of nonprofits are mentioned just once or twice. So it is quickly apparent where the consensus lies among even a diverse group of experts.

Of course, like any methodology, our process has limitations. Because the responses are open-ended for experts, we can't guarantee to collect a review on every organization out there. This means that we will naturally miss some great organizations. Additionally, because we are asking experts to donate their time to participate in this research, we cannot go as in-depth in the reviews as we might like. Therefore, we cannot offer a completely comprehensive review of an organization. Nonetheless, we believe we are able to unlock the knowledge of sector experts in a way that has never been done before. By sharing this information with the public, we can help donors learn more about the great work nonprofits are doing and, we hope, direct more money to those outstanding organizations.

We bet you still have a lot of questions about our work and how we do things. In future articles, we'll address some frequently asked questions about such things as how we define impact, what makes an expert, how we find experts, what questions we ask experts, how we come up with our rankings, etc. We hope you will stay tuned to learn more!

Read More in the Series

Read Part II, The Value of Experts

Read Part III, A Deeper Look at Philanthropedia's Experts

Read Part IV, Philanthropedia's Survey Process

Erinn Andrews, Philanthropedia
© 2011, Philanthropedia

Erinn has been the chief operating officer of Philanthropedia since the nonprofit's inception in June 2009. She was primarily responsible for developing and scaling Philanthropedia's methodology and conducting the social cause research. As a new member of the GuideStar family, Erinn, now Director of Data, Research, and Partner Relationships will continue to oversee Philanthropedia's research but is assuming new responsibilities within GuideStar, as well.

Topics: Impact