Funders increasingly want to see evidence of impact, but what does 'impact' really mean for the thought-leaders in this field? Jenny Ramage speaks to NPC, Garfield Weston Foundation and others to find out.

Today, much more than in the past, funders want to know about the impact their investment is making. This is understandable - the pot of money for charities is limited, and so you’ve got to ensure that money works as hard as it can. But the concept of impact investing is an early science, and one that is currently plagued with more questions than answers.

For a start, different funders have different views on what impact means, as Dan Corry, chief executive at New Philanthropy Capital (NPC) points out: “Some funders just want to get some kind of feel for what their grantees are doing, whereas others want the grantee to have done some quite complicated follow-up longitudinal work”. And there is any number of variations in between.

This, he says, can give rise to problems for under-resourced charities - particularly where several funders are involved in a project, all asking for different measures of impact. A practice that is, in Dan’s words, “completely nuts”.

The problem with the numbers game

Demonstrating impact is often conceptually problematic, too, especially if you’re a charity that can’t easily count its outputs in simple terms of number of people helped or trees planted.

Cathy Pharoah, professor of charity funding at Cass Business School, thinks all this rhetoric around explicitly identifying impacts and outcomes may be misplaced. “Academic experience has shown how very hard it is to identify those single factors that indicate change, and that where it is possible, it’s usually complex, costly and time consuming.”

For those charities trying to tackle complex social issues with limited skills and resources, this can be a nightmare. A youth offending charity, for example, because of the various factors influencing crime rates, is going to find it very hard to make a very noticeable impact on statistics, even at the local level.

“Even though they may be doing a huge amount of effective work, there may be other factors in play that are simultaneously exacerbating a particular problem - like prisons getting more crowded, or the cutting back of benefits to young people.” Cathy points out.

Softer outcomes and self-reporting

Cathy says that in many cases, there are much easier ways of demonstrating impact than trying to see in very hard, quantitative terms whether you’ve had any effect on statistics at the macro level. “Self-reporting, for example, can be a very effective way of demonstrating impact - whether you’re collating beneficiary survey results or more narrative accounts of how they’ve benefited.”

Sally Booth, new fundraising manager for the HELP (Helicopter Emergency Landing Pads) Appeal, has a long history in both corporate and private funding, and has tackled the question of impact measurement from many angles. In her experience, there is always some kind of measure you can put in place. “If you find the quantitative stuff more difficult to gather, then you can always get anecdotal evidence to give you the qualitative data. In some respects this can be just as important for funders - if not more so.”

Such anecdotal evidence can be drawn from any or all the sources feeding into the work you’re doing, says Sally. “If, for example, you’re helping support school students with their reading, you’d want to measure how the paired readers were developing their reading and self-confidence skills. You could do this by asking the children to tell a story about what it means to them to have a reading mentor; you could speak to the classroom teachers to determine what they think about how the children were benefiting; you could even speak to the reading mentor to find out how they think they are helping that child develop.”

Being realistic and proportionate

There are certainly moves towards funders becoming more accepting of these ‘softer’ ways of showing impact. Garfield Weston Foundation places a lot of emphasis on impact but also believes it “has to be seen in the context of, and proportionate to, the type and size of organisation we’re funding, and be relative to the nature of the work that’s being done”, according to the foundation’s director, Philippa Charles.

“If we are funding, for example, a volunteer-led, practical handyman’s service for the elderly which includes an element of befriending, it would seem inappropriate to launch a very large-scale, randomised, controlled trial study on this kind of organisation in order to compare it with the town next door that doesn’t have the same type of intervention.

“While it might be possible to do a complex costs analysis over a long period, in general, I think we can all see what the benefits will be of such a service, and we also just know that these things work.”

Rather than use a standardised questionnaire for applying charities, the foundation provides clear guidelines setting out the kind of initial information it’s looking for and later, particularly with the more complex cases, “delves into that information with the charities individually and on a case-by-case basis. We then try to adjust our approach to be pragmatic.”

Investing in measuring

Dan Corry thinks that if a charity is being told by the funder that they need to measure impact in a particular way, it would be helpful for that funder to then supply them with the tools to do so. “Sometimes charities will say the funder wants them to do a certain form of impact measurement, but won’t give them any funding for it. Sometimes the charity simply hasn’t got the skills or resources to do it themselves.”

He thinks that the solution needs to come from both sides: “I think charities need to be a bit firmer, and to state in their grant application that they want to include some metrics in their bid; meanwhile the funders themselves have got to recognise that if they care about this, it’s worth them spending a bit of money and that ultimately it means they’ll be spending their money better.”

Being realistic

However you decide to measure your impact, Cathy Pharoah thinks it will come much more easily to those charities who set realistic goals. “Very often, charities set goals in hugely optimistic and visionary terms, whereas they really should set modest and realistic goals whereby they say, ‘we’re not trying to change the world, we’re trying to get these children in this school here to achieve better exam results’, for example.

“It’s then about stating how you think you’re going to get there, discussion of whether you’ve got the right steps in place, and having a good rationale. You can then look at what you’ve achieved and assess your progress towards your goals.

“I think that’s much more important than trying to use indicators to measure impact which may be quite far from what you were actually doing.”

The future of impact measurement

The impact debate is set to continue for quite some time, and NPC intends to maintain its position as a thought-leader in this field. “We at NPC will always argue for impact measurement”, says Dan, “but as a sector, we’ve got to find ways to be sensible and proportionate”.

Some of NPC’s current work is focused on encouraging funders to come together and try to figure out a shared system of impact measurement. “Not only would this make it easier for the charities, but if you start asking other charities doing similar work for the same kind of information, you can start comparing what they are doing - and then we can all start learning from each other.” He says there’s still a long way to go, but that “on the whole, things are headed the right way - and I’m pleased that we’re on this journey.”

Meanwhile, Cathy urges those pushing for better impact measurement to be realistic in their expectations. “There’s a huge history of academic science behind impact measurement, and charities are only at the beginning of the learning curve at the moment. Sophisticated measurement is beyond most of them, in terms of both skills and resources”.

Cathy does, however, share some of Dan’s optimism about the movement, feeling it “will make lots of charities look at what they’re doing and construct a stronger case for the difference that they make to lives.”

Keeping up a dialogue

Cathy suspects that eventually, the rhetoric about impact will give way to a debate focused more on what charities can do with the resources they’ve got, the setting of more realistic goals, clear dialogue about how they are going to get there, and acceptance that measurement of change is a very inexact science. “I think we will end up with a sector that is much clearer about what it’s doing and where it’s most likely to have an impact, and that is able to talk more honestly about its failures.

“The reality is, charities can do a huge amount to make people’s lives better, even if tackling society’s deep-rooted problems in the long-term is largely beyond the sector’s small resources”, she concludes.