Meredith Niles provides a funder’s perspective on why impact measurement matters

It’s no secret that the competition among charities for funding has become increasingly intense. Public sector and charitable funders are demanding greater evidence of impact from the organisations they support.

Despite a clear business case for greater emphasis on impact measurement and evaluation (“M&E”) many small and medium-sized charities struggle in this area. Common questions include: “What systems should we be using?”, “How much time and money should we/can we spend on this?” and in some organisations, the more fundamental question: “Why are we doing this again?”

At Impetus Trust, we spend a lot of time thinking about impact. It’s a subject that is important to us for three reasons.

1. Like a significant and growing number of other funders, our decisions about which organisations we back are influenced by the ability of those organisations to demonstrate the difference that they make.

In fact, our investment decisions may be even more influenced by impact reporting than many other funders as we have no endowment. This means that our ability to carry out our work is dependent on raising several million pounds each year from our own donors: a mix of grantmaking trusts, corporations and individuals.

2. Moreover, our donor base is particularly concerned with understanding the difference that their money is making, which in turn means that we have to be especially focused on measuring and communicating our impact.

Time and again, our donors tell us that a major reason they decided to support us was the knowledge that their funds would be well managed – and this in no small part because of our commitment to measuring outcomes.

3. Because our primary activity is helping other charities and social enterprises to scale up their operations, we can only demonstrate the impact of our work if we can get good quality impact data from our investees.

This means we commit a substantial level of resource to helping our investees raise their game with respect to impact measurement and evaluation. Most of our investee charities have annual turnover in the range of £500,000 to £2m when we start working with them, and they all have relatively lean staff teams. These aren’t massive organisations with huge infrastructures to support complex M&E programmes, so we have to help them be smart about how they design their systems.

Theory of change

When we’re assessing an application, we don’t expect the charity to have perfect evidence of their impact already, but we do expect them to have a well-developed theory of change that makes a logically sound link between the work they do and the results they hope to achieve. For example, when we were looking at applications for our reducing reoffending initiative, we knew it would be unrealistic to demand that relatively small organisations, working in a sector where sometimes sensitive data is hard to access, would have been able to prove that their intervention alone could produce a statistically significant reduction in reoffending.

We did expect applicants to be able to make a coherent case, pointing to research about the causes of offending behaviour and what contributes to desistance, showing that their intervention produces outcomes linked to a reduction in the likelihood of their clients reoffending. We also required applicants to have systems in place to monitor their activity to ensure they were doing what they said they do.

In addition to a strong theory of change, we look for a cultural commitment throughout all levels of the organisation to improving their M&E processes. We want to know that the organisation isn’t interested in M&E purely to please its funders, but because they are genuinely committed to learning what about their service works and what could be improved so that they can be as effective as possible.

Measuring what matters

Once we’ve chosen to support an organisation, we work closely with them to help them build their skills and capacity for improved impact measurement. We invest a lot of time upfront agreeing what measures are really important in understanding the social value that is being created; just because something can be measured doesn’t mean it’s worth someone’s time to track it and report on it.

At Impetus, we usually focus on no more than two or three key social impact metrics when we assess the performance of our portfolio charities, but we spend a lot of time with our investees thinking about what these should be and co-developing tools to track them. We want to measure what matters.

When we’ve agreed what the charity will measure, we then work out how this will be done. We don’t think there’s a perfect one size fits all tool out there, so we encourage our investees to use the tools that are most comfortable for their organisation. This might include a mix of data collected from surveys, focus groups or third party datasets, as well as quotes and case studies.

We also try to help our investees think about their current and future stakeholders and what kind of information and evidence they might require. A charity might receive most of its funding today from donors who believe so much in its work that they don’t demand significant evidence of impact. However, as their operation grows, they may want to be in a position to attract new funders, potentially through government commissions.

Public bodies (and their prime contractors) will often require a much higher standard of evidence and may offer terms to which no charity should sign up unless they have a high degree of confidence in their ability to achieve the required results. We help our organisations develop their understanding of what will be required of them by these potential future stakeholders, and then we help them produce this evidence.

Nobody’s perfect

We try to be realistic about what’s achievable and not to let “perfect” be the enemy of “really good”. Sometimes, it is simply not possible for a charity to collect all the evidence it needs for a “gold standard” evaluation of its service for one reason or another. For example, a charity might lose contact with its clients before it is able to make an assessment of the long-term outcome of its work. In this case, we might look for alternative ways to collect information about the clients. We could also try to help the investee develop intermediate outcomes associated with the long-term outcome that they are trying to achieve, which can then be measured and used as a proxy. We make the best of the information that we can access, acknowledge the limitations of what we can achieve, and move on with confidence, knowing we’ve done the best that we can.

Stepping back

In our experience, organisations are often reluctant to invest in M&E. Often this is due to a well-intentioned but misguided attempt to maximise the funds available for front-line work with clients.

Why do I say misguided? Imagine your organisation has developed a promising experimental treatment for a rare disease, and now you need to develop a budget for testing whether the treatment is effective. No one would take seriously the claim that it would be better to allocate your resources at this stage simply to “more medicine” in order to treat more patients, without having a clear understanding of whether the treatment actually made people better (and certainly without establishing that it didn’t make patients sicker). Your charity is treating a social problem: don’t you owe it to your clients to make sure that you know the medicine you’re offering is safe and effective? What if through your evaluation, you learned that a simple change to what you’re doing could allow your work to have even greater impact, or reach even more people, for the same spending? Just doing “more of the same” without taking a step back and evaluating the results you’re achieving would never provide an opportunity for improvement.

Ultimately, improving your services and resource allocation so that you can make a bigger difference with the money you have is the real reason that organisations should invest in impact measurement. By building a culture of continual assessment and improvement, combined with honest and transparent communication with your stakeholders, you’ll naturally attract more funding as well.

How to communicate impact measurement and evaluation to your funders

1. Approach your funders as collaborative partners.

Treat conversations with funders as a constructive dialogue. Many will have a standard reporting template that they ask their grantees to complete on a regular schedule. However, that doesn’t mean that you can’t and shouldn’t have a conversation with them about reporting in a different way if you think it would make more sense for your organisation. Perhaps you have a regular data collection schedule that doesn’t match the reporting timeframe they’ve requested, or maybe they’ve asked for data that is difficult to collect or that you don’t think is particularly relevant to your work. Most funders would be quite happy for you to suggest amendments to their reporting framework on a reasonable basis such as this, although some funders have specific metrics on which they need to report for their own stakeholders, so you should make an effort to be respectful of these and try to accommodate them where possible. It’s a two-way street.

2. Plan ahead.

The best time for your first conversation with a funder about how you’ll report back to them is at the time the grant is made (or even before if the funder is willing to engage at this stage), not the day before your first report is due. It’s hard to recreate data you haven’t been tracking from the start, so getting buy-in on both sides as early as possible about what will be measured is crucial.

3. Build it into your budget.

Evaluation doesn’t have to be expensive, but it’s never entirely without cost. Most funders expect that there will be a line item in your funding proposal to them for evaluation, so be sure that you build it into your plans.

4. Keep it proportional.

In their quest to impress, some charities overpromise what they can deliver in terms of impact evaluation. Think about what’s really called for from the particular situation and what is realistically achievable. Most funders will accept a level of reporting that is consistent with the scale of the commitment they have made to you.

Meredith Niles is an investment director at venture philanthropy organisation Impetus Trust

This article first appeared in The Fundraiser magazine, Issue 15, March 2012