What impact data should charities collect and report?

Outcomes are the positive, usually immediate or short-term differences that result from a charity's interventions. This includes any benefits felt directly by the people the charity supports, such as improved confidence, self-esteem, health or wellbeing. Impact is a term often reserved for the sustained, broader difference that a charity achieves for the people it aims to support or wider society.

That said, outcomes and impact are often used interchangeably and they go hand in hand. A strong impact report will contain evidence of key outcomes.

Theory of Change has multiple benefits and informing impact data collection is one of them. A well constructed Theory of Change with clearly defined outcomes (and other key elements such as mechanisms) is the perfect basis for producing an impact measurement framework. Rather than measuring what's easiest, or what's always been measured, it puts impact data on a strategic, purposeful footing.

Choosing the right measures is a key part of the impact data process. Choose measures that will resonate with your audience (for example policy makers) and that the people you support feel are meaningful and reflect their priorities. Standardised measures such as the ONS's harmonised questions are a good place to start. But think also about whether measures need adapting to be accessible to your audience. Draw up a shortlist then consult widely and test them out.

Three big risks are data protection breaches, flawed evidence, and causing harm. But they can all be managed through good practice and a bit of expert advice.

Collecting impact data nearly always involves asking people about their experiences. This is likely to mean processing personal data, often about vulnerable individuals (this might be the case even if you are not collecting names or other direct identifiers). It also means asking people to reflect on challenges and upsetting experiences in their lives, so there are ethical issues to consider.

Collecting impact data takes time and resources and the worst outcome is ending up with seriously flawed evidence. Careful sample selection, unbiased questionnaire design and correct use of statistical methods all play a part.

In our experience, impact takes time to achieve and rarely gets much bigger or smaller from one day to the next (unlike feedback for example). It's far more important to measure impact robustly and convincingly. That means careful planning, building up enough data and different types of evidence, and summarising it objectively. That kind of impact data will have a long shelf life.

Some charities report impact in a real time dashboard. This can be very engaging and motivating. But it's not always possible or necessary. At the other extreme, many charities relegate impact data to a corner of their website or their annual report and accounts. We recommend having a dedicated impact report - this helps potential funders and the people you support find this key data, and it sends a strong signal about the importance a charity places on impact. We've developed a 20 point impact reporting checklist to help.

We can help at every stage from Theory of Change development, to measurement choices, data collection and reporting. Please get in touch to discuss your needs - we'd be happy to have a free initial discussion about how best to make a start.