EVPA Measuring And Managing Impact A Practical Guide
User Manual:
Open the PDF directly: View PDF 
.
Page Count: 140 [warning: Documents this large are best viewed by clicking the View PDF Link!]
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION JUNE 2015
A PRACTICAL
GUIDE TO
MEASURING
AND MANAGING
IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
2A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
www.fsc.org
©1996 Fore st StewardshipCouncil
Cert no.SA-COC-001530
Mixed Sources
Product groupfromwell-managed
forestsand othercontrolle dsources
Published by the European Venture Philanthropy Association
This edition June 2015
Copyright © 2015 EVPA
Email: info@evpa.eu.com
Website: www.evpa.eu.com
Creative Commons Attribution-Noncommercial-No Derivative Works 3.0.
You are free to share – to copy, distribute, display, and perform the work
– under the following conditions:
• Attribution: You must attribute the work as
A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
Copyright © 2015 EVPA.
• Non commercial: You may not use this work for commercial purposes.
• No Derivative Works: You may not alter, transform or build upon this work.
• For any reuse or distribution, you must make clear to others the licence
terms of this work.
Authors: Dr Lisa Hehenberger, Anna-Marie Harling, Peter Scholten
Design and typesetting: Pitch Black Graphic Design The Hague/Berlin
ISBN 9789082316087
This publication is supported under the EU Programme for Employment and
Social Innovation “EaSI” (2014–2020).
For more information see: http://ec.europa.eu./social/easi
The information contained in this publication does not necessarily reect the position
or opinion of the European Commission.
With the nancial support of the European Commission.
JUNE 2015 3
A PRACTICAL
GUIDE TO
MEASURING
AND MANAGING
IMPACT
DR. LISA HEHENBERGER, ANNAMARIE HARLING AND PETER SCHOLTEN | JUNE 2015
EVPA is grateful to:
Acanthus Advisers, Adessium
Foundation, BMW Foundation
and Omidyar Network for their
structural support
EVPA is grateful to:
Fondazione CRT for the support
of its Knowledge Centre
4A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
JUNE 2015 5
Preface
Executive Summary
Part 1:
Introduction and Overview
1.0 Introduction and Overview
1.1 Background
1.2 How is social impact currently
measured by social investors and
venture philanthropists?
1.3 Five-step framework
1.4 Methodology
1.5 Denition of social impact
Part 2:
The Impact Measurement Process
2.0 Step 1: Setting Objectives
2.1 What?
2.2 How to?
2.3 Practical tips
2.4 Recommendations for managing
impact
2.5 Worked Example
3.0 Step 2: Analysing Stakeholders
3.1 What?
3.2 How to?
3.3 Practical tips
3.4 Recommendations for managing
impact
3.5 Worked Example
4.0 Step 3: Measuring Results:
Outcome, Impact and Indicators
4.1 What?
4.2 How to?
4.3 Practical tips
4.4 Recommendations for managing
impact
4.5 Worked Example
6
14
28
29
30
30
31
33
34
36
37
37
37
46
46
46
48
48
49
54
54
55
57
57
60
70
70
71
76
76
78
84
85
85
86
86
87
93
93
94
95
101
102
103
109
116
121
126
132
133
137
5.0 Step 4: Verifying & Valuing Impact
5.1 What?
5.2 How to?
5.3 Practical tips
5.4 Recommendations for managing
impact
5.5 Worked Example
6.0 Step 5: Monitoring & Reporting
6.1 What?
6.2 How to?
6.3 Practical tips
6.4 Recommendations for managing
impact
6.5 Worked Example
7.0 Managing Impact
8.0 Conclusions
Part 3:
Case studies
9.1 Case Study:
Ferd Social Entrepreneurs investing
in The Scientist Factory
9.2 Case Study:
Impetus Trust investing in Blue Sky
9.3 Case Study:
Oltre Venture investing in PerMicro
9.4 Case Study:
Esmée Fairbairn Foundation investing
in Social Impact Partnership
(developed and run by Social Finance)
9.5 Case Study:
Auridis investing in Papilio
Part 4:
Appendices
10.0 Glossary of Terms
11.0 Sources
TABLE OF CONTENTS
6A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
Preface
JUNE 2015 7
PREFACE
Introduction to the second edition from Lisa Hehenberger,
Research & Policy Director of EVPA
This is the second edition of A Practical Guide to Measuring and Managing Impact
(“the Guide”), rst published in 2013. In what follows, we will provide a brief update of
the uptake of the Guide, the remaining challenges that practitioners face, the contribution
of the guide to policy work, and nally what EVPA’s future plans are in terms of research
on impact measurement and management.
When we started developing the Guide in 2011, we responded to the need of EVPA members
for more clarity in terms of impact measurement. We had noticed that the problem was not
the lack of information, but rather the absence of guidance in how to make sense of the
information on impact measurement. Therefore, we engaged in a meta-analysis of almost
1,000 different approaches as included in resources such as the TRASI database1 curated
by the US-based “Foundation Center”. From these approaches, EVPA, informed by the
convening of an Expert Group of twenty-seven venture philanthropy and social invest-
ment practitioners, consultants, academics and representatives of other organisations
involved in impact measurement, selected the most commonly used approaches and then
further distilled these approaches into a ve step process. The objective was to derive the
commonalities between various approaches to come up with clear recommendations on
how to measure impact.
We discovered during the process that the most important aspect of impact measurement
is not the actual value or numbers you obtain from the exercise, but the integration of
an impact approach in the organisation so that impact becomes an integral part of the
entire management or investment process. By undertaking and learning from the process
of measuring impact, an organisation can work more effectively towards achieving societal
impact. That is why we moved from working on just measuring to also managing impact.
Uptake of the Guide and recent developments
The EVPA guide has been well received, with over 2,000 copies downloaded since its
launch in April 2013 and more than 500 hard copies distributed. It has been translated
to Swedish, Spanish and French. As shown by the results of EVPA’s 4th Industry Survey2,
an increasing number of organisations are using the ve steps of impact measurement
outlined in the Guide.
1. http://trasi.foundationcenter.org/
2. Hehenberger, L., Boiardi, P.,
Gianoncelli, A., (2014), “European
Venture Philanthropy and social
investment 2013/2014 – The EVPA
Survey”, EVPA.
8A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
PREFACE
Additionally, the survey shows that the large majority of respondents are measuring
outcomes and trying to assess the impact of its activities, pointing to the importance the
practice has for VP/SI practitioners, and their increased sophistication in the use of the
practice.
The objective of our best practice research is to increase the effectiveness of practitioners and
we see the results as encouraging, although direct attribution to EVPA’s work is difcult
to claim. Many challenges remain for both funders and investees who still consider impact
measurement a complex and technical practice. However, we do believe that the Guide and
the dissemination and policy work around it have contributed to raising awareness for the
topic of impact measurement and management in our sector.
EVPA and its members are being recognised as important actors in the practice of impact
measurement. EVPA’s work on impact measurement is being referenced in the European
Commission’s Standard3 on impact measurement, and we have participated in and contrib-
uted to the report4 produced by the Working Group on Impact Measurement of the taskforce
on social impact investment established by the G8.
In terms of the European Standard, when the Guide was in its nal stages, the European
Commission set up it Sub-group on Impact Measurement to advise the Commission on
the topic. EVPA participated in the sub-group (and in GECES) and presented the ve steps
Outputs Outcomes Impact
84 87
70
Objectives of social impact
measurement by %
of respondents
n=91
multiple choice
numbers in %
3. http://europa.eu/rapid/
press-release_IP-14-696_
en.htm?locale=EN
4. http://www.
socialimpactinvestment.org/
reports/Measuring%20Impact%20
WG%20paper%20FINAL.pdf
% of VPOs that perform
each step of societal
impact measurement
FYs 2011 and 2013
2013 n=92
2011 n=57
numbers in %
0 20 40 60 80 100
84
72
Step 5
72
72
Step 4
90
88
Step 3
73 79
Step 2
90
86
Step 1
JUNE 2015 9
PREFACE
of the Guide during one of the meetings. The experts in the Sub-group agreed that the
European Standard of Impact Measurement should be set at the process level (adopting
EVPA’s ve steps) and not at the indicator level (indicators can only be standardised at
social sector level and they have to be chosen in accordance with relevance to the social
organisation measuring impact). The European Standard was formally adopted in June
20145 and the report can be downloaded for free6.
Another important development for our sector has been the study conducted by the
Working Group on Impact Measurement (WGIM) of the Taskforce on Social Impact Invest-
ment established by the G8. The WGIM was composed of experts from the G8 countries
(later to be G7) as well as representatives of the European Commission and OECD. WGIM
built on the work of both EVPA and GECES, and extended it by including specic steps for
data collection and data analysis.
Considering the uptake of the Guide and its contribution to European and Global standards
as outlined above, we are condent that the Guide reects best practice globally as it
currently stands today. Core principles that have come out of these work streams, and that
will guide our work on impact measurement going forward, are as follows:
•Impact measurement has to be relevant to the organisation measuring so that it becomes
part of their management system and helps them improve their work to achieve greater
impact.
•Impact measurement also needs to be proportionate to the organisation at hand, keeping
in mind that it is a means towards an end, not an end in itself.
Attempts are being made at standardising social impact measurement indicators, across
social sectors and on broad levels, leaving room for some local adaptation at project-level.
Several databases (e.g. IRIS, Global Value Exchange) exist that have collected key perfor-
mance indicators commonly used. Reporting standards are already being developed by
social investors in cooperation with investees in many parts of Europe (e.g. Social Reporting
Standard in Germany). New and more sophisticated tools (e.g. Sinzer, PULSE) have been
created and are being developed to support practitioners in measuring and managing
impact.
Next steps for EVPA
Although the eld has come a long way since EVPA held its rst workshop on performance
measurement in 2007, EVPA members are still in need of additional training and guidance
on impact measurement. The lack of benchmarking measures, the absence of standards in
terms of evidence needed, and the lack of data of the impact of funders on their investees
are a few frequently mentioned issues. We particularly see the need to make the research
even more hands-on, practical and relevant with concrete case studies that run through
the impact measurement process in a VPO. Furthermore, EVPA’s guide should be seen as
5. http://europa.eu/rapid/
press-release_IP-14-696_
en.htm?locale=EN
6. http://ec.europa.eu/social/main.
jsp?catId=738&langId=en&pubId
=7735&type=2&furtherPubs=yes
10 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
PREFACE
an evolving document that incorporates new developments and provides up-to-date and
practical guidance. Therefore, we have decided to invest further in research on the topic.
Concretely, the next steps include conducting and writing a number of in-depth case studies
to be published in a separate report on how to measure and manage impact in practice, to
be further developed as training material.7 This research will provide VP/SI practitioners
with practical real-life and in-depth cases of how impact measurement can be performed.
The exploratory case studies will help us revise the Guide in 2016, based on ndings
regarding what works/what doesn’t work and what organisations are struggling with
using the ve steps proposed in our Guide. The case studies are not a means for EVPA to
“prove” its 5-steps, but a way to reect on what organisations are struggling with when
measuring impact. We also aim to include, where relevant, any new, upcoming method-
ologies in impact measurement and emergent issues (e.g. how to evaluate outcomes, issue
of proportionality, levels of evidence needed and use of control groups, measurement
standards that allow comparability, etc.).
Other plans include developing a micro-site on impact measurement, to develop a
community of practice, making research on impact measurement a process of continuous
learning that builds on existing knowledge and on the experiences of VP/SI organisa-
tions. The micro-site will help EVPA collect knowledge and best practices around the topic
and make it available for practitioners in a user-friendly way. Through the community of
practice we will also collect experiences and practical cases to help EPVA and its members
upgrade and revise the learnings.
It is our aim that the research on impact measurement and management will allow us to
provide even more practical guidance that will facilitate the work of VP/SI practitioners
and the investees they support. We also intend to build on EVPA’s reputation as a leading
actor in terms of setting standards in the VP and Social Investment industry, and thus
reduce problems of multiple standards in impact measurement that increase the work
of both investor and investee. And we should never lose track of the overall purpose of
impact measurement; to help both funders and investees work towards greater impact –
while being relevant and proportionate.
Lisa Hehenberger
Research and Policy Director, EVPA
7. A shortened and simplified
version of the case studies will
then be used for training purposes
at a later stage.
JUNE 2015 11
PREFACE
Introduction from Daniela Barone Soares,
Chief Executive of Impetus Trust
What’s Impact Measurement for? Ask a social purpose organisation (SPO), and they’ll tell
you it helps them prove that what they do makes an impact, gives funders reassurance that
their money is well-spent, and provides the stories and case studies they need for further
fundraising. They might add at the end that the data helps them rene and improve their
programmes, and inform their decision-making.
Venture Philanthropy Organisations (VPOs), like Impetus, work for a social sector where
the work of impact measurement is driven by the need to extract maximum value from our
nite resources. Where SPOs stop doing things that don’t work, even if funders love that
project. Where new projects are explicitly based on learnings from previous work, and bear
the imprint of past successes and failures.
“Managing impact” might not be a phrase to set the world on re but we believe the
benets of embedding the concept across the social sector would be transformational – and
immediately tangible. SPOs are often experts in the needs of their beneciaries, but lack
the data on their own activities to make informed resource allocation decisions, or build an
organisation that really plays to its own strengths. Data doesn’t just reveal impact – it is a
prerequisite for making impact. It’s also the mother and father of innovation. Innovation
isn’t just about ‘new’; it has to be about ‘better’. Data reveals where an SPO could do better,
and tells them when they’ve got there.
This Practical Guide to Measuring and Managing Impact is a timely resource with a wealth
of much-needed information for Venture Philanthropy Organisations (VPOs), and the SPOs
they back. It’s one of the many reasons we’re proud to partner with the EVPA Knowledge
Centre, because sharing best practice is an essential part of the development of our sector.
VPOs are in a strong position to take impact measurement and management practice to the
next level. Collecting relevant data, and crucially putting it to good use, is a key challenge
for SPOs. Our unrestricted funding backs the unglamorous, but essential, work of building
capacity. And we’re in it for the long haul: we know this sort of organisational change
cannot happen overnight, and we don’t expect short-term projects to do the trick.
At Impetus, we are committed to building this capacity in the organisations we support.
We build relationships of trust that allow us to push our organisations to achieve more than
they might have thought possible. Our deep understanding of the sector is complemented
by the private sector expertise we bring in, and our long-term engagement means that
support packages can see an SPO right through the process of nding out what to measure,
collecting the data, and putting in place the virtuous circle that connects performance data
to performance improvement.
12 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
PREFACE
And of course we are an SPO too! We are acutely aware that we have a duty to expend the
resources entrusted to us for maximum impact, and that we only identify impact through
data. We need to know what wouldn’t have happened without us – whether that be more
lives changed, improved cost-efciency, greater sustainability, or all of the above, in the
organisations we support. This is undoubtedly difcult to measure. But we are committed
to nding better, and more useful ways to do so; we know our own funders value this
information, but rst and foremost we are doing it to ensure that, year on year, we do what
we do better. This guide helps us on our journey.
A nal word: managing impact is not about removing risk, as this is often the partner of
innovation. We believe SPOs should be intelligent risk takers, with venture philanthropy
providing the ultimate risk capital. Data allows you to know when you are taking a risk,
as well as whether it pays off. And when the “pay off” can mean changing the lives of
thousands, or even millions, we all need to know about it.
Daniela Barone Soares
Chief Executive, Impetus Trust
JUNE 2015 13
PREFACE
Name Organisation
Brad Presner Acumen Fund
Ken Ito Asian Venture Philanthropy Network
Claudia Leissner Auridis
Bettina Windau Bertelsmann Stiftung
Richard Kennedy CAN Breakthrough
Camilla Backström Charity Rating / NAYA AB
Nalini Tarakeshwar CIFF
Uli Grabenwarter EIF
Iana Petkova /
Gina Crane
Esmée Fairbairn Foundation
Emeline Stievenart ESSEC Business School
Rosien Herweijer European Foundation Centre
Øyvind Sandvold FERD Social Entrepreneurs
Fabrizio Ferraro IESE Business School
Anne Holm Rannaleet IKARE / EVPA Board
Meredith Niles Impetus Trust
Filipe Santos INSEAD
Sarah Gelfand IRIS / GIIN
Thomas Kagerer LGT Venture Philanthropy
Eva Varga NESsT
Lorenzo Allevi Oltre Venture
Emma Lane Spollen One Foundation
Pieter Oostlander Shaerpa
Alex Nicholls Skoll Centre for Social
Entrepreneurship
Marlon Van Dijk Social Evaluator
Claire Coulier Social Impact Analyst Association
Jeremy Nicholls SROI Network
Sophie Robin Stone Soup / ESADE Business School
Expert Group composition
EVPA is grateful for the contribution of the following Expert
Group to the development of this manual.
This list refers to the experts’ afliations at the time the rst
edition of the report was published in April 2013.
14 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
Executive summary
JUNE 2015 15
EXECUTIVE SUMMARY
8. We use investment through-
out as including the range of
nancing instruments from
grants, loans and equity.
This manual is targeted specically at venture philanthropy organisations and social investors
(“VPO/SI”), and more generally at impact investors, foundations and any other funders
interested in generating a positive impact on society. Throughout the document, we use
the term “VPO/SI” to refer to such social sector funders. The rst objective of the manual
is to create a roadmap or guidebook to help VPO/SIs navigate through the current maze of
existing methodologies, databases, tools and metrics on social impact measurement. There-
fore, we do not take a stand to recommend a particular tool, but rather have attempted to
distil best practice from the various ways of measuring and managing social impact. The
second objective is to trigger a movement towards best practice when it comes to measuring
and managing impact. We would like the manual to become a working document that
evolves with new versions over time as our industry knowledge develops.
The manual should be useful both for beginners in impact measurement, who are consid-
ering how to get started, and for more advanced investors who are struggling with how to
better integrate an impact focus into everyday investment management decisions. Within
the VPO/SI, the person (or team) assigned to measure impact will be the natural reader/
user of the manual, but we also recommend executive directors, boards of directors and
investment managers to use the manual as a reference for key decisions on topics such as
resource allocation, deal selection and investment management. The manual uses plenty
of real-life examples from VPO/SIs as well as ve longer case studies that were developed
by the impact measurement initiative (IMI) Expert Group members. The manual does not
consider how to measure nancial impact but focuses solely on social impact (using a
broad denition of social that may also include environmental or cultural).
Our starting point has been to devise a process of Impact measurement for a VPO/SI wanting
to measure the impact of their investment8 in a Social Purpose Organisation (“SPO”). The
guide focuses on two levels: how to measure and manage the impact of specic investments
(level of SPO) and how the VPO/SI itself contributes to that impact (level of VPO/SI).
This process is the “how to” of impact measurement and is often what is most needed by
venture philanthropy organisations and social investors in general to get started. Analysis
of existing methodologies for impact measurement and the experience of working together
with VPO/SIs showed that most methods and tools to measure impact share a general
framework.
16 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
EXECUTIVE SUMMARY
We represent the framework as having ve steps as shown in the following diagram:
The steps are presented in sequential order and we recommend that VPO/SIs go through
the steps in this order. However within the process it is possible to go back to steps and
revise them as you gain more information and more familiarity with the process. Some
VPO/SIs may nd it useful to go through each of the steps at a theoretical level before
implementing them in practice.
The goal of impact measurement is to manage and control the process of creating social
impact in order to maximise or optimise it (relative to costs). Managing impact occurs
continuously and is facilitated by integrating impact measurement in the investment
management process. It is important to identify what may need to change within the
investment management process so that you are able to maximise social impact. That is
why Managing Impact is the core of the impact measurement process. For each step in
the process, one should consider how this relates to the everyday work of funding and
building stronger social purpose organisations.
The impact value chain was the starting point for the denitions used in this manual as it
clearly sets out the differences between inputs, outputs, outcome and impacts.
The 5 steps of social
impact measurement
Source: EVPA
Managing
Impact
5
.
M
o
n
i
t
o
r
i
n
g
4
.
V
e
r
i
f
y
i
n
g
&
3
.
M
e
a
s
u
r
i
n
g
2
.
A
n
a
l
y
s
i
n
g
a
n
d
R
e
p
o
r
t
i
n
g
V
a
l
u
i
n
g
I
m
p
a
c
t
R
e
s
u
l
t
s
S
t
a
k
e
h
o
l
d
e
r
s
1
.
S
e
t
t
i
n
g
O
b
j
e
c
t
i
v
e
s
JUNE 2015 17
EXECUTIVE SUMMARY
The Impact Value Chain
Source: Elaborated by EVPA from Rockefeller Foundation Double Bottom Line Project
In this manual, the following denitions are used:
To accurately (in academic terms) calculate social impact, you need to adjust outcomes for:
(i) what would have happened anyway (“deadweight”); (ii) the action of others (“attribution”);
(iii) how far the outcome of the initial intervention is likely to be reduced over time (“drop
off”); (iv) the extent to which the original situation was displaced elsewhere or outcomes
displaced other potential positive outcomes (“displacement”); and for unintended conse-
quences (which could be negative or positive).
Inputs: all resources, whether capital or human, invested in the activities of the
organisation.
Activities: the concrete actions, tasks and work carried out by the organisation to
create its outputs and outcomes and achieve its objectives.
Outputs: the tangible products and services that result from the organisation’s
activities.
Outcomes: the changes, benets, learnings or other effects (both long and short term)
that result from the organisation’s activities.
Social
Impact:
the attribution of an organisation’s activities to broader and longer-term
outcomes.
SPO’s Planned Work SPO’s Intended Results
4. Outcomes 5. Impact2. Activities1. Inputs 3. Outputs
Resources (capital,
human) invested in
the activity
€, number of people
etc.
Development &
implementation of
programs, building
new infrastructure
etc.
Number of people
reached, items sold,
etc.
Effects on target
population e.g.
increased access to
education
Attribution to changes
in outcome. Take
account of alternative
programs e.g. open air
classes
€50k invested, 5 people
working on project
Land bought, school
designed & built
New school built with
32 places
Students with
increased access to
education: 8
Students with access
to education not
including those with
alternatives: 2
Tangible products
from the activity
Changes resulting
from the activity
Outcomes adjusted
for what would have
happened anyway,
actions of others &
for unintended
consequences
Concrete actions of
the SPO
18 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
EXECUTIVE SUMMARY
EVPA’s recommendation for measuring social impact is to calculate outcomes while
acknowledging (and if possible adjusting for) those factors that contribute to increasing
or decreasing the impact of the organisation; rather than aiming to calculate very specic
impact numbers.
In what follows, we provide a quick glance at the recommended impact measurement
process as detailed in the manual.
Step 1: Setting Objectives
This step includes dening the scope of the VPO/SI’s impact measurement and setting
objectives. Setting objectives is a vital step in any impact measurement process and needs
to be considered at both the level of the VPO/SI and the investee SPO. Often VPO/SIs do
not spend enough time upfront considering their own impact objectives and why they
want to measure impact, which later makes it difcult to take decisions regarding what is
relevant and what is not when faced with scarce resources.
The more specic the objectives the better the impact measurement that can be prepared
For a VPO/SI, objectives should be set at two levels:
(i) Level of the VPO/SI.
On the rationale and scope of impact measurement, the VPO/SI should aim to answer ve
questions upfront:
a. What is your motivation for measuring social impact?
There are many different purposes for using impact measurement and these could each
imply different target audiences and outlook.
b. What resources can you dedicate to impact measurement?
Resources to be considered include nancial, human, technological and time.
c. What type of SPOs are you working with?
The maturity i.e. the stage of development of the SPO will potentially limit the type of
information that the SPO can provide to you.
d. What level of rigour do you require in your impact analysis?
Depending on how accountable you expect your investees to be, you can increase the
rigour of your analysis and thereby reduce the risk of any impact claims made.
e. What is your time frame for measuring impact?
The time period over which you measure impact should be determined by the most
important outcomes and estimated length of time required to achieve them. But in
practice there may be internal or external pressures to invest for a certain period of time.
Depending on your timeframe, you will be able to draw either very specic or more
general conclusions about the impact of the SPO.
JUNE 2015 19
EXECUTIVE SUMMARY
Recommendations for managing impact:
•A VPO/SI must formulate its overarching social problem or issue so as to choose
investments in SPOs that can contribute to solving that social issue.
•Understanding the current and expected social impact of an SPO early in the decision-
making process is extremely valuable: it creates a common understanding of the
impact of an organisation; allows the VPO/SI and SPO to “speak the same language”;
and facilitates assessment of achievement of impact at later stages. A VPO/SI should
convince the SPO of the value of impact measurement, provide assistance where
possible and dene with them the responses to the essential questions to help them
express their objectives.
•Decisions have to be made about the amount of time and resources that a SPO should
dedicate to impact measurement.
On its impact objectives, the VPO/SI should aim to answer these questions:
a. What is the overarching social problem or issue that the VPO/SI is trying to solve?
This can be more or less difcult depending on how broad or focused your approach
but a clearly articulated response is necessary to be able to choose investments that can
contribute to solving the social issue that the VPO/SI is addressing.
b. What objective does the VPO/SI want to achieve?
Looking at your overall objectives and the relationship to be built with investees.
c. What are the expected outcomes?
The VPO/SI should evaluate the expected outcome of its investment in the SPO, i.e. the
expected outcome of the SPO and how the VPO/SI expects to contribute to achieving
that outcome. It is important to consider potential unintended consequences of the
VPO/SI’s activities.
(ii) Level of the SPO.
At a minimum you should answer these questions about the SPO:
a. What is the social problem or issue that the SPO is trying to solve?
The response should include information about the nature and magnitude of the
problem or opportunity; which populations are affected; whether the matter is changing
or evolving as well as in what way it is changing or evolving.
b. What activities are the SPO undertaking to solve the social problem or issue?
This should include a description of exactly what the SPO is doing to try to effect a change.
c. What resources or inputs (as per the impact value chain) does the SPO have and need
to undertake its activities?
This should include the time, talent, technology, equipment, information and other assets
available to conduct the activities, as well as the VPO/SI’s contribution to helping the
SPO to solve the issue.
d. What are the expected outcomes?
This should include what the SPO must achieve to be considered successful and will form
the basis of the milestones against which the SPO will be measured. It is also important
to consider the unintended consequences of the SPO’s activities.
20 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
EXECUTIVE SUMMARY
Step 2: Analysing Stakeholders
VPO/SI investments generate value for a variety of stakeholders. A stakeholder is dened
as, “Any party effecting and/or affected by the activities of the organisation.”
This is an important step because the VPO/SI needs:
•To understand the expectations of the stakeholders, their contribution to and the potential
impact the SPO’s work will have on them.
•The co-operation of the main stakeholders in the impact measurement process.
Applying to both the VPO/SI and SPO level, there are two aspects to stakeholder analysis:
(i) Stakeholder identication; which includes stakeholder mapping (direct and indirect
contributors and beneciaries), stakeholder selection (using concepts such as materi-
ality, accountability and relevancy) and analysis of stakeholder expectations.
(ii) Stakeholder engagement; which includes communicating with the selected stakeholders
and is vital to be able to understand their expectations and, later in the process, verify
if their expectations have been met. This is described in more detail in Step 4.
Step 3: Measuring Results: Outcomes, Impact and Indicators
This step again occurs at two levels:
•VPO/SI level: its own outputs, outcomes, impact and indicators as per its own objectives
(theory of change etc); impact measurement at a portfolio level; impact of the VPO/SI’s
work on the SPO.
•SPO level: transforming its objectives into measurable results via outputs, outcomes,
impact and indicators.
To transform the objectives set in Step 1 into measureable results a VPO/SI and SPO must
consider outputs, outcomes, impact and indicators. For a VPO/SI, it is not sufcient to
just consider the impact achieved by the SPO, it is also important to assess the impact of
the work of the VPO/SI on the SPO. Outputs are directly related to the activities of the
organisation i.e. what is done to try and make a change in the base case, hence these are
Recommendations for managing impact:
•A VPO/SI must get the buy-in of key stakeholders (donors/investors, staff/human
resources, SPOs) to the impact objectives of the VPO so that their expectations are
managed and their contributions are aligned.
•Engagement with a VPO/SI’s key stakeholders should happen upfront and any major
changes in the impact objectives should be properly communicated.
•When a VPO/SI makes an investment in a SPO, stakeholder analysis should be
included during the due diligence phase. As the investment period proceeds, it is
advisable to regularly get back to these stakeholders to verify that their expectations
are being met (more details on how to do this in Step 4).
•Consider upfront when would be the appropriate time to revisit stakeholder analysis
together with the SPO (e.g. change to outcomes being achieved, major new funding
streams, new business lines, policy changes).
JUNE 2015 21
EXECUTIVE SUMMARY
Internal vs external focus:
the use of outputs,
outcomes or impacts
generally easier to measure. Outcomes and impacts are related to the expected and unex-
pected effects of the activities of the organisation, hence they are outside the scope of the
organisation’s activities (but within their scope in terms of accountability) and generally
more difcult to measure.
Indicators are used to show progress towards or away from outputs or outcomes. If output
indicators are required these should be sourced as much as possible from public databases
such as IRIS, Global Value Exchange or other databases. If these output indicators point in
the same direction as the outcome you are targeting or if there exists independent research
showing that specic outputs do result in specic outcomes then some may also be used
as outcome indicators. If not we recommend the following process to select outcome indi-
cators:
(i) Dene outcomes as change statements, target statements or benchmark statements.
(ii) Select outcomes: you may have a list of outcome statements but you must select those
outcomes that are most important, material, useful and feasible (in achievement not in
measurement).
(iii) Select indicators i.e. identify two to three factors that provide measurable evidence
for a sub-optimal situation. There are four aspects to a good indicator:
a. Indicators should generally be aligned with the purpose of the organisation, although
if a potential unintended outcome has been identied, relevant indicators for this
out-come may by denition not be aligned with the purpose of the organisation.
b. Indicators should be “SMART”.
c. Indicators should be clearly dened so they can be reliably measured and ideally
comparable with those used by others.
d. More than one indicator should be used, with preference for two or three.
Impact itself is a technical and often academic discussion including concepts such as drop
off, displacement, deadweight and attribution. The rationale for this guide is to remove the
complexity around the issue and provide practical guidance.
Base Case Changed Case
Outcomes
Impacts
External Focus
Internal Focus
Outputs
Inputs
Organisation
Source: EVPA
22 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
EXECUTIVE SUMMARY
The ability of an organisation to measure impact will depend on the sector and geography
in which it is operating. The propensity of European governments to move towards pay for
performance contracts means the measurement of impact is becoming more important for
those organisations active in these areas. However, for many organisations, access to inde-
pendent statistics and the creation of control groups in order to assess displacement, dead-
weight, drop off and attribution is not possible due to the expense and specialist skill-set
to carry them out. In these cases we encourage organisations to measure impact by calcu-
lating outcomes and acknowledging those factors that may mean that the outcomes are not
equal to the impact i.e. can increase or decrease impact. In some cases it may be possible to
think about some evidence as to what a control group may look like and could be used for
comparison purposes, for example based on research of comparable situations elsewhere.
Step 4: Verifying & Valuing Impact
In this step, we need to verify whether the claim we make on having positive social impact
is true, and if so, to what extent (i.e. to what value). The responses to these questions will
allow us to rene the target outcomes and associated indicators, creating a positive
feedback loop in the impact measurement process. This step also helps identify the
impacts with the highest social value, which can help an organisation focus their resources
towards those initiatives that create most impact on society.
Again, this step needs to occur at two levels: both at the level of the VPO/SI as well as at
the level of the SPO.
Recommendations for managing impact:
•For a VPO it is not enough to just consider the impact achieved by the SPO, it is also
important to assess the impact of the work of the VPO/SI on the SPO.
•The denition of portfolio level indicators may be required to measure how well a
VPO/SI has achieved its objectives as an organisation.
•The VPO/SI should ask the SPO to focus on those indicators that are directly related
to the SPO’s theory of change and hence in line with their operational process. Any
additional indicators required for the VPO/SI to satisfy its impact measurement needs
should be collected by the VPO/SI.
•The expected outputs, outcome and impact, and the corresponding indicators should
be dened before the investment is made and agreed upon by the VPO/SI and the
SPO.
•Clarify at the beginning of the relationship (i.e. during due diligence and within deal
structuring) who is responsible for measuring what. This can evolve over time and
should be reviewed on an annual basis.
JUNE 2015 23
EXECUTIVE SUMMARY
The VPO/SI must verify (or at least record) the non-nancial assistance provided to their
investees. They should then conrm with the investees that this assistance was in fact
valued. It may also be necessary for VPO/SIs to verify at regular intervals that the expecta-
tions of other stakeholders (donors/investors and human resources) are met so that correc-
tive actions can be undertaken if necessary.
At the level of the SPO, it is important to verify whether the outcomes make sense for the
stakeholder i.e. if the outcomes were realised during the timeframe and in the quantities
expected.
Verifying impact can be done via:
•Desk research: conrming whether the trends detected and interpreted by the SPO can
be triangulated with other data (external research reports, databases, government statis-
tics etc.);
•Competitive analysis: comparing the results of the SPO with other comparable organisa-
tions in terms of similar issues, geographies and populations targeted;
•Interviews / focus groups: asking neutral questions to a representative sample of your
key stakeholders. This format can be particularly useful when the VPO/SI is assessing
the value of its non-nancial assistance to the SPO. However it is recommended that a
neutral party conduct these interviews so as to ensure SPOs are comfortable providing
the most truthful responses).
The next step is to understand if the outcome was important i.e. of value to the stakeholder.
Numerous techniques and methodologies exist for measuring value created. We have
chosen not to list all the possible techniques preferring instead to cite certain useful refer-
ences. Two general categories can be identied: qualitative and quantitative (monetisation).
•Qualitative: storytelling, client satisfaction surveys, participatory impact assessment
groups, progress out of poverty index.
•Quantitative (monetisation): techniques for valuing e.g. perceived value / revealed pref-
erence and Value Game or techniques for cost / benet analysis e.g. cost-saving methods
and quality adjusted life years calculations.
Whether you select a quantitative or qualitative technique or a combination of both for
valuing impact will depend on your rationale for measuring impact in the rst place. For
example, often governments tend to prefer quantitative approaches whereas the general
public may prefer qualitative methods.
24 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
EXECUTIVE SUMMARY
Step 5: Monitoring & Reporting
The nal step in the impact measurement process involves monitoring – tracking progress
against (or deviation from) the objectives dened in the rst step and made concrete
through the indicators set in the third step; and reporting – transforming data into present-
able formats that are relevant for key stakeholders. Monitoring and reporting are iterative
processes that go hand in hand because what is monitoring to one stakeholder is reporting
to another e.g. when a VPO/SI is monitoring the progress of an investee SPO, that SPO is
reporting relevant data to the VPO/SI. As in the other steps we must consider the process
at two levels: the VPO/SI and SPO.
(i) Monitoring
Once an organisation has decided on the indicators to be measured and veried that they
make sense to the key stakeholders, they need to start collecting data in a systematic way.
In practice, the type of system may be considered upfront but we urge organisations to
go through the impact measurement process at least theoretically prior to setting up the
system to understand the type of information that needs collection.
The VPO/SI should be collecting and analysing data on:
•Specic indicators that measure its progress towards reaching its overarching social
objectives.
•Time invested and/or € provided in non-nancial support to its investees.
•The investee SPOs, according to the objectives and indicators previously dened.
There is also a need to evaluate if the SPO is effectively monitoring its activities and
outcomes e.g. are the selected indicators appropriate (providing a balanced picture of the
situation and picking up potentially positive and negative aspects) and if the VPO/SI has a
role to play in improving the impact measurement practices of the investee
Recommendations for managing impact:
•Perform this step at the beginning of an investment (as part of the due diligence), at
least once during the investment period (to check that the impact is achieved and
valued) and again at the time of exit (as a way to check that the desired impact has
been achieved and makes sense).
•Make clear assignments between the SPO and VPO/SI about who is responsible for
which parts of the verifying and valuing process.
•Unless you verify whether you have created value through your support of the SPO,
you cannot credibly make that statement.
•VPO/SIs should use independent studies to assess the value they provide to the
SPOs as directly questioning investees may be a delicate matter, resulting in them not
always providing truthful answers.
•VPO/SIs should verify at regular intervals that the expectations of other stake-
holders (donors/investors and human resources) are met so that corrective action
can be undertaken if necessary.
JUNE 2015 25
EXECUTIVE SUMMARY
Recommendations for managing impact:
•To remove a reliance on and/or culture of “gut feeling”, VPO/SIs should work with
the SPO to develop an impact monitoring system that can be integrated into the
management processes of the organisation.
•Check whether the system the SPO already works with is sufcient to meet your
requirements – if not, a VPO/SI should be prepared to contribute to improving it
through pro-bono partners or other resources (although generally this support doesn’t
extend to the actual data collection). The objective should be a system that is of value
to the SPO as a management tool.
•The cost to support and maintain a SPO’s impact monitoring system (including
personnel time and costs) should be part of the SPO’s budget and hence part of the
negotiation with the investor in order to decide how costs should and/or could be split.
•When working with very early stage SPOs and helping them develop business plans,
integrate requirements on impact measurement at this stage.
•Agree on reporting requirements upfront with SPO and co-investors to eliminate the
burden of multiple reporting on the SPO.
•Manage expectations about frequency and level of detail for reporting, and the way
SPOs report; will they just report on numbers or also on verication (and if so, with
what frequency).
The SPO needs to evaluate the outcomes or impacts that are being achieved through the
activities of its organisation and the practical lessons that can be learned from the results.
With this information it is then possible to decide what actions are needed to increase
impact.
(ii) Reporting
Once the data has been collected and analysed, an organisation needs to consider how
to present this information. The purpose of reporting affects the information that should
be included. Depending whether the focus is on an internal or an external audience, the
various stakeholders may require different types of reports. The stakeholder analysis
conducted in Step 2 should guide the development of reporting, considering the stake-
holders’ multiple objectives.
One of the challenges of the social sector is that each SPO needs to report in different ways
to each funder. Some initiatives (e.g. Social Reporting Standard) are trying to overcome this
problem, but there is still a problem of lack of standardisation that leads to inefciencies.
26 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
EXECUTIVE SUMMARY
Managing Impact
The impact measurement process outlined in the ve steps should allow the VPO/SI to
better manage the impact generated by its investments. To manage impact, the VPO/SI
should continuously use the impact measurement process to identify and dene correc-
tive actions if the overall results deviate from expectations. It will also have become clear
that impact measurement is very closely aligned to your investment management process.
Given most VPO/SIs are aiming to maximise impact, the corrective actions taken may
apply as much to the investment management process as to impact measurement itself.
Managing impact in the investment process
Assess whether invest-
ment opportunity fits
with VPO/SI strategy
by asking questions
detailed in setting
objectives.
Decide on the
overarching social
impact objectives
of the VPO/SI –
these will guide the
investment process.
Dig deeper into
questions asked in
setting objectives.
Perform stakeholder
analysis.
Verify and value
expected results.
Map outputs, out-
comes and impacts
and decide on key
indicators against
which progress will
be measured.
Decide on monito-
ring and reporting
content and frequency
and assign responsi-
bilities.
Regularly assess
impact results against
key indicators.
Verify and value
reported results at
regular intervals.
Revise indicators
if significant changes
are made in the
business and impact
model.
Investment
management Exit
Due diligence
(detailed screening)
Deal
screening
Investment
strategy Deal
structuring
Investment process
Perform thorough
analysis of impact
results against
objectives –
verifying
and valuing
reported
results.
JUNE 2015 27
28 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
PART 1:
Introduction
and Overview
JUNE 2015 29
INTRODUCTION AND OVERVIEW
1.0 Introduction and Overview
This manual is targeted specically at venture philanthropy organisations and social investors
(“VPO/SI”), and more generally at impact investors, foundations and any other funders
interested in generating a positive impact on society. Throughout the document, we use the
term “VPO/SI” to refer to such social sector funders.
The rst objective of the manual is to assist investors in improving the way they measure
impact, providing practical tips and recommendations for how it works in real-life situations.
For that purpose, the manual is a roadmap or guidebook to help VPO/SIs navigate through
the current maze of existing methodologies, databases, tools and metrics on social impact
measurement. The manual does not recommend a particular tool, but rather attempts to distil
best practice from the various ways of measuring and managing social impact. The manual
should be useful both for beginners in impact measurement, who are considering how to get
started, and for more advanced investors who are struggling with how to better integrate an
impact focus into everyday investment management decisions. The manual does not consider
how to measure nancial impact but focuses solely on social impact (using a broad denition
of social that may also include environmental or cultural impact). The second objective is to
trigger a movement towards best practice when it comes to measuring and managing impact.
We would like the manual to become a working document that evolves with new versions
over time as our industry knowledge develops.
The manual focuses on two levels, how to measure the impact of specic investments and
how the VPO/SI itself contributes to that impact. It focuses on devising a process of impact
measurement for a VPO/SI evaluating the impact of their investment in a SPO. This process
is the “how to” of impact measurement and is often what is most needed by VPO/SIs to get
started. The ultimate goal is for impact to become an integral part of the investment manage-
ment process. Within the VPO/SI, the person (or team) assigned to measure impact will be
the natural reader/user of the manual, but we also recommend executive directors, boards of
directors and investment managers use the manual as a reference for key decisions on topics
such as resource allocation, deal selection and investment management.
In order to ensure the inclusion of the opinions and experiences of various stakeholders, EVPA
convened an Expert Group consisting of twenty-seven venture philanthropy practitioners,
consultants, academics and representatives of other organisations involved in impact measure-
ment. We have beneted greatly from the collaboration of these experts who freely and enthu-
siastically contributed their time and knowledge to the development of this document. The
members of the Expert Group are listed in the preface and we are extremely grateful to them.
The manual uses plenty of real-life examples from VPO/SIs as well as ve longer case studies
that were developed by the impact measurement initiative (IMI) Expert Group members. In
this version of the manual, we also include the feedback received during the workshop we
organised on the topic with 80 participants and individual feedback collected during a consul-
tation period of around three months following the publication of the rst draft.
30 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
INTRODUCTION AND OVERVIEW
This practical guide is presented through a framework of ve steps that an investor should
go through when measuring impact. The process nishes with a section on managing
impact that attempts to integrate the elements of impact measurement into the investment
process. We have stayed away from set methodologies and instead tried to provide specic
recommendations and practical examples. Five concrete and detailed case studies are
provided to further show how real VPO/SIs are dealing with impact measurement. These
cases studies are examples of the current state of the eld and show how VPO/SIs are
addressing the challenges they face in measuring impact. Finally, the document provides a
glossary and additional resources.
1.1 Background
Venture philanthropy (VP) works to build stronger investee organisations with a social
purpose (SPOs) by providing them with both nancial and non-nancial support in order
to increase their social impact. Although we use the word social we include impacts that
maybe social, environmental or cultural. The venture philanthropy approach includes both
the use of social investment (equity and debt instruments) and grants. The key character-
istics of venture philanthropy include high engagement, organisational capacity-building,
tailored nancing, non-nancial support, involvement of networks, multi-year support
and performance measurement.
An integral part of the VP approach is the measurement of performance; placing emphasis
on good business planning, measurable outcomes, achievement of milestones and nancial
accountability and transparency. The focus of this manual is social impact measurement.
1.2 How is social impact currently measured by social investors and venture
philanthropists?
The rationale for undertaking this impact measurement initiative was inspired by the
out-come of a workshop on impact measurement organised by EVPA in June 2011, and
the results of the 2011 EVPA Survey of European VPO/SIs, collecting data on 50 VPO/SIs
based in Europe with investments in Europe and abroad. The general opinion that came
out of the workshop was that there was a strong need for further direction on how to
approach impact measurement.
The second annual EVPA survey of Venture Philanthropy and Social Investment in Europe9,
released on 1st March 2013, collecting data on 61 VPO/SIs also reinforced the importance
of social impact measurement.
The key highlights of the survey with respect to impact measurement were as follows:
•There is increased attention to measuring social impact: The focus on social impact measure-
ment increased, with 90% of respondents measuring social impact on at least an annual
basis during the investment period. Although impact measurement still occurs less
frequently than nancial performance measurement.
9. Hehenberger, L.; Harling,
A., (2013), “European Venture
Philanthropy and Social Investment
2011/2012”, EVPA.
JUNE 2015 31
INTRODUCTION AND OVERVIEW
•VPO/SIs still focus on outputs more than outcomes or impact: The objectives of the impact
measurement system are, in the majority of cases (84%), still based on output measures.
Nevertheless we saw an increase in the percentage of VPO/SIs attempting to measure
changes in outcome or impact.
•Increase in budget assigned to impact measurement: In scal year 2011, the average annual
budget for measuring social impact was just over €63k (compared to €18k in 2010), with
a median spend of €15k.
•Lack of standardisation indicates a high degree of fragmentation in the use of impact measurement
tools and systems: In line with last year’s survey a majority of VPO/SIs (73%) indicated that
they were not using a standardised tool to measure social impact. Among those that did
use such a tool, the most frequently mentioned were Social Evaluator and SROI, although
a quarter of people did say they were using IRIS indicators or theory of change. Interest-
ingly, when asked whether they used one of the steps of the 5-step process developed
herein, between 70-90% of respondents used each of the steps.
•Impact measurement is not fully integrated into the decision-making process: 53% never or only
sometimes take the social performance into account before releasing new funds.
•Impact measurement does not inform employee compensation: Only 12% of the VPO/SIs
include social performance in the compensation schemes for their own staff.
The outcome of the workshop and the results of the EVPA Industry Survey 2011/2012 rein-
forced EVPA’s opinion that there was a need for additional clarity and guidance on impact
measurement.
1.3 Five-step framework
Analysis of existing resources on impact measurement and the experience of working
with VPO/SIs showed that most methods and tools to measure impact share a general
framework. This general framework was the starting point for the discussions on impact
measurement.
We see the framework as having ve steps, which will be explored in greater detail in the
main body of the manual (Part 2). Each of the ve steps applies to the VPO and how it
should consider its own impact, as well as to the SPO. The ve steps are as follows:
1. Setting Objectives: setting the scope of the impact analysis (why and for whom), the level
(portfolio of social investments/individual social enterprise) and what the desired social
change is. Objectives should be set at:
•Level of VPO/SI (dening scope of impact measurement and the overarching social
objectives the VPO/SI wants to achieve)
•Level of investee (social issue to be solved, inputs/activities, expected outcomes)
32 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
INTRODUCTION AND OVERVIEW
2. Analysing Stakeholders: ranking the multitude of potential stakeholders in order of
priority, weighing their contribution to the completeness of the analysis against the
resources required, and analysing their inputs (if any), activities and potential outputs.
•Level of VPO/SI (employees, board of directors, investors / donors)
•Level of investee (direct and indirect contributors and beneciaries)
3. Measuring Results – Outcome, Impact and Indicators: measuring the output, outcome and
impact10 that derive from your activity for the key stakeholders, and understanding how
different types of indicators can be used to map the social result of the social enterprise’s
and VPO/SI’s work.
•Level of VPO/SI (based on the objectives of VPO/SI, you can map results and consider
portfolio level indicators)
•Level of investee (outputs, outcomes, impact and indicators relating to the objectives of
the SPO)
4. Verifying & Valuing Impact: verifying that the impact is not too subjective and whether
it indeed was valued by the key stakeholders – considering quantitative and/or
qualitative methods (by calculating the social value of an investment or otherwise) and
comparing the results of the work against relevant benchmarks.
•Level of VPO/SI (was non nancial support provided to investees, valued by the
investee etc.)
•Level of investee (verifying and valuing impact for key stakeholders)
5. Monitoring & Reporting: collecting data and devising a system to store and manage the
data as well as integrating this information into overall operations and reporting the
data to relevant stakeholders.
•Level of VPO/SI (what systems are required to collect, store and manage data, reporting
formats)
•Level of investee (collection, management and reporting requirements for the SPO)
The manual presents the steps in a sequential order and we recommend that VPO/SIs go
through the steps in this order. However within the process it is possible to go back to steps
and revise them as you gain more information and more familiarity with the process. Some
VPO/SIs may nd it useful to go through each of the steps at a theoretical level before
implementing them in practice. For example it may be difcult for the SPO to engage with
certain stakeholders on a frequent basis, therefore in practice you may need to gain the
information required for Steps 2 and 4 at the same time.
Working through impact measurement it will become clear that each step also has ramica-
tions for the investment management process. Given VPO/SIs are interested in maximising
impact it is important to identify what may need to change within the investment manage-
ment process so you are indeed able to maximise impact. Within this manual we call this
managing impact. For each step in the process, the VPO/SI should consider how it relates to
the everyday work of funding and building stronger social purpose organisations.
10. The definition of these terms
are explored in section 1.5.
JUNE 2015 33
INTRODUCTION AND OVERVIEW
1.4 Methodology
EVPA proposed a ve-step process for how to measure social impact based on our own
research on impact measurement and the practical experience of working with VPO/SIs
that measure impact. A brief description of the contents of the ve-step process was circu-
lated to the Expert Group in the spring of 2012. Between April and July of 2012, six webinars
were held, each webinar related to a particular step in the process (plus an introductory
session). The members of the Expert Group were divided into ve working groups and
asked to prepare a presentation, including a case study on a particular step. The experiences
and discussions among the participants in these webinars have served to adjust and edit
the frame-works put forward in this manual to ensure it is well grounded in the practice
of EVPA members and other social investors. The data gathered from the Expert Group
members was complemented with more in-depth interviews with selected VPO/SIs.
The rst draft of A Practical Guide to Measuring and Managing Impact was released for
consultation in November 2012 and the 80 participants of EVPA’s impact measurement
workshop provided initial feedback. Between November 2012 and March 2013 additional
feedback was garnered from VPO/SI practitioners in order to improve the guide. The
timeline of the Impact Measurement Initiative is shown below.
The 5 steps of social
impact measurement
Source: EVPA
Managing
Impact
5
.
M
o
n
i
t
o
r
i
n
g
4
.
V
e
r
i
f
y
i
n
g
&
3
.
M
e
a
s
u
r
i
n
g
2
.
A
n
a
l
y
s
i
n
g
a
n
d
R
e
p
o
r
t
i
n
g
V
a
l
u
i
n
g
I
m
p
a
c
t
R
e
s
u
l
t
s
S
t
a
k
e
h
o
l
d
e
r
s
1
.
S
e
t
t
i
n
g
O
b
j
e
c
t
i
v
e
s
Webinar
Series
April – July
2012
Case Study
Development
April – August
2012
1st Draft
Manual &
Workshop
November 2012
Consultation
Period
November 2012
– March 2013
“Version 1.0”
Manual
April 2013
Timeline of the Impact
Measurement Initiative
34 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
INTRODUCTION AND OVERVIEW
Inputs: all resources, whether capital or human, invested in the activities of the
organisation.
Activities: the concrete actions, tasks and work carried out by the organisation to
create its outputs and outcomes and achieve its objectives.
Outputs: the tangible products and services that result from the organisation’s
activities.
Outcomes: the changes, benets, learnings or other effects (both long and short term)
that result from the organisation’s activities.
Social
Impact:
the attribution of an organisation’s activities to broader and longer-term
outcomes.
1.5 Denition of social impact
There is currently a large amount of discussion and debate around social impact meas-
urement. However, before diving into the topic it is important to agree the denitions of
certain frequently used words in the impact measurement dialogue.
The impact value chain has become a popular starting point for dening social impact as it
clearly sets out the differences between inputs, outputs, outcome and social impacts.
The impact value chain was also the starting point for the denitions used in this manual.
Based on the discussions in the Expert Group, EVPA has agreed the following denitions:
The Impact Value Chain
Source: Elaborated by EVPA from Rockefeller Foundation Double Bottom Line Project
SPO’s Planned Work SPO’s Intended Results
4. Outcomes 5. Impact2. Activities1. Inputs 3. Outputs
Resources (capital,
human) invested in
the activity
€, number of people
etc.
Development &
implementation of
programs, building
new infrastructure
etc.
Number of people
reached, items sold,
etc.
Effects on target
population e.g.
increased access to
education
Attribution to changes
in outcome. Take
account of alternative
programs e.g. open air
classes
€50k invested, 5 people
working on project
Land bought, school
designed & built
New school built with
32 places
Students with
increased access to
education: 8
Students with access
to education not
including those with
alternatives: 2
Tangible products
from the activity
Changes resulting
from the activity
Outcomes adjusted
for what would have
happened anyway,
actions of others &
for unintended
consequences
Concrete actions of
the SPO
JUNE 2015 35
INTRODUCTION AND OVERVIEW
To accurately (in academic terms) calculate social impact you need to adjust outcomes for:
(i) what would have happened anyway (“deadweight”); (ii) the action of others (“attribution”);
(iii) how far the outcome of the initial intervention is likely to be reduced over time (“drop
off”); (iv) the extent to which the original situation was displaced elsewhere or outcomes
displaced other potential positive outcomes (“displacement”); and for unintended conse-
quences (which could be negative or positive).
EVPA’s recommendation for measuring social impact is to calculate outcomes while
acknowledging (and if possible adjusting for) those factors that contribute to increasing
or decreasing the impact of the organisation, rather than aiming to calculate very specic
impact numbers. This is a general recommendation however we accept that there are
certain organisations (for example those who interact with government for pay for perfor-
mance type contracts) that may be required to produce more scientically accurate social
impact numbers.
As with all denitions, they are most effectively demonstrated through the use of an
example11. Let us look at an investment in an organisation that focuses on increasing access
to education for primary school age children in developing countries. We have introduced
the key factors from the case in the impact value chain above to illustrate the difference
between input, out-put, outcome and impact.
The theory of change for this organisation is that lack of access to education is a key factor
in preventing the poor from moving out of poverty. Hence to increase access to education
the organisation builds educational infrastructure in developing countries. Its inputs are
the money invested and the people employed to build the educational infrastructure.
Their principal activity (although it may have other complementary ones) is building new
schools. One particular output would be a new school built with places for 32 primary
school children, although the actual outcome with respect to increased access to education
is only 8 as 24 of the other potential primary school children were kept at home to work
on the harvest and do other essential work for the family. In fact, the impact is even less
when adjusting for the change that would have taken place if the SPO had not performed
its activity: of those 8 primary school children, 6 were already receiving some form of
education through open air classes and visiting teachers.
This example shows the importance of understanding the difference between impact,
out-comes and outputs when considering the social impact of a SPO.
11. Elaborated from Grabenwarter &
Liechtenstein, (2011), “In search
of gamma: an unconventional
perspective on impact investing”.
36 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
PART 2:
The Impact
Measurement
Process
JUNE 2015 37
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
In the following sections, we will go through each step in the impact measurement process.
For each step, we will explain what it means, how the step is implemented at two levels
(i) at the level of the Investor, the VPO/SI, and (ii) at the level of the Investee, the SPO itself;
provide concrete recommendations and illustrate by using a real-life example. The reason
why the manual contemplates two levels is because a VPO/SI achieves impact indirectly
by investing in a SPO that is solving a particular social issue. A VPO/SI needs to consider
both levels and how to achieve an appropriate alignment between the two.
2.0 Step 1: Setting Objectives
2.1 What?
This step includes the denition of the scope of impact measurement by the VPO/SI and
then the setting of objectives. Setting objectives may appear an intuitively simple task but
in practice there is often confusion. Without a clear understanding of objectives it is difcult
to proceed with the impact measurement process and this can lead to overburdening the
SPO and even the VPO/SI with excessive data collection requests.
The more specic the objectives the better the impact measurement can be prepared. Objec-
tives should be set at two levels:
(i) At the level of the VPO/SI; and
(ii) At the level of the SPO
2.2 How to?
Level of VPO/SI
Before thinking about measuring the social impact of an investee, VPO/SIs should dene
the scope of their impact measurement and set their own objectives in terms of impact and
their relationships with the SPOs. Our conversations with VPO/SIs have highlighted that
often VPO/SIs begin with an opportunistic approach to venture philanthropy and social
investment. There may also be other issues e.g. the views of potential donors / investors
that may condition what you invest in and could risk being outside your specic objectives.
The two levels of impact
measurement
Achieve Impact:
Ensure progress towards
impact objectives
Investor
(VPO/SI)
Investee
(SPO)
38 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
Setting and communicating your impact objectives upfront minimises the risk of straying
from them due to opportunistic approaches or forceful donors / investors.
There are ve factors to consider when dening the scope of impact measurement:
(i) What is your motivation for measuring social impact?
There are many different purposes of impact measurement and these imply different target
audiences and outlooks.
A VPO/SI may want to use impact measurement for several reasons. The following list
is not exhaustive but provides the main reasons why a VPO/SI should strive to measure
impact. Each motivation in turn has implications for how impact is measured:
1. A tool to assist with investment selection: allowing the VPO/SI to prioritise where to invest
its resources for greatest impact. In this case the target audience of the impact measure-
ment will be internal to the VPO/SI, most likely the VPO/SI portfolio managers, and the
outlook will be forward looking.
2. Evaluation of the progress of a SPO: again the target audience is internal, however this will
also include the management and board of the VPO/SI as well as the individual portfolio
managers and, rather than being prospective, monitoring occurs on a continuous basis.
3. A management tool to ensure that social impact is integrated into strategy and operations: is of
great use to the management of the VPO/SI. This form of impact measurement would
also be done on a continuous basis.
4. Facilitation of aligning of incentives: can be done either with an internal audience in mind:
incentive schemes for portfolio managers based on social impact achieved to steer their
work towards achieving maximum impact; or with an external party in mind, speci-
cally the SPO management: setting funding milestones based on social impact objectives
achieved. In both these cases there are elements of continuous but also retrospective
measurement of impact.
5. Reporting purposes: so it can communicate the social impact achieved to external stake-
holders in order to facilitate marketing or fundraising efforts. This is almost always done
on a retrospective basis.
In practice a VPO/SI is likely to use impact measurement for a combination of purposes.
Even though Ferd Social Entrepreneurs (please refer to case study) has only one owner
rather than a large external investor group, they still believe it is important to measure
impact. They do so for a number of reasons including demonstrating to Ferd’s board
and to owner Johan Andresen that it is possible to create social impact in a country with
a well-developed welfare state and to motivate other investors to follow a VP approach.
JUNE 2015 39
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
The Bill & Melinda Gates Foundation12 cites 3 different reasons for impact measure-
ment:
(i) Track their progress i.e. for monitoring: hold themselves accountable for what they
do and how they do it by measuring inputs, activities and outputs of their work as
well as those of their investments.
(ii) Inform their strategies i.e. as a management tool: test assumptions and track
achievements by measuring outputs, outcomes and impacts as well as under-
standing how and why they have succeeded or failed.
(iii) Contribute to the eld i.e. for reporting: contribute to accomplishing shared
goals by measuring outcomes and impact, sharing results and collaborating with
partners to understand what works and why in the populations they serve.
The table below provides an overview of the principal motivations for measuring impact
and their associated audiences and outlooks.
Reality Check
The reality is that no two VPO/SIs are the same, and your understanding of your motiva-
tion for impact measurement needs to be framed in the context of what is reasonable with
your resources, the type of SPOs you invest in, the level of rigour you require in your
analysis and the timeframe you are considering for your analysis.
Motivation Target Audience Outlook
Investment Selection: prioritise where
to invest resources for greatest impact
Internal (VPO/SI portfolio
managers)
Prospective
Monitoring: evaluate progress of the
SPO against milestones, with increased
transparency
Internal (VPO/SI portfolio
managers, management, board of
VPO/SI)
Continuous
Management tool: a framework to
integrate social impact into strategy
and operations
Internal (VPO/SI portfolio managers
and management)
Continuous
Aligning incentives: generates
incentive schemes that steer work
towards achieving impact, and/or for
setting funding milestones with the
SPO
Internal (VPO/SI portfolio
managers, management and
board of VPO/SI) & external (SPO
management)
Continuous/
Retrospective
Reporting: communicate impact to
external stakeholders for marketing
and fundraising purposes
External (other stakeholders) Retrospective
Source: EVPA
12. Nelson & Ratcliffe, (2010),
“A Guide to Actionable
Measurement”, Bill & Melinda
Gates Foundation.
40 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
(ii) What resources can you dedicate to impact measurement?
Resources to be considered include nancial, human, technological and time. The more
resources available, the higher your expectations can be as to what you can achieve from
impact measurement and the greater the rigour and complexity that can be applied in the
process. But with limited resources, what you would like to achieve from impact measure-
ment needs to be much more tempered and focused.
In fact there are two parts to this resources question. The rst is the resources required
to set up the process and the second those required to implement and use the process.
Depending on the complexity of your approach, you could expect to spend three to six
months establishing the methodology and training the team. To implement and use the
process you could expect to have one person dedicated part time to impact measurement.
The aim is for impact measurement to become an integral part of the investment process
so that it is used by all VPO/SI team members on a daily basis, but it is useful to have
someone responsible for the overall process.
The issue of constrained resources is often heard as a reason preventing VPO/SIs and SPOs
from getting started on impact measurement. Sometimes this is more a mental barrier, and
we hope this manual provides sufcient practical recommendations in order to get started
on impact measurement without incurring high costs. Other times impact measurement
is seen as a burden, driven by VPO/SIs and/or to be outsourced to external consultants.
A resource called “The Good Investor: A Book of Best Impact Practice”13 focuses on
integrating impact measurement into the investment process. The guide recommends
impact investors to include the following functions to make impact measurement an
integral part of the investment process:
•An investment team that understands the essentials of impact measurement.
•Some in-house expertise regarding impact analysis (either within the investment
team, or active in supporting it).
•A person with a head of impact role (if not a full time position, this responsibility is
clearly assigned to someone, and included in their job description).
•An investment committee with diverse membership, including social and invest-
ment expertise, with members who are able to read impact reports, understand the
key parameters at play, and integrate impact into the making of reasoned investment
decisions.
A survey of 1000 SPOs in the UK by New Philanthropy Capital14 showed that more than
half put meeting funders’ requirements as a key driver for impact measurement versus
only 5% saying that the main driver was improving services. However the main benet
that SPOs said they found when they did measure their impact was not increased
funding but improved services!
13. Hornsby, A. & Blumberg, G.,
(2013), “The Good Investor: A Book
of Best Impact Practice”. Investing
for Good.
http://cdn.goodinvestor.co.uk/
wp-content/uploads/2013/01/
thegoodinvestor.pdf
14. Ní Ógáin, E.; Lumley T.;
Pritchard, David., (October
2013), “Making an Impact: Impact
measurement among charities and
social enterprises in the UK” New
Philanthropy Capital.
JUNE 2015 41
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
On the other point, consultants can indeed provide useful guidance and advice; however
allocating internal resources to focus on impact measurement is vital. To perform a good
impact analysis it is important to know the organisation well and assigning internal
resources to these tasks ensures any learning about impact measurement remains within
the organisations so as to inform their strategy, structure, policies and procedures i.e. to
improve services.
(iii) What type of SPOs are you working with?
All SPOs are different. Specically, the maturity i.e. the stage of development of the SPO
will potentially limit the type of information that the SPO can provide. You should also
consider what assistance the SPO requires in order to provide you with the data needed
to measure impact. In addition, the complexity of the issue that the SPO addresses may
also constrain your impact measurement process and should be considered upfront when
deciding on the scope of your impact measurement.
(iv) What level of rigour do you require in your impact analysis?
A point that we develop further in Step 2 is the concept of how accountable do you expect
your investees to be when assessing their impact? By increasing their accountability you
increase the rigour of your analysis and thereby reduce the risk associated with any impact
claims made. However the ability to do so depends on the type of SPOs you are working
with and the resources that can be dedicated to the process.
(v) What is your time frame for measuring impact?
In theory time frame should not be the driver for impact measurement as the time period
over which you measure impact should be determined by the most important outcomes
and the estimated length of time required to achieve them. However in practice VPO/
SIs may have external or internal pressures to invest for a certain period of time, which
will affect their ability to collect sufcient data to measure impact. We recommend that
although there is often a temptation to measure only outputs, especially when looking at
shorter investment periods (less than 5 years), all VPO/SIs should aim to go a step further
and concentrate on the outcomes of their investments. We discuss the difference between
outputs, outcomes and impacts as well as how to select appropriate indicators in Step 3.
LGT Venture Philanthropy (“LGT VP”) estimates that it took them six months to
establish the methodology and another 3 months to train the team. For them termi-
nology was the main issue as it was important to establish a common dialogue within
the team. They selected the logic model as the principal framework, given its clear de-
nitions, guidelines and examples. However despite the clarity in the framework it took
a while for the team to get up to speed. LGT VP then mapped the logic model to the
Millennium Ecosystem denitions of quality of life. This enhancement of the original
model to describe how out-comes improve a specic dimension of quality of life added
complications and increased the time needed for the team to become comfortable with
the approach.
42 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
To set their objectives a VPO/SI should answer, at a minimum, the following questions:
(i) What is the overarching social problem or issue that the VPO/SI is trying to solve?
Some VPO/SIs decide early on to focus on a particular social issue such as the problem
of youth unemployment. Others have a broader social sector focus, which makes it more
difcult to clearly dene the social problem or issue that they are trying to solve. The
response should include information about the nature and magnitude of the problem or
opportunity; which populations are affected; whether the issue is changing or evolving and
in what way it is changing or evolving. This analysis will allow you to understand the base
case and therefore, at a later stage, allow you to see whether there has been any change from
this base case. A clearly articulated response is necessary to be able to choose investments
in SPOs that can contribute to solving that social issue that the VPO/SI is addressing. For
the impact measurement process, the VPO/SI needs to consider this question clearly before
starting to make investments, and regularly revise and adapt as its investment strategy
develops.
(ii) What objective does the VPO/SI want to achieve?
The response should look at both their overall impact objectives as well as the relationship
to be built with investees. For the overall impact objectives, the VPO/SI should consider
what changes it wishes to achieve as opposed to the base case social issue previously iden-
tied. The next question will include how to achieve those changes by investing in SPOs
whose work is aligned with the objectives of the VPO/SI. The role of the VPO/SI will be
to provide the SPO with the support needed to help the SPO achieve those objectives. Sub-
questions to assist in answering the question on the VPO/SI – SPO relationship include:
•What is the problem SPOs are facing?
•What solutions are available which are provided by the VPO/SI?
•What is the correlation between these two points?
(iii) What are the expected outcomes?
This should include what the VPO/SI must achieve in order to be considered successful
and will form the basis of the milestones against which the VPO/SI will be measured.
For the VPO/SI it is important to evaluate the expected outcome of its investment in the
SPO, i.e. the expected outcome of the SPO and how the VPO/SI expects to contribute to
achieving that outcome. Given these are likely to evolve over time, it is best to organise
by time, ranging from specic (i.e. immediate) to broad (i.e. long-term). It is important
to consider potential unintended consequences of the VPO/SI’s activities. For example, a
VPO/SI that provides a large grant to one of the players in a particular social sector and
region may distort the market by creating an unfair competitive advantage (even though
that is not its intention). This risk may be mitigated for example by offering other nancing
instruments.
Tools such as theory of change, logic model or the initial steps of social return on invest-
ment (“SROI”) may be useful at this stage.
JUNE 2015 43
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
Level of SPO
To understand and set the objectives of a particular investment or intervention, a wide range
of support systems, methods and tools are available. Tools recommended to assist VPO/SIs
in setting their own objectives, such as theory of change, logic model, and particular parts
of methodologies such as SROI or balanced score card (which are themselves based on the
theory of change) are equally useful when working with SPOs on this step of the process.
The manual has extracted the commonalities of the various tools mentioned to come up
with a recommended list of questions to go through when dening objectives at SPO level.
At a minimum you should answer these questions about the SPO15
(i) What is the social problem or issue that the SPO is trying to solve?
As per the recommendation for VPO/SIs, the response should include information about
the nature and magnitude of the problem or opportunity; which populations are affected;
whether the issue is changing or evolving and in what way it is changing or evolving. This
analysis will allow you to understand the base case and therefore, at a later stage, allow
you to see whether there has been any change from this base case.
(ii) What activities are the SPO undertaking to solve the social problem or issue?
This should include a description of exactly what the SPO is doing to try to effect a change.
It should include a set of specic steps, strategies or actions arranged in a logical sequence
demonstrating how each activity relates to another.
(iii) What resources or inputs, as per the impact value chain, does the SPO have and need to
undertake its activities?
This should include the time, talent, technology, equipment, information and other assets
available to conduct the activities. Ideally it should also consider whether a mismatch
exists between the activities and the resources available to execute those activities. As a VP
investor, you should also consider what would be your contribution to helping the SPO to
solve the issue (access to networks, capacity building etc.) as a key input.
15. Elaborated from Centers for
Disease Control & Prevention,
“Framework for program evaluation
in public health”.
In 2011, Noaber Foundation (“Noaber”) changed its strategy completely to focus solely
on healthcare. The rational for this change was that Noaber saw one of its roles as that
of connecting people/organisations and creating synergies so as to achieve impact at
an aggregated scale. This was more feasible when investees were active in the same
sector. With the denition of this new strategy, Noaber had to think of its own impact
objectives. For that it created a theory of change for Noaber. Now each time they
consider a new investee, they map the investee to Noaber’s theory of change to under-
stand how the new investee adds value to Noaber’s overall objectives and its goal of
collective impact.
44 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
(iv) What are the expected outcomes?
As per the recommendation for VPO/SIs, this should include what the SPO must achieve
in order to be considered successful and will form the basis of the milestones against which
the SPO will be measured. Given these are likely to evolve over time, it is best to organise
by time, ranging from specic (i.e. immediate) to broad (i.e. long-term). Some forethought
should be given to anticipate potential unintended consequences of the SPO’s activities.
There may be cases where the SPO is not clear on their own objectives, and input and
guidance is required from the VPO/SI. The VPO/SI can work in collaboration with the
investee to assist in setting objectives. However the VPO/SI must bear in mind that its own
objectives may be slightly different from the objectives of the SPO. As long as the objectives
are not opposing it is feasible to move ahead with the relationship, but in the case they are
not, then serious questions need to be asked on the appropriateness of the investment as
part of the investment selection process.
Jan Lübbering and Katrin Elsemann from Streetfootballworld’s Partnership Develop-
ment team had the following advice when thinking about a SPO’s theory of change.
“First of all an organisation needs to be clear about its goals: What would you like your
organisation to be recognised for in terms of actual changes? What is the long-term
change you want to see as a result of your work? Once these basic questions have been
answered it is crucial to think about the pre-conditions that need to be in place for the
long term impact: what changes need to happen at what level – within the target group,
the community and the society as a whole – to lead to the desired impact? How do
external stakeholders inuence these changes and how do the organisations’ own activ-
ities and initiatives contribute to change? What can only be achieved through collabora-
tion and partnerships, and how does that inuence your offering?”
They added that, “It is important to think out of the box when developing a theory of
change. While an organisation explores how change happens it is tempting to simply
explain why it does the respective activities. There is often a certain hesitation to leave
the comfort zone of what one already knows. This new thinking requires considering
many external factors which lead to the desired change and which one may not have
any inuence on. But we advise our partner organisations to take time to brainstorm
freely. There is no reason to fear the outcomes, as this process is valuable and can only
Ferd Social Entrepreneurs investing in The Scientist Factory illustrates a complex issue.
Their vision is that by providing interesting and exciting after-school science classes to
primary school children, more children will be inspired to consider natural sciences as
a career path and opt for science classes in high school and at university. Trying to show
the impact that these classes have on the children who participate is very difcult given
the timeline involved as well as the problem of attributing any decision by the children
to later pursue a scientic career path to the inuence of the classes.
JUNE 2015 45
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
16. Doran, G. T., (1981), “There’s
a S.M.A.R.T. way to write
management’s goals and objectives”.
Management Review, Volume
70, Issue 11 (AMA FORUM), pp.
35–36.
17. Examples thanks to Social
Performance Management
Resource Centre:
http://www.themix.org/social-
performance
“SMART” Objectives16
The concept of “SMART” objectives is now commonplace in management dialogue and
business school text books, but the principles should also be applied to objective setting
in the context of venture philanthropy and social investment.
An objective is considered as “SMART” if it is specic, measureable, attainable, realistic,
and time bound:
•“S” – specic: if it is clearly written so relevant parties easily understand it. The party
should be able to dene what is to be done, the rationale or benet related to meeting
the outcome or goal and what requirements are necessary.
•“M” – measurable: the objective is measurable if it covers at least one measure of
a quality metric, quantity, time and/or cost-effectiveness. Measurable means not
just meeting a standard but evaluating to what extent the standard needs to be met.
Without a specic measure the party is not able to self-monitor how they are doing
relative to the overall objectives of the organisation.
•“A” – attainable: the objective is attainable by the SPO if it is appropriate given the
resources (time, human, capital, technology) it has at its disposal. It should allow for
some stretch to encourage the organisation to meet its goals.
•“R” – realistic: the objective is realistic if it is within reach of the SPO to achieve given
the external context in which the SPO’s activities take place.
•“T” – time bound: the objective is time bound if it can be accomplished within the
evaluation period that has been set by the SPO and/or VPO/SI.
“SMART” objectives can be focused on process objectives, such as infrastructure, human
resources, systems, policies and procedures or on results objectives such as outputs
(or outreach) and outcomes, which usually have a quantitative target with a deadline.
An example of a SMART process objective would be, “Create a new loan product to
t the needs of rural women by the end of 2014”. An example of a “SMART” results
objective would be, ”25% of our clients will move above the poverty line by 2016.”17 In
impact measurement we are generally focused on results objectives when considering
the specic objectives of an organisation, however in very early stage organisations it
may be relevant to also include process objectives, the attainment of which are vital in
order to reach any longer term results objectives.
lead to an increased under-standing of the underlying reasons for existing or future
activities. We consider it as standard good practice for the communities (clients/target
groups) to be actively involved in coming up with the theory of change (constituency
voice), as well as in all relevant planning steps along the way (not as a one-off effort).”
46 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
2.3 Practical Tips
•Setting objectives is a vital step in any impact measurement process and needs to be
considered at both the level of the VPO/SI and the SPO.
•Often VPO/SIs do not spend enough time upfront considering why they want to measure
impact, which makes it difcult to take decisions regarding what may be relevant and
what not when faced with scarce resources.
2.4 Recommendations for Managing Impact
•A VPO/SI needs to formulate clearly what is its overarching social problem or issue so as
to be able to choose investments in SPOs that can contribute to solving that social issue.
•Understanding the current and expected social impact of an organisation early in the
decision process is extremely valuable: it creates a common understanding of the impact
of an organisation among all stakeholders; allows the VPO/SI and SPO to “speak the
same language” and assess at a later stage whether impact has been achieved.
•A VPO/SI should convince the SPO of the value of impact measurement, provide assis-
tance where possible and dene with them the responses to the essential questions to
help them express their objectives.
•Decisions have to be made about the amount of time and resources that a SPO should
dedicate to impact measurement.
2.5 Worked Example18
Throughout the manual we will be illustrating the various steps in the process through the
use of a worked example. The worked example focuses on a VPO/SI that is investing in
early-stage SPOs in Africa.
In setting the scope of impact measurement they needed to consider that their rationale for
impact measurement was driven by three reasons:
•Investment selection: to ensure they are selecting investments that are not only nan-
cially viable but also having signicant impact in their area of focus.
•Ongoing monitoring: facilitating their offering of technical assistance.
•Reporting: to existing shareholders as well as to assist in raising additional funding from
other parties.
In addition, like many VPO/SIs, the investment team is small and resources are tight, which
frames how much time and money they can dedicate to impact measurement. However
they do have a de facto head of impact and are keen to pursue a rigorous process focusing
on outcomes even if they may not be able to accurately determine impact (according to the
technical denition) in all cases. Their investment approach includes a focus on providing
technical assistance and measuring impact as well as the other inherent characteristics of
a venture philanthropy approach. Their timeframe for each investment is generally ve to
seven years.
18. Thanks go to Beyond Capital Fund
for introducing us to this example,
which is inspired by and elaborated
from Sanergy’s website:
http://saner.gy
The views contained in this
document are those of EVPA and not
of Beyond Capital Fund.
JUNE 2015 47
THE IMPACT MEASUREMENT PROCESS
STEP 1: SETTING OBJECTIVES
The VPO/SI‘s overarching objective is to improve the lives of people living under the
poverty line in Africa. They believe this objective is best fullled by investing in early-
stage for prot social enterprises operating in the region. They have performed substantial
research and decided that focusing investments in the sectors of water, sanitation and health
will most effectively and efciently allow them to full their objective. Given their focus
on early-stage SPOs, technical assistance, particularly access to networks and mentoring, is
expected to contribute signicantly to the success of their investees.
In this worked example we consider one of their investments in a for-prot organisa-
tion that is aiming to build and scale viable sanitation infrastructure in Kenyan slums,
beginning with Nairobi.
The objectives for the SPO can be considered as follows:
•Social problem or issue19: 2.6 billion people do not have access to adequate sanitation and
this number is not decreasing despite billions of dollars of aid. The resulting disease and
water pollution cause 1.7 million deaths and a loss of $84 billion in worker productivity
each year. In Kenya’s slums, 8 million people lack access to adequate sanitation causing
disease and death.
•Activities: the model involves 4 parts: (i) building a network of low-cost sanitation centres
in slums; (ii) distributing them through franchising to local entrepreneurs; (iii) collecting
the waste produced; (iv) processing the waste into electricity and fertiliser.
•Resources or inputs: equipment (sanitation centres, vehicles for collection, digesters to
convert faeces to fertiliser and to generate electricity); staff (qualied personnel on the
ground in Kenya to supervise building of sanitation centres and selection of franchisees,
employees to collect waste products and transport to digesters, operators of digesters to
produce electricity and fertiliser); partners (implementation partners for education about
sanitation, technical partners in the design of toilets, digesters / composters, micro-
nance organisations to support franchisee purchases); funding (grants and investments
from foundations and social investors).
•Expected outcomes: positive expected outcomes at a local level include increased access to
sanitation facilities for slum dwellers, increased employment levels among slum dwellers,
improved health for toilet users and overall slum; increased income for toilet operators;
improved environmental situation (less waste in open waterways). At a national level,
positive outcomes could include a decrease in the number of power shortages, a decrease
in carbon emissions, a decreased reliance on imported fertilisers and a decrease in the
use of chemical fertilisers leading to positive environmental effects. Potential negative
outcomes could be displacement with respect to existing operators of toilets in the slum;
zero job creation through people leaving one organisation to work with this one; reduc-
tions in sales and hence livelihoods of existing producers of fertiliser.
19. Source: Sanergy website –
http://saner.gy
48 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 2: ANALYSING STAKEHOLDERS
3.0 Step 2: Analysing Stakeholders
3.1 What?
There is a strong link between setting objectives and analysing stakeholders for both VPO/
SIs and SPOs as, depending on your scope of impact measurement and your objectives for
social impact, the stakeholders to be considered will be different.
VPO/SI investments generate value for a variety of stakeholders. We will analyse the stake-
holders at two different levels (VPO/SI and SPO), however we can dene a stakeholder so
that it is relevant for both levels of analysis:
“Any party effecting and/or affected by the activities of the organisation.”
There are different categories of stakeholders (which are not necessarily mutually
exclusive). Some contribute with inputs in the process, others contribute to or benet from
the results and/or impacts, and others only contribute to or benet from the outcomes.
These stakeholders can be considered direct or indirect and as beneciaries or contribu-
tors. Beneciaries can be positively or negatively affected by impact and contributors can
enhance or decrease impact.
Stakeholder analysis is an important part of impact measurement because:
•We need to understand the expectations of the stakeholders, their contribution to and the
potential impact our work will have on them. If these expectations are in opposition
to each other then it is likely that the VPO/SI or SPO will have severe difculties in
achieving its social impact objectives.
A SPO that focused on getting long-term unemployed people back into employment
based on a variation of “welfare to work” programmes is a good example. For two years
these people received a salary from the SPO (subsidised by the government) rather than
from the employing company. Other than the participants themselves, two important
stakeholders were the government (subsidising the salaries for two years) and the
employing company (accepting to take on the long-term unemployed for two years). For
the government, the expectation was that after the 2-year period, the people receiving
subsidised salaries would be offered a permanent job and taken onto the payroll of the
company. However, the company saw this as an opportunity to have free labour for
2-years and did not intend to hire the participants at the end of the 2-year period. Unsur-
prisingly the SPO did not achieve its impact objectives and eventually closed.
JUNE 2015 49
THE IMPACT MEASUREMENT PROCESS
STEP 2: ANALYSING STAKEHOLDERS
•The co-operation of the main stakeholders in the impact measurement process is critical.
3.2 How to?
Applying to both the VPO/SI and SPO level, there are two important aspects of stakeholder
analysis: stakeholder identication and stakeholder engagement
(i) Stakeholder identication
Under stakeholder identication we have identied three separate but equally important
tasks: (a) stakeholder mapping, (b) stakeholder selection and (c) understanding stakeholder
expectations.
(a) Stakeholder mapping
To perform the stakeholder mapping we need to keep in mind the objectives that have been
set in Step 1 at the level of the VPO/SI and the SPO.
Level of VPO/SI
At the level of the VPO/SI we need to remind ourselves what is the overall scope of the
VPO/SI’s impact measurement and who is the target audience for impact measurement.
This will ensure that when the VPO/SI reaches Step 5 it is in a better position to customise
its data analysis and prepare the various reports.
More immediately we need to consider the impact objectives of the VPO/SI and who are
the relevant stakeholders contributing to achieving those objectives and ultimately who is
affected by the intervention.
Level of SPO
At the level of the SPO, we have already answered questions around the issue being
addressed, the activities of the SPO, the available resources and the expected outcomes.
These answers should guide us as we list the direct and indirect contributors as well as the
direct and indirect beneciaries from the SPO’s actions.
As an example we can consider a SPO that supports ex-offenders in seeking employment
with the aim of reducing re-offending rates. In this case we can highlight certain stakeholders
within this framework. The direct contributors are the staff at the SPO, the indirect contributor
is the family of the ex-offender, the direct beneciary is the ex-offender who is the focus of
the SPO and the indirect negative beneciaries are those people who do not receive job offers
because the ex-offender was employed instead (an effect also known as job displacement).
LGT VP interviews during its due diligence process people who have already benet-
ted from the products or services of an organisation. For them these real case studies
provide an important source of information regarding the organisation’s impact on less
advantaged people.
50 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 2: ANALYSING STAKEHOLDERS
The matrix below classies the various types of stakeholders.
(b) Stakeholder selection
Level of VPO/SI
For a VPO/SI the stakeholder selection process as it relates to the scope of impact measure-
ment and eventual reporting in Step 5 should be relatively straightforward. For example, if
your focus of impact measurement is investment selection then your key stakeholders will
be the staff of the organisation (portfolio managers especially) and the board of directors
(or which ever entity approves investments). However if your objective of impact meas-
urement is external reporting and communication then you will select those stake-holders
mostly affected by this activity i.e. investors / donors.
However, for each specic investment, the VPO should identify the key stakeholders of
the intervention. The key stakeholders contributing would be donors/investors in terms
of nancial resources, as well as staff, consultants, volunteers of the VPO, and also the
broader networks in terms of providing human and social capital. The beneting stake-
holders would be the SPO and its ultimate beneciaries. To mitigate the risk of unintended
consequences one would need to consider other organisations or communities that might
be affected by the intervention. That should be part of the due diligence process.
Level of SPO
To mitigate potential selection bias when asking the SPO to provide a list of stakeholders
for you to contact you can:
•Explicitly ask the organisation to include some parties where the outcomes were not
ideal.
•Reach out to parties through your own network who were not necessarily identied by
the SPO but who are familiar with its work.
•Always ask the stakeholders to discuss the successes and failures they have experienced.
•A the end of the interaction with the stakeholders ask them to identify other parties with
whom they think you should speak in order to build a balanced view of the SPO’s work.
At this stage it is likely that you will have a long list of stakeholders. Informed by the objec-
tives of the VPO/SI and those of the SPO you should be able to rank those stakeholders
Direct Indirect
Contributor Direct contributor e.g. Staff at SPO Indirect contributor e.g. family of
ex-offender
Beneciary Direct (positive) beneciary e.g. ex-
offender who is the focus of the SPO
Indirect (negative) beneciary
e.g. those people who do not
receive job offers due to the ex-
offender being employed
Source: EVPA
JUNE 2015 51
THE IMPACT MEASUREMENT PROCESS
STEP 2: ANALYSING STAKEHOLDERS
in order of importance. Our suggestion is not to try to measure everything so you should
select the top ve to ten stakeholders to be the focus of the rest of the analysis. At this
point concerns regarding resources (time, manpower, capital) come to the fore, as you must
decide what level of accountability (further described below) you want to accept in order
to perform a valid analysis.
Two important questions you can ask to help reduce the number of stakeholders are:
(i) How material are the benets and inputs provided by these stakeholders?
(ii) How relevant is the stakeholder group to my primary mission?
One common question asked is how to decide which stakeholders experience material
out-comes in advance. Clearly this entails risks that should be acknowledged and can
be framed as per the question of how accountable the SPO should be. Additionally by
focusing on stakeholder groups relevant to the primary mission and not including certain
stakeholders in the analysis we may miss large positive or negative outcomes that would
affect our overall impact analysis. However it is important to keep in mind that this is a
learning process and as you go through the impact measurement process you can reassess
the list of stakeholders and make adjustments.
(c) Stakeholder expectations
Level of VPO/SI
Once you have selected the stakeholders you should understand their expectations. It is
important that key stakeholders buy in to the impact objectives of the VPO so that their
expectations are managed and their contributions are aligned. For example, this means
that donors/investors should be clear about the objectives of the VPO when they commit
money, staff and consultants should know what the goals are that they are trying to achieve
with their work, and SPOs should know what the VPO is expecting them to change. If for
example certain investors/donors have very different expectations to yours, then you may
need to consider how appropriate an investor/donor they are for your VPO/SI, so as to
avoid potential issues at a later stage.
In your engagement with SPOs it is also important to understand what they expect from
the relationship, for example in terms of non-nancial support, to ensure alignment on
these topics.
Level of SPO
With the list of 5 to 10 stakeholders you should then understand their expectations.
Even if the stakeholders share a common objective, the expectation of how impact materi-
alises for each of them in a tangible way may differ considerably. For example, in the UK
a social impact bond linked to an organisation that aims to reduce the re-offending rate of
ex-prisoners has the UK government and the entrepreneur of the SPO among its stakeholders.
The objective of both these stakeholders is to reduce the re-offending rate of ex-prisoners,
52 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 2: ANALYSING STAKEHOLDERS
however the UK government’s expectation for impact is in the reduced problems (particu-
larly budgetary and prison over-crowding) caused by re-offenders, whereas the entrepre-
neur sees impact more in increasing the quality of life of the ex-prisoner so they have no
desire to re-offend.
Importantly there is a distinction between differing expectations, something that is natural
and inherent in venture philanthropy and social investment, and opposing expectations,
which as we demonstrated earlier, can be disastrous for the success of a VP investment. If
it is found that stakeholders do have opposing expectations, then you should take action
in the form of assessing how this may impact on the success of the investment and decide
whether or not to continue or not with the investment. Generally the recommended way to
nd out the expectations of your stakeholders is to ask them. We discuss how to do so in
more detail in the section on stakeholder engagement below.
Mapping stakeholders with respect to accountability: a more in depth way of thinking
about stakeholder relevance
A more in-depth way to consider mapping and then selecting the most relevant stake-
holders is to determine the level of accountability of the SPO in question. For example
should the SPO be accountable for just the intended outcomes on the target beneciaries
or for the outcomes on all stakeholders (positive and negative). We have identied a
spectrum of levels of accountability between these two concepts and illustrate them
through an example of an organisation that wants to help people nd employment:
(i) Accountability for the intended outcomes on the main beneciaries. For example
you would focus on the employment outcomes.
(ii) Accountability for material but only positive outcomes on the main beneciaries,
generalised for the whole group. For example we would consider the trainees who
gained employment but we would not consider the extent to which family support
was critical.
(iii) Accountability for material but only positive outcomes on the main beneciary
group but analysing this for sub-groups. For example you would consider those
trainees who gained employment and who had family support.
(iv) Accountability for material positive and negative outcomes on the main bene-
ciary group and sub-group. For example you would also consider those trainees
who gained employment who had family support and those that did not.
(v) Accountability for material positive and negative outcomes on a selection of the
stakeholders (i.e. not just focusing on the main beneciary group and sub-groups)
For example you would consider the trainees with family support, those without,
the families of the trainees and the employees of the organisation but you would
not consider all stakeholders.
(vi) Accountability for material positive and negative outcomes on all stakeholders e.g.
SROI. For example you would consider the trainees with family support (positive
JUNE 2015 53
THE IMPACT MEASUREMENT PROCESS
STEP 2: ANALYSING STAKEHOLDERS
(ii) Stakeholder engagement
Engaging in communication with the selected stakeholders is recommended to be able to
understand their expectations and, later in the process, verify if their expectations have
been met, which is discussed in more detail in Step 4.
For VPO/SIs, this means engaging regularly with donors/investors, staff and other human
resources, as well as with the investee SPOs so that you are aware of their expectations and
can correct any misalignment before further harm is done.
At the SPO level, based on the outcomes of the stakeholder identication process, there will
be a number of stakeholder groups identied as key. Engaging with them will be part of the
due diligence process of a VPO/SI.
Stakeholder mapping
and selection based
on the concept of
accountability
It is evident that focusing solely on level 1 will bring a quicker estimation of social
impact. However there is higher risk that the impact is misstated and that the SPO
could even be having an overall negative social impact. Level 6 is certainly a slower
and more resource intensive way of considering the social impact of the SPO, however
there is less risk that impact is misstated as social impact on all potential stakeholders
is considered. This trade-off is a decision for the VPO/SI and should be based on its
motivation for impact measurement, its resources (human, capital, time) as well as the
relationship with the SPO and its resources and motivations.
Source: EVPA20
20. Inspired by conversations with
Jeremy Nicholls, SROI Network
(now Social Value UK).
2
1
3
4
5
6
SPO
outcome, may gain employment), trainees without family support (negative
out-come as they did not gain a qualication or employment, in fact they became
more depressed meaning they are less likely to gain employment in the future),
families of the trainees, employees, suppliers, funders etc.
1. Accountable for SPO intended outcomes on
main beneciary
2. Accountable for material but only positive
outcomes on main beneciary generalized for
whole groups
3. Accountable for material but only positive
outcomes on main beneciary groups &
analysing these for sub-groups.
4. Accountable for material outcomes on main
beneciary group & analysing these for
subgroups (positive & negative)
5. Accountable for material outcomes on some
stakeholders (positive & negative)
6. Accountable for material outcomes on all
stakeholders (positive & negative)
54 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 2: ANALYSING STAKEHOLDERS
Within each particular stakeholder group you should aim to construct a sample that is of
an appropriate size and diversity, for example a mix of male and female, older and younger
people. The size of the sample will depend on the reach of the SPO. However what is
important is ensuring a good sample selection that is non-biased and random. Discussions
on scientic (econometric) sampling methods suggest that a good rule of thumb is about
20–120 respondents for the sample to be credible, depending on the size of the population.
After that the increased sample size merely decreases the standard error of the ndings.
The communication channel selected should be appropriate for the stakeholder, and may
require different methods for different stakeholders. For example, an elderly population
will need to be approached via face-to-face interviews, while a group of youths can be
polled via internet surveys. One point to keep in mind in any interaction however is the
importance of “neutral” questioning, so that the stakeholders can give their answer without
overt direction or pressure from the VPO/SI.
In some cases it may appear difcult if not impossible to communicate with a stakeholder
(for example the families of the ex-offenders). Our recommendation is that if a stakeholder
is to be included in the analysis then a method of communication should be found, even
if this is via an intermediary. Without engaging with the stakeholders it is impossible to
understand their expectations and then verify whether those expectations have been met.
3.3 Practical Tips
•Begin stakeholder analysis focusing on a small number of relevant stakeholders and
expand from there, rather than trying to measure everything in one go. This guide is not
set up to advise you to measure everything!
•Screen the list of stakeholders for the materiality of the benets or contributions of these
stakeholders and the relevance of the stakeholder to the SPO’s achievement of its mission
(with the understanding that this is a learning process so over time the risk of missing
large positive or negative outcomes decreases).
•As you become more experienced in impact measurement you can consider those stake-
holders who contribute to or benet from the side effects (negative or positive) of the
SPO’s work.
•Engaging with stakeholders on multiple occasions may not be feasible. Assess when is
the optimal time to engage and then ensure all possible preparation has been completed
prior to such time in order to get the most out of the interaction.
3.4 Recommendations for Managing Impact
•Engagement with a VPO/SI’s key stakeholders (donors/investors, staff/human resources,
SPOs) should happen upfront by making sure they understand and support impact objec-
tives, and any major changes in these objectives should be properly communicated.
•Regularly engage with the VPO/SI’s key stakeholders to ensure that objectives continue
being aligned, and otherwise implement corrective measures.
JUNE 2015 55
THE IMPACT MEASUREMENT PROCESS
STEP 2: ANALYSING STAKEHOLDERS
•When a VPO/SI makes an investment in a SPO, stakeholder analysis should be part of
the due diligence phase.
•To avoid wasting resources, increase the intensity, i.e. more stakeholders, more involve-
ment from the same stakeholders and higher numbers involved from each group, up to
the number required for a non-biased and random sample of the analysis as you increase
your condence that you will pursue an actual investment.
•As the investment period proceeds, regularly get back to the stakeholders to verify that
their expectations are being met (more details on how to do this in Step 4).
•Consider upfront when would be the appropriate time to revisit stakeholder analysis
with the investee SPO. For example this could be when signicant developments occur,
such as a change to outcomes being achieved, major new funding streams, new business
lines being entered, changes to policy environment etc.
3.5 Worked Example
In our example we focus on the stakeholders of the SPO. The stakeholders are the toilet
users, the toilet operators, the waste collectors, the broader slum dwellers, the employees
of the SPO, other health & sanitation organisations working on educational initiatives,
micronance organisations, the government, existing fertiliser producers, existing power
companies, farmers and the VPO/SI itself. These are classied as direct or indirect and
contributor or beneciary in the table below.
The VPO/SI would rank the importance of these stakeholders as follows: toilet users, toilet
operators, broader slum dwellers, waste collectors, SPO employees, farmers, existing ferti-
liser producers, existing power companies, government. Given the resources and time that
the VPO/SI has available, the early-stage nature of the SPO and the view that these stake-
holders are most relevant for the VPO/SI to decide if it is achieving its mission; the VPO/SI
decides to focus its analysis on the rst three stakeholders: toilet users, toilet operators and
broader slum dwellers.
Direct Indirect
Contributor Toilet operators
SPO’s employees
VPO/SI
Government Health & sanitation
organisations
Micronance organisations
Beneciary Toilet users
Toilet operators
Waste collectors
Slum dwellers
SPO’s employees
Government
Farmers
Existing fertiliser companies Existing
power companies
Source: EVPA
56 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 2: ANALYSING STAKEHOLDERS
You can consider the expectations of these stakeholders as follows:
•Toilet users: pay an amount of money to use a clean toilet, they therefore expect the toilet
to be clean and may expect to have fewer health problems.
•Toilet operators: earn income from the toilets and pay the franchise fee. They expect to
have a steady stream of customers for their toilets and the necessary franchisor support
from the SPO in case of any problems with the toilet.
•Broader slum dwellers: if the installation of toilets results in less human waste in the
slums then all slum dwellers may have fewer health problems. However it is unlikely
that slum dwellers will necessarily have this expectation.
Although these expectations do differ, none of them are opposing, therefore we can assume
that the SPO will not have difculties in this area. To understand the expectations, you
should engage with the specic stakeholder, remembering the neutral questioning tech-
niques and the advice on sampling detailed above.
JUNE 2015 57
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
4.0 Step 3: Measuring Results: Outcome, Impact, Indicators
4.1 What?
To transform the objectives set in Step 1 into measureable results we need to consider
outputs, outcomes, impact and indicators.
In section 1.5 we dened, through the use of the impact value chain and the example of the
SPO building schools in Africa, the rst three of these concepts:
To accurately (in academic terms) calculate social impact you need to adjust outcomes for:
(i) what would have happened anyway (“deadweight”); (ii) the action of others (“attri-
bution”); (iii) how far the outcome of the initial intervention is likely to be reduced over
time (“drop off”); (iv) the extent to which the original situation was displaced elsewhere
or outcomes displaced other potential positive outcomes (“displacement”); and for unin-
tended consequences (which could be negative or positive).
Many VPO/SIs and SPOs may be tempted to focus their measurement on outputs, but
often, simple output measures say very little about the actual outcomes. Imagine a nature
conservancy organisation whose mission is to conserve natural species, which measures
member-ship numbers (an output measure) as a measure of its effectiveness. From 1980 to
2010, membership numbers increase signicantly, hence they conclude that they are being
effective and achieving their mission. However, the membership numbers might have
increased due to the escalating problem of depleted biodiversity. Indeed, if they were to
look at the number of species present in the geographic area where they are active during
the same period they would see that this number has decreased signicantly. By focusing
on an output measure, which was not aligned with their mission of conserving species, they
were unable to measure the true impact of their work. On the other hand, output measures
may be sufcient when there is research that specic outputs do result in specic outcomes.
For example if their mission had been to increase awareness of the nature conservancy
issue then membership numbers (despite being an output measure) could have been one
of the relevant indicators.
The difference between outcomes and impact can very quickly become theoretical when
considering concepts such as attribution, deadweight, drop-off and displacement. In
reality there is no tool or methodology to accurately measure these aspects. The types of
Outputs: the tangible products and services that result from the organisation’s
activities.
Outcomes: the changes, benets, learnings or other effects (both long and short term)
that result from the organisation’s activities.
Social
Impact:
the attribution of an organisation’s activities to broader and longer-term
outcomes.
58 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
studies, which would stand up to scrutiny (e.g. randomised control trials etc.), are very
costly, time consuming and may also open up ethical questions when it comes to excluding
potential beneciaries from the SPO’s solution for the sake of the study. Part of the rational
for this manual is to be a practical guide hence we recommend that for VPO/SIs and
SPOs to measure impact that they calculate the outcomes of their investments while
acknowledging (and where possible adjusting for) where other programmes could have
contributed (e.g. the effect of the welfare state in developed countries) or where there may
be negative effects. i.e. those factors that increase or decrease impact. In some situations
comparing to potential control groups (for example based on research of comparable situ-
ations elsewhere) may also be feasible.
One could argue that impact should be very closely related to outcome, as venture philan-
thropists and social investors should already be aware of the other parties working in their
sector of focus. If there is already a large amount of activity we could question whether
investing in that sector is the best use of the VPO/SI’s funds or whether they should be
targeting different areas where they can really add value. In practice, a rule of thumb
could be to focus on out-comes and impacts that the organisation can actually inuence.
If outcomes and impacts become too detached from the operations of the organisation, the
organisation will lose ownership of the impact analysis.
Reverting back to Step 1: setting objectives, you can consider outputs as directly related to
the activities of the organisation i.e. what is done to effect change in the base case. These
outputs are internal to the organisation and hence easy to measure whereas outcome and
impact are related to the expected and unexpected effects of the activities of the organisa-
tion i.e. what effects the activities of the organisation have on the base case. These are by
denition, outside the scope of the organisation’s activities (but within their scope in terms
of accountability) and hence more difcult to measure.
Source: EVPA
Internal vs external focus: the
use of outputs, outcomes or
impacts
Base Case Changed Case
Outcomes
Impacts
External Focus
Internal Focus
Outputs
Inputs
Organisation
JUNE 2015 59
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
VPO/SIs and SPOs identify and use indicators to manage outputs, outcomes and determine
impacts.
We dene output indicators as “Specic and measurable actions or conditions that assess
progress or regression against specic operational activities.”
We dene outcome indicators as, “Specic and measureable actions or conditions that
demonstrate progress towards or away from specied outcomes.”
An indicator can be expressed in different ways, for example as numbers, as a ranking of
systems or as changes in the level of user approval and further be used to express qualita-
tive and/or quantitative information. Quantitative indicators are numerical. Qualitative
indicators are those based on individual perceptions, for example responses to interview
questions. The types of indicators that exist can also be described at a more granular level
e.g. sector specic, leading, lagging etc. We do not go into more detail in the manual on the
different types of indicators as no one type of indicator is better than another; its suitability
depends on how it relates to the result it intends to describe.
For example21, if a VPO/SI is investing in a SPO focused on increasing access to clean water
then two output indicators could be the number and type of wells installed. The specied
outcome could be a reduction in ill health and mortality and a relevant outcome indicator
could be the increase in the number and proportion of the target population with sustained
availability of clean water for domestic use.
If instead we consider a SPO focused on women’s empowerment through the use of
micro-nance, a target outcome could be improved economic control, choice and status
with respect to men. An output indicator could be the number of loans given and repaid
as agreed. Two outcome indicators could be the % of women with increased disposable
income; and the expansion of their options towards diverse social and economic roles.
21. Inspired from Ruby Sandhu-
Rojon, UNDP, “Selecting
Indicators for impact evaluation”.
22. http://www.un.org/
millenniumgoals/
The United Nations Millennium Development Goals are lofty goals22, however the
UN has identied specic indicators to demonstrate progress towards those goals. For
example goal 1 is to eradicate extreme poverty and hunger, specically to:
(i) Halve, between 1990 and 2015, the proportion of people whose income is less than
one dollar a day.
(ii) Achieve full and productive employment and decent work for all, including women
and young people.
(iii) Halve, between 1990 and 2015, the proportion of people who suffer from hunger.
60 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
4.2 How to?
Level of VPO
Although the explanation above focuses predominantly on the SPO; the VPO/SI, via the
objectives set in Step 1 and the stakeholders analysed in Step 2, should consider its own
outputs, outcomes and impacts and set indicators. The principles of how to select outcomes
and indicators described for SPOs below are equally valid for VPO/SIs.
Measuring impact at the portfolio level is a hot topic in impact measurement at the moment
and there is no common practice yet. VPO/SIs should be aware that the following practices
exist and are being tested by leading VPO/SIs:
•Aggregation of output data e.g. lives touched. VPO/SIs can view the Impetus Trust Impact
Report 2010–201123 as an example. The TONIIC Institute also recently published an
E-guide24 recommending the use of certain IRIS indicators (client individuals, jobs main-
tained in nanced enterprise, earned revenue, net income, new investment capital) at a
cross-portfolio level.
•Measurement of success in achieving dened goals i.e. different indicators per investment
but with an additional overlay of assessing whether or not the goals have been achieved
per investment e.g. Grabenwarter & Liechtenstein’s “Gamma” factor. Further details can
be found in the paper: Grabenwarter & Liechtenstein, 2011, “In search of Gamma: an
unconventional perspective on impact investing.” Based on this general idea, the
European Investment Fund is currently experimenting with an approach that uses an
“impact multiple” to compare an impact objective against an outcome. The result is a
relative measure that can be aggregated. For example, if you make an investment in the
education sector and use rate of attendance as the indicator that reects your objectives.
If the objective is to improve the attendance rate from 50% to 65% and you attain 70%,
the relative multiple is 70/65. This multiple can be used to aggregate across the portfolio.
23. Accessed from
www.impetus.org.uk
24. “TONIIC E-Guide: Impact
Measurement”, (Fall 2012),
TONIIC Institute.
For each of these outcomes between 2 and 4 of indicators have been identied. These are
then monitored on a country-by-country basis. The indicators selected are:
1.1 Proportion of population below $1 (PPP) per day
1.2 Poverty gap ratio
1.3 Share of poorest quintile in national consumption
2.1 Growth rate of GDP per person employed
2.2 Employment-to-population ratio
2.3 Proportion of employed people living below $1 (PPP) per day
2.4 Proportion of own-account and contributing family workers in total employment
3.1 Prevalence of underweight children under-ve years of age
3.2 Proportion of population below minimum level of dietary energy consumption
JUNE 2015 61
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
•Denition of indicators that reect the outcome objectives of the VPO/SI. For example, Noaber
Foundation ensures that it aligns the outcomes targeted through its own theory of change
to the outcomes selected by the SPO. If these outcomes are not aligned it does not go
ahead with the investment. Rather than aggregating all individual SPO indicators, the
VPO/SI can measure how well those general outcome objectives have been achieved.
•Selection of common outcomes at the portfolio level. For example Big Society Capital25 has
dened with the UK government, venture philanthropists, social investors and SPOs a
number of outcomes in each sector in which it operates. Their focus will be on ensuring
that reporting from investees focuses on these outcomes. However they do clearly state
that although outcomes can be used as a mapping tool to show a VPO/SI where they
are active, an outcome map can only be used for aggregation purposes if truly like-for-
like numbers and contexts are involved and issues such as double-counting have been
dealt with.
For a VPO/SI, it is not enough to just consider the impact achieved by the SPO, it is also
important to assess the impact of the work of the VPO/SI on the SPO. As set out in the
Good Investor26, in practice the impact of the VPO/SI on the SPO is apparent in four areas:
•Scale of the investment: the percentage contribution of the investment to the SPO forms a
baseline for the extent to which a VPO/SI can link impacts achieved by the SPO back to
the investment e.g. 25% of SPO capitalised by the VPO/SI translates to 25% of the impact
attributable to the VPO/SI’s investment.
•Growth and strength of the SPO: growth in nancial turnover, increase in strength or resil-
ience of the SPO, growth in impact generating activities and delivery of services, growth
in outcomes and impact.
•Access to other and further capital: here we consider the “deadweight” of the VPO/SI i.e.
without access to other sources of nance, the impact of the VPO/SI’s investment is at
its highest.
• Expertise and networks: this is the important area of non-nancial support that needs to
be tracked and valued.
These areas would need to be evaluated in addition to the objectives that are directly to the
activities of the SPO when a VPO/SI considers its own impact.
The key point for VPO/SIs to remember is that the SPO should report on those outcomes
and indicators that are in line with its objectives. If the VPO/SI requires further informa-
tion in order to full its own information requirements then it should be the VPO/SI that
invests the resources to achieve this. It is important not to overburden the SPO. 25. More information on the
outcomes matrix can be found at:
http://www.bigsocietycapital.
com/outcomes-matrix
26. Hornsby, A; Blumberg, G.,
(2013), “The Good Investor, A Book
of Best Impact Practice.” Investing
for Good.
62 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
Level of SPO
Output and outcome measures are different and should be used in different circumstances.
Output measures are suitable when the focus is on the operational aspects of the SPO (e.g. as
a management tool or for day to day monitoring). However they may also be useful in deter-
mining outcomes when they point in the same direction as the specied outcome or when
there is research that a particular output does result in a particular outcome. For example
if the objective of the SPO were to raise awareness through advocacy then the number of
participants at an event organised by the SPO (an output measure) would be an appropriate
measure to use. However if the objective were to change people’s opinions about a certain
issue, then counting the number of participants would not be appropriate as it says nothing
about whether the event had any effect on the opinions of those participants.
When focusing on output measures, there are a few databases that include a large number
of output indicators e.g. IRIS and Global Value Exchange. Where possible we would
recommend that if you do require an output indicator that you rst see if an appropriate
indicator exists within one of these databases and only if it does not, develop your own
indicator. Indeed a number of VPO/SIs follow this policy.
Standardisation of output indicators serves two important purposes:
(i) Ensuring that you and the SPO are aligned on the specics of the indicator (the indica-
tors in databases are very clearly dened).
(ii) Reducing the burden on the SPO, as if all VPO/SIs can request the same output indica-
tors then this reduces the multiple reporting burden of the SPO.
Outcomes should be your key focus as soon as your rationale for impact measurement
moves beyond the operational towards investment selection, external reporting etc.
Whether you then consider output or outcome indicators as relevant for showing your
progress towards your outcomes will depend on the nature of the business and the
outcomes you are targeting. Output indicators may be sufcient if the operations of the
SPO are very directly generating impact or if there exists independent research showing
that specic outputs do result in specic outcomes.
This focus on outcomes is reected in other impact measurement initiatives that are taking
place at the European level.
LGT Venture Philanthropy and Bamboo Finance try as much as possible to use IRIS
indicators. In case indicators do not exist in IRIS then they dene their own in close
collaboration with the SPOs.
JUNE 2015 63
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
27. Hornsby, A. ; Blumberg, G.,
(2013), “The Good Investor, A Book
of Best Impact Practice.” Investing
for Good.
28. Organisation Research Services.
“Outcomes for Success!” A product
of the Evaluation Forum, Jane
Reisman, Judith Clegg, (2000),
pg 3–22 (inclusive).
The change or desired eect In what For whom
Such as:
increase, decrease, maintain,
improve, reduce, expand
Attitude, perceptions,
knowledge, skill, behaviour,
condition, agency,
organisation, community
Population group, programme
participant, client, individual,
family, neighbourhood
Example:
Increase
Awareness of environmental
protection activities
Among community members
Big Society Capital (“BSC”), the UK government’s social investment initiative, has spear-
headed a project to agree with leading venture philanthropists and social investors the
outcomes for various target social sectors in the UK27. The resulting outcomes matrix
provides an overall framework for outcomes in relation to beneciaries. Each cell within
the matrix houses a list of the high level outcomes that can be achieved within that
outcome area for the dened beneciary group. These high level outcomes break down
further into detail outcomes, and the indicators that can be used to measure them. The
full outcomes matrix, with the complete list of indicators is not yet complete but will
soon be available for download, it will be integrated into the Global Value Exchange
platform, and will sync with IRIS indicators.
Given our recommendation to VPO/SIs to focus on outcomes and then select appropriate
indicators, the next paragraphs provide guidelines on how to do this in practice.
(i) Dening outcomes
As a starting point to transform objectives into more concrete and measurable results, an
organisation may state outcomes in a number of different ways28. The desired outcomes
should be in line with the objectives set in Step 1 and the organisation should be aware that
different stakeholders seek different outcomes.
We identify three main types of outcomes:
•Outcomes focused on change: including the increase, maintenance, or decrease in behaviour,
skill, knowledge or attitude e.g. increase immunisation among young children.
•Outcomes focused on targets: stating specic levels of achievement e.g. immunise 80% of
2 year old children in the community according to recommended public health schedules.
•Outcomes focused on benchmarks: including comparative targets, generally related to other
time periods or organisations e.g. increase the current 70% immunisation rate for children
aged 0–24 months to 90% by the year 2015.
The following tables can help you dene specic types of outcomes.
(i) Outcomes focused on change
64 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
29. Organisation Research Services.
“Outcomes for Success!” A product
of the Evaluation Forum, Jane
Reisman, Judith Clegg, (2000),
pg 3–22 (inclusive).
(ii) Outcomes focused on targets
(iii) Outcomes focused on benchmarks (converted from a target statement)
There is however an issue when using any form of percentage statements in that without a
proper context you cannot know whether the change you are seeing is positive or negative.
For example if the % of community members who are active in environmental protection
increases from 55% to 60% but the community itself reduces in size, then the % increasing
on its own does not tell you much about whether more or less people are involved in envi-
ronmental protection activities.
(ii) Selecting outcomes
Outcomes are often lofty and abstract, so how do you set a concrete target for whether the
desired outcome has been achieved or not? This is where indicators come into play because
if you claim you have an outcome you need to be able to measure it.
Having gone through the process you may have a number of outcome statements, but it
is important to select only the relevant outcomes as informed by your mission, rationale
for impact measurement and the stakeholders you are focusing on. Some methodologies
aim to assign outcomes per stakeholder. However we prefer to use stakeholders as a lter
to select among outcomes. To assist in the selection you can ask yourself the following
questions29:
•Which outcomes are most important to achieve (this will depend on the prioritisation
you assign to the stakeholders)? Which are most closely related to the core business of
the organisation?
•Are the outcomes material? Is the change or benet something that makes a real differ-
ence for the key stakeholders?
The amount of change For whom In what
Such as:
Percentage, rate, ratio,
amount
Population group, programme
participant, client, individual,
family, neighbourhood
Attitude, perceptions,
knowledge, skill, behaviour,
condition, agency,
organisation, community
Example:
55%
Of community members Will increase their involvement
in environmental protection
activities
The amount of change For whom In what Against what standard
Example:
55%
Of community members Will increase their involvement
in environmental protection
activities
As compared to the 2010 rate
or To exceed the national
standard of 50%
JUNE 2015 65
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
•Which outcomes are most useful? Which will provide the best information for manage-
ment decision-making, investment selection, reporting or whatever other purpose you
have for impact measurement?
•Which outcomes are most feasible? Which are most likely achievable with the resources
available? Which are likely achievable within the designated evaluation period? It is
important to reiterate that this question relates to achievability of the outcomes and not
the feasibility of their measurement.
(iii) Selecting indicators
Having selected the outcomes you need to select appropriate indicators. The key challenge
with indicators is to ensure their quality and integrity. Indicators should generate data
that are needed as well as useful because if they are not used carefully they can consume
extensive resources and generate data with little or no value.
A guiding principle for selecting indicators is that if you are looking at a sub-optimal
situation e.g. low self-esteem of adolescents then there must be some measurable evidence
of this within that group versus the norm e.g. not nishing school and/or not paying debts.
It is that type of evidence that needs to form the basis of the indicator. We recommend that
you select the top three issues that demonstrate that a situation is sub-optimal. These issues
should form the basis of your indicators.
The Global Reporting Initiative30 uses the principles of materiality, stakeholder inclu-
siveness, sustainability context and completeness, when identifying the topics that
are of relevance. These principles can also be applied in social impact measurement:
•Materiality: information should cover topics that (a) reect the organisation’s signi-
cant (i.e. require active management or engagement by the organisation) economic,
environ-mental and social impacts, or that (b) would substantively inuence the
assessments and decisions of stakeholders.
•Stakeholder inclusiveness: the reporting organisation should identify its stakeholders
and explain how it has responded to their reasonable expectations and interests.
Failure to identify and engage with stakeholders is likely to result in reports that are
not suitable and therefore not fully credible to all stakeholders.
•Sustainability context: the report should present the organisation’s performance in the
wider context of sustainability i.e. discussing the performance of the organisation in
the context of the limits and demands placed on environmental or social resources at
the sectoral, local, regional or global level.
•Completeness: Coverage of the material topics and indicators and denition of the
report boundary should be sufcient to reect signicant economic, environmental
and social impacts and enable stakeholders to asset the reporting organisation’s
performance in the reporting period.
30. “Sustainability Reporting
Guidelines”; Version 3.1; Global
Reporting Initiative.
66 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
31. For more information refer to
progressoutofpoverty.org
We have identied four factors that constitute a “good” indicator:
(i) Indicators should generally be aligned with the purpose of the organisation. Although if
a potential unintended outcome has been identied, relevant indicators for this outcome
may by denition not be aligned with the purpose of the organisation.
(ii) Indicators should be SMART: specic, measurable, achievable, relevant, time-bound.
(iii) Indicators should be clearly dened so that they can be reliably measured, and ideally,
comparable with those used by others so that performance can be better benchmarked
and understood in a broader context.
(iv) Indicators i.e. more than one should be used, with a preference for two to three. For
example if your objective is to increase women’s empowerment and one outcome is
that they take better care of their health, then an appropriate indicator could be the
number of times they visit their doctor in a certain period. However whether this
number goes up or down, it is very difcult to draw a conclusion as to whether they
are taking better care of their health. At least one other indicator is required and a
conclusion can only be drawn about whether the outcome is achieved by seeing if they
all point in the same direction.
Grameen Foundation’s Progress Out of Poverty Index (“PPI”)31 estimates the likelihood
that an individual falls below the national poverty line, the $1/day/PPP and $2/Day/
PPP international benchmarks. The PPI uses 10 simple indicators that eld workers can
quickly collect and verify. These indicators are derived from the most recent national
household income or expenditure survey or the country-specic World Bank Living
Standards Measurement Survey, depending upon which dataset has the most complete
information, for each country. All indicators on the national household survey are
ranked according to how strongly they predict poverty levels. The full list of 400-1000
indicators is narrowed to the 100 most powerful ones. Using both statistics and expert
judgement a 10-indicator scorecard is constructed. Each possible response is assigned
a point value on the original national survey responses. The total score (summing from
0 to 100) is then linked to the probabilities of falling above or below the poverty lines.
Jan Lübbering and Katrin Elsemann from Streetfootballworld’s Partnership Develop-
ment team had the following advice:
•Dene measurable indicators for the key outputs and outcomes that are useful and
meaningful to the organisation.
•Choose a mixture of both quantitative and qualitative indicators and refer to already
existing indicators from other players in the same eld.
One helpful question is: why do we need this information and do we have the capacity
to collect it ourselves or is somebody else collecting this already? Is there an easier
way to get the relevant information/feedback from the stakeholders/beneciary/
community etc.?
JUNE 2015 67
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
(iv) What do you need to consider if you are aiming to measure impact?
To move from outcome measures to understand if the organisation is having an impact,
ve factors need to be considered:
•Drop off: relates to the fact that over time the importance of impact decreases. Impacts
don’t last forever so you need to make some estimation as to the time period for the
impact. The organisation should also be aware of which beneciaries are dropping off
and if there are commonalities among them, so as to be able to improve services.
•Displacement: relates to the fact that with some interventions the positive effect that is
seen in a certain group can be offset by the negative effect seen in a different group
(which was not the target beneciary of the organisation). For example, a new business
opening in a community may bring about the closure of another business already active
in the community.
•Deadweight: relates to a consideration as to what would have happened anyway i.e. in the
absence of the organisation’s activities. It includes the progress that beneciaries would
have made without the organisation’s activities (reducing the impact of the organisation)
as well as the negative consequences of no intervention (increasing the impact of the
organisation).
•Attribution: relates to understanding how much of the change that has been observed is
the result of the organisation’s actions or of the actions of other organisations / govern-
ment etc. taking place at the same time.
•Unintended consequences: are those effects that come about as a result of the organisation’s
activities, but are not part of the desired effect.
The ability of an organisation to measure impact will depend very much on the sector
and geography in which it is operating. For example, in the UK, the development of the
rst Social Impact Bond by Social Finance was made possible by the involvement of the
government, access to public sector statistics on costs for reconvictions, the ability to create
a control group through the Propensity Score Matching method and the involvement of
organisations such as QinetiQ and the University of Leicester to independently assess
Unintended consequences: By dening outcomes in line with objectives implies that
an organisation’s focus is on intended consequences. For an organisation to have a
more accurate calculation of impact they should consider the unintended consequences
of the organisation’s activities, which may be positive or negative. Some unintended
consequences may be foreseen because, although the results of the activities on a
particular community or group were not intended, they are a clear result of the organi-
sation’s activities and hence they should be factored into the dened outcomes and
assigned indicators. However others may only manifest themselves once the activities
of the organisation are underway e.g. beneciaries responding in an unexpected way,
or effects on more peripheral stakeholders than direct beneciaries. To pick up these
unintended consequences an organisation should review its activities periodically as
part of the monitoring and evaluation process (further discussed in Step 5) and then
assess what this means for their overall impact objectives and activities.
68 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
the method and outcomes (more details in the Esmée Fairbairn Foundation case study
in section 9.4). In fact the propensity of European governments to move towards pay for
performance contracts means the measurement of impact is becoming more important for
those organisations active in these areas.
However, for many VPO/SIs and SPOs access to independent statistics and the creation
of control groups in order to assess displacement, deadweight, drop off and attribution
is not possible due to the expense and specialist skills needed to carry them out. In these
cases the resources required to estimate these aspects in a rigorous manner are beyond the
scope of most VPO/SIs and SPOs. Therefore we recommend that social impact should be
measured by calculating outcomes, while acknowledging those factors that could serve to
increase or decrease impact. In some cases it may be possible to think about some evidence
as to what a control group may look like and could be used for comparison purposes, for
example based on research of comparable situations elsewhere. To gain an idea of what is
involved in measuring impacts in an academically rigorous manner we would recommend
reviewing the study undertaken of Grameen Danone Foods Ltd32 in Bangladesh by the
NGO GAIN and John Hopkins university and MIT’s Department of Economics working
paper33, “Up in Smoke: the inuence of household behaviour on the long-run impact of
improved cooking stoves.”
Øyvind Sandvold, Business Development at Ferd Social Entrepreneurs shared that,
“We try to be as cost effective as possible in our measurement of impact but still show
meaningful results. Since we are only working within Norway, a country with a well-
functioning welfare state, it is difcult to isolate the direct impact for each SPO because
there are so many ways impact can be inuenced. What we try to do is to assign indi-
cators that show the effect and if those numbers were greater or lesser (depending on
the context) compared to average numbers for a comparable group we would claim it
is appropriate to assume that there is an impact. At the same time we always collect
good stories from the SPOs to provide the context behind the numbers, so we have
“witnesses” to strengthen the results. We know that this is by no means “bullet proof”
evidence, but it provides us comfort in our impact beyond reasonable doubt.”
32. Found at
www.danonecommunities.com
33. Rema Hanna, Esther Duflo,
Michael Greenstone, Working
Paper 12–10, (April 16 2010),
Reviser (April 30 2012), “Up in
Smoke: the influence of household
behaviour on the long-run impact
of improved cooking stoves”.
34. Source: LGT Venture Philanthropy
LGT Venture Philanthropy’s (“LGT VP”) investment in a ready to use food (“RUF”)
producer in Haiti called MFK illustrates the challenges in moving from outputs to
outcomes and then to impact. MFK dries, stores, roasts and then grinds peanuts into a
paste, before mixing them with proteins, vitamins and minerals. The resulting mixture
is packed into sachets and sold to institutional clients who distribute them for free to
malnourished children in Haiti. LGT VP use the logic model to understand the SPO’s
objectives and map their inputs, outputs, outcomes and impacts. They then overlay the
ve dimensions of quality life, inspired by the UN Millennium Ecosystem Assessment.
JUNE 2015 69
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
Logic Model applied to MFK34
Intended Results
Resources Organisation Activities Impact (systemic)
Products:
MTs of RUFs produced p.a.: 75 (2011), 800
(2015e)
# of products: 2 (2011), 5 (2015e)
MTs of local peanuts purchased p.a.: 40MT
(2011), 400MT (2015e)
Services:
# of farmers trained in agricultural skills
and provided with a stable market at fair
prices: 100 (2011), 1000 (2015e)
Improve physical wellbeing:
In 6-8 weeks, a child treated with RUF has 80%
likelihood of recovery. Once severe malnutrition has
been treated the child can survive on a local diet.
Children cured of severe malnutrition before age 5
perform better at school and develop to be healthier
and stronger.
# patients treated p.a.: 80,000
# patients treated against severe acute malnutrition:
20,000
# children saved from becoming malnourished:
60,000
Improve social well-being:
Preventing a child’s illness and eventual death leads
to avoiding negative impacts, severe trauma and
emotional shock for the family circle
Improve material well-being:
Parents of malnourished children treated with RUFs
can go on with their lives normally as the medicine
does not require medical supervision, cooking of
cooling.
Farmers supported by MFK’s agricultural operations
are provided with technical support and access to a
stable market
Improve security: N/A
Improve freedom: N/A
Eradicate malnutrition in Haiti
Build food security in Haiti
Quality of Life Assessments (“QOL”)
No impact
Low impact
Medium impact
Strong impact
Very strong impact
Model
Resources Organisation Activities
Equipment: peanuts processing factory, transportation vehicles
Supplies: peanuts / peanut paste, vitamins & mineral mix
Staff: qualied personnel with medical and technical expertise
on the ground in Haiti, trained labour force to run factory,
international support team in the USA
Partners: institutional programmes / demand for RUFs,
international support for agricultural development operations
Funding: philanthropic support to combat malnutrition
Production of medicines known as RUFs: MFK produces 75MT
of fortied peanut based foods (RUFs) p.a. in its current factory
and expects to be producing 800MT p.a. by 2015 in an upgraded
facility.
MFK Agricultural Development: MFK conducts 3–5 workshops
p.a. with Georgia University to teach subsistence peanut growers
how to increase yield and quality of harvests, MFK manages 5
demonstration plots and sources 40% of its peanuts locally
70 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
4.3 Practical Tips
•It is not advisable to “pick and mix” indicators from existing databases without the back-
ground work associated with following the impact measurement process. It may save
some time upfront, but you are more likely to conclude that you are wasting resources
collecting data on irrelevant points if you have not gone through the process.
•You should rst reect on the relevant indicators and only then check the existing
databases to see whether some are aligned to yours.
•Don’t just select indicators that are likely to show short-term positive impact. For example
if the theory of change of a SPO states that providing language training to economic
migrants will empower these people, making them less dependent on government
provided services and therefore reduce costs for the government; then a potential indicator
would be the number of people from this community using government services. In the
short term this number may increase as enhanced language skills mean the people can
now ask for these services, however in the long-term the number may reduce.
•Always try to include at least one non-self-reported indicator for each outcome.
•Don’t get too “bogged down” in calculating an accurate number for impact (unless
working with specic governments that have this as a pre-requisite!) Focus your impact
measurement on calculating outcomes and acknowledging those factors that could
increase or decrease impact.
•To pick up unintended consequences review your activities periodically as part of the
monitoring and evaluation process (further discussed in Step 5) and then assess what this
means for their overall impact objectives and activities.
4.4 Recommendations for Managing Impact
•A VPO/SI needs to consider whether to dene portfolio level indicators to measure
how well it has achieved its objectives as an organisation. Measurement of impact at
the portfolio level is a hot topic in impact measurement at the moment and there is no
common practice yet.
•For a VPO, it is not enough to just consider the impact achieved by the SPO, it is also
important to assess the impact of the work of the VPO/SI on the SPO.
For LGT VP the most difcult part of dening their impact measurement process was
to nd a rating method that described the contribution of an organisation towards the
improvement of quality of life. For example MFK improves the health of children. On
average, the families of healthy children have more money than those of sick children
(less spent on medicine etc). Hence, MFK contributes to the material well-being of the
families of the healed children. But how large is that contribution towards the improve-
ment of their quality of life? Answering this question remains a challenge. They try
to nd a pragmatic though reliable way to rate the impact but it’s still the most chal-
lenging exercise.
JUNE 2015 71
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
•The VPO/SI should ask the SPO to focus on those indicators that are directly related to
the SPO’s theory of change and hence in line with their operational process. Any addi-
tional indicators required for the VPO/SI to satisfy their impact measurement needs
should be collected by the VPO/SI.
•Clarify at the beginning of the relationship (i.e. during due diligence and within deal
structuring) who is responsible for measuring what. The responsibilities of who measures
what could and probably should evolve over time as the SPO grows and develops and
should be reviewed on an annual basis. The expected outputs, outcome and impact, and
the corresponding indicators should be dened before the investment is made and agreed
upon by the VPO/SI and the SPO. The indicators can be revised if signicant changes are
made in the business and impact model of the SPO during the investment process.
4.5 Worked Example
Having a clear idea of the VPO/SI’s and SPO’s objectives and the key stakeholders, we are
in a position to consider outputs, outcomes and impacts, as well as the appropriate indica-
tors. The table below shows the SPO’s theory of change and highlights the various outputs,
outcomes and impacts.
Business Model
Inputs Activities
Equipment: sanitation centres, vehicles for collection, digester to process
faeces to fertilisers to generate electricity
Installing toilets
Staff: qualied personnel on the ground in Kenya to supervise building
of sanitation centres and selection of franchisees, employees to collect
waste products and transport to digester, operators of digester to
produce electricity and fertiliser
Recruitment of
franchisees
Partners: implementation partners for education about sanitation,
technical partners in design of toilets and digesters / composters,
micronance partners to support franchisee purchase
Sale of sanitation
services
(via franchisee)
Funding: grants and investments from foundations and social investors Waste removal,
collection and
processing
Electricity generation
Fertiliser production
72 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
Given one of the objectives of impact measurement for this VPO/SI is to monitor the opera-
tions of the SPO, we need to set certain output indicators. These output indicators need to
be in line with the theory of change shown above and to promote standardisation we will
look where possible to use the IRIS indicators.
Expected Eects
Outputs Outcomes Impact
Number of toilets installed
$ revenue from toilet sales
Increased access to sanitation
facilities for slum dwellers
Improved physical well-
being (reduce disease)
Number of toilet operators Increased employment levels
among slum dwellers
Improved material well-
being
Number of users (per toilet &
total)
Number of visits to toilets
$ income of toilet operators
Improve health for toilet users
and overall slum
Increased income for toilet
operators
Improved physical well-
being
Improved material well-
being
Kg waste collected (assuming
kg processed = kg collected)
Improved environmental
situation in slums (less waste in
waterways)
Improved physical well-
being
kWh of electricity produced
$ revenue from electricity sales
Decreased number of power
shortages / outages
Decreased carbon emissions
Improved energy security
Improved environment
Kg of fertiliser produced
Kg of fertiliser sold
$ revenue from fertiliser sales
Decreased reliance on costly
imported fertilisers
Decreased reliance on chemical
fertilisers
Improved material well-
being
Improved environment
JUNE 2015 73
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
Despite being important for monitoring the operations of the SPO, these output indicators
do not necessarily tell us whether the SPO is making progress towards its outcomes. To
do that rstly we need to select the outcomes that are relevant for the VPO/SI to focus on.
Given the VPO/SI’s objective is to improve the lives of people living in poverty we would
naturally focus on those outcomes related to physical and material wellbeing, over and
above those related to the environment. We mentioned previously that the stakeholders of
focus were the toilet users, toilet operators and slum dwellers. With this lter we should
therefore focus on the following outcomes arranged according to the themes of material
and physical well-being.
Improved physical well-being:
1. Increased access to sanitation facilities for slum dwellers
2. Improved health for toilet users and overall slum
3. Improved environmental situation in the slum (less waste in waterways)
Output IRIS Indicator
Number of toilets installed PI9601: Number of units installed by the SPO during the
reporting period
$ revenue from toilet sales PI1775: Revenue from the sales of the product or service
during the reporting period
Number of toilet operators PI2758: Number of micro-entrepreneurs distributing the
SPO’s products/services during the reporting period
Number of visits to toilets PI8783: Average number of client visits to facilities during
the reporting period
Number of users of toilets (per toilet
& total)
PI4060: Number of individuals who were clients during the
reporting period
$ income of toilet operators PI4881: Total earnings generated by the micro-
entrepreneurs from selling the SPO’s products / services
Kg of waste collected Not within IRIS so indicator created as: Number of kgs of
waste collected from the toilets during the reporting period
kWh of electricity produced PI8706: Energy produced during the reporting period
$ revenue from electricity sales PI1775: Revenue from the sales of the product or service
during the reporting period
Kg of fertiliser produced PI1290: Amount of product or service produced by the
organisation during the reporting period
Kg of fertiliser sold PI1263: Amount of the product or service sold by the
organisation during the reporting period
$ revenue from fertiliser sales PI1775: Revenue from the sales of the product or service
during the reporting period
74 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
Improved material well being:
1. Increased employment levels among slum dwellers
2. Increased income for toilet operators
Given the technical equipment needed to test the level of sewage in the slum waterways
the VPO/SI decided to focus on the remaining four outcomes. With each of these you need
to think of the two to three issues that evidence the situation is sub-optimal at present in
order to select appropriate indicators.
For increased access to sanitation facilities, two appropriate outcome indicators can be
found among the output indicators detailed above:
•Number of toilet units installed by the SPO during the reporting period.
•Number of individuals that were clients in the reporting period.
However an important indicator to add to these two would be to understand how the sani-
tation situation has evolved generally:
•Increase (versus the beginning of the SPO’s operations) in the number of toilet type
(including latrines etc.) units installed (by the SPO or by any other organisation) during
the reporting period.
For the improved health of the toilet users and slum dwellers, the users may have to be
surveyed so as to collect data on the following indicators:
•Number of days a toilet user has not been able to be up and about during the reporting
period due to some stomach related illness (deliberately left broad to include the possi-
bility of diarrhoea, intestinal worms etc.).
•Number of outbreaks of typhoid or cholera in the slum area served by the toilets during
the reporting period.
•Average Number of days a slum dweller has not been able to be up and about during the
reporting period due to some stomach related illness.
For the increased employment levels among slum dwellers, it is important to track the
following indicators:
•Proportion of community with some form of regular income through full time and part
time work as at the end of the reporting period.
•Number of employees (toilet operators, waste collectors etc.) of the SPO, including
full-time and part-time (but not temporary), as at the end of the reporting period that
reside in the community where the toilets are situated.
JUNE 2015 75
THE IMPACT MEASUREMENT PROCESS
STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
For the increased income of the toilet operators, another of the output indicators can be
used as well as two indicators that point towards increased wealth:
•Total earnings generated by the micro-entrepreneurs from selling the SPO’s products /
services.
•Proportion of toilet operators with all their children attending school.
•Proportion of toilet operators with their house’s outer walls made from strong materials
(e.g. iron, aluminium, tile, concrete, bricks, stone, wood).
Of the eleven outcome indicators selected, three are also used as output indicators and
one other (SPO employees from the slum) should be relatively easy for the SPO report.
However the remaining seven, which are required to show the progress (or not) towards
the target outcomes, require further investment of time and resources (e.g. information
gathering via surveys) on the part of the SPO. Given the SPO is claiming these outcomes
they should be willing to spend the time necessary to collect the data. However in very
early stage entrepreneurs it is important for the VPO/SI and SPO to agree when this level
of reporting should begin, although the VPO/SI should not be too accommodating in this
respect.
76 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
5.0 Step 4: Verifying & Valuing Impact
5.1 What?
A focus on verifying and valuing companies’ products and services has been present in
management for a long time. The initial focus of this research and within commercial
organisations was on the quality of the product offered i.e. the process of how the product
was made or the service was provided. Management then realised that quality alone was
not enough to satisfy the customer so the focus moved to client satisfaction. Today the
focus has moved again, to demonstrating the value the customer gets from the product or
service. The importance of putting customer value at the centre of your assessment is not
just relevant for commercial organisations but also for social entrepreneurs, their organisa-
tions and VPO/SIs.
Verifying & valuing impact is a step that occurs at 2 levels: that of the VPO/SI and the SPO.
(i) VPO/SI level: as a VPO/SI you believe you are creating value by providing non-
nancial assistance. Unless you verify whether this has occurred and how much the
SPO values this assistance, you cannot credibly make that statement. It may also be
necessary for VPO/SIs to verify at regular intervals that the expectations of other stake-
holders (donors/investors and human resources) are met so that corrective actions can
be undertaken if necessary.
(ii) SPO level: When we set objectives, identify the stakeholders and select the relevant
out-comes and indicators (Steps 1–3), we need to know whether we are really making
progress towards the desired change and the desired outcomes. We need to know
whether we are achieving our objectives, and if so, whether we are achieving them in
the expected amounts.
The focus of this step is predominantly on the second level, the SPO, given there are
generally more challenges in this area. However VPO/SIs should not overlook the impor-
tance of verifying and valuing their own impact on the SPOs and we do discuss this briey.
Given that, with respect to the SPO level, we may have different stakeholders with different
expected outcomes; we need to verify the results at the level of these stakeholders. This can
be a time consuming activity, so it is preferable to start with your most relevant stakeholder
group(s), which in many cases are the beneciaries of the intervention.
In addition, when we verify whether the outcome makes sense for the stakeholders and
if the expected outcomes are realised (within the timeframe and quantity expected) we
also need to verify whether this outcome was important i.e. valuable to the stakeholder(s).
The latter is what we call “valuing impact”. In other words: we need to verify whether the
claim we make on having positive social impact is true, and if so, to what extent (i.e. to
what value). The responses to these questions will allow us to rene the target outcomes
and associated indicators, creating a positive feedback loop in the impact measurement
process and enabling us to effectively manage impact.
JUNE 2015 77
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
Clearly at this point we are assuming that the SPO is a going concern and given the work
done in developing the services that the SPO already knows that the outcomes are of some
value. Otherwise one would question why the SPO has implemented the products or
services in the rst place.
Value is weighing the benets versus the costs/sacrices for the stakeholder (whoever that may
be). VPO/SIs and SPOs normally incur costs, to create value for other stakeholders (i.e. the
direct or indirect beneciaries). And it is common that the direct or indirect beneciaries
reap the benets, without incurring (nancial) costs themselves. In other words costs are
incurred by one stakeholder in order to create value for another. This is one of the reasons
why the impact of venture philanthropy and social investment is difcult to value. How
value creation (“VC”) is linked to benets and sacrices is illustrated in the chart below.
Source: Woodall, 2003,
“Conceptualizing Value for the
Customer”
Net VC
Benefits
Goods Quality
Attributes
Service Quality
Core Product
Features
Added Service
Features
Customisation
Strategic Benefits
Outcomes
Personal Benefits
Social Benefits
Practical Benefits
Financial Benefits
Sacrifices
Price
Monetary
Search Costs
Acquisition Costs
Relationship
Costs
Non Monetary
Psychological
Costs
Time
EffortOpportunity Costs
Distribution Costs
Learning Costs
Costs of Use
Maintenance Costs
Disposal Costs
(Marketing VC) (Derived VC)
(When reduced = Sale VC)
78 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
This step is also important in assessing whether the SPO has improved its product or
service delivery post VPO/SI intervention (this is why it would be important to make such
an assessment at the beginning and the end of the intervention). The choice of method
depends on the mind-set of the VPO/SI, the characteristics of the investors (whether more
or less focused on numerical or emotional value), and resources available.
Additionally, verifying and valuing impact helps identify the impacts with the highest
social value, which can help the SPO and VPO/SI focus their resources towards initiatives
that create most impact on society.
Verifying and valuing results should not only be done at the last phase of an investment: it
should be repeated as a “reality check” at several points during the investment and value
creation process of a VPO/SI. We recommend that this step be performed at the beginning
of an investment (as part of the due diligence), at least once during the investment period
(to check that the impact is achieved and valued) and again at the time of exit (as a way to
check that the desired impact has been achieved and makes sense).
One question that is often raised is who is responsible? At the level of the VPO/SI, it must
be the VPO/SI that takes responsibility for verifying and valuing the impact of their non-
nancial assistance on their investees. At the level of the SPO, a SPO may claim they are
too busy or do not have the time or the incentives to do it. VPO/SIs often do not want to
“burden the investee”. It is up to the VPO/SI to encourage the SPO to dedicate the time
and the resources to this step, given it adds more credibility to any information provided.
Unfortunately in practice, many investors in the social sector tend to “trust their gut
feeling” rather than invest in the verifying and valuing process. We hope that this step
“demysties” what is required to verify that expected outcomes are actually realised and
that the outcomes are valuable to the key stakeholders.
5.2 How to?
Verifying results
What we need to verify is what has been developed through the rest of the impact measure-
ment process.
Level of VPO
With respect to the VPO/SI, this is the added value provided to the SPO from the VPO/SI’s
non-nancial support. It is recommended that VPO/SIs use independent studies to assess
the value they provide to their SPOs as directly questioning investees may be a delicate
matter not always providing truthful answers.
Level of SPO
With respect to the SPO these are the outcomes that the SPO plans to or claims to be deliv-
ering i.e. how the key stakeholders are / were affected by the work of the SPO.
JUNE 2015 79
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
In other words, in both cases, we are triangulating the information we have received by
verifying it against other sources.
There are three principal approaches for this:
(i) Desk research
By looking at external research reports, databases, government statistics etc. it is
possible to conrm the trends that the organisation has detected through the outcome
indicators. This can be done by the VPO/SI and/or SPO or can be outsourced. For the
VPO/SI this desk research can and should occur at various points in the investment
process. Prior to investment this information provides data on the size and impor-
tance of the issue and establishes a base case. During the investment this data is useful
for triangulation purposes.
(ii) Competitive analysis
We can compare the data of the organisation with the data of other comparable organ-
isations operating in similar geographies on similar issues. We can ask the question
of whether the activity has been tried before and what were the results and learnings.
Competitive analysis helps with setting objectives and with judging the outcomes.
But the danger with this method is that organisations may only share “good” results
and not always the information about projects that failed or were less successful.
(iii) Interviews / Focus groups
Probably the best way to verify expectations and results is to ask the stakeholders: by
personal interviews or in the form of focus groups. In both cases you ask your stake-
holder about the results of the intervention. This is particularly the case when the
VPO/SI is assessing the value of its non-nancial assistance to the SPO.
One of the most crucial issues is to ask questions in a neutral way so as to prevent ”leading
questions”. For example, if a project manager asks a participant “Do you like my project?”
there is the risk that the participant will answer the way the project manager would prefer.
It is preferable to have a neutral interviewer (i.e. relative outsider) asking open questions
such as “What do / did you need?” “What has changed?”
Don’t worry about criticisms on the subjectivity of this method. At this stage we are loo-king
for people’s opinions to triangulate with data we already have. However what is important
is ensuring that the sample is representative.
We highlight a few references that we believe are useful for this method:
•http://www.roguecom.com/interview/overview.html
•http://techinlibraries.com/cowgill.pdf
80 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
Valuing results
There are numerous methods and techniques to measure the value created. They can be
divided in two categories:
•Qualitative
•Quantitative (monetised)
The objective of this manual is not to list the full plethora of tools available for measuring
value, instead we focus on a number of the most commonly used methods and describe
them briey. We have also provided specic websites and reports where you can nd out
more about a particular technique.
(i) Qualitative
•Storytelling
Almost all organisations use storytelling in one way or another. These stories can be
found in annual reports, project reports, and magazines, etc. In fact storytelling is a struc-
tured approach which describes the outcomes of an intervention / investment from the
point of view of a stakeholder. Through structured interviews, stakeholders are asked
about their experiences with the organisation. Every interview is executed with the same
framework of questions. Finally a picture (story) will emerge about the change that
the particular stakeholder experienced. A number of frameworks are available on the
internet to help create a structured interview and hence effective story.
Website: http://www.eldrbarry.net/roos/eest.htm
The reason why storytelling is popular is that numbers do not always tell a story, and it
is often easier to communicate the value of an outcome through a story. The downside of
storytelling is that it is generally unclear how many people are having or have had, that
particular experience i.e. the story may not be representative. We recommend the use of
storytelling as one component of valuing; not as the only way of valuing.
•Client satisfaction survey
This is an often-used method to measure the level of satisfaction among one’s (target)
stakeholders. On the internet you will nd a large numbers of alternatives for this type
of research, including online questionnaires, interviews, focus groups, etc. Often this will
be done by an outside organisation.
VPO/SIs may nd this technique especially useful for assessing the value of non-nan-
cial support provided to their investees.
One Foundation commissions independent feedback from their grantees through a
quantitative survey, carried out by Centre for Effective Philanthropy every second
year. The Grantee Perception Report® (GPR) shows an individual philanthropic
funder its grantee perceptions relative to a set of perceptions of other funders whose
grantees were surveyed by CEP.35
35. More information can be found at
www.effectivephilanthropy.org
JUNE 2015 81
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
Using client (or customer) satisfaction research can deliver important information on the
value of the product or service to the stakeholders. However do note that “satisfaction”
does not always imply the issue at stake is (very) important to the specic stakeholder.
You should therefore include questions focused on value, for example: how important is
the change for the beneciary?
Many organisations ask the question, “How many interviews or how much feedback is
required? 40%? 80%?” In general it really depends on availability of resources. In reality
the representativeness of the research population is much more important than just the
number of interviews. It is better to have 20% coverage of a good representative group,
than 50% of a non-representative group.
•Participatory impact assessment (focus groups)
This method is popular in developing countries with target groups that cannot read or
write as it makes it possible to rank preferences among stakeholders through the use of
pictures. Participants are shown a number of pictures (or in some cases they rst make
these pictures themselves) of products that are relevant and signicant to them. A new
item (the offer from the SPO or VPO/SI for example) is inserted. Participants receive
small stones as “money” and can rank their preferences by paying more or less stones to
the different products.
•Progress out of poverty index (Grameen Foundation)
We have already described this method in Step 3 where the focus was on selecting appro-
priate indicators. In Step 4 we would use the PPI as a measure of the effectiveness of the
SPO at moving people out of poverty and therefore the value of the SPO’s work to the
group in question.
(ii) Quantitative (monetisation)
Different stakeholders are likely to require different techniques. We identify two
principal techniques:
•Perceived value
•Cost-savings / cost reallocation
Please also note that social return on investment or cost / benet analysis are not tech-
niques in themselves, rather frameworks using one of the two techniques.
In the case study of Esmée Fairbairn Foundation investing in the social impact bond
managed by Social Finance, it is clear that the bond focused on cost-savings because
this was the governments focus and because the buy-in from the government was
crucial to the whole structure.
82 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
For individuals and/or target populations cost-savings are hardly ever relevant as it is
not generally the individual or broader population who is bearing the cost. For them,
perceived value methods should be use. On the other hand, governments, institutions
and organisations generally prefer cost-saving methodologies, given this is their focus.
One benet assigned to monetisation techniques is the ease of aggregating values
across the portfolio. But be aware that this can only be done if in each case you are
looking at either values or costs i.e. they are genuinely like-for-like quantities.
(a) Perceived value
•Perceived value / revealed preference
These techniques infer prices from related market-traded goods. The idea is that people
are “revealing” their preferences every time they make a trade. In scientic literature
these methods may be referred to as contingent valuation methods. Basically they
address two main questions to infer:
(i) Willingness to pay
(ii) Willingness to accept
Because these methods use “money” in the research, the answers of the respondents may
be biased: either they give strategic answers (lower value when they are afraid their will-
ingness to pay will lead to a higher real price); or, if they can’t afford the service anyway,
they may give unrealistically high answers.
Useful references in order to nd out more about this technique are:
•Mitchell, R. and R. Carson, (2005). “Using surveys to value public goods; the contingent
valuation method”. Washington USA.
•Champ, P., Koyle, K. and Brown, T., (2003). “A Primer on nonmarket valuation”. Dordrecht
(NL): Kluwer.
•The Value Game
A specic form of the revealed preferences method is the Value Game. The Value Game
combines participatory impact assessment (as described above) and the willingness to
pay-method, without the “money-component” of willingness to pay. Participants rank
pictures with relevant products, and a picture of the service/activity/impact with
unknown value, in order of their preference. The ranking gives information about the
rating of the service and can be compared to the (money) value of the surrounding products.
More information on Value Game can be found at www.valuegame.org
JUNE 2015 83
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
(b) Cost-savings
•Cost-saving methods / stated preferences
Stated preference methods use real nancial data to assess the value of the outcome by
using information about prevented costs, spending, and changes in nancial income. The
most commonly used methods are:
- Prevention costs method: for example when a new hospital treatment results in a shorter
stay in hospital for the patient, hospital costs will be prevented.
- Travel cost method: for example the costs people are incurring to get to a service. These
costs indicate the minimum price they are paying to receive the service.
- Hedonic pricing model: is measuring the value of a change, resulting from changes in
the environment. For example: a house has a value of 1 million euros. When an airport
is built right beside the house, the value may drop down (although the house is still
the same).
- Well-being valuation: a recently developed technique for valuing the effect, in monetary
terms, of a health problem on an individual’s well-being. The method involves calcu-
lating the compensating variation necessary to maintain the same level of well-being
after suffering from a particular health problem, and is hoped to offer a solution to the
problems of revealed preference and contingent valuation methods. Ref.: www.ncbi.
nlm.nih.gov/pubmed/17380470
- These methods give a good indication of the volume of value created, and are popular in
cost / benet analysis. Originating from the infrastructural and environmental sectors,
these methods are increasingly nding their way into social sectors.
•Quality Adjusted Life Years (“QALY”) is a form of cost / benet analysis
The basic idea of a QALY is straightforward. It takes one year of perfect health-life expec-
tancy to be worth 1, and regards one year of less than perfect life expectancy as less than
1. The QALY is based on the number of years of life that would be added by an interven-
tion. Each year in perfect health is assigned the value of 1.0 down to a value of 0.0 for
death. If the extra years would not be lived in full health, for example if the patient were
to lose a limb, or go blind or have to use a wheelchair, then the extra life-years are given
a value between 0 and 1 to account for this.
Although one treatment might help someone live longer, it might also have serious side
effects. For example, it might make them feel sick, put them at risk of other illnesses or
leave them permanently disabled. Another treatment might not help someone to live
as long, but it may improve their quality of life while they are alive (for example, by
reducing their pain or disability). The QALY method helps to measure these factors so
that we can compare the cost of different treatments for the same and different condi-
tions. A QALY gives an idea of how many extra months or years of life of a reasonable
quality a person might gain as a result of treatment.
84 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
QALYs have been criticised because there is an implication that some patients will be
refused or not offered treatment for the sake of other patients and, yet such choices have been
made and are being made all the time. However big the pot, choices still have to be made.
Further information on QALY can be found at:
http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/QALY.pdf
The above list is not and is not meant to be conclusive. We have highlighted some of the
often-used methods that we think are most prevalent and most useful. More information
on these techniques (and many others) can be found in the TRASI database: http://trasi.
foundationcenter.org/
5.3 Practical Tips
•Take this step (more) seriously as it may prevent poor investments, and can create a
learning and entrepreneurial environment.
•Be clear about what needs to be veried: different processes and timings will apply to
verify the results of the SPO to the beneciaries, and the VPO/SI’s role to the SPO (for
example: learning and growth of SPO).
•Desk research is a good starting point for verication but may not be sufcient.
•The decision for a VPO/SI to go out into the eld and be fully condent in its impact
verication must weigh the size of the investment with the cost of getting to the eld.
•Your choice of quantitative or qualitative techniques or a combination of both to value
the impact should be driven by the objectives of your impact measurement process and
by the prioritisation you assign to different stakeholders.
•The amount of time you will need to verify and value impact should be budgeted upfront
in your time plan for the year (VPO/SIs in the Expert Group suggested 5–7% of time) as
it is vital to talk to people to ensure that impact is being achieved.
Pieter Oostlander from Shaerpa shared that, “What we do in relevant cases is:
•Search for academic research that supports claims that are made on impact in the
analysis;
•Interview stakeholders and beneciaries to verify whether claims made by the project
or organisation are actually acknowledged by those groups;
•Search for statistical information and academic research to underpin the values of
indicators on impact and, if need be, organise focus groups with beneciaries and
stakeholders to assess the perceived value with them.”
JUNE 2015 85
THE IMPACT MEASUREMENT PROCESS
STEP 4: VERIFYING & VALUING IMPACT
5.4 Recommendations for Managing Impact
•For a VPO/SI, verifying and valuing results should not only be done at the last phase
of an investment: it should be repeated as a “reality check” at several points during the
investment and value creation process of a VPO/SI.
•Make clear determinations between the SPO and VPO/SI regarding who is responsible
for which parts of the verifying and valuing process.
•Unless you verify whether you have created value through your support of the SPO, you
cannot credibly make that statement.
•VPO/SIs should use independent studies to assess the value they provide to their SPOs
as directly questioning investees may be a delicate matter not always providing truthful
answers.
•VPO/SIs should verify at regular intervals that the expectations of other stakeholders
(donors/investors and human resources) are met so that corrective actions can be under-
taken if necessary.
5.5 Worked Example
The VPO/SI in our worked example wants to verify and value the technical assistance
it has provided to the SPO and the outcomes of the SPO. To achieve the rst the VPO/SI
ensures that it tracks all the pro-bono assistance it provides to the SPO in terms of type,
hours, and where possible assigning a $ value to how much that assistance would cost if
it were to be purchased on the market. On an annual basis it surveys all its investees to
understand the value the SPOs see in the technical assistance. The rst such survey was
developed with the help of an external consultant. However it now does the surveying and
the necessary tweaking itself to reduce costs.
At the same time the VPO/SI collects as much data for the verifying process as possible by
using desk research combined with competitive analysis. For example it tracks government
data on disease outbreaks in and around Nairobi as well as keeps an “ear to the ground” on
the activities and results of any similar companies working in a similar setting (although
not necessarily the same country).
The VPO/SI is not yet working with the SPO to value the outcomes of the SPO’s activities,
given they do not have sufcient resources to support either of the two techniques that
they would consider (perceived value or Progress out of Poverty Index). Given the early
stage nature of the SPO they feel the time is better spent on rening the business model
and consolidating its sales. Additionally the required outcome indicators already take up
a signicant amount of time and the VPO/SI does not want to overburden the SPO with
other requests at this early stage.
86 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 5: MONITORING & REPORTING
6.0 Step 5: Monitoring & Reporting
6.1 What?
The nal step in the impact measurement process involves monitoring – tracking progress
against (or deviation from) the objectives dened in the rst step and made concrete
through the indicators set in the third step; and reporting – transforming data into present-
able formats that are relevant for key stakeholders. Monitoring and reporting are iterative
processes that go hand in hand because what is monitoring to one stakeholder is reporting
to the other, e.g. when a VPO/SI is monitoring the progress of an investee SPO, that SPO
is reporting relevant data to the VPO/SI. When considering monitoring and reporting we
again consider the step at two levels: the VPO/SI and SPO.
(i) Monitoring
Once an organisation has decided on the indicators to measure and veried that they make
sense to the key stakeholders, they need to start collecting data in a systematic manner to
track performance against objectives. In practice, the type of monitoring system may be
considered upfront, however we urge organisations to go through the impact measure-
ment process at least theoretically prior to setting up the system so us to understand the
type of information that needs collecting and therefore avoid technological related issues
at a later stage.
A VPO/SI needs to systemise the data it tracks (from the SPO as well as independently)
across its portfolio to assess whether it is meeting its own impact objectives.
The SPO needs to collect and track data related to the indicators set in Step 3 and the
relevant information dened in Step 4 for verifying and valuing impact.
A VPO/SI should also gain an understanding of the ways its investees are already gathering
data and assess whether or not the relevant data is collected in a systematic way. VPO/SIs
need to monitor the information received from the SPO (timeliness, completeness, quality
of information provided etc.). In fact some VPO/SIs go a step further and look beyond
what a SPO can monitor today and try to conjure up a few criteria they would like to strive
to monitor in the future, if the SPO is able to develop according to their strategic plan.
A VPO/SI also needs to monitor data about its own activities as a high engagement
investor. This involves keeping track of all non-nancial and nancial support provided to
each investee and the total costs thereof. It also involves tracking how the support is used
by the SPO and where the gaps are.
Using the data collected to track progress against objectives means that the data needs
to be processed, performing the necessary analyses to gain a better and more complete
understanding of the impact achieved. The main objective of monitoring is to learn from
the data collected and analysed so that changes can be identied and corrective actions
JUNE 2015 87
THE IMPACT MEASUREMENT PROCESS
STEP 5: MONITORING & REPORTING
implemented. The organisation uses the data collected to analyse the results against the
initial objectives and decides which strategies and interventions worked and which did
not. It is important also to analyse the unintended consequences of the organisation’s activ-
ities and if they are signicant enough to warrant a change in strategy.
The VPO/SI needs to analyse its role in the change process asking questions such as: Is the
support offered to the SPO adequate and sufcient? What can be done differently and are
there resources available to implement such corrective actions?
(ii) Reporting
Once the data has been collected and analysed, an organisation needs to consider how to
present this information. The purpose of reporting affects the information that should be
included. Depending whether the focus is on an internal or external audience, the various
stakeholders may require different types of reports. The stakeholder analysis conducted in
Step 2 should guide the development of reporting, considering their multiple objectives.
One of the challenges of the social sector is that each SPO needs to report in different ways
to each funder. Some initiatives (for example the Social Reporting Standard) are trying to
overcome this problem, but there is still a problem of lack of standardisation that leads to
inefciencies.
6.2 How to?
A single system can be used that includes functionality for both monitoring and reporting,
so that the monitoring system is set up to produce reports, or different tools can be used
for each part.
Social Reporting Standard
An initiative to deal with multiple reporting requirements is the Social Reporting
Standard (SRS)36. Apart from providing guidelines on reporting, it provides the
following framework for impact-oriented reporting of SPOs:
1. Problem to be solved
2. Scale of the problem
3. Contribution of the organisation to a solution and expected impact
4. Actual social impact
•resources used (input)
•work performed (output)
•impact (incl. outcome)
5. Plan and outlook
6. Organisation
7. Finances
36. www.social-reporting-standard.de
88 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 5: MONITORING & REPORTING
(i) Monitoring
Level of VPO/SI
The VPO/SI should be collecting and analysing data on the specic indicators that measure
its progress towards reaching its overarching social objectives, and also monitoring the
time invested and/or € provided in non-nancial support to its investees.
Monitoring at the VPO/SI level is not yet well developed in many cases, whereas the moni-
toring of individual investments is a much more widespread practice. Depending on how
the VPO/SI measures impact at portfolio level, the data collection needed and the necessary
analysis will be different. It is important that the approach used is coherent. For example,
if the overall objectives of the VPO/SI are related to improving the long-term conditions
of a certain population, data should be collected at regular intervals to assess changes in
that particular population, and an evaluation should be made as to the contribution of the
VPO/SI to that change. Implementing the monitoring into the standard internal processes
of the VPO/SI and assigning a person responsible for this aspect should go some way to
addressing the problem.
VPO/SIs need to collect and analyse data from their investee SPOs, according to the objec-
tives and indicators previously dened. An important step for a VPO/SI is to gain an
understanding of the data already collected by the SPO and assess whether the data is of
sufciently high quality and enables the VPO/SI to evaluate whether the SPO is achieving
its impact objectives. Many times SPOs have systems in place to collect data on output
indicators, but not on outcomes.
The key recommendation for any VPO/SI is not to ask the SPO to collect data that will not
be useful to the management of the SPO itself. The danger is to start asking the SPO for long
lists of data that take time and effort to collect, when in the end only some of this data is
truly relevant. This is why it is so important for both VPO/SIs and SPOs to go through the
entire impact measurement process and spend some time on setting objectives and dening 37. http://pulse.app-x.com
Auridis, for example, has developed an investee database using Microsoft Access. It
collects information such as nancial data, grant history, essential documents such as
grant agreements, investees’ progress reports, and the milestones. At a portfolio level,
Auridis does not aggregate output, outcome, or impact data because the indicators
are not comparable across the portfolio. Some very basic aggregate indicators such as
“number of lives touched” can be aggregated across its investments.
PULSE37 is a numeric metric data collection and reporting tool and was created between
Acumen Fund and engineers from Google.com and developed with help from the Skoll
Foundation, the WK Kellogg Foundation, the Lodestar Foundation and Salesforce.com
Foundation. A VPO/SI that works with multiple organisations, and has metrics to track
and report back to your stakeholders, can use PULSE to facilitate the process.
JUNE 2015 89
THE IMPACT MEASUREMENT PROCESS
STEP 5: MONITORING & REPORTING
38. http://1068899683.n263075.test.
prositehosting.co.uk/wp-content/
uploads/2013/03/SEL-Balanced-
Scorecard-article.pdf
relevant indicators, before starting to collect data. If an organisation discovers at this stage
that it is impossible to gather data on a specic indicator, it makes sense to go back to step
3 and reconsider the indicators to align them with the real situation at hand.
There is a need to evaluate if the SPO is effectively monitoring its activities and outcomes
e.g. are the selected indicators appropriate (providing a balanced picture of the situation
and picking up potentially positive and negative aspects) and if the VPO/SI has a role
to play in improving the impact measurement practices of the investee. If the VPO/SI
asks the SPO to implement major changes in its information management system, it must
also be willing to contribute some of the resources (nancial and non-nancial support)
required. Using the VPO/SI’s network of pro-bono consultants can also be valuable to
provide key resources.
Level of the SPO
The SPO needs to evaluate the outcomes or impacts that are being achieved through the
activities of the organisation and the practical lessons that can be learned from the results.
With this information it is then possible to decide what actions are needed to increase
impact.
For example, in the social balanced scorecard developed in the UK for social enterprises38,
a performance measurement schedule is used to evaluate performance against objec-
tives, including the initiatives that lead to achieving the objective and who is responsible.
In the case of Papilio, the investee of Auridis highlighted in the case study on Step 5,
the investee itself developed an information system to collect relevant data. The Papilio
team previously used a mix of Excel sheets, Word lists, and paper lists spread all over
the team, which made it very difcult to manage the data and gain a quick overview
of how the organisation was progressing. Supported by Auridis and another major
funder, the Papilio team started to develop their own information management system.
The recommendations when implementing an information management system are as
follows:
•Usability is the key success factor of any system.
•The underlying processes are more important than technology, which is why the
system has to be well planned.
•Unless the SPO has relevant technological skills, it is advisable to hire a separate IT
consultant to implement the system.
•The whole SPO team and some of the other (external) users need to be integrated in
the development process, as they will be the main beneciaries of the system. It has
to be useful for them.
The development of an information management system needs an iterative process and
a lot of end consumer testing and reversing.
90 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 5: MONITORING & REPORTING
It provides a template to be used as a management tool for the social enterprise. An example
is included in the table below:
(ii) Reporting
For a VPO/SI, reporting can be external or internal, but generally it is related to reporting
to donors or investors. This reporting has different levels of detail depending on the stage
of the investment process. At a deal screening phase, the report to investors includes a low
level of detail, whereas much more information will be reported on after a due diligence
has been conducted. Once the investment has been made, the agreed-upon impact targets
should be communicated to investors. During the investment period, reporting should
allow investors to determine whether impact targets are being met, and at the end of an
investment, a detailed report should be completed with more long-term impacts included
and how the VPO/SI has helped the SPO achieve those. A VPO/SI should also consider
how to report the progress of its entire portfolio.
For Streetfootballworld, it is important to harmonise all monitoring and evaluation
activities with the theory of change and the strategic priorities of an organisation. There
is no point in collecting data that is not used either for reporting or for learning and
improvements. They have often encountered data overload on the one hand and a lack
of relevant data that helps organisations respond to internal or external questions about
their programmes on the other hand. If an organisation has a clear theory of change and
has identied its strategic objectives, this informs the data it needs to assess if the organ-
isation is still on track towards achieving them. In addition, it is essential to consider
community and investor requirements from the start to avoid adding additional data
collection, which often results in isolated forms and reports rather than synchronised
work streams. Involving stakeholders (communities/target groups, investors, board
etc.) in the development of the theory of change and the resulting monitoring and eval-
uation system is a very good practice to ensure backing and support for what you want
to measure anyway. Sometimes this can clash with specic reporting requirements to
investors or partners who want to know different things about the programmes, and
what communities/customers consider effective and necessary.
Outcome: Promote nancial sustainability
Objective Indicators Targets
Q1
Targets
Q2
Targets
Q3
Targets
Q4
Initiatives Owner
Increase trading
income
% of income
from new
products &
services
1% of
overall
budget
2% of
overall
budget
3% of
overall
budget
4% of
overall
budget
Launch of
online shop
Director of
marketing
JUNE 2015 91
THE IMPACT MEASUREMENT PROCESS
STEP 5: MONITORING & REPORTING
VPO/SIs and SPOs should agree before the investment the level of reporting required.
Considerations include:
•What to report on: which information should be included in the report?
•Format of reporting: which format can easily be used by the SPO based on the management
system they have developed, and is clear and transparent for the VPO/SI?
•Ownership: who is responsible for reporting within the SPO?
•Frequency and time horizon of reporting: how often (monthly, quarterly, annually, etc.)
should the reporting take place and what should be the time frame included for compar-
ison (one year, three years)?
If the VPO/SI co-invests with other funders, they should consider the possibility of devel-
oping common reporting frameworks so that the SPO is spared the burden of multiple
requirements. As long as the funders are able to extract the necessary information from the
report, they should not necessarily push their own format.
Some indicators may be reported more frequently than others. Typically, output indica-
tors can be captured more frequently than outcome indicators that might require more
time and effort to collect relevant data. The SRS recommends that the reporting period
should be the calendar year and relate to the prior nancial year. It is recommended that
the SPO completes the report by the end of the rst quarter of the nancial year following
the reporting period, in parallel with your annual nancial statements.
LGT VP’s social impact reporting tool was developed in-house using a combination of
Excel and Word. SPOs can add their latest outcome gures. In addition, LGT VP intro-
duced Pulse in their Salesforce tool to capture the key indicators. Investee organisations
can use their reporting tool if the information provided ts LGT VP’s requirements.
If this is not the case, investees are kindly asked to use LGT VP’s tools. The standard
frequency of reporting is every 3 months. Social impact might be reported less often, as
many of the indicators are not easy to capture. Effort/benets to capture reliable data
frequently must be balanced.
Ferd Social Entrepreneurs (“Ferd”) do not expect the social entrepreneurs to report
every month - only once or twice a year, and then maybe spending a couple days at
a maximum each time. For them it is crucial to keep the amount of time as low as
possible (for many SPOs the gathering of output data is more or less automated). For
Ferd, they spend time collecting data perhaps 3-4 weeks a year, as well as a lot of time
working with the system and dening the right measures, which probably takes much
more time. Standard reporting formats for both SPOs and VPO/SIs may help a little,
but Ferd believes they need to tailor the reporting for each of the SPOs to make it as
relevant as possible for their business.
92 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 5: MONITORING & REPORTING
39. “Sustainability Reporting
Guidelines”; Version 3.1; Global
Reporting Initiative.
For Streetfootballworld, agreeing on common standards for good reporting is denitely
the right way to support investees and investors, beneciaries and other stakeholders
in their collaboration and collective efforts. For them good reporting should always
include a reference to the organisation’s theory of change, its activities, and the related
progress made towards their desired outcomes, as well as key lessons learnt. While they
strongly believe in standardising reporting requirements, they also see some limitations
regarding exibility with the current reporting standards and formats. Rigid formats
and xed templates pose a challenge when reporting to different audiences, such as
the target beneciaries, the community, the board of the organisation or the different
funding partners. Ideally, standardised reporting formats remain exible to be modied
in its form and cover a high percentage of the basics, so that only a small amount of
additional information needs to be adapted for other stakeholders. At Streetfootball-
world they use SRS as a basis for reports to Ashoka and the Schwab foundation. They
have also observed an increasing openness by investors to accept existing (standard-
ised) reporting formats rather than asking for burdensome additional information and
strongly encourage existing and new investors to support such proposals.
The Global Reporting Initiative39 cites the following principles for dening report
quality:
•Balance: the report should reect positive and negative aspects of the organisation’s
performance to enable a reasoned assessment of overall performance.
•Comparability: Issues and information should be selected, compiled and reported
consistently. Reported information should be presented in a manner that enables
stakeholders to analyse changes in the organisation’s performance over time, and
could support analysis relative to other organisations.
•Accuracy: the reported information should be sufciently accurate and detailed for
stakeholders to assess the reporting organisation’s performance.
PhiTrust does not ask for social impact reporting on a monthly basis, but rather some
top-line criteria every six months and a more in-depth look at the impact performance
of its investees on a yearly basis. They nd that social impact reporting is a very time
intensive process for some of their investees (notably those who deal with an array of
stakeholders) and the information is not always easily collected. PhiTrust have seen
that many of their investees have or are moving towards creating internal reporting
‘standard’ formats in the form of monthly or quarterly reports on nancial and operating
metrics and at least annual social impact/performance. The internal format allows them
to both take on the process formally in-house with all of the internal buy-in necessary
as well as produce one document that they can share with all of their investors. For
investees choosing to develop these internal formats, PhiTrust have worked closely
with the entrepreneurs in the development of documents to ensure that both the pres-
entation format and content is as relevant as possible to their own needs.
JUNE 2015 93
THE IMPACT MEASUREMENT PROCESS
STEP 5: MONITORING & REPORTING
6.3 Practical Tips
•Early stage SPOs may not be ready to implement a complex monitoring and reporting
system – start with simple (e.g. Excel) and increase the level of sophistication as the
organisation matures and can free up resources.
•For the VPO/SI, it can be challenging to aggregate results across the portfolio with many
different systems and types of impacts. Some tools such as Pulse may be helpful.
•Try to share the development of systems with others, and do not invest on your own
in expensive systems e.g. a specic IT system. One of the main problems of specic IT
systems is that they are generally standalone and cannot communicate with common le
formats e.g. Excel.
•As a sector, we should move towards standardisation on reporting to remove inefcien-
cies. The Social Reporting Standard is a positive step in that direction.
6.4 Recommendation for Managing Impact
•To remove a reliance on and/or culture of “gut feeling”, the VPO/SI should work with
the SPO to develop an impact monitoring system which can be integrated into manage-
ment processes of the organisation, dening timings for each indicator (as not all impact
happens at the same time), tools to be used and responsibilities.
•Check whether the system the SPO already works with is sufcient to meet your require-
ments – if not, be prepared to contribute to improving it through pro-bono partners or
other resources. The objective should be a system that is of value to the SPO as a manage-
ment tool!
•The cost to support and maintain a SPO’s impact monitoring system (including personnel
time and costs) should be part of the SPO’s budget and hence part of the negotiation with
the investor in order to decide how costs should and/or could be split.
•When working with very early stage SPOs and helping them develop business plans,
integrate requirements on impact measurement at this stage.
•Agree on reporting requirements upfront with SPO and co-investors to eliminate multiple
reporting for SPOs.
•Manage expectations about frequency and level of detail for reporting, and the way
SPO’s report: will they just report on numbers, or also on verication (and if so, with
what frequency)?
•Timeliness: Reporting occurs on a regular scale and information is available in time for
stakeholders to make informed decisions.
•Clarity: Information should be made available in a manner that is understandable and
accessible to stakeholders using the report.
•Reliability: Information and processes used in the preparation of the report should be
gathered, recorded, analysed and disclosed in a way that could be subject to examina-
tion and that establishes the quality and materiality of the information.
94 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
THE IMPACT MEASUREMENT PROCESS
STEP 5: MONITORING & REPORTING
6.5 Worked Example
In our example, given the SPO is at a very early stage it is sensible to begin collecting data
through the use of spreadsheets. The VPO/SI can take the initiative with other investors
and create a template for reporting that the other investors are also happy to receive. This
can therefore reduce the burden on the SPO. Given the VPO/SI’s objective is also to provide
technical assistance to the SPO, one such assistance could be the use of pro-bono consult-
ants to help the SPO develop a more robust (vs. Excel) internal monitoring system to facil-
itate the monitoring of the output and outcome indicators alongside standard nancial
information.
The VPO/SI itself should have some form of internal system for collecting and aggregating
data (where feasible). Given the VPO/SI is also a young organisation, this system can begin
as an access database, but given the VPO/SI has plans in the short term to implement a
“Salesforce” style CRM system, moving the monitoring to a system based on Pulse may be
a good medium term option.
JUNE 2015 95
MANAGING IMPACT
7.0 Managing Impact
The goal of impact measurement is to manage and control the process of creating social
impact in order to maximise or optimise it (relative to costs). The impact measurement
process outlined in the ve steps should allow the VPO/SI to better manage the impact
generated through its investments. To manage impact, the VPO/SI should continuously
use the impact measurement process to identify and dene corrective actions if the overall
results deviate from expectations. This involves revising and readjusting the steps in the
impact measurement process as lessons are learned, additional data is collected, or the
feasibility of objectives set is questioned. It is important to see impact measurement as a
learning process.
Throughout the document, the impact measurement process has been related to the invest-
ment management process of the VPO/SI. Given that most VPO/SIs are aiming to maximise
impact, the corrective actions taken may apply as much to the investment management
process as to impact measurement itself. In the table that follows, the components of the
impact measurement process have been integrated into the overall investment process of
a VPO/SI. The objective of the table is to assist VPO/SIs that are trying to make impact
measurement integral to their investment process. Such an approach may facilitate the end
goal of maximising impact:
Several VPO/SIs that have worked many years on impact measurement, such as Noaber
Foundation and LGT Venture Philanthropy, have fully integrated impact analysis into their
investment process. Although the aim is for the impact measurement to become an integral
part of the investment process so that it is used by all VPO/SI team members on a daily
basis, it is useful to have someone responsible for the overall process. Drawing from the
recommendations on managing impact developed for each step in the impact measurement
process, the following elements should be taken into account when developing an invest-
ment strategy and for the investment process as a whole.
Assess whether invest-
ment opportunity fits
with VPO/SI strategy
by asking questions
detailed in setting
objectives.
Decide on the
overarching social
impact objectives
of the VPO/SI –
these will guide the
investment process.
Dig deeper into
questions asked in
setting objectives.
Perform stakeholder
analysis.
Verify and value
expected results.
Map outputs, out-
comes and impacts
and decide on key
indicators against
which progress will
be measured.
Decide on monito-
ring and reporting
content and frequency
and assign responsi-
bilities.
Regularly assess
impact results against
key indicators.
Verify and value
reported results at
regular intervals.
Revise indicators
if significant changes
are made in the
business and impact
model.
Investment
management Exit
Due diligence
(detailed screening)
Deal
screening
Investment
strategy Deal
structuring
Investment process
Perform thorough
analysis of impact
results against
objectives –
verifying
and valuing
reported
results.
Managing impact
in the investment
process
Source: EVPA
96 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
MANAGING IMPACT
Investment strategy
The rst question the VPO/SI needs to answer is what the overarching social problem or
issue is that the VPO/SI is trying to solve. A clearly articulated response is necessary to be
able to choose investments in SPOs that can contribute to solving the social issue that the
VPO/SI is addressing. For the impact measurement process, the VPO/SI needs to consider
this question clearly before starting to make investments, and regularly revise and adapt
as its investment strategy develops. Next, the VPO/SI needs to dene its overall impact
objectives as well as the relationship to be built with investees. For the overall impact objec-
tives, the VPO/SI should consider what changes it wishes to achieve as opposed to the
base case social issue previously identied. The next question will include how to achieve
those changes by investing in SPOs whose work is aligned with the objectives of the VPO/
SI. The role of the VPO/SI will be to provide the SPO with the support needed to help the
SPO achieve those objectives.
For the VPO/SI, it is important to get the buy-in of key stakeholders (donors/investors,
staff/human resources, SPOs) to the impact objectives of the VPO so that their expectations
are managed and their contributions are aligned. Therefore, engagement with a VPO/SI’s
key stakeholders should happen upfront by making sure they understand and support
impact objectives, and any major changes in these objectives should be properly communi-
cated. It is useful to regularly engage with these key stakeholders to make sure that objec-
tives continue being aligned, and otherwise implement corrective measures.
To better manage its overall impact, a VPO/SI needs to consider whether to dene overall
indicators to measure how well it has achieved its objectives as an organisation. Measure-
ment of impact at the portfolio level is a hot topic in impact measurement at the moment
and there is no common practice as of yet.
Deal screening
The impact objectives of the VPO/SI will guide the deal-screening step in the investment
process, narrowing down the type of SPO that will be considered for investment. For each
potential investment, it is important to evaluate the expected outcome of its investment in
the SPO, i.e. the expected outcome of the SPO and how the VPO/SI expects to contribute
to achieving that outcome.
Due diligence (detailed screening)
At the due diligence stage, it is vital to gain a detailed understanding of the current and
expected social impact of the SPO. It not only reduces the risk of making the wrong invest-
ment, but also creates a common understanding of the impact of an organisation among all
stakeholders and allows the VPO/SI and SPO to “speak the same language”.
Stakeholder analysis should be an integral part of the due diligence phase. To avoid wasting
resources, it is advisable for VPO/SIs to increase the intensity (i.e. more stakeholders, more
involvement from the same stakeholders and higher numbers involved from each group
JUNE 2015 97
MANAGING IMPACT
(up to the number required for a non-biased and random sample) of the analysis as it
becomes more likely that the investment will be realised.
If a SPO is claiming a certain outcome then they need to prove it. If the SPO cannot deliver
the data then the VPO/SI must consider whether they will bring in the expertise and
provide the necessary support so the system for data collection can be set up (although not
necessarily assisting the SPO in collecting the data per se) or question whether the SPO is
an appropriate investment at all.
It is useful as part of the due diligence phase to check whether the impact monitoring
system the SPO already works with is sufcient to meet the requirements of the VPO/SI.
Otherwise, the VPO/SI may need to contribute to improving it through pro-bono partners
or other resources – and those costs should be factored in before making an investment
decision.
Deal structuring
The resources of any SPO are limited and decisions have to be made about the amount of
time and resources that a SPO should dedicate to impact measurement. An important role
of the VPO/SI is to convince the SPO of the value of impact measurement, provide assis-
tance where possible and dene with them the responses to the essential questions to help
them express their objectives. Dening in the initial stages of the relationship with a SPO
exactly what they want to deliver makes it is much easier at a later stage to assess whether
this has been achieved.
It is important to clarify in the deal structuring phase who is responsible for measuring
what. The responsibilities of who measures what could and probably should evolve over
time as the SPO grows and develops and therefore should be reviewed on an annual basis.
The expected outputs, outcome and impact, and the corresponding indicators should be
dened before the investment is made and agreed upon by the VPO/SI and the SPO. The
VPO/SI should ask the SPO to focus on those indicators that are directly related to the SPO’s
theory of change and hence in line with their operational process. Any additional indicators
required for the VPO/SI to satisfy its impact measurement needs should be collected by the
VPO/SI. Also make clear determinations between the SPO and VPO/SI regarding who is
responsible for which parts of the verifying and valuing process – and when would be the
appropriate time to revisit stakeholder analysis during the investment period.
To remove a reliance on and/or culture of “gut feeling”, it is essential that the VPO/SI
works with the SPO to develop an impact monitoring system which can be integrated into
management processes of the organisation, dening timings for each indicator (as not all
impact happens at the same time), tools to be used and responsibilities. The cost to support
and maintain such a system (including personnel time and costs) should be part of the
SPO’s budget and hence may be part of the negotiation with the investor in order to decide
how costs should and/or could be split.
98 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
MANAGING IMPACT
The objective should be an impact measurement system that is of value to the SPO as a
management tool! When working with very early stage SPOs and helping them develop
business plans, it is useful to integrate requirements on impact measurement.
Reporting requirements should be agreed upfront between the VPO/SI and the SPO, if
possible involving co-investors in the decision-making process to eliminate a multiple
reporting burden for the SPO. Managing expectations about frequency and level of detail
for reporting, and the way SPO’s report will reduce the risk of problems later on in the
process.
Investment management
Monitoring of progress against objectives needs to be conducted regularly during the
investment process. Some indicators may be reported by the SPO more frequently than
others. Typically, output indicators can be captured more frequently than outcome indi-
cators that might require more time and effort to collect relevant data. VPO/SIs usually
require their investees to report against the predened indicators every quarter, every six
months or on an annual basis during the investment period. For a VPO, it is not enough to
just consider the impact achieved by the SPO, it is also important to assess the impact of the
work of the VPO/SI on the SPO. It is recommended that VPO/SIs use independent studies
to assess the value they provide to their SPOs, as directly questioning investees may be a
delicate matter not always providing truthful answers.
Stakeholder analysis may need to be repeated either at predened intervals during the
investment period or when signicant developments occur, such as a change to outcomes
being achieved, major new funding streams, new business lines being entered, changes
to policy environment etc. It is advisable to get back to the key stakeholders to verify that
their expectations are being met. Verifying and valuing results should be repeated as a
“reality check” at several points during the investment and value creation process of a
VPO/SI. We recommend that this step be performed at least once during the investment
period to check that the impact is achieved and valued.
The main objective of monitoring is to learn from the data collected and analysed so that
changes can be made and corrective actions implemented. The VPO/SI together with the
SPO should use the data collected to analyse the results against the initial objectives and
decide which strategies and interventions worked and which did not. The indicators set at
the deal structuring stage can be revised if signicant changes are made in the business and
impact model of the SPO during the investment process.
JUNE 2015 99
MANAGING IMPACT
Exit
At the time of exit, a VPO/SI should aim to measure the outcomes of the investment against
initial objectives. The outcomes should be veried using the various methods recom-
mended in Step 4. The resulting information will be useful for the VPO/SI itself to assess its
success as a “high-engagement” investor and take away learnings for future investments.
It will also be used to report back to donors and investors on the “social return” on their
investment. The impact of the SPO itself may also be a selling argument when “handing
over the baton” to future social investors.
For NESsT, managing impact takes place at the SPO and VPO/SI level. At the SPO level,
the objectives, indicators and regular measurement are used as a management tool by
the SPO and as signals for both the SPO and NESst for required intervention if things
don’t go accor-ding to plan. That intervention can be done by either the SPO or the
VPO/SI with their parti-cular “toolboxes”. At the VPO/SI level, they track portfolio
performance and if targets are not being reached they intervene to adjust the investment
strategy so as to better reach their goals. The most important question for NESsT is how
to decide when to intervene: how far off track to you need to go? What processes can be
put in place? Can this be supported in investment documentation? The NESsT approach
is to review performance data three times a year and that is when discussions about
intervention take place. They also relate their decision to scale or exit a SPO based on
performance and impact data.
100 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
MANAGING IMPACT
The chart below illustrates how LGT VP integrates social impact into the overall invest-
ment process and who is involved and what is produced. Prior to an investment,
during the due diligence process, the principal users of the social impact information
are internal to LGT VP both at a team and board level. However once the deal is in
execution the SPO itself also needs to be on board with regards to the denition of social
impact targets. Post investment, monitoring and reporting assesses how the SPO has
performed relative to the social impact targets. Here there are two principal audiences,
internal and external, and LGT VP produces different reports for each group.
Sign contracts &
disburse capital
Due diligence
Post-Investment
Monitoring
Who uses the
social impact
information?
LGT VP Team Board LGT VP Team / Board
/ Orgs
How is the
social impact
information
used?
Initial understanding
of impact
Decide if org will be
presented to the board
Deeper understanding
of impact
Board approval of
resources for local DD
Board approval of
engagement
Dene impact targets
Deliverables Impact model
light
Impact model Investment memo
(impact targets)
Who uses the
social impact
information?
Team + Board + Funders/Investors + Public
How is the
social impact
information
used?
Assess and report on achieved social impact
(qualitative and quantitative)
Deliverables Impact reports
(internal + public)
Deal executionDeal screening Preliminary review
ExitPortfolio controlling & reporting
Y1 Y2 Y3 Y4
JUNE 2015 101
CONCLUSIONS
8.0 Conclusions
The rst objective of this manual was to provide a guide on how to measure social impact
for VPO/SIs and other funders interested in generating a positive impact on society. For
that purpose, we researched the various existing approaches, interviewed several VPO/
SIs to nd out how impact measurement is currently done in the eld, and convened an
Expert Group that helped us develop practical case studies. Importantly, we tried not to
be partial to any existing approach, but rather attempted to provide practical recommen-
dations to social sector funders. The second objective was to trigger a movement towards
best practice on impact measurement. We envisage further work during coming years to
provide more high-level guidelines on impact measurement and reporting following this
hands-on practical guide. To support the implementation of our recommendations in the
manual, EVPA will launch a training centre.
A survey40 of 1000 charities by New Philanthropy Capital in the UK in 2012 cited a number
of barriers preventing SPOs from using impact measurement to its full potential. Among
these barriers was the point that SPOs do not know how to decide outcomes or where to
nd tools, an issue hopefully addressed in part by this practical guide. Another key barrier
was the fact that different investors ask SPOs for different types of information – over two
thirds asking their investees for information tailored to them. We believe that within this
manual we have the foundations of a shared measurement system for venture philanthro-
pists and social investors. The next step in EVPA’s impact measurement initiative is to
build on the content of the practical guide to create a code of good practice, which can then
be disseminated further across the sector.
At EVPA, we encourage our members to work hard to measure, monitor and report impact,
but also to increasingly integrate an impact approach into each important decision along
the investment process, from deal selection to exit. This is why managing impact is at the
core of the impact measurement process. For each step in the process, one should consider
how this relates to the everyday work of funding and building stronger social purpose
organisations. Our aim is for this practical guide to encourage more and better work on
impact measurement within EVPA’s membership and beyond.
40. Pritchard, D; Ní Ógáin, E;
Lumley, T., (2012), “Making an
impact: impact measurement among
charities and social enterprises in the
UK” New Philanthropy Capital.
102 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
PART 3:
Case Studies
JUNE 2015 103
CASE STUDIES
9.1 Step 1: Setting Objectives
Case study: Ferd Social Entrepreneurs investing in The Scientist
Factory (“TSF”)
This case study rst introduces the social issue at hand and how the social purpose organi-
sation TSF supported by the venture philanthropy organisation Ferd Social Entrepreneurs
is trying to solve the issue. It then discusses the impact measurement undertaken by the
social purpose organisation and nally moves into how the VPO/SI can go about setting
objectives for its own impact measurement as well as for the particular case.
Introduction – social issue
The challenge in question in this case study is that we live in an era where natural sciences
and technology develop at a very rapid rate. The gap between what children learn in
school and what is happening in the real world grows bigger every year. At the same time,
research within natural sciences and technology is central and vital for the development
of society. The issues we face in the years to come include climate change, food production
and distribution, and medicine and health. These issues cannot be solved without the use
of natural sciences and new technologies. TSF was founded in 2002 to meet the educational
challenges related to natural sciences and technology. TSF’s main goals were to increase
the number of students who choose an education in natural sciences, and to develop an
interest in research and technology among children and young people.
Ferd Social Entrepreneurs (FSE) was established in 2008 and TSF was the rst investment
we did. Ferd itself is part of the Norwegian industrial group of the same name. Ferd recog-
nizes its corporate social responsibility as an integral part of its business activities. We also
consider it natural to play a role beyond this, principally by supporting social entrepreneurs
who reect Ferd’s vision to create enduring value and leave clear footprints. Ferd therefore
supports selected organisations, projects and individuals who work to help ensure that
people – and particularly children and young people – can realise their opportunities and
ambitions. The due diligence process with TSF was initiated by the CEO of Ferd at the same
time as he hired the CEO of FSE.
The investment decision of FSE was based on a “gut feeling” regarding the impact that TSF
would have and a belief in the work of the entrepreneur Dr. Hanne Finstad. After inter-
viewing her and visiting some of the classes, FSE believed this project would create added
value for the participants (learning, motivation, fun) and hopefully in a longer term create
a change in the way natural science is taught in Norwegian schools.
How TSF assesses impact: starting point
Our starting point for measuring the impact of TSF before participating in the EVPA initia-
tive was to check what the organisation itself was already doing. In the autumn of 2011,
after TSF had been running for 10 years a web-based survey was launched to seek answers
104 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
to what we considered to be the most important question: does TSF matter? The survey was
answered by 75 out of 229 participants who followed a TSF course in the period 2002–2003
(a response rate of 32%, which is considered high for this type of survey).
The following results were found:
•all respondents have positive memories of the course
•93% are still interested in natural sciences
•18 respondents (25%) state that the TSF courses were an important factor when they
decided to choose natural science in their secondary education
•a positive correlation between the level of motivation and the interest in natural science
•a positive correlation between the number of courses pupils attended and the focus on
natural science in their secondary education
Motivation and interest in the natural sciences among children are the main goals of TSF,
and are therefore important factors that they have in mind when creating the courses. At
the same time, the courses centre on the most important concepts in the elds of chemistry,
physics and biology. The participants perform experiments and receive a few facts about
the different assignments before, during, and after each course session. Furthermore, TSF
always asks the pupils if they feel that they have learned something from the course. 99% of
the participants say that they learned a lot, or something, and many of them can talk about
the topics they have learned in their own words. Therefore, Ferd has reason to believe that
the courses provide a good learning platform where children gain knowledge, in addition
to giving them exciting and motivating experiences in natural sciences.
In the context of impact measurement the important question for us is whether the survey
provides information about the impact created by TSF.
To consider whether impact is measured with this survey we would have to ask certain
critical questions:
•Can we establish a base case? i.e. What percentage of children generally choose a natural
sciences-focused education later on (secondary/tertiary)?
•What percentage of participants of TSF classes chose a natural science focused education?
•How many of the participants in TSF were already interested in natural sciences?
•How many of them would have chosen to focus on the natural sciences anyway (a concept
that in some methodologies is called dead weight)?
Although the survey can provide us with some information about the % of participants of
TSF classes that choose a natural sciences focused education, it cannot provide answers to
the latter two questions. However some comparison of the TSF group to the general popu-
lation of Norway can probably be made.
One of the suggestions of the Expert Group, and particularly relevant for this step was the
importance of accurately describing the situation at the start of the period under analysis
JUNE 2015 105
CASE STUDIES
i.e. the situation 10 years ago and then thinking through the outcome objectives or social
changes that TSF would like to achieve.
One of the biggest problems for us is that the typical starting age of participants in TSF
classes is 10–12 years (5/6 graders). The reason for targeting children of this age was that
children nd science interesting and are curious about nature, so it is essential to maintain
their interest and curiosity and not lose it on the way to higher education. However when
the survey was conducted the original children were just about to nish their secondary
education (high school) or were at the beginning of their tertiary education (college/univer-
sity). Given so many changes occur during this time of their life it was difcult for them to
remember their views back in 2002/2003 and think about whether TSF was responsible for
any change in opinion or increase in motivation towards the natural sciences.
Input from the Expert Group – Setting Objectives
From our discussions in the Expert Group we decided to take another look at TSF and how
we could potentially understand and measure their social impact.
Firstly we realised that it was vital to understand why we (as Ferd Social Entrepreneurs)
wanted to measure the impact of TSF. Ferd is different from many VPO/SIs as it has just
one owner, so our focus does not need to be directed to a larger external owner group.
Nevertheless we believe it is important to measure impact for a number of reasons:
1. To demonstrate to Ferd’s board and to Johan Andresen that it is possible to create social
impact in a country that has a well-developed welfare state. And in addition, that small
amounts of money (as a proportion of the total welfare spend) can achieve quite some
impact.
2. To encourage the social entrepreneurs themselves to measure social impact so they can
improve their sales message and more effectively compete for government contracts or
sell their products / services.
3. To more effectively manage our portfolio. Our focus is on how we scale social impact
(versus scaling the economics per se) so we need to have a very clear understanding
of what elements of a social entrepreneur’s work generates the most impact so we can
ensure any scaling strategy focuses on these areas.
4. To select investments. Although we have not yet fully implemented social impact
analysis into our investment process we know this is important. We have scarce resources
(in terms of people and available funds) so we need to be sure we are spending our time
and funds with the social entrepreneurs that are generating the most social impact.
5. To motivate other investors to follow a VP approach. We feel that if we can demon-
strate our own social impact then it will be easier for other organisations to work with
social entrepreneurs and more generally in the eld of social innovation. At the moment,
although we are not entirely alone in the VP landscape in Norway we are denitely a
dominant player and it would be good to have some other organisations doing the same.
106 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
Quote from Johan Andresen41 to illustrate Ferd’s approach:
In the context of the TSF case study we were specically interested in nding evidence to
prove that TSF’s approach was an important factor in motivating younger people to choose
education within the eld of natural sciences. In the longer term we would like to know
whether TSF is creating a system change in the way in which natural sciences are taught at
primary schools in Norway. It was basically a retrospective evaluation of TSF, which would
allow us to better consider any future investments in TSF and also work with the entrepre-
neur on their scaling strategy.
Secondly, from the Expert Group discussions, it was decided to use the theory of change to
better understand the objectives of TSF and how they are working towards achieving them.
The rationale for this choice was that it is a simple framework that can guide you as you
think about the overall objectives of an investee especially when the impact of that investee
is potentially difcult to measure, intangible and only observable after signicant period of
time. We generally spend a lot of time (in person) with our entrepreneurs so we gain a good
understanding of what they are doing and why. We found that the theory of change was a
good technique / methodology for helping an entrepreneur “get down on the paper” the
key points of what they are trying to achieve. However, it is important to highlight that in
these initial stages we are just “building the foundation” for the rest of the impact measure-
ment process. The elements of the theory of change will need to be renewed and rened
throughout our journey with the entrepreneur, as things will and do change.
With regards to TSF we asked ourselves the following questions:
•What is the social issue TSF is trying to solve and why is it important?
•What is TSF doing to try and nd a solution to this issue?
•When can TSF be considered successful?
•What would happen without TSF?
“An advantage of focusing on a signicant social problem is that
when you nd a solution that works, it is worth investing in it to
allow as many people as possible to benet from it. There is a need,
but also an enormous challenge, to try to measure impact in order
to document that real social value is created, a value that someone,
private, business, or government should be willing to pay for”.
41. http://www.forbes.com/sites/
rahimkanani/2012/09/16/
johan-andresen-of-ferd-on-social-
entrepreneurship/
JUNE 2015 107
CASE STUDIES
Our responses were as follows:
•Social issue: There is a lack of young people who choose an education in natural sciences
(chemistry, physics, biology), which creates unfullled vacancies in the workforce. This is
caused by the way education is delivered: many teachers in primary school are not able
to motivate and stimulate children in natural science because of a lack of condence and
equipment. This has a negative affect on the self-image among children regarding this topic
therefore not enough children choose/focus on natural sciences in their secondary education.
As we made this statement, we realised that there were even more questions that we
needed answered upfront especially to try and understand why this topic is important.
On the one side we can make the statement that research within natural sciences and
technology is central and vital for the development of society. As stated above, we believe
that the issues society faces (climate, food production, distribution, medicine and health)
can’t be solved without the use of natural science and new technologies. However we were
encouraged to provide sources and support for this statement, which led to more questions.
These questions are as follows but we still don’t have answers to all of them. We accept that
the macro level questions can probably be answered from generally available statistics but
the mezzo and micro levels would require us to conduct surveys from a large enough sample
group and we do not have the resources (human or nancial) to do that at the moment.
Macro level:
•How many young people choose an education in the natural sciences (at the moment)?
•How many unlled vacancies are there in the workforce?
•i.e. how large is this gap?
•Mezzo level:
•How many (attending) teachers have a lack of condence and equipment?
•And how does a lack of condence show?
•And what kind of equipment?
Micro level:
•What is the level of interest of the participating children in natural science?
•How many children have a negative self-image regarding this topic?
•And what causes a negative self-image regarding this topic?
TSF Solution: The (after-school) courses provided by TSF are designed to create a long-
lasting inner motivation for natural sciences for children from 9 till 13 years old. There
are 4 to 6 courses within a school year and each course (2–3 hours) focuses on a different
topic. The courses aim to create positive experiences, through fun and creative teaching
situations (socially interactive, practice and theory and experiments to stimulate the ve
senses). Alongside the courses for children TSF also offer, “teach the teacher” programmes.
108 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
Success: We decided that there were two ways of considering whether TSF had been
successful. First the implementation of the teaching principles of TSF within the primary
education system of Norway, and second the increase in the number of students who
choose an education in natural science (secondary and beyond).
What would happen without TSF: Despite acknowledging the importance of this question
for us to be able to truly measure impact, we think that this is a question is almost impos-
sible to answer due to the age of participants. What children want to be when they grow
up when asked at the age of 10 is in most cases not what they end up being. One could try
to set up randomized control groups to provide more information and data but we believe
this would be a waste of resources. We therefore will be focusing on comparing the % of
participants in TSF classes who elect natural sciences at the level of higher education to the
average in Norway.
Conclusions
Some of the challenges of measuring the impact of TSF are due to the difculty of setting
the goals and then isolating the ndings from other sources of inuence. We can state our
main objective as, “The percentage of TSF participants choosing natural science in higher
education should be higher than the average in Norway”. But, again, it is difcult to tell if
TSF is the key differentiator or not. The participant children may have parents or siblings
with this interest, they may have an exceptional teacher, choose a role model with that type
of background, watch a video clip on YouTube, etc. and one of these events may be the true
trigger. However if the proportion of children from TSF classes choosing natural sciences
in higher education is signicantly greater than the average and in the surveys the children
state that the TSF courses were important to them – then it is safe to make the assumption
that TSF works. And that is the best answer we think we can get and for us it is good enough!
Recommendations
From our experience and the Expert Group discussions we have a number of recommenda-
tions for other VPO/SIs:
•Setting objectives is an absolutely vital step in any impact measurement process. Under-
standing why you want to measure impact is an obvious step. And if you can dene
exactly what the social entrepreneur wants to deliver then it is much easier at a later stage
to assess whether this has been achieved. It may be easier to follow a “gut feeling” but it
is much better to be more specic.
•If possible the discussion on objective setting should begin within the due diligence process
as this then sets the tone for your future collaboration with the entrepreneur. The theory of
change can help the entrepreneur better express their goals, aided by the VPO/SI.
•The investor should convince the social entrepreneur of the value of considering impact
measurement and using the theory of change methodology, even if it means they have a
few hours less sleep some nights!
JUNE 2015 109
CASE STUDIES
9.2 Step 2: Analysing Stakeholders
Case Study: Impetus Trust Investing in Blue Sky
This case study considers stakeholder analysis through the lens of Impetus Trust’s invest-
ment in Blue Sky Development and Regeneration (Blue Sky).
Introduction – social issue
There is substantial evidence that employment is the single most effective way to enable
an individual to escape the cycle of re-offending. Blue Sky offers ex-offenders “a proper
job with a proper company,” employing individuals with criminal records in entry-
level positions in the grounds maintenance and recycling sectors for up to six months.
The ex-offenders work in small teams and are supervised by an ex-offender team leader,
who serves as a mentor, and the Blue Sky team provides additional pastoral support (for
example, help to secure a bank account or to nd accommodation, etc.). Four months in,
employees are given a training budget that they may use to pursue a qualication of their
choice. As employees near the end of their six months with Blue Sky, they are offered help
in nding onward employment.
In 2008, Impetus invested in Blue Sky, a social enterprise that helps reduce re-offending by
employing ex-offenders and supporting them into sustained onward. Impetus Trust works
to break the cycle of poverty by investing in ambitious charities and social enterprises
that ght economic disadvantage using its highly effective venture philanthropy model.
Impetus Trust pioneered the venture philanthropy model of long-term nancial support
plus specialist business support delivered on a pro bono basis plus careful hands-on invest-
ment management in the UK. Since its launch in 2002, Impetus has invested in 24 charities
and social enterprises, helping them achieve average annual growth in income and people
helped of 23% and 30%, respectively.
Working Denition of Stakeholder Analysis
For the purposes of this case study, we have dened a stakeholder to be “any person or
organisation who is effecting and/or affected by the venture philanthropy investment.”
The primary stakeholders are the intended direct beneciaries of the project (in Blue Sky’s
case, these are the ex-offender employees), but there are also indirect beneciaries, other
parties that contribute to the change experienced by the ex-offenders, as well as stake-
holders who either indirectly contribute to or are affected by the project. From our perspec-
tive, as an investor, the investee is also a stakeholder, one with which it is important for the
investor to build and maintain a positive relationship. When we think about stakeholder
analysis, it involves both stakeholder identication as well as stakeholder engagement.
110 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
Impetus’s Approach to Stakeholder Analysis vis-à-vis the Blue Sky Investment
The Impetus team considered stakeholder analysis at three stages during its investment in
Blue Sky: as part of our pre-investment due diligence, as part of our ongoing investment
management, and during a one-off impact evaluation project.
Pre-investment due diligence: We do not necessarily need, at this stage, a perfect under-
standing of all the stakeholders – just a working picture of how the organisation creates
impact and an opportunity to speak to some of the most critical stakeholders to conrm
that our assumptions have validity.
Stakeholder analysis is important during this phase because we cannot develop a picture
of the impact that is created by the organisation without identifying and then speaking to
the individuals who are affected by Blue Sky’s intervention and without understanding
who else, besides Blue Sky, plays a role in creating the change that is experienced by stake-
holders.
A key consideration at all stages but which is of particular importance during the pre-
investment stage is stakeholder selection: in order to form an accurate view of the impact
created by the organisation, we need to ensure that the stakeholders we interview have not
been “cherry picked.” The need for a balanced view from stakeholders is complicated by
the fact that we often have to rely on the potential investee to identify and provide access to
its service users and other parties who are closely familiar with its work, and the potential
investee obviously has an interest in presenting as positive a view of its work as possible.
In addition, an organisation is often, almost by denition, less likely to be in a position
to maintain contact with individuals who drop out of their programme. To mitigate the
potential for selection bias, we have a number of strategies:
•When developing the list of service users / other partners to be interviewed, we explic-
itly ask the organisation to include some parties where the outcomes were not ideal.
•Using our own contacts, we reach out to parties who were not necessarily identied by
the potential investee, but who are familiar with its work.
•In all interviews, we ask stakeholders to discuss both successes and failures that they and
others have experienced, and we ask them to identify any other parties with whom they
think we should speak in order to build a balanced view of the potential investee’s work.
Also of critical importance to Impetus at the pre-investment stage is effective resource allo-
cation: we don’t want to commit too much time to organisations in which we don’t ulti-
mately make an investment. We therefore seek to employ a method of stakeholder analysis
that involves increasing intensity only as we increase our condence that we will pursue
the investment. We typically invest c. two to three days of staff time in total in stakeholder
analysis during due diligence, and this was the case with Blue Sky. This is largely managed
in-house, although we do on occasion commission external consultants on a pro bono basis,
always led by an Impetus investment executive, to assist with due diligence.
JUNE 2015 111
CASE STUDIES
We begin with conversations with the senior management team probing the theory of
change and developing a picture of who are the key stakeholders who would need to
be interviewed prior to making an investment decision. Our discussions led us to the
following conclusions:
•Key social change the organisation is trying to achieve: reducing re-offending through
the employment of ex-offenders.
•Primary beneciaries of this change: Ex-offender employees (who benet from employ-
ment and from support in turning their lives around) and government/local communi-
ties (which save money through fewer crimes and related costs).
•Other parties that might contribute to this change: We understood that families, probation
ofcers, and other support workers might be involved in helping the employees turn
their lives around, and we wanted to investigate further how much of the observed
change could be attributed to these groups.
•Other parties who might be impacted by Blue Sky’s work (either positively or nega-
tively): we were interested in understanding what happens to employees who drop out
of the programme and also whether the programme creates any job displacement.
The next step involved desk research and interviews with key stakeholders identied in
step one. We focused our work at this stage on three groups:
•Employees: We interviewed some current and former Blue Sky employees to understand
better what changes for them and how Blue Sky contributes to this. We also wanted
to know if there were any negative changes experienced and the extent to which the
employees felt that other parties were responsible for helping them turn their lives
around. Finally, we wanted to understand more about the employees, in particular, how
similar they are to an “average” person coming out of prison, so that we could develop
an understanding of how much change we might expect to have occurred even without
Blue Sky. Through our due diligence, including interviews with employees, reference
checks with other agencies familiar with Blue Sky’s work, and analysis of the Blue Sky
employee database (and comparison with publicly available datasets), we got comfort-
able that Blue Sky’s employees were not particularly “easy to reach” relative to other
people coming out of prison. We also satised ourselves that while there might be other
groups of stakeholders who played some part in an individual’s journey towards turning
his/her life around (people like probation ofcers, family, etc.), that the Blue Sky inter-
vention was very decisive and without it, the change would not have happened. This
gave us condence that we didn’t need to discount signicantly the outcomes achieved
by the Blue Sky employees signicantly for deadweight or for the contribution from
other stakeholders. As such, we did not conduct signicant further interviews with these
other stakeholders at this time.
•Government / local communities: We did undertake some reference checking with local
authorities that were familiar with Blue Sky, but they were primarily able to comment on
the quality of the work performed by the employees rather than the social impact created.
112 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
As such, we amplied the ndings of these conversations with desk research looking at
the potential for cost savings when positive outcomes are achieved.
•Potentially displaced employees: Our interviews with SMT around the prole of employees
they hire, plus further desk research on the labour market, gave us condence that
permanent job displacement was de minimus, so we also did not consider this further at
this time.
On-going Investment Management: Having committed to invest in Blue Sky, our primary
objective during the investment management phase is ensuring that the organisation
continues to meet the social impact objectives agreed at the time of our investment. Our
investment management model involves
•Monthly meetings with the CEO to review progress against investment milestones and to
get an update on developments within the organisation.
•Biannual reviews with CEO and Chair to reect on “bigger picture” issues.
•Periodic site visits, including informal interviews with employees, and preparation
of further case studies each year to help keep our understanding of the work “on the
ground” fresh.
•Quarterly reporting by Impetus to our Board on progress against investment milestones.
•Biannual published reports to Impetus’s external stakeholders on the impact achieved
within our portfolio; more regular reporting to particular stakeholders, such as investors
in our various initiatives or co-investors in particular organisations, on a case-by-case
basis.
As the core business model doesn’t change that much, we do not have to revisit our stake-
holder analysis often. Signicant developments that might lead us to return to stakeholder
analysis include:
•A change to outcomes being achieved: we would want to understand what is driving this
new outcome and whether there are any stakeholder groups that need to be considered
•Major new funding streams, particularly if the funder has specic objectives: we would
want to consider the impact that these have on Blue Sky’s work.
•New business lines being entered: we would want to look at whether these involve
different market dynamics and therefore increased displacement or other positive or
negative impacts that we would need to consider.
•New recruitment practices: we would want to understand whether this changes the
make-up of the employee group, either increasing or decreasing what we should assume
about deadweight and attribution.
•Changes to the policy environment: we keep an eye on the overall environment around
ex-offenders and employment of hard to reach groups to understand the impact that
other actors might have on the Blue Sky employees and our assumptions around dead-
weight and attribution.
JUNE 2015 113
CASE STUDIES
Given our relatively light-touch and by exception approach to stakeholder analysis during
the on-going investment management phase, stakeholder engagement usually takes up no
more than a day or two of staff time per year and is managed entirely in-house.
One-Off Impact Analysis: Mid-investment, we decided to undertake an in-depth review
of the impact created by Blue Sky. Our objective was to deepen our understanding of and
to quantify the social value created by the Blue Sky intervention using a methodology
that would be externally recognised as rigorous. We chose to conduct a social return on
investment (SROI) analysis that was capable of achieving assurance by the Social Value UK
(formerly SROI Network), which meant closely following the principles laid out in their
guide to SROI.
The stakeholder analysis and engagement undertaken during this phase was similar to,
but more in-depth than the process adopted during due diligence, as we needed to justify
and document all the assumptions we had made regarding stakeholders. As an initial step,
we identied a very long list of potential stakeholders (including groups considered earlier
and many more). We then screened this list for relevance (how relevant is the stakeholder
group to Blue Sky’s primary mission of reducing re-offending through employment) and
signicance (how signicant are the benets and inputs provided by these stakeholders).
Once we had identied the groups of stakeholders to be interviewed, we then created lists
of individual stakeholders within these groups to consult, taking care to reect the range
of stake-holder experiences by constructing a sample that was of an appropriate size and
diversity. For example, when selecting which employees to interview, we went for a mix of
male and female, current and former, older and younger employees, and we also explicitly
sought out some employees who had left the programme early. Next, we conducted detailed
stake-holder interviews focused on understanding how they experience and contribute to
the social change delivered by Blue Sky. Finally, we “played back” our conclusions to the
stake-holders to ensure that we had accurately reected their views and to see whether
there were any additional points that we needed to consider. We engaged a team of about
four people from Blue Sky and Impetus to carry out the stakeholder engagement, and in
total, we invested about eight to ten days of staff time to select the stakeholders, develop
the interview questions, set up and carry out the interviews, and then to follow up with the
stakeholders once the SROI analysis was complete. Although there were some arguments
in favour of engaging a third party to conduct the stakeholder engagement during this
phase, we ultimately opted to conduct this work in-house in order to proceed quickly and
to conserve resources (we had a very limited budget for this project and, going into the
project, were unclear as to how much time it would consume).
The Social Value UK’s principles emphasise the importance of evidence collected from
stake-holders. This led to an interesting debate within the team working on the analysis as
to how to treat the families of ex-offenders. We knew that in the relatively small number
of instances where employees’ families had proactively contacted Blue Sky to share their
stories that the intervention had made a very signicant difference in their lives, one which
114 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
could have had a material impact on our calculation of the overall value created and which
therefore should be captured as part of the analysis. We could point to a number of third
party studies that enumerated in detail the positive impact on families of the ex-offender
turning his life around and of the negative impact that an offender’s return to prison has on
families. However, the Social Value UK’s principles would have required us to interview
Blue Sky families directly to substantiate these claims, and the Blue Sky team felt strongly
that this would be inappropriate. Their position was that an important part of their inter-
vention is that they treat their employees as would any other employer, and we did not
think that asking to interview a random selection of employees’ families was something
that would happen at other companies. This left us in the unsatisfactory position of
knowing that there was material value that we were not able to capture and include in the
analysis. Because we wanted to have our study externally assured, we excluded the value
of changes that we believe are experienced by this group of stakeholders.
Another challenge we encountered was around extracting from the stakeholder engage-
ment the evidence we needed for our SROI analysis in a manner that preserved the integrity
of the process and of our relationships with the stakeholders. For instance, we needed the
employees to share with us as much detail about what had changed, both positively and
negatively, in their lives as a result of working with Blue Sky. However, even though we
made it clear that we wanted employees to be as open an honest with us as possible, we
could not get around the risk that some current employees might be nervous about saying
something that wasn’t positive about their employer. We tried to focus on open-ended
questions, but some employees were understandably reluctant to open up with us about
a time in their life that has been challenging, meaning that some of their answers to our
open-ended questions were quite brief and not very illuminating; we, therefore, had to nd
a way to ask for additional information gently and without leading the employees towards
a particular answer. The part of the process where we played back our ndings to stake-
holders was particularly challenging: while it was straightforward to conrm that we had
accurately captured the employees’ stories of change and had concentrated our analysis
on the outcomes that were most meaningful to them, we found it difcult to talk about
how we translated those stories of change into nancial proxies in a way that was mean-
ingful to them. Consistently, employees told us that the most meaningful change they had
experienced was that they had managed to secure and hold onto their freedom, and they
also consistently told us that it was impossible to put a value on this – that it was literally
priceless. While we could reect those conversations in the paper, we ultimately had to try
to assign some type of nancial value to each of the signicant outcomes detailed in the
paper, and nding a way to talk about this constructively was challenging. There are some
resources that provide suggestions on how to engage with stakeholders (see Sources in the
Appendix), which we found helpful. Ultimately, though, each situation is unique and will
require an exercise of judgment to balance the requirements of the analysis with the need
to preserve the dignity of the stakeholders and the integrity of your relationship with them.
JUNE 2015 115
CASE STUDIES
Lessons Learned / Tips for Other Venture Philanthropy Organisations
In the end, the results that we achieved from the more rigorous, structured examination
of stakeholders that we performed as part of our SROI analysis did not lead us to materi-
ally different conclusions as those achieved during our due diligence: the more rigorous
examination did not uncover stakeholders who were either signicant beneciaries of or
contributors to the social change achieved by Blue Sky that had not been considered earlier.
This conrmed our view that stakeholder analysis is an area where experienced investors
may safely rely on intuitive processes that work for them. For newer investors or ones
who are more comfortable operating within a dened framework, there are a number of
options available for both stakeholder identication and engagement (see Sources in the
Appendix). VP investors will also need to think about how much time and resources they
have to devote to the project.
In all three stages of Impetus’s engagement with Blue Sky, we found direct engagement
with primary stakeholders to be quite useful and would encourage any VP organisation
to make this a regular part of their investment procedures. However, our experience with
the SROI analysis we conducted also led us to conclude that direct engagement with stake-
holders may not be the only way to understand value creation; we believe there is a place
for considering available third party research with similar stakeholders, particularly if
there are resource constraints that would prevent an organisation from conducting their
own stakeholder engagement or if there are ethical issues involved, as was the case with
Blue Sky.
Our top tips in approaching stakeholder analysis are to:
•Make it meaningful: Link your stakeholder analysis to your investment objectives at each
particular stage.
•Put it in proportion: Stakeholder analysis is an area that could, in theory, consume as
much resource as an organisation is willing to invest in it. Be thoughtful about how much
is required for your current stage of investment.
•Keep it current: Stay in touch with your stakeholders regularly and be clear about what
would trigger a need for a major refresh of your initial work.
•Sample soundly: Try to construct stakeholder samples that are of an appropriate size and
reect the diversity of your service users and partners.
116 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
9.3 Step 3: Measuring Results – Outcome, Impact and Indicators
Case Study: Oltre Venture investing in PerMicro
Introduction – social issue
Oltre Venture started its activity in 2006 and has been investing in social enterprises since
then; bringing capital, managerial skills and knowledge to the social sector. Its purpose is
to assist companies that have a social impact to create value through the creation of sustain-
able businesses, offering a positive nancial return to those who have invested in the fund.
In this way, value created is not only human and social but also economic and nancial,
proving that traditional nancial tools can also be used in a new and innovative sector that
is positioned between the for-prot and non-prot sector, by creating for-prot companies
that seek primarily a social impact. The challenge is to attract private capital in a sector
that has historically beneted from public funds, through a balance between nancial and
social return.
On one hand we aim to provide investors with an IRR equal to ination plus 2%, on
the other hand we aim to create a durable positive impact on communities involved in
the project. The current fund amounts to €8m and was raised with the contributions of
several private investors, some corporations and one bank foundation (Fondazione CRT).
Currently Oltre focuses its investments on microcredit institutions (20%), social housing
(25%), health services (39%), and job creation enterprises (16%).
Our approach to venture philanthropy and social investment is tailor made for the market
in which we are investing: Italy. We truly partner with the organisations we invest in
and this is made very clear from the outset. For us understanding impact begins right
at the start. Any organisation that we consider must have nancial and social outcomes
embedded in their mission, for example offering services at a price at least 50% lower
than the market rate to customers who would not usually have access (e.g. low cost dental
care to poor families) or working within a sector that by denition is social (e.g. micro-
nance). We generally invest in very early stage or start-up organisations so our key focus is
ensuring their nancial sustainability. Unless the organisation is successful there can be no
impact and if the organisation cannot reach nancial sustainability during our investment
period (7–10 years) then it is unlikely to survive after we exit meaning any potential social
impact is then lost. In addition the investors in our fund expect at least the return of their
capital and this can only be achieved if we help build nancially sustainable companies.
PerMicro Case
An example of one of our investees is PerMicro. PerMicro is a microcredit institution
founded in 2007. Its mission is to give the opportunity of social and nancial inclusion to
“non-bankable” populations through microcredit, providing loans directly to businesses
and individuals. Operating initially in the multi-ethnic neighbourhoods of Torino, PerMicro
has grown to national level by opening 12 branches throughout Italy. PerMicro’s activity
JUNE 2015 117
CASE STUDIES
is based on the concept of network credit: the social network of reference is the interme-
diary between PerMicro and the clients, providing a moral guarantee and supporting them
before and after the loan disbursement. PerMicro is the rst Italian microcredit provider.
Its business model has been recognised and rewarded also at European level and won
the “Fondazione Giordano dell’Amore” award. Since its inception, PerMicro has screened
about 10,700 candi-dates and distributed more than 2,000 microloans, for a total nancing
amount of €11,4m. The average duration of a loan is 36 months, the average size of a loan
is €4,000 for family loans and €7,300 for business loans. Oltre Venture currently owns 12%
of PerMicro’s equity, which has recently beneted from the entrance of BNL (BNP Paribas
group) among its shareholders as its industrial partner.
Current approach to measuring results
PerMicro’s objective is to create improvement in overall life conditions for its customers by
distributing microloans and thus positively affecting the micro business/family nancial
condition to achieve its mission. PerMicro wants to understand the outputs, outcomes, and
impact of its activity and has developed an in-house approach to impact measurement that
addresses its specic queries.
PerMicro has developed different types of reports and performance screening tools, which
address different objectives and are intended for different recipients.
•On-going Performance Tracking & Management: PerMicro produces monthly,
quarterly, and annual reports that summarize its activities, which are shared during
monthly commit-tee meetings. The indicators in the report include measures of outreach,
client satisfaction, and nancial performance. These are produced for internal use, as a
tool for management to monitor the ongoing progress towards (i) fullling the mission
and reaching the target population, and (ii) reaching the economic / nancial objectives
stated in the business plan (break-even point).
Type of Report Information Covered Purpose
Monthly Reports •Client information: nationality, gender, civil status,
business activity of clients and purpose of the loan.
•Loan information: disbursed and outstanding port-
folio, the number of contracts, the number of opened
les, the number of closed les.
•Monitor data on new clients and existing clients’
attrition rate
•Provide detailed information on the monthly activity
of PerMicro
Risk Reports •Bad debt
•Repayment
•Other performance measures
•Evaluate cost of risk
•Evaluate quality of portfolio
•Set benchmarks among branches, evaluate other risks
•This is a tool under development. It will be a monthly
report and will provide information across the
following areas:
•Administration
•Production and development
•Risk and recovery
•Provide a comprehensive view on the social and
economic performance of PerMicro
118 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
The form, content and frequency of the reporting were agreed between Oltre and PerMicro
at the beginning of our investment and focus on the operations of PerMicro, for the reasons
we detailed before (i.e. in line with our objectives for impact measurement).
•External Reporting: PerMicro produces a series of different reports for different stake-
holders. Equity investors are the stakeholders that are mostly interested in the assess-
ment of the projects, and they need to have information in relation to their expectations
(achievement of the break even point and value created through their investment). Apart
from clients and investors of PerMicro, other interested stakeholders are mainly local
municipalities and in general public institution working in the nearby environment,
which may benet from a constant update on the evolution of PerMicro’s activities, and
other local associations or non-prot organisations.
Indicators
PerMicro has identied a set of objectives and related performance indicators that are
summarised in the table below. To monitor progress towards nancial goals, PerMicro has
chosen the standard tools used for this purpose, which are nancial statements and nancial
modelling, with constant monitoring and review of the business plan made available by
monthly budget reports. On the Social side, PerMicro constantly monitors the demographics
of clients it reaches, comparing this with its goals and its mission, as well as monitoring the
type of engagement of their clients and any possible difculties they are facing.
Although we are aware of and follow the development of standardised indicators (from the
likes of IRIS and Global Value Exchange) and can understand what these organisations are
trying to achieve, we have chosen not to use them. This is rstly because it is important for
us to assign indicators in Italian, and this is not possible with the current IRIS taxonomy.
Secondly we believe it is important to work with the SPO in the development of the indica-
tors given the peculiarities of each of the organisations we work with.
Target Audience Information
Potential clients of PerMicro •Social reports •Communication instruments
Investors (e.g. Oltre Venture) •Qualitative reports about outreach
(monthly) and client satisfaction.
•Reports that monitor portfolio risk
•Balance sheet and income statement
•Business plan
•Social reports
•Market research
Other stakeholders (networks,
government)
•Social reports
•Reports on risk prole of clients
•Market research
•Reports on clients
JUNE 2015 119
CASE STUDIES
Impact Measurement
Following the social and economic contextualization of the microcredit institution activity,
PerMicro goes a step further in the evaluation of impact, focusing on the analysis of changes
made in the quality of life of its clients (or their families and local communities) and deter-
mining whether there have been any positive, negative or neutral effects.
The denition of impact used by PerMicro stems from two main elements:
•Changes that take place in an individual’s life, in its family, its business or its community.
•The extent to which these changes are related to the individual’s loan undertaking.
To identify and measure impact, one must prove in a credible manner that changes observ-
able in clients, with reference to the different analysis levels, are directly related to the
clients’ relationship with the institution.
In the last few years, PerMicro participated in two scientic working groups and identied
some potential methodologies to evaluate the impact of its activity. These methodologies,
however, presented some hurdles in terms of cost of implementation and of the so-called
attribution problem, which is more marked in the western world, where the existence of a
more structured public welfare system makes it hard to isolate the effect of micro lending
from other types of intervention.
The nal decision made by PerMicro was to perform a retrospective impact evaluation
focusing on a proxy of Impact: the change in nancial inclusion. Below is a summary of
the evaluation method showing how it will be implemented in time. As per PerMicro’s
in-house developed denition, impact occurs and it is positive if a client becomes bankable
after taking a microloan.
The end of the evaluation period was set to be end of 2014, at which time PerMicro was also
expected to reach its nancial break-even point.
As an investor we are fortunate that PerMicro themselves were willing to commit the
required resources to these more in depth studies about their impact and it does provide us
with further information to communicate to our own stakeholders. However if PerMicro
Financial side Social side
Objectives Financial objectives: break even point Social objectives: lending to non-
bankable people
Indicators Financial data: balance sheets, income
statement, nancial modelling
Client Demographics: gender,
nationality, education, age
Client Engagement: account types,
pending loans, non performing
loans
120 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
were not keen to perform these impact studies we would not require them to do so as we
believe that output measures can sufciently demonstrate that a business is on the right
track (or not) to nancial sustainability and therefore achieving social impact.
Conclusions
The rationale of building nancially sustainable companies informs Oltre’s approach to
impact measurement. For us nancial sustainability is the key to achieving social impact
so we predominantly use impact measurement as a management tool, focusing on output
indicators to understand how the business is progressing vis-à-vis its business plan. This is
reinforced by the difculties that exist in measuring impact in a developed country such as
Italy. The strong welfare state and other safety nets means it is very difcult (and expensive)
to isolate the longer-term effects of any organisation we are supporting to provide an
accurate measure of impact. For example in the case of one of our micro-nance invest-
ments, we can accurately measure the number of loans disbursed and number of new busi-
nesses created, but to go a step further and consider how that relates to the physical well-
being of the family who now has a business would be very difcult. A long-term study
using randomized control groups would probably be required and then we also have the
moral issue of excluding groups of people who could have benetted from a loan but for
the purpose of the study were selected not to so as to have an appropriate control group.
We may eventually consider a more comprehensive study of the “impact” of our fund, but
that is most likely to occur once we close the fund and are distributing the proceeds to its
shareholders. This is because any sufciently rigorous impact study is likely to have to
be in place for at least half the time of our total investment period and we think it is more
important to focus our efforts on supporting the entrepreneurs in growing their business.
We recommend other social investors to develop ways of measuring results that are clearly
in line with their objectives.
Timeline Phase Actions
June-September
2012
Sample of non-bankable 1. Settle on criteria to dene a sample of non-bankable people:
•Absence (or presence for less than 6 months) of a bank account
•Loans with credit institutions
•Loans with banks
2. Selection of a sample composed of non-bankable people within PerMicro
portfolio who received a loan in 2010.
September-
December 2012
Interview 3. Phone interview with the client or the bank to understand whether after the
disbursement of a microloan the client become bankable i.e. did they start a
stable relationship with a bank, open a bank account asked and/or obtain
another loan.
January 2013 Elaboration of data 4. Analysis of data. Impact evaluation.
JUNE 2015 121
CASE STUDIES
9.4 Step 4: Verifying & Valuing Impact
Case Study: Esmée Fairbairn Foundation investing in Social
Impact Partnership (developed and run by Social Finance)
Introduction
Esmée Fairbairn Foundation aims to improve the quality of life for people and communities
in the UK both now and in the future. We make grants in the region of £30–35 million every
year in the arts, education and learning, the environment and social change. In addition,
we operate a £21 million Finance Fund, which invests in organisations that aim to deliver
both a nancial return and a social benet.
For both our grant making and nance fund activities we are in the process of imple-
menting a systematic approach of asking grantees and investees to dene 3 key outcomes
at the start of their grant of investment. We then track progress towards these outcomes
over the course of the investment period via standard reporting. For the majority of grants
and investments it is not a good use of resources for us to independently verify or value
the impact achieved, although for strategic interventions on particular sectors or themes or
for large individual grants we may commission a broader evaluation, we do not generally
verify or value the impact that is achieved by our investees.
We are aware that verifying and valuing impact is becoming an important topic in the
sectors in which we work and investing in the Social Impact Partnership gave us rst hand
experience of the nancial return on our investment being directly linked to a measurable
social return.
Investment: Social Impact Partnership (developed and run by Social Finance)
The Social Impact Partnership is the rst social impact bond, developed in 2010 by Social
Finance with the aim of reducing re-offending in a cohort of prisoners. A social impact bond
is an outcomes-based contract in which private investors pay the costs of an intervention,
which is delivered by service providers with a proven track record, and nancial returns
are made to the investors by the public sector if the agreed improved social outcomes take
place. If outcomes do not improve, then investors do not recover their investment.
Several social impact bonds are now being developed, but this is the rst and is still in
progress. It is hoped that, in a time of reduced public sector spending, social impact bonds
will be a way of attracting new investment in interventions with positive social outcomes.
122 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
Approaching Valuing Impact – Considering Objectives, Stakeholders & Impact
Step 1: Setting Objectives
The objectives of the Social Impact Partnership were agreed in a dialogue between Social
Finance, the Government, potential investors and the voluntary sector.
Social Finance canvassed offenders, prison staff, local stakeholders, voluntary organisa-
tions working in the eld and criminal justice experts to hear what they thought might
help stop the revolving door of short sentenced re-offending. They also began talks with
the Ministry of Justice to understand what might make a difference if an alternative source
of funding was found to deliver support to this target group. In addition, Social Finance
engaged with Trusts and Foundations, some of whom were already committed to the
Criminal Justice sector, to test whether they were prepared to support an untested but
potentially transformational proposition.
After 18 months of intense discussions, a contract was signed with the Ministry of Justice
to launch the rst social impact bond. The model aims to:
•Provide intensive support to 3,000 short-term prisoners leaving Peterborough prison
over a six year period, leading to a reduction in re-offending of at least 7.5% or more
which would trigger payments to investors; and
•Prove the social impact bond as a model which could attract new investment in future.
Investors Ministry
of Justice
3,000 male prisoners sentenced to less then 12 months
Social Impact Bond
Ormiston
Trust
St Giles
Trust YMCA Other
Interventions
On-going operating funding
£5 million
Reduction
re-offending
% of cost savings
from reducing
re-offending
Support in prison,
at the prison
gates and in
the community
Support to
prisoners’ families
while they are in
prison and post
release
Providing a
community base
Support needed
by the prisoner,
in prison and the
community.
Funded as the
need is identified
Source: Social Impact Through Effective Finance, Emily Bolton, Social Finance, Ltd., 2010
Structure of the Social
Impact Partnership:
JUNE 2015 123
CASE STUDIES
Step 2: Stakeholder Analysis
Most of the stakeholders of the Social Impact Partnership are taking part in or are directly
affected by the project, and are instrumental in its success or failure:
•Government – the public sector (Ministry of Justice).
•Investors – 17 charitable foundations, primarily from the UK and two from the US;
•Service Providers – voluntary sector charities (St Giles Trust, Ormiston Children and
Families Trust and YMCA) provide the core services, supplemented by additional
services purchased as needs are identied e.g. mental health services by MIND.
•Service Users – the prisoners taking part in the project, and those not taking part who
represent the control group.
•Her Majesty’s Prison Peterborough – run by Sodexo Justice Services. The prison resettle-
ment team works alongside the service providers to provide pre-release services.
Step 3: Measuring Results: Outcomes, Impacts and Indicators
Due to the nature of the project, each stakeholder will have their own outcomes for this
project. The Government may be looking for cost-saving and off-loading risk, whilst the
service users may want a wide range of outcomes (good housing, job prospects, a better
future for their family). For us, the most important outcome was reducing re-offending,
however the cost-saving element for the government became a key driver given the impor-
tance of the government in facilitating the whole transaction
Outcomes:
•A proven reduction in re-offending in a cohort of short-term sentence prisoners.
•A wider impact on the social investment market – evidence on whether this model works,
or how it can be improved, which is taken up by the market.
Indicator:
•Reduction in the frequency of reconviction* events (number of times an offender is recon-
victed at court in the 12 months following release from prison calculated using data held
on the Police National Computer) of the cohort group when compared to a comparison
group of prisoners discharged from other prisons during the same period (to normalize
for the inuence of external events on reconviction levels).
*It was agreed to use the indicator of reconviction events rather than re-offending, as
cost savings to Government are linked to reconviction events rather than incidences of
re-offending.
Verifying & Valuing Impact
In the case of the Social Impact Partnership, we were primarily interested in the perfor-
mance of the model itself and what lessons this provides for the future – would the stake-
holders be able to work together to deliver the main goal, reducing re-offending?
124 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
How was the Social Impact Partnership to assess the value of reducing re-offending? It
was agreed before the project began that this value could be assigned a nancial value. The
costs of reconviction saved by the public sector (the Ministry of Justice) would represent
both the value of the social outcomes achieved and the return to the private investors in
the Partnership.
Why was the cost-saving methodology selected?
•The outcome metric of the Social Impact Partnership is the foundation of its structure,
which is in essence a contract between the public sector and private investors. The
government was the crucial player in this discussion and they dened that cost savings
were the most important measure
•In order for the Partnership model to work, the target social outcomes must be tied to a
desired social change and a direct cost to the public sector.
•Being able to measure clearly and provide evidence for the social outcomes of the invest-
ment and link them directly to the costs saved by the Ministry of Justice by achieving the
social outcomes was essential to attract both the public sector and the private investors
into the Partnership.
•It was a pragmatic approach: transparent, objective & independently veriable.
Costs & Cost-Saving
•Costs were estimated before the start of the project, using:
•Data that was available and easily collected on public sector costs.
•Cost calculations that were probability-weighted.
•Average public sector costs per individual.
The cost calculations were limited to the direct cost of a reconviction, and did not include:
insurance costs, costs to victims and costs borne by society for crime prevention due to the
difculty in reliably calculating these costs.
Probability / Cost
The public sector court cost of a reconviction within 1 year (in terms of
police work, court costs, etc.)
£13,000
Reconviction cost £13,000
The likelihood of a reconviction leading to a further prison sentence
The costs associated with that further prison sentence
40%
£37,000
Average prison cost £14,800
The likelihood of a reconviction leading to a community sentence
The costs associated with that community sentence
60%
£6,000
Average community sentence cost £3,600
Average cost of a reconviction within 1 year £31,400
Source: Towards a New Economy, Emily Bolton & Louise Savell, Social Finance Ltd., 2010
(please note that the data in the chart is illustrative only)
JUNE 2015 125
CASE STUDIES
Impact measurement: current status
The project began work in August 2010 at Peterborough prison and will work with three
cohorts of 1,000 unique short-term male prisoners each over 6 years.
In order to measure the outcome, the Social Impact Partnership tracked a baseline control
group of prisoners not involved in the project, using the Propensity Score Matching method
to match the cohort to a suitable control group. The method normalized the groups for demo-
graphics & criminal history background. Both the method and outcomes are independently
assessed by QinetiQ and the University of Leicester.
There are minimum thresholds in place that must be reached to ensure that the outcomes
achieved are statistically signicant:
•A xed unit payment for each reduced reconviction event is paid provided reconviction events
in each of the three cohort groups are reduced by at least 10% relative to a control group.
•If a 10% reduction is not achieved in any of the three cohorts, then the three cohorts are
measured together at the end of the pilot. If a 7.5% reduction is achieved in total, then
investors receive payment for any cohorts that have not been paid for to date.
•There is a cap on total outcome payments to investors. Investors will therefore receive an
increasing return effectively capped at a maximum of 13% per year over the eight-year period.
Limitations of this method:
•Data integrity – the measurement of outcomes assumes that data on the prisoners and the
control group is captured and recorded correctly on the Police National Computer.
•Propensity Score Matching method (“PSM”) – the model assumes that the PSM method-
ology is successful in matching the cohort to a control group based on each individual’s
characteristics.
We are not yet able to value the impact of the Social Impact Partnership, but as a model it has
proved that it is possible to get Government, the voluntary sector and private sector investors
working together for a common goal. Whether this is taken up more widely will depend on its
success, and that of other Social Impact Bonds, which have begun to be taken up in other sectors.
Anecdotally, it is believed that the Social Impact Partnership is already having an impact.
Clients have reported a better control over their lives and lower incidences of re-offending.
Local police have conveyed similar ndings. However, the rst results will not be available
until Year 4 as it takes approximately two years for the rst cohort of 1,000 prisoners to be
released, a further 18 months to track reconviction events and a further 3–6 months to measure
the outcome against the control group.
Lessons Learned
There is a balance to be struck between robustness and complexity, time & cost. Whilst the
PSM method proved successful in developing an appropriate control group, it is a complex
and time-consuming process. This could be a barrier to replicating the model more widely.
126 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
There are also limitations for control group comparison if the social impact bond models
are scaled. With higher numbers, the population from which the control group is derived
becomes smaller, which may restrict the quality of the matching and ultimate results.
The outcome chosen in this case is the one that is best linked to cost-savings for the Govern-
ment, but this may not necessarily be the best measure of the outcomes of the project when
considered from the point of view of other stakeholders, including ourselves. For a charitable
foundation like Esmée Fairbairn, judging the value of our investment in a social intervention
(either through a grant or a social investment) is usually done through a mix of qualitative
and quantitative methods. We want to be convinced by the facts and gures of a project, but
we also want to be told the story of the work itself and its potential impact on its beneciaries.
Will the estimated cost-savings for Government materialise? It is likely that the outcomes
achieved at Peterborough will be too low to be able to shut a prison wing or close a court;
hence the cashable cost-savings may be limited.
9.5 Step 5: Monitoring & Reporting
Case Study: Auridis investing in Papilio
This case study considers monitoring & reporting through the lens of Auridis’ (a German
charitable limited company) investment in the German non-prot organisation, Papilio e.V.
(“Papilio”).
Introduction – social issue
Auridis invests in organisations and programmes that sustainably improve opportunities
for socially disadvantaged families and their small children.
The investment focus is on the dissemination and replication of successful approaches. The
core portfolio consists of 19 organisations that mostly receive grants for 3 to 10 years. Due to
the fact that we are investing in early childhood development, most investments do preven-
tion work with impact that cannot be easily related to the activities of the investees.
Since 2010 Auridis has been supporting Papilio. Papilio has developed and promotes
a kindergarten programme for early childhood prevention of addiction and violence.
Substance addiction and violence are widespread, in particular among the juvenile popula-
tion, with extremely high negative effects on the society and national economy. The likeli-
hood of young people developing a substance addiction or violent behaviour is to a relevant
degree determined by the individual’s capacity to cope with stress and adversity, her or his
so-called socio-emotional competences (resilience). Children develop these competences in
early childhood, i.e. at age 3 to 6. The Papilio programme intends to enhance child educators’
abilities to support young children in developing positive social and emotional competences.
JUNE 2015 127
CASE STUDIES
Papilio integrates as a part of the pedagogic concept in the kindergarten, with elements
like the “toys-go-on-holiday-day” or “Paula and the trunk pixies”, a puppet play with
pixies representing the four main emotions (joy, anger, sadness, fear). Other than other
programmes offered in German kindergarten, Papilio accompanies the children during
their whole kindergarten time (as opposed to a curricular one-time activity).
The Papilio programme is disseminated by way of a train-the-trainer model, with head-
quarters in Augsburg, Germany. Since 2002 close to 5000 child care workers in 11 federal
states all over Germany have been trained with the Papilio programme and approximately
100,000 children (extrapolated) could be reached. We accompanied Papilio by nancing a
business planning phase from 2010 to 2011 and is currently supporting the growth phase
from 2012 to 2017 (estimated).
Auridis’ approach to Monitoring & Reporting
Prior to investing long-term in a SPO we nance and actively accompany a business
planning phase (“impact planning”). During this phase a shared understanding of the
social issue, the theory of change, the expected impact, the main levers for organisational
success, and the relevant indicators is developed between Auridis and the investee. During
the growth nancing phase the organisations’ development and performance are tracked
on a quarterly and annual basis using milestones and specic metrics agreed upon between
Auridis and the investee.
We track the development in our investee database which collects information such as
nancial data, grant history, essential documents such as grant agreements, investees’
progress reports, and the milestones. The investee database has been developed in house
using Microsoft Access. All other data is stored in a le storage system. We do not aggregate
output, outcome, or impact data of the portfolio organisations, as we believe this data
would lead to misinterpretation.
Prevention and impact measurement dilemma
The expected social long-term impact:
•The early development of protection factors (social-emotional competences) prevents
the risks that lead to addiction and violence.
•This forms the basis for a self-paced and independent adult life.
Challenge:
It is a great challenge to measure the long-term social impact of prevention work with
quality assurance.
Specic early childhood interventions can only be linked to later developments or
outcomes based on large-scale randomised longitudinal studies – if at all.
Ethical problem of working with comparison groups in scientic studies for longer time
period (because it would require specic target groups to be excluded from change)
Proxy:
Short-term output indicators can give an indication of sustainable impact.
128 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
We support the development of information management and controlling systems of our
investees by agreeing on reporting requirements, nancing the development of tailor-made
systems, bringing in pro bono consultants, and sharing experiences throughout the portfolio.
The level of sophistication of the information management and controlling systems of our
investees varies across our portfolio. We estimate that approximately one quarter of our
investees have good systems in place, on a par with Papilio, which we discuss shortly; one
quarter of our investees are about to develop a robust system; one quarter are considering the
development of an information management system; and for the remaining quarter it is not
an issue on their radar screen given they are very early stage and are needing to focus their
efforts and resources elsewhere.
Papilio’s approach to monitoring and reporting
Papilio commissioned a scientic study on the outcomes of the programme from 2002 to
2005 with 700 children and their families. The results showed positive outcomes for children,
kindergarten, and parents, such as reduction of rst deviant behaviour of the children and
better learning abilities at school, positive effects on cooperation within the kindergarten
team, and a better basis for education partnerships with the parents.
As outcomes are not always easy to measure in the short term we decided to use large scale
output indicators to serve as proxies for outcome. For example: the number of actively prac-
ticing and certied Papilio child care workers; the number of parents ordering Papilio books
and DVDs for their children, etc. The underlying assumption is that these indicators are good
proxies for the expected long-term outcomes.
1. Aggregation of impact data
Papilio introduced an online, web-based database system for the Papilio trainers to report
their activities to headquarters. Information such as names and contact details of trainers, child
care workers, and kindergarten as well as number, date, place, and participants of trainings
and supervisions and the progress of the certication process are recorded by the trainers.
In addition, Papilio tracks the quantities of materials ordered (books, DVDs, educational
material, etc.). The Papilio team gets monthly reports of all aggregated data. The prerequisite
for Papilio to introduce such a tool was a German-language, very simple web-log-in system.
JUNE 2015 129
CASE STUDIES
2. Aggregation of nancial data
Financial data tracked by the book-keeping and accounting system as well as output data
recorded in the web-based database are integrated in monthly and quarterly reports. This
is done semi-automatically by the controller of Papilio using Microsoft Excel templates
for summarizing the web-based database and the accounting software. Data processed
includes actual cash ow, ACT vs. PLAN data, organisational development indicators, and
output indicators as described above.
3. Stakeholder presentation of the data
The data collected is presented to different stakeholders in different formats:
A monthly dashboard report is produced for the organisation’s management, summarising
key nancial and output indicators. This is the basis for the organisation’s day-to-day
management. More detailed reports are produced for a variety of funders in accordance
with their respective requirements.
In order to streamline reporting and to increase the efciency of the reporting process,
Papilio has started to produce annual reports in accordance with the German Social
Reporting Standard (SRS). The SRS has been developed by a consortium of German high-
impact funders such as Auridis, BonVenture, and Ashoka, in cooperation with experts and
researchers. SRS provides a structure to report on the problem to be solved, the contribu-
tion of the SPO to the solution and the achieved social impact together with organisational
and nancial data. Reports based on SRS should satisfy most reporting requirements of
different funders. To the extent this is not the case the reports can be complemented by
additional annexes.
Papilio started to use the SRS structure during its business planning phase. Many of the
elements developed during this phase are being reused for reporting purposes, such as the
concise description of Papilio’s theory of change.
We encourage our investees to use the SRS, however to date we haven’t pushed any of our
investees to do so, preferring to offer them assistance in introducing it. In our view, using
SRS will improve the consistency and comparability of the information that we receive.
130 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
CASE STUDIES
Auridis’ investment in Papilio
In its rst years of operation the Papilio team had an expanding system with a mix of Excel
sheets, Word lists, and paper lists spread all over the team, which made it very difcult to
aggregate the data. As Papilio evolved a more holistic system was needed. Supported by
Auridis and one other major funder, the Papilio team started to develop their own informa-
tion management system. No German-language system that could be tailored to Papilio’s
needs could be identied. The denition and streamlining of processes took 1.5 years and was
supported by IT and nance experts. The result – a self-developed, tailor-made data system
– was put into action in 2012 and will need approximately half a year of implementation. So
far Papilio only counts hard facts, but is thinking about how to measure soft outcomes in the
future.
What has Papilio learned from the development process?
•Usability is the key success factor for the usage of the system. Therefore, simplicity is the
most important requirement for the information management system.
•The underlying processes are more important than technology.
•The process should be steered by an experienced IT developer who can, and does, ask the
team for input regarding the reporting contents and formats required and translates them
into a technical solution.
•The whole team and some of the other (external) users need to be integrated in the develop-
ment process as they will be the main beneciaries of the system.
•The development of an information management system needs an iterative process and a lot
of end consumer testing and reversing.
End-users do have a broad variety of experience with, and afnity to, web-based systems.
Therefore, user training is required to ensure the same understanding of data and time periods,
to check the technical usability of the system on the users’ hardware (social workers tend to
have only access to defunct technology) and to agree on reporting timelines. The end-user
should optimally also understand the added value of using the new tool.
The costs for the development of the system were 20,000 to 30,000 Euro for staff time and the
IT developer. In many cases this kind of work does have a high potential for pro bono work
from external consultants and IT companies. The hours saved if the information manage-
ment system is working uid and properly are expected to outweigh the upfront investment
(although no calculation was made for this).
Recommendations
We believe that investees should be encouraged to allocate substantial money to information
management, as it is a key to sustainable growth and stakeholder reporting. Excel is only
suitable for the early development stage. In most cases, the necessity to introduce more or less
sophisticated monitoring and evaluation systems only becomes apparent once the scaling-
up, or dissemination, starts to accelerate following the VPO/SI’s investment. In our experi-
ence, the monitoring and evaluation systems used by one organisation can only inspire the
JUNE 2015 131
CASE STUDIES
development of tailor-made solutions for other organisations with a different business model,
but cannot be transferred “as is”.
Importantly, the investees need external help to implement these systems, which can be facili-
tated by the VPO/SI. In a number of cases, the organisations in the Auridis portfolio were
supported on a pro bono basis by consultants of OC&C Strategy Consulting. Their focus was
on asking strategic questions in order to dene the expected end product before starting with
the “how to questions”.
Regarding impact measurement, substantial scientic impact studies are usually very
expensive (>0.5 million Euros), and such funding is difcult to obtain (if not provided by
the VPO/SI). In most cases gut feeling, proxies, and scientic assumptions based on other
studies need to be used, especially in prevention work. But be aware to not only count what
is countable – soft facts matter more. However, it is important to be transparent about the
assumptions and their basis. Gut feeling alone won’t do it.
Financing an information management system
The development of an information management system will need signicant work by an
experienced IT developer. If the service would be purchased in the for-prot market, signi-
cant costs would accrue. VPO/SIs should provide cash and encourage their investees to invest
in IT infrastructure to streamline processes and strengthen the operational capacities of the
investee.
Nevertheless, given the usual shortage of money in SPOs, this topic offers the opportunity to
fundraise a service grant from a for prot service provider. In combination with a pro bono
consultant the development and implementation process can be realised with minimum cash
spend. VPO/SIs can play an active role in connecting their investees to service providers
and pro bono resources. Investments in a sound information management system should be
written off in many years and maybe shared with other organisations to make the investment
worth wile.
Typical costs if no pro bono support applied:
Phase Cost range (Euro)
Specication 5,000 – 15,000
Realisation 20,000 – 50,000
Testing 1,000 – 5,000
Pilot phase 2,500 – 5,000
Yearly operation 1,000 – 5,000
Note: these gures are just
indicative, based on the Auridis
experience in this specic case.
However, it can be used as a
general indication of expected
costs.
132 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
PART 4:
Appendices
JUNE 2015 133
APPENDICES
GLOSSARY OF TERMS
42. Definitions defined so as to
be aligned with definitions in
the glossary of Hornsby, A;
Blumberg, G., (2013), “The Good
Investor: A book of best impact
practice”, Investing for Good.
10.0 Glossary of Terms
Accountability
The obligation of an organisation to account
for or take responsibility for the effect of its
activities.
Activities
The concrete actions, tasks and work carried
out by the organisation to create its outputs
and outcomes and achieve its objectives.
Attribution
Attribution takes account of how much of
the change that has been observed is the
result of the organisation’s activities, and
how much is the result of actions taken
simultaneously by others (e.g. other SPOs,
government).
Balanced scorecard
Developed by Robert Kaplan and David
Norton, the balanced scorecard denes
what an organisation means by “perfor-
mance” and measures whether the organi-
sation is achieving desired results. The
Balanced Scorecard translates mission and
vision statements into a comprehensive set
of objectives and performance measures
that can be quantied and appraised.
The traditional balanced scorecard of the
business world has also been adapted by
Social Enterprise London with the aim of
assisting social enterprises to examine their
strategies and desired outcomes, which can
be tracked over time.
Beneciaries
The people, communities, broader society
and environment that a SPO seeks to reach
through its activities. Beneciaries can be
affected positively or negatively by the
activities of the SPO.
Contributors
The people, communities, broader society
and environment that contribute to the SPO
performing its activities. Contributors can
enhance or decrease the effect of the activi-
ties of the SPO.
Cost / benet analysis
A measurement of the benets of an organi-
sation’s activities in monetary terms com-
pared to their costs. A cost / benet ratio is
determined by dividing the projected ben-
ets of an activity by the projected costs.
SROI is an example of cost / benet analy-
sis applied to SPO activities.
Deadweight
Deadweight is the change that would have
happened anyway i.e. the outcomes the ben-
eciaries would be expected to experience
if the organisation were not active. This is
sometimes called the “baseline” or “coun-
terfactual”. Deadweight includes the pro-
gress or regress beneciaries typically make
without the organisation’s intervention.
Displacement
Displacement occurs when the positive out-
comes experienced by beneciaries access-
ing the organisation’s services are offset by
negative outcomes experienced by another
group elsewhere (also as a result of the
organisation’s activities).
Drop-off
Drop-off occurs when, over time, the effects
of the output and the observed out-comes
decreases (e.g. beneciaries relapse, lose the
job attained, revert to previous behaviours).
The organisation’s denition of its out-
comes sets the scope for how long they are
expected to last. Drop-off occurring within
this period is accounted for in assessing the
organisation’s true impact.
42
134 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
APPENDICES
GLOSSARY OF TERMS
Expert group
The Expert Group is the 27 strong group of
practitioners, consultants, academics and
representatives from other networks who
contributed to the development of this prac-
tical guide.
Global Value Exchange
Global Value Exchange is a database
of values, indicators and outcomes for
stakeholders.
Impact
See: Social Impact
Impact investor
See denition for social investor.
Impact measurement initiative
The initiative undertaken by the European
Venture Philanthropy Association with the
support of the Expert Group to create this
practical guide for impact measurement
with the aim of spreading best practice in
the venture philanthropy and social invest-
ment sector.
Impact value chain
Represents how an organisation achieves
its impact by linking the organisation to its
activities and the activities to outputs, out-
comes and impacts.
Inputs
The resources, whether capital or human,
invested in the activities of the organisation.
Indicators
Indicators are specic and measurable
actions or conditions that assess progress
towards or away from outputs or out-
comes. Indicators may relate to direct quan-
tities (e.g. number of hours of training pro-
vided) or to qualitative aspects (e.g. levels
of beneciary condence).
Investee
A SPO that receives investment from a
VPO/SI.
Investment
We use investment throughout this docu-
ment as including the range of nancing
instruments from grants, loans to equity.
IRIS
IRIS is the Impact Reporting & Investment
Standards initiative of the Global Impact
Investing Network (“GIIN”) and was devel-
oped to provide a common reporting lan-
guage for impact related terms and metrics.
IRIS indicators
IRIS indicators are a set of standardised
metrics that can be used to describe an
organisation’s social, environmental and
nancial performance.
Logic model
Logic models are usually a graphical depic-
tion of the logical relationships behind how
an organisation does its work i.e. the rela-
tionships between the activities, outputs,
outcomes and impacts.
Materiality
Materiality refers to an assessment made to
determine the factors that are relevant and
material to include in a true account of the
organisation’s impact.
Monetisation
Monetisation is the process of transforming
the value of outcomes and/or impacts into a
unit of currency. SROI is a way to monetise
the value of social impact in nancial terms.
Organisation
In this case an entity working to bring about
positive social impact i.e. the term includes
SPOs and VPO/SIs.
JUNE 2015 135
APPENDICES
GLOSSARY OF TERMS
Outcomes
The changes, benets, learnings, or other
effects (both long and short term) that result
from the organisation’s activities.
Outcomes matrix
A classication tool, developed by Big Soci-
ety Capital in combination with Investing
for Good and other UK based VPOs, for
use by investors and SPOs to map areas in
which, and beneciaries for whom, their
impacts are being achieved.
Outputs
The tangible products and services that
result from the organisation’s activities.
Participatory impact assessment
Participatory impact assessment is the pro-
cess of engaging people and communities in
the actual measurement of impact on their
livelihoods, for example through the use of
focus groups or survey.
Perceived value
Perceived value is a beneciary’s opinion
of a product’s or service’s value. It may
have little or nothing to do with the prod-
uct’s or service’s price, and depends on the
product’s or service’s ability to satisfy their
needs or requirements.
Progress out of poverty index (“PPI”)
Developed by the Grameen Foundation, the
progress out of poverty index estimates the
likelihood that an individual falls below the
national poverty line, the $1/day/PPP and
$2/Day/PPP international benchmarks.
The PPI uses 10 simple indicators that eld
workers can quickly collect and verify.
Quality adjusted life year (“QALY”)
A quality adjusted life year is an expression
of health in terms of time (life years) and
quality of that life (adjusted for years lived
with diseases). It is based on the number of
years of life that would be added by a par-
ticular medical intervention and the quality
of the life lived during those years.
Revealed preference
Revealed preference theory was pioneered
by American economist Paul Samuelson and
is based on the assumption that the prefer-
ence of beneciaries can be revealed by their
purchasing behaviour. It tries to understand
preferences of beneciaries among bundles
of goods, given their budget constraints.
Social balanced scorecard
The traditional balanced scorecard adapted
by Social Enterprise London with the aim of
assisting social enterprises to examine their
strategies and desired outcomes, which can
be tracked over time.
Social impact
The attribution of an organisation’s activi-
ties to broader and longer-term outcomes.
To accurately (in academic terms) calcu-
late social impact you need to adjust out-
comes for: (i) what would have happened
anyway (“deadweight”); (ii) the action of
others (“attribution”); (iii) how far the out-
come of the initial intervention is likely
to be reduced over time (“drop off”); (iv)
the extent to which the original situation
was displaced elsewhere or outcomes dis-
placed other potential positive outcomes
(“displacement”); and for unintended con-
sequences (which could be negative or
positive).
Social investor (“SI”)
An organisation pursuing a social invest-
ment approach.
Social investment
Social investment is the provision and use
of capital to generate social as well as nan-
cial returns.
136 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
APPENDICES
GLOSSARY OF TERMS
The social investment approach has many
overlaps with the key characteristics of ven-
ture philanthropy, however social invest-
ment means investment mainly to gener-
ate social impact, but with the expectation
of some nancial return (or preservation of
capital).
Social purpose organisation (“SPO”)
An organisation that operates with the pri-
mary aim of achieving measurable social
and environmental impact. Social purpose
organisations include charities, non-prot
organisations and social enterprises.
Social return on investment (“SROI”)
Social return on investment is a framework
for measuring and accounting for the broad
concept of value. It tells the story of how
change is being created by measuring social,
environmental and economic outcomes and
uses monetary values to represent them.
This enables a ratio of benets to costs to be
calculated e.g. a ratio of 3:1 indicates that an
investment of €1 delivers €3 of social value.
Stakeholder
Any party that is effecting or affected by
the activities of the organisation. The most
prominent stakeholders are the direct or
target beneciaries, though stakeholders
as a group also includes the organisation’s
staff and volunteers, its shareholders and
investees, its suppliers and purchasers and
most likely the families of beneciaries and
those close to them, and the communities in
which they live.
Stated preference
Stated preference is a method used to assess
the value of an outcome or impact by using
real nancial data such as prevented costs,
spending and changes in nancial income.
Theory of change
A theory of change denes all building
blocks required to bring about a given
long-term goal. This set of connected build-
ing blocks is depicted on a map known as
a pathway of change or change framework,
which is a graphic representation of the
change process.
Unintended consequences
Unintended consequences are those that
come about as a result of the organisation’s
activities, but are not part of the desired
effect. They may be foreseen (anticipated but
not intended), or unexpected (positive or
negative). Unintended consequences often
relate to effects upon stakeholders other
than the organisation’s target beneciaries.
Value Game
The Value Game is a survey tool that asks
questions to stakeholders in order to reveal
the value of outcomes. It shows how stake-
holders value the outcomes they experience
relative to other products they also value.
Venture philanthropy (“VP”)
Venture philanthropy is an approach that
includes both the use of social investment
(equity and debt instruments) and grants.
The key characteristics of venture philan-
thropy include high engagement, organisa-
tional capacity-building, tailored nancing,
non-nancial support, involvement of net-
works, multi-year support and performance
measurement.
Venture philanthropy organisation
(“VPO”)
Organisations following the venture phi-
lanthropy approach.
JUNE 2015 137
APPENDICES
SOURCES
11.0 Sources
Step 1: Setting Objectives
General Resources on Goal Setting
•Locke, E. A. & Latham, G. P., (1990). “A theory of goal setting and task performance”.
Upper Saddle River, NJ: Prentice-Hall.
•Locke, Edwin A.; Latham, Gary P. “Building a practically useful theory of goal setting and
task motivation: A 35-year odyssey.” American Psychologist, Vol 57(9), Sep 2002, 705–717.
•Doran, G. T., (1981). “There’s a S.M.A.R.T. way to write management’s goals and objectives”.
Management Review, Volume 70, Issue 11 (AMA FORUM), pp. 35-36.
Setting Objectives in Impact Measurement
•Sept, Naylor and Weston. 2011. “Measuring the impact of social programs: A review of best
practices.” Stanford Global Supply Chain Management Forum; Socially & Environmen-
tally Responsible Supply Chain Program: http://www.gsb.stanford.edu/sites/default/
les/documents/MeasuringPerformance0fSocialPrograms-040811-1.pdf
Note that this provides a slightly different framework for considering approaches to
social performance measurement:
•Kellogg Foundation’s Logic Model (Chapter 1: The “What” and the “Why” of Logic
Models): http://www.wkkf.org/knowledge-center/resources/2006/02/WK-Kellogg-
Foundation-Logic-Model-Development-Guide.aspx
•Grantcraft, The Ford Foundation, “Mapping Change: Using a Theory of Change to Guide
Planning and Evaluation” (Compares “Theory of Change” and “Logic Model”):
http://portals.wi.wur.nl/les/docs/ppme/Grantcraftguidemappingchanges_1.pdf
•www.theoryofchange.org
•Annie E. Casey Foundation (www.aecf.org). “Theory of Change: A Practical Tool for
Action, Results and Learning”.
Step 2: Analysing Stakeholders
•Accountability, “The Stakeholder Engagement Manual”. http://www.accountability.org/
images/content/2/0/207.pdf
http://www.accountability.org/images/content/2/0/208.pdf
•The New Economics Foundation, “Participation Works!” http://www.neweconomics.
org/publications/participation-works
•Robert Wood Johnson Foundation, “A Practical Guide to Engaging Stakeholders in Deter-
mining Evaluation Questions” http://www.rwjf.org/pr/product.jsp?id=49951
•Forthcoming guide by Social Value UK (formerly SROI Network) “Supplementary
Guidance on Stakeholder Involvement” http://socialvalueuk.org/publications/publica-
tions/doc_download/368-supplementary-guidance-on-stakeholder-involvement
•The Value Game – a stakeholder led valuation tool http://www.valuegame.org/
•Geoff Mulgan, “Measuring Social Value”, Stanford Social Innovation Review, (2010).
http://www.ssireview.org/articles/entry/measuring_social_value
138 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
APPENDICES
SOURCES
Step 3: Measuring Results: Outcome, Impact, Indicators
•Nelson & Ratcliffe, (2010), “A Guide to Actionable Measurement”, Bill & Melinda Gates
Foundation
•IRIS database of indicators: iris.thegiin.org
•Global Value Exchange database of indicators: http://www.globalvaluexchange.org/
•Social Balanced scorecard and other tools: http://www.proveandimprove.org/tools/
socialenterprise.php
•Ruby Sandhu-Rojon, UNDP, “Selecting Indicators for impact evaluation”
•Millennium Development Goals: http://www.un.org/millenniumgoals/
•Progress Out of Poverty Indicator: http://www.progressoutofpoverty.org/
Step 4: Verifying & Valuing Impact
•http://www.roguecom.com/interview/overview.html
•http://techinlibraries.com/cowgill.pdf
•http://www.eldrbarry.net/roos/eest.htm
•Mitchell, R. and R.Carson, (2005). „Using surveys to value public goods; the contingent
valuation method“. Washington USA.
•Champ, P., Koyle, K. and Brown, T. (2003). „A Primer on nonmarket valuation“.
Dordrecht (NL): Kluwer.
•Wellbeing valuation: www.ncbi.nlm.nih.gov/pubmed/17380470
•Further information on QALY can be found at: http://www.medicine.ox.ac.uk/
bandolier/painres/download/whatis/QALY.pdf
•More information on these techniques (and many others) can be found in the TRASI
database: http://trasi.foundationcenter.org/
Step 5: Monitoring & Reporting
•Social Reporting Standard: http://social-reporting-standard.de/ by Auridis, BonVen-
ture, Phineo, Ashoka, PWC a.o.
http://srs.aufbau-server.de/en : English webpage
•www.mande.co.uk : website on monitoring and evaluation; lot of information, docu-
ments, cases, etc.
•http://www.unfpa.org/monitoring/toolkit.htm (United Nations Population Fund)
•http://web.undp.org/evaluation/handbook PDF-Handbook UNDP.
•“Principles of Good Impact Reporting”, by NPC a.o.
•World Bank, “Monitoring & Evaluation: Some Tools, Methods & Approaches”.
•PULSE (http://pulse.app-x.com)
•GIIRS (www.giirs.org) provides both company and fund impact ratings, each with
current and historical analyses of impact performance for comparative use. In order to
scale the impact investing marketplace, investors require an independent third-party
impact ratings product that is comparable, transparent, and easy to use.
JUNE 2015 139
APPENDICES
SOURCES
Webinars
The Expert Group members were divided into working groups to focus on a particular step
in the impact measurement process. Their ndings resulted in a webinar-based presenta-
tion to the other members of the Expert Group and the case studies found in section 9.1. The
working groups for each step were as follows. A “*” denotes the author of the case study.
•Step 1 - Setting Objectives: Van Dijk, M., Social Evaluator; Presner, B., Acumen Fund;
Kagerer, T., LGT Venture Philanthropy; *Sandvold, O., Ferd Social Entrepreneurs;
Ferraro, F., IESE Business School.
•Step 2 – Analysing Stakeholders: Grabenwarter, U., European Investment Fund; *Niles,
M., Impetus Trust; Kennedy, R., CAN Breakthrough; Robin, S., Stone Soup.
•Step 3 – Measuring results: outcome, impact, indicators: Gelfand, S., the GIIN; Lane
Spollen, E., One Foundation; *Allevi, L., Oltre Venture; Stievenart, E., ESSEC Business
School.
•Step 4 – Verifying & Valuing Impact: Nicholls, J., SROI Network (now Social Value UK);
Varga, E., NESsT; *Petkova, I., Esmée Fairbairn Foundation; Nicholls, A., Skoll Centre
for Social Entrepreneurship.
•Step 5 – Monitoring & Reporting: Scholten, P., Scholten & Van der Meij; Backstrom, C.,
Naya AB; Tarakeshwar, N., Children’s Investment Fund Foundation; *Leissner, C.,
Auridis; Santos, F., INSEAD Business School.
Interviews
•Allevi, L., Managing Director, Oltre Venture (September 24, 2012)
•Blokhuis, M., Director, Noaber Foundation (October 19, 2012)
•Crane, G., Impact and Learning Ofcer, Esmee Fairbairn Foundation (September 26, 2012)
•Kagerer, T., COO, LGT Venture Philanthropy (September 10, 2012, by email)
•Leissner, C., Project Manager, Auridis (October 8, 2012)
•Luebbering, J; Elsemann, K., Partnership Development, Streetfootballworld (September
10, 2012)
•Lumley, T., Head of Development, New Philanthropy Capital (September 7, 2012)
•Mason, C., COO, Big Society Capital (September 27, 2012)
•Niles, M., Investment Director, Impetus Trust (September 24, 2012)
•Sandvold, Ø., Director of Business Development, Ferd Social Entrepreneurs (September
17, 2012)
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
The European Venture Philanthropy Association (EVPA)
Established in 2004, EVPA aims to be the natural home as well as the highest-
value catalytic network of European Social Investors committed to using venture
philanthropy and social investment tools and targeting societal impact.
EVPA’s membership covers the full range of venture philanthropy and social
investment activities and includes venture philanthropy funds, social investors,
grant-making foundations, impact investing funds, private equity rms and
professional service rms, philanthropy advisors, banks and business schools.
EVPA members work together across sectors in order to promote and shape the
future of venture philanthropy and social investment in Europe and beyond.
Currently the association has over 180 members from 25 countries, mainly based
in Europe, but also outside Europe showing the sector is rapidly evolving across
borders.
EVPA is committed to support its members in their work by providing
networking opportunities and facilitating learning. Furthermore, EVPA aims to
strengthen our role as a thought leader in order to build a deeper understanding
of the sector, promote the appropriate use of venture philanthropy and social
investment and inspire guidelines and regulations.
http://www.evpa.eu.com
ISBN 9789082316087
VENTURE PHILANTHROPY
SOCIAL INVESTMENT
Rue Royale 94
1000 Brussels, Belgium
Tel: +32 (0) 2 513 21 31
Email: info@evpa.eu.com
EVPA is grateful to:
Acanthus Advisers, Adessium
Foundation, BMW Foundation
and Omidyar Network for their
structural support
EVPA is grateful to:
Fondazione CRT for the support
of its Knowledge Centre
