There were three big steps in our 2016 impact assessment process:
Collecting data on changes at outcome and impact level
Three main sources provided information on changes in relation to VECO’s work:
- Farmer surveys offer us data from a random selection of targeted farmers on income, profit margins, diversification of income, environmental production practices, gender relations and the role of youth in the value chain.
- SCOPEinsight assessments rate the professionalisation of farmer organisations on several dimensions, ranging from internal and financial management skills to business-savviness, access to finance and their influence on trading relations (VECO started to use this tool in 2016 – a conversion table allowed for comparisons with the matrix used to calculate the baseline and intermediate scores).
- Half-yearly monitoring reports, in which colleagues update the outcome indicators, capture evolutions in sector policies, and collect stories that illustrate the dynamics behind the observed changes.
Carrying out a contribution/attribution analysis
A critical step in the impact study is to connect the implementation of VECO’s interventions with the observed changes at outcome and impact level. To ensure the validity of these connections, a set of interviews and focus group discussions were organised with partners, farmers and key stakeholders.
These discussions also helped to distinguish between those observed changes at outcome and impact level that are attributable to our interventions (i.e. the observed changes have been brought about uniquely thanks to the intervention and would not have occurred without VECO) and those to which our interventions have contributed (i.e. the intervention played a role in bringing about the observed changes but VECO was not the sole reason why they occurred).
Finally, a counterfactual reflection complemented this analysis: what would have happened if VECO had not implemented its interventions? In absence of comparison groups, it was not possible to carry out a profound counterfactual analysis during the 2016 impact assessments. Starting from 2017, however, VECO shall pilot the use of comparison groups in some interventions to allow for increased rigour in its assessments and gain an even more thorough understanding of how it brings about change.
Disseminating the results and collective learning
Upon terminating the impact assessment studies, VECO shares its results and recommendations with its partners organisations, key stakeholders, donors and the wider public. Furthermore, these studies nurture the internal discussions and learning processes in the organisation as a whole and between the regional offices. We keep track of good practices, lessons learnt and challenges, and exchange them among the topical implementers within our offices, to improve our intervention strategies and attain a greater impact.