Tender: end-line impact assessment

Tender: end-line impact assessment

03/07/2020
in Jobs

1. Context

Rikolto is an international NGO with more than 40 years’ experience in partnering farmer organisations and food chain stakeholders across Africa, Asia, Europe and Latin America. Rikolto runs programmes in 16 countries worldwide through seven regional offices. We’re a close-knit network of accessible and knowledgeable colleagues, willing to share experience and eager to inspire others. This network of Rikolto offices is supported by a global support team, which fulfils key advisory roles and offers shared services such as finance, monitoring & evaluation, people & organisation, communication and fundraising.

1.1 Overall evaluation framework

Rikolto wishes to evaluate its impact on three levels:

  1. Improved livelihoods of smallholder farmers (m/f);
  2. Improved business and organisational capacities of targeted farmer organisations;
  3. Improved institutional environment (public/private policies, regulations, etc.).

All Rikolto interventions have frameworks in which pathways of change are described along with relevant outcome indicators, which are updated on a yearly basis. These form the basis for half-yearly strategic reflections within each regional office and with partner organisations, which result in updates to the intervention strategies and target values.

Complementing these regular monitoring data and practices, Rikolto has developed a farmer survey to obtain more insight into the livelihood of farmers, such as production, commercialization and income figures, environmental sustainability practices, participation in the value chain, etc. At the level of farmer organisations, Rikolto uses SCOPEinsight assessments to measure every 18-24 months their business and organisational capacities.

The current global impact measurement framework of Rikolto covers the five-year period (2017-2021) in which most of its interventions are inscribed:

In the last two years of the current programme-period, Rikolto seeks to conduct the following two evaluation processes:

  • September 2020-March 2021: Efficiency/cost-benefit analysis of its work from a systems perspective, building on the 2019 Mid-Term Review (accompanied self-assessment), which should deliver a practical and easily replicable methodology.
  • April-December 2021: End-line impact assessment of its work, with an emphasis on the potential of attained results to contribute towards global impact through commodity and sustainable cities platforms.

Since tackling the efficiency question within the end-line evaluation would risk overburdening both our teams and the team of evaluators, we have decided to turn the efficiency analysis into a separate process prior to the end-line evaluation. These two evaluation processes are therefore tendered separately, although they are closely connected. We understand that the efficiency analysis may deliver inconclusive results that need to be complemented with data from the end-line assessment. The methodology and results of the efficiency analysis will therefore have to be taken into account during the end-line assessment to make sure it can be nourished with relevant data for conclusive results.

1.2 Accountability requirements

Per the current regulatory framework governing the DGD programme, each outcome should be assessed by an independent evaluator. In the case of Rikolto’s DGD programme, each outcome equals a country programme, which is typically constituted by the sum of 2 to 3 commodity-specific programmes.

With the means available, it is unrealistic to conduct 13 full-fledged country evaluations covering 30+ commodity-programmes. However, the regulatory framework is flexible in allowing the following modes of evaluation:

  • Mix of field & desk evaluation: evaluation with in-country field visits only for outcomes with poor performance, i.e. where actual achievements deviate largely from expected results, or for outcomes where there is high learning potential; the remainder of outcomes may be evaluated through desk study based on the organisation’s M&E data.
  • Case study per outcome: one or more representative interventions for each outcome may be evaluated in depth, from where a general assessment is made for the whole outcome.
  • Participatory evaluation: for organisations with a thorough M&E system, (part of) the evaluation can be conducted internally under supervision of an external evaluator. The external evaluator bears full responsibility for the evaluation process and outcome.

Overall, we are looking for practical solutions that can satisfy our need for accountability, learning and strategy development.

2. Scope of the evaluation

2.1 Evaluation object

The object of this evaluation are the 13 specific outcomes (1 per country) of the DGD funded 2017-2021 programme implemented by Rikolto. The underpinning Theory of Change is provided in Annex 1; the full programme proposal can be consulted on our website.

In line with the above-mentioned flexibility regarding the accountability requirements, Rikolto decided to focus on assessing one case in depth per country, which will be conducted by the external evaluator, with support of Rikolto staff regarding available data and sense-making processes. The evaluation of the remaining interventions which contribute to the same outcome shall be conducted internally under supervision of the external evaluator, who shall validate the internal assessments and bear responsibility for the evaluation process and outcomes. Based on the assessment of the in-depth case study and the complementary internal assessments of the other interventions at country level the external consultant is expected to formulate a general assessment for the whole outcome. The idea is that one local consultant would support the evaluation per outcome as part of the overall external evaluation team.

2.2 Research questions

Retrospective research focus

The retrospective research focus should be focusing on the impact of Rikolto’s work at farm, farmer organisation and institutional level. However, since we would be overstretching the available budget if we wanted to assess our impact at all 3 levels, we decided to delimit our research focus and questions to the levels that are of most interest to us, connected to the systemic impact of our work. The research questions shall focus on those change dynamics that we’re not certain of yet and that have high potential for impact in the food system.

The following table summarizes a first expression of interests per cluster, however a further concretization of retrospective research focus and specific questions shall be done together with the consultant during the inception phase.

Prospective research focus

As mentioned above, the prospective evaluation component shall focus on the potential of the generated evidence (per case study and the sum of it) to be leveraged at a global level.

The proposed prospective research questions are:

  • What is the potential to leverage the evidence generated in the cases studied at a higher-level, global platform? [rice: SRP; FSC: Milan Pact; Cocoa: World Cocoa Foundation; Coffee: ?]
  • What country-specific factors influence the potential to leverage the findings of these case studies at a higher level?
  • Do our current strategies allow us to build convincing business cases (for private sector/public authorities/impact investors to invest in)? What cases have already convincingly been made (also taking into account social and environmental return on investment)? What are promising strategies to build these business cases?

While it is clear that these questions will be nourished by the findings to the retrospective research questions at country level, these questions shall be evaluated at cluster level. We therefore expect a separate report per cluster, elaborating on these questions for several countries, which contribute to the respective cluster. These research questions are agreed upon as equally valid across all clusters, can however still be further discussed and concretized with the consultant during the inception phase.

Specific evaluation question related to the Covid-19 crisis

In light of the current events and dynamics unfolding around the Covid-19 pandemic and its effects on food systems, Rikolto would be interested to additionally explore one or two evaluation question on our response to the pandemic. Possible options are:

  • What were the factors that facilitated a quick and impactful response to Covid-19?
  • Which impact did Covid-19 responses have on the target groups?
  • To which extent has Rikolto’s response to the COVID-19 outbreak left a more resilient food system in place, able to respond more swiftly to a next systemic crisis?

Despite the interest and relevance of these evaluation questions we are aware of the possible constraints to such an evaluation (e.g. data availability). The consultants shall therefore explore the evaluability of these questions together with Rikolto during the inception phase.

2.3 End users

As mentioned above, this end-line assessment has a twofold functionality for Rikolto: 1) to satisfy our accountability needs towards DGD and 2) at the same time feed our internal strategy development for the following programme period. The target audience of the evaluation results are therefore both external (DGD portfolio managers but also other implementing partners to which we would like to communicate about the impact of our work; and Rikolto’s strategic partners with whom we collaborate in the food system) and internal (Rikolto management and programme teams).

3. Relevant remarks for the evaluation methodology

The following data is available at Rikolto to assess impact at the different levels:

  • At Rikolto we have collected farm level data from all countries and all our interventions to get better insight and understanding of farmer livelihoods and adjust our interventions to the needs of our end beneficiaries. These data were collected at baseline (2017), mid-term (2019) and will again be collected at end-line (2021). At baseline, we have collected data from an extended sample of beneficiaries and additionally collected comparison group data for 8 commodities in 7 countries. The case selection took place per region, focusing on the most representative commodity programme per region. The final sample size for target and comparison groups for these 8 cases was determined based on the effect size anticipated for changes in income per ha from 2017 to 2021. It makes sense to have these cases as part of the mix for external evaluations, after an evaluability study by the evaluator to confirm the suitability of the baseline data. We have comparison group data for rice (DRC, Mali, Indonesia), coffee (DRC, Peru), vegetables (Vietnam; Tanzania (highest probability to drop out > unreliable data) and cocoa (Honduras).
  • At farmer organisation level we systematically collect data via SCOPEinsight to measure developments in the FOs’ business and organisational capacities. These assessments are being carried out every 18-24 months and are further complemented by country-specific indicators for more regular monitoring (usually annual periodicity).
  • At institutional level we have no common monitoring framework within Rikolto, but only country-specific monitoring data, which is updated on a yearly basis.
  • Regarding the Covid-19-specific research questions we are foreseeing major data gaps. Its’ evaluability shall therefore be jointly assessed with the consultant during the inception phase.

During the inception phase, the consultant is expected to create a list with additional data requirements based on the review of the data and documents made available to them. It shall be the responsibility of the consultant’s team to collect the additional data for the in-depth case studies subject to external evaluation. Should there be data gaps identified for a reliable internal assessment, Rikolto staff will be responsible for additional data collection.

Sources to be consulted for the evaluation:

3.2 Assessment methodology

The chosen methodology for the assessment shall comply with the following principles:

  • Mix of field & desk evaluation: evaluations with in-country field visits shall only be conducted for outcomes with poor performance or with high learning potential. In the case of such field visits being identified as necessary they shall be conducted by the local consultant, which is part of the overall evaluation team. The remainder of outcomes shall be evaluated via desk study based on Rikolto’s M&E data.
  • Mix of internal and external evaluation: due to budgetary limitations the external consultant shall focus on assessing one case in depth per country, which shall be complemented by an internal evaluation of the remaining interventions contributing to the same outcome and which shall be supervised by the external evaluator. The external evaluator is responsible for the methodological setup of both the external and internal evaluation framework.
  • Participatory approach for learning and strategy development: As mentioned earlier, next to the accountability needs towards DGD and partners, Rikolto wants to seize this evaluation as an opportunity for learning and improving its intervention strategies for the future. We therefore would like to see a participatory and co-creative approach applied to this evaluation, which involves Rikolto staff and strengthens their critical reflection and strategic thinking skills alike. Apart from that we would also like to see other key stakeholders, like implementing partners, involved in the process to foster joint learning, planning and ownership.

4. Expected results

The following results are expected as part of the assignment:

  1. Evaluability study of the proposed in-depth case studies (taking into account data availability and quality of different sources mentioned above), the prospective research questions for each cluster and the research question specific to the Covid-19 pandemic.
  2. Inception workshop to determine final research focus, questions, and cases per cluster.
  3. Inception report detailing the outline of evaluation methodology for: External, in-depth case study evaluations; internal evaluations complementing the in-depth case studies for the overall outcome evaluation; overall outcome evaluation at country level; prospective evaluations per cluster; evaluation specific to the Covid-19 pandemic.
  4. Inception meetings of local evaluation teams with local Rikolto teams to gain insight in local context and implementation dynamics, discuss and operationalise the evaluation methodology and finetune list of stakeholders to be consulted during the evaluation and identify additional data needs.
  5. Short inception report summarising main agreements with regards to operationalisation of the methodology, list of stakeholders to be consulted, additional data requirements and other important agreements made with the Rikolto team.
  6. Participatory workshops with participation of relevant Rikolto staff and partners at country and cluster level to make sense of the collected data and the intermediate evaluation findings
  7. 13 impact evaluation reports (1 per outcome) uniting findings from the internal and external assessments at country level, answering above outlined research questions.
  8. 4 cluster reports evaluating the prospective research questions based on the country-level findings.
  9. Sense-making with participation of relevant Rikolto staff and partners at country and cluster level to critically discuss evaluation results and translate them into strategy adjustments (modalities of process still to be agreed upon with the respective teams).

5. Roles and responsibilities

The following list of actors and their roles & responsibilities is to be finetuned during the inception phase of the evaluation, in collaboration with the consultant:

  • Rikolto’s evaluation committee (Executive Director, Inclusive Business Manager, 1 Regional Director and 2 Global PLA Advisors – point of contact: Michaela Boyen): briefing of the global organisational context; provides relevant documentation available at global level; ensures alignment of the evaluation with the organisational goals and needs; validates evaluation methodology; approves evaluation products.
  • PLA team (composed of all regional PLA focal points and 2 global PLA advisors – point of contact: Michaela Boyen): practical coordination and accompaniment of the evaluation process; first point of contact for questions, to validate evaluation process and to address bottlenecks or stumbling blocks – both internationally and at country level.
  • Regional teams (composed of Regional Director, relevant programme and support staff – point of contact: regional PLA focal point): provide context, programme briefing and available data and documentation; validation of the list of actors to be consulted and the additional data collection needs; facilitate contact with actors to be consulted; participate in the assessment at agreed-upon moments for sense-making, contextualisation etc.; conduct internal evaluations under guidance and supervision of the external evaluator; take care of additional data collection for internal assessments if applicable; validate the pertinence of the evaluation results.
  • Lead consultant: coordinate the overall evaluation, incl. the development of the methodological setup of both the external and internal evaluation frameworks; ensure alignment of local evaluation teams with the evaluation method, comprehension of the evaluation task and direct line of methodological problem-solving; main spokesperson throughout the evaluation; produce cluster evaluation reports addressing the prospective research questions; accountable for all evaluation products; ensure timely implementation of all steps, allowing for adequate participation of involved Rikolto staff and partners during the sense-making process and validation of results.
  • Local evaluation team (composed of local evaluator(s) and support researchers if needed for additional data collection): discuss and operationalise evaluation framework at country level with local Rikolto team; conduct the evaluation of the in-depth case study selected for the country-level outcome; responsible for additional data collection, triangulation and sense-making for the assigned case, and any other evaluation duties as agreed upon with the lead consultant; guidance and supervision of the internal evaluation conducted by the regional teams, incl. validation of the final assessments and bear responsibility for the evaluation process and outcomes; write a general assessment for the whole outcome (=impact evaluation report per country) based on the external and internal assessments.

6. Budget and timeline

  • The total budget for this end-line impact evaluation is 160.000 Euros.
  • Inception phase 1st semester of 2021: Given the uncertainty of how COVID-19 will affect our work in the coming year and a half, we would like to spread out the evaluation process, starting with the inception phase in the first semester of 2021 already. This is to foresee enough time throughout all stages of the evaluation, including to allow for a proper review of the draft report, for a participatory process to formulate recommendations and for a proper restitution by the evaluator. Travel conditions permitting, we aim to combine the inception meeting with a physical meeting of Rikolto’s PLA team and the Inclusive Business working group, planned for May 2021. We planned to dedicate 1-2 days to operationalising the end-line evaluation and establishing collaboration mechanisms between the Rikolto team and that of the evaluators. Starting the inception phase already early in 2021, also offers the opportunity to connect some aspects of this phase with the proposal writing process of the new DGD programme (e.g. stakeholder mapping, power analysis, …), thus maximizing the gains from these moments.
  • Data collection starting from July
  • Final reports ready 31st Dec 2021

7. Required profile

We are looking for a team of consultants that has:

  • Expertise with impact assessments, preferably in food systems (min. 5 years)
  • Solid understanding of change dynamics in food systems, upscaling dynamics, policy environment, agricultural value chains and farmer organisation capacity development (min. 5 years)
  • Capacity to provide critical-constructive feedback and transmit methodological improvements
  • Ability to write clear, realistic and relevant recommendations
  • Professional fluency in English; at least a good passive dominion of French and Spanish
  • Methodological affinity with Rikolto’s impact assessment framework

8. Tenders

8.1 Information to be included in the tender

Tenderers' attention is drawn to the general principles of Articles 4, 5, 6, 7 and 11 of the Law of 17 June 2016, which apply to this placement procedure. The tender and the annexes to the tender form shall be drawn up in English. The tenderer shall clearly indicate in his tender which information is confidential and/or relates to technical or commercial secrets and must not therefore be divulged by the contracting authority.

The following information shall be included in the tender:

  • proposed methodology and implementation timeline (max. 5 pages);
  • proposed team of local consultants (1 per country): This proposition shall be subject to validation of the regional offices. Should the lead consultant have no leads on suitable local consultants, the Rikolto can suggest candidates or facilitate contacts to identify this local consultant.
  • relevant experience of the lead and local evaluators in relation to the described project;
  • estimated working days for each of the consultancy’s aspects, disaggregated for the lead and local evaluators;
  • daily fee rate and total amount of fees (incl. VAT) for the lead evaluator;
  • estimated evaluation costs for the local evaluation team;
  • the total amount of the offer (excl. VAT);
  • the amount of VAT;
  • the total amount of the offer (incl. VAT);
  • the signature of the submission report of the authorised or mandated person(s), as the case may be, to bind the tenderer;
  • the capacity of the person or persons, as the case may be, signing the offer(s); and
  • the full registration number of the subscriber with the Crossroads Bank for Enterprises (for Belgian subscribers).

8.2 Prices

This is an order at a global price which means that the global price is fixed. The contractor shall be deemed to have included in his unit price all possible costs weighing on the services, with the exception of VAT.

The exact prices of those parts of the assignment that cover local work on the selected cases, shall be fixed upon agreeing on the case selection and upon validating the local consultants by the regional offices. The local work should fit within the global available budget.

8.3 Period of validity of the tender

Tenderers shall remain bound by their tenders for a period of 60 calendar days commencing the day after the final date for the receipt of tenders.

9. Application procedure

Because of the budget of the assignment, Rikolto is obliged to publish this call for tender both on the European and Belgian e-procurement platforms. You can find the respective links here:

All communication and information exchange between Rikolto and potential applicants, including electronic submission and receipt of tenders, should be carried out using electronic means of communication at all stages of the procurement procedure. For this we will use the Belgian e-procurement platform exclusively. Tenders must reach Rikolto via this platform by the 16th of August 2020 at 23:59 CET and include the filled European Single Procurement Document (ESPD) for this assignment (see below), which guarantees compliance with the conditions laid down in Article 14, §6 and §7 of the Act of 17 June 2016. A tender sent by e-mail does not meet these conditions and will therefore be disregarded for the selection process.