moneval_experts_2012_worldcafe_MandE

= Africa RISING M&E Expert Meeting = Back to the event agenda
 * 5-7 September 2012 **
 * Large auditorium, ** ** ILRI Ethiopia, Addis Ababa **

World cafe discussion on M&E:
The three questions were:
 * What are the most important questions we need to ask ourselves and how can we use them to learn and inform management? (Gary)
 * What methods do we need to answer these questions? (Carlo)
 * What are functions / roles we need about monitoring and evaluation and how should we split them (Sibiry)

What are the most important questions we need to ask ourselves and how can we use them to learn and inform management? (Gary)
Nice overlap between the three different questions. This was quick and I tried to categorize the questions and put a star for unique issues.
 * Each group came up with high level questions:
 * What is the contribution of SI to poverty alleviation ?
 * Did we increase productivity?
 * Scale: Are our approaches replicable?
 * Technologies and packages of technology: how do they complement each other? Are there synergies between them?
 * Who are we helping etc. hh types, landscapes etc.?
 * Process: Which processes and institutions were helpful? How participatory/top down was it?
 * Tradeoffs: the farming system, the environment
 * Adoption - demonstrated which farm is adopted
 * Partnerships: Can we use existing IPs? Did we improve the efficiciency/effectiveness of different partners?
 * Learning: What did we researchers learn and what did we learn about farmer learning?
 * How abut women labor use?
 * How does intervention complement each other?
 * Variation and targets (we we are helping, the different domains, household)?
 * Sustainability ? for household
 * USAID favorite question, what is worked out and what didn’t work ?
 * Did Africa rising make contribution in multiple project in the larger context, did it fill the large gap in farming system ?
 * Collectiveness of information to fill the data collection efficiency ?
 * Which pathways we provide ?
 * Sequencing? To talk about farmers adopting in different sequencing, which sequencing is best to achieve impact?
 * Which advantage do we have on community level?


 * Unique questions / categories**:


 * What worked and what didn’t work?
 * Did AfR make a contribution in the context of multiple projects?
 * Did AfR bridge the gap among commodity projects?
 * Did we collect enough to complement existing data?
 * Which pathways did we provide?
 * Sequencing: farmers adopting in various sequences etc. What is the best sequence to achieve impact?
 * What are technology advantages for certain groups or others?


 * What** **to do next about** **this**:


 * Organizing this to be written down and organized in groups to share with people
 * Next steps after this is prioritization.
 * Then we need to pair questions with methods etc and a back-and-forth between implementers and evaluation team.

What methods do we need to answer these questions? (Carlo)
__Group 2__: > __Group 3__:
 * For monitoring, each project has milestones and indicators. The indicators will assess milestones.
 * For evaluation: qualitative surveys, baseline and follow-up surveys.
 * We need to discuss the differences between implementing CG centres and IFPRI. We (IFPRI) suggesting controls in other villages to encourage spillovers but that might be different to others. IFPRI might be in charge of collecting data and implementing CG centres too. Efforts need to be coordinated.
 * Implementers will be administering the project
 * M&E methods (no distinction between M and E): forms and surveys to be standardized across the 3 megasites, at household level, for input markets, food markets and
 * 3 surveys
 * Monitor environmental and land use changes, using satellite imagery (hi res)
 * Management techniques, including soil management.
 * Central database to collect and capture all data
 * RCTs ex-post and ex-ante but participation fatigue from farmers so minimize the surveys...
 * We didn't refer evaluation a lot but we have discussed RCTs, ex-post and ex-ante (why we need ex-ante? you can use it for predicating and screening, with simulation models, it shouldn’t be before the project starts)
 * We should limit materials

Q: Why are we suggesting ex-ante evaluations? There is no time for ex-ante any more. A: By ex-ante we're just talking about evaluations going on during the project, not before. It's not about actual measurements.

__Group 1__:
 * Monitoring tools - quantitative approaches and strategies (researchers evaluating themselves)
 * Surveys and discussions with stakeholders, meetings and observations by beneficiaries and researchers
 * Reports about what needs to be done, what's happening where and how
 * Keeping track of project activities
 * Outcome mapping and dynamic monitoring e.g. progress markers
 * Quantitative methods:
 * before and after
 * with or without intervention
 * RCTs

What are functions / roles we need about monitoring and evaluation and how should we split them amongst us - who should do what? (Sibiry)


 * Monitoring**:
 * IFPRI backstorps output monitoring:
 * Provides efficient recipes (PMT etc.) to collate, process, ?? (maps etc.)
 * Defines standardized indicators and collects procedures
 * Backstops on sampling design (@MegaSites research planning workshops) and data sharing e.g. technology performance, biophysical data

Researchers on the ground are helping but IFPRI are responsible for it all.
 * Evaluation**:
 * IFPRI et al. evaluate the outcomes
 * They design, execute BLm MT, EL surveys
 * Report on survey to MS terms (responsible for learning to take place)
 * Engage national ?? surveys ??
 * Hires staff to embed in MS terms under IITA / ILRI
 * Is responsbile for modelling / GIS work towards scaleability

Timelines will vary across mega sites. Need to stagger M&E deployment (IFPRI to coordinate).

Possible big gap: Strong coordination between IFPRI and mega site coordination teams. Key scientists from IFPRI should attend the planning workshops and that they hire and embed IFPRI staff in the megasite teams.

Q: What do you mean with IFPRI ensuring data sharing? A: It's not about IFPRI dispossessing data from scientists but about providing other data... There is no tension because we work on the same collective goal.

**‘What are M&E function and roles we need and how should we split them among us (who should do what)?’ (Sibiry)**
The following is a synthesis from the input of 3 successive groups, which organically converged on most issues. The main unknown highlighted by most participants relates to the role of IFPRI in the design and implementation of the M&E system. For simplicity and clarity, results from the discussion hereafter focus on IFPRI’s responsibilities and roles. The responsibilities and roles of other Africa RISING stakeholders (the implementing centers: IITA/ILRI and their collaborators: sister centers, NARS, FOs, NGOs, private sector, etc.) can be deducted. For example, while IFPRI (and its technical collaborators SpatialDev, MSU, ABT, etc.) plays a seminal role in the **//design//**, **//deployment//** and **//technical operation//** of the M&E system, thereafter it is typically the responsibility of mega-site implementers (IITA, ILRI, collaborating centers, NARS, FOs, NGOs, private sector, etc.) to feed the latter with **//monitoring indicators//**. Implementers (IITA, ILRI & al.) are responsible for properly feeding the M&E system with monitoring indicators || IFPRI (& technical collaborators SpatialDev, MSU, ABT, etc.) ** evaluates ** **//outcomes//** and is responsible for evaluation activities. Implementers (IITA, ILRI & al.) assist in the execution thereof. || 1. Provides efficient technical mechanisms for collation, processing, quality check and serving (mapping) of outputs monitoring indicators 2. Provides a function of ‘independent watchdog’, verifying the indicators reporting by ground research teams 3. Provides a generic set of standardized indicators and indicator collection procedures and tools at various granularity levels (from mega-site to country to district to cluster to village levels) 4. Provides backstopping on research sampling design, data flows and standardization of methods through active participation in mega-site research planning workshops 5. Provides data sharing policy, protocols and vehicles for aggregate (typically: village-level) biophysical and technology performance data || IFPRI (& al.): 1. Is responsible for the design and execution of baseline, mid-term and end-line surveys, including engagement of national statistical offices to help with survey design and administration 2. Is responsible for ensuring interactive learning takes place within and across mega-site teams through regular reporting to the latter 3. Is responsible for organizing modeling and GIS work towards scalability of Africa RISING successful interventions || - Hires three M&E officers:
 * M&E focus || Monitoring || Evaluation ||
 * Who || IFPRI (& technical collaborators SpatialDev, MSU, ABT, etc.) ** backstops ** on **//outputs monitoring.//**
 * Funding || IFPRI funds AND delegates with facilitation (co-responsibility) || IFPRI funds AND executes (lead responsibility) ||
 * Functions || IFPRI (&al.):
 * M&E human resources |||| IFPRI:
 * o to be embedded in each of mega-site research teams
 * o tasked with facilitation of outputs monitoring indicators collection, transfer and serving back to research teams for accelerated learning and efficiency
 * o tasked with end-to-end organization of baseline, mid-term and end-line surveys, data
 * o etc. (see TORs provided by IFPRI) ||

About methods and approaches
We will make use of PMT.

What M&E approaches and methods do we need:


 * Monitoring**


 * 1) Reports
 * 2) Dynamic monitoring -keeping track of project activities (what has been done, what has happened, where?)-
 * 3) Outcome mapping -progress markers-
 * 4) Forms and surveys need to be standardized across the 3 sites
 * 5) Surveys at various levels: household, village, input markets, food markets.
 * 6) Monitor environmental and land-use changes –possibly through the use of high-resolution satellite imagery-
 * 7) Water and soil management techniques
 * 8) Central database as data repository
 * 9) Monitoring is automatic, as each activity is associated to some FtF or custom indicators


 * Evaluation**

//Quantitative methods// //Qualitative methods//
 * 1) Instrument: Quantitative surveys (baseline –midpoint –end line) on both treated and control farmers
 * 2) Methods: Diff-in-Diff and RCTs -taking advantage of local knowledge-. Control farmers should be the same for implementers and IFPRI
 * 3) Ex-ante and ex-post analysis
 * 4) Possible trade-off: respondent’s fatigue (AR should minimize the burden of surveys to farmers)
 * 1) Surveys/discussion with stakeholders, both at baseline and follow-up
 * 2) Meetings and observations
 * 3) Self-assessment by beneficiaries (agrodealers, researchers)

About roles and responsibilities
We need to look at budgets - so we may not be able to accommodate all wishes. --> We could be a bit more selective about evaluation but should probably be quite systematic about monitoring.