Terminal Evaluation of Jatropha farming Project

TERMS OF REFERENCE

in Malindi and Magarini Districts, Coast Province, Kenya

___________________________________________________________________________

1.0  BACK GROUND INFORMATION

Comitato Internazionale per lo Sviloppo dei Popoli (CISP) in partnership with Kenya Forest Research Institute (KEFRI) and associate with the Ministry of Agriculture (MoA) is implementing a project in Malindi and Magarini Districts (Kilifi County) titled “Support to and Expansion of Malindi Bio-fuel Cluster-Jatropha-Farming”. This is a four-year project funded by the European Commission through the European Development Fund. CISP is the lead implementing organization executing all project activities while KEFRI and MoA are involved in research and monitoring activities respectfully. The project action-plan was informed by a brainstorming discussion between, KEFRI, CISP and the District Agricultural Officer (DOA) of Malindi on the possibilities of implementing a project on sustainable Jatropha farming. The project was incepted in June 2011 and is expected to end in June 2015.

The project targeting 1200 farmers within the mentioned areas is designed to improve self sufficiency and increase sustainability of local communities in terms of energy supply.

It is within the project a midterm evaluation is foreseen. This mid-term evaluation will cover Jatropha activities for the period June 2011 to December 2013. The rationale of the mid-term evaluation (MTE) is to enable CISP and project stakeholders to:

(a) Assess the continued relevance of project objectives and planned outcomes, including the implicit and explicit assumptions and risks of the project.

(b) Assess the outputs produced and outcomes achieved as compared to those planned and to verify prospects for development impact and sustainability.

(c) Assess the efficiency of implementation: quantity, quality, cost and timeliness of inputs and activities.

(d) Provide an analytical basis and recommendations for the continuation of the project.

(e) Draw lessons of wider application for the replication of the experience gained in other projects.

	Intervention logic
Overall objective	To promote the oil tree Jatropha as a renewable, sustainable and decentralized source of biofuel for cooking and lighting for small scale farming communities in Malindi and Magarini Districts on the Kenyan North Coast.
Specific objective/ outcomes	To substitute the use of kerosene and wood fuel in households, SMEs and hotels within the project area and beyond by introducing the Jatropha value chain To enable farmers to increase their income by building their capacity in agri-business and marketing. To establish a production, supply and maintenance chain for the mechanically produced and manually operated low-tech biomass conversion technologies (e.g. seed crusher, oil expeller, briquette press, etc.) as well as energy end-use appliances (e.g. socially acceptable plant oil lamps) by training the informal sector on producing the equipment for the local market.
Expected results	1) Production: 1200 farmers have adopted the Jatropha as hedge plant (farmland) and buffer zone (forest) and 160 selected farmers and 20 government extension workers are trained as trainer on FaaB. 2) Processing: 1200 farmers produce crude Jatropha oil (CJO) and Jatropha seed cake briquettes (JSCB) for local consumption and sale and 30 artisans are trained ad trainees on how to manufacture energy end-use appliance. 3) Usage/marketing: At least 1200 households have access to CJO for domestic lighting and JSCB for cooking. This will have the following positive impact on the target groups/beneficiaries: A)    Energy Security: Target group will substitute the use of kerosene with CJO and firewood with the use of JSCB. B)     Poverty reduction: Target group would save the money spent on kerosene for lighting and wood fuel for cooking. Additionally they would generate an income from the surplus of CJO and JSCB. C)     Food security: The dense Jatropha hedge will help increase on-farm productivity by protecting farmland from erosion and browsing animals. Protection of natural resources: The dense Jatropha buffer zone at the Arabuko Sokoke forest will reduce pressure on the forest which serves as firewood and charcoal.
	Activities 1) Production: cultivation of Jatropha in form of hedges around farmland and as a buffer zone around identified areas of the Arabuko Sokoke Forest; Training of farmers and government extension workers in agri-business (Farming as a Business- FaaB) to increase income of farm activities. 2) Processing: introduction of locally adapted biomass conversion technologies; training of the informal sector in manufacturing low-tech biomass conversion technology and energy end-use appliance. 3) Usage/marketing: introduction of socially accepted energy end-use appliances, marketing strategies for the sale of JSCB/CJO surplus.

2.0 PURPOSE THE EVALUATION

This midterm evaluation of the project is intended principally for learning and accountability purposes. The evaluation is expected to generate relevant findings, lessons learned and major recommendations which will provide the key stakeholders of the project with sufficient information to make informed judgment about the past performance of the project and inform future programming. This information will be shared with key stakeholders of the project namely. 

3.0 SCOPE OF THE EVALUATION

The evaluation will assess accomplishments of the project as detailed in project document and in the annual work plans for the period 2 years and 6 months. All the different project components and activities stated in the original log frame will be assessed.  In view of the fact that the project is on-going, the evaluation will also cover the links between the different phases of the project.

Standard criteria like relevance, efficiency, effectiveness, sustainability and impact should be used to structure the evaluation.

4.0 METHODOLOGY

The  evaluation  will  be  undertaken  through  a  combination  of  qualitative  and  quantitative approaches. Data will be collected from main stakeholders of  the project that include farmers, project coordinator, officers, Kenya Forestry Research Institute (KEFRI), Kenyan Agricultural Research Institute, Ministry of Energy, Ministry of Agriculture, Practical Action, District agricultural officers and other government bodies implementing energy project, other  actors  in  similar  projects  etc. The data and information gathered will be systematically analyzed and presented following the methodology below. The evaluation will encompass the following steps:

Document/ desk review: This will comprise review of the project documentation, including but not confined to:

• Annual work plans and progress reports
• Inception Report
• Technical project reports
• Workshops and training reports
• Strategies and policies on renewable energy and energy efficiency in Kenya
• Strategies and programme of other development cooperation agencies active in this field

·         Situational analysis and market potential of Jatropha in Malindi and Magarini District.

·         EIA Report and EIA study

Focus Group Discussions: These will be held with relevant target groups and beneficiaries. 

Semi-Structured Interview (SSI): SSI will be carried out with selected, individual male and female concerned government authorities, project staff, and partners, selected representatives of organizations active in the field of renewable energy and energy efficiency potential funders of Jatropha projects in the future etc

Observations: Careful and systematic observation regarding the process of Jatropha will be carried to capture primary information.

Key Informant Interviews: This method will be used to capture the views and professional opinions of people who know the implementation context of Jatropha project objectives and expected results in particular.

Case studies:  It is suggested that the consultants document key success stories that represent significant achievements whether in process or results of the project.

Data analysis

Data gathered using various methods and tools have to be collated, analyzed and interpreted systematically. The evaluators will be expected to come up with detailed data/information analysis methods. It would be possible to utilize social science data analysis software such as the Statistical Package for the Social Sciences

5.0 ROLES AND RESPONSIBILITIES

CISP will facilitate and coordinate staff members’ or stakeholders’ participation in the evaluation, pay the professional fee and other arising costs from the contract and provide logistical support (transport, accommodation, food, etc. during the assignment). The overall supervision of the evaluators will rest with CISP, the lead agency for the project. The consultants will report to the Project Manager Malindi and the Country Coordinator based in Nairobi. The consultants will report to the Country Coordinator (or a designated officer) based in Nairobi but the Project Manager in Malindi will provide day-to-day supervision and support to the consultant. The Country Coordinator will be responsible for overall quality assurance of the evaluation and collating feedback on the findings and recommendations from the evaluation.

6.0 DELIVERABLES

The consultants should deliver the following outputs and services;

• A technical proposal with a proposed evaluation framework/design.
• Upon signing a contract submit an inception report detailing the evaluation design, methodology and data collection tools to be discussed and agreed upon with CISP before the evaluation starts.
• Plan and over see the data collection process in consultation with CISP.
• Produce a draft report of the evaluation.
• Facilitate validation workshops in Malindi on the basis of the draft report.
• Present the findings and recommendations for further action.
• Submit a final report to CISP.  The report structure should follow the EU PCM guidelines for evaluation report structure.

The consultants should submit the final report in soft and 3 hard copies

7.0  TIME FRAME

The consultancy will run for 16 effective days in Feb 2014. The following are indicative number of days for executing the activities involved:

	Activity	Number of days	Location	Person responsible
1	Briefing	1/2		Lead consultant
2	Document review	2		Lead Consultant
3	Training of evaluation team	½		Lead consultant
6	Pre-testing and revision of evaluation tools	1		Lead consultants
7	Field work (including meetings with stakeholders)	7		Lead consultants
8	Data analysis & report	2		Lead consultants
9	Validation & debriefing meeting	1		Lead consultants
11	Revision and submission of draft reports	1		Lead consultants
13	Final revision and submission of the reports based on feedback of CISP	1		Lead consultant
		16	Lead consultant = 16 days

The following attributes are required of the consultant 

Lead Consultant  

·         Postgraduate qualification (Masters or above) in Agriculture/Agronomy/Renewable energy or a discipline relevant to this assignment.

·         Background knowledge and experience in biomass preferably in Jatropha curcas.

·         Experience in the evaluation of technical assistance project, preferably EU-funded projects, particularly those under the energy portfolio.

·         Proven capacity in working across the levels of institutions from policy, to legislation and organizations and ability to assess institutional capacity and incentives.

·         Excellent report writing skills

MODE OF APPLICATION

Applications with non returnable copies of CVs, testimonials and sample of previous work should be submitted on or before 15^th February 2014 and via email to njau@cisp-nairobi.org.

All applications should include the following;

• Cover letter (maximum 1 page)
• Technical proposal (max 8 pages):

The technical proposal should include (I) brief explanation about the Consultant with particular emphasis on previous experience in this kind of work; (ii) profile of the Consultant to be involved in undertaking the evaluation, (iii) Understanding of the TOR and the task to be accomplished, (iv) Draft evaluation frame work and plan

·        Financial Proposal: The financial proposal should provide cost estimates for services rendered including daily consultancy fees excluding: accommodation and living costs; transport cost; stationeries, and supplies needed for data collection; costs related to persons that will take part from consortium partners and government authorities during evaluation process, enrichment workshop.

•  Samples: At least two samples of previous work i.e. evaluations of similar projects.

Annex 1: Detailed Evaluation criteria

Relevance

The analysis of relevance would focus on the extent to which the design

Effectively/ appropriately:

✓ analyzed the project’s coherence with the EC’s Country Strategy Paper and National Indicative Programme

✓ analyzed the project’s coherence with the Partner Government’s development policy and sector policies

✓ identified key stakeholders and target groups (including gender analysis and analysis of vulnerable groups such as the disabled), assessed institutional capacity issues and effectively promoted local ownership

✓ clearly and accurately identified real problems

✓ analyzed lessons learned from past experience and ensured coherence with current/ongoing initiatives

✓ provided a clear analysis of strategy options and justified the recommended implementation strategy

✓ established a clear and logically coherent set of project objectives (Goal, purpose, outputs) and a set of indicative activities for delivering each project output

✓ developed a clear and useful Logframe matrix with supporting activity and resource/cost schedules

✓ analyzed assumptions/risks

✓ established appropriate management and coordination arrangements

✓ established appropriate and effective monitoring and evaluation systems

✓ provided an analysis of sustainability issues – including the financial and economic sustainability of the proposed measure, environmental impact, benefits to both women and men and the use of appropriate technology

Efficiency

The efficiency criterion concerns how well the various activities transformed the available resources into the intended outputs (sometimes referred to as results), in terms of quantity, quality and timeliness. The assessment of Efficiency would therefore focus on such issues as:-

✓ the quality of day-to-day management, for example in (i) management of the budget (including whether an inadequate budget was a factor); (ii) management of personnel, information, property, etc, (iii) whether management of risk was adequate, i.e. whether flexibility was demonstrated in response to changes in circumstances; (iv) relations/coordination

With local authorities, institutions, beneficiaries, other donors; (v) respect for deadlines.

✓ Costs and value-for-money: how far the costs of the project were justified by the benefits _ whether or not expressed in monetary terms in comparison with similar projects or known alternative approaches,

Taking account of contextual differences;

✓ Partner country contributions from local institutions and government (e.g offices, experts, reports, tax exemption, as set out in the Log Frame resource schedule), target beneficiaries and other local parties: were they provided as planned, could re-allocation of responsibilities have improved performance, were communications good?

✓ Commission HQ/Delegation inputs (e.g. procurement, training, contracting, either direct or via consultants/bureaux): key questions as for local/government inputs (above);

✓ Technical assistance: how well did it help to provide appropriate solutions and develop local capacities to define and produce results?

✓ Quality of monitoring: its existence (or not), accuracy and flexibility, and the use made of it; adequacy of baseline information;

✓ did any unplanned outputs arise from the activities?

Effectiveness

The effectiveness criterion, in LogFrame terminology, concerns how far the project’s outputs were used, and the project purpose realized. The analysis of Effectiveness would therefore focus on such issues as:

✓ Whether the planned benefits have been delivered and received, as perceived by all key stakeholders (including women and men and specific vulnerable groups such as the disabled)

✓ in institutional reform projects, whether behavioural patterns have changed in the beneficiary organisations or groups at various levels; and how far the changed institutional arrangements and characteristics have produced the planned improvements (e.g. in communications, productivity, ability to generate actions which lead to economic and social development); if the assumptions and risk assessments at results level turned out to be inadequate or invalid, or unforeseen external factors intervened, how flexibly management adapted to ensure that the results would still achieve the purpose; and how well it was supported in this by key stakeholders including Government, Commission (HQ and locally), etc.

✓ Whether the balance of responsibilities between the various stakeholders was appropriate, which accompanying measures were or should have been taken by the partner authorities, and with what consequences;

✓ How unplanned results may have affected the benefits received;

✓ Whether any shortcomings at this level were due to a failure to take account of cross-cutting or over-arching issues such as gender, environment and poverty during implementation.

Impact

The term impact, sometimes referred to as outcome, denotes the relationship between the project’s purpose and goal, that is the extent to which the benefits received by the target beneficiaries had a wider overall effect on larger numbers of people in the sector or region or in the country as a whole. At Impact level the analysis generally examines such aspects as:

✓ to what extent the planned goal have been achieved, and how far that was directly due to the project;

✓ in institutional reform projects, how far enhanced economic and social development resulted from improved institutional capabilities and communications;

✓ In infrastructure-type projects, how far did they also enhance economic and social development beyond the level of their immediate users?

✓ if there were unplanned impacts, how they affected the overall impact;

✓ Where appropriate, all gender-related, environmental and povertyrelated impacts were achieved; and

✓ how the economic effects were spread between economic growth, salaries and wages, foreign exchange, and budget, and how this relates to the project’s overall objectives.

Sustainability

The fifth criterion, sustainability, relates to whether the positive outcomes of the project at purpose level are likely to continue after external funding ends. An analysis of sustainability would therefore focus on such issues as:

✓ Ownership of objectives and achievements, e.g. how far all stakeholders were consulted on the objectives from the outset, and whether they agreed with them and remained in agreement throughout the duration of the project;

✓ Policy support and the responsibility of the beneficiary institutions, e.g. how far donor policy and national policy corresponded, and the effects of any policy changes; how far the relevant national, sectoral and budgetary policies and priorities affected the project positively or adversely; and the level of support from governmental, public, business

And civil society organizations.

✓ Institutional capacity, e.g. the degree of commitment of all parties involved, such as Government (e.g. through policy and budgetary support) and counterpart institutions; the extent to which the project is embedded in local institutional structures; if it involved creating a new institution, how far good relations with existing institutions were established; whether the institution appears likely to be capable of continuing the flow of benefits after the project ends (is it well-led, with adequate and trained staff, sufficient budget and equipment?); whether counterparts were properly prepared for taking over, technically, financially and managerially;

✓ The adequacy of the project budget for its purpose;

✓ Socio-cultural factors, e.g. whether the project is in tune with local perceptions of needs and of ways of producing and sharing benefits; whether it respects local power-structures, status systems and beliefs, and if it seeks to change any of those, how well-accepted are the changes both by the target group and by others; how well it was based

on an analysis of such factors, including target group/ beneficiary participation in design and implementation; and the quality of

relations between the external project staff and local communities.

✓ Financial sustainability, e.g. whether the products or services provided were affordable for the intended beneficiaries and remained so after funding ended; whether enough funds were available to cover all costs (including recurrent costs), and continued to do so after funding ended; and economic sustainability, i.e. how well the benefits (returns) compared to those on similar undertakings once market distortions are eliminated.

✓ Technical (technology) issues, e.g. whether (i) the technology, knowledge, process or service provided fits in with existing needs, culture, traditions, skills or knowledge; (ii) alternative technologies were considered, where there was a choice; and (iii) the intended beneficiaries were able to adapt to and maintain the technology acquired without further assistance.

✓ wherever relevant, cross-cutting issues such as gender equity, environmental impact and good governance; were appropriately accounted for and managed from the outset of the project