In our previous article, we discussed Evaluation Designs in Monitoring & Evaluation. Today we look at the process of evaluation. An evaluation is the systematic assessment of an ongoing or finished project, policy, or program, its design, implementation, and outcome. An evaluation aims to determine the significance and fulfillment of objectives, development effectiveness, efficiency, result, and sustainability. Here’s a breakdown of the whole process, step by step.

Preparation

1. Planning & Budgeting for the Evaluation

The evaluation plan is part of project planning and budgeting. The duration of the evaluation process should be considered and its commencement scheduled as early as the project planning stage to ensure enough time for preparation, realization, and financing. This is done on application/planning of the project or as soon as the need is identified. It takes approximately one week. To plan and budget for an evaluation;

• Review the donors’ stipulations on evaluations.
• Secure funding and fix the budget.
• Define the objectives and form broadly.
• Specify time points considering the framework conditions (access to project region, availability of target groups and employees, etc.)

The aim of drawing up a specific plan and making sufficient provisions in the budget is to ensure that the evaluation can be done at the right time. It also ensures that adequate methods are used, the evaluation is conducted within the defined scope, and the questions asked are answered. The process of drawing up the ToR should begin in good time so that the evaluation can be completed within the time scheduled. The planning phase encompasses a provisional budget before the project starts. This plan needs subsequent modification during the process of drawing up the Terms of Reference (ToR) or if the conditions of the framework change, evaluator teams are recruited, additional questions arise, etc.

2. Drawing up the Terms of Reference (ToR)

Terms of Reference (ToR) is a description of the performance to be done for an evaluation. According to international quality standards, evaluations should assess the respective subject matter under the five DAC criteria (relevance, effectiveness, efficiency, sustainability, and impact). It is for this reason that the formulation of questions relating to all five criteria is helpful in the ToR. However, additional questions may also be formulated.

A ToR provides evaluators with a reference framework and is attached and enclosed to the contract. ToR should be drawn by the commissioning organization as it consults with other stakeholders. This step is done at least ten weeks before the planned field phase and it takes approximately 4-6 weeks to finish.  To draw up the ToR;

• Identify the stakeholders and parties interested and their inclusion throughout the evaluation.
• Prepare the ToR, potentially with other stakeholders.

3. Obtaining Offers

The Terms of Reference create the basis on which potential evaluators can submit an offer. Obtaining offers ensures that both financial estimates and methodological approaches are compared. Offers from external evaluators often include a financial estimate, a content-related part, and their CVs that depict their prior knowledge, to ensure their independence is also assessed. Obtaining offers should be done after the ToR has been drawn up, to establish the qualifications needed to conduct the evaluation. It is done for approximately 4 weeks (in consideration of the submission period).

Several offers for an evaluation should always be obtained. As a rule, a minimum of three offers should be obtained on basis of the ToR. Similarly, the evaluators should be unbiased and independent, having the requisite technical expertise. Often, offers from external evaluators open up different ways of perceiving things, raise new questions, or suggest different methodologies. To obtain offers, distribute the Terms of Reference to known evaluators through networks and Internet forums.

4. Selecting Evaluators

This is done immediately after the submission period ends and it takes a minimum of one week. When selecting evaluators, the process should be transparent and participatory, following the pre-defined criteria. CVs should be handled in compliance with provisions for data protection. On receiving the offers, the external evaluators are selected in a participatory order to account for the full range of perspectives at this stage as well. It is imperative to set down criteria before the actual selection that can serve as a basis for assessing the offers and selection evaluators. To guarantee transparency in awarding of the contract, there must be documentation of the assessment and the reasons for either a selection or a rejection. This also ensures that evaluators get feedback on their offer on request and also ascertains accountability to auditors. To select evaluators;

• Review the received offers.
• Select the evaluators, potentially with other stakeholders.
• Document your decision and justify it.

5. Concluding the Contract

The conclusion of the contract is done after the evaluators are selected and it takes one day to two weeks. The ToR and offer should be appended as scheduled to the contract. The contract should bindingly show what each party should deliver, how, and by when. The contract is a legally binding document which by signing, the contracting parties agree to the obligations and terms set down within it. Many organizations retain templates for the contract that simply need to be adapted to the specific assignment, which is incredibly helpful since the relevant laws of a country should be considered and specific clear legal formulations must be deployed when drafting a contract.

In the case of international contracts, if an evaluator gets contracted from abroad, the place of jurisdiction or the legal system of the contract must be specified. Both the evaluator’s offer and the Terms of Reference should be adjoined to the contract as its integral component. Provisions regarding billing that should also be considered may be attached. To conclude the contract;

• Draft the contract.
• Have all contracting parties sign the contract.

Implementation

6. Kick-off & Clarification Meeting

The kick-off and clarification meeting is a discussion between the evaluators and the commissioning organization prior to the start of the evaluation. It only takes place after the signing of the contracts as sensitive data may be handed to the evaluators. It is held to discuss in detail
the assignment, provide a chance for questions to be answered, and hand over existing information, definitely also from the target group’s representatives or other stakeholders. The evaluation’s stipulations, objectives, opportunities, and boundaries should be discussed during this meeting.

If target groups or stakeholders need to be visited, a run-up of at least two weeks before the field visit must be scheduled. This meeting takes approximately one day. To hold a kick-off & clarification meeting;

• Organize and provide documents, material, and contacts to the evaluators.
• Discuss with other stakeholders and evaluators the dates, logistical and other requisite support plus formal expectations, options, and boundaries.
• Prepare a results protocol.

It is recommended to prepare brief minutes of the meeting containing the discussed key points.

7. Inception Report

The inception report consists of the assignment concept presentation, a time frame, the evaluation methods, and potential restrictions. It is prepared by the evaluators and approved by the commissioning organization in writing. It is prepared approximately one week after the kick-off and clarification meeting and takes approximately 1-2 weeks. The inception report is a key document in an evaluation as it gives a further opportunity for the assignment to be properly understood and the evaluation to be done at a quality and intensity corresponding to the commissioning organization’s standards and, if relevant, the donor’s. The inception report is a report prepared by the evaluators in which at least;

• The assignment is presented again in detail.
• Any difficulties and challenges are laid out.
• The proposed methodology is explained.
• A detailed timetable is drafted.

8. Debriefing/ Presentation of the Results

A debriefing session is basically a short presentation and analysis of the results with the commissioning organization. A presentation of the results entails communicating the results to the commissioning organization. During a debriefing session, the evaluators get an opportunity to present their temporary results and interpretations to the stakeholders and the commissioning organization and discuss them together. They either make corrections or include important details from the discussion in the final report. Debriefing is done at the end of the field phase or before the submission of the final report and takes half a day to one day.

At the end of the field visits, the evaluators hand over the collected data and initial results to the target group. The results and recommendations are then handed over to the commissioning organization and other potential stakeholders and interested parties at a debriefing session/presentation.

The objectives of a debriefing session are;

• To notify stakeholders, the target group’s representatives, and the commissioning organization of the results.
• To provide another opportunity for rectification of incorrect information or misunderstandings.

9. Assessment of the Final Report

This is done after receipt of the draft report and it takes up to 2 weeks. During this assessment, the evaluation report draft should be commented on and incorrect factual statements amended by the evaluator. In any case, the ToR’s formal stipulations must be assessed for compliance. To assess the final report;

• Review the draft report and request any rectifications.
• Approve the final report.

The objective of assessing the final report is to review the target accuracy of descriptions and statements and to rectify them if need be. It also permits a critical analysis of the compliance of quality requirements such as evaluation criteria and standards. It should be established whether the agreed-on inception report approach has been followed and questions from the ToR answered.

Utilization

10. Dealing with the Results

This is done after receipt of the final report and it takes approximately two months to draft an implementation plan. After every evaluation, the commissioning organization must draft an implementation plan that guides on how to deal with the recommendations. Each person or group in the implementation plan with some responsibility should be included in the drawing up of the plan and the dates for the implementation should be specified.

This step is essential to any evaluation’s meaningfulness because only the implementation of realistic and acceptable recommendations guarantees the evaluation’s positive effect on learning and change. Results and recommendations should be considered when planning similar projects or follow-up projects. To deal with the results;

• Discuss with stakeholders.
• Draft an implementation plan.
• Where relevant: make the evaluation report accessible to a broad public.
• Notify interested, relevant actors of results, recommendations, and planned utilization.
• Implement and monitor the implementation plan.

If you are a Monitoring and Evaluation professional and need help with advancing your career, simply start here for the best experience. Get a 10% discount today!