NGO support toolkit (English)  >  Monitoring and evaluation  >  Evaluations 

Designing an evaluation

Whether process or outcome, the steps involved in designing an evaluation are broadly similar. Clearly defined questions are the first step in ensuring that an evaluation remains focused. It is sometimes helpful to identify broad objectives for the evaluation and use these objectives as a basis for more specific research questions. Often questions are generated directly from issues identified during programme implementation. Stakeholder participation in generating and refining questions should lead to results that can be owned by a broad range of actors.

There is a range of methods commonly used for answering questions in the evaluation. Quantitative methods (which answer ‘how many?’ questions) include questionnaire surveys and checklists. Qualitative methods (which focus on ‘how?’, ‘what?’, and ‘why?’ questions) include focus groups, semi-structured interviews and observation. Each method has strengths and weaknesses; the challenge is to identify the most appropriate method for the question. An evaluation question such as, ‘Did the training programme lead to increased capacity among our partner NGOs?’ might be answered through interviews with staff and focused discussions, as well as checklists. There may be ‘tried and tested’ tools for measuring capacity that can be adapted. Throughout, it is important to clearly define the terms used within the evaluation. In this example, ‘increased capacity’ should be broken down into specific measurable components (for instance ‘organisational strength’, ‘HIV/AIDS technical capacity’).

Consider who needs to be included as study participants and how they should be selected. The evaluation questions and choice of methods should guide these decisions. Quantitative methods generally involve a larger number of randomly selected respondents, whereas qualitative methods generally rely on a smaller number of respondents, selected for specific reasons (such as their involvement in an activity).

Assigning responsibilities, drafting a work plan and devising a budget for the evaluation are essential next steps. It may be that the design will have to be scaled down to fit with resource constraints (though ideally these constraints will have been borne in mind during the design phase). Consultant-led evaluations can be expensive, but may be useful where organisations do not have the time or resources to undertake the work themselves. It is often felt that an ‘outsider’ will be more objective, although this may depend how the consultant is being paid! In general, consultants are hired for outcome/impact evaluations, while process evaluations are often undertaken internally. Whether internally or externally led, a clear terms of reference, detailing expected outputs, is essential in avoiding disappointment with the final results. Good evaluations should make evidence-based and realistic recommendations for improved practice.

• Send to a friend
• Printer-friendly