File Name: pnaeb936.pdf
File Size: 332.27 KB
File Type: Application/pdf
Last Modified: 4 years
Status: Available
Last checked: 3 hours ago!
This Document Has Been Certified by a Professional
100% customizable
Language: English
We recommend downloading this file onto your computer
L-MEP Guide to Developing USAID Project Performance Management Plans (PMPs) May 27, 2011 DEVELOPED BY THE LIBERIA MONITORING AND EVALUATION PROGRAM (L – MEP)Disclaimer: Even though the content of this guide is based on requirements of USAID’s ADS203, L-MEP – and not USAID – is solely responsible for its content
ABOUT L-MEPThe overall objective of the Liberia Monitoring and Evaluation Program (L-MEP) is to designand Implement a comprehensive Monitoring and Evaluation (M&E) and PerformanceMeasurement System for USAID/Liberia so that the Mission can measure and report on theachievement of their Development Objectives. The program is also intended to assist USAID’sImplementing Partners to strengthen their M&E systems. In addition, L-MEP will work with theMinistry of Planning and Economic Affairs and key line ministries in coordinating M&Eactivities for Liberia’s National Development Plan
L-MEP intends to meet its overall objective through the following five key components: A. PERFORMANCE MONITORING AND RESULTS REPORTING B. PERFORMANCE MANAGEMENT PLAN DEVELOPMENT AND UPDATE C. FIELD MONITORING D. CAPACITY BUILDING E. EVALUATION ASSISTANCE 2 TABLE OF CONTENTSINTRODUCTION…………………………………………………………………………………………….4 1. FIRST STEPS……………………………………………………………………………………….4 2. ORGANIZING THE INITIAL PMP DEVELOPMENT SESSION………………………………..5 3. HOW TO MANAGE THE FIRST PMP PLANNING SESSION………………………………….5 4. DOCUMENTING THE INDICATORS……………………………………………………………7 5. COMPLETING OTHER ELEMENTS OF THE PMP……………………………………………..7 6. SETTING TARGETS AND BASELINES………………………………………………………....8 7. FINAL PACKAGING OF THE PMP……………………………………………………………...8 8. WHAT NEXT?..................................................................................................................................9APPENDICES……………………………………………………………………………………………….10APPENDIX 1: PMP TOOLKIT (WORKSHEETS)………………………………………………………...10 Worksheet 1: PMP Development Team Skills Matrix…………………………………………….10 Worksheet 2: PMP Development Work Plan……………………………………………………...11 Worksheet 3: Results Statement Assessment……………………………………………………..13 Worksheet 4: Performance Indicator Quality Assessment………………………………………..14APPENDIX 2: TEMPLATE & INSTRUCTIONS ONCOMPLETING THE RESULT FRAMEWORK..........................................................................................16APPENDIX 3: TEMPLATE & INSTRUCTIONS ONCOMPLETING THE PERFORMANCE INDICATOR REFERENCE SHEETS (PIRS).............................................17APPENDIX 4: TEMPLATE & INSTRUCTIONS ONCOMPLETING THE PERFORMANCE INDICATOR DATA TABLE.....................................................22APPENDIX 5: TEMPLATE & INSTRUCTIONS ONCOMPLETING CALENDAR OF PERFORMANCE MANAGEMENT TASK…………………………………23APPENDIX 6: BRAINSTORMING TECHNIQUES……………………………………………………...24APPENDIX 7: NOMINAL GROUP TECHNIQUE……………………………………………………….26APPENDIX 8: QUESTIONS TO GUIDE INDICATOR DEVELOPMENT……………………………...28APPENDIX 9: DEFINITION OF KEY TERMS…………………………………………………………..29APPENDIX 10: PMP SELF-ASSESSMENT TOOL……………………………………………………...31 3 GUIDE TO DEVELOPING A PROJECT’S PERFORMANCE MANAGEMENT PLANINTRODUCTIONThis brief guide is intended to help M&E Officers of USAID Liberia implementing partners leadthe Performance Management Plan (PMP) development process for their organization. Itassumes that the M&E Officer or another relevant staff has either attended the USAID’sManaging for Results Training conducted by L-MEP, or met with the L-MEP M&E Specialistassigned to assist with PMP development in an initial orientation
The key requirements for a PMP are: • Narrative describing the project’s Development Hypothesis • A Results Framework graphic • A Performance Indicator Data Table in matrix format documenting, among other things, baseline value, annual and life-of-project targets and disaggregation type • Performance Indicator Reference Sheets (PIRS) detailing all indicators • Calendar of Performance Management Task, a M&E Task ScheduleThe PMP should also include a narrative description of how the project’s M&E system willfunction
Developing a PMP will take a number of sessions and/or meetings and require the inputs ofmany of the project’s staff over a period of time. Strong leadership is required to ensure that thePMP development process is successful and that project staff buy into the final product
L-MEP developed this short guide to help M&E Officers of Implementing Partner organizationslead the process of developing a PMP that both meets Agency requirements and that is useful totheir organization
1. FIRST STEPS Familiarize yourself thoroughly with your Project Description by reading your project’s proposal and the agreed Project Description in your project’s agreement with USAID
Obtain the most current version of your USAID Sector Team’s PMP and Results Framework so that you understand where your project fits into the bigger picture. Review your own Results Framework
4 Obtain and review the list of indicators that are on the Team PMP against which your project is required to report from your project’s COTR/AOTR or L-MEP – these must be included in your PMP
Complete Worksheet 1: PMP Development Team Skills Matrix in order to identify project staff with all or some of the skills listed in the top row. This will ensure that your meetings have the right skill mix in order to make progress. (See Appendix 1) Complete Worksheet 2: PMP Development Work Plan (see Appendix 1) in order to plan your own work as the M&E Officer in charge of this process. Phase1 would be the set of meetings needed to develop the indicators and Phase 2 could be the PMP finalization stage
2. ORGANIZING THE INITIAL PMP DEVELOPMENT SESSION • Invite the project staff to an initial PMP Development meeting. Consider inviting your USAID COTR and L-MEP M&E Specialist
• Prepare a short introductory briefing on what a PMP is and why it is required • Make copies of Worksheet 3 (Results Framework Assessment) and Worksheet 4 (Performance Indicator Quality Assessment) on flip-chart sized paper • Prepare questions for brainstorming on flip chart sheets for each working group (See Annex 8) • Obtain flip charts, pens, paper etc
• Assign a note-taker3. HOW TO MANAGE THE FIRST PMP PLANNING SESSIONOne possible way of managing the PMP development process is outlined below
3.1 OverviewProvide a short presentation on what a PMP is, why it is important and the elements thatcomprise a complete PMP (result framework indicators, precise definitions, baselines, targets,narrative, and M&E task schedule) so that everyone is on the same page
Outline your plan to develop and submit the PMP. Project staff needs to know up-front that theywill be engaged with the process after this initial meeting
5 3.2 Results Framework ReviewReview the project’s Results Framework and its link to the USAID Team’s Results Frameworkwith a view to ensuring that everyone understands the bigger picture. Then use Worksheet 3:Results Statement Assessment (see Appendix 1) if you think that there are problems with theproject’s Results Framework. The assessment tool is used to assess each result at each level ofyour Results Framework (Project Objective, Intermediate Result and Sub-Intermediate result)
Agree any changes and assign the Chief of Party to communicate these to the project’sCOTR/AOTR. You may decide to wait until approval has been given before proceeding withIndicator Development
The key elements of the result framework (see Appendix 2) are as followed: • The Development Objective => this refers to the overall or final result that is hoped to be achieved with the assistance provided at the completion of projects/programs. This is placed at the highest level of the graphical structure of the framework
• Intermediate Result (IR) => this refers to middle level results/outcomes when achieve could lead to the highest level result being achieved. This is placed at the next level of the structure below the Assistance Objective
• Sub-Intermediate Results => this refers to lower level results that lead to intermediate results being achieved. Based on the structure of a program/project, it is very possible to have another level of sub intermediate results
• Critical Assumptions => these are conditions that missions and programs may not have control over but must hold for the development hypothesis to lead to the relevant outcome. They are always placed in a separate block of the framework
3.3 Performance Indicator Development & SelectionDivide the participants into groups according to each Intermediate Result in your project’sResults Framework. Have senior staff float between the groups. Put up the list of questions thathas been pre-documented on flip chart paper (see Appendix 8)
Invite the senior project staff members in charge of results for each Intermediate Result to clarifythe results sought for his/her Intermediate Result and associated Sub-Intermediate Results(Question 1) for his/her group and to explain what activities will be undertaken in pursuit ofthese results (Question 2). Have these documented on flip charts. Ensure that there is clarity andconsensus in the groups on these two items before moving on. Then, move the groups onto the3rd question which is about indicators
6 Have each group document all ideas for indicators for their assigned result on flip chart paper
Ideas for managing a brainstorming exercise are in Appendix 7
Have each group systematically work through Worksheet 4: Performance Indicator QualityAssessment (see Appendix 1) for each proposed indicator. They can do this as a group or theycan divide up the proposed indicators amongst a small number of sub-groups or pairs
Have each group rank the indicators from best to worst in terms of the criteria on Worksheet 5and give them some time to prepare a presentation to the Plenary
Conduct a Plenary Session at which each group reports on their best indicators. Elicit feedbackfrom the other members. Identify any gaps. Identify some indicators for the highest level of yourproject’s Results Framework
This is probably a good point at which to end the initial session. Communicate to staff that morework will be required to precisely define the indicators, set targets and baselines, document datacollection methodologies etc
4. DOCUMENTING INDICATORSAfter the initial meeting with staff, you can start to draft Performance Indicator ReferenceSheets (see Appendix 3), working more or less closely with relevant project staff depending onyour level of knowledge of the sector. You will find that some indicators previously selected fallaway as it becomes apparent that they are not feasible to collect. The PIRS is a comprehensivetool covering definition, data collection methodologies, data analysis techniques, roles andresponsibilities. They usually take several iterations to develop. Also draft the PIRS for themandatory indicators
Once PIRS are complete and have been reviewed internally, you can send to your L-MEP M&ESpecialist for an independent review. L-MEP staff can also provide you with guidance on datacollection methodologies
5. COMPLETING OTHER ELEMENTS OF THE PMPComplete the Performance Indicator Table (see Appendix 4) with title, description, unit ofmeasure, frequency of data collection etc. Your Administrative Assistant can help here withcutting and pasting the indicator definitions from the PIRS into the Table
Working from the project’s annual work plan and the frequency column of the PerformanceIndicator Table, complete the Calendar of Performance Management Task (see Appendix 5)
7 Remember to schedule a mid-term and end-of project evaluation. This is helpful also to theM&E Officer as it constitutes an outline of your own personal annual work plan
6. SETTING TARGETS AND BASELINESAsk project staff to complete the baseline and targets on the Performance Indicator Table. It isadvisable to do this in a meeting setting with senior project managers and financial staff presentas the magnitude of your project’s achievements is closely related to its resources. Targets mayhave already been set in your project’s annual work plan – in which case, review with programand senior staff to see if they are still valid. Your L-MEP M&E Specialist can provide you tip ontargets and baselines setting. Be sure to document the rationale or reason why a target was set onthe PIRS in the section on Notes on Baselines & Targets. This provides everyone with a recordof your project’s thinking at the time the target was set
If baselines are not yet known, devise and document a plan to collect them. Many baselines willbe zero e.g. number of people trained whereas others will require more complex data collectionmethods e.g. a survey to determine pre-project household income levels
7. FINAL PACKAGING OF THE PMP Draft the narrative section of the PMP which outlines your project’s development hypothesis, your reasons for selecting the indicators, plans for monitoring, data collection and evaluation
When finalizing the narrative describing the development hypothesis you should: 1. Articulate causality or a Causal Logic – a way to demonstrate your theory of how the activities delivered by your project lead to the objective (“change in the condition of the customer”). Clearly explain the linkages between your result framework and the USAID result to which your project is linked, detailing how achieving your results help accomplish the USAID result
2. Apply the “Necessary and Sufficient Rule” – Results within the results framework should be necessary to achieving the objective; an indicator should also be necessary to tracking the results it is intended to measure. Additionally, the set of intermediate results or sub-intermediate results should be sufficient for achieving the result to which they are linked. Similarly, the set of indicators selected for a given result should be adequate to ensure that progress toward the result is sufficiently captured
This does not mean that one intermediate result or one indicator is not sufficient for a given result, but such intermediate result or indicator should be capable of performing its functions
8 3. Define Critical Assumptions – a general condition under which the development hypothesis or strategy for achieving the objective will hold true. Critical assumptions are not results; they are outside the control or influence of USAID and its partners, but they reflect conditions likely to affect the achievement of results in the result framework
Assemble the whole package – Narrative, Results Framework, Performance Indicator Table, PIRS, and M&E Task Schedule - and ask L-MEP to review once any internal reviews have been performed
Have your Chief of Party submit to the project’s COTR/AOTR for approval
8. WHAT NEXT? Once you have submitted the PMP for approval, you can start working on data collection tools for each indicator. These will vary according to the indicator. For example, it is a good idea to have a standardized training participation sheet that collects the sex of the training participant as USAID usually requires people-level data to be disaggregated by sex
One standardized form used across your project by whoever conducts training will make your work aggregating the data easier
Consider providing sub-grantees and other partners with data collection tools to ensure that data are captured exactly as you want them to be. Also work with your project staff to devise suitable data collection tools that they are comfortable using
Ensure baseline data are collected as soon as possible. L-MEP staff can review Scopes of Work for consultants if you decide to contract out this task
Plan for data aggregation by developing a series of Excel spreadsheets to capture simple indicator data such as people trained, project beneficiaries, commodities sold etc. It is advisable to roll up data as soon as it is available in time for your project’s quarterly reports
Consider preparing periodic internal performance data reviews for project management and project staff in which you calculate % performance achieved to-date against the annual target. This alerts everyone to areas of under-performance vis-à-vis the target and areas of over-performance which is sometimes indicative of poor planning, a more favorable operating environment than anticipated, more funds etc
Consider reviewing the templates for Success Stories on the USAID website and encouraging project staff to collect qualitative data in one of these recommended formats
9 Consider developing a collection of captioned photos for use in your project’s reports or its website. Captions should include: Photographer’s Name, date, location, event
APPENDICES:APPENDIX 1: PMP Toolkit (worksheets)Worksheet 1: PMP Development Team Skills MatrixList all of the proposed team members of the PMP development team and their proposed roles
Check-mark each skill that the team members have. Use this worksheet to ensure that you havea good cross-section of skills represented on the team
processes, culture methods and best Has facilitation, MFR approach Knows USAID Knows USAID report writing analytical and conditions in- Knows local Knows PM Name Role training or experience Has sector Has sector structure, education practices depth skills 1
10 5Worksheet 2: PMP Development Work planUse this worksheet to list all of the major tasks and sub-tasks needed to prepare the PMP
Expand the worksheet by including additional rows in the table as needed. Another approachwould be to use Microsoft Project to develop the work plan if someone on the team is familiarwith it
START LEVEL OF DESRIPTION END DATE STAFF DATE EFFORT Phase 1: Task 1: Sub-task 1: Sub-task 2: Task 2: Sub-task 1: Sub-task 2: Phase 2: Task 1: Sub-task 1: 11 Sub-task 2:Task 2:Sub-task 1:Sub-task 2: 12
5 Obtain and review the list of indicators that are on the Team PMP against which your project is required to report from your project’s COTR/AOTR or L-MEP – these must be included in your …
Less frequently, a project's focus could be a DO, for a small program, or, for a large program, a project's focus might be on a sub-IR. Project design, implementation planning and oversight and project monitoring and evaluation are USAID staff responsibilities.
The MEL Plans contribute to the effectiveness of the CDCS-level Performance Management Plan (PMP), as well as the project itself, by assuring that comparable data will be collected on a regular and timely basis. A Project MEL Plan is technically an annex to a Project Appraisal Document (PAD).
Two sections of this MEL Handbook will likely be as useful for teams designing trade project MEL plans as they will be for project designers focusing on the status of a country's business enabling environment.