Monitoring & Evaluation Systems
- What is M&E?
- Issues to consider
artKenya develops and implements effective web-based M&E systems using the latest technologies. We work closely with our clients for tailor-made solutions for their specific area of competence. Our M&E systems can easily be embedded either in your intranet or your website and allow for various levels of security and user authentication.
Our M&E solution is a tool for donor-funded aid projects to improve their policy/program/project measurement that specifically targets gathering, collating, analysing and disseminating policy/program/project information. It is an easy to use and flexible software tool combining various graphics and internet technologies and modeling tools such as economic analysis without stretching the organisation's capacity limitations.
The solution is quick and economical to implement on national and regional level, satisfying the needs of the national and regional management. The implementation process can be customised and quickly adapted to the institutional capacity development levels of the organisation.
Our M&E solution enhances management capacity through:
- Transparency, Accountability, Flexibility, Adaptability, Participation, Predictability, and Continuity
- Multiple management objectives
- Nationwide and grass root solution
- Organisation Data Architecture
- Organisation Data Synchronization
- Commonisation and Agreement on Performance Indicators
- The logical framework approach
Mande.net is designed to maximize the Monitoring and Evaluation process. It enables any organization to track its progress in achieving the goals and objectives it has set out. Regardless of the industry, cause or sector, it is possible to convert any plan into objectives, activities and indicators and the questions that will be used to establish the performance according to these measurements.
Every goal can be broken down into objectives. These group together the planned activities that will be used to achieve the goal. An objective states where you expect to at the end of a given period (a year or more down the line).
These get into the details of how you expect to reach your goals. An activity is planned event or undertaking that has resources assigned to it and an expected output.
An indicator is the milestone that tells you when you are on track or have reached your target. It lets you know what to ask in order to find out where you stand in terms of achieving your goals.
The questions you pose to your responders allow you to get the feedback that you need to establish whether you are meeting your targets. We have separated these because there are many instances when you need to ask more than one question to establish the correct answer.
Once the plans have been set up, users will be able to access the system and respond to the questions that are intended for them to answer. A report is logged every time a user responds to all the questions for a selected reporting period and finalizes it.
When the user has entered their answers, they can view their report online, print it out or download it as a Word document, Excel spreadsheet or PDF document. Users with administrative rights can also do this.
The information entered into these reports is stored in the database and can be used to compile summary and comparison reports for regions, organizational units or time periods. These reports can be in plain text form or can be presented in graphs and pie charts.
The system comes with in-built search capabilities that allow you to narrow down reports or questions according to the date, region, organizational unit, objective, question type, user, etc.
Mande.net offers a large variety of question types that allow you to capture virtually any response. The options aim to make it easy for the user to respond quickly and accurately while avoiding confusion.
This presents the responder with a list of choices to select from. This type of question can be set up to allow only one choice to be selected or several of the available choices. This type of question is useful when the range of responses is known and you want to weed out unwanted answers. It can also be used when you want to suggest a list of responses to the user and only allow them to add an "other" response when theirs does not fall in the list.
This type of question is ideal when the response is expected to be a number. It can be configured to allow either integer or decimal values.
When you want the responder to answer in their own words, this question type allows them to type in their response to the question. Two modes can be selected, either long text (up to 16,000 characters) or short text (limited to 200 characters) to encourage brevity.
This question type is used to prompt the responder to upload documents or spreadsheets to go along with their reports. These are uploaded to the server and can be accessed as part of the report. Apart from choosing the type of question, it is possible to include parameters that dictate whether the question has to be answered.
The type of question you select is also used to validate the responses given. The responses will be limited to either the options you provide, or numbers depending on these choices. Users will not be allowed to enter any invalid data that is not permitted for that field, e.g. text into a numeric field.
When you designate a question as mandatory, it must have a response of the allowed type before the report can be completed. Each time the user tries to finalize a report that has not been completely answered, they will be prompted to provide the missing responses.
In certain cases, a question's relevance may depend on the response to a previous question. When a question is linked to another through a dependency, it will be disabled until the required answer is selected or entered.
Regions and Organizational Units
Mande.net has been designed to work for organizations that have multiple branches or regional offices across several countries. In this regard, the web-based nature of the system makes it possible for various users across these geographic locations to access the same system and respond to the same questions without the need for installing any additional software.
Mande.net supports multiple languages which enhances its usability in many locations even when the preferred language of each user is not the same. Each user can opt to view the system in the language of their choice.
Users can also be categorized into organizational units. This governs the type of questions they can respond to and reports they can see. The planner is able to pose questions to certain users or groups of users when it is relevant to these users. This avoids situations where a user has to respond to a question that they are not in a position to answer. They can spend less time clicking or entering none or not applicable and only answer the questions that are meant for them.
This can also be done on a regional basis. You can set up your plan so that only the countries/regions that are carrying out a certain project are required to answer questions regarding that project. Other responders would not even need to know that the project is being undertaken and would not need to provide answers for it.
During the process of setting up your plans, you can also determine how frequently the users need to respond to your questions. The available frequencies are:
Users will be automatically reminded every time they log into the system that they have a report pending for the next reporting period until they finalize that report. The reminder includes a link that takes them directly to the report so they can provide the required information. It is also possible to setup the system to send out reminders by email when the reports are pending from the users.
- For end users, only a web browser and an internet connection are required to access the system.
- The core files that enable the system's functions must be hosted on a server to make them available on the internet or within an intranet.
- A domain name is required when the system is hosted on the internet. An SSL certificate is optional and is based on the level of security required.
- The system can be installed on either a Microsoft Windows or Linux operating system. • It runs equally well on Apache HTTP Server or Microsoft's Internet Information Services
- The server environment must be running PHP and MySQL.
- In all cases, it is recommended to have the latest stable versions of the software installed both on the server and for the users accessing the system. This is important in reducing the number of bugs and security risks the system will be exposed to.
- We strongly recommend allowing artKenya.net Ltd. to host the system because it has been thoroughly tested in our hosting environment. Where issues arise, we can easily intervene to resolve them and we can often preempt errors before they occur. The system cannot be guaranteed to work in the same way on different hosting platforms because the configurations and permissions each provider allows vary.
Although Mande.net is designed to meet a wide range of monitoring and evaluation requirements, there are number of customizations that likely need to be made every time it is installed:
- Language - when required, additional languages may need to be added to the system and translation done for the interface into these languages.
- Organizational units – these often vary from one organization to another, so these may need to be added/removed on a case by case basis.
- Regions and Chapters/Branches – again these are specific to each organization depending on the nature of the operation and the areas being focused on.
- Reports – these need to be tweaked to the organizations needs and can sometimes be altered on an annual basis.
"Generally it has massively improved the way we work, the standard of case recording, statistics, evidencing what we do, which has been very well received. We can produce detailed reports."
- Facilitate stakeholders to appreciate a better M&E process where technology can later be enhanced.
- Increased accountability in funding utilisation.
- Facilitate transparency throughout policy/project, which can be propagated down to the grassroots level, while providing room for adjustments.
- Indicators can be quickly incorporated and maintained.
- Can be adapted to complex information systems.
- A solution can be quickly customised by any M&E specialist.
- Promotes public and private sector good governance.
- All changes can be applied to artKenya's M&E by end users and stakeholders.
- All measurement tools can be created by policy makers and distributed for regional self-monitoring.
Results-based monitoring and evaluation (M&E) is a powerful management tool that is used to help stakeholders track progress and demonstrate the impact of their projects or programs. While results-based M&E tracks basic short-term program outputs, it puts an especially strong focus on assessing the program’s medium- and long-term outcomes and impact.
Monitoring is a routine process that is used to determine the extent to which a project or program has been effectively implemented at different levels, in time and at what cost.
Evaluation is a periodic process designed to determine the value of a specific project or program. It links particular outputs or outcomes directly to interventions.
In recent years, an increasing number of donors have been focusing on assessing results rather than just monitoring progress on planned activities. As a result of analysis done within many NGO networks, they decided to adopt a results-oriented approach to its work in order to keep track of progress on its strategic programs and the corresponding outcomes and impacts, as well as to meet the increasingly rigorous requirements of their various donors and partners. In fact, NGO’s Strategic Plans call for the institutionalization of Monitoring & Evaluation across their networks in order to help satisfy the information needs of the various stakeholders, including the Executive Committee, the Donors, the National Chapters, the Regional Secretariat, and other strategic partners.
Given the wide geographic distribution and linguistic diversity of many NGO’s, they choose to implement an online M&E system that can be accessed through the internet by all staff, especially those at the National Chapter level, in both English and French (or other languages).
artKenya have made our M&E system easy to use and have designed it to flexibly meet the diverse needs of the verious stakeholders.
Some of the reasons why monitoring is done include:
- ensuring that implementation is moving according to plans and identify areas needing further support for NGOs/CBOs;
- improving the quality of routine work at the local level;
- to provide baseline information for evaluations;
- to feed into project planning and development.
The purposes of evaluation include:
- demonstrating the value of NGO work to outsiders;
- comparing different project approaches for future programme development;
- contributing to lessons learned;
- for policy and advocacy.
Both monitoring and evaluation are management tools. In the case of monitoring, information is routinely gathered for tracking progress according to previously agreed plans and schedules. Evaluation is more episodic than monitoring. It is facilitated by monitoring but utilises additional sources of information. Many such sources are identified during project reviews when there is a need to understand why inputs did not lead to planned outputs or what the impact of a programme has been. More details are provided in evaluations.
M&E is sometimes regarded as a donor-driven requirement, with little or no use for the NSP or NGO/CBO. This view may lead to unreliable and unsystematic reporting. Beyond accountability to donors, there are many good reasons for undertaking M&E. The reasons for undertaking M&E may differ between NGOs/CBOs and the NSP, but these reasons should be clear to all stakeholders.
As mentioned above, monitoring and evaluation play key roles in programme management. Easily accessible M&E data can provide a solid basis for project reviews and can be useful in identifying problems. M&E data can also be used to highlight potential solutions to problems and justify changes to workplans and budgets. Tracking progress using M&E data can also assist managers in identifying areas for technical support or capacity building, both among staff and NGO partners. Regular feedback of monitoring results can be encouraging both to NGO partners and NSP staff. M&E also provides useful opportunities for staff and stakeholder participation.
- The reasons for undertaking M&E may differ across organisations and between NGOs/CBOs and NSPs, so it is helpful to establish common understanding amongst programme staff and all other stakeholders about the priorities in M&E.
- The priorities in M&E need to be matched with organisational capacity and human and financial resources. M&E activities should aim to meet information requirements without overloading the staff involved and using up a lot of resources. Ideally, the budget for M&E activities should be less than 10% of the overall programme budget.
- A participatory approach to M&E requires that a range of stakeholders are involved. Stakeholders are those with a ‘stake’ or interest in the outcome of the project and may include staff responsible for collecting and analysing monitoring data, those implementing the project, the beneficiaries or project participants, and those providing financial support or technical support.
- Ideally, the development of M&E systems should take place alongside organisational level planning; in other words, at the planning or re-planning stages of the project cycle. This ensures that M&E procedures stay relevant to programme implementation, and avoids the common pitfall of developing M&E as an afterthought.