Our MERL team has developed comprehensive systems to support the implementation of key project interventions while collecting relevant data for formative analysis. Key considerations ensured that project beneficiaries were not overwhelmed with data requests, and that data collection is well-aligned to achieving strategic milestones in the midst of a global health pandemic. In an effort to mitigate severe logistical risks, 95% of the internal MERL systems for High Gear are online (electronic) based. Additionally, our programme benefits from innovative dashboarding tools such as Power Bi Dashboard Report. In turn, these data collection and reporting efficiencies are reviewed on a quarterly-basis as part of our Theory of Change (TOC) review meetings. The dashboard report is an innovation on its own and takes the posture of a progress tracker, a project management tool, and a central hub for all project data analysis occurring within the High Gear team.
We have continued this software-strengthening trend in our Project-based-learning (PBL) portfolio. As a key intervention for our TVET curricula upgrades, our MERL team undertook the development of a dynamic monitoring system to track implementation and document impact.
As a starting point, the MERL team participated in the conceptualization and design of the PBL intervention. This was followed by the identification of five project indicators whose results will be influenced by the successful implementation of PBL with TVET colleges, namely:
- # of beneficiaries receiving training, coaching or capacity building support,
- % of college lecturers that indicate that their teaching has improved as a result of High Gear support,
- # of TVET students who receive updated course content,
- #/% of TVET graduates that indicate the quality and resources of the programme was of high standard,
- and # of TVET courses aligned to industry needs.
Subsequent to this thorough and participatory process, four tools were developed to collect data against the identified indicators, namely:
Scheduling Assistant – Used to schedule mentoring sessions with lecturers for a full trimester. Mentors update the schedule daily, allowing the team the ability to track real-time scheduling with lecturers. This is particularly important for data-driven recommendations for continuous quality improvement of the communication with a large group of lecturers – 27 in total.
PBL Demokit Logbook – Online tool designed to track the frequency of use of the PBL Demokits, and to serve as a resource management tool for the colleges through the logging of faults/damage to the kits for maintenance purposes
Mentoring Session Reflection Tool – A collaborative tool completed by mentors and lecturers for each of the mentoring sessions. The tool involves quantitative and qualitative metrics track new PBL pedagogy implementation and is designed to measure the quality of teaching while continually identifying areas for improvement in subsequent mentoring sessions.
the Lecturer Self-Assessment Tool – Similar to the Mentoring Session Reflection Tool, except that it is completed by lecturers only once per quarter, and is designed for lecturers to reflect honestly on the quality of their teaching, and to serve as an added layer of verifying data collected through the Mentoring Session Reflection Tool. This tool has since been deployed, prior to the start of mentoring sessions, to establish our PBL baseline we can use to track improvements in teaching throughout the duration of the project
In order to embed these systems within College operations, IYF conducted a two-day workshop with the mentors. The team was orientated on proper-use of all the tools and an online (Dropbox) system to allow for real-time access to data by the IYF team. In an effort to further optimize our systems, IYF embarked on another two-day workshop with mentors, lecturers, and campus managers where the college staff was trained on the system, with the key output of mentoring schedules for the 3rd Trimester being developed.
The orientation to our MERL PBL system was then concluded with a virtual session for preparation of campus managers to professionalize and train their relevant staff over proper-use of the extremely user-friendly online Demokit Management System.
The system has been received very positively by all college staff, with some campus managers requesting to receive continual IYF reports on the results of data analysis within High Gear. In effect, the success of swift execution of our workstream in KZN is due to the technological efficiencies that we have introduced to the programme since inception. These measures and management systems are being used and in some cases transferred to our direct beneficiaries (Lecturers & Students) by ensuring that we:
Maintain convenient access through e-platforms;
Expedited analysis of data for improved decision making and human-centered approaches; and;
targeted strategic development based on quantitative and qualitative evidence.