USAID Guide: A Roadmap for Measuring Distance Learning

As countries around the world have closed learning institutions in response to the COVID-19 pandemic, teaching and learning have pivoted from in-person instruction to distance learning. Simultaneously, there has been a surge in efforts to promote access to distance learning programming.

Distance learning is commonly used to reach learners who need flexible learning opportunities, as well as to reach groups excluded from formal education institutions. It can serve as the main form of instruction or can complement or supplement in-person learning. As countries and education agencies take up distance learning, it is important to design and implement evidence-based strategies for monitoring and evaluation to measure whether distance learning efforts are serving the intended communities and achieving intended objectives.

The purpose of this review, A Roadmap for Measuring Distance Learning, is to support education practitioners, host country government representatives, donors, implementers, non-governmental organizations (NGOs), civil society organizations, and other stakeholders in applying best practices to monitor and evaluate distance learning initiatives designed for diverse learners and implemented both within and outside of learning institutions. This review covers the four key distance learning modalities: radio/audio, television/video, mobile phone, and online learning. Printed texts, which are often developed to accompany these first four modalities, can also be a fifth modality in contexts where technology is not used.

Most of the data sources were drawn from work in the primary education sub-sector. However, much of the guidance can be applied to secondary and tertiary-level distance learning. This review is also applicable to data collection in both crisis and non-crisis contexts.

This review presents a roadmap that guides users through four steps of planning and designing how distance learning delivered through any of these modalities can be monitored and evaluated.

  • Step 1: Determine the Objectives of Monitoring and Evaluating Distance Learning
  • Step 2: Determine What Will Be Measured (Reach, Engagement, and Outcomes)
  • Step 3: Determine How Data Will Be Collected (In-Person or Remotely)
  • Step 4: Determine the Methods and Approaches for Measurement

Based on emerging global evidence, this review guides users through the process of measuring the reach, engagement, and outcomes of distance learning initiatives. 

    1. Reach  – captures access to technology (devices and software); infrastructure for technology (electricity, internet, or phone connectivity); and distance learning programming and content.
    2. Engagement  – assesses the extent to which users participate as intended in the programming, including the degree to which users perceive the content to be relevant, captivating, and of high quality.
    3. Outcomes  – measure changes in learning of content knowledge as well as social and emotional learning (SEL).

In addition to providing step-by-step guidance, this review provides three overarching recommendations for developing and implementing evidence-based monitoring, evaluation, and learning (MEL) plans for distance learning initiatives. The recommendations are to:

  1. Integrate in-person and remote approaches, use multimodal interfaces, and employ mixed methods to measure distance learning. Integrating in-person and remote data collection (e.g., in-person testing and remote interviews), using multi-modal interfaces (e.g., phone calls and SMS surveys), and mixed-methods approaches (e.g., interviews, surveys, and photographs) helps promote greater participation and leads to more accurate results.
  2. Encourage innovative solutions to measure reach, engagement, and outcomes during a quick pivot to distance learning, while also developing high-quality MEL strategies for the longer term. The guidance in this review helps teams think about short-term MEL (monitoring, evaluation, and learning) needs while working toward longer-term strategies for assessing the effectiveness of distance learning.
  3. Design equitable monitoring and evaluation approaches and conduct systematic equity analyses of distance learning initiatives. Evaluative approaches to distance learning must attempt to measure and analyze whether marginalized individuals and groups are being systematically included or excluded through distance learning programming as well as during in-person and remote data collection.

Download full review here: A Roadmap for Measuring Distance Learning

Leave a Reply