Adapting evaluation in the time of COVID-19

Evaluation needs to respond to the changes brought about by the Covid-19 pandemic. As well as direct implications for the logistics of collecting data and managing evaluation processes, the pandemic has led to rapid changes in what organisations are trying to do and how evaluation can best be used to support these changes.

In the third of our Adapting Evaluation in the time of COVID-19 blog series, we are focusing on the third cluster of tasks in the Rainbow Framework – FRAME the boundaries of an evaluation – identify the primary intended users, decide purposes (primary intended uses), specify the key evaluation questions, and determine what ‘success’ looks like. We’ve brought together some useful advice and resources for addressing these tasks. Please add other suggestions in the comments below.

You can also catch up on the first two posts in this series:

1. Identify primary intended users

Clarify who will actually use the evaluation—not in vague, general terms (e.g. “decision-makers”) but in terms of specific identifiable people (e.g. the manager and staff of the programme; the steering committee; funders deciding whether to fund this programme or similar programmes in the future). View full task page in the Rainbow Framework.

While your primary intended users may remain the same, it’s worth reviewing this in light of potentially new organisational circumstances. Some things to look at include:

Have there been any major staffing changes or restructuring? If some of your primary intended users are no longer in the same role or have left, will there be a replacement who you should identify as a new intended user? If so, What orientation will you need to give your new intended users? How will you help them to feel ownership of and buy-in to the evaluation? This can often be helped with involving intended users with the decision-making
Have the primary intended users fundamentally changed? In some cases there is now increasing attention to prioritising local implementers and local communities as primary intended users of evaluation to inform locally appropriate adaptations.

Resources

Utilization-Focused Evaluation Checklist: Step 3 of this checklist contains some useful advice about identifying, monitoring engagement and facilitating change of primary intended users.