Why should you establish a standardized process for Data Review?
Standardizing a Data Review process helps set a clear expectation that the end product of Data Review is not knowledge, but action. Any Data Review Protocol should ensure that raw data is converted into a clear and digestible format before the reflection process so that reviewers can focus their energies on reflecting on the data, rather than synthesizing the data.
By separating the work of creating and refining a Data Review process from the work of implementing the process in practice, a standardized Data Review Protocol helps you focus your designated Data Review time on what you’re reviewing, not how to review it.
How should you conduct a Data Review?
- When: As soon as possible after collecting relevant data. The more dated the data, the less relevant it will be for making timely decisions.
- Why: The goal is to learn and improve — not to assign blame for shortfalls. Set norms accordingly.
- Who: The facilitator guides the conversation, but they do not have all the answers. Every voice matters.
- What: Don’t just review aggregate data. Disaggregate by demographic to reveal impact across lines of difference.
- How: Prioritize quality over speed, but adjust the time allotment based on how comprehensive the dataset is:
- A single tutor reviewing daily assessment data for all of their students should only need about 15 minutes to complete the protocol.
- An entire team reviewing the past year’s worth of data could take half a day to complete the protocol.
|Standard Data Review Protocol|
|This is a standard protocol you can use for a wide variety of reflections. It is broadly applicable whenever someone has data (qualitative, quantitative, or both) to review. Your organization might apply it to end-of-year outcome data; a head of program might apply it to training data at the end of tutor preservice training; a leadership team might apply it to quarterly parent feedback. There are also versions of this protocol specifically tailored for tutors reviewing student data.|
|Step 1: WHAT did we want to happen?||Ensure all participants are on the same page about what the goal or intended outcome was.||
|Step 2: WHAT actually happened?||Ensure all participants are on the same page about what the actual outcome or result was. Explore the divergences between expectations and realities.||
|Step 3: SO WHAT did we learn?||Reflect on successes and failures during the course of the project, activity, event or task. The question ‘Why?’ generates understanding of the root causes of these successes and failures.||
|Step 4: SO WHAT can we do better in the future?||Generate clear, actionable recommendations and next steps for future projects.||
|Step 5: NOW WHAT changes do we need to make to our project and individual plans?||Incorporate key lessons into your future actions. Document all key lessons for those who may inherit this project in the future.||