Data Use: Program Effectiveness and Improvement

Program Effectiveness and Improvement: The program has demonstrated a commitment to understanding overall program effectiveness and processes for ongoing improvement. 

• What student and tutor outcomes represent the vision of success for the program?
• What data will the program collect to measure these outcomes? 
• How will these data reflect a holistic understanding of students’ and tutors’ experiences?
• How do the measures address both the impact on K-12 students and the impact on HEI tutors?
• What processes will be in place to review and act upon collected data?
• How will these review processes promote equity and reduce bias?

• Define measures of success in alignment with your logic model, including process measures of implementation and non-academic measures of impact.
• Develop and/or leverage existing tools to collect data on the identified measures, including both quantitative and qualitative data.
• Set benchmarks to monitor progress towards outcomes.
• Put systems in place for collecting data that can be disaggregated by race, gender, individualized education program statutes, home language, and other important factors to ensure equity of services.
• Consider using the Tutoring Quality Improvement System to assess overall program’s alignment with Quality Standards as part of continuous improvement efforts.
• Meet requirements and use best practices for data privacy.

HEI Specific Tools:

• If you are interested in learning more about current research at other institutions or how to conduct a study at your higher education institution (HEI), contact: info@studentsupportaccelerator.org. 

From Existing Resources: 

• Developing a Performance Measurement Plan 
• Tutoring Survey Instruments
• Examples of Data Collection Tools
• Performance Management Plan Template
• Tutoring Quality Improvement System
• Quality Standards
• Student Data Privacy Guidance
• Building in Continuous Improvement

Develop a holistic data collection strategy that includes non-academic measures of impact for both students and tutors.

• While academic improvement is the primary purpose of a tutoring program, it is not the only goal. Programs need to collect data across multiple dimensions to ensure that they are building students’ overall well being and are serving students equitably. As a results they benefit from collecting data to evaluate student experiences with tutors, not just student academic growth. HEI programs should also set goals and collect data to assess progress on tutor outcomes.
• Programs should collect feedback from all stakeholders (students, families, teachers, tutors, and administrators) to understand and improve program impact at all levels. While achievement data and feedback from school partners is critical, programs should include student voices when evaluating program impact: tutoring programs exist primarily to serve students. 
• These pre-developed Tutoring Survey Instruments can support the program’s understanding of student and tutor experiences. 
• Modeling use of comprehensive data measures will also support your student tutors with deepening their understanding of these practices, furthering tutoring as a career development opportunity for participating student tutors.
• Consider phasing the approach to data collection to ensure you are able to commit to the data you plan to collect. For example, you may begin with collecting feedback from 1-2 stakeholder groups during the first year of implementation, and then increase the number of stakeholder groups in subsequent years. 

Set specific benchmarks with expected dates to help stay on track.

• Programs should set benchmarks with expected dates for all measures — not just for student growth, but also for aspects like student/tutor/teacher/parent satisfaction. Routinely reviewing data and comparing it to benchmarks helps programs understand where they are on-track or off-track; this is critical for establishing a data- to-action cycle of insights and iterative improvements. These processes will also support the program with communicating progress and improvements to funders, ensuring the HEI is able to sustain the tutoring program into the future. 

Align routine assessments with session targets (and, ideally, with classroom curriculum).

• Well-aligned, routine assessments can help programs quickly identify student knowledge gaps and target upcoming sessions to meet specific student needs as they emerge.
• If partner schools have existing interim assessments, leverage those data to reduce the need for another assessment and assure that tutoring success is tied to outcomes the school and district sees as relevant.
• In order for formative assessments to result in more student learning, tutors need time and support to review the assessment and formulate a plan to address each student’s needs.

Develop systems for visualizing data for stakeholders.

• Programs should develop in-house capability for distilling data so that information can be presented in a digestible and actionable format. Some programs may have databases and utilize software such as Tableau to visualize data, while other programs that operate at a smaller scale may find it sufficient to store data in well-designed Google spreadsheets. Consider reaching out to faculty and/or staff at your HEI with expertise in this area - there are likely data management and analysis software products already purchased by your HEI to use. 
• Ultimately, the method chosen for visualizing data should allow for users to sort the data and easily extract insights. 
• Programs should regularly gather feedback on their data collection and visualization systems and improve upon these as part of their continuous improvement processes. If your program has access to research assistants, consider working with them to support your program’s data collection and measurement work.