Key Takeaways
To measure and understand student learning with a focus on unfinished learning, schools and districts should follow a strategic process.
Determine if you are ready to measure learner growth by reviewing your data for measurement feasibility.
Develop research questions that are strong, measurable, and specifically aligned to the areas you would like to measure.
Collect data that answer your research questions and include the sample groups or subgroups you want to examine.
What is the problem?
From the outset of the pandemic, ‘learning loss’ emerged as a key concern at the national level. Researchers predicted group-level performance gaps on standardized assessments, highlighting the deleterious effects of the pandemic. However, negative or reduced rates of learner performance indicate potential for growth – not a loss of knowledge. Schools and districts need a different approach to understanding learner progress in order to identify areas of success.
Why is it important?
By focusing on unfinished learning (versus ‘learning loss’), districts can take an asset-based approach to examining student growth over time. This allows them to better identify which students (or groups of students) have made the most progress. With these patterns identified, it becomes possible to examine and identify factors that contributed to student growth, allowing these enabling supports to be extended to all students.
The research says...
Throughout the pandemic, reports have contributed to the national concern over ‘learning loss,’ as well as the short- and long-term impact of the COVID-19 crisis on learner outcomes. However, negative or reduced rates of learner performance do not indicate a loss of knowledge but rather slower achievement and growth. Therefore, measuring unfinished learning is essential for understanding what is possible, instead of what is lost, thus placing the onus for improvement on school systems, not students.
Measurement should focus on individual learner context and process to uncover more precise factors that contributed to learner growth and avoid placing emphasis on ‘learning loss.’
Educational leaders can more immediately understand and address student needs by focusing on measuring unfinished learning (versus ‘learning loss’).
Measuring unfinished learning assists educational leaders in shifting towards opportunities for educational transformation.
How: Solution
It is important for school and district leaders to take an asset-based approach to measuring student learning, identifying areas of progress and the associated enabling systems and structures, rather than a deficit stance that focuses primarily on gaps or loss. Using a diverse research team with a multitude of perspectives, educators and leaders should take time to ensure they are creating a strong foundation for identifying whether they are ready to start the process and if the questions they are asking are directly aligned to the data they hope to collect.
As a team, it is important to work through the initial steps outlined below to ensure that all stakeholders are brought into the process early on and to establish a trajectory for the work that will be actionable, responsive, and has the best chance for success.
Review the Data You Have
Before jumping in, it is important to first make sure that you have all of the data you need to measure learner growth in your school or district. The data should come from multiple sources and points in time during the year. This helps you to get a full picture of each student and the school as a whole. Some example platforms for data collection include:
iReady
NWEA MAP
Scholastic Reading Inventory (SRI)
Diagnostic Reading Assessment (DRA)
The data should also be organized in a way that it can easily be compared (e.g., grade, school year) and aligned with the correct student groups. Research or data teams can then self-assess to determine whether the needed data exists to assess learner growth over time or if they need to collect additional data to create a full picture.
Strategies/Resources:
In addition to our Unfinished Learning Measurement Guide, the following strategies and resources can be used to determine whether you have the correct data.
Develop Your Research Questions
Developing specific research questions – and grounding your research and data in those questions – sets a clear path toward how you plan to measure outcomes (in this case, student learning). As you develop research questions, it is important to create questions that are inclusive of all stakeholders’ (including students, teachers, and families) needs so that all involved can, in turn, understand and provide insight for learning growth.
The following steps can help you to develop your research questions.
Step 1: Create your research questions with a purpose in mind.
Your research questions should guide your data collection and analysis. To do this:
Identify the topic or purpose of what you are measuring (e.g., learner growth in math during the 2021-22 school year).
Specify the context (e.g., online learning).
Determine what you want to achieve (e.g., discover the enabling math systems that improved learning).
Note any potential relationships you want to explore (e.g., increase in math comprehension for students whose teacher received professional learning).
Step 2: Research questions should be clearly stated and focused.
Make sure to avoid ambiguity. Your research questions should address who, what, where, and when.
Unclear and Broad | Clear and Focused |
| What effect did the pandemic have on student learning? | What effect did the pandemic have on math comprehension scores for students who are learning English engaged in remote learning during the 2020-21 school year? |
Step 3: Make sure that your research questions are actually researchable.
Determine if your questions can be answered using your available data sources – both those you have direct access to (i.e., progress reports, standardized assessment data, benchmark scores) as well as other sources that may need additional permission such as surveys, qualitative interviews, or focus groups.
Strategies/Resources:
Explore the following strategies to learn more about developing concrete research questions that will enable successful unfinished learning measurement.
Collect Your Data
Before collecting data, make sure that it is both numeric and standardized – meaning that you are comparing apples to apples (such as lexile scores for reading or scale scores in math). To determine what data you should collect and whether the data will allow you to conduct your analysis, use these three self-assessment questions:
Do you want to examine changes in scores over time in any subject where you have consistent data? If so, determine which school years you want to use in your analysis and then make sure that you have consistent data for each of those years that can be compared.
Which students should be included in the sample? A sample is a group (or groups) to be investigated. To measure unfinished learning, the sample could be students for whom you have consistent data across multiple years; those from specific age groups or school buildings; or those who have participated in different learning contexts such as remote, hybrid, or in-person environments.
Do you want to look at any specific subgroups within your sample, such as students who are learning English or those receiving special education services? Make sure to take sample size into account when identifying your subgroups. If the number of students in a particular group is not large enough, then you will not be able to make valid observations.
Strategies/Resources:
The following are supports that can be used to help guide successful data collection. When implementing these strategies, be sure to keep in mind the type of data you want to collect and the objectives or purpose for which you are collecting the data.
Take it further
By focusing on unfinished learning, districts can examine student growth over time. This allows them to take an asset-based approach and better identify which students (or groups of students) have made the most progress. From there, they can examine and identify the supports that contributed to student growth so that the same supports can be extended to all students.
Additional Resources and Content:
This resource can be used to extend your knowledge about creating evidence based strategies to support learners during unfinished learning:
Once you complete this guide, you are ready to learn more about the process for effectively organizing, analyzing, and making sense of data.