mcginnis
Fri, 09/13/2024 - 12:21
Edited Text
Understanding the Impact of Culturally Responsive Instructional Materials on
Middle School Students’ Math Achievement Scores in Westmoreland County
A Doctoral Capstone Project
Submitted to the School of Graduate Studies and Research
Department of Education
In Partial Fulfillment of the
Requirements for the Degree of
Doctor of Education
Rebecca M Henderson
Pennsylvania Western University
July 2023
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Rebecca M. Henderson
July 2023
ii
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
iv
Dedication
To my children, Neil and Elizabeth.
The most meaningful things we do are often the hardest things we do. It's a
concept that may seem scary at first, but it's an essential truth that I want you to
understand and embrace. During your lives, you will encounter situations that appear
overwhelming, and it's normal to feel a sense of fear or doubt. Fear is a natural response,
but it should not determine your actions or hinder your progress. Instead, let courage
guide you and push through the fear. Trust in your abilities and believe in yourself. You
are capable of incredible things.
Remember, we do hard things.
Love, Mom
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
v
Acknowledgements
The completion of this Capstone Research Project would not have been possible
without the love, support, guidance, and encouragement that I received from family,
friends, and colleagues during this process. I want to start by thanking my husband,
Scott, and my children, Neil and Elizabeth. Thank you for believing in me and
supporting me, even when it meant spending less time together so that I could complete
this work. A special thank you to my parents, Jim and Betty, and the rest of my family
for their continued love and support. To my friends who have supported me, and laughed
with me, and have always kept me grounded. A special thank you to the following
colleagues: Dr. Jason Stragand and Dr. Denise Lohr, for encouraging me to enter this
program and going on this journey with me, Dr. Matt Thomas, Dr. Jason Boone, and Mr.
Timothy Hammill, for their friendship, humor, and wisdom, and Mr. Blaine Howard, for
suggesting the focus of my research. Thank you to Dr. J. Kevin Lordon, Faculty
Capstone Committee Advisor, who encouraged me throughout the process. A very
special thank you to Dr. Jason A. Conway, for your guidance, support, feedback,
encouragement, and wisdom during this process. I deeply value the environment that you
have established at the Westmoreland Intermediate Unit, where the focus on personal and
professional growth is paramount and nurtured for every employee. And as always,
thank you to Steve Perry and Corky St. Clair.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
vi
List of Figures and Tables
Figure 1: Westmoreland County Student Enrollments by Race / Ethnicity
7
Figure 2: Countywide Average by Anchor 2015 - 2019
8
Figure 3: What Culturally Responsive Competencies do State Teaching
Standards Address?
31
Figure 4: Growing Awareness Inventory Theory of Action
33
Figure 5: Greater Latrobe School District Percent Enrollment by Race/Ethnicity
50
Figure 6: Greater Latrobe School District Percent Enrollment by Student Groups
50
Figure 7: Greater Latrobe School District Algebra I Proficiency Percentages
2013 - 2022
51
Figure 8: Greater Latrobe School District Total Number of Student Tested vs.
Year Administered
52
Figure 9: Monessen City School District Percent Enrollment by Race/Ethnicity
53
Figure 10: Monessen City School District Percent Enrollment by Student
Groups
54
Figure 11: Monessen City School District Algebra I Proficiency Percentages
2013 - 2022
55
Figure 12: Monessen City School District Total Number of Student Tested vs.
Year Administered
56
Figure 13: Yough School District Percent Enrollment by Race/Ethnicity
57
Figure 14: Yough School District Percent Enrollment by Student Groups
58
Figure 15: Yough School District Algebra I Proficiency Percentages 2013 - 2022
59
Figure 16: Yough School District Total Number of Student Tested vs. Year
Administered
60
Figure 17: Group Summary: Winter 2012 Performance Keystone, Algebra I
78
Figure 18: Group Summary: Spring 2022 Performance Keystone, Algebra I
79
Table 1: All Districts’ Total Student Enrollments by Race/Ethnicity
80
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
vii
Figure 19: Percent Enrollments Economically Disadvantaged by District
81
Figure 20: Percent Enrollments ELL by District
82
Figure 21: Percent Enrollments Special Ed. by District
83
Figure 22: Enrollments by Gender (M/F) by District
84
Tabe 2: Building Principal Perception Survey Questions
85
Table 3 Teacher Perception Survey Questions
86
Table 4: Culturally Responsive STEAM Scorecard
88
Table 5: Group Summary, Performance Levels by Grade 08 and Ethnicity
92
Figure 23: GLSD Culturally Responsive STEAM Scorecard Results
95
Figure 24: YSD Culturally Responsive STEAM Scorecard Results
98
Table 6: Number of Algebra I Eligible Content Items Present by Vendor
99
Figure 25: Teacher Perceptions of Standards Alignment of Instructional
Materials
101
Figure 26: Building Principals’ Perceptions of Instructional Materials
Alignment to Standards
102
Figure 27: Teacher Perceptions of PD to Effectively Use Instructional Materials
103
Figure 28: Money Spent Annually By Districts on Teacher PD to Utilize
Instructional Material
104
Figure 29: Teacher Perceptions on the Impact of Instructional Materials on
Student Achievement
106
Figure 30: Teacher Perceptions on Physical Materials Preparing Students for
Keystone Exams
107
Figure 31: Teacher Perceptions on Digital Materials Preparing Students for
Keystone Exams
107
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
viii
Figure 32: Teachers' Perceptions of Student Reflections on Instructional
Materials
108
Figure 33: Building Principals’ Perceptions of Teacher Modifications to
Resources
109
Figure 34: Building Principals’ Perceptions of Teacher Development of Original
Resources
109
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
ix
Table of Contents
Dedication
iv
Acknowledgement
v
List of Figures
vi
Abstract
xiii
Chapter I. Introduction
1
Background of Study
1
Identification of Capstone Focus
3
Research Questions
4
Expected Outcomes
4
Fiscal Implications
4
Summary
5
Chapter II. Literature Review
6
Standards-Based Education Reform in the United States
8
The National Council of Teachers of Mathematics
10
National Education Goals
12
No Child Left Behind
14
Common Core State Standards
15
Pennsylvania Common Core Standards
16
The Every Student Succeeds Act
17
Publishers’ Responses to Standards-Based Education Reform
18
Textbook Content vs. Proficiency Exam Content
19
Textbook Structure
20
Standards Alignment
21
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Educators’ Response to Standards-Based Instructional Materials
x
22
Teacher Professional Development
23
Process for Adopting and Implementing New Standards in PA
24
A Mindset Shift in Mathematics
26
History of Culturally Responsive Teaching in the United States
27
Culturally Responsive Teaching in Mathematics
28
Culturally Responsive Teaching in Pennsylvania
30
The Impact of Reform on Culturally Responsive Teaching
32
Culturally Responsive Teaching as a Pedagogical Practice
34
Implementing OER to Facilitate Culturally Responsive Teaching
35
Perceived Benefits of Using Open Educational Resources
36
Perceived Barriers to Increasing Student Achievement
37
Understanding the Impact of COVID-19 on Student Achievement
38
NAEP Results by Student Group
41
Steep Declines in Algebra
41
Summary
42
Chapter III. Methodology
44
Methodology
44
Purpose of Study
45
Setting
46
Participants
49
Research Plan
60
Research Methods & Data Collection
66
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
xi
Data Collection Timeline and Plan Described
67
Validity
69
Summary
71
Chapter IV. Data Analysis and Results
Data Analysis
73
74
Participants
74
Data Analysis Procedure
75
Results
91
Research Question 1
91
Research Question 2
102
Research Question 3
105
Discussion
110
Summary
115
Chapter V. Conclusion and Recommendations
Conclusion
118
119
Research Question 1
120
Research Question 2
122
Research Question 3
123
Overview of Results
125
Limitations
136
Future Directions for Research
137
Summary
139
References
140
APPENDIX A. Building Principals Survey
149
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
xii
APPENDIX B. Teacher Perception Survey
153
APPENDIX C. Culturally Responsive STEAM Scorecard
159
APPENDIX D. IRB Approval
161
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
xiii
Abstract
Building Administrators and Algebra I teachers in Westmoreland County have worked
hard to determine the root causes of students’ historically low achievement scores on the
Algebra I Keystone Exam over the past decade. Through intensive data analysis, they
have investigated the alignment of their curriculum to Pennsylvania Core State Standards,
their instructional practices, and the alignment of their in-class assessments to Eligible
Content Items covered on the Algebra I Keystone Exam. Yet one area that has not been
intensively investigated is the impact of culturally responsive instructional materials on
student achievement scores. This research study focused on building administrator and
teacher perceptions of their primary instructional materials used in Algebra I classrooms,
as well as the amount of time and money dedicated to support educator professional
development to successfully implement those resources in their classrooms. Participants
completed perception surveys to share their thoughts about their instructional material,
supplemental materials, Open Educational Resources, professional development, and
time and funding spent by their district to support teachers. They also completed the
Culturally Responsive STEAM Scorecard to evaluate the cultural responsiveness of their
primary instructional materials. In addition, the researcher evaluated the presence of
Eligible Content Items present on the Algebra I Keystone Exam on primary Algebra I
instructional materials used by three case study districts representing Westmoreland
County. While there was not a strong correlation between the culturally responsive
nature of instructional materials and student achievement scores, there was evidence that
shows that the lack of Eligible Content Items present in primary instructional materials
negatively impacts student achievement scores.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
1
Chapter I
Background
Each year, the Curriculum Services Team of the Westmoreland Intermediate Unit
(WIU) works with the seventeen public school districts within their geographic region to
analyze student achievement data, evaluate their curriculum and instruction, and support
student achievement and success. This work focuses more on standards alignment and
state-mandated standardized test scores and less on instructional materials' impact on
student achievement. Previous actions to complete this work include mapping standards
to a district’s curriculum, supporting the development of a more thorough curriculum and
concept maps that address tested content, and a deeper understanding of the state
standards and how to align instruction to them.
While there is less consideration given to the actual instructional materials used to
support instruction, those materials make a significant impression on students and
teachers and steer teaching and learning within the classroom. Districts rarely separate
their curriculum discussions into defined categories. Instead, they use the term
curriculum to refer to standards, the scope and sequence of content taught, the instruction
delivered, the instructional materials used by teachers to engage students, and the
assessments developed and used to determine student achievement.
The researcher has been a part of the Curriculum Services Team at the WIU since
August 2018. She has served as Distance Learning Supervisor and Curriculum Services
Supervisor during this time. In this capacity, she has been responsible for supporting
districts as they work to implement instructional initiatives and mandates as set forth by
the Pennsylvania Department of Education (PDE) in both traditional and online
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
2
classroom environments. In serving as a liaison between PDE and the seventeen
public school districts in Westmoreland County, the WIU’s Curriculum Services team
utilizes innovative practices to advance the practice of teaching and learning across
Westmoreland County.
School Improvement and Accountability is a central focal point of this work for
the WIU Curriculum Services Team, as the United States Department of Education
(USDOE) has outlined annual requirements that both states and local school districts
must meet to qualify for funding. As such, the WIUs Curriculum Services Team is
committed to supporting and monitoring the school improvement efforts of districts.
These efforts typically include curriculum writing, standards analysis and alignment,
assessment data analysis, instructional strategy, and resource review and implementation.
As part of this work, the WIU’s Curriculum Services Team regularly analyzes
district-specific data on the annual Pennsylvania System of School Assessment (PSSA)
and Keystone Exams. The PSSA is a standards-based assessment administered to every
student in grades 3 through 8. Students are assessed in English Language Arts and
Mathematics each year and in Science in grades 4 and 8. The Keystone Exams serve as
end-of-course assessments determining a student’s proficiency in Algebra I, Literature,
and Biology. These assessments meet the federal accountability criteria established in the
Every Student Succeeds Act (ESSA) of 2015.
Each year, PDE releases the scores of all students, all districts, and the statewide
average score for each subject area. Historically, Algebra I scores across Westmoreland
County’s seventeen public schools have been below the state average, which has been an
ongoing concern for administrators. Administrators want to understand the root cause of
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
3
these county-wide historically low scores to support teachers and students better. They
have asked the WIU Curriculum Services Team to help them analyze their specific
curriculum, instruction, instructional materials, and assessment circumstances and
provide consultative services that can promote increased student achievement.
Identification of Capstone Focus
This research will investigate the possible correlation between the historically low
Keystone Algebra I Scores in middle school and building principal and Algebra I
teachers’ perceptions of student identification with examples in the word problems found
within their middle school instructional materials. There are many barriers to
implementing Open Educational Resources (OER) and developing culturally-responsive
instruction that reflects a district’s community, demographics, and values to establish
equitable instructional practices. Despite the amount of funds made available to districts
through the Elementary and Secondary School Emergency Relief (ESSER) Fund, and the
resources compiled by the Pennsylvania Department of Education on their Equitable
Practices Hub, there appears to be hesitancy to develop instructional materials that meet
the needs of their individual student populations.
The research is showing that when students see themselves reflected in their
instructional content, they learn more and understand content and concepts on a deeper
level. Yet, the conversations around school districts in Westmoreland County focus on
purchasing technologies, tools, and programs developed by national vendors that do not
reflect the students within the county. This study seeks to better understand the
correlation between the instructional materials currently in use by 7th and 8th-grade
mathematics teachers, their perceived cultural responsiveness by district building
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
4
principals, mathematics teachers, and students, and student achievement scores in
Operations with Real Numbers and Expressions in Algebra I classes.
Research Questions
The following questions will guide this case study:
1. What are the factors that impact student achievement in Algebra I?
2. What are the factors that impact instructional delivery in Algebra I?
3. What are the teacher and administrator perceptions of the impact of instructional
delivery on student achievement scores in Algebra I?
Expected Outcomes
There are many barriers to implementing Open Educational Resources (OER) and
developing culturally-responsive instruction that reflects a district’s community,
demographics, and values to develop equitable instructional practices. Despite the
amount of funds made available to districts through the Elementary and Secondary
School Emergency Relief (ESSER) Fund, and the resources compiled by the
Pennsylvania Department of Education on their Equitable Practices Hub, there appears to
be hesitancy to develop instructional materials that meet the needs of their individual
student populations. The results of this case study will provide building administrators
and Algebra I teachers with additional guidance and support when determining which
instructional materials may significantly impact student achievement within their student
populations.
Fiscal Implications
The fiscal implications of this capstone research project consist solely of the cost
associated with the salaries and benefits of the participants for the time required to
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
5
complete the activities within this case study. Each participant will complete a
Perception Survey at the start of the case study, which will take 20 minutes to complete.
They will then complete the Culturally Responsive STEAM Scorecard, developed by
NYU Steinhardt, to share their perception of the cultural relevance of their primary
instructional material used for Algebra I as the second step of data collection. This
activity will take approximately two hours to complete. Since the participants are
employees at three representative districts across Westmoreland County, the cost will be
distributed evenly among those districts. The researcher will utilize Google Forms to
collect perception survey data at no additional cost to the school districts or the
Westmoreland Intermediate Unit. The researcher has also been granted permission to
fully use and publish components of the Culturally Responsive STEAM Scorecard, a
freely available online resource, by NYU Steinhardt at no additional charge. All costs
associated with this capstone research project are indirect, and the overall financial
impact for an individual school district will be minimal.
Summary
Chapter I introduced the background context of this case study. It also explained
why studying the impact of instructional materials on student achievement scores in
Algebra I across Westmoreland County is essential for administrators and educators to
develop culturally responsive instructional experiences that promote increased student
achievement. Chapter II will provide a literature review that helps create a thorough
understanding of the currently available research to help inform instructional practices.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
6
Chapter II
Literature Review
Since the early 1980s, Standards-Based Education Reform has driven public
education in the United States. This type of education reform calls for the establishment
of academic content standards, which are used to determine critical elements that students
must know within each content area. Student achievement is also measured against these
established standards, making high-stakes testing a key component to monitoring student
progress annually. Historically, the individuals that determine the need for
Standards-Based Education Reform are not the same individuals that are responsible for
the implementation of the standards at the classroom level.
While structures are in place to ensure that educators have access to high-quality,
job-embedded professional development that is designed to help them implement newly
adopted standards within the classroom, much of that professional development is
focused on instructional and assessment strategies. Historically, little attention has been
given to assisting educators in critically examining instructional materials in order to
determine the accuracy of their standards alignment. Unfortunately, instructional
materials are one component of a district’s implemented curriculum that has the greatest
impact on student achievement and mastery of newly adopted standards.
Westmoreland County, located in the Southwestern region of Pennsylvania, is
home to approximately 43,710 students in grades Kindergarten through twelfth grade
across 17 different public school districts. The smallest school district serves
approximately 712 students, while the largest serves approximately 5,304 students.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 1 shows student enrollments in Westmoreland County public school districts by
Race / Ethnicity.
Figure 1
Westmoreland County Student Enrollments by Race / Ethnicity
While student populations across the county tend to be overwhelmingly white,
each district has a unique culture and climate based on geographic location, population
demographics, and societal norms. A commonality between all districts is their
historically low scores in Algebra I on state standardized tests. Since 2015, countywide
averages have only risen above a 60% proficiency rating one time. Figure 2 shows the
countywide average scores by Algebra I Anchor Analysis from 2015 - 2019.
7
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
8
Figure 2
Countywide Average by Anchor 2015 - 2019
These scores have become a major concern for district administrators and building
leaders across Westmoreland County who are analyzing their intended curriculum and
implemented curriculum in order to determine how to better support students in the
classroom and foster higher levels of student achievement.
Standards-Based Education Reform in the United States
Standards-based education reform in the United States of America has its roots in
the 1983 publication titled, “A Nation At Risk: The Imperative for Educational Reform.”
This report, authorized by President Ronald Reagan, was developed by an
eighteen-member commission of individuals working within private companies,
government agencies, and education organizations. The authors determined that declines
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
9
in the efficacy of public education were due to inconsistencies in four key areas of
education: the content or curriculum being covered in classrooms, district expectations
for students, the time that is dedicated to education over the course of the year, and the
quality of programs designed to prepare candidates for the teaching profession as well as
continuing professional development once teachers enter the profession.
As the authors laid out their case for the need for a more rigorous approach to
K-12 public education in the United States, and warned of an increase in mediocrity
among students, the authors state, “Our goal must be to develop the talents of all to their
fullest. Attaining that goal requires that we expect and assist all students to work to the
limits of their capabilities. We should expect schools to have genuinely high standards
rather than minimum ones, and parents to support and encourage their children to make
the most of their talents and abilities” (National Commission on Excellence in Education,
1983, p. 13). Furthermore, the authors felt that there must be a focus on equity for
students. They cautioned that sacrificing equity over education reform, or education
reform over equity, would lead to potentially disastrous consequences.
We do not believe that a public commitment to excellence and educational reform
must be made at the expense of a strong public commitment to the equitable
treatment of our diverse population. The twin goals of equity and high-quality
schooling have profound and practical meaning for our economy and society, and
we cannot permit one to yield to the other either in principle or in practice. To do
so would deny young people their chance to learn and live according to their
aspirations and abilities. It also would lead to a generalized accommodation to
mediocrity in our society on the one hand or the creation of an undemocratic
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
10
elitism on the other (National Commission on Excellence in Education, 1983, p.
13).
The strongest conclusion from this report was that, in order to effectively ensure
that students would be able to productively contribute to society and that future
generations would be able to compete on a global stage, we would need to develop a
more rigorous and structured curriculum that would address the relevant concepts and
skills that students would need to master prior to graduation in an equitable manner.
Doing so would instill a lifelong love of learning that would encourage individuals to
continue seeking learning opportunities throughout their adult lives. It would also
continue to foster support for public education well into the future.
The report was distinguished historically from others for its strikingly alarmist
tone. Mehta (2015) concluded:
Public confidence in schooling, which had already been falling as part of the
post-Watergate decline of confidence in public institutions, hit a new low after the
release of the report in 1983. States rushed to issue their own reports to evaluate
whether they were falling short of the new measures of excellence specified by A
Nation at Risk. Despite the claims of the critics, A Nation at Risk had been the
rare report that galvanized not only debate but also action. (p. 22)
The National Council of Teachers of Mathematics
The action that resulted from A Nation at Risk is commonly referred to as
Standards-Based Education Reform. One of the first recommendations supporting
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
11
Standards-Based Education reform came in 1988 from the National Council of Teachers
of Mathematics (Owens, 1988), which released a report titled, “Curriculum and
Evaluation Standards for School Mathematics: Report of the National Council of
Teachers of Mathematics’ Commission on Standards for School Mathematics.” While
there were other recommendations, this report focused heavily on the need for schools to
“implement a core curriculum that allows all students the opportunity to study the
important ideas and methods of mathematics.” (Owens, 1988, p. 2) In order to allow for
time to address new concepts within the core curriculum, the authors recommended a
de-emphasis on practicing computational skills with pencil and paper. Instead, they
encouraged the use of technology such as calculators and computers to assist students in
computation so that they were able to prioritize developing a solid conceptual foundation
for skills.
Computational skills are defined as more than mere arithmetical competencies.
They include, at the secondary level, the extensive manipulation of expressions
and equations that are central to current study in Algebra--a considerable portion
of the current Algebra I curriculum, for example, is devoted to manipulation skills
associated with solving equations, simplifying complex rational expressions, and
factoring. As with arithmetic skills, these are not envisioned as unnecessary but
as areas that should be de-emphasized; conceptual development of the ideas
underlying such operations should take precedence, calculator and computer
solutions are stressed, and applications serve as both motivators of operations and
practice opportunities for developing skills. (Owens, 1988, p. 3)
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
12
This strategy of de-emphasizing paper and pencil computational skills was
developed to better assist teachers in putting a priority on supporting students’
developmental readiness for instruction. Teachers would now have time to focus on
instructional strategies designed to support students in developing their conceptual
thinking and problem solving skills at various age levels. The end result would be
students that can make real-world connections to the content, solve complex problems,
make connections to the content year after year, and communicate about mathematical
constructs.
National Education Goals
This was further reinforced in 1990 by President George H.W. Bush and the
Nation’s Governors, who adopted six National Education Goals that were to be achieved
within the next ten years. These goals formed the basic structure for Standards-Based
Education Reform that are still present in the public education system of the United
States today.
The six National Education Goals establish that, by the year 2000:
● All children in America will start school ready to learn.
● The high school graduation rate will increase to at least 90 percent.
● American students will leave grades 4, 8, and 12 having
demonstrated competency in challenging subject matter including
English, mathematics, science, history, and geography; and every
school in America will ensure that all students learn to use their
minds well, so they may be prepared for responsible citizenship,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
13
● Further learning, and productive employment in our modern
economy.
● United States students will be first in the world in science and
mathematics achievement.
● Every adult American will be literate and will possess the
knowledge and skills necessary to compete in a global economy
and exercise the rights and responsibilities of citizenship.
● Every school in America will be free of drugs and violence and
will offer a disciplined environment conducive to learning.
(Educate America Act, 1994)
These goals, intended to be seen as overlapping, intricate goals that complimented
each other, were designed to provide a more structured framework for public education.
Officially becoming known as the Neighborhood Schools Improvement Act, the 102d
Congress debated the merits of the six goals, as well as the roles of individuals and
organizations that would be involved in monitoring public schools as they worked
towards achieving those goals.
Ultimately, the 102d Congress did not formally pass the Neighborhood Schools
Improvement Act and did not formally adopt the goals developed by President Bush and
the Nation's Governors at this time. Instead, the previously adopted Elementary and
Secondary Education Act of 1965 (ESEA) was reauthorized in 1994 under President Bill
Clinton. This reauthorization, called the Improving America’s Schools Act of 1994,
acknowledged the importance of the National Education Goals, and began to develop
parameters in which schools would need to operate so that students could meet those
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
14
goals. As stated in the document on page 3, since 1988 the country has learned
that “All children can master challenging content and complex problem-solving skills.
Research clearly shows that children, including low-achieving children, can succeed
when expectations are high and all children are given the opportunity to learn challenging
material.” Again, there was a reiteration of the need for a focus on problem-solving skills
over computational skills, and in setting higher expectations for students in order to push
their understanding and mastery of a more challenging curriculum.
No Child Left Behind
This pattern of reauthorizing ESEA and renaming it would continue into both the
George W. Bush and Barack Obama Administrations. During the George W. Bush
Administration’s reauthorization, also known as the No Child Left Behind Act of 2001
(NCLB), we still see a strong preference for establishing high achievement standards,
however now we see a shift away from a common set of national academic standards.
Instead, states are directed to develop their own set of academic standards as well as
accompanying assessments to measure student understanding and mastery of content. To
stress the importance of this development, Title VI, referred to as Section 601 Flexibility
and Accountability, laid forth several conditions for states to receive grant funding for the
development of standards-aligned, state assessments designed to ensure validity and
reliability by assessing student achievement through multiple measures (United States
Congress, 2002).
We now see a shift in control for K-12 public education. While accountability has
been a central tenant of Standards-Based Education Reform since the publication of A
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
15
Nation at Risk, the control has largely laid at the feet of the individual states. With the
inception of NCLB, control over accountability methods now shifted to the federal
government. However, it would not be the last shift in control that we would see.
Despite a tradition, or at least a conventional myth, of local control that
dominated education federalism for decades, federalism boundaries are
increasingly noted for their fluidity in the public education sector. Not only are
federalism boundaries becoming increasingly fluid, but the pace of change to
these boundaries has also quickened over time, no doubt a reflection of
increasingly contested federalism claims. (Heise, 2017)
The replacement for NCLB, the Every Student Succeeds Act of 2015 (ESSA), shifted
that control back to the states again.
Common Core State Standards
In the midst of this back and forth regarding accountability oversight and control,
the conversation regarding the establishment of a core curriculum continued. In 2008,
the National Governors’ Association established a group of experts to develop a common
set of core standards that could be utilized across all fifty states. In order to do so, this
group of experts needed to analyze the current standards already in place in each state
and determine whether or not those standards were high-quality, rigorous, and effective.
“The common critique was that most American grade-level guidelines were “a mile wide
and an inch deep,” in stark contrast to the fewer but more intense expectations in
high-achieving countries like Japan and Singapore” (Garland, 2020). Out of their work,
the Common Core State Standards Initiative (CCSS) was developed to address learning
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
16
standards in English Language Arts and Mathematics. “The Standards for Mathematical
Practice describe varieties of expertise that mathematics educators at all levels should
seek to develop in their students” (National Governors Association Center for Best
Practices & Council of Chief State School Officers, 2022). The focus of the mathematics
standards became processes and proficiency in order to ensure that students were able to
not only complete each step of a given problem, but also explain those steps as well as
the ways in which the steps were connected to each other.
This emphasis on processes and proficiency was a strategic design meant to
support students and help to prepare them for their future endeavors. “The goal is not
just to have students master calculations that they will perform only on standardized tests;
the goal is to have them think in ways that will prepare them for college, careers, and
life” (Tampio, 2018). Again, we see a prioritization of skills that will support students as
they move beyond high school.
Pennsylvania Common Core Standards
While CCSS were developed to be relevant, rigorous, and applicable to students
in all fifty states, not all fifty states adopted the standards as written. As the Pennsylvania
Department of Education (PDE) publicized, “On July 1, 2010, the State Board of
Education adopted improved standards in math and English to replace the original
standards adopted in 1999. The standards were based off the Common Core initiative”
(Pennsylvania Department of Education, 2013). PDE recognized the work that went into
the development of CCSS, but acknowledged that the state-specific standards that had
been developed and approved for Pennsylvania, referred to as Pennsylvania Common
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
17
Core (PA Core), were tailored to meet the specific needs of students within the
Commonwealth. Furthermore, PDE took steps to ensure that the public understood that
the standards were written to serve as guidelines that would assist local districts and their
personnel in developing their curriculum.
The Every Student Succeeds Act
The most current reauthorization of ESEA, known as the Every Student Succeeds
Act of 2015 (ESSA), was signed into law in 2015 by President Barack Obama. At this
point, Standards-Based Education Reform in the United States was now forty years old.
This latest reauthorization continued to emphasize the importance of increased rigor and
accountability.
In 2012, the Obama administration began granting flexibility to states regarding
specific requirements of NCLB in exchange for rigorous and comprehensive
state- developed plans designed to close achievement gaps, increase equity,
improve the quality of instruction, and increase outcomes for all students.
(USDOE, 2015)
While NCLB was written in such a way that it did not prevent the federal
government from promoting a specific set of standards to states, ESSA was written so
that the federal government was unable to influence a state’s adoption of a specific set of
standards in order to maintain a clear delineation between state and federal authority over
K-12 public education (Every Student Succeeds Act, 2015).
The flexibility of ESSA and the inherent shift back to state accountability and
oversight was touted by many advocates as a key triumph of the new law. The Center for
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
18
American Progress (CAP) noted that this flexibility would be an opportunity to empower
local school districts to become innovative in their approaches to accountability. They
stressed that this could be achieved by collaborating with stakeholder groups to critically
analyze their current testing structures. In doing so, this would allow them to design
better-aligned assessment systems that “include the smart and strategic use of formative
and interim tests that provide real-time feedback to inform instruction, as well as
high-quality summative tests that measure critical thinking skills and student mastery of
standards.” (Brown et al., 2016)
Publishers’ Response to Standards-Based Education Reform
To study the impact of Standards-Based Education Reform, the National Center
for Education Statistics (NCES) established the Trends in International Mathematics and
Science Study (TIMSS) in 1995. This study, which has been administered to students in
grade 4 and grade 8 every four years, assesses and compares student achievement and
educational experiences in math and science by collecting data from students, parents,
educators, and administrators (NCES, 2022). The United States has been participating in
this study since its inception, providing us with 24 years worth of data as of this writing.
The TIMSS Framework helps us to critically examine instructional materials,
instruction, assessment, and their alignment to each other in order to determine their
impact on student understanding and achievement. In working within the TIMSS
Framework, we focus on the following terms:
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
19
● Intended Curriculum: National, social, and educational contexts such as
standards documents, curriculum documents, vision statements, and educational
objectives.
● Implemented Curriculum: Home, school, teacher, and classroom contexts such as
classroom management, instruction, instructional materials used, and the ways in
which content is interpreted by teachers.
● Attained Curriculum: Student achievement and attitudes, most commonly
obtained through student test scores, and student belief measures. (NCES, 1995)
Textbook Content vs. Proficiency Exam Content
As Standards-Based Education Reform gained prominence in both federal and
state education policy and practice, K-12 textbook and instructional content publishers
took notice. Since textbooks were designed to be the instructional tools that help teachers
bridge the gap between the intended curriculum of an educational system and the
attained curriculum of an individual student, publishers recognized the importance that
Standards-Based Education Reform would have, and marketed their products
accordingly. In her analysis of mathematics textbooks utilized by countries all over the
world, Johanssen (2003) observed that, while many of the textbooks used within the
United States have a strong focus on contextual features and performance requirements,
the content covered within the textbooks was disproportionately matched to the content
that was on student proficiency exams, with the greatest areas of mismatch being in
arithmetic, measure, and algebra (p. 28). This is of great concern, as Johanssen goes on
to note that in the United States, teachers use textbooks in the least flexible manner,
preferring to navigate through them as a part of their implemented curriculum in order,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
20
lesson by lesson, using the information as presented, regardless as to whether that
information aligns to their intended curriculum or locally developed assessment.
Textbook Structure
In analyzing the response to Standards-Based Education Reform made by
textbook and instructional content publishers, an obvious data point comes to us in the
number of pages within mathematics textbooks. Overwhelmingly, publishers in the
United States include more pages within their mathematics textbooks than publishers
from other countries (Johanssen, 2003). Even with this inclusion of additional pages, the
structure of reform-oriented textbooks in the United States focused on including as much
content as possible within the textbook itself. However, in comparing textbooks to one
another, there is a great deal of variance in the specific content within each textbook
(Davis et al., 2014).
What this clearly shows is that while textbook and instructional content publishers
understood the importance of Standards-Based Education Reform on their products, they
were unable to develop an industry-wide approach to addressing the changes in content
and instruction that came out of this reform initiative. However, similar marketing
approaches were employed by publishers. Textbook and instructional content publishers
focused on their marketing efforts, adjusting language to show that their products were
aligned to CCSS or state-specific standards, however they made little changes to the
content itself, continuing to perpetuate the gap between the intended curriculum and the
attained curriculum. As Kaufman et al. (2021) acknowledge, “One critical reason for
this disconnect between state standards and instructional practice is that teachers’
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
21
curriculum materials—the textbooks and other materials that drive lesson content—are
frequently not aligned with state standards (Kaufman et al., 2020; Polikoff et al., 2020).
Standards Alignment
“The influence of the idea of alignment is evident today in the large number of
textbooks, assessment systems, and professional development materials that are explicitly
marketed as being aligned with state standards” (Hamilton et al., 2008). They go on to
note that while standards alignment is now the universally accepted measure of
endorsement for use by educators, “there is no widely accepted method for aligning
standards, curriculum, instruction, and assessments, nor is there a consensus on how to
determine if existing components are aligned” (Hamilton et al., 2008).
The implication is that while most textbook and instructional content publishers
market their materials as being aligned to either CCSS, or state-specific standards, there
is no mechanism in place for those publishers to determine the consistency and accuracy
of their alignment. The consistency and accuracy of their alignment, instead, falls to
individual districts as they review materials in consideration of their purchase and use
within their classrooms. However, most educators are not provided with adequate
training or professional development designed to help them analyze the consistency and
accuracy of perceived alignments. Rather, they look to documentation provided to them
from the publisher that explains their rationale for their alignment.
Furthermore, there is a gap present in the development of math textbooks and
instructional content regarding the connection between content standards, teacher’s
content knowledge, and their pedagogical knowledge (Copur-Gencturk et al., 2022). This
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
22
gap makes the connection between the intended curriculum and the attained curriculum
much harder to bridge if the materials used for the implemented curriculum are not
designed to consider this connection.
Educators’ Response to Standards-Based Instructional Materials
As Loeb et al. (2008) point out, teachers’ responses to Standards-Based Education
Reform and their actions in the classroom are central to students achieving the desired
attained curriculum. As such, they must be provided with high-quality, job-embedded
professional development that is designed to help them understand the intricacies of the
reform initiatives, the ways in which student achievement is assessed, and the ways in
which their instructional materials are aligned to reform initiatives. In doing so, we are
able to address the assumptions about Standards-Based Education Reform and the ways
in which teachers will respond in order to align and improve their instruction. Those
assumptions include:
● Assumption One: Teachers will pay attention to the reform and become
familiar with the standards and what they imply for practice.
● Assumption Two: Teachers will take the reform seriously, as will their
supervisors and other local leaders, who will exhort teachers to meet the
demands of the policy, and offer support, as needed.
● Assumption Three: Teachers will adjust their instruction to align with the
standards and associated assessments, including preparation for
assessment.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
23
● Assumption Four: Teachers will expect all of their students to
succeed—and believe that they are capable of succeeding.
● Assumption Five: Teachers will have access to appropriate professional
learning opportunities. (Loeb et al., 2008, p.4)
Teacher Professional Development
The time that is needed for teachers to be adequately trained on the
implementation of newly adopted standards is substantial. To be most effective, this
professional development should be provided in an ongoing manner so that teachers can
engage in new learning frequently, allowing them to have regular opportunities to
collaborate with their peers. This collaboration should include rich discussion around
designing instructional strategies that engage students and help them achieve mastery of
new concepts and skills brought forth from new standards, and a critical review of
instructional materials to ensure that they are aligned to the standards and will effectively
support the implemented curriculum. This can become a daunting task, as it may require
the adjustment of teacher schedules to ensure that sufficient time is dedicated towards
providing them with meaningful professional development.
There must be a clear delineation between understanding the impact of
Standards-Based Education Reform and the intended curriculum on student achievement,
the impact of teacher instruction and the implemented curriculum on student
achievement, and the impact of the instructional materials used by teachers as a part of
the implemented curriculum on student achievement. We have evidence that shows a
clear connection between teacher professional development for instructional strategies
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
24
and an increase in student achievement. As Chingos and Whitehurst (2012) show, there
is also evidence that shows that the instructional materials utilized by teachers as a part of
the implemented curriculum have a large impact on student achievement and the attained
curriculum. To drive this point home, they state:
Student learning occurs primarily through interactions with people (teachers and
peers) and instructional materials (textbooks, workbooks, instructional software,
web-based content, homework, projects, quizzes, and tests). The contexts within
which these interactions occur are surely important. The effectiveness of
teachers, the behavior of peers, and the instructional materials with which
students have the opportunity to interact are affected by layers of influences that
range from circumstances in the home, to leadership in the school, to the
international macro-economy and everything in between. But students do not
engage in instructional interactions with the governance mechanism for their
school or with state standards or with a school district’s evaluation system for
teachers or with collective bargaining agreements or with the leadership qualities
of their school principal - they learn by engaging in cognitive processes that are
triggered and shaped by interactions with people and instructional materials.
(Chingos & Whitehurst, 2012, p. 3)
Process for Adopting and Implementing New Standards in PA
In the Commonwealth of Pennsylvania, there is a formalized process for adopting
and implementing new standards. Once the standards’ language has been approved by
the Pennsylvania State Board of Education, the Pennsylvania Department of Education
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
25
(PDE), in collaboration with their 29 intermediate unit partners, will work together to
present the new standards language to educators, and provide training for classroom
implementation. This process ensures that all administrators and educators are aware of
the new standards, the manner in which they will be assessed, and are well-equipped to
provide high-quality instruction that addresses them within their classrooms.
As new standards are adopted in the Commonwealth of Pennsylvania, there is a
direct impact on the ways in which districts, school buildings and individual educators
are held accountable for their implementation and student’s understanding and mastery of
them. This occurs through a combination of educator evaluation tools, student
assessment tools, and school progress tools. Educators are required to address the
implementation of the newly adopted standards within the classrooms as the adoption is
directly tied to their individual annual evaluations.
As noted by Kaufman et al. (2021), “ By themselves, state academic standards do
not lead to major shifts in what teachers do in their classrooms.” In order to adequately
support teachers as they adjust their instruction in order to meet new standards, teachers
need access to high-quality, standards-aligned instructional materials as well as engaging,
job-embedded professional development that helps them to understand the most effective
and efficient ways in which to use those materials in their classrooms in order to help all
students achieve academic success. “This means that states and districts must continue
working to push use of standards-aligned materials, along with curriculum-focused
professional development that can improve use of those materials and learning outcomes”
(Kaufman et al., 2021). Without this training, teachers will struggle to accurately address
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
26
the standards in a comprehensive manner as they select instructional materials for their
implemented curriculum.
A Mindset Shift in Mathematics
The overarching disconnect between the intended curriculum, the implemented
curriculum, and the attained curriculum comes down to the fact that instructional
materials are not as flexible or as well-aligned to standards as teachers need them to be in
order to ensure that students master the concepts and skills outlined within their adopted
standards. It is at this point that we need to consider the goal of Standards-Based
Education Reform, and teachers’ responses to it. A central tenet of Standards-Based
Education Reform is that all students are capable of achieving mastery in difficult content
areas, and that rigorous standards should be adopted in order to ensure that students do
so.
This is a relatively recent shift in mindset. Traditionally, more advanced content
was taught only to students that needed to engage with that content, going all the way
back to the Ancient Egyptian and Babylonian Eras. Katz (2009) points to this clearly
with regards to Algebra instruction,
“In fact, throughout most of recorded history, algebra was only taught to the
“few”, not the “many”. So the concept of “algebra for all” is a very recent one
and one that will take a lot of work to make into a reality.”
While each ESEA reauthorization has attempted to ensure that instruction and
learning would be equitable, the statewide adoption of common standards has decreased
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
27
opportunities for personalized instruction designed to meet the needs of students as
teachers have felt the pressures of increased assessments for accountability purposes.
While the implementation of common standards in education was to ensure that
all students receive the same education regardless of where they live through the intended
curriculum, their use within the implemented curriculum has decreased instances of
personalized instruction tailored to students’ specific needs. This shift away from
personalized instruction is not a new phenomenon in the history of education in the
United States. It reflects a long-standing tension between the ideals of standardization
and the need for culturally responsive teaching practices that recognize and value the
diverse backgrounds and experiences of students.
History of Culturally Responsive Teaching in the United States
The roots of Culturally Responsive Teaching in the United States can be traced
back to the 1970s, before the publication of A Nation At Risk. Also referred to as
culturally appropriate, culturally relevant, culturally congruent, or culturally compatible
teaching, this pedagogy is designed to be highly responsive to the backgrounds and
identities of the individuals within a given classroom in order to tailor instructional
materials, examples, and scenarios so that students can personalize them to their own life
experiences. The most widely accepted formal definition of Culturally Responsive
Teaching, defined by Gay (2010, p. 31) is “using the cultural knowledge, prior
experiences, frames of reference and performance styles of ethnically diverse students to
make learning encounters more relevant to and effective for them.”
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
28
As teachers have struggled to ensure that students meet mandated achievement
benchmarks due to Standards-Based Education Reform, the gap between the intended
curriculum and the attained curriculum has become more prevalent. In response,
teachers have begun to recognize that students need instructional materials and
instructional experiences that are reflective of their personal, cultural experiences for the
implemented curriculum to be more effective in meeting their learning needs (Harmon,
2012). Engaging in Culturally Responsive Teaching has become an effective method for
ensuring that students not only understand the concepts and skills being taught in the
classroom, it helps to ensure that students are able to relate to those concepts and skills by
introducing them to students through culturally familiar scenarios and explanations that
allow them to engage in meaningful learning (Harmon, 2012). “It is a means for
improving achievement by teaching diverse students through their own cultural filters”
(Gay, 2010, p. 50). This pedagogical approach becomes a transformative experience for
students and teachers by empowering them to engage in a personalized instructional
experience during the implemented curriculum.
Culturally Responsive Teaching in Mathematics
Most educators are quick to see the connection between Culturally Responsive
Teaching and literacy-centric subjects, such as English Language Arts. However
Culturally Responsive Teaching is effective in all content areas, as demonstrated by
Matthews and López (2019), who sought to reinforce this point as they centered their
work on the importance of Culturally Responsive Teaching in mathematics. They found
that initially most teachers did not recognize the importance of Culturally Responsive
Teaching on academic goals, favoring the pedagogy to reinforce socio-engagement goals
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
29
for students. However, as teachers engaged more fully in Culturally Responsive
Teaching within mathematics classrooms they noticed increased levels of student
achievement among Latino students (Matthews & López, 2019). The emphasis on
designing learning experiences that are reflective of students’ lived experiences provide
them with additional conceptual resources that allow them to make stronger connections
to the implemented curriculum, thus increasing the amount of attained curriculum that
they experience.
While mathematics is often viewed as a culturally neutral subject, it is deeply
rooted in culture and history, making the need for culturally responsive teaching in
mathematics crucial to creating inclusive and equitable learning environments for all
students. By incorporating culturally relevant examples and activities into math
instruction, teachers can help students see the relevance and applicability of math in their
own lives and communities. Culturally responsive teaching in math can also help address
equity gaps by ensuring that all students have access to rigorous and meaningful math
instruction that is culturally relevant and responsive to their needs.
Comparing the current demographics of our teaching force to the demographics of
our student populations further highlights the need to engage in Culturally Responsive
Teaching.
The need for culturally responsive teaching is more pressing than ever before,
especially when you consider the deep demographic gaps between teachers and
students. A teaching workforce that remains overwhelmingly female, white,
middle-class, and monolingual is increasingly likely to teach students who are of
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
30
a different race, ethnicity, socio-economic status, language group, and so on.
(Muñiz, 2019, p. 6)
This disparity between those delivering instruction and those receiving instruction has not
gone unnoticed by policymakers.
Culturally Responsive Teaching in Pennsylvania
Many states are now recognizing the importance of providing teachers with
access to ongoing, high-quality, and job-embedded professional development that will
assist them in developing culturally responsive instructional experiences for students. In
Pennsylvania, the Commonwealth uses the term Culturally Relevant and Sustaining
Education (CR-SE) to refer to a set of agreed-upon standards and competencies that all
Pennsylvania educators are accountable for when working with students (Cole-Malott et
al., 2021). These competencies, built upon the 50-state survey of culturally responsive
standards completed by New America, ask educators to reflect upon their own cultural
lens, deepen their understanding of bias within the system as a whole, design educational
experiences that are reflective of students’ lived cultural experiences, and provide all
students with equitable and differentiated learning opportunities.
Each competency is accompanied by a set of “Questions to Stimulate
Reflexivity,” which educators can use to critically examine their current practices and
approaches in order to identify areas in which they can improve their practices. These
questions help educators to ensure that their instruction and instructional practices
“connect academics to their learners' daily lives, cultural backgrounds, challenges and
build upon the funds of knowledge of learners and their families” (Cole-Malott et al.,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
31
2021, p. 2). They are also designed to ask educators to critically examine the resources
that they use to assist in their instructional delivery so that students see positive and
accurate representations of themselves.
In evaluating instruction and instructional resources through a CR-SE lens,
Pennsylvania educators have a unique opportunity to help bridge the gap between the
intended curriculum, the implemented curriculum and the attained curriculum for their
students. In reviewing the standards of all 50 states, Muñiz (2019) observed that, while
most state standards generally include some form of Culturally Responsive Teaching
within their language, there are varying levels of depth and nuance to the language.
Figure 3 shows the ways in which the New America Culturally Responsive
Competencies are addressed within all 50 states:
Figure 3
What Culturally Responsive Competencies do State Teaching Standards Address?
Note: Culturally responsive competencies addressed in state teaching standards. (Muñiz, 2019)
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
32
The Impact of Reform on Culturally Responsive Teaching
Abdulrahim and Orosco (2020) note that historically, the focus on mathematics
instruction has not been to make a strong connection between students’ lived
experiences and math activities in order to promote math achievement. They feel that
this is due to the fact that the standardization of the curriculum has created an
instructional environment that is more reflective of the teacher and less reflective of
the individual learners. Abdulrahim and Orosco (2020, p. 2) state,
Despite reform agendas calling for the improvement of mathematics
education for all students, traditional approaches to instruction in favor of
basic skills instruction continue to dominate practice (Ellis and Berry, 2005).
The majority of mathematics education in the U.S. is based on standardized
curricula, pedagogy, and testing that is not normed on CLD (Culturally
Linguistic or Diverse) students (Bartell et al., 2017; Kress, 2005). This
one-size-fits-all approach separates students’ cultural strengths and
experiences from teaching and learning mathematics, which influences their
response to mathematics instruction (Leonard, 2008).
To assist in moving away from this standardized approach to instruction,
Brown and Crippen (2016) detail the utilization of the Growing Awareness Inventory
(GAIn) with preservice teachers. Designed as a structured observation protocol to
assist in the development of Culturally Responsive Teaching, the GAIn helps
educators to build their capacity to deliver Culturally Responsive Teaching by
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
33
engaging in continuous, reflective review of their instructional materials and
practices. Figure 4 shows the GAIn review process.
Figure 4
Growing Awareness Inventory Theory of Action
Note: The Theory of Action cycle of implementation from GAIn (Brown & Crippen, 2016, p. 136)
The GAIn is just one tool that can be utilized by educators to help them
implement Culturally Responsive Teaching within their implemented curriculum so that
their lessons and materials are reflective of the individuals that comprise the community
in which they teach. As each community is unique in its structure, demographics, and
culture (Smith & Ayers, 2006), so too should be a teacher’s approach to the implemented
curriculum for that community. In order for a teacher to be able to provide instruction
and utilize instructional materials that are reflective of their student population, they must
engage in reflective practices in order to understand the impact that their instruction and
instructional materials are having on student understanding and achievement.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
34
Culturally Responsive Teaching as a Pedagogical Practice
Howard and Rodriguez-Minkoff (2017) caution that Culturally Responsive
Teaching is not just a simple way of teaching or utilizing out-of-the-box units of
instructional materials. Rather, it is a pedagogical practice that is rooted in deeply
understanding the lived experiences of the students in a teacher’s classroom and actively
designing instruction and instructional materials that reflect those experiences. In order
to do so, Culturally Responsive Teaching must become a pedagogical practice that is
taught extensively to both pre-service and in-service teachers.
In order to ensure that all teachers are familiar with and actively include CR-SE as
a core component of their implemented curriculum, the Pennsylvania State Board of
Education has amended Chapter 49 of the Public School Code to include the mandatory
inclusion of CR-SE training as a requirement for pre-service teachers, as a component of
new teacher induction programs, and as a part of districts’ continuing education plans.
This guidance, released in April 2022, states,
The existing requirements of Chapter 49 further require that professional
education plans must address training in meeting the needs of diverse learners
(defined as students with limited English language proficiency or students with
disabilities), improving language and literacy acquisition, and closing the
achievement gap among students. (Molchanow, 2022, p. 117)
In order to do so, in-service educators are required to engage in continuing professional
education courses regarding CR-SE. The scope and depth of this training has not been
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
35
defined, nor have the strategies for resource evaluation to support Culturally Responsive
Teaching within the implemented curriculum inside individual classrooms.
Despite the growing awareness of the importance of culturally responsive
teaching, the scope and depth of training for educators in this area have not been clearly
defined. This lack of clarity can create challenges for teachers who want to implement
culturally responsive teaching practices in their classrooms. One possible solution to this
challenge is the use of open educational resources (OERs) that focus on culturally
responsive teaching. OERs can provide teachers with a variety of resources, such as
lesson plans, activities, and assessments, that have been designed and curated to support
culturally responsive teaching. OERs can also help teachers access a wider range of
perspectives and ideas, and allow for customization and adaptation to meet the needs of
their specific students and communities.
Implementing OER to Facilitate Culturally Responsive Teaching
In order to support students as they address Culturally Responsive Teaching
within their implemented curriculum, many teachers have begun to utilize Open
Educational Resources (OER) as their preferred instructional materials. The term, first
coined by UNESCO at their 2002 Forum on Open Courseware, is defined as, “learning,
teaching and research materials in any format and medium that reside in the public
domain or are under copyright that have been released under an open license, that permit
no-cost access, re-use, re-purpose, adaptation and redistribution by others” (UNESCO,
2002). Due to their ability to be adapted, teachers are able to easily personalize these
materials to better reflect the lived experiences of their students, making them more
relevant and engaging.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
36
Perceived Benefits of Using Open Educational Resources
When asked about the benefits of utilizing OER, teachers discussed many positive
aspects of their use (Tang, 2020). From feeling empowered as instructional experts who
had the capacity to deliver personalized instruction that was tailored to the specific needs
of individual student populations, to being able to have access to a larger repository of
materials with no impact to their classroom or district budgets, OER have become a
powerful asset to teachers looking to increase student achievement through Culturally
Responsive Teaching. Many see the implementation of OER as a transformative element
of their instruction.
Teachers now commonly utilize OER to create a culturally responsive classroom.
Garth and Sebastian (2020) highlight the increased flexibility and options that teachers
now have by using OER by stating:
With an ever-expanding library of digital content at their disposal - including
online publications, instruction materials, and institutional peer-reviewed assets instructors have countless pathways to embrace the entire community of learners
by designing culturally relevant classrooms that invite and incorporate every
student into the process. (Garth & Sebastian, 2020, para. 8)
Furthermore, there is now evidence that student performance has remained
unchanged or increased through the use of OER in the classroom (Tang et al., 2020).
Multiple studies from 2014 - 2019 show that the use of OER in K-12 classrooms has
allowed students of varying cultural backgrounds to maintain or increase their
achievement scores. Moreover, these studies also showed that students were more
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
37
motivated and engaged in their learning when they interacted with OER (Tang et al.,
2020).
Perceived Barriers to Increasing Student Achievement
While many teachers feel empowered to use OER as a component of their
implemented curriculum, most teachers report multiple barriers to the effective use of
OER in the classroom. These barriers, whether real or perceived, tend to outweigh the
reported benefits of OER on student achievement. Unfortunately, they prevent teachers
from engaging with instructional materials that may be able to be designed to be more
culturally reflective of individual students and increase student achievement (Tang,
2020).
The first identified barrier is related to adequate professional development
regarding the understanding of and intricacies surrounding OER use in the classroom.
Many teachers report that they only have a surface-level understanding of OER.
Teachers are uncomfortable in actively engaging in the work that is related to finding,
reviewing, and modifying OER to use in their classrooms (Tang, 2020).
The lack of professional development provided to teachers in the implementation
of OER also results in a lack of time to engage in a thorough review of the resources that
are found. As stated earlier in this review, teachers need consistent opportunities to
engage in professional development that helps them to understand the most effective and
efficient ways in which to review and use instructional materials in their classrooms in
order to increase the academic achievement of students. In order for OER use to become
a sustainable component of Culturally Responsive Teaching designed to increase student
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
38
achievement in the implemented curriculum, districts must provide adequate time for
professional development that is designed to assist teachers in learning the process of
efficiently finding OER that relate to their students’ needs, critically reviewing those
resources for rigor and quality, and strategically aligning those resources to the intended
curriculum (Kaufman et al., 2021).
Understanding the Impact of COVID-19 on Student Achievement
As conversations regarding student achievement, Standards-Based Education
Reform, and Culturally Responsive Teaching continue today, the discussion will naturally
turn towards the impact of COVID-19. The National Center for Education Statistics
(NCES) administers the National Assessment of Educational Progress (NAEP) every two
years. As stated on their website,
The National Assessment of Educational Progress (NAEP), first administered in
1969, is the largest continuing and nationally representative assessment of what
our nation’s students know and can do in subjects such as mathematics, reading,
science, and writing. Standard administration practices are implemented to
provide a common measure of student achievement. Teachers, principals, parents,
policymakers, and researchers all use NAEP results to assess progress and
develop ways to improve education in the United States. The results of NAEP are
released as The Nation’s Report Card, and are available for the nation, states, and
in some cases, urban districts. NAEP is a congressionally mandated project
administered by the National Center for Education Statistics (NCES), within the
U.S. Department of Education and the Institute of Education Sciences (IES).
(National Assessment of Educational Progress, 2022, para. 1)
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
39
Due to the COVID-19 pandemic, the NAEP was not administered in 2020.
Because of this, the results of the 2022 NAEP administration were the first to show the
impact of COVID-19 on student achievement in the United States. The results showed
the “largest score declines in NAEP mathematics at grades 4 and 8 since initial
assessments in 1990” (National Assessment of Educational Progress, 2022). In
comparing scores to 2019, student scores in grade 4 dropped an average of 5 points and
student scores in grade 8 dropped an average of 8 points. A sample of the headlines that
resulted from the report included:
● “Two Decades of Progress, Nearly Gone: National Math, Reading Scores Hit
Historic Lows” (Sparks & Tomko, 2022)
● “Nation’s report card: Massive drop in math scores, slide in reading linked to
COVID disruption” (Petrin, 2022)
● “NAEP ‘Nation’s Report Card’ Shows Steep Fall in Math Scores” (Mollenkamp,
2022)
● “Nation's Report Card shows how badly we failed kids during pandemic. Now
what do we do?” (Jacques, 2022)
The alarmist language that was so unique to A Nation At Risk appears to be used
as common language to discuss student achievement scores in the wake of COVID-19.
However, it is important to note that while the gap in student achievement has become
larger, due in part to school closures and interruptions to consistent approaches to
instructional delivery, there is compelling evidence that student achievement scores had
been persistent well before the onset of the pandemic.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
40
While the pandemic has certainly exacerbated existing inequities and highlighted
the deep-seated disparities in access to education and resources, the existence of the
achievement gap is not a new phenomenon. The gap in academic achievement between
students from different racial, ethnic, and socioeconomic backgrounds has persisted over
time, even when controlling for factors such as parental education and income. The
COVID-19 pandemic has certainly magnified the challenges faced by students and
schools, but it is not the root cause of the achievement gap. Rather, it is a symptom of the
systemic and structural inequities that have long plagued the US education system.
The US academic-achievement gap was first identified in 1966. Its persistence is
troubling. The possibility that COVID-19 could make it worse deserves focused
attention. The achievement gap costs the United States hundreds of billions of
dollars - and also exacts a long-term cost in social cohesion. This is a moment and a challenge - that calls for urgency and energy. (Dorn et al., 2020)
This quote by Dorn et al. (2020) highlights the troubling persistence of the
achievement gap in the US education system, and recognized the fact that the COVID-19
pandemic had the potential to exacerbate this long-standing issue. As previously
mentioned, the achievement gap has been identified as early as 1966 and has persisted
despite various efforts to address it. Dorn et al. (2020) take their research one step further
by also addressing the cost associated with addressing the achievement gap, suggesting
that it costs the US hundreds of billions of dollars and exacts a long-term cost in our
ability to develop the skills needed to strengthen our communities.
To address the achievement gap effectively, it is essential for us to examine
student achievement scores by different student groups. Such analysis can reveal
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
41
disparities in performance and highlight the need for targeted interventions to support
historically marginalized students. It is also crucial to recognize the intersectionality of
identities and experiences that contribute to the achievement gap. By analyzing
achievement data through an equity lens and developing evidence-based interventions,
educators and policymakers can work towards creating a more equitable education
system that supports all students in achieving their full potential.
NAEP Results by Student Group
In examining the 2022 results of the NAEP, most student groups had lower
average scores than they did in 2019. American Indian / Alaska Native students’ average
scores dropped by 4 points, Asian students’ average scores dropped by 7 points,
Asian/Pacific Islander students’ average scores dropped by 6 points, Black students’
average scores dropped by 7 points, Hispanic students’ average scores dropped by 7
points, Native Hawaiian / Other Pacific Islander students’ average scores dropped by 2
points, students of Two or More Races average scores dropped by 10 points, and White
students’ average scores dropped by 7 points (National Assessment of Educational
Progress, 2022).
Steep Declines in Algebra
In the Algebra content area, the subtopic with the steepest decline in scores is
Equations and Inequities, which dropped from 35% correct in 2019 to 26% correct in
2022. While NCTM recommended the prioritizing of developing a solid conceptual
foundation for skills before moving into the utilization of calculators or computers to
assist with the development of more involved problem-solving skills, (NCTM, 1989) as
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
42
was mentioned earlier, these results are indicative of the gap that exists between
education policy that is created in the wake of Standards-Based Education Reform and
the instructional practices that are adopted by teachers in the classroom.
Summary
Standards-Based Education Reform has become a deeply embedded reality of the
K-12 public education system in the United States for the past 40 years. As we look at
the history of Standards-Based Education Reform, there has been a clear response from
both the publishers of instructional content and the consumers of instructional content in
an effort to increase student achievement in order to meet reform expectations.
Instructional content publishers have gone to great lengths to market their
materials as being aligned to national and state content standards. Their documentation
that details this alignment has become a trusted source of information that many
education decision makers use to determine which materials to purchase for their
students. In the same vein, administrators and teachers have gone to great lengths to
understand new content standards and synthesize them into concepts and skills that can
be taught in classrooms. They spend countless hours working to determine the best ways
to present content to students from varied cultural backgrounds, and with varied lived
experiences so that they can all meet standardized proficiency benchmarks.
While this standardization makes monitoring student achievement easy to
quantify, it highlights gaps that exist between the intended curriculum that is set forth by
national or state standards and a district’s established curriculum, the implemented
curriculum that is taught within individual teachers’ classrooms, and the attained
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
43
curriculum that is evident through student achievement scores. In order to bridge this
gap between policy, practice, and performance, teachers look to pedagogical techniques
such as Culturally Responsive Teaching and more flexible instructional materials such as
Open Educational Resources to support the individual needs of their students.
The inclusion of Culturally Responsive Teaching pedagogy is a critical element in
all content areas, and not just those with a strong literature-based focus, such as English
Language Arts and Social Studies. Teachers are looking at the ways in which they can
develop culturally responsive instructional experiences for students in order to help them
increase their understanding and mastery of skills and concepts in content areas such as
Algebra I.
In Westmoreland County, educators and administrators are curious to understand
whether or not there is a correlation between instructional materials, Culturally
Responsive Teaching pedagogy, and increases in student achievement. Due to the fact
that Algebra I test scores have been historically low countywide, districts are looking at
the ways in which their intended curriculum and implemented curriculum are reflecting
their student populations in order to determine whether or not their Algebra I
instructional materials and strategies are culturally responsive to their student populations
and whether or not there is an impact on student achievement.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
44
Chapter III
Methodology
There is a need for school districts to separate their curriculum discussions into
defined categories to understand better how standards, instructional materials, and the
scope and sequence of instruction impact student achievement scores. In doing so,
districts can engage in a more comprehensive analysis of the impact of their intended
curriculum, implemented curriculum, and attained curriculum. When school districts
take the time to understand the connection between state-mandated academic standards,
the alignment and relevance of instructional materials, and the scope and sequence of the
instruction that the teacher provides, they can better adjust their efforts to ensure that
learning experiences are designed to be culturally responsive and personalized to the
individual student populations of each classroom.
The literature shows us that Standards-Based Education Reform was adopted in
the early 1980s to provide broad, top-down guidance to K-12 public schools in the United
States to increase academic rigor and develop a structured curriculum. While
Standards-Based Education Reform was designed to impact all content areas,
mathematics was a particular concern. Organizations such as the National Council of
Teachers of Mathematics provided detailed recommendations to overhaul K-12
mathematics education in the United States, including the adoption of a common, core
curriculum that would be utilized in all schools to ensure that all students would be
introduced to the same skills and concepts in mathematics.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
45
Another critical recommendation was to focus on equity in education and not just
on education reform. Researchers noted the importance of honoring the diverse
populations of K-12 public school districts by developing diverse learning experiences.
When educators create learning experiences that students feel reflect their unique cultures
and backgrounds, they can quickly connect to the content. This connection can result in a
deeper understanding and mastery of the concepts and skills presented within the
implemented curriculum and increase the student’s attained curriculum.
Purpose of Study
This study investigated the possible correlation between the historically low
Keystone Algebra I Scores and building principal and Algebra I teachers’ perceptions of
student identification with examples in the word problems found within their
instructional materials. To thoroughly investigate this topic, the researcher used a
mixed-methods embedded design in the form of a case study of three school districts that
represent Westmoreland County public schools. The following research questions guided
this case study:
1. What are the factors that impact student achievement in Algebra I?
2. What are the factors that impact instructional delivery in Algebra I?
3. What are the teacher and administrator perceptions of the impact of instructional
delivery on student achievement scores in Algebra I?
Qualitative data has been generated from district curriculum documents, standard
alignment maps, lesson plans, and instructional material reviews and policies to
determine how academic standards are addressed within the curriculum and addressed
through teacher instruction. Quantitative data has been generated from student
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
46
assessment scores, district budgets, and survey responses regarding instructional
materials to determine the amount spent on them, their alignment with Pennsylvania Core
Standard, and their cultural relevancy.
The desired outcome of this case study is that the data will show whether or not
the instructional resources currently being purchased and utilized within the classroom
effectively support student academic understanding and achievement. The results of this
case study will provide building administrators and Algebra I teachers with additional
guidance and support when determining which instructional materials may significantly
impact student achievement within their student populations.
Setting
The Pennsylvania General Assembly established intermediate units in 1970. As
the report “Intermediate Units in Pennsylvania: The Role of Educational Service
Agencies in Promoting Equity in Basic Education” states:
In 1970, the General Assembly passed Act 102, creating a system of 29
intermediate units (Us), which replaced the county superintendent offices as of
July 1, 1971. The lUs were mandated to develop a broad program of educational
services to be offered to public and nonpublic schools, including curriculum
development and instructional improvement services; educational planning
services; instructional materials services; continuing professional education
services; pupil personnel services; State and federal agency liaison services; and
management services. The lUs were created as instruments of federal, State, and
local education policies. They were empowered to create new services needed by
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
47
public and nonpublic schools. (Pennsylvania Joint State Government
Commission, 1997).
Each of the Commonwealth’s 29 intermediate units is now charged with
providing cost-effective and management-efficient programs designed to support school
districts, charter schools, and nonpublic schools and serves as the liaison between these
organizations and the Pennsylvania Department of Education (PDE). Intermediate units
have four significant areas of focus as service agencies. The first is Advancing
Professionals through high-quality training and professional development. Through this
area of focus, intermediate units work with over 52,000 educators each year in data
analysis, curriculum development, technology integration, school safety, and educator
certifications (Pennsylvania Association of Intermediate Units [PAIU], 2022). The
second area of Safe and Healthy Students is an area of focus that supports schools in
addressing individual students' behavioral, emotional, and mental health needs. These
supports include services such as partial hospitalization programs, alternative education
programs, student assistance programs, and student evaluations. These supports also
extend to implementing services designed to prevent school threats, such as creating
Crisis Response Teams (PAIU, 2022).
The third area of Serving Students is broken into two categories: Early Childhood
and Young Children and School Age Students. Through this service area, intermediate
units can provide educational opportunities to students from birth to 18 years of age.
These services include Early Intervention programs, online education programs, and
special education services such as speech therapy, occupational therapy, and behavior
intervention (PAIU, 2022). The fourth area of focus for intermediate units is Supporting
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
48
Schools. This service helps create purchasing consortiums so districts can take advantage
of lower pricing for health insurance, energy purchases, transportation, and administrative
services (PAIU, 2022). Each of these focus areas allows intermediate units to be
responsive to the needs of the member districts within their geographic region.
The Westmoreland Intermediate Unit is in Greensburg, Pennsylvania. Situated in
southwestern Pennsylvania, the WIU serves seventeen public schools, three career and
technology centers, one charter school, and forty-two nonpublic schools within
Westmoreland County. The WIU provides Curriculum Services, Executive Services,
Financial Services, Student Services, and Technology and Infrastructure Services to their
member schools. These services are delivered both in-person and online and are
designed to help districts to build their capacity to support their communities.
“The Curriculum Services Division's primary goal is to advance the practice of
teaching and learning to the schools and students we serve. Through innovative practice,
we strive to be the premier leading experts in curriculum services, operating as an
integral part of our local, regional, and state education community” (Westmoreland
Intermediate Unit, 2023). The Curriculum Services Division comprises a team of one
director, three supervisors, one support specialist, and one secretary. While this team is
small, its capacity to support districts continues to grow through its increased
involvement in local, state, and national educational initiatives.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
49
Participants
Three public school districts participated in this case study. The first is the
Greater Latrobe School District. Founded in 1960, the Greater Latrobe School District is
a midsized district in Westmoreland County. As stated on their district website,
The Greater Latrobe School District is in southwestern Pennsylvania at the
foothills of the Laurel Highlands. It is approximately 50 miles east of downtown
Pittsburgh and 30 miles southwest of Johnstown, Pennsylvania. U.S. Route 30
bisects much of the district, which encompasses the City of Latrobe, Youngstown
Borough, and Unity Township, with approximately 30,000 residents. The Greater
Latrobe area has diversified industries which afford job opportunities in specialty
steel, plastics, molds and casting ceramics, and brewing beverages. The local
hospital, four area colleges, the airport, and specialty stores and shopping areas
provide many forms of employment. (Greater Latrobe School District, 2022)
As of the 2021-2022 school year, Greater Latrobe School District reported a
district enrollment of 3,512 students in grades K-12. Of these students, 51.8% are male,
and 48.2% are female. The district states that 2.2% of their students are enrolled in their
Gifted program, 7% in Eastern Westmoreland Career and Technical Center, and 3% in
outside charter or cyber charter schools. Figure 5 shows the percentage of enrollments by
race/ethnicity for the 2021-2022 school year. Figure 6 shows a demographic breakdown
of the district’s student enrollments for the 2021-2022 school year. (Pennsylvania
Department of Education, 2022)
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 5
Greater Latrobe School District Percent Enrollment by Race/Ethnicity
Note: 2021 - 2022 Greater Latrobe School District student enrollments by race/ethnicity. Data retrieved
from Future Ready PA Index, December 2022
Figure 6
Greater Latrobe School District Percent Enrollment by Student Groups
Note: 2021 - 2022 Greater Latrobe School District student enrollments by student group. Data retrieved
from Future Ready PA Index, December 2022
50
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
51
The Greater Latrobe School District carefully reviews their Algebra I Keystone
Exam student achievement scores to work toward meeting Pennsylvania’s statewide
established proficiency goal. The Pennsylvania Department of Education has set the
statewide goal of having 71.8% of all students reach proficiency in Algebra I by 2030.
Districts work to increase their proficiency percentage annually incrementally to achieve
this goal. Figure 7 shows the proficiency percentages of all Greater Latrobe students
taking the Algebra I Keystone Exam from 2013 - 2022.
Figure 7
Greater Latrobe School District Algebra I Proficiency Percentages 2013 - 2022
Since the 2013-2014 school year, the Greater Latrobe School District’s highest
percentage area has been in the Basic category. There is a correlation between the
number of students in the Basic category and the overall number of students tested each
year. Figure 8 shows the total number of students tested annually from 2013 - 2022. The
district’s goal is to review its instructional materials and strategies to determine strategies
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
52
to increase student achievement and move students from the Basic category into the
Proficient category to achieve the 2030 statewide goal of 71.8% proficiency.
Figure 8
Greater Latrobe School District Total Number of Student Tested vs. Year Administered
The second district participating in this case study is the Monessen City School
District. The Monessen City School District is a small, urban district in Westmoreland
County. Serving the City of Monessen, the district spans approximately 2 square miles in
total. As stated on their district website, “Monessen, a very caring and unique school
district, relies on versatility in meeting the needs of our very special young people by
assisting them to become productive citizens and confident individuals in our own and
world community” (Monessen City School District, 2023).
As of the 2021-2022 school year, Monessen City School District reported a
district enrollment of 712 students in grades K-12. Of these students, 48.7% are male,
and 51.3% are female. The district states that 1.5% of their students are enrolled in their
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
53
Gifted program, 4% are enrolled in Mon Valley Career and Technical Center, and 7% are
enrolled in an outside charter or cyber charter schools. Figure 9 shows the percentage of
enrollments by race/ethnicity for the 2021-2022 school year. Figure 10 shows a
demographic breakdown of the district’s student enrollments for the 2021-2022 school
year (Pennsylvania Department of Education, 2022).
Figure 9
Monessen City School District Percent Enrollment by Race/Ethnicity
Note: 2021 - 2022 Monessen City School District student enrollments by race/ethnicity. Data retrieved from
Future Ready PA Index, December 2022
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
54
Figure 10
Monessen City School District Percent Enrollment by Student Groups
Note: 2021 - 2022 Monessen City School District student enrollments by student group. Data retrieved
from Future Ready PA Index, December 2022
The Monessen City School District also carefully reviews their Algebra I
Keystone Exam student achievement scores to work toward meeting Pennsylvania’s
statewide established proficiency goal. Figure 11 shows the proficiency percentages of
all Monessen City students taking the Algebra I Keystone Exam from 2013 - 2022.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
55
Figure 11
Monessen City School District Algebra I Proficiency Percentages 2013 - 2022
Historically, Monessen City School District has not had any students in the
Advanced category on the Algebra I Keystone Exam since 2013. Instead, most of their
students were in the Basic category until the Spring 2018 administration of the Keystone
Exam. At this point, we see the same percentage of students in the Basic and Below
Basic categories. From this point on, we see more students in the Below Basic category.
Figure 12 shows the total number of students tested yearly from 2013 - 2022. The district
is concerned about the increasing number of students entering the Below Basic category
and is investigating the reasons for this shift.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
56
Figure 12
Monessen City School District Total Number of Student Tested vs. Year Administered
The third district participating in this case study is the Yough School District. The
largest of the three districts participating, the Yough School District is a rural district that
spans 77 square miles and serves the Arona Borough, Madison Borough, Sewickley
Township, Smithton Borough, South Huntingdon Township, Sutersville Borough, and
West Newton Borough. As stated on their district website, “The Yough School District is
enthusiastic about the growing local industries, which is providing unlimited
opportunities for our students and families to gain meaningful careers. Employment,
coupled with local community and recreation organizations, allows individuals to be
actively involved, while preserving hometown pride” (Yough School District, 2020).
As of the 2021-2022 school year, Yough School District reported a district
enrollment of 1,805 students in grades K-12. Of these students, 50.1% are male, and
49.9% are female. The district reports that 2.4% of their students are enrolled in their
Gifted program, 6.9% are enrolled in Central Westmoreland Career and Technical Center,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
57
and 4.5% are enrolled in outside charter or cyber charter schools. Figure 13 shows the
percentage of enrollments by race/ethnicity for the 2021-2022 school year. Figure 14
shows a demographic breakdown of the district’s student enrollments for the 2021-2022
school year (Pennsylvania Department of Education, 2022).
Figure 13
Yough School District Percent Enrollment by Race/Ethnicity
Note: 2021 - 2022 Yough School District student enrollments by race/ethnicity. Data retrieved from Future
Ready PA Index, December 2022
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
58
Figure 14
Yough School District Percent Enrollment by Student Groups
Note: 2021 - 2022 Yough School District student enrollments by student group. Data retrieved from Future
Ready PA Index, December 2022
The Yough School District also carefully reviews their Algebra I Keystone Exam
student achievement scores to work toward meeting Pennsylvania’s statewide established
proficiency goal. Figure 15 shows the proficiency percentages of all Yough students
taking the Algebra I Keystone Exam from 2013 - 2022.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
59
Figure 15
Yough School District Algebra I Proficiency Percentages 2013 - 2022
Like the Greater Latrobe School District, Yough School District’s highest
percentage area has historically been in the Basic category since 2013. However, unlike
the Greater Latrobe School District, there is no direct correlation between the number of
students taking the Algebra I Keystone Exam and the percentage of students in the Basic
category. Figure 16 shows the total number of students tested yearly from 2013 - 2022.
The district is concerned with the percentage of students in the Basic category. They
have not been able to raise their Algebra I proficiency level beyond 33.9%, which
occurred in 2015. In recent years, their highest proficiency level was 28.3% in Spring
2021.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
60
Figure 16
Yough School District Total Number of Student Tested vs. Year Administered
Student demographic data, achievement score data, attendance data, and
disciplinary data from each of the three participating school districts were collected to
investigate this topic thoroughly. In addition, teacher and building administrator
perception data were contributed by teachers currently instructing Algebra I classes
within the three participating school districts and their building-level principals. All
participants volunteered to participate in this study, completing and submitting a
participant informed consent form.
Research Plan
As Standards-Based Educational Reform has become more prevalent in K-12
education in the United States, public school districts have relied on content publishers to
align their instructional materials to educational standards to help them increase student
achievement scores, investing large amounts of time and money into effectively using
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
61
these materials in classrooms. Districts have prioritized purchasing these materials in
Mathematics, which has been one of the most globally competitive content areas since
the 1980s. They hope these materials will help teachers better align their implemented
curriculum with the intended curriculum of the district. Unfortunately, the literature
shows us that, in many cases, content publishers are investing more effort into updating
their marketing materials to show that their instructional materials are aligned to content
standards instead of updating and aligning the instructional materials themselves.
As districts struggle to increase student achievement scores in mathematics, they
are now looking to supplement the instructional materials they purchase with open
educational resources (OER) that are more reflective of their student population. They
are also taking the time to evaluate the instructional materials they buy from content
developers to determine whether they are culturally reflective of their student population
and whether there is a correlation between culturally reflective instructional materials and
student achievement scores.
To explore this issue in Westmoreland County, the established research plan
included independent actions by the researcher and voluntary actions by participants from
districts within Westmoreland County. The researcher began by investigating the
Algebra I test scores in all seventeen public school districts in Westmoreland County
since 2013. This data was collected from the eMetric Data Interaction for Pennsylvania
Student Assessments system and was generated as a Group Summary report showing the
performance results of all students that took the Algebra I Keystone Exam during each
administration period from 2013 through 2022. This resulted in a total of 322 unique
reports showing the test results of 64,959 students.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
62
The researcher also analyzed current demographic data for all seventeen public
school districts in Westmoreland County as reported to PDE for the 2021 - 2022 school
year. This data was collected from the Future Ready PA Index, a website maintained by
PDE annually. As stated on PDE’s website, “The Future Ready PA Index illustrates
student and school success via three color-coded categories: academic performance,
student progress, and college and career readiness, providing the public with a more
comprehensive look at how Pennsylvania's schools are educating students” (Pennsylvania
Department of Education, 2023).
To gather perception data regarding the instructional materials being used in
Algebra I classrooms in Westmoreland County, building principals and Algebra I teachers
in Westmoreland County were given the opportunity to participate in a perception survey.
The survey for building principals (Appendix A) was 14 questions in length and covered
topics such as the amount spent on instructional materials annually, administrators’
awareness of OER, and their perception of the following:
● the alignment of purchased materials to PA Core Standards
● their teachers’ awareness of OER
● internal and external barriers toward the use of OER in the classroom
● the frequency in which teachers need to modify instructional materials to meet the
needs of students
● the frequency in which teachers need to develop their own instructional materials
to meet the needs of students
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
63
The survey for Algebra I teachers (Appendix B) was 20 questions in length and covered
topics such as teachers’ academic background, instructional environments, and their
perception of the multiple items relating to their instructional materials, including:
● the alignment of their 7th and 8th grade mathematics curriculum with PA Core
Standards
● the alignment of instructional materials being used with PA Core Standards
● the amount of professional development given to effectively use the instructional
materials being purchased by the district
● the amount of professional development given to effectively develop their own
instructional materials
● the impact of instructional materials on student achievement scores
The final step in the established research plan involved working with three
selected districts to complete the Culturally Responsive STEAM Scorecard (Appendix
C), developed by NYU Steinhardt, in order to determine the cultural responsiveness of
the primary instructional materials for Algebra I for each school district. During this
step, the researcher held a virtual meeting with each district's building principal and
Algebra I teachers to explain the purpose of the Culturally Responsive STEAM
Scorecard and how to use it. The groups were then given time to review their primary
instructional materials, evaluating them through the scorecard parameters and
determining individual scores for each of the four categories: Representation, Social
Justice, Teacher’s Materials, and Materials & Resources. Each district group was
allowed to complete the scorecard in a group setting or individually. The researcher then
created an average score for each category based on the individual scores provided.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
64
Based on the average score, the researcher was able to recommend to each district the
cultural responsiveness of the primary instructional materials their district purchased for
Algebra I.
Once all steps were completed, the researcher looked at the historic Algebra I test
scores for each of the three participating school districts, their 2021 - 2022 demographic
data, and their Culturally Responsive STEAM Scorecard results for their primary
instructional materials for Algebra I. This was done to determine if there is a correlation
between low test scores and the use of instructional materials that do not reflect the
district's student population. This research plan is designed to help districts determine
their instructional materials' efficacy and impact on student achievement in Algebra I.
This research plan meets the needs of the research problem because the research
will investigate the possible correlation between the historically low Keystone Algebra I
scores in Westmoreland County and building principal and Algebra I teachers’
perceptions of student identification with examples in the word problems found within
their instructional materials in Algebra I classrooms in Westmoreland County.
There are minimal costs associated with this Doctoral Capstone Research Project.
The main cost associated with it is the time and effort of the researcher. This work aligns
with her current work for the Westmoreland Intermediate Unit. The budget below shows
an allotment of 30% of the researcher’s annual salary. Should the work result in the
recommendation for districts to adjust their instructional materials, there may be a shift in
district resources allotted to both instructional materials and professional development.
As a result, one year of funding for the implementation of this study will be
supported through the general operating budget of the Westmoreland Intermediate Unit
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
65
under the function of Instruction for Regular Programs. The funding source will allocate
dollars from the Administrative Budget, as all administrators within the Westmoreland
Intermediate Unit are not in this category and are not assigned to specific instructional
levels. The Westmoreland Intermediate Unit is classified as a non-instructional building.
Therefore all expenditures within this budget are not split by individual schools, as you
see within a school district. All project costs fall under Mathematics Instruction for the
Curriculum Specialist Assignment job description.
To effectively analyze each district’s instructional materials for Algebra I, the
researcher must dedicate time to thoroughly understanding the resources being used,
educator perspectives, and student achievement data. This budget allots 30% of the
researcher’s time to this specific project, as the researcher has additional duties and
responsibilities that must be completed throughout the year. Budget line items include
the Supervisor’s salary and benefits such as health care, insurance, retirement,
unemployment, and worker’s compensation.
The indirect costs associated with this project, external of the researcher’s time,
include the use of a laptop and iPad, access to a Westmoreland Intermediate Unit-issued
Google Account, and the cost of secure digital storage on the Westmoreland Intermediate
Unit’s Amazon Web Services (AWS) server. As the researcher also utilizes all of these
indirect cost items to complete the other assigned duties and responsibilities, the amount
listed reflects 30% of the annual usage of each of these items.
The immediate cost associated with the project on the part of the individual
districts is minimal as well, resulting in approximately 2 ½ hours in salary and benefits
per participant. There may be a future cost to each district. Should the project result in
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
66
providing districts with a recommendation for adjusting their instructional materials, the
researcher will collaborate with each district to determine the budgetary impact of
implementing the recommendation. This will help districts to project the sustainability of
the recommendations made by the researcher.
Research Methods & Data Collection
The researcher studied this topic by using a mixed-methods embedded design in
the form of a case study of three school districts that are representative of all seventeen
Westmoreland County public school districts. This mixed-methods research allowed the
researcher to utilize one form of data to support the other. As stated by Mertler when
discussing mixed-methods research, “Its focus must always be on integration - that is, on
the idea of integrating not only multiple types of data but also perhaps integrating
theoretical perspectives, research design, research results, etc. Integration is the central
concept in this methodological approach to conducting research.” (pg. 140) Utilizing a
mixed-method research approach allowed the researcher to efficiently collect qualitative
and quantitative data as she progressed through her research. Once all data was collected,
she could begin looking for potential correlations between instructional materials, student
achievement scores, perceptions of those resources, and reasons why teachers utilize
specific resources within their classrooms.
The researcher utilized qualitative data from district curriculum documents,
standard alignment maps, lesson plans, and instructional material reviews and policies to
determine the ways in which academic standards are addressed within the curriculum and
addressed through teacher instruction. The researcher also used this qualitative data to
support the quantitative data collected from student assessment scores, district budgets,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
67
and survey responses regarding instructional materials to determine the amount being
spent on instructional materials, their alignment to Pennsylvania Core Standard and their
cultural relevancy and to show whether or not the resources that are currently being
purchased and utilized within the classroom are effective in supporting student academic
understanding and achievement.
This design enhanced the study by giving the researcher the opportunity to more
thoroughly explain the research and data that was collected and interpreted. It also
helped the researcher to better manage her time to ensure that everything was complete
within the defined timeline as established within the research plan. Data that was
available via public, openly available records, such as budget documents, student
achievement scores, and district policies were able to be accessed at any time once
released online. Building administrator and teacher perception survey data was collected
via Google Form. The data from the Culturally Responsive STEAM Scorecard was
collected via Google Sheet. This ensured that each district team and the researcher had
instant access to the results of their team’s scorecard.
Data Collection Timeline and Plan Described
The first research question, What are the factors that impact student achievement
in Algebra I?, was investigated from November 2022 through February 2023. In order to
thoroughly investigate this question, the researcher analyzed Algebra I Keystone Exam
scores, available local assessment scores, Future Ready PA Index Information, student
attendance and discipline records, and the results of the Culturally Responsive STEAM
Scorecard. Each of these items provided historical data for the three school districts
being studied within this case study.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
68
Keystone Exam scores and local assessment scores were obtained from the
Pennsylvania Value-Added Assessment System (PVAAS) website and the eMetric
website. All Future Ready PA Index information was obtained from the Future Ready PA
Index website. Attendance and discipline data was collected from the individual school
districts. Teachers and administrators completed the Culturally Responsive STEAM
Scorecard digitally.
The second research question, What are the teacher and administrator
perceptions of the impact of instructional delivery on student achievement scores in
Algebra I?, was investigated from November 2022 through January 2023. Due to the
nature of this study, the researcher was able to investigate this question at the same time
as the first research question. In order to thoroughly investigate this question, the
researcher analyzed the results of the building administrator and teacher perception
surveys.
These two surveys helped the researcher to understand the perceptions of both
teachers and administrators regarding the use of instructional materials in Algebra I
classrooms, their cultural responsiveness, and their potential impact on student scores.
Surveys were shared electronically via email, and results were automatically delivered to
the researcher once participants completed the survey.
The final research, question What are the factors that impact instructional
delivery in Algebra I?, investigated from March 2023 through April 2023. In order to
thoroughly investigate this research question, the researcher analyzed budget documents,
available professional development plans, instructional material adoption policies,
available instructional material standards alignment documentation, available district
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
69
curriculum map, and available scopes and sequences of instruction. Each of these items
helped the researcher to understand the correlation between the intended curriculum, the
implemented curriculum, and the attained curriculum. All documents were obtained
from district websites, or were requested from the participating districts if they had been
developed.
The researcher submitted proposal #PW22-007 to the Institutional Review Board
(IRB) at the Pennsylvania Western University before initiating the outlined research plan.
The IRB granted approval effective 9/20/22 with an expiration date of 9/19/23 (Appendix
D).
The data collection tools utilized for this study were available online at no cost,
therefore the fiscal implications of conducting the study were minimal. Participants
completed perception surveys at times that were most convenient to their daily schedules,
and worked on completing the Culturally Responsive STEAM Scorecard during
established collaborative planning time during their workday, ensuring that districts
would not accrue additional costs for participating in the case study.
Validity
Hendricks (2017) describes the validity of action research as the “trustworthiness”
of the research. Relying heavily on the criteria established by Lincoln and Guba (1985),
Hendricks (2017) reminds researchers that they must consider both the credibility and the
validity of their work. The four types of validity established by Lincoln and Guba (1985)
are
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
70
● Credibility: the plausibility of the research findings for the context that was
studied.
● Transferability: the extent to which results of a study are applicable to other
contexts and other individuals
● Dependability: the degree to which research results would replicate with the same
or similar participants and/or contexts.
● Confirmability: Showing that results are an accurate representation of what
occurred rather than the results of the researcher’s bias, motivation, or interest.
The researcher designed the research plan to address the credibility of the study
by utilizing two specific strategies recommended by Hendricks (2017): triangulation and
accurate data recording. In order to thoroughly investigate this topic, the researcher
analyzed data from multiple sources over a span of 13 years. In order to ensure that data
was recorded accurately, the data files were downloaded and utilized to ensure an
accurate transfer of information.
Transferability was addressed by providing a detailed description of the setting
and the steps of the study. By using a mixed-methods embedded design, the researcher
had the ability to provide a rich description of the setting, participants, interventions, and
research methods employed in the study. This will allow future researchers to apply this
work to other contexts.
In order to address dependability, and determine the degree to which this research
would replicate with the same or similar participants and/or contexts, the researcher
relied upon triangulation and providing a detailed description of the setting and steps of
the study. The researcher also engaged in an ongoing investigation with different
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
71
participants by selecting three representative school districts from across Westmoreland
County to participate in the case study.
Confirmability was also addressed within the research plan through triangulation
and accurate data recording. The amount of data collected necessitated the accurate
recording of all information gathered. In addition, providing a thick description of the
setting and study allowed the researcher to show that the results of the study are an
accurate representation of what occurred during the course of the study rather than the
results of the researcher’s own bias, motivations, or interests (Hendricks 2017).
Summary
The researcher designed a plan to investigate the possible correlation between the
historically low Algebra I test scores in Westmoreland County and the culturally
responsive nature of the instructional materials currently being used in Algebra I
classrooms. In order to investigate this possible correlation, the researcher analyzed data
from multiple sources, including student achievement scores over a period of eight years,
district demographic data as reported to PDE, district budget documents, instructional
material standards alignment documentation, district curriculum maps, and scopes and
sequences of instruction. In addition, perception data was collected and analyzed from
budiling administrators and teachers. Finally, participants engaged in the completion of a
Culturally Responsive STEAM Scorecard in order to determine the responsiveness of
their Algebra I instructional materials.
Chapter III provided an overview of the research problem identified within this
case study. It then provided a detailed explanation of the purpose, setting, participants,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
72
and research plan for this study. Finally, Chapter III discussed the methods of data
collection and validity.
Chapter IV will provide a thorough report of the data analysis that was conducted.
It will also share the results of this study. Finally, Chapter IV will provide a detailed
discussion of the interpretation of those results.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
73
Chapter IV
Data Analysis and Results
This case study aimed to determine the possible correlation between the
historically low Keystone Algebra I Scores in middle school and building principal and
Algebra I teachers’ perceptions of student identification with examples in the word
problems found within their middle school instructional materials. The research plan for
this case study was designed to determine if a correlation existed by examining the
relationship between district curriculum documents, standard alignment maps, lesson
plans, and instructional material reviews and policies to determine how academic
standards are addressed within the curriculum and addressed through teacher instruction
and comparing those documents to student assessment scores, district budgets, and
survey responses regarding instructional materials to determine the amount spent on
them, their alignment with Pennsylvania Core Standard, and their cultural relevancy.
This chapter presents an analysis of the data collected to determine whether or not
a correlation between student test scores and the perceived cultural responsiveness of
instructional materials in middle school Algebra I courses exists through the framework
of the identified research questions. This chapter also provides a detailed discussion of
the interpretations of the data concerning the research questions for each school district
identified within the case study.
Research Questions
1. What are the factors that impact student achievement in Algebra I?
2. What are the factors that impact instructional delivery in Algebra I?
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
74
3. What are the teacher and administrator perceptions of the impact of instructional
delivery on student achievement scores in Algebra I?
Data Analysis
Participants
Eighteen schools are designated as middle schools, junior high schools, or
intermediate middle schools within the 17 public school districts in Westmoreland
County. The building administrators and Algebra I teachers within those 18 buildings
were invited to participate in the perception surveys for this case study. Eleven building
administrators agreed to participate in the Building Principal Perception Survey and
signed the established consent form before completing the survey. All of the building
administration participants remained in the study, resulting in the completion of the
survey by all eleven participants. Six Algebra I teachers agreed to participate in the
Teacher Perception Survey and signed the established consent form before completing
the survey. All of the teacher participants remained in the study, resulting in the
completion of the survey by all six participants.
In addition, building administrators and Algebra I teachers from three districts
agreed to complete the Culturally Responsive STEAM Scorecard to review their primary
middle school Algebra I instructional materials. To access the Culturally Responsive
STEAM Scorecard, building administrators and teachers first agreed to participate in the
study and sign the established consent form. While all participants agreed, the teachers
from one of the three districts did not return the signed consent forms and could not
participate in the study. Due to this, the study only reflects the Culturally Responsive
STEAM Scorecard responses for two of the three case study districts.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
75
The Building Principal Perception Survey and the Teacher Perception Surveys
were administered via Google Forms. Survey responses did not collect identifying
information, such as participants’ names, to ensure confidentiality. Their email addresses
were also not collected upon submission. Once all responses were collected, each
participant was assigned a letter of the alphabet and a corresponding number to associate
with the reported data.
The researcher speculates that the lack of participation by teachers is due to the
current political climate in education and the connection of this case study to the phrase
“culturally responsive teaching.” The researcher further speculates that external factors
impacting teachers in one of the case study districts, such as the unexpected death of a
student, staffing shortages, and changes in district administration, may have caused
teachers to struggle to find the time to complete the Culturally Responsive STEAM
Scorecard.
Data Analysis Procedure
The research aimed to investigate the correlation between historically low
Keystone Algebra I Scores in Westmoreland County middle schools, principals' and
teachers' perceptions of instructional materials alignment and efficacy, training and
support provided to teachers, and the cultural responsiveness of the instructional
materials being used in middle school Algebra I classrooms. Quantitative data included
exam scores, assessment data, budget information, and standards alignment from three
school districts. Data sources included eMetric, PVAAS, PDE, the Future Ready PA
Index, district and content vendor websites. Qualitative data included surveys and the
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
76
Culturally Responsive STEAM Scorecard. The data were analyzed to determine if a
correlational relationship existed.
The quantitative data that was gathered for this case study was compiled by the
researcher from multiple sources. This data includes overall student Algebra I Keystone
Exam scores for all 17 public school districts in Westmoreland County from 2012 - 2022,
countywide Algebra I Anchor Analysis based on student achievement scores from 2015 2021, local assessment data for Algebra I from seven of the 17 public school districts in
Westmoreland County from 2019 - 2022, Future Ready PA Index data for all public
school districts in Westmoreland County from 2001 - 2022, annual budget information
for all 17 public school districts in Westmoreland County from 2016 - 2023,
Comprehensive Plans for all 17 public school districts in Westmoreland County from
2016 - 2026, and Pennsylvania Common Core Standards alignment information for the
primary Algebra I instructional resources used by Greater Latrobe School District,
Monessen City School District, and Yough School District. This data was gathered from
the eMetric Data Interaction for Pennsylvania Student Assessments web-based system
(eMetric), the web-based Pennsylvania Value Added Assessment System (PVAAS), the
Pennsylvania Department of Education’s website (PDE), PDE’s web-based Future Ready
PA Index, individual district websites, and individual content vendor websites.
The quantitative data gathered for this case study was then organized by the
researcher for analysis according to the research plan outlined in Chapter III. This
allowed the researcher to discover patterns and trends in student achievement scores over
time, determine funds allocated to instructional materials, and determine whether the
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
77
materials used by Greater Latrobe School District, Monessen City School District, and
Yough School District are aligned with Pennsylvania Common Core Standards.
The eleven building administrators voluntarily submitted the first qualitative data
gathered for this case study through a Google Form. The six teachers voluntarily
submitted the second set of qualitative data for this case study through a Google Form.
This format helped the researcher to compile the data to facilitate analysis according to
the research plan outlined in Chapter III. Participant responses were used to support the
researcher's efforts to determine a potential correlation between instructional materials,
Culturally Responsive Teaching pedagogy, and increases in student achievement.
First, the researcher gathered the total number of students tested for each testing
window from the 2012 - 2013 school year through the 2021 - 2022 school year. Then the
researcher determined the percentage of students with scores in the Below Basic, Basic,
Average, and Proficient ranges. From there, the scores were separated into categories by
Algebra I Assessment Anchors separated the scores into categories to determine student
achievement within each Anchor. The change in these percentages by district is shown
below. Figure 17 shows the percentage of each district’s scores in each performance
category in the winter of 2012. Figure 18 shows the percentage of each district’s scores
in each performance category in the spring of 2022.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 17
Group Summary: Winter 2012 Performance Keystone, Algebra I
78
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
79
Figure 18
Group Summary: Spring 2022 Performance Keystone, Algebra I
The researcher next gathered demographic data from all 17 public school districts
in Westmoreland County for the 2021 - 2022 school year, as reported to PDE. This
demographic data allowed the researcher to separate each district’s total student
enrollments by race/ethnicity, as shown in Table 1 The researcher was also able to
separate each district’s enrollments by the following subgroups:
● Economically Disadvantaged, as shown in Figure 19
● English Language Learners (ELL), as shown in Figure 20
● Special Education, as shown in Figure 21
● Gender (male/female), as shown in Figure 22
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
80
Table 1
All Districts’ Total Student Enrollments by Race/Ethnicity
District
American
Indian /
Asian
Alaska
Native
Black
Native
Hawaiian
Two or
/ other Hispanic White More
Pacific
Races
Islander
Belle Vernon Area SD
0%
1.3%
3.2%
0%
2.9%
87.3%
5.2%
Burrell SD
0%
0.5%
1.9%
0.1%
1.5%
91.7%
4.4%
Derry Area SD
0.1%
0.5%
1.7%
0%
0.6%
93.6%
3.5%
Franklin Regional SD
0.1%
7.8%
1.8%
0.1%
2.5%
83.7%
4.1%
Greater Latrobe SD
0.2%
0.9%
1%
0%
1.8%
9%
3.6%
0%
0.3%
6.3%
0.2%
3.2%
80.3%
9.7%
Hempfield Area SD
0.1%
1.1%
1.2%
0.1%
1.8%
91.5%
4.3%
Jeannette City SD
0.7%
0%
14.2%
0%
3.1%
62.4%
19.6%
Kiski Area SD
0.1%
0.2%
6.3%
0%
1.5%
90.7%
1.1%
Ligonier Valley SD
0.1%
0.1%
0.6%
0.1%
1.4%
95.4%
2.3%
Monessen City SD
0%
0.7%
37.5%
0%
3.5%
37.2%
21.1%
Mount Pleasant Area SD
0%
0.1%
0.7%
0.1%
0.6%
95%
3.5%
New Kensington-Arnold SD
0.3%
0.4%
32.9%
0%
3.6%
43.7%
19.2%
Norwin SD
0.1%
1.5%
1%
0%
2%
91.4%
4%
0%
1.1%
1.8%
0.2%
1.1%
92.7%
3.1%
0.1%
0.4%
0.7%
0%
0.9%
94.3%
3.7%
0%
0.1%
1.4%
0%
0.9%
93.3%
4.3%
Greensburg Salem SD
Penn-Trafford SD
Southmoreland SD
Yough SD
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 19
% Enrollments Economically Disadvantaged by District
81
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 20
% Enrollments ELL by District
82
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 21
% Enrollments Special Ed. by District
83
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
84
Figure 22
% Enrollments by Gender (M/F) by District
Next, the first group of participants submitted their qualitative data by accessing
the Building Principal Perception Survey via Google Forms. This allowed the researcher
to quickly compile and organize the data collected by the participants to determine their
perception of the instructional materials used in the Algebra I classrooms in their
buildings. Data was collected from this survey by utilizing Open-Ended Questions,
Intervals Scales, and Ordinal Scales. The questions from the Building Principal
Perception Survey are shown in Table 2 and in Appendix A.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
85
Table 2
Building Principal Perception Survey Questions
Building Principal Perception Survey Questions
To what degree are you aware of Open Educational Resources (OER)?
To what degree are teachers aware of Open Educational Resources (OER)?
How many administrators in your building feel adequately informed about OER?
How many teachers in your building feel adequately informed about OER?
What are the internal factors that cause teachers to not use OER?
What are the external factors that cause teachers to not use OER?
How regularly do teachers modify instructional resources to better meet their students’
needs?
How regularly do teachers develop their own original instructional resources to better
meet their students’ needs?
What percentage of purchased instructional materials are accurately aligned to
Pennsylvania Core Academic Standards?
How often do teachers evaluate the efficacy of their purchased instructional materials?
How often do administrators evaluate the efficacy of their purchased instructional
materials?
How much money is spent annually on purchased instructional materials?
How much money is spent annually on teacher professional development designed to
help teachers utilize purchased instructional materials in their classrooms?
How much money is spent annually on teacher professional development designed to
help teachers curate, evaluate, and implement OER in their classrooms?
The same process was used for the second group of participants as they submitted
their qualitative data by accessing the Teacher Perception Survey via Google Forms.
Again, this allowed the researcher to quickly compile and organize the data the
participants collected to determine their perception of the instructional materials being
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
86
used within their Algebra I classrooms. Data was collected from this survey by utilizing
Short Answer Questions and Likert Scale questions. The questions from the Teacher
Perception Survey are shown in Table 3 and Appendix B.
Table 3
Teacher Perception Survey Questions
Teacher Perception Survey Questions
Which best describes your education level?
In which District do you teach Algebra I?
In which building do you teach Algebra I?
Generally speaking, do you believe that the 7th or 8th grade mathematics curriculum
within your district is well-aligned to the PA Core Standards for Mathematics?
Generally speaking, do you believe that the 7th or 8th grade mathematics instructional
materials used within your district are missing critical standards addressed on the
Algebra I Keystone Exam?
I believe that the physical and digital instructional materials purchased and used with
our students were well researched and help us to prepare students for success on the
Algebra I Keystone Exam.
I have been given plenty of professional development focused on how to effectively
use the instructional materials provided to me with my students.
I have been given plenty of professional development focused on how to effectively
develop my own instructional materials with my students.
I believe that the instructional materials used in my classroom have a direct impact on
my students’ achievement scores.
In terms of the physical instructional materials used with students, how would you
describe the way in which the skills addressed prepare students for the Algebra I
Keystone Exam?
In terms of the digital instructional materials used with students, how would you
describe the way in which the skills addressed prepare students for the Algebra I
Keystone Exam?
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
87
I believe that the instructional materials used accurately reflect my student population.
I believe that my students can see themselves and their life experiences in their
instructional materials in my class.
I believe that when students are able to see themselves in their instructional materials
they will do better on assessments.
I have been adequately trained in understanding the culture and values of my student
population.
I have been adequately trained to evaluate instructional materials for cultural relevance.
I have been adequately trained to modify existing resources or develop new resources
that are culturally relevant to my student population.
I am aware of Open Educational Resources (OER) and use them within my learning
environment.
My colleagues are aware of Open Educational Resources (OER) and use them in their
learning environments.
My building administrators are aware of Open Educational Resources (OER) and
encourage their use.
The final data collection step was to have the middle school Algebra I teachers of
the three representative school districts in Westmoreland County complete the Culturally
Responsive STEAM Scorecard to evaluate their primary Algebra I instructional
materials. The Culturally Responsive STEAM Scorecard collects data utilizing a Likert
Scale. It evaluates teachers’ perceptions of the cultural responsiveness of their
instructional materials in four categories: Representation, Social Justice, Teachers’
Materials, and Materials & Resources. During this step, the researcher met with the
building principal and Algebra I teachers of Greater Latrobe School District, Monessen
City School District, and Yough School District to explain the components of the
Culturally Responsive STEAM Scorecard, how to use the Culturally Responsive STEAM
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
88
Scorecard to evaluate their primary Algebra I instructional materials, and to establish the
timeline for completion and submission of the scorecard. Once each district’s group of
teachers compiled the data, the researcher created an average score for each statement in
all four categories and a final score for each. The statements from the Culturally
Responsive STEAM Scorecard are shown in Table 4 and Appendix C.
Table 4
Culturally Responsive STEAM Scorecard
Representation Statements
The curriculum centers the origins of science, technology, engineering, arts and/or
math within BIPOC communities (Black, Indigenous, people of color).
The curriculum elevates mathematicians, artists, and/or scientists with historically
marginalized identities (i.e. non-binary or trans people, women, people of color, people
with disabilities, working class people, multilingual people) and their discoveries.
The curriculum acknowledges and/or incorporates the expertise of diverse
communities, their cultures, and their historical and/or contemporary experiences.
The curriculum has photos/pictures, names, scenarios, and text that reflect the
experiences and interests of students of color in your community.
The curriculum elevates not just “scientists, artists, and mathematicians”, but the
everyday users of math, science, arts, technology, and engineering.
The curriculum affirms the multiple forms of communication or language systems
during mathematical and scientific argumentation rooted in historically marginalized
cultures.
Social Justice Statements
Curriculum highlights and affirms the knowledge systems of Indigenous,
Black/African, Brown, and non-Western conceptions of science, technology,
engineering, arts, and math (such as interdependence, sustainability, and continual
change).
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
89
The curriculum and instructional activities promote or provoke critical questions about
science, technology, engineering, arts, and/or math and the societal status quo.
The curriculum and instructional activities present alternatives or allow students to
present alternatives about the role science, technology, engineering, arts, and/or math
plays in existing social structures.
The curriculum presents multiple understandings of a scientific, mathematical, or
artistic concept or theory, especially highlighting points of view from marginalized
people/communities.
The curriculum provides avenues for students to see STEAM as a way to understand
and improve their world, take actions that combat inequity or promote equity, and
connect learning to social, political, and/or environmental concerns.
The curriculum presents social situations and problems not as individual problems but
as embedded within a societal and/or systemic context.
The curriculum encourages students to critically examine dominant knowledge systems
as scientists, mathematicians, and artists.
Teachers’ Materials Statements
The authors of the teachers’ materials are people of diverse identities (race/ethnicity,
gender, other identities).
Guidance is provided to teachers on being aware of one’s biases, assumptions, and the
gaps between one’s own culture and students’ cultures (implicit bias).
The teachers’ materials ask teachers to reflect on their own practices and experiences
learning STEAM subjects and critique them through the lens of cultural
responsiveness.
Guidance is provided on combating the legacy of STEAM education related trauma
amongst historically marginalized communities and on designing healing and joyful
STEAM experiences.
Guidance is provided on using students’ everyday lives as the starting point for
learning, and making real-life connections between academic content and the local
neighborhoods, culture, environment, community issues, and current events.
Guidance is provided on creating opportunities to meaningfully engage students’
families to enhance lessons.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
90
Guidance is provided on appropriately using Indigenous and non-Western resources to
understand math and science, including oral histories, legends, and community
knowledge.
Guidance is provided on engaging students in culturally responsive experiential
learning activities.
Guidance includes, for applicable lessons, a range of possible student responses that
could all be valid, given the range of student experiences and perspectives.
Guidance is provided on how to design lessons or engage in conversations that use
STEAM to disrupt power inequities or create opportunities for students to practice
disruption.
Guidance is provided on planning opportunities for students to create and teach lessons
to their peers.
Guidance is provided to help teachers foster environments of collaboration that go
beyond group work by meshing individual's assets, resources, and strengths (such as
various ways of knowing, doing and being) to advance group and individual learning.
Materials & Resources Statements
The curriculum provides opportunities for students to use accessible technology and
manipulatives to explore STEAM concepts in ways that reflect a variety of ways of
“doing.”
Issues of social justice, equity, and Black, Indigenous, and people of color’s
experiences and contributions are reflected in homework/classroom assignments and
assessments.
The curriculum incorporates the arts as a tool to center experiences within STEM and
innovation.
Curriculum rigor is not dependent on access to resources, materials and technology that
students and schools may not have. In other words, the curriculum materials are fully
accessible; all resources, materials and technology options are rigorous and interesting.
(Ex. If students can engage curriculum materials with a computer or paper, the paper
materials should be just as rigorous, interesting, and engaging as using the computer).
The resources, materials and technology referenced in lessons include resources from
BIPOC communities (Black, Indigenous, people of color).
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
91
As stated in Chapter III, the researcher looked at the historic Algebra I test scores
for each of the three participating school districts, their 2021 - 2022 demographic data
and their Culturally Responsive STEAM Scorecard results to effectively answer the three
research questions established for this case study.
Results
Research Question 1
The first research question in this case study aims to answer “What are the factors
that impact student achievement in Algebra I?” After student assessment data was
compiled, scores were disaggregated into achievement categories and by Algebra I
Assessment Anchors. District demographic data were also disaggregated into the
subgroups that PDE uses for reporting. These subgroups are consistent with existing
Civil Rights Data Collection standards established by the United States Department of
Education.
To thoroughly investigate this question, student assessment scores for Greater
Latrobe School District, Monessen City School District, and Yough School District were
further disaggregated by combining two or more standard subgroups that PDE uses for
reporting to create unique subgroups. Those distinctive subgroups include:
● Ethnicity and Gender
● Ethnicity and Economically Disadvantaged
● Ethnicity and English Language Learners
● Ethnicity and IEP
The unique subgroups were then analyzed in comparison with the results of the
Culturally Responsive STEAM Scorecard for Greater Latrobe School District and Yough
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
92
School District. To determine a correlation between student achievement scores and the
results of the Culturally Responsive STEAM Scorecard, the researcher focused on
Algebra I Keystone performance levels for middle school students, disaggregated by
ethnicity, for Spring 2022. Table 5 shows those results below for Greater Latrobe School
District, Monessen City School District, and Yough School District.
Table 5
Group Summary, Performance Levels by Grade 08 and Ethnicity
Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022
Best Score
Below
Basic
Basic Proficient
Advanced
Number
Administration Tested
Mean
SS
Greater Latrobe SD
Spring 2022
383
1494.3
12.5
45.7
25.1
16.7
Grade 08
Spring 2022
111
1545.4
0
11.7
40.5
47.7
Monessen City SD
Spring 2022
49
1422.7
75.5
20.4
4.1
0
Grade 08
Spring 2022
11
1449.5
54.5
27.3
18.2
0
Yough SD
Spring 2022
183
1476.6
17.5
53
24.6
4.9
Grade 08
Spring 2022
41
1503
2.4
39
43.9
14.6
Group
%
%
%
%
Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022, Grade 08 and Black
Best Score
Group
Number
Administration Tested
Mean
SS
Below
Basic
%
Basic Proficient
%
%
Advanced
%
Greater Latrobe SD
Grade 08
Spring 2022
0
0
0
0
0
0
Monessen City SD
Grade 08
Spring 2022
3
1425.3
66.7
33.3
0
0
Yough SD Grade 08
Spring 2022
0
0
0
0
0
0
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
93
Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022, Grade 08 and Hispanic/Latino
Best Score
Group
Number
Administration Tested
Mean
SS
Below
Basic
%
Basic Proficient
%
Advanced
%
%
Greater Latrobe SD
Grade 08
Spring 2022
1
1591
0
0
0
100
Monessen City SD
Grade 08
Spring 2022
3
1435
100
0
0
0
Yough SD Grade 08
Spring 2022
0
0
0
0
0
0
Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022, Grade 08 and White
Best Score
Below
Basic
Basic Proficient
Advanced
Number
Administration Tested
Mean
SS
Greater Latrobe SD
Grade 08
Spring 2022
108
1544.4
0
12
41.7
46.3
Monessen City SD
Grade 08
Spring 2022
6
1469.2
33.3
33.3
33.3
0
Yough SD Grade 08
Spring 2022
39
1502.1
2.6
38.5
46.2
12.8
Group
%
%
%
%
The researcher compared this information to the Culturally Responsive STEAM
Scorecard results for Greater Latrobe School District. The primary Algebra I
instructional material utilized by the middle school Algebra I teachers is the 8th Grade
Algebra edition of Go Math by Houghton Mifflin Harcourt. Overall, teachers in the
Greater Latrobe School District felt satisfied that the curriculum of their primary Algebra
I instructional materials adequately represented diverse communities, including Black,
Indigenous, and People of Color (BIPOC), those with historically marginalized identities.
Furthermore, they are satisfied that the curriculum of their primary Algebra I
instructional materials incorporates the expertise of these diverse and historically
marginalized communities and elevates those communities throughout their
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
94
student-facing materials. Due to this, the total score for the Representation category was
2.
In reviewing the Social Justice category, teachers in the Greater Latrobe School
District were not satisfied with how their primary Algebra I instructional materials
addressed concepts such as interdependence and continual change. They were also not
satisfied that the materials’ instructional activities adequately presented alternatives or
allowed students to present alternatives about the role science, technology, engineering,
arts, and/or math plays in existing social structures. Due to this, the total score for the
Social Justice category was -5.
The only category where teacher opinions varied greatly was the Teachers’
Materials category. Some teachers were very satisfied that the teachers’ materials for
their primary Algebra I instructional materials adequately provided guidance to teachers
on being aware of one’s biases, assumptions, and the gaps between one’s own culture and
students’ cultures that the materials ask teachers to reflect on their own practices and
experiences learning STEAM subjects and critique them through the lens of cultural
responsiveness and that the materials provide guidance on using students’ everyday lives
as the starting point for learning and making real-life connections between academic
content and the local neighborhoods, culture, environment, community issues, and
current events. Others felt that the teachers’ materials were very lacking in these areas.
Due to this, the total score for the Teachers’ Materials category was -1.5.
Finally, the Greater Latrobe School District teachers had the most confidence in
the materials and resources associated with their primary Algebra I instructional
materials. Overall, they felt satisfied that the materials and resources provide
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
95
opportunities for students to use accessible technology and manipulatives to explore
STEAM concepts in ways that reflect a variety of ways of “doing,” that issues of social
justice, equity, and Black, Indigenous, and people of color’s experiences and
contributions are reflected in homework/classroom assignments and assessments, and that
the rigor of the curriculum is not dependent upon the access to resources, materials and
technology that students and schools may not have. Due to this, the total score for the
Materials & Resources category was 2.5, which was the highest score given by teachers
in the Greater Latrobe School District. Figure 23 shows the scorecard results for the
Greater Latrobe School District.
Figure 23
GLSD Culturally Responsive STEAM Scorecard Results
The researcher then compared the Algebra I Keystone performance levels for
middle school students, disaggregated by ethnicity, for Spring 2022 to the results of the
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
96
Culturally Responsive STEAM Scorecard for the Yough School District. The primary
Algebra I instructional material utilized by the middle school Algebra I teachers is a
district-specific, OER-based Algebra I Flexbook that was customized on CK-12’s digital
platform. Overall, teachers in the Yough School District felt satisfied that the curriculum
of their primary Algebra I instructional materials adequately represented diverse
communities, including Black, Indigenous, and People of Color (BIPOC), those with
historically marginalized identities. Furthermore, they are satisfied that the curriculum of
their primary Algebra I instructional materials incorporates the expertise of these diverse
and historically marginalized communities and elevates those communities throughout
their student-facing materials. Due to this, the total score for the Representation category
was 3.5, which was the highest score given by teachers in the Yough School District.
In reviewing the Social Justice category, teachers in the Yough School District
were less satisfied with how their primary Algebra I instructional materials addressed
concepts such as interdependence and continual change. It was unclear whether the
materials’ instructional activities adequately presented alternatives or allowed students to
present alternatives about the role science, technology, engineering, arts, and/or math
plays in existing social structures. Due to this, the total score for the Social Justice
category was -1.
As we also saw in the Greater Latrobe School District, the only category where
teacher opinions varied greatly in the Yough School District was the Teachers’ Materials
category. Some teachers were very satisfied that the teachers’ materials provided
guidance on using students’ everyday lives as the starting point for learning, and making
real-life connections between academic content and the local neighborhoods, culture,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
97
environment, community issues, and current events, and to help teachers foster
environments of collaboration that go beyond group work by meshing individual's assets,
resources, and strengths to advance group and individual learning. However, other
teachers felt that the teachers’ materials were unclear in their ability to adequately
provide guidance to teachers on being aware of one’s biases, assumptions, and the gaps
between one’s own culture and students’ cultures that the materials ask teachers to reflect
on their own practices and experiences learning STEAM subjects and critique them
through the lens of cultural responsiveness, and in how to design lessons or engage in
conversations that use STEAM to disrupt power inequities or create opportunities for
students to practice disruption. Due to this, the total score for the Teachers’ Materials
category was 1.
Finally, the teachers in the Yough School District felt satisfied that the materials
and resources for their primary Algebra I instructional materials provide opportunities for
students to use accessible technology and manipulatives to explore STEAM concepts in
ways that reflect a variety of ways of “doing,” and that the rigor of the curriculum is not
dependent upon the access to resources, materials and technology that students and
schools may not have. They felt unclear that issues of social justice, equity, and Black,
Indigenous, and people of color’s experiences and contributions are reflected in
homework/classroom assignments and assessments. Due to this, the total score for the
Materials & Resources category was 2. Figure 24 shows the results of the scorecard for
the Yough School District.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
98
Figure 24
YSD Culturally Responsive STEAM Scorecard Results
The researcher had planned to review the Algebra I Keystone performance levels
for middle school students, disaggregated by ethnicity, for Spring 2022 to the results of
the Culturally Responsive STEAM Scorecard for the Monessen City School District.
Unfortunately, the teachers in that school district did not complete the scorecard. The
primary Algebra I instructional material utilized by the middle school Algebra I teachers
is the enVision - Algebra I online program by Savvas, formerly Pearson.
Next, the researcher completed a standards-analysis review for each of the
primary Algebra I instructional materials utilized by the middle school Algebra I teachers
in the Greater Latrobe School District, the Monessen City School District, and the Yough
School District. Since each resource was developed to be aligned to Common Core State
Standards, the researcher had to complete a Standards Crosswalk to align the materials to
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
99
Pennsylvania Core Standards and then determine which Standards, Assessment Anchors
and Eligible Content were present within each. Table 6 shows the number of Eligible
Content items present in each primary instructional material used by the Greater Latrobe
School District, Monessen City School District, and Yough School District.
Table 6
Number of Algebra I Eligible Content Items Present by Vendor
Algebra I Assessment Anchor
A1.1.1 - Operations With Real
Numbers and Expressions
# of Eligible Content # of Eligible # of Eligible # of Eligible
Items within Anchor in
Content
Content
Content
PA Core Standards For
Items
Items
Items
Mathematics
Addressed Addressed Addressed
Curriculum Framework
in HMH
in CK-12
in Savvas
Algebra 1
Resource
Resource
Resource
9
7
7
5
A1.1.2 - Linear Equations
19
7
13
14
A1.1.3 - Linear Inequalities
10
0
10
2
A1.2.1 - Functions
24
9
12
11
A1.2.2 - Coordinate Geometry
13
4
8
4
5
2
0
3
A1.2.3 - Data Analysis
When comparing Algebra I Keystone performance levels for middle school
students, disaggregated by ethnicity, for Spring 2022 to the results of the Standards
Crosswalk for all three primary Algebra I instructional materials, the findings indicate
that there may be an impact on student learning and achievement since the Algebra I
Assessment Anchors are not adequately addressed in each of the materials being used.
Go Math by covers 36.25% of the available Eligible Content items for Algebra I
within their curriculum. CK-12’s Algebra I flexbook covers 62.5% of the available
Eligible Content items for Algebra I within their curriculum in its original form before
being modified by users. Savvas’ enVision Algebra I program covers 48.75% of the
available Eligible Content items for Algebra I within their
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
100
curriculum. If teachers are not aware of the items that are not addressed within the
curriculum of the primary Algebra I instructional materials that their district has
approved, students may not be exposed to this content in the classroom, resulting in a
lack of understanding of concepts and mastery of skills needed to achieve higher
performance levels on the Algebra I Keystone Exam.
Finally, data were collected from the Teacher Perception Survey and compared
with historical student assessment scores in Algebra I and the results of the Culturally
Responsive STEAM Scorecard. Data was also collected from the Building Principal
Perception Survey and compared with the results of the Teacher Perception Survey,
historical student assessment scores in Algebra I, and the results of the Culturally
Responsive STEAM Scorecard.
In reviewing the results from the Teacher Perception Survey, teachers in
Westmoreland County overwhelmingly feel that their middle school mathematics
curriculum is well-aligned to Pennsylvania Core Standards for Mathematics, with 66.7%
of respondents reporting that their curriculum is mainly aligned to the standards.
However, their views began to vary as they began to evaluate the alignment of their
primary instructional resources to Pennsylvania Core Standards. Figure 25 shows the
differences in their perceptions of the alignment of the primary instructional materials
used in middle school Algebra I classes to Pennsylvania Core Standards for Mathematics.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
101
Figure 25
Teacher Perceptions of Standards Alignment of Instructional Materials
Of the teachers participating in the survey, 66.7% felt that their materials were not
missing critical standards addressed on the Algebra I Keystone Exam. In reviewing the
Building Principal Perception Survey results, most building principals participating in the
survey feel that at least 75% of their purchased instructional materials accurately align
with Pennsylvania Core Academic Standards. Only 27.3% of participants felt that 50%
of their purchased instructional materials were accurately aligned with Pennsylvania Core
Academic Standards. Figure 26 shows building principals' perceptions regarding the
accurate alignment to Pennsylvania Core Academic Standards for their purchased
instructional materials.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
102
Figure 26
Building Principals’ Perceptions of Instructional Materials Alignment to Standards
However, the results of the Standards Crosswalk completed by the researcher, as shown
above in Figure 30, show that resources being used within Westmoreland County are
missing substantial amounts of Eligible Content items directly connected to the
Pennsylvania Core Standards. The perception surveys' findings indicate that teachers and
building principals feel that the primary instructional materials used in middle school
Algebra I classrooms are more accurately aligned to Pennsylvania Core Standards and are
addressing Eligible Content items in Algebra I than they really are.
Research Question 2
The second research question in this case study aims to answer “What are the
factors that impact instructional delivery in Algebra I?” Data from both the Teacher
Perception Survey and the Building Principal Perception Survey were compiled and
analyzed to research this question effectively. Many of the questions presented to
teachers and building principals were related to their perceptions about the time and
funds spent on professional development for utilizing instructional materials.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
103
The researcher first focused on the results of the Teacher Perception Survey.
When asked if teachers had been given plenty of professional development focused on
using the instructional materials their district had purchased effectively, the answers
varied greatly. Some teachers strongly agreed they had been given plenty of time, while
others disagreed. Figure 27 shows the differences in teachers' perceptions regarding the
amount of time they feel that they have to use the instructional materials they have been
given effectively.
Figure 27
Teacher Perceptions of PD to Effectively Use Instructional Materials
The researcher next compared these results to data from the Building Principal
Perceptions Survey. When asked how much money is spent annually on teacher
professional development experiences designed to help teachers utilize their purchased
instructional materials in their classrooms, 81.8% of building principals who participated
in the survey said they spend between $0.00 and $25,000.00. Figure 28 shows the
breakdown of funds spent annually by districts on teacher professional development to
utilize instructional materials in their classroom.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
104
Figure 28
Money Spent Annually By Districts on Teacher PD to Utilize Instructional Materials
In comparing this data with the results of the Culturally Responsive STEAM
Scorecard, as shown in Figures 23 and 24 above, the findings indicate that teachers feel
they need more confidence in their ability to evaluate the instructional resources that their
districts have purchased. Teachers also need more professional development experiences
designed to help them utilize their primary instructional materials more effectively and
develop culturally responsive learning experiences to meet students' needs better. Each
of these factors directly impacts the instructional delivery of Algebra I instruction
occurring in middle school classrooms in Westmoreland County.
When comparing Algebra I Keystone performance levels for middle school
students, disaggregated by ethnicity, for Spring 2022 to the results of the Culturally
Responsive STEAM Scorecard, the findings indicate that the guidance provided to
teachers on how to use their instructional materials from the vendor that developed those
materials may have an impact on student learning and achievement. When teachers feel
that this guidance needs to be clarified, they may struggle to create instructional
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
105
experiences for students that reflect the lived experiences of the students in their
classroom and actively design instruction and instructional materials that reflect those
experiences.
The researcher speculates that this may be addressed by implementing the
Culturally Relevant and Sustaining Education Guidelines and Framework that the
Commonwealth of Pennsylvania has adopted. As of this writing, Pennsylvania school
districts have been directed to include Culturally Relevant and Sustaining Education as a
part of their continuing education plan for their educators starting with the 2023-2024
school year. Based on the competencies within the framework, this professional
development will assist teachers in developing instructional experiences for students that
reflect their lived experiences in their classroom and actively design instruction and
instructional materials that reflect those experiences.
Research Question 3
The third research question in this case study aims to answer “What are the
teacher and administrator perceptions of the impact of instructional delivery on student
achievement scores in Algebra I?” To investigate this question, the researcher analyzed
data from the Teacher and Building Principal Perception Surveys relating to instruction,
instructional materials, and student achievement scores. Overall, teachers believe that
their instructional materials directly impact student achievement scores, as shown in
Figure 29. These perceptions appear to correlate to the Standards Crosswalk results and
the historical student achievement scores in Algebra I.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
106
Figure 29
Teacher Perceptions on the Impact of Instructional Materials on Student Achievement
In examining the data further to determine the impact on instructional delivery,
teachers feel split in their opinions that the physical instructional materials used with
students adequately address the skills needed to help prepare students for the Algebra I
Keystone Exam. They feel less confident in the ability of their digital instructional
materials to address the skills required to help prepare students for the Algebra I
Keystone Exam. Figures 30 and 31 show the results of these questions from the Teacher
Perception Survey.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
107
Figure 30
Teacher Perceptions on Physical Materials Preparing Students for Keystone Exams
Figure 31
Teacher Perceptions on Digital Materials Preparing Students for Keystone Exams
Teachers overwhelmingly believe that students do better on assessments when
they are able to see themselves in their instructional materials, as seen in Figure 32
below. However, teachers acknowledge that their primary instructional materials are not
always culturally reflective of their entire student population. They must supplement their
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
108
instruction with Open Educational Resources or other materials to develop more
culturally relevant learning experiences.
Figure 32
Teachers' Perceptions of Student Reflections on Instructional Materials
Building principals also acknowledge that teachers need to modify resources to meet
students' cultural and academic needs. If teachers are unable to modify the instructional
materials that are available to them, they will develop their own original materials
instead. Figures 33 and 34 provide a breakdown of the perceptions that building
principals have regarding the frequency in which teachers modify resources to meet the
needs of students or develop their own resources to meet the needs of students.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
109
Figure 33
Building Principals’ Perceptions of Teacher Modifications to Resources
Figure 34
Building Principals’ Perceptions of Teacher Development of Original Resources
Although building principals acknowledge that teachers are engaging in the
modification of existing resources or the creation of original resources to meet the
cultural and academic needs of their students as a way to supplement the instruction that
is provided within their primary instructional materials for Algebra I, they also
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
110
acknowledge that there are internal and external factors that cause teachers to shy away
from taking these action. Building principals feel that the internal factors that cause
teachers not to use OER in their Algebra I classrooms include:
● “A perception that OER are not aligned to PA Core Standards”
● “Lack of knowledge and how they can be incorporated into what they do and a
perceived lack of time during the day”
● “Ignorance”
● “Lack of research and commitment to utilizing various resources that may be
different from what they are comfortable already using”
They also feel that the external factors that cause teachers not to use OER in their
Algebra I classrooms include:
● “Not explicitly approved as per the curriculum / textbook approval process.”
● “Potential pressure from community members”
● “Focus on school curriculum”
● “School board”
These findings suggest that the internal and external factors preventing the
broader adoption of Open Educational Resources in Algebra I classrooms in
Westmoreland County are rooted in a lack of understanding about OER and a fear of
judgment from others for utilizing OER.
Discussion
The research aimed to examine the possible correlation between the historically
low Keystone Algebra I Scores in middle schools in Westmoreland County, building
principals and Algebra I teachers’ perceptions of the standards-alignment and efficacy of
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
111
their primary instructional materials, the amount of training and support provided to
teachers to effectively utilize those materials in their Algebra I classrooms, and the
culturally responsive nature of their materials. As stated earlier in Chapter IV, the
quantitative data gathered by the researcher included overall student Algebra I Keystone
Exam scores, completed a countywide Algebra I Anchor Analysis, local assessment data
for Algebra I, Future Ready PA Index data, annual budget information relating to
instructional materials expenses, district Comprehensive Plans, and Pennsylvania
Common Core Standards alignment information for the primary Algebra I instructional
resources used by Greater Latrobe School District, Monessen City School District, and
Yough School District. This data was gathered from the eMetric Data Interaction for
Pennsylvania Student Assessments web-based system (eMetric), the web-based
Pennsylvania Value Added Assessment System (PVAAS), the Pennsylvania Department
of Education’s website (PDE), PDE’s web-based Future Ready PA Index, individual
district websites, and individual content vendor websites. The qualitative data gathered
by the researcher included Building Principal and Teacher Perception Surveys and the
completion of the Culturally Responsive STEAM Scorecard. The data were analyzed to
address all research questions to determine if a correlational relationship existed.
To analyze the first research question, “What are the factors that impact student
achievement in Algebra I?” student assessment data was compiled and then
disaggregated into achievement categories and by Algebra I Assessment Anchors.
District demographic data were also disaggregated into the subgroups that PDE uses for
reporting. Student achievement data was then compared to the results of the Culturally
Responsive STEAM Scorecard and the Standards Crosswalk for the instructional
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
112
materials. Analyses revealed that several factors may be impacting student achievement
in Algebra I.
The first factor revealed in the data analysis for this research question was that
middle school Algebra I teachers do not feel that their primary instructional materials for
Algebra I provide them with guidance and support in developing culturally responsive
learning experiences that meet the needs of their student populations. Disaggregating
district demographic data by ethnicity and comparing it to the results of the Culturally
Responsive STEAM Scorecard shows that culturally responsive instruction may impact
student achievement. Teachers believe this to be true but need more guidance and
support in creating those learning experiences for their students.
The second factor revealed in the data analysis for this research question was that
the primary instructional materials used by the three identified case study school districts
do not adequately address the content covered on the Algebra I Keystone Exam.
Moreover, teachers must be made aware that this content must be addressed in their
instructional materials. While teaching to the test is not recommended, reviewing
instructional materials to ensure that they adequately support instruction to cover
concepts and skills that will be assessed on the Algebra I Keystone Exam is a highly
recommended action by PDE. To thoroughly complete this work, teachers and
administrators cannot simply accept the supporting alignment documentation that content
vendors provide. Instead, they must conduct their own independent evaluation to
determine if all Assessment Anchors and Eligible Content items are present and can
support the instructional experiences that their teachers develop. If absent, teachers and
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
113
administrators will need a plan to supplement their materials and instruction to prepare
students for the Algebra I Keystone Exam.
To analyze the second research question, “What are the factors that impact
instructional delivery in Algebra I?” data from both the Teacher Perception Survey and
the Building Principal Perception Survey that related to their perceptions about time and
funds spent on professional development for utilizing instructional materials were
compiled and analyzed. This research question aimed to explore how teachers grow in
their knowledge of their student populations, the use of their instructional materials, and
how to design effective learning experiences that help to increase student achievement.
Unfortunately, there was not much teacher participation in the Teacher Perception
Survey. The researcher speculates that the current political climate and the connection to
the term “culturally responsive” may have caused teachers to hesitate to participate in the
survey. However, findings based on the received responses show that teachers feel they
are given enough time to learn how to effectively utilize their instructional resources
when districts spend money on professional development specifically designed to help
teachers use those materials in their classrooms.
The findings based on the Culturally Responsive STEAM Scorecard show that
teachers still feel uncertain about developing culturally responsive learning experiences
or identifying how resources may not be meeting the needs of their students from a
culturally responsive standpoint. While the low participation numbers on the Teacher
Perception Survey make it challenging to determine if a lack of culturally responsive
instruction impacts student achievement scores, teachers in Westmoreland County believe
there is a correlation between culturally responsive pedagogy and increased student
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
114
achievement. As districts begin to evaluate their instructional materials through a more
culturally responsive lens with the guidance of the Culturally Relevant and Sustaining
Education Guidelines and Framework developed by PDE, they will need to design
professional development experiences for teachers that guide them through the process of
how to balance the effective use of instructional materials to present concepts and skills,
and better understand the specific needs of their particular student populations from a
culturally responsive point of view and modify or supplement instruction with OER to
address those needs.
To analyze the third and final research question, “What are the teacher and
administrator perceptions of the impact of instructional delivery on student achievement
scores in Algebra I?” data was gathered from the Teacher and Building Principal
Perception Surveys relating to instruction, instructional materials, and student
achievement scores. This research question aimed to analyze whether or not teachers and
administrators believe that instructional experiences and resources directly impact student
achievement.
The findings indicate that while teachers and building principals recognize that
instruction and instructional materials will impact student achievement, they must learn
to analyze their resources from a standards-alignment and culturally responsive
perspective. They may also need more time to engage in observational activities that
allow them to see examples of standards-alignment activities and culturally responsive
instruction to learn how to develop and implement these skills.
The findings also indicate a significant disconnect in understanding Open
Educational Resources while teachers regularly engage in activities commonly associated
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
115
with their use. Survey results show that teachers regularly modify instructional materials
to meet the needs of their students, with respondents indicating that many teachers do this
at least weekly, if not daily, to ensure that students can successfully engage with their
instructional materials. The act of revising a resource to adapt or modify it to meet the
needs of students better is one of the main ways in which educators engage with Open
Educational Resources. Teachers across Westmoreland County engage in this critical
practice to support student academic success.
In summary, the most significant finding is a direct correlation between student
achievement scores and the amount of Assessment Anchors and Eligible Content items
present within the primary instructional materials that middle school Algebra I teachers
use. When instructional materials have a higher level of Assessment Anchors and
Eligible Content items present, students have a better chance of having higher
achievement scores on the Algebra I Keystone Exam. This shows that teachers strongly
align their instruction to the content in their primary instructional materials. Factors that
prevent them from deviating from their primary instructional materials include pressure
only to use their purchased materials, a lack of knowledge about the Assessment Anchors
and Eligible Content items that are present within their primary instructional materials,
and a lack of understanding of how to utilize Open Educational Resources effectively.
Summary
Chapter IV of the study introduced a mixed-methods embedded research design
aimed at investigating a potential correlation between the historically low Keystone
Algebra I Scores of middle school students in Westmoreland County and the perceptions
of building principals and Algebra I teachers regarding student identification with
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
116
examples in word problems found within their middle school instructional materials. The
chapter provided a comprehensive analysis of both quantitative and qualitative data to
address the three research questions formulated for this particular case study.
To start, the chapter offered a detailed explanation of the research methodology
used by the researcher, which combined quantitative and qualitative approaches to gain a
thorough understanding of the research problem. The mixed-methods design allowed the
researchers to gather numerical data through the collection of assessment scores while
also gathering subjective insights through perception surveys. This comprehensive
approach aimed to provide a more robust analysis and interpretation of the research
findings.
The chapter then delved into the data analysis process, describing the techniques
used to analyze both the quantitative and qualitative data. The quantitative analysis
sought to identify any correlations or patterns between the Keystone Algebra I Scores and
analyses of the middle school instructional materials. The qualitative data analysis
involved a thorough examination of surveys and the perceptions of principals and
teachers. This qualitative analysis aimed to provide a deeper understanding of the
perceptions and experiences of principals and teachers regarding student identification
within the word problems presented to students within their materials.
The chapter then presented the results obtained from both the quantitative and
qualitative approaches. The quantitative results provided evidence of any correlations
between low Keystone Algebra I Scores and the alignment of instructional materials to
PA Core Standards and Eligible Content Standards. In addition, the qualitative results
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
117
offered rich descriptions and interpretations of the identified themes and patterns within
the perceptions of principals and teachers.
Finally, Chapter IV engaged in a comprehensive discussion of the interpretation
of the results. It analyzed the implications of the findings, considering both the
quantitative and qualitative perspectives. The discussion aimed to identify the possible
reasons behind the findings, providing insights into the underlying factors influencing
student identification in the context of word problems.
Chapter IV serves as a bridge between data analysis and the upcoming
conclusions and recommendations of the research study, which will be presented in the
next chapter. Chapter V, which will focus on drawing conclusions and formulating
recommendations based on the interpretation of the results from the previous chapter. It
will also synthesize the findings, discuss their significance, and propose actionable
recommendations that can contribute to addressing the challenges identified in the study.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
118
Chapter V
Conclusion and Recommendations
Administrators and teachers in public school districts in Westmoreland County
have struggled to understand their students’ historically low middle school mathematics
test scores on PSSA and Keystone Exams. The Keystone Exam test scores of middle
school students have been particularly concerning since the inception of the Keystone
Algebra I Exam in 2010. To understand the root cause of these scores, building
administrators and Algebra I teachers have engaged in intensive data analysis of student
test scores over the years. This analysis has driven the adoption of instructional
materials, the refinement of scopes and sequence of instruction, the evaluation of and
alignment of curriculum maps to Pennsylvania Core State Standards, and the
implementation of new instructional initiatives designed to increase student achievement
scores.
The mixed-methods embedded design of the research set out to answer three
research questions, which sought to further determine the root cause of students’
historically low test scores by understanding the impact of the instructional materials
used in Algebra I classrooms on student achievement. They also sought to understand
building administrator and Algebra I teacher perceptions of instructional materials and
professional development time devoted to implementing instructional materials with
fidelity, as well as understanding the perceived cultural responsiveness of their primary
instructional materials for Algebra I in middle school classrooms.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
119
Eleven building principals and six middle school Algebra I teachers employed in
K-12 public schools in Westmoreland County, Pennsylvania, participated in the study.
The researcher collected data related to student Algebra I achievement scores, assessment
data, budget data, and standards alignment data to discover patterns and trends in student
achievement scores over time. The participants completed perception surveys and the
Culturally Responsive STEAM Scorecard to provide a potential correlation between
instructional materials, Culturally Responsive Teaching pedagogy, and increases in
student achievement.
Conclusion
The mixed-method embedded research design sought to determine if a correlation
exists between the historically low Keystone Algebra I Scores of middle school students
in Westmoreland County and the perceptions of building principals and Algebra I
teachers regarding student identification with examples in word problems found within
their middle school instructional materials. The researcher investigated this issue to
provide both building administrators and Algebra I teachers with guidance to help
support their efforts to understand the gaps between the intended curriculum better, as
identified by state standards and district curriculum maps, the implemented curriculum,
as observed through the alignment of resources to standards and culturally responsive
pedagogy, and the attained curriculum, as shown by student achievement scores. The
following sections discuss specific conclusions related to each of the established research
questions, explain how the findings support those conclusions, and describe how those
findings may be impacting current instructional practices in middle school Algebra I
classrooms.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
120
Research Question 1: “What are the factors that impact student achievement in
Algebra I?”
The first research question analyzed the factors that impact student achievement
in Algebra I by reviewing disaggregated, historic student achievement data. The
researcher then completed a standards-analysis review for each of the primary Algebra I
instructional materials utilized by the middle school Algebra I teachers in three case
study districts representing all of Westmoreland County: Greater Latrobe School District,
the Monessen City School District, and the Yough School District. Finally, the researcher
compared disaggregated test scores and the standards analysis with the results of the
Culturally Responsive STEAM Scorecard by participants from the case study school
districts. The study design aimed to determine if the instructional materials used in
Algebra I classrooms impacted student proficiency scores on the Algebra I Keystone
Exam.
By disaggregating student scores on the Algebra I Keystone Exam by race, the
researcher gained insight into the impact of the wording of the exam questions on student
subgroups. There was no significant difference in Algebra I Keystone Exam scores
between subgroups, regardless of gender, race, socioeconomic status, special education
status, or English language learner status. These factors alone do not have a significant
impact on student proficiency. This suggests that the wording used in the Algebra I
Keystone Exam questions is mainly free from bias.
Perception data showed that both building principals and Algebra I teachers have
high confidence levels in the accuracy of their primary instructional materials. They feel
strongly that their materials are closely aligned with Pennsylvania Core Academic
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
121
Standards and will support students as they prepare for the Algebra I Keystone Exam.
Each district’s process for adopting primary instructional materials includes, at a
minimum, a review of the standards alignment documentation provided by the content
vendor. Despite high levels of confidence from building principals and Algebra I
teachers that their primary instructional materials are aligned with Pennsylvania Core
Academic Standards, a comprehensive standards analysis shows that the primary
instructional materials used by the three case study districts are missing critical Eligible
Content Items that are addressed on the Algebra I Keystone Exam. The implication is
that if teachers rely heavily on instructional materials that do not address Eligible Content
Items that are presented to students on the Algebra I Keystone Exam, students may not
understand the content, contributing to their low test scores.
In evaluating materials with the Culturally Responsive STEAM Scorecard,
Algebra I teachers were able to quantify their perception of the cultural relevance of their
primary instructional material used for Algebra I. The results of the Culturally
Responsive Scorecard show that teachers are not fully confident in their abilities to
identify bias within their instructional materials accurately. The adoption of
Pennsylvania’s Culturally Relevant and Sustaining Education Framework, including its
standards and competencies, is designed to assist educators in completing this work to
help teachers critically examine their instructional materials to determine if students can
see themselves reflected in their instructional materials. The required training
components associated with the framework may assist teachers in building their
confidence in analyzing instructional materials through a culturally responsive lens.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
122
Research Question 2: “What are the factors that impact instructional delivery in
Algebra I?”
The second research question analyzed the time and funds spent on professional
development for utilizing instructional materials. The study design aimed to determine
whether or not time and funds for professional development related to using instructional
materials in the classroom impacted student achievement. It also sought to understand
the differences in perceptions regarding time and funds for professional development
from building administrators and Algebra I teachers.
There was a discrepancy in teacher opinions on the amount of time given for
professional development, with some strongly agreeing that they had been given plenty
of time and others disagreeing. This may indicate that the amount of time spent on
professional development varies across districts in Westmoreland County. Conversely, it
highlights teachers' different confidence levels in their abilities to utilize their
instructional materials without additional support effectively.
When asked how much money is spent annually on teacher professional
development experiences designed to help teachers utilize their purchased instructional
materials in their classrooms, 81.8% of building administrators who participated in the
survey said they spend up to $25,000.00 However, it is important to note that this is a
building-wide cost. It is not isolated to the funds spent only on professional development
to support implementing Algebra I instructional materials in classrooms.
As discussed in the conclusions for Research Question 1, the Culturally
Responsive STEAM Scorecard found that teachers in Westmoreland County need more
confidence in their ability to evaluate instructional resources. The scorecard also found
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
123
that they need more professional development on using their primary instructional
materials effectively and more support in developing culturally responsive learning
experiences, as teachers feel strongly that students better understand concepts and master
skills when they see themselves in their instructional materials. These factors directly
impact the instructional delivery of Algebra I in middle school classrooms. This may be
influenced positively by implementing the Culturally Relevant and Sustaining Education
framework.
Research Question 3: “What are the teacher and administrator perceptions of the
impact of instructional delivery on student achievement scores in Algebra I?”
The third research question analyzed Teacher and Building Principal Perception
Surveys data relating to instruction, instructional materials, and student achievement
scores. The study design aimed to determine whether there was a correlation between the
Standards Crosswalk results and the historical student achievement scores in Algebra I.
Teachers believe that their instructional materials directly impact student
achievement scores. Teachers are divided on whether the physical instructional materials
they use with students adequately address the skills needed to prepare students for the
Algebra I Keystone Exam. They are less confident that their digital instructional
materials can do so. These results indicate that teachers are not confident in effectively
analyzing instructional materials to identify standards and Eligible Content Items. It may
also suggest that they cannot verify the accuracy of the standards documentation provided
to them by content vendors. Teacher confidence levels may also vary across districts in
Westmoreland County and may be influenced by factors related to Research Question 2.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
124
Teachers overwhelmingly believe students do better on assessments when they
see themselves in their instructional materials. However, teachers acknowledge that their
primary instructional materials are only sometimes culturally reflective of their entire
student population. To address this, teachers must supplement their instruction with
resources that they create other purchased instructional materials that are more culturally
relevant. In doing so, teachers can add variety to how they introduce concepts and skills
to students through their materials. However, the need to supplement their instruction
with additional instructional materials means that teachers also need extra professional
development time to curate or develop those materials. Due to the additional time
required, teachers hesitate to implement supplemental instructional materials within their
classrooms.
Teachers may also utilize Open Educational Resources (OER) to develop
instructional experiences that are more culturally reflective of their student population.
Using OER in their classrooms enables teachers to modify resources to meet the needs of
their students better. However, as we saw with additional purchased instructional
materials, teachers are hesitant to implement OER in their classrooms. This is due to
several factors, including a need for more understanding about OER and a lack of time to
explore or have comprehensive professional development and ongoing support regarding
OER initiatives within their schools.
These findings suggest that the main reasons Open Educational Resources are not
widely adopted in Algebra I classrooms in Westmoreland County are a need for more
understanding about OER and a fear of being judged for using them. The current
political climate makes teachers hesitant to explore materials beyond their approved
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
125
primary materials. Building administrators are also reluctant to encourage their use due
to the current political environment. Both groups fear that the broad acceptance of OER
and open discussion of culturally responsive pedagogy will open them up to criticisms
from stakeholders and that these criticisms will detract from the positive benefits of both
for students. At this point, instead of working to educate stakeholders, most building
principals and teachers feel that it makes sense to find an amicable path forward to
support students without causing additional strife among their communities.
Overview of Results
The overall research design aimed to investigate the relationship between low
Keystone Algebra I Scores in middle schools in Westmoreland County, the perceptions of
building principals and Algebra I teachers regarding instructional materials, training and
support provided to teachers, and the cultural responsiveness of the materials. The
researcher collected data from various sources, such as student assessment scores, district
demographics, budget information, and surveys. The analysis revealed two key factors
that may impact student achievement in Algebra I. The first key factor was that teachers
felt their instructional materials lacked guidance and support for culturally responsive
learning experiences. The second key factor was that the primary materials used by
teachers did not adequately cover the content assessed in the Algebra I Keystone Exam,
requiring teachers to supplement their instruction with additional materials from sources,
including OER.
Regarding instructional delivery, the survey data showed that teachers believed
they had enough time and support to use the instructional resources when provided with
professional development effectively. However, teachers needed to feel more confident
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
126
about developing culturally responsive experiences and identifying resource gaps.
Regarding teacher and administrator perceptions, they recognized the impact of
instruction and materials on student achievement but needed to analyze resources from a
standards-alignment and culturally responsive perspective. There also needed to be a
greater understanding of OER despite teachers regularly modifying materials to meet
student needs.
The most significant finding was the correlation between student achievement
scores and the presence of Assessment Anchors and Eligible Content Items in the
instructional materials used by Algebra I teachers. The identified case study districts,
Greater Latrobe School District, Monessen City School District, and Yough School
District, revealed a more detailed look into the results of this study. The following
sections discuss specific results for each of the three case study districts,
recommendations, and fiscal implications for each.
Greater Latrobe School District
The Greater Latrobe School District has been using Go Math by Houghton
Mifflin Harcourt as their primary instructional resource for middle school Algebra I. A
comprehensive analysis of this resource reveals that only 36.25% of the Algebra I
Eligible Content Items are present in this textbook. With the use of this resource, plus the
addition of supplemental materials that teachers both developed and found from external
sources, 8th-grade student scores on the Algebra I Keystone Exam were in the following
proficiency categories:
● Below Basic: 0%
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
127
● Basic: 11.7%
● Proficient: 40.5%
● Advanced: 47.7%
The teachers within the Greater Latrobe School District expressed a strong sense
of satisfaction with the curriculum of their primary Algebra I instructional materials,
emphasizing its strong alignment with Pennsylvania Core Academic Standards. Teachers
were also content with the curriculum's rigor, feeling that the material presented to
students is appropriately demanding and helps them to apply concepts and skills to
further their understanding. They also feel that the materials help to prepare students for
the Algebra I Keystone Exam.
Teachers felt that their primary Algebra I instructional materials have the ability
to represent diverse communities effectively. Moreover, the teachers were pleased with
how the curriculum went beyond mere representation and actively incorporated the
expertise of these diverse and historically marginalized communities. By integrating
their knowledge and insights, the instructional materials provided a comprehensive
understanding of algebraic concepts and served as a platform to elevate the voices and
contributions of these communities. The teachers found this approach essential in
fostering a classroom environment that values diversity and promotes inclusivity.
Throughout the student-facing materials, the teachers felt that the curriculum
consistently highlighted these communities' achievements, struggles, and unique
perspectives. The teachers recognized the significance of this inclusive approach, as it
empowered students to embrace diversity and recognize the contributions of historically
marginalized people. Teachers expressed dissatisfaction with how their primary Algebra
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
128
I instructional materials approached essential concepts such as interdependence and
continual change. They felt that the topics were not adequately addressed within the
instructional materials, leaving gaps in students' understanding of the interconnectedness
of mathematical concepts and their relevance in real-world scenarios. The teachers
recognized the significance of teaching these concepts as they are crucial for developing
critical thinking skills and preparing students for the complexities of the modern world.
Teachers also found the instructional activities in the materials lacking when presenting
alternatives or allowing students to explore and offer their own alternatives regarding the
role of science, technology, engineering, arts, and math (STEAM) in existing social
structures.
A range of perspectives emerged among teachers regarding their satisfaction with
the teachers' primary Algebra I instruction materials in addressing crucial aspects such as
cultural responsiveness and real-life connections. While some teachers expressed high
satisfaction, others felt that the materials fell short in these areas. Teachers recognize the
importance of making real-life connections between academic content and the local
neighborhoods, culture, environment, community issues, and current events and feel that
their primary instructional materials do not give them the flexibility to do so. When they
have materials that provide them with this flexibility, they can better foster meaningful
engagement and relevance in the learning process, enabling students to see the direct
applicability of Algebra I concepts in their own lives.
Based on these results, the researcher recommends that the Greater Latrobe
School District adopt a new primary instructional resource for their middle school
Algebra I classrooms. During their resource evaluation process, the researcher suggests
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
129
Algebra I teachers and building administrators engage in a thorough standards-analysis
review to determine which Algebra I Eligible Content Items are present within each
resource being considered for adoption. In doing so, teachers will better understand the
items not present in their final decision. The fiscal implication of this recommendation is
that the district will need to incur the cost of purchasing a new textbook for their middle
school Algebra I classes, which may be more than expected if this action falls outside
their typical adoption cycle.
The researcher also recommends that the Greater Latrobe School District partner
with the Curriculum Services Team at the Westmoreland Intermediate Unit to engage in
ongoing professional development for all staff around the effective and ongoing use of
Open Educational Resources in their classrooms to support culturally responsive
pedagogy. This training will help teachers and building administrators learn how to
curate, modify, and remix resources to meet the needs of their student population. It will
also help them develop their instructional skills to effectively represent diverse
communities in their lessons and design lessons and materials that better represent the
students within their classrooms while also aligning to content standards and Eligible
Content Items. The fiscal implication of this recommendation is minimal. It may include
minor upgrades to the technical infrastructure of the district to ensure that teachers and
students can effectively utilize digital resources as needed.
Finally, the researcher recommends that the Greater Latrobe School District
utilizes a new primary Algebra I resource and supports the implementation of OER
through ongoing professional development for multiple years to track the efficacy of
these recommendations. The district will not see sustainable results with a one-year
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
130
implementation. Instead, they must commit to a systematic approach to professional
development that supports teachers in learning to use new resources, develop their own
resources, and grow their knowledge and skills. Adding an OER professional
development initiative to the district’s Comprehensive Plan will help the district to
commit to a long-term plan for growth and to see their efforts reflected in student test
scores.
Monessen City School District
The Monessen City School District has been using Savvas’ enVision Algebra I
program as their primary instructional resource for middle school Algebra I. A
comprehensive analysis of this resource reveals that only 48.75% of the Algebra I
Eligible Content Items are present in this textbook. With the use of this resource, plus the
addition of supplemental materials that teachers both developed and found from external
sources, 8th-grade student scores on the Algebra I Keystone Exam were in the following
proficiency categories:
● Below Basic: 54.5%
● Basic: 27.3%
● Proficient: 18.2%
● Advanced: 0%
Unfortunately, no perception or Culturally Responsive STEAM Scorecard data is
available from the Monessen City School District. Both building administrators and
Algebra I teachers did not complete the perception surveys or the scorecard. This means
that the researcher is unable to determine if the teachers in the district with the most
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
131
diverse population of the three case study districts feel that their instructional materials
are culturally responsive or if their instructional materials are doing an adequate job of
addressing the standards and preparing their students for the Algebra I Keystone Exam.
Based on the information that the researcher collected, the researcher can
recommend that the Monessen City School District adopt a new primary instructional
resource for their middle school Algebra I classrooms. The researcher recommends that
the district consider an OER-based primary resource to provide teachers with a higher
level of flexibility in addressing the wide variety of needs that the students within their
classrooms have. This will also allow the district to reallocate funds typically used for
resource adoption and instead dedicate them to additional professional development to
better support teachers in understanding how to utilize OER to meet the needs of students
effectively.
As was recommended for the Greater Latrobe School District, the researcher
recommends that the Monessen City School District partner with the Curriculum Services
Team at the Westmoreland Intermediate Unit to engage in ongoing professional
development for all staff around the effective and ongoing use of Open Educational
Resources in their classrooms to support culturally responsive pedagogy. This training
will help teachers and building administrators develop their instructional skills to
effectively represent diverse communities in their lessons and design lessons and
materials that better represent the students within their classrooms while also aligning to
content standards and Eligible Content Items. The fiscal implication of this
recommendation is minimal. It may include minor upgrades to the technical infrastructure
of the district to ensure that teachers and students can effectively utilize digital resources
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
132
as needed. The researcher also recommends that the district commits to ongoing
professional development for multiple years to track these recommendations' efficacy.
Adding an OER professional development initiative to the district’s Comprehensive Plan
will help the district to commit to a long-term plan for growth and to see their efforts
reflected in student test scores.
Yough School District
The Yough School District has been using CK-12’s Algebra I flexbook as their
primary instructional resource for middle school Algebra I. A comprehensive analysis of
this OER-based resource reveals that 62.5% of the available Eligible Content items for
Algebra I within their curriculum are in its original form before users modify it. With the
use of this resource, plus the addition of supplemental materials that teachers both
developed and found from external sources, 8th-grade student scores on the Algebra I
Keystone Exam were in the following proficiency categories:
● Below Basic: 2.4%
● Basic: 39%
● Proficient: 43.9%
● Advanced: 14.6%
Overall, the teachers' satisfaction with the curriculum of their primary Algebra I
instructional materials reflected a commitment to inclusivity and equity within the Yough
School District. By adequately representing and incorporating the expertise of diverse
and historically marginalized communities, the curriculum promoted a positive and
inclusive learning environment that valued the experiences and contributions of all
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
133
students. Teachers in the Yough School District expressed less satisfaction with how
their primary Algebra I instructional materials tackled key concepts such as
interdependence and continual change. They felt that these areas were not adequately
addressed, leaving gaps in students' understanding of the interconnected nature of these
concepts and their relevance in real-world contexts. The teachers believed that a
comprehensive understanding of interdependence and continual change is essential for
developing critical thinking skills and preparing students for the complexities of the
modern world.
Teachers were uncertain if the materials effectively encouraged students to
analyze and challenge how STEAM subjects impact society critically. However, they
acknowledge that they need more confidence to evaluate resources through this lens
effectively. The teachers emphasized the need for instructional materials to provide more
explicit guidance and resources for addressing interdependence, continual change, and
the societal implications of STEAM. They believed that by encouraging students to
explore alternatives and present their own ideas, the materials would foster a deeper
understanding of these concepts and promote critical thinking skills.
In the Yough School District, the Teachers' Materials category stood out as the
only area where teacher opinions varied greatly. Some teachers expressed high
satisfaction with the teachers' materials, appreciating their guidance on utilizing students'
everyday lives as a foundation for learning. These teachers found value in the materials'
emphasis on making real-life connections between academic content and various aspects
of the local community, including neighborhoods, culture, environment, community
issues, and current events. They also commended the materials for promoting
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
134
collaboration and leveraging individuals' unique assets, resources, and strengths to
enhance group and individual learning experiences.
Conversely, other teachers held reservations regarding Teachers' Materials. They
felt that the guidance provided for teachers to be aware of their biases, assumptions, and
cultural gaps between themselves and their students was unclear. These teachers believed
that the materials did not adequately prompt teachers to reflect on their own practices and
experiences in learning science, technology, engineering, arts, and math (STEAM)
subjects through the lens of cultural responsiveness. The contrasting perspectives among
teachers underscored the need for more explicit and comprehensive guidance within the
Teachers' Materials category.
Teachers in the Yough School District expressed satisfaction with the materials
and resources associated with their primary Algebra I instructional materials. They
appreciated that these materials offered students opportunities to engage with STEAM
concepts using accessible technology and manipulatives and were pleased that the rigor
of the curriculum was not contingent upon access to specific resources, materials, or
technology that students and schools may not have. This ensured that all students could
actively participate in and benefit from the curriculum's challenges and academic rigor
regardless of their available resources.
Based on these results, the researcher recommends that the Yough School District
continue utilizing the OER-based CK-12 Algebra I flexbook with their middle school
Algebra I classes. However, the researcher also suggests that the district leverage the
resource's flexible nature and modify the flexbook's structure to better align the book to
Pennsylvania Core Academic Standards by adding in the missing Eligible Content Items
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
135
from other instructional content within the CK-12 system. This will require teachers to
understand where to find those items within the CK-12 system and know how to add
them to their Algebra I flexbook to truly develop a personalized resource that matches the
scope and sequence of instruction that Yough School District has adopted, is aligned to
state standards, and covers all Eligible Content Items covered on the Algebra I Keystone
Exam. Since CK-12 is an OER-based resource, there will be no additional cost to the
district to enhance its already adopted resources. However, the researcher does
recommend that the district reallocate funds to increase the amount of professional
development and collaboration time for teachers to engage in this work.
As with the Greater Latrobe School District and the Monessen City School
District, the researcher also recommends that the Yough School District partner with the
Curriculum Services Team at the Westmoreland Intermediate Unit to engage in ongoing
professional development for all staff around the effective and ongoing use of Open
Educational Resources in their classrooms to support culturally responsive pedagogy.
This training will help teachers and building administrators learn how to curate, modify,
and remix resources to meet the needs of their student population beyond the CK-12
system and introduce teachers and building administrators to several reputable
repositories for OER. The fiscal implication of this recommendation is minimal. It may
include minor upgrades to the technical infrastructure of the district to ensure that
teachers and students can effectively utilize digital resources as needed.
The researcher also recommends that the Yough School District include these
recommendations in its Comprehensive Plan to engage in this work over multiple years.
In doing so, both building principals and teachers can commit to actively engaging in the
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
136
professional development and collaboration needed to implement OER within the
classroom in a sustainable manner effectively. This will also give the district time to see
their work reflected in student test scores.
Limitations
The established research design and implementation resulted in one significant
limitation, which impacted the results and findings. This significant limitation was the
low participation numbers by teachers across Westmoreland County. This particularly
impacted the results and findings for the Monessen City School District, as the researcher
could not gather any perception data for that school district. At the start of the study, four
Algebra I teachers from Monessen City School District indicated their willingness to
participate. However, none of the teachers took any steps to sign the Informed Consent
Letter, complete the Teacher Perception Survey, or the Culturally Responsive STEAM
Scorecard.
In addition, only six Algebra I teachers across the county were willing to engage
in any of the activities for this study. As the results suggested, teachers are concerned
about criticism from stakeholders regarding culturally responsive pedagogy. The
researcher feels that the lack of participation may be due to the topic of this study, the use
of the term culturally responsive, and fear over the current political climate in relation to
that term.
While not a limitation of the study itself, it is worth mentioning that the statewide
adoption and implementation of the Culturally Relevant and Sustaining Education
(CR-SE) guidelines and framework has been put in jeopardy during the study. Originally
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
137
passed into law on April 23, 2022, as part of Chapter 49, relating to the Certification of
Professional Personnel, the law has been challenged several times. Most recently, several
western Pennsylvania school districts have filed a lawsuit to prevent the implementation
of the CR-SE guidelines.
In addition, legislation was introduced to the House of Representatives Education
Committee on May 31, 2023, proposing the abrogation of Chapter 49 regulations related
to CR-SE. Should that legislation pass, it will go into effect immediately, eliminating the
CR-SE guidelines and framework, preventing their use, and any training for teachers
regarding culturally responsive pedagogy. While these developments did not directly
impact the study itself, they may have an impact on the researcher's recommendations.
Future Directions for Research
The findings of this research warrant further study to gain a better understanding
of the challenges building administrators and teachers face when working to bridge the
gaps between the intended curriculum, the implemented curriculum, and the attained
curriculum in order to positively influence student achievement. The researcher
recommends that future researchers continue this work and seek out larger participant
groups to gain a more comprehensive understanding of teachers' perceptions. Future
researchers can achieve this by expanding the work beyond middle school, looking at
math scores across transition years, and including all math teachers in the study. Doing
so will allow participants to work together more collaboratively and engage in
meaningful discussion while completing the Culturally Responsive STEAM Scorecard
rather than in isolation.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
138
The researcher recommends replicating this study with other public school
districts within Westmoreland County to understand better the resources being used
across the county. This will also provide county administrators and teachers with a
common critical analysis experience regarding their instructional materials and student
test scores and can serve as the starting point for a countywide Algebra I community of
practice to meet regularly to share resources, experiences, instructional practices, and to
engage in problem-solving around common concerns related to Algebra I test scores in
Westmoreland County.
In replicating the study, the researcher recommends also looking at the
assessments used by Algebra I teachers in their classrooms throughout the year, the
alignment of those assessments to the Eligible Content Items on the Algebra I Keystone
Exam, and the implementation of common assessments by teachers in the same districts.
This will give future researchers better insight into how Algebra I teachers regularly
assess students on their understanding and mastery of skills and concepts taught to
students with the help of the primary instructional materials used by teachers throughout
the school year as they prepare for the Algebra I Keystone Exam.
Finally, the researcher recommends expanding this study into other content areas
beyond mathematics. Doing so will test to see if the research design is effective and if
the study applies to other content areas beyond mathematics. With the new adoption of
the Science, Technology & Engineering, and Environmental Literacy & Sustainability
(STEELS) Standards and the current focus that many districts have on finding new
resources that align with new instructional strategies, this study may be well-suited to be
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
139
replicated for districts that are concerned with student test scores on the Biology
Keystone Exam next.
Summary
To understand the root cause of students’ historically low test scores on the
Algebra I Keystone Exam, the researcher developed a mixed-method embedded research
design that examined the impact of the instructional materials used in Algebra I
classrooms on student achievement. Chapter V presented the conclusion of this research
by providing a comprehensive analysis of the findings for each of the three identified
research questions. The chapter also included specific implications for the three case
study districts identified within the research study, providing each study district with
personalized recommendations and the fiscal implications for implementing those
recommendations. Finally, Chapter V discussed the limitations of the research and
recommendations for further study by future researchers in the hopes of continuing the
work to support building administrators and teachers as they work to meet the needs of
their individual student populations.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
140
References
Abdulrahim, N., & Orosco, M. J. (2019, April 5). Culturally responsive mathematics
teaching: A research synthesis. The Urban Review, 52(1), 1-25.
https://doi.org/10.1007/s11256-019-00509-2
Ayers, D. F. (2006, June). Culturally responsive pedagogy and online learning:
Implications for the globalized community college. Community College Journal
of Research and Practice, 30(5-6), 401-415.
https://doi.org/10.1080/10668920500442125
Brown, C., Boser, U., Sargrad, S., & Marchitello, M. (2016, January 29). Implementing
the Every Student Succeeds Act toward a coherent, aligned assessment system.
Center for American Progress.
https://www.americanprogress.org/article/implementing-the-every-student-succee
ds-act/
Brown, J. C., & Crippen, K. J. (2016, March 2). The Growing Awareness Inventory:
Building capacity for culturally responsive science and mathematics with a
structured observation protocol. School Science and Mathematics, 116(3),
127-138. https://doi.org/10.1111/ssm.12163
Chingos, M. M., & Whitehurst, G. J. ". (2012, April). Choosing blindly: Instructional
materials, teacher effectiveness, and the Common Core. Brookings.
https://www.brookings.edu/research/choosing-blindly-instructional-materials-teac
her-effectiveness-and-the-common-core/
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
141
Cole-Malott, D.-M., Parker Thompson, K., Peterson-Ansari, R., & Whitaker, II, R. W.
(2021, April). The Pennsylvania Culturally Relevant and Sustaining Education
Competencies.
https://s3.amazonaws.com/newamericadotorg/documents/The_Pennsylvania_Cult
urally_Relevant_and_Sustaining_Education_Competencies.pdf
Copur‑Gencturk, Y., Jacobson, E., & Rasiej, R. (2021, February 18). On the alignment of
teachers’ mathematical content knowledge assessments with the common core
state standards. Journal of Mathematics Teacher Education, 25, 267–291.
https://doi.org/10.1007/s10857-021-09486-4
Dorn, E., Hancock, B., Sarakatsannis, J., & Viruleg, E. (2020, June 1). COVID-19 and
student learning in the United States: The hurt could last a lifetime. McKinsey.
https://www.mckinsey.com/industries/education/our-insights/covid-19-and-studen
t-learning-in-the-united-states-the-hurt-could-last-a-lifetime
Elementary and Secondary Education Act. (1965, April 11). 20 U.S.C. § 6301.
https://www.congress.gov/89/statute/STATUTE-79/STATUTE-79-Pg27.pdf
eMetric. (2023). Data Interaction for Pennsylvania Student Assessments. eMetric Data
Interaction. https://pa.emetric.net
Every Student Succeeds Act. (2015, December 10). 20 U.S.C. § 6301.
https://www.congress.gov/114/plaws/publ95/PLAW-114publ95.pdf
Garland, S. (2020, March 30). Who was behind the Common Core math standards, and
will they survive? The Hechinger Report.
https://hechingerreport.org/behind-common-core-math-standards-will-survive/
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
142
Garth, R., & Sebastian, R. (2020, November 18). Using open educational resources to
create a culturally relevant classroom. Achieving the Dream.
https://achievingthedream.org/using-open-educational-resources-to-create-a-cultur
ally-relevant-classroom/
Gay, G. (2013, January). Teaching to and through cultural diversity. Curriculum Inquiry,
43(1), 48-70. https://doi.org/10.1111/curi
Greater Latrobe School District. (2022). District Information – About Us.
https://www.glsd.us/apps/pages/index.jsp?uREC_ID=1756118&type=d&pREC_I
D=1937431
Hamilton, L. S., Stecher, B. M., & Yuan, K. (2008, December). Standards-based reform
in the United States: History, research, and future directions. Institute of
Education Sciences.
https://www.rand.org/content/dam/rand/pubs/reprints/2009/RAND_RP1384.pdf
Harmon, D. A. (2012, Spring). Culturally responsive teaching though a historical lens:
Will history repeat itself? Interdisciplinary Journal of Teaching and Learning,
2(1), 12-22.
Heise, M. (2017). From No Child Left Behind to Every Student Succeeds: Back to a
future for education federalism. Columbia Law Review, 117(7), 1859-1896.
https://columbialawreview.org/content/from-no-child-left-behind-to-every-student
-succeeds-back-to-a-future-for-education-federalism/
Hendricks, C. (2017). Improving schools through action research: A reflective practice
approach. Pearson.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
143
Improving America's Schools Act. (1994, October 20). 20 U.S.C. § 6301.
https://www.congress.gov/103/statute/STATUTE-108/STATUTE-108-Pg3518.pdf
Jacques, I. (2022, October 24). NAEP math, reading scores: Report card shows how
school closures hurt. USA Today.
https://www.usatoday.com/story/opinion/columnist/2022/10/24/report-card-studen
t-test-scores-reveal-how-much-closing-schools-hurt/10551234002/
Johansson, M. (2003, November). Textbooks in mathematics education A study of
textbooks as the potentially implemented curriculum. Luleå University of
Technology.
Katz, V. J. (2009). A history of mathematics : an introduction. Addison-Wesley.
Kaufman, J. H., Doan, S., & Fernandez, M.-P. (2021). The rise of standards-aligned
instructional materials for U.S. k–12 mathematics and english language arts
instruction [Findings from the 2021 American Instructional Resources Survey].
RAND Corporation.
https://www.rand.org/pubs/research_reports/RRA134-11.html
Lincoln, Y. S., & Guga, E. (1985). Naturalistic inquiry. Sage.
Loeb, H., Knapp, M. S., Elfers, A. M., & University of Washington. (2008, April 21).
Teachers’ response to standards-based reform: Probing reform assumptions in
Washington state. Education Policy Analysis Archives, 16(8).
https://doi.org/10.14507/epaa.v16n9.2008
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
144
Matthews, J. S., & López, F. (2019). Speaking their language: The role of cultural content
integration and heritage language for academic achievement among Latino
children. Contemporary Educational Psychology.
https://doi.org/10.1016/j.cedpsych.2018.01.005
Mehta, J. (2015, Summer). Escaping the shadow: A nation at risk and its far-reaching
influence. American Educator, 39(2), 20-26.
Mertler, C. A. (2021). Introduction to educational research. SAGE Publications,
Incorporated.
Molchanow, K. (2022, April 23). Pennsylvania bulletin (17, 52nd ed.). PA Code and
Bulletin.
https://www.pacodeandbulletin.gov/secure/pabulletin/data/vol52/52-17/52-17.pdf
Mollenkamp, D. (2022, October 24). NAEP ‘nation’s report card’ shows steep fall in
math scores. EdSurge.
https://www.edsurge.com/news/2022-10-24-naep-nation-s-report-card-shows-stee
p-fall-in-math-scores
Monessen City School District. (2023). About Us – District – Monessen City SD.
https://www.monessenschooldistrict.com/apps/pages/index.jsp?uREC_ID=13007
04&type=d&pREC_ID=1504939
Muñiz, J. (2019, March 28). Culturally Responsive Teaching. New America.
https://www.newamerica.org/education-policy/reports/culturally-responsive-teach
ing/
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
145
National Assessment of Educational Progress. (2022). 2022 Mathematics and Reading
Report Cards at Grades 4 and 8. The Nation's Report Card.
https://www.nationsreportcard.gov/
National Center for Education Statistics. (1995). Pursuing excellence: A study of U.S.
fourth-grade mathematics and science achievement in international context.
https://nces.ed.gov/timss/results95.asp
National Commission on Excellence in Education. (1983). A nation at risk: The
imperative for educational reform. U.S. Government Printing Office.
National Governors Association Center for Best Practices & Council of Chief State
School Officers. (2022, October 31). Common Core State Standards for
Mathematics. Council of Chief State School Officers.
https://ccsso.org/resource-library/common-core-state-standards-mathematics
No Child Left Behind Act. (2002, January 8). 20 U.S.C. § 6301.
https://www.congress.gov/107/plaws/publ110/PLAW-107publ110.pdf
Owens, J. E. (1988, August 9). Curriculum and evaluation standards for school
mathematics: Report of the national council of teachers of mathematics'
commission on standards for school mathematics. Institute of Education Sciences.
Pennsylvania Association of Intermediate Units. (2022). Focus. PAIU.
https://paiu.org/Focus
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
146
Pennsylvania Department of Education. (2013). Fact sheet: PA Academic Standards and
the Common Core.
https://www.sasd.k12.pa.us/Downloads/FACT%20SHEET%20for%20PA%20Aca
demic%20Standards%20and%20the%20Common%20Core%20April%202013.pd
f
Pennsylvania Department of Education. (2022). Future Ready PA Index district fast facts
- Greater Latrobe SD.
https://futurereadypa.org/District/FastFacts?id=195212081008215029032253089
092020162089031176052
Pennsylvania Department of Education. (2023). Future Ready PA Index.
https://www.education.pa.gov/K-12/ESSA/FutureReady/Pages/default.aspx
Pennsylvania Department of Education, Pennsylvania Educator Diversity Consortium, &
U.S. Department of Education's Region 4 Comprehensive Center. (2021, April).
The Pennsylvania Culturally Relevant and Sustaining Education Competencies.
https://s3.amazonaws.com/newamericadotorg/documents/The_Pennsylvania_Cult
urally_Relevant_and_Sustaining_Education_Competencies.pdf
Pennsylvania Joint State Government Commission. (1997, December). Intermediate units
in Pennsylvania: The role of educational service agencies in promoting equity in
basic education.
Pennsylvania State Board of Education. (2010). Implementing the Common Core
Standards in Pennsylvania: Reflections from the field. Pennsylvania Department
of Education.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
147
https://www.stateboard.education.pa.gov/Documents/Research%20Reports%20an
d%20Studies/Common%20Core%20White%20Paper.pdf
Petrin, K. (2022, October 23). NAEP scores show big drop in wake of COVID. Chalkbeat.
https://www.chalkbeat.org/2022/10/24/23417139/naep-test-scores-pandemic-scho
ol-reopening
Smith, D. R., & Ayers, D. F. (2006, June). Culturally responsive pedagogy and online
learning: Implications for the globalized community college. Community College
Journal of Research and Practice, 30, 401-415.
https://doi.org/10.1080/10668920500442125
Sparks, S. D., & Tomko, G. (2022, October 24). Two decades of progress, nearly gone:
National math, reading scores hit historic lows. Education Week.
https://www.edweek.org/leadership/two-decades-of-progress-nearly-gone-national
-math-reading-scores-hit-historic-lows/2022/10
Superfine, B. M. (2009). The courts and standards based reform. Oxford University
Press, USA.
Tampio, N. (2018). Common Core: National education standards and the threat to
democracy. Johns Hopkins University Press.
Tang, H. (2020, September). A qualitative inquiry of k–12 teachers’ experience with open
educational practices: Perceived benefits and barriers of implementing open
educational resources. International Review of Research in Open and Distributed
Learning, 21(3), 211-229. https://doi.org/10.19173/irrodl.v21i3.4750
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
148
Tang, H., Lin, Y.-J., & Qian, Y. (2020, April). Understanding K-12 teachers’ intention to
adopt open educational resources: A mixed methods inquiry. British Journal of
Educational Technology, 51(6), 2558–2572. https://doi.org/10.1111/bjet.12937
UNESCO. (n.d.). Open educational resources. UNESCO.org. Retrieved November 12,
2022, from
https://www.unesco.org/en/communication-information/open-solutions/open-educ
ational-resources
United States Department of Education. (2008). A nation accountable: Twenty-five years
after a nation at risk. U.S. Department of Education.
https://www2.ed.gov/rschstat/research/pubs/accountable/accountable.pdf
Watt, M. G. (2011). The Common Core State Standards Initiative: An Overview. Institute
of Education Services.
Westmoreland Intermediate Unit. (n.d.). Curriculum Services General Information Curriculum - Westmoreland IU 7. Westmoreland Intermediate Unit.
https://www.wiu7.org/apps/pages/index.jsp?uREC_ID=1559997&type=d&pREC
_ID=1605947
Xie, K., Nelson, M. J., Cheng, S.-L., & Jiang, Z. (2021, August). Examining changes in
teachers’ perceptions of external and internal barriers in their integration of
educational digital resources in K-12 classrooms. Journal of Research on
Technology in Education. https://doi.org/10.1080/15391523.2021.1951404
Yough School District. (2020). General Information/Alma Mater. Yough School District.
https://www.youghsd.net/general_informationalma_mater
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Appendix A
149
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
150
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
151
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
152
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Appendix B
153
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
154
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
155
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
156
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
157
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
158
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Appendix C
159
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
160
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Appendix D
161
Middle School Students’ Math Achievement Scores in Westmoreland County
A Doctoral Capstone Project
Submitted to the School of Graduate Studies and Research
Department of Education
In Partial Fulfillment of the
Requirements for the Degree of
Doctor of Education
Rebecca M Henderson
Pennsylvania Western University
July 2023
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Rebecca M. Henderson
July 2023
ii
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
iv
Dedication
To my children, Neil and Elizabeth.
The most meaningful things we do are often the hardest things we do. It's a
concept that may seem scary at first, but it's an essential truth that I want you to
understand and embrace. During your lives, you will encounter situations that appear
overwhelming, and it's normal to feel a sense of fear or doubt. Fear is a natural response,
but it should not determine your actions or hinder your progress. Instead, let courage
guide you and push through the fear. Trust in your abilities and believe in yourself. You
are capable of incredible things.
Remember, we do hard things.
Love, Mom
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
v
Acknowledgements
The completion of this Capstone Research Project would not have been possible
without the love, support, guidance, and encouragement that I received from family,
friends, and colleagues during this process. I want to start by thanking my husband,
Scott, and my children, Neil and Elizabeth. Thank you for believing in me and
supporting me, even when it meant spending less time together so that I could complete
this work. A special thank you to my parents, Jim and Betty, and the rest of my family
for their continued love and support. To my friends who have supported me, and laughed
with me, and have always kept me grounded. A special thank you to the following
colleagues: Dr. Jason Stragand and Dr. Denise Lohr, for encouraging me to enter this
program and going on this journey with me, Dr. Matt Thomas, Dr. Jason Boone, and Mr.
Timothy Hammill, for their friendship, humor, and wisdom, and Mr. Blaine Howard, for
suggesting the focus of my research. Thank you to Dr. J. Kevin Lordon, Faculty
Capstone Committee Advisor, who encouraged me throughout the process. A very
special thank you to Dr. Jason A. Conway, for your guidance, support, feedback,
encouragement, and wisdom during this process. I deeply value the environment that you
have established at the Westmoreland Intermediate Unit, where the focus on personal and
professional growth is paramount and nurtured for every employee. And as always,
thank you to Steve Perry and Corky St. Clair.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
vi
List of Figures and Tables
Figure 1: Westmoreland County Student Enrollments by Race / Ethnicity
7
Figure 2: Countywide Average by Anchor 2015 - 2019
8
Figure 3: What Culturally Responsive Competencies do State Teaching
Standards Address?
31
Figure 4: Growing Awareness Inventory Theory of Action
33
Figure 5: Greater Latrobe School District Percent Enrollment by Race/Ethnicity
50
Figure 6: Greater Latrobe School District Percent Enrollment by Student Groups
50
Figure 7: Greater Latrobe School District Algebra I Proficiency Percentages
2013 - 2022
51
Figure 8: Greater Latrobe School District Total Number of Student Tested vs.
Year Administered
52
Figure 9: Monessen City School District Percent Enrollment by Race/Ethnicity
53
Figure 10: Monessen City School District Percent Enrollment by Student
Groups
54
Figure 11: Monessen City School District Algebra I Proficiency Percentages
2013 - 2022
55
Figure 12: Monessen City School District Total Number of Student Tested vs.
Year Administered
56
Figure 13: Yough School District Percent Enrollment by Race/Ethnicity
57
Figure 14: Yough School District Percent Enrollment by Student Groups
58
Figure 15: Yough School District Algebra I Proficiency Percentages 2013 - 2022
59
Figure 16: Yough School District Total Number of Student Tested vs. Year
Administered
60
Figure 17: Group Summary: Winter 2012 Performance Keystone, Algebra I
78
Figure 18: Group Summary: Spring 2022 Performance Keystone, Algebra I
79
Table 1: All Districts’ Total Student Enrollments by Race/Ethnicity
80
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
vii
Figure 19: Percent Enrollments Economically Disadvantaged by District
81
Figure 20: Percent Enrollments ELL by District
82
Figure 21: Percent Enrollments Special Ed. by District
83
Figure 22: Enrollments by Gender (M/F) by District
84
Tabe 2: Building Principal Perception Survey Questions
85
Table 3 Teacher Perception Survey Questions
86
Table 4: Culturally Responsive STEAM Scorecard
88
Table 5: Group Summary, Performance Levels by Grade 08 and Ethnicity
92
Figure 23: GLSD Culturally Responsive STEAM Scorecard Results
95
Figure 24: YSD Culturally Responsive STEAM Scorecard Results
98
Table 6: Number of Algebra I Eligible Content Items Present by Vendor
99
Figure 25: Teacher Perceptions of Standards Alignment of Instructional
Materials
101
Figure 26: Building Principals’ Perceptions of Instructional Materials
Alignment to Standards
102
Figure 27: Teacher Perceptions of PD to Effectively Use Instructional Materials
103
Figure 28: Money Spent Annually By Districts on Teacher PD to Utilize
Instructional Material
104
Figure 29: Teacher Perceptions on the Impact of Instructional Materials on
Student Achievement
106
Figure 30: Teacher Perceptions on Physical Materials Preparing Students for
Keystone Exams
107
Figure 31: Teacher Perceptions on Digital Materials Preparing Students for
Keystone Exams
107
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
viii
Figure 32: Teachers' Perceptions of Student Reflections on Instructional
Materials
108
Figure 33: Building Principals’ Perceptions of Teacher Modifications to
Resources
109
Figure 34: Building Principals’ Perceptions of Teacher Development of Original
Resources
109
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
ix
Table of Contents
Dedication
iv
Acknowledgement
v
List of Figures
vi
Abstract
xiii
Chapter I. Introduction
1
Background of Study
1
Identification of Capstone Focus
3
Research Questions
4
Expected Outcomes
4
Fiscal Implications
4
Summary
5
Chapter II. Literature Review
6
Standards-Based Education Reform in the United States
8
The National Council of Teachers of Mathematics
10
National Education Goals
12
No Child Left Behind
14
Common Core State Standards
15
Pennsylvania Common Core Standards
16
The Every Student Succeeds Act
17
Publishers’ Responses to Standards-Based Education Reform
18
Textbook Content vs. Proficiency Exam Content
19
Textbook Structure
20
Standards Alignment
21
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Educators’ Response to Standards-Based Instructional Materials
x
22
Teacher Professional Development
23
Process for Adopting and Implementing New Standards in PA
24
A Mindset Shift in Mathematics
26
History of Culturally Responsive Teaching in the United States
27
Culturally Responsive Teaching in Mathematics
28
Culturally Responsive Teaching in Pennsylvania
30
The Impact of Reform on Culturally Responsive Teaching
32
Culturally Responsive Teaching as a Pedagogical Practice
34
Implementing OER to Facilitate Culturally Responsive Teaching
35
Perceived Benefits of Using Open Educational Resources
36
Perceived Barriers to Increasing Student Achievement
37
Understanding the Impact of COVID-19 on Student Achievement
38
NAEP Results by Student Group
41
Steep Declines in Algebra
41
Summary
42
Chapter III. Methodology
44
Methodology
44
Purpose of Study
45
Setting
46
Participants
49
Research Plan
60
Research Methods & Data Collection
66
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
xi
Data Collection Timeline and Plan Described
67
Validity
69
Summary
71
Chapter IV. Data Analysis and Results
Data Analysis
73
74
Participants
74
Data Analysis Procedure
75
Results
91
Research Question 1
91
Research Question 2
102
Research Question 3
105
Discussion
110
Summary
115
Chapter V. Conclusion and Recommendations
Conclusion
118
119
Research Question 1
120
Research Question 2
122
Research Question 3
123
Overview of Results
125
Limitations
136
Future Directions for Research
137
Summary
139
References
140
APPENDIX A. Building Principals Survey
149
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
xii
APPENDIX B. Teacher Perception Survey
153
APPENDIX C. Culturally Responsive STEAM Scorecard
159
APPENDIX D. IRB Approval
161
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
xiii
Abstract
Building Administrators and Algebra I teachers in Westmoreland County have worked
hard to determine the root causes of students’ historically low achievement scores on the
Algebra I Keystone Exam over the past decade. Through intensive data analysis, they
have investigated the alignment of their curriculum to Pennsylvania Core State Standards,
their instructional practices, and the alignment of their in-class assessments to Eligible
Content Items covered on the Algebra I Keystone Exam. Yet one area that has not been
intensively investigated is the impact of culturally responsive instructional materials on
student achievement scores. This research study focused on building administrator and
teacher perceptions of their primary instructional materials used in Algebra I classrooms,
as well as the amount of time and money dedicated to support educator professional
development to successfully implement those resources in their classrooms. Participants
completed perception surveys to share their thoughts about their instructional material,
supplemental materials, Open Educational Resources, professional development, and
time and funding spent by their district to support teachers. They also completed the
Culturally Responsive STEAM Scorecard to evaluate the cultural responsiveness of their
primary instructional materials. In addition, the researcher evaluated the presence of
Eligible Content Items present on the Algebra I Keystone Exam on primary Algebra I
instructional materials used by three case study districts representing Westmoreland
County. While there was not a strong correlation between the culturally responsive
nature of instructional materials and student achievement scores, there was evidence that
shows that the lack of Eligible Content Items present in primary instructional materials
negatively impacts student achievement scores.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
1
Chapter I
Background
Each year, the Curriculum Services Team of the Westmoreland Intermediate Unit
(WIU) works with the seventeen public school districts within their geographic region to
analyze student achievement data, evaluate their curriculum and instruction, and support
student achievement and success. This work focuses more on standards alignment and
state-mandated standardized test scores and less on instructional materials' impact on
student achievement. Previous actions to complete this work include mapping standards
to a district’s curriculum, supporting the development of a more thorough curriculum and
concept maps that address tested content, and a deeper understanding of the state
standards and how to align instruction to them.
While there is less consideration given to the actual instructional materials used to
support instruction, those materials make a significant impression on students and
teachers and steer teaching and learning within the classroom. Districts rarely separate
their curriculum discussions into defined categories. Instead, they use the term
curriculum to refer to standards, the scope and sequence of content taught, the instruction
delivered, the instructional materials used by teachers to engage students, and the
assessments developed and used to determine student achievement.
The researcher has been a part of the Curriculum Services Team at the WIU since
August 2018. She has served as Distance Learning Supervisor and Curriculum Services
Supervisor during this time. In this capacity, she has been responsible for supporting
districts as they work to implement instructional initiatives and mandates as set forth by
the Pennsylvania Department of Education (PDE) in both traditional and online
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
2
classroom environments. In serving as a liaison between PDE and the seventeen
public school districts in Westmoreland County, the WIU’s Curriculum Services team
utilizes innovative practices to advance the practice of teaching and learning across
Westmoreland County.
School Improvement and Accountability is a central focal point of this work for
the WIU Curriculum Services Team, as the United States Department of Education
(USDOE) has outlined annual requirements that both states and local school districts
must meet to qualify for funding. As such, the WIUs Curriculum Services Team is
committed to supporting and monitoring the school improvement efforts of districts.
These efforts typically include curriculum writing, standards analysis and alignment,
assessment data analysis, instructional strategy, and resource review and implementation.
As part of this work, the WIU’s Curriculum Services Team regularly analyzes
district-specific data on the annual Pennsylvania System of School Assessment (PSSA)
and Keystone Exams. The PSSA is a standards-based assessment administered to every
student in grades 3 through 8. Students are assessed in English Language Arts and
Mathematics each year and in Science in grades 4 and 8. The Keystone Exams serve as
end-of-course assessments determining a student’s proficiency in Algebra I, Literature,
and Biology. These assessments meet the federal accountability criteria established in the
Every Student Succeeds Act (ESSA) of 2015.
Each year, PDE releases the scores of all students, all districts, and the statewide
average score for each subject area. Historically, Algebra I scores across Westmoreland
County’s seventeen public schools have been below the state average, which has been an
ongoing concern for administrators. Administrators want to understand the root cause of
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
3
these county-wide historically low scores to support teachers and students better. They
have asked the WIU Curriculum Services Team to help them analyze their specific
curriculum, instruction, instructional materials, and assessment circumstances and
provide consultative services that can promote increased student achievement.
Identification of Capstone Focus
This research will investigate the possible correlation between the historically low
Keystone Algebra I Scores in middle school and building principal and Algebra I
teachers’ perceptions of student identification with examples in the word problems found
within their middle school instructional materials. There are many barriers to
implementing Open Educational Resources (OER) and developing culturally-responsive
instruction that reflects a district’s community, demographics, and values to establish
equitable instructional practices. Despite the amount of funds made available to districts
through the Elementary and Secondary School Emergency Relief (ESSER) Fund, and the
resources compiled by the Pennsylvania Department of Education on their Equitable
Practices Hub, there appears to be hesitancy to develop instructional materials that meet
the needs of their individual student populations.
The research is showing that when students see themselves reflected in their
instructional content, they learn more and understand content and concepts on a deeper
level. Yet, the conversations around school districts in Westmoreland County focus on
purchasing technologies, tools, and programs developed by national vendors that do not
reflect the students within the county. This study seeks to better understand the
correlation between the instructional materials currently in use by 7th and 8th-grade
mathematics teachers, their perceived cultural responsiveness by district building
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
4
principals, mathematics teachers, and students, and student achievement scores in
Operations with Real Numbers and Expressions in Algebra I classes.
Research Questions
The following questions will guide this case study:
1. What are the factors that impact student achievement in Algebra I?
2. What are the factors that impact instructional delivery in Algebra I?
3. What are the teacher and administrator perceptions of the impact of instructional
delivery on student achievement scores in Algebra I?
Expected Outcomes
There are many barriers to implementing Open Educational Resources (OER) and
developing culturally-responsive instruction that reflects a district’s community,
demographics, and values to develop equitable instructional practices. Despite the
amount of funds made available to districts through the Elementary and Secondary
School Emergency Relief (ESSER) Fund, and the resources compiled by the
Pennsylvania Department of Education on their Equitable Practices Hub, there appears to
be hesitancy to develop instructional materials that meet the needs of their individual
student populations. The results of this case study will provide building administrators
and Algebra I teachers with additional guidance and support when determining which
instructional materials may significantly impact student achievement within their student
populations.
Fiscal Implications
The fiscal implications of this capstone research project consist solely of the cost
associated with the salaries and benefits of the participants for the time required to
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
5
complete the activities within this case study. Each participant will complete a
Perception Survey at the start of the case study, which will take 20 minutes to complete.
They will then complete the Culturally Responsive STEAM Scorecard, developed by
NYU Steinhardt, to share their perception of the cultural relevance of their primary
instructional material used for Algebra I as the second step of data collection. This
activity will take approximately two hours to complete. Since the participants are
employees at three representative districts across Westmoreland County, the cost will be
distributed evenly among those districts. The researcher will utilize Google Forms to
collect perception survey data at no additional cost to the school districts or the
Westmoreland Intermediate Unit. The researcher has also been granted permission to
fully use and publish components of the Culturally Responsive STEAM Scorecard, a
freely available online resource, by NYU Steinhardt at no additional charge. All costs
associated with this capstone research project are indirect, and the overall financial
impact for an individual school district will be minimal.
Summary
Chapter I introduced the background context of this case study. It also explained
why studying the impact of instructional materials on student achievement scores in
Algebra I across Westmoreland County is essential for administrators and educators to
develop culturally responsive instructional experiences that promote increased student
achievement. Chapter II will provide a literature review that helps create a thorough
understanding of the currently available research to help inform instructional practices.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
6
Chapter II
Literature Review
Since the early 1980s, Standards-Based Education Reform has driven public
education in the United States. This type of education reform calls for the establishment
of academic content standards, which are used to determine critical elements that students
must know within each content area. Student achievement is also measured against these
established standards, making high-stakes testing a key component to monitoring student
progress annually. Historically, the individuals that determine the need for
Standards-Based Education Reform are not the same individuals that are responsible for
the implementation of the standards at the classroom level.
While structures are in place to ensure that educators have access to high-quality,
job-embedded professional development that is designed to help them implement newly
adopted standards within the classroom, much of that professional development is
focused on instructional and assessment strategies. Historically, little attention has been
given to assisting educators in critically examining instructional materials in order to
determine the accuracy of their standards alignment. Unfortunately, instructional
materials are one component of a district’s implemented curriculum that has the greatest
impact on student achievement and mastery of newly adopted standards.
Westmoreland County, located in the Southwestern region of Pennsylvania, is
home to approximately 43,710 students in grades Kindergarten through twelfth grade
across 17 different public school districts. The smallest school district serves
approximately 712 students, while the largest serves approximately 5,304 students.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 1 shows student enrollments in Westmoreland County public school districts by
Race / Ethnicity.
Figure 1
Westmoreland County Student Enrollments by Race / Ethnicity
While student populations across the county tend to be overwhelmingly white,
each district has a unique culture and climate based on geographic location, population
demographics, and societal norms. A commonality between all districts is their
historically low scores in Algebra I on state standardized tests. Since 2015, countywide
averages have only risen above a 60% proficiency rating one time. Figure 2 shows the
countywide average scores by Algebra I Anchor Analysis from 2015 - 2019.
7
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
8
Figure 2
Countywide Average by Anchor 2015 - 2019
These scores have become a major concern for district administrators and building
leaders across Westmoreland County who are analyzing their intended curriculum and
implemented curriculum in order to determine how to better support students in the
classroom and foster higher levels of student achievement.
Standards-Based Education Reform in the United States
Standards-based education reform in the United States of America has its roots in
the 1983 publication titled, “A Nation At Risk: The Imperative for Educational Reform.”
This report, authorized by President Ronald Reagan, was developed by an
eighteen-member commission of individuals working within private companies,
government agencies, and education organizations. The authors determined that declines
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
9
in the efficacy of public education were due to inconsistencies in four key areas of
education: the content or curriculum being covered in classrooms, district expectations
for students, the time that is dedicated to education over the course of the year, and the
quality of programs designed to prepare candidates for the teaching profession as well as
continuing professional development once teachers enter the profession.
As the authors laid out their case for the need for a more rigorous approach to
K-12 public education in the United States, and warned of an increase in mediocrity
among students, the authors state, “Our goal must be to develop the talents of all to their
fullest. Attaining that goal requires that we expect and assist all students to work to the
limits of their capabilities. We should expect schools to have genuinely high standards
rather than minimum ones, and parents to support and encourage their children to make
the most of their talents and abilities” (National Commission on Excellence in Education,
1983, p. 13). Furthermore, the authors felt that there must be a focus on equity for
students. They cautioned that sacrificing equity over education reform, or education
reform over equity, would lead to potentially disastrous consequences.
We do not believe that a public commitment to excellence and educational reform
must be made at the expense of a strong public commitment to the equitable
treatment of our diverse population. The twin goals of equity and high-quality
schooling have profound and practical meaning for our economy and society, and
we cannot permit one to yield to the other either in principle or in practice. To do
so would deny young people their chance to learn and live according to their
aspirations and abilities. It also would lead to a generalized accommodation to
mediocrity in our society on the one hand or the creation of an undemocratic
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
10
elitism on the other (National Commission on Excellence in Education, 1983, p.
13).
The strongest conclusion from this report was that, in order to effectively ensure
that students would be able to productively contribute to society and that future
generations would be able to compete on a global stage, we would need to develop a
more rigorous and structured curriculum that would address the relevant concepts and
skills that students would need to master prior to graduation in an equitable manner.
Doing so would instill a lifelong love of learning that would encourage individuals to
continue seeking learning opportunities throughout their adult lives. It would also
continue to foster support for public education well into the future.
The report was distinguished historically from others for its strikingly alarmist
tone. Mehta (2015) concluded:
Public confidence in schooling, which had already been falling as part of the
post-Watergate decline of confidence in public institutions, hit a new low after the
release of the report in 1983. States rushed to issue their own reports to evaluate
whether they were falling short of the new measures of excellence specified by A
Nation at Risk. Despite the claims of the critics, A Nation at Risk had been the
rare report that galvanized not only debate but also action. (p. 22)
The National Council of Teachers of Mathematics
The action that resulted from A Nation at Risk is commonly referred to as
Standards-Based Education Reform. One of the first recommendations supporting
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
11
Standards-Based Education reform came in 1988 from the National Council of Teachers
of Mathematics (Owens, 1988), which released a report titled, “Curriculum and
Evaluation Standards for School Mathematics: Report of the National Council of
Teachers of Mathematics’ Commission on Standards for School Mathematics.” While
there were other recommendations, this report focused heavily on the need for schools to
“implement a core curriculum that allows all students the opportunity to study the
important ideas and methods of mathematics.” (Owens, 1988, p. 2) In order to allow for
time to address new concepts within the core curriculum, the authors recommended a
de-emphasis on practicing computational skills with pencil and paper. Instead, they
encouraged the use of technology such as calculators and computers to assist students in
computation so that they were able to prioritize developing a solid conceptual foundation
for skills.
Computational skills are defined as more than mere arithmetical competencies.
They include, at the secondary level, the extensive manipulation of expressions
and equations that are central to current study in Algebra--a considerable portion
of the current Algebra I curriculum, for example, is devoted to manipulation skills
associated with solving equations, simplifying complex rational expressions, and
factoring. As with arithmetic skills, these are not envisioned as unnecessary but
as areas that should be de-emphasized; conceptual development of the ideas
underlying such operations should take precedence, calculator and computer
solutions are stressed, and applications serve as both motivators of operations and
practice opportunities for developing skills. (Owens, 1988, p. 3)
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
12
This strategy of de-emphasizing paper and pencil computational skills was
developed to better assist teachers in putting a priority on supporting students’
developmental readiness for instruction. Teachers would now have time to focus on
instructional strategies designed to support students in developing their conceptual
thinking and problem solving skills at various age levels. The end result would be
students that can make real-world connections to the content, solve complex problems,
make connections to the content year after year, and communicate about mathematical
constructs.
National Education Goals
This was further reinforced in 1990 by President George H.W. Bush and the
Nation’s Governors, who adopted six National Education Goals that were to be achieved
within the next ten years. These goals formed the basic structure for Standards-Based
Education Reform that are still present in the public education system of the United
States today.
The six National Education Goals establish that, by the year 2000:
● All children in America will start school ready to learn.
● The high school graduation rate will increase to at least 90 percent.
● American students will leave grades 4, 8, and 12 having
demonstrated competency in challenging subject matter including
English, mathematics, science, history, and geography; and every
school in America will ensure that all students learn to use their
minds well, so they may be prepared for responsible citizenship,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
13
● Further learning, and productive employment in our modern
economy.
● United States students will be first in the world in science and
mathematics achievement.
● Every adult American will be literate and will possess the
knowledge and skills necessary to compete in a global economy
and exercise the rights and responsibilities of citizenship.
● Every school in America will be free of drugs and violence and
will offer a disciplined environment conducive to learning.
(Educate America Act, 1994)
These goals, intended to be seen as overlapping, intricate goals that complimented
each other, were designed to provide a more structured framework for public education.
Officially becoming known as the Neighborhood Schools Improvement Act, the 102d
Congress debated the merits of the six goals, as well as the roles of individuals and
organizations that would be involved in monitoring public schools as they worked
towards achieving those goals.
Ultimately, the 102d Congress did not formally pass the Neighborhood Schools
Improvement Act and did not formally adopt the goals developed by President Bush and
the Nation's Governors at this time. Instead, the previously adopted Elementary and
Secondary Education Act of 1965 (ESEA) was reauthorized in 1994 under President Bill
Clinton. This reauthorization, called the Improving America’s Schools Act of 1994,
acknowledged the importance of the National Education Goals, and began to develop
parameters in which schools would need to operate so that students could meet those
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
14
goals. As stated in the document on page 3, since 1988 the country has learned
that “All children can master challenging content and complex problem-solving skills.
Research clearly shows that children, including low-achieving children, can succeed
when expectations are high and all children are given the opportunity to learn challenging
material.” Again, there was a reiteration of the need for a focus on problem-solving skills
over computational skills, and in setting higher expectations for students in order to push
their understanding and mastery of a more challenging curriculum.
No Child Left Behind
This pattern of reauthorizing ESEA and renaming it would continue into both the
George W. Bush and Barack Obama Administrations. During the George W. Bush
Administration’s reauthorization, also known as the No Child Left Behind Act of 2001
(NCLB), we still see a strong preference for establishing high achievement standards,
however now we see a shift away from a common set of national academic standards.
Instead, states are directed to develop their own set of academic standards as well as
accompanying assessments to measure student understanding and mastery of content. To
stress the importance of this development, Title VI, referred to as Section 601 Flexibility
and Accountability, laid forth several conditions for states to receive grant funding for the
development of standards-aligned, state assessments designed to ensure validity and
reliability by assessing student achievement through multiple measures (United States
Congress, 2002).
We now see a shift in control for K-12 public education. While accountability has
been a central tenant of Standards-Based Education Reform since the publication of A
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
15
Nation at Risk, the control has largely laid at the feet of the individual states. With the
inception of NCLB, control over accountability methods now shifted to the federal
government. However, it would not be the last shift in control that we would see.
Despite a tradition, or at least a conventional myth, of local control that
dominated education federalism for decades, federalism boundaries are
increasingly noted for their fluidity in the public education sector. Not only are
federalism boundaries becoming increasingly fluid, but the pace of change to
these boundaries has also quickened over time, no doubt a reflection of
increasingly contested federalism claims. (Heise, 2017)
The replacement for NCLB, the Every Student Succeeds Act of 2015 (ESSA), shifted
that control back to the states again.
Common Core State Standards
In the midst of this back and forth regarding accountability oversight and control,
the conversation regarding the establishment of a core curriculum continued. In 2008,
the National Governors’ Association established a group of experts to develop a common
set of core standards that could be utilized across all fifty states. In order to do so, this
group of experts needed to analyze the current standards already in place in each state
and determine whether or not those standards were high-quality, rigorous, and effective.
“The common critique was that most American grade-level guidelines were “a mile wide
and an inch deep,” in stark contrast to the fewer but more intense expectations in
high-achieving countries like Japan and Singapore” (Garland, 2020). Out of their work,
the Common Core State Standards Initiative (CCSS) was developed to address learning
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
16
standards in English Language Arts and Mathematics. “The Standards for Mathematical
Practice describe varieties of expertise that mathematics educators at all levels should
seek to develop in their students” (National Governors Association Center for Best
Practices & Council of Chief State School Officers, 2022). The focus of the mathematics
standards became processes and proficiency in order to ensure that students were able to
not only complete each step of a given problem, but also explain those steps as well as
the ways in which the steps were connected to each other.
This emphasis on processes and proficiency was a strategic design meant to
support students and help to prepare them for their future endeavors. “The goal is not
just to have students master calculations that they will perform only on standardized tests;
the goal is to have them think in ways that will prepare them for college, careers, and
life” (Tampio, 2018). Again, we see a prioritization of skills that will support students as
they move beyond high school.
Pennsylvania Common Core Standards
While CCSS were developed to be relevant, rigorous, and applicable to students
in all fifty states, not all fifty states adopted the standards as written. As the Pennsylvania
Department of Education (PDE) publicized, “On July 1, 2010, the State Board of
Education adopted improved standards in math and English to replace the original
standards adopted in 1999. The standards were based off the Common Core initiative”
(Pennsylvania Department of Education, 2013). PDE recognized the work that went into
the development of CCSS, but acknowledged that the state-specific standards that had
been developed and approved for Pennsylvania, referred to as Pennsylvania Common
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
17
Core (PA Core), were tailored to meet the specific needs of students within the
Commonwealth. Furthermore, PDE took steps to ensure that the public understood that
the standards were written to serve as guidelines that would assist local districts and their
personnel in developing their curriculum.
The Every Student Succeeds Act
The most current reauthorization of ESEA, known as the Every Student Succeeds
Act of 2015 (ESSA), was signed into law in 2015 by President Barack Obama. At this
point, Standards-Based Education Reform in the United States was now forty years old.
This latest reauthorization continued to emphasize the importance of increased rigor and
accountability.
In 2012, the Obama administration began granting flexibility to states regarding
specific requirements of NCLB in exchange for rigorous and comprehensive
state- developed plans designed to close achievement gaps, increase equity,
improve the quality of instruction, and increase outcomes for all students.
(USDOE, 2015)
While NCLB was written in such a way that it did not prevent the federal
government from promoting a specific set of standards to states, ESSA was written so
that the federal government was unable to influence a state’s adoption of a specific set of
standards in order to maintain a clear delineation between state and federal authority over
K-12 public education (Every Student Succeeds Act, 2015).
The flexibility of ESSA and the inherent shift back to state accountability and
oversight was touted by many advocates as a key triumph of the new law. The Center for
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
18
American Progress (CAP) noted that this flexibility would be an opportunity to empower
local school districts to become innovative in their approaches to accountability. They
stressed that this could be achieved by collaborating with stakeholder groups to critically
analyze their current testing structures. In doing so, this would allow them to design
better-aligned assessment systems that “include the smart and strategic use of formative
and interim tests that provide real-time feedback to inform instruction, as well as
high-quality summative tests that measure critical thinking skills and student mastery of
standards.” (Brown et al., 2016)
Publishers’ Response to Standards-Based Education Reform
To study the impact of Standards-Based Education Reform, the National Center
for Education Statistics (NCES) established the Trends in International Mathematics and
Science Study (TIMSS) in 1995. This study, which has been administered to students in
grade 4 and grade 8 every four years, assesses and compares student achievement and
educational experiences in math and science by collecting data from students, parents,
educators, and administrators (NCES, 2022). The United States has been participating in
this study since its inception, providing us with 24 years worth of data as of this writing.
The TIMSS Framework helps us to critically examine instructional materials,
instruction, assessment, and their alignment to each other in order to determine their
impact on student understanding and achievement. In working within the TIMSS
Framework, we focus on the following terms:
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
19
● Intended Curriculum: National, social, and educational contexts such as
standards documents, curriculum documents, vision statements, and educational
objectives.
● Implemented Curriculum: Home, school, teacher, and classroom contexts such as
classroom management, instruction, instructional materials used, and the ways in
which content is interpreted by teachers.
● Attained Curriculum: Student achievement and attitudes, most commonly
obtained through student test scores, and student belief measures. (NCES, 1995)
Textbook Content vs. Proficiency Exam Content
As Standards-Based Education Reform gained prominence in both federal and
state education policy and practice, K-12 textbook and instructional content publishers
took notice. Since textbooks were designed to be the instructional tools that help teachers
bridge the gap between the intended curriculum of an educational system and the
attained curriculum of an individual student, publishers recognized the importance that
Standards-Based Education Reform would have, and marketed their products
accordingly. In her analysis of mathematics textbooks utilized by countries all over the
world, Johanssen (2003) observed that, while many of the textbooks used within the
United States have a strong focus on contextual features and performance requirements,
the content covered within the textbooks was disproportionately matched to the content
that was on student proficiency exams, with the greatest areas of mismatch being in
arithmetic, measure, and algebra (p. 28). This is of great concern, as Johanssen goes on
to note that in the United States, teachers use textbooks in the least flexible manner,
preferring to navigate through them as a part of their implemented curriculum in order,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
20
lesson by lesson, using the information as presented, regardless as to whether that
information aligns to their intended curriculum or locally developed assessment.
Textbook Structure
In analyzing the response to Standards-Based Education Reform made by
textbook and instructional content publishers, an obvious data point comes to us in the
number of pages within mathematics textbooks. Overwhelmingly, publishers in the
United States include more pages within their mathematics textbooks than publishers
from other countries (Johanssen, 2003). Even with this inclusion of additional pages, the
structure of reform-oriented textbooks in the United States focused on including as much
content as possible within the textbook itself. However, in comparing textbooks to one
another, there is a great deal of variance in the specific content within each textbook
(Davis et al., 2014).
What this clearly shows is that while textbook and instructional content publishers
understood the importance of Standards-Based Education Reform on their products, they
were unable to develop an industry-wide approach to addressing the changes in content
and instruction that came out of this reform initiative. However, similar marketing
approaches were employed by publishers. Textbook and instructional content publishers
focused on their marketing efforts, adjusting language to show that their products were
aligned to CCSS or state-specific standards, however they made little changes to the
content itself, continuing to perpetuate the gap between the intended curriculum and the
attained curriculum. As Kaufman et al. (2021) acknowledge, “One critical reason for
this disconnect between state standards and instructional practice is that teachers’
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
21
curriculum materials—the textbooks and other materials that drive lesson content—are
frequently not aligned with state standards (Kaufman et al., 2020; Polikoff et al., 2020).
Standards Alignment
“The influence of the idea of alignment is evident today in the large number of
textbooks, assessment systems, and professional development materials that are explicitly
marketed as being aligned with state standards” (Hamilton et al., 2008). They go on to
note that while standards alignment is now the universally accepted measure of
endorsement for use by educators, “there is no widely accepted method for aligning
standards, curriculum, instruction, and assessments, nor is there a consensus on how to
determine if existing components are aligned” (Hamilton et al., 2008).
The implication is that while most textbook and instructional content publishers
market their materials as being aligned to either CCSS, or state-specific standards, there
is no mechanism in place for those publishers to determine the consistency and accuracy
of their alignment. The consistency and accuracy of their alignment, instead, falls to
individual districts as they review materials in consideration of their purchase and use
within their classrooms. However, most educators are not provided with adequate
training or professional development designed to help them analyze the consistency and
accuracy of perceived alignments. Rather, they look to documentation provided to them
from the publisher that explains their rationale for their alignment.
Furthermore, there is a gap present in the development of math textbooks and
instructional content regarding the connection between content standards, teacher’s
content knowledge, and their pedagogical knowledge (Copur-Gencturk et al., 2022). This
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
22
gap makes the connection between the intended curriculum and the attained curriculum
much harder to bridge if the materials used for the implemented curriculum are not
designed to consider this connection.
Educators’ Response to Standards-Based Instructional Materials
As Loeb et al. (2008) point out, teachers’ responses to Standards-Based Education
Reform and their actions in the classroom are central to students achieving the desired
attained curriculum. As such, they must be provided with high-quality, job-embedded
professional development that is designed to help them understand the intricacies of the
reform initiatives, the ways in which student achievement is assessed, and the ways in
which their instructional materials are aligned to reform initiatives. In doing so, we are
able to address the assumptions about Standards-Based Education Reform and the ways
in which teachers will respond in order to align and improve their instruction. Those
assumptions include:
● Assumption One: Teachers will pay attention to the reform and become
familiar with the standards and what they imply for practice.
● Assumption Two: Teachers will take the reform seriously, as will their
supervisors and other local leaders, who will exhort teachers to meet the
demands of the policy, and offer support, as needed.
● Assumption Three: Teachers will adjust their instruction to align with the
standards and associated assessments, including preparation for
assessment.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
23
● Assumption Four: Teachers will expect all of their students to
succeed—and believe that they are capable of succeeding.
● Assumption Five: Teachers will have access to appropriate professional
learning opportunities. (Loeb et al., 2008, p.4)
Teacher Professional Development
The time that is needed for teachers to be adequately trained on the
implementation of newly adopted standards is substantial. To be most effective, this
professional development should be provided in an ongoing manner so that teachers can
engage in new learning frequently, allowing them to have regular opportunities to
collaborate with their peers. This collaboration should include rich discussion around
designing instructional strategies that engage students and help them achieve mastery of
new concepts and skills brought forth from new standards, and a critical review of
instructional materials to ensure that they are aligned to the standards and will effectively
support the implemented curriculum. This can become a daunting task, as it may require
the adjustment of teacher schedules to ensure that sufficient time is dedicated towards
providing them with meaningful professional development.
There must be a clear delineation between understanding the impact of
Standards-Based Education Reform and the intended curriculum on student achievement,
the impact of teacher instruction and the implemented curriculum on student
achievement, and the impact of the instructional materials used by teachers as a part of
the implemented curriculum on student achievement. We have evidence that shows a
clear connection between teacher professional development for instructional strategies
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
24
and an increase in student achievement. As Chingos and Whitehurst (2012) show, there
is also evidence that shows that the instructional materials utilized by teachers as a part of
the implemented curriculum have a large impact on student achievement and the attained
curriculum. To drive this point home, they state:
Student learning occurs primarily through interactions with people (teachers and
peers) and instructional materials (textbooks, workbooks, instructional software,
web-based content, homework, projects, quizzes, and tests). The contexts within
which these interactions occur are surely important. The effectiveness of
teachers, the behavior of peers, and the instructional materials with which
students have the opportunity to interact are affected by layers of influences that
range from circumstances in the home, to leadership in the school, to the
international macro-economy and everything in between. But students do not
engage in instructional interactions with the governance mechanism for their
school or with state standards or with a school district’s evaluation system for
teachers or with collective bargaining agreements or with the leadership qualities
of their school principal - they learn by engaging in cognitive processes that are
triggered and shaped by interactions with people and instructional materials.
(Chingos & Whitehurst, 2012, p. 3)
Process for Adopting and Implementing New Standards in PA
In the Commonwealth of Pennsylvania, there is a formalized process for adopting
and implementing new standards. Once the standards’ language has been approved by
the Pennsylvania State Board of Education, the Pennsylvania Department of Education
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
25
(PDE), in collaboration with their 29 intermediate unit partners, will work together to
present the new standards language to educators, and provide training for classroom
implementation. This process ensures that all administrators and educators are aware of
the new standards, the manner in which they will be assessed, and are well-equipped to
provide high-quality instruction that addresses them within their classrooms.
As new standards are adopted in the Commonwealth of Pennsylvania, there is a
direct impact on the ways in which districts, school buildings and individual educators
are held accountable for their implementation and student’s understanding and mastery of
them. This occurs through a combination of educator evaluation tools, student
assessment tools, and school progress tools. Educators are required to address the
implementation of the newly adopted standards within the classrooms as the adoption is
directly tied to their individual annual evaluations.
As noted by Kaufman et al. (2021), “ By themselves, state academic standards do
not lead to major shifts in what teachers do in their classrooms.” In order to adequately
support teachers as they adjust their instruction in order to meet new standards, teachers
need access to high-quality, standards-aligned instructional materials as well as engaging,
job-embedded professional development that helps them to understand the most effective
and efficient ways in which to use those materials in their classrooms in order to help all
students achieve academic success. “This means that states and districts must continue
working to push use of standards-aligned materials, along with curriculum-focused
professional development that can improve use of those materials and learning outcomes”
(Kaufman et al., 2021). Without this training, teachers will struggle to accurately address
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
26
the standards in a comprehensive manner as they select instructional materials for their
implemented curriculum.
A Mindset Shift in Mathematics
The overarching disconnect between the intended curriculum, the implemented
curriculum, and the attained curriculum comes down to the fact that instructional
materials are not as flexible or as well-aligned to standards as teachers need them to be in
order to ensure that students master the concepts and skills outlined within their adopted
standards. It is at this point that we need to consider the goal of Standards-Based
Education Reform, and teachers’ responses to it. A central tenet of Standards-Based
Education Reform is that all students are capable of achieving mastery in difficult content
areas, and that rigorous standards should be adopted in order to ensure that students do
so.
This is a relatively recent shift in mindset. Traditionally, more advanced content
was taught only to students that needed to engage with that content, going all the way
back to the Ancient Egyptian and Babylonian Eras. Katz (2009) points to this clearly
with regards to Algebra instruction,
“In fact, throughout most of recorded history, algebra was only taught to the
“few”, not the “many”. So the concept of “algebra for all” is a very recent one
and one that will take a lot of work to make into a reality.”
While each ESEA reauthorization has attempted to ensure that instruction and
learning would be equitable, the statewide adoption of common standards has decreased
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
27
opportunities for personalized instruction designed to meet the needs of students as
teachers have felt the pressures of increased assessments for accountability purposes.
While the implementation of common standards in education was to ensure that
all students receive the same education regardless of where they live through the intended
curriculum, their use within the implemented curriculum has decreased instances of
personalized instruction tailored to students’ specific needs. This shift away from
personalized instruction is not a new phenomenon in the history of education in the
United States. It reflects a long-standing tension between the ideals of standardization
and the need for culturally responsive teaching practices that recognize and value the
diverse backgrounds and experiences of students.
History of Culturally Responsive Teaching in the United States
The roots of Culturally Responsive Teaching in the United States can be traced
back to the 1970s, before the publication of A Nation At Risk. Also referred to as
culturally appropriate, culturally relevant, culturally congruent, or culturally compatible
teaching, this pedagogy is designed to be highly responsive to the backgrounds and
identities of the individuals within a given classroom in order to tailor instructional
materials, examples, and scenarios so that students can personalize them to their own life
experiences. The most widely accepted formal definition of Culturally Responsive
Teaching, defined by Gay (2010, p. 31) is “using the cultural knowledge, prior
experiences, frames of reference and performance styles of ethnically diverse students to
make learning encounters more relevant to and effective for them.”
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
28
As teachers have struggled to ensure that students meet mandated achievement
benchmarks due to Standards-Based Education Reform, the gap between the intended
curriculum and the attained curriculum has become more prevalent. In response,
teachers have begun to recognize that students need instructional materials and
instructional experiences that are reflective of their personal, cultural experiences for the
implemented curriculum to be more effective in meeting their learning needs (Harmon,
2012). Engaging in Culturally Responsive Teaching has become an effective method for
ensuring that students not only understand the concepts and skills being taught in the
classroom, it helps to ensure that students are able to relate to those concepts and skills by
introducing them to students through culturally familiar scenarios and explanations that
allow them to engage in meaningful learning (Harmon, 2012). “It is a means for
improving achievement by teaching diverse students through their own cultural filters”
(Gay, 2010, p. 50). This pedagogical approach becomes a transformative experience for
students and teachers by empowering them to engage in a personalized instructional
experience during the implemented curriculum.
Culturally Responsive Teaching in Mathematics
Most educators are quick to see the connection between Culturally Responsive
Teaching and literacy-centric subjects, such as English Language Arts. However
Culturally Responsive Teaching is effective in all content areas, as demonstrated by
Matthews and López (2019), who sought to reinforce this point as they centered their
work on the importance of Culturally Responsive Teaching in mathematics. They found
that initially most teachers did not recognize the importance of Culturally Responsive
Teaching on academic goals, favoring the pedagogy to reinforce socio-engagement goals
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
29
for students. However, as teachers engaged more fully in Culturally Responsive
Teaching within mathematics classrooms they noticed increased levels of student
achievement among Latino students (Matthews & López, 2019). The emphasis on
designing learning experiences that are reflective of students’ lived experiences provide
them with additional conceptual resources that allow them to make stronger connections
to the implemented curriculum, thus increasing the amount of attained curriculum that
they experience.
While mathematics is often viewed as a culturally neutral subject, it is deeply
rooted in culture and history, making the need for culturally responsive teaching in
mathematics crucial to creating inclusive and equitable learning environments for all
students. By incorporating culturally relevant examples and activities into math
instruction, teachers can help students see the relevance and applicability of math in their
own lives and communities. Culturally responsive teaching in math can also help address
equity gaps by ensuring that all students have access to rigorous and meaningful math
instruction that is culturally relevant and responsive to their needs.
Comparing the current demographics of our teaching force to the demographics of
our student populations further highlights the need to engage in Culturally Responsive
Teaching.
The need for culturally responsive teaching is more pressing than ever before,
especially when you consider the deep demographic gaps between teachers and
students. A teaching workforce that remains overwhelmingly female, white,
middle-class, and monolingual is increasingly likely to teach students who are of
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
30
a different race, ethnicity, socio-economic status, language group, and so on.
(Muñiz, 2019, p. 6)
This disparity between those delivering instruction and those receiving instruction has not
gone unnoticed by policymakers.
Culturally Responsive Teaching in Pennsylvania
Many states are now recognizing the importance of providing teachers with
access to ongoing, high-quality, and job-embedded professional development that will
assist them in developing culturally responsive instructional experiences for students. In
Pennsylvania, the Commonwealth uses the term Culturally Relevant and Sustaining
Education (CR-SE) to refer to a set of agreed-upon standards and competencies that all
Pennsylvania educators are accountable for when working with students (Cole-Malott et
al., 2021). These competencies, built upon the 50-state survey of culturally responsive
standards completed by New America, ask educators to reflect upon their own cultural
lens, deepen their understanding of bias within the system as a whole, design educational
experiences that are reflective of students’ lived cultural experiences, and provide all
students with equitable and differentiated learning opportunities.
Each competency is accompanied by a set of “Questions to Stimulate
Reflexivity,” which educators can use to critically examine their current practices and
approaches in order to identify areas in which they can improve their practices. These
questions help educators to ensure that their instruction and instructional practices
“connect academics to their learners' daily lives, cultural backgrounds, challenges and
build upon the funds of knowledge of learners and their families” (Cole-Malott et al.,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
31
2021, p. 2). They are also designed to ask educators to critically examine the resources
that they use to assist in their instructional delivery so that students see positive and
accurate representations of themselves.
In evaluating instruction and instructional resources through a CR-SE lens,
Pennsylvania educators have a unique opportunity to help bridge the gap between the
intended curriculum, the implemented curriculum and the attained curriculum for their
students. In reviewing the standards of all 50 states, Muñiz (2019) observed that, while
most state standards generally include some form of Culturally Responsive Teaching
within their language, there are varying levels of depth and nuance to the language.
Figure 3 shows the ways in which the New America Culturally Responsive
Competencies are addressed within all 50 states:
Figure 3
What Culturally Responsive Competencies do State Teaching Standards Address?
Note: Culturally responsive competencies addressed in state teaching standards. (Muñiz, 2019)
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
32
The Impact of Reform on Culturally Responsive Teaching
Abdulrahim and Orosco (2020) note that historically, the focus on mathematics
instruction has not been to make a strong connection between students’ lived
experiences and math activities in order to promote math achievement. They feel that
this is due to the fact that the standardization of the curriculum has created an
instructional environment that is more reflective of the teacher and less reflective of
the individual learners. Abdulrahim and Orosco (2020, p. 2) state,
Despite reform agendas calling for the improvement of mathematics
education for all students, traditional approaches to instruction in favor of
basic skills instruction continue to dominate practice (Ellis and Berry, 2005).
The majority of mathematics education in the U.S. is based on standardized
curricula, pedagogy, and testing that is not normed on CLD (Culturally
Linguistic or Diverse) students (Bartell et al., 2017; Kress, 2005). This
one-size-fits-all approach separates students’ cultural strengths and
experiences from teaching and learning mathematics, which influences their
response to mathematics instruction (Leonard, 2008).
To assist in moving away from this standardized approach to instruction,
Brown and Crippen (2016) detail the utilization of the Growing Awareness Inventory
(GAIn) with preservice teachers. Designed as a structured observation protocol to
assist in the development of Culturally Responsive Teaching, the GAIn helps
educators to build their capacity to deliver Culturally Responsive Teaching by
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
33
engaging in continuous, reflective review of their instructional materials and
practices. Figure 4 shows the GAIn review process.
Figure 4
Growing Awareness Inventory Theory of Action
Note: The Theory of Action cycle of implementation from GAIn (Brown & Crippen, 2016, p. 136)
The GAIn is just one tool that can be utilized by educators to help them
implement Culturally Responsive Teaching within their implemented curriculum so that
their lessons and materials are reflective of the individuals that comprise the community
in which they teach. As each community is unique in its structure, demographics, and
culture (Smith & Ayers, 2006), so too should be a teacher’s approach to the implemented
curriculum for that community. In order for a teacher to be able to provide instruction
and utilize instructional materials that are reflective of their student population, they must
engage in reflective practices in order to understand the impact that their instruction and
instructional materials are having on student understanding and achievement.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
34
Culturally Responsive Teaching as a Pedagogical Practice
Howard and Rodriguez-Minkoff (2017) caution that Culturally Responsive
Teaching is not just a simple way of teaching or utilizing out-of-the-box units of
instructional materials. Rather, it is a pedagogical practice that is rooted in deeply
understanding the lived experiences of the students in a teacher’s classroom and actively
designing instruction and instructional materials that reflect those experiences. In order
to do so, Culturally Responsive Teaching must become a pedagogical practice that is
taught extensively to both pre-service and in-service teachers.
In order to ensure that all teachers are familiar with and actively include CR-SE as
a core component of their implemented curriculum, the Pennsylvania State Board of
Education has amended Chapter 49 of the Public School Code to include the mandatory
inclusion of CR-SE training as a requirement for pre-service teachers, as a component of
new teacher induction programs, and as a part of districts’ continuing education plans.
This guidance, released in April 2022, states,
The existing requirements of Chapter 49 further require that professional
education plans must address training in meeting the needs of diverse learners
(defined as students with limited English language proficiency or students with
disabilities), improving language and literacy acquisition, and closing the
achievement gap among students. (Molchanow, 2022, p. 117)
In order to do so, in-service educators are required to engage in continuing professional
education courses regarding CR-SE. The scope and depth of this training has not been
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
35
defined, nor have the strategies for resource evaluation to support Culturally Responsive
Teaching within the implemented curriculum inside individual classrooms.
Despite the growing awareness of the importance of culturally responsive
teaching, the scope and depth of training for educators in this area have not been clearly
defined. This lack of clarity can create challenges for teachers who want to implement
culturally responsive teaching practices in their classrooms. One possible solution to this
challenge is the use of open educational resources (OERs) that focus on culturally
responsive teaching. OERs can provide teachers with a variety of resources, such as
lesson plans, activities, and assessments, that have been designed and curated to support
culturally responsive teaching. OERs can also help teachers access a wider range of
perspectives and ideas, and allow for customization and adaptation to meet the needs of
their specific students and communities.
Implementing OER to Facilitate Culturally Responsive Teaching
In order to support students as they address Culturally Responsive Teaching
within their implemented curriculum, many teachers have begun to utilize Open
Educational Resources (OER) as their preferred instructional materials. The term, first
coined by UNESCO at their 2002 Forum on Open Courseware, is defined as, “learning,
teaching and research materials in any format and medium that reside in the public
domain or are under copyright that have been released under an open license, that permit
no-cost access, re-use, re-purpose, adaptation and redistribution by others” (UNESCO,
2002). Due to their ability to be adapted, teachers are able to easily personalize these
materials to better reflect the lived experiences of their students, making them more
relevant and engaging.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
36
Perceived Benefits of Using Open Educational Resources
When asked about the benefits of utilizing OER, teachers discussed many positive
aspects of their use (Tang, 2020). From feeling empowered as instructional experts who
had the capacity to deliver personalized instruction that was tailored to the specific needs
of individual student populations, to being able to have access to a larger repository of
materials with no impact to their classroom or district budgets, OER have become a
powerful asset to teachers looking to increase student achievement through Culturally
Responsive Teaching. Many see the implementation of OER as a transformative element
of their instruction.
Teachers now commonly utilize OER to create a culturally responsive classroom.
Garth and Sebastian (2020) highlight the increased flexibility and options that teachers
now have by using OER by stating:
With an ever-expanding library of digital content at their disposal - including
online publications, instruction materials, and institutional peer-reviewed assets instructors have countless pathways to embrace the entire community of learners
by designing culturally relevant classrooms that invite and incorporate every
student into the process. (Garth & Sebastian, 2020, para. 8)
Furthermore, there is now evidence that student performance has remained
unchanged or increased through the use of OER in the classroom (Tang et al., 2020).
Multiple studies from 2014 - 2019 show that the use of OER in K-12 classrooms has
allowed students of varying cultural backgrounds to maintain or increase their
achievement scores. Moreover, these studies also showed that students were more
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
37
motivated and engaged in their learning when they interacted with OER (Tang et al.,
2020).
Perceived Barriers to Increasing Student Achievement
While many teachers feel empowered to use OER as a component of their
implemented curriculum, most teachers report multiple barriers to the effective use of
OER in the classroom. These barriers, whether real or perceived, tend to outweigh the
reported benefits of OER on student achievement. Unfortunately, they prevent teachers
from engaging with instructional materials that may be able to be designed to be more
culturally reflective of individual students and increase student achievement (Tang,
2020).
The first identified barrier is related to adequate professional development
regarding the understanding of and intricacies surrounding OER use in the classroom.
Many teachers report that they only have a surface-level understanding of OER.
Teachers are uncomfortable in actively engaging in the work that is related to finding,
reviewing, and modifying OER to use in their classrooms (Tang, 2020).
The lack of professional development provided to teachers in the implementation
of OER also results in a lack of time to engage in a thorough review of the resources that
are found. As stated earlier in this review, teachers need consistent opportunities to
engage in professional development that helps them to understand the most effective and
efficient ways in which to review and use instructional materials in their classrooms in
order to increase the academic achievement of students. In order for OER use to become
a sustainable component of Culturally Responsive Teaching designed to increase student
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
38
achievement in the implemented curriculum, districts must provide adequate time for
professional development that is designed to assist teachers in learning the process of
efficiently finding OER that relate to their students’ needs, critically reviewing those
resources for rigor and quality, and strategically aligning those resources to the intended
curriculum (Kaufman et al., 2021).
Understanding the Impact of COVID-19 on Student Achievement
As conversations regarding student achievement, Standards-Based Education
Reform, and Culturally Responsive Teaching continue today, the discussion will naturally
turn towards the impact of COVID-19. The National Center for Education Statistics
(NCES) administers the National Assessment of Educational Progress (NAEP) every two
years. As stated on their website,
The National Assessment of Educational Progress (NAEP), first administered in
1969, is the largest continuing and nationally representative assessment of what
our nation’s students know and can do in subjects such as mathematics, reading,
science, and writing. Standard administration practices are implemented to
provide a common measure of student achievement. Teachers, principals, parents,
policymakers, and researchers all use NAEP results to assess progress and
develop ways to improve education in the United States. The results of NAEP are
released as The Nation’s Report Card, and are available for the nation, states, and
in some cases, urban districts. NAEP is a congressionally mandated project
administered by the National Center for Education Statistics (NCES), within the
U.S. Department of Education and the Institute of Education Sciences (IES).
(National Assessment of Educational Progress, 2022, para. 1)
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
39
Due to the COVID-19 pandemic, the NAEP was not administered in 2020.
Because of this, the results of the 2022 NAEP administration were the first to show the
impact of COVID-19 on student achievement in the United States. The results showed
the “largest score declines in NAEP mathematics at grades 4 and 8 since initial
assessments in 1990” (National Assessment of Educational Progress, 2022). In
comparing scores to 2019, student scores in grade 4 dropped an average of 5 points and
student scores in grade 8 dropped an average of 8 points. A sample of the headlines that
resulted from the report included:
● “Two Decades of Progress, Nearly Gone: National Math, Reading Scores Hit
Historic Lows” (Sparks & Tomko, 2022)
● “Nation’s report card: Massive drop in math scores, slide in reading linked to
COVID disruption” (Petrin, 2022)
● “NAEP ‘Nation’s Report Card’ Shows Steep Fall in Math Scores” (Mollenkamp,
2022)
● “Nation's Report Card shows how badly we failed kids during pandemic. Now
what do we do?” (Jacques, 2022)
The alarmist language that was so unique to A Nation At Risk appears to be used
as common language to discuss student achievement scores in the wake of COVID-19.
However, it is important to note that while the gap in student achievement has become
larger, due in part to school closures and interruptions to consistent approaches to
instructional delivery, there is compelling evidence that student achievement scores had
been persistent well before the onset of the pandemic.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
40
While the pandemic has certainly exacerbated existing inequities and highlighted
the deep-seated disparities in access to education and resources, the existence of the
achievement gap is not a new phenomenon. The gap in academic achievement between
students from different racial, ethnic, and socioeconomic backgrounds has persisted over
time, even when controlling for factors such as parental education and income. The
COVID-19 pandemic has certainly magnified the challenges faced by students and
schools, but it is not the root cause of the achievement gap. Rather, it is a symptom of the
systemic and structural inequities that have long plagued the US education system.
The US academic-achievement gap was first identified in 1966. Its persistence is
troubling. The possibility that COVID-19 could make it worse deserves focused
attention. The achievement gap costs the United States hundreds of billions of
dollars - and also exacts a long-term cost in social cohesion. This is a moment and a challenge - that calls for urgency and energy. (Dorn et al., 2020)
This quote by Dorn et al. (2020) highlights the troubling persistence of the
achievement gap in the US education system, and recognized the fact that the COVID-19
pandemic had the potential to exacerbate this long-standing issue. As previously
mentioned, the achievement gap has been identified as early as 1966 and has persisted
despite various efforts to address it. Dorn et al. (2020) take their research one step further
by also addressing the cost associated with addressing the achievement gap, suggesting
that it costs the US hundreds of billions of dollars and exacts a long-term cost in our
ability to develop the skills needed to strengthen our communities.
To address the achievement gap effectively, it is essential for us to examine
student achievement scores by different student groups. Such analysis can reveal
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
41
disparities in performance and highlight the need for targeted interventions to support
historically marginalized students. It is also crucial to recognize the intersectionality of
identities and experiences that contribute to the achievement gap. By analyzing
achievement data through an equity lens and developing evidence-based interventions,
educators and policymakers can work towards creating a more equitable education
system that supports all students in achieving their full potential.
NAEP Results by Student Group
In examining the 2022 results of the NAEP, most student groups had lower
average scores than they did in 2019. American Indian / Alaska Native students’ average
scores dropped by 4 points, Asian students’ average scores dropped by 7 points,
Asian/Pacific Islander students’ average scores dropped by 6 points, Black students’
average scores dropped by 7 points, Hispanic students’ average scores dropped by 7
points, Native Hawaiian / Other Pacific Islander students’ average scores dropped by 2
points, students of Two or More Races average scores dropped by 10 points, and White
students’ average scores dropped by 7 points (National Assessment of Educational
Progress, 2022).
Steep Declines in Algebra
In the Algebra content area, the subtopic with the steepest decline in scores is
Equations and Inequities, which dropped from 35% correct in 2019 to 26% correct in
2022. While NCTM recommended the prioritizing of developing a solid conceptual
foundation for skills before moving into the utilization of calculators or computers to
assist with the development of more involved problem-solving skills, (NCTM, 1989) as
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
42
was mentioned earlier, these results are indicative of the gap that exists between
education policy that is created in the wake of Standards-Based Education Reform and
the instructional practices that are adopted by teachers in the classroom.
Summary
Standards-Based Education Reform has become a deeply embedded reality of the
K-12 public education system in the United States for the past 40 years. As we look at
the history of Standards-Based Education Reform, there has been a clear response from
both the publishers of instructional content and the consumers of instructional content in
an effort to increase student achievement in order to meet reform expectations.
Instructional content publishers have gone to great lengths to market their
materials as being aligned to national and state content standards. Their documentation
that details this alignment has become a trusted source of information that many
education decision makers use to determine which materials to purchase for their
students. In the same vein, administrators and teachers have gone to great lengths to
understand new content standards and synthesize them into concepts and skills that can
be taught in classrooms. They spend countless hours working to determine the best ways
to present content to students from varied cultural backgrounds, and with varied lived
experiences so that they can all meet standardized proficiency benchmarks.
While this standardization makes monitoring student achievement easy to
quantify, it highlights gaps that exist between the intended curriculum that is set forth by
national or state standards and a district’s established curriculum, the implemented
curriculum that is taught within individual teachers’ classrooms, and the attained
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
43
curriculum that is evident through student achievement scores. In order to bridge this
gap between policy, practice, and performance, teachers look to pedagogical techniques
such as Culturally Responsive Teaching and more flexible instructional materials such as
Open Educational Resources to support the individual needs of their students.
The inclusion of Culturally Responsive Teaching pedagogy is a critical element in
all content areas, and not just those with a strong literature-based focus, such as English
Language Arts and Social Studies. Teachers are looking at the ways in which they can
develop culturally responsive instructional experiences for students in order to help them
increase their understanding and mastery of skills and concepts in content areas such as
Algebra I.
In Westmoreland County, educators and administrators are curious to understand
whether or not there is a correlation between instructional materials, Culturally
Responsive Teaching pedagogy, and increases in student achievement. Due to the fact
that Algebra I test scores have been historically low countywide, districts are looking at
the ways in which their intended curriculum and implemented curriculum are reflecting
their student populations in order to determine whether or not their Algebra I
instructional materials and strategies are culturally responsive to their student populations
and whether or not there is an impact on student achievement.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
44
Chapter III
Methodology
There is a need for school districts to separate their curriculum discussions into
defined categories to understand better how standards, instructional materials, and the
scope and sequence of instruction impact student achievement scores. In doing so,
districts can engage in a more comprehensive analysis of the impact of their intended
curriculum, implemented curriculum, and attained curriculum. When school districts
take the time to understand the connection between state-mandated academic standards,
the alignment and relevance of instructional materials, and the scope and sequence of the
instruction that the teacher provides, they can better adjust their efforts to ensure that
learning experiences are designed to be culturally responsive and personalized to the
individual student populations of each classroom.
The literature shows us that Standards-Based Education Reform was adopted in
the early 1980s to provide broad, top-down guidance to K-12 public schools in the United
States to increase academic rigor and develop a structured curriculum. While
Standards-Based Education Reform was designed to impact all content areas,
mathematics was a particular concern. Organizations such as the National Council of
Teachers of Mathematics provided detailed recommendations to overhaul K-12
mathematics education in the United States, including the adoption of a common, core
curriculum that would be utilized in all schools to ensure that all students would be
introduced to the same skills and concepts in mathematics.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
45
Another critical recommendation was to focus on equity in education and not just
on education reform. Researchers noted the importance of honoring the diverse
populations of K-12 public school districts by developing diverse learning experiences.
When educators create learning experiences that students feel reflect their unique cultures
and backgrounds, they can quickly connect to the content. This connection can result in a
deeper understanding and mastery of the concepts and skills presented within the
implemented curriculum and increase the student’s attained curriculum.
Purpose of Study
This study investigated the possible correlation between the historically low
Keystone Algebra I Scores and building principal and Algebra I teachers’ perceptions of
student identification with examples in the word problems found within their
instructional materials. To thoroughly investigate this topic, the researcher used a
mixed-methods embedded design in the form of a case study of three school districts that
represent Westmoreland County public schools. The following research questions guided
this case study:
1. What are the factors that impact student achievement in Algebra I?
2. What are the factors that impact instructional delivery in Algebra I?
3. What are the teacher and administrator perceptions of the impact of instructional
delivery on student achievement scores in Algebra I?
Qualitative data has been generated from district curriculum documents, standard
alignment maps, lesson plans, and instructional material reviews and policies to
determine how academic standards are addressed within the curriculum and addressed
through teacher instruction. Quantitative data has been generated from student
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
46
assessment scores, district budgets, and survey responses regarding instructional
materials to determine the amount spent on them, their alignment with Pennsylvania Core
Standard, and their cultural relevancy.
The desired outcome of this case study is that the data will show whether or not
the instructional resources currently being purchased and utilized within the classroom
effectively support student academic understanding and achievement. The results of this
case study will provide building administrators and Algebra I teachers with additional
guidance and support when determining which instructional materials may significantly
impact student achievement within their student populations.
Setting
The Pennsylvania General Assembly established intermediate units in 1970. As
the report “Intermediate Units in Pennsylvania: The Role of Educational Service
Agencies in Promoting Equity in Basic Education” states:
In 1970, the General Assembly passed Act 102, creating a system of 29
intermediate units (Us), which replaced the county superintendent offices as of
July 1, 1971. The lUs were mandated to develop a broad program of educational
services to be offered to public and nonpublic schools, including curriculum
development and instructional improvement services; educational planning
services; instructional materials services; continuing professional education
services; pupil personnel services; State and federal agency liaison services; and
management services. The lUs were created as instruments of federal, State, and
local education policies. They were empowered to create new services needed by
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
47
public and nonpublic schools. (Pennsylvania Joint State Government
Commission, 1997).
Each of the Commonwealth’s 29 intermediate units is now charged with
providing cost-effective and management-efficient programs designed to support school
districts, charter schools, and nonpublic schools and serves as the liaison between these
organizations and the Pennsylvania Department of Education (PDE). Intermediate units
have four significant areas of focus as service agencies. The first is Advancing
Professionals through high-quality training and professional development. Through this
area of focus, intermediate units work with over 52,000 educators each year in data
analysis, curriculum development, technology integration, school safety, and educator
certifications (Pennsylvania Association of Intermediate Units [PAIU], 2022). The
second area of Safe and Healthy Students is an area of focus that supports schools in
addressing individual students' behavioral, emotional, and mental health needs. These
supports include services such as partial hospitalization programs, alternative education
programs, student assistance programs, and student evaluations. These supports also
extend to implementing services designed to prevent school threats, such as creating
Crisis Response Teams (PAIU, 2022).
The third area of Serving Students is broken into two categories: Early Childhood
and Young Children and School Age Students. Through this service area, intermediate
units can provide educational opportunities to students from birth to 18 years of age.
These services include Early Intervention programs, online education programs, and
special education services such as speech therapy, occupational therapy, and behavior
intervention (PAIU, 2022). The fourth area of focus for intermediate units is Supporting
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
48
Schools. This service helps create purchasing consortiums so districts can take advantage
of lower pricing for health insurance, energy purchases, transportation, and administrative
services (PAIU, 2022). Each of these focus areas allows intermediate units to be
responsive to the needs of the member districts within their geographic region.
The Westmoreland Intermediate Unit is in Greensburg, Pennsylvania. Situated in
southwestern Pennsylvania, the WIU serves seventeen public schools, three career and
technology centers, one charter school, and forty-two nonpublic schools within
Westmoreland County. The WIU provides Curriculum Services, Executive Services,
Financial Services, Student Services, and Technology and Infrastructure Services to their
member schools. These services are delivered both in-person and online and are
designed to help districts to build their capacity to support their communities.
“The Curriculum Services Division's primary goal is to advance the practice of
teaching and learning to the schools and students we serve. Through innovative practice,
we strive to be the premier leading experts in curriculum services, operating as an
integral part of our local, regional, and state education community” (Westmoreland
Intermediate Unit, 2023). The Curriculum Services Division comprises a team of one
director, three supervisors, one support specialist, and one secretary. While this team is
small, its capacity to support districts continues to grow through its increased
involvement in local, state, and national educational initiatives.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
49
Participants
Three public school districts participated in this case study. The first is the
Greater Latrobe School District. Founded in 1960, the Greater Latrobe School District is
a midsized district in Westmoreland County. As stated on their district website,
The Greater Latrobe School District is in southwestern Pennsylvania at the
foothills of the Laurel Highlands. It is approximately 50 miles east of downtown
Pittsburgh and 30 miles southwest of Johnstown, Pennsylvania. U.S. Route 30
bisects much of the district, which encompasses the City of Latrobe, Youngstown
Borough, and Unity Township, with approximately 30,000 residents. The Greater
Latrobe area has diversified industries which afford job opportunities in specialty
steel, plastics, molds and casting ceramics, and brewing beverages. The local
hospital, four area colleges, the airport, and specialty stores and shopping areas
provide many forms of employment. (Greater Latrobe School District, 2022)
As of the 2021-2022 school year, Greater Latrobe School District reported a
district enrollment of 3,512 students in grades K-12. Of these students, 51.8% are male,
and 48.2% are female. The district states that 2.2% of their students are enrolled in their
Gifted program, 7% in Eastern Westmoreland Career and Technical Center, and 3% in
outside charter or cyber charter schools. Figure 5 shows the percentage of enrollments by
race/ethnicity for the 2021-2022 school year. Figure 6 shows a demographic breakdown
of the district’s student enrollments for the 2021-2022 school year. (Pennsylvania
Department of Education, 2022)
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 5
Greater Latrobe School District Percent Enrollment by Race/Ethnicity
Note: 2021 - 2022 Greater Latrobe School District student enrollments by race/ethnicity. Data retrieved
from Future Ready PA Index, December 2022
Figure 6
Greater Latrobe School District Percent Enrollment by Student Groups
Note: 2021 - 2022 Greater Latrobe School District student enrollments by student group. Data retrieved
from Future Ready PA Index, December 2022
50
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
51
The Greater Latrobe School District carefully reviews their Algebra I Keystone
Exam student achievement scores to work toward meeting Pennsylvania’s statewide
established proficiency goal. The Pennsylvania Department of Education has set the
statewide goal of having 71.8% of all students reach proficiency in Algebra I by 2030.
Districts work to increase their proficiency percentage annually incrementally to achieve
this goal. Figure 7 shows the proficiency percentages of all Greater Latrobe students
taking the Algebra I Keystone Exam from 2013 - 2022.
Figure 7
Greater Latrobe School District Algebra I Proficiency Percentages 2013 - 2022
Since the 2013-2014 school year, the Greater Latrobe School District’s highest
percentage area has been in the Basic category. There is a correlation between the
number of students in the Basic category and the overall number of students tested each
year. Figure 8 shows the total number of students tested annually from 2013 - 2022. The
district’s goal is to review its instructional materials and strategies to determine strategies
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
52
to increase student achievement and move students from the Basic category into the
Proficient category to achieve the 2030 statewide goal of 71.8% proficiency.
Figure 8
Greater Latrobe School District Total Number of Student Tested vs. Year Administered
The second district participating in this case study is the Monessen City School
District. The Monessen City School District is a small, urban district in Westmoreland
County. Serving the City of Monessen, the district spans approximately 2 square miles in
total. As stated on their district website, “Monessen, a very caring and unique school
district, relies on versatility in meeting the needs of our very special young people by
assisting them to become productive citizens and confident individuals in our own and
world community” (Monessen City School District, 2023).
As of the 2021-2022 school year, Monessen City School District reported a
district enrollment of 712 students in grades K-12. Of these students, 48.7% are male,
and 51.3% are female. The district states that 1.5% of their students are enrolled in their
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
53
Gifted program, 4% are enrolled in Mon Valley Career and Technical Center, and 7% are
enrolled in an outside charter or cyber charter schools. Figure 9 shows the percentage of
enrollments by race/ethnicity for the 2021-2022 school year. Figure 10 shows a
demographic breakdown of the district’s student enrollments for the 2021-2022 school
year (Pennsylvania Department of Education, 2022).
Figure 9
Monessen City School District Percent Enrollment by Race/Ethnicity
Note: 2021 - 2022 Monessen City School District student enrollments by race/ethnicity. Data retrieved from
Future Ready PA Index, December 2022
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
54
Figure 10
Monessen City School District Percent Enrollment by Student Groups
Note: 2021 - 2022 Monessen City School District student enrollments by student group. Data retrieved
from Future Ready PA Index, December 2022
The Monessen City School District also carefully reviews their Algebra I
Keystone Exam student achievement scores to work toward meeting Pennsylvania’s
statewide established proficiency goal. Figure 11 shows the proficiency percentages of
all Monessen City students taking the Algebra I Keystone Exam from 2013 - 2022.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
55
Figure 11
Monessen City School District Algebra I Proficiency Percentages 2013 - 2022
Historically, Monessen City School District has not had any students in the
Advanced category on the Algebra I Keystone Exam since 2013. Instead, most of their
students were in the Basic category until the Spring 2018 administration of the Keystone
Exam. At this point, we see the same percentage of students in the Basic and Below
Basic categories. From this point on, we see more students in the Below Basic category.
Figure 12 shows the total number of students tested yearly from 2013 - 2022. The district
is concerned about the increasing number of students entering the Below Basic category
and is investigating the reasons for this shift.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
56
Figure 12
Monessen City School District Total Number of Student Tested vs. Year Administered
The third district participating in this case study is the Yough School District. The
largest of the three districts participating, the Yough School District is a rural district that
spans 77 square miles and serves the Arona Borough, Madison Borough, Sewickley
Township, Smithton Borough, South Huntingdon Township, Sutersville Borough, and
West Newton Borough. As stated on their district website, “The Yough School District is
enthusiastic about the growing local industries, which is providing unlimited
opportunities for our students and families to gain meaningful careers. Employment,
coupled with local community and recreation organizations, allows individuals to be
actively involved, while preserving hometown pride” (Yough School District, 2020).
As of the 2021-2022 school year, Yough School District reported a district
enrollment of 1,805 students in grades K-12. Of these students, 50.1% are male, and
49.9% are female. The district reports that 2.4% of their students are enrolled in their
Gifted program, 6.9% are enrolled in Central Westmoreland Career and Technical Center,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
57
and 4.5% are enrolled in outside charter or cyber charter schools. Figure 13 shows the
percentage of enrollments by race/ethnicity for the 2021-2022 school year. Figure 14
shows a demographic breakdown of the district’s student enrollments for the 2021-2022
school year (Pennsylvania Department of Education, 2022).
Figure 13
Yough School District Percent Enrollment by Race/Ethnicity
Note: 2021 - 2022 Yough School District student enrollments by race/ethnicity. Data retrieved from Future
Ready PA Index, December 2022
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
58
Figure 14
Yough School District Percent Enrollment by Student Groups
Note: 2021 - 2022 Yough School District student enrollments by student group. Data retrieved from Future
Ready PA Index, December 2022
The Yough School District also carefully reviews their Algebra I Keystone Exam
student achievement scores to work toward meeting Pennsylvania’s statewide established
proficiency goal. Figure 15 shows the proficiency percentages of all Yough students
taking the Algebra I Keystone Exam from 2013 - 2022.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
59
Figure 15
Yough School District Algebra I Proficiency Percentages 2013 - 2022
Like the Greater Latrobe School District, Yough School District’s highest
percentage area has historically been in the Basic category since 2013. However, unlike
the Greater Latrobe School District, there is no direct correlation between the number of
students taking the Algebra I Keystone Exam and the percentage of students in the Basic
category. Figure 16 shows the total number of students tested yearly from 2013 - 2022.
The district is concerned with the percentage of students in the Basic category. They
have not been able to raise their Algebra I proficiency level beyond 33.9%, which
occurred in 2015. In recent years, their highest proficiency level was 28.3% in Spring
2021.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
60
Figure 16
Yough School District Total Number of Student Tested vs. Year Administered
Student demographic data, achievement score data, attendance data, and
disciplinary data from each of the three participating school districts were collected to
investigate this topic thoroughly. In addition, teacher and building administrator
perception data were contributed by teachers currently instructing Algebra I classes
within the three participating school districts and their building-level principals. All
participants volunteered to participate in this study, completing and submitting a
participant informed consent form.
Research Plan
As Standards-Based Educational Reform has become more prevalent in K-12
education in the United States, public school districts have relied on content publishers to
align their instructional materials to educational standards to help them increase student
achievement scores, investing large amounts of time and money into effectively using
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
61
these materials in classrooms. Districts have prioritized purchasing these materials in
Mathematics, which has been one of the most globally competitive content areas since
the 1980s. They hope these materials will help teachers better align their implemented
curriculum with the intended curriculum of the district. Unfortunately, the literature
shows us that, in many cases, content publishers are investing more effort into updating
their marketing materials to show that their instructional materials are aligned to content
standards instead of updating and aligning the instructional materials themselves.
As districts struggle to increase student achievement scores in mathematics, they
are now looking to supplement the instructional materials they purchase with open
educational resources (OER) that are more reflective of their student population. They
are also taking the time to evaluate the instructional materials they buy from content
developers to determine whether they are culturally reflective of their student population
and whether there is a correlation between culturally reflective instructional materials and
student achievement scores.
To explore this issue in Westmoreland County, the established research plan
included independent actions by the researcher and voluntary actions by participants from
districts within Westmoreland County. The researcher began by investigating the
Algebra I test scores in all seventeen public school districts in Westmoreland County
since 2013. This data was collected from the eMetric Data Interaction for Pennsylvania
Student Assessments system and was generated as a Group Summary report showing the
performance results of all students that took the Algebra I Keystone Exam during each
administration period from 2013 through 2022. This resulted in a total of 322 unique
reports showing the test results of 64,959 students.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
62
The researcher also analyzed current demographic data for all seventeen public
school districts in Westmoreland County as reported to PDE for the 2021 - 2022 school
year. This data was collected from the Future Ready PA Index, a website maintained by
PDE annually. As stated on PDE’s website, “The Future Ready PA Index illustrates
student and school success via three color-coded categories: academic performance,
student progress, and college and career readiness, providing the public with a more
comprehensive look at how Pennsylvania's schools are educating students” (Pennsylvania
Department of Education, 2023).
To gather perception data regarding the instructional materials being used in
Algebra I classrooms in Westmoreland County, building principals and Algebra I teachers
in Westmoreland County were given the opportunity to participate in a perception survey.
The survey for building principals (Appendix A) was 14 questions in length and covered
topics such as the amount spent on instructional materials annually, administrators’
awareness of OER, and their perception of the following:
● the alignment of purchased materials to PA Core Standards
● their teachers’ awareness of OER
● internal and external barriers toward the use of OER in the classroom
● the frequency in which teachers need to modify instructional materials to meet the
needs of students
● the frequency in which teachers need to develop their own instructional materials
to meet the needs of students
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
63
The survey for Algebra I teachers (Appendix B) was 20 questions in length and covered
topics such as teachers’ academic background, instructional environments, and their
perception of the multiple items relating to their instructional materials, including:
● the alignment of their 7th and 8th grade mathematics curriculum with PA Core
Standards
● the alignment of instructional materials being used with PA Core Standards
● the amount of professional development given to effectively use the instructional
materials being purchased by the district
● the amount of professional development given to effectively develop their own
instructional materials
● the impact of instructional materials on student achievement scores
The final step in the established research plan involved working with three
selected districts to complete the Culturally Responsive STEAM Scorecard (Appendix
C), developed by NYU Steinhardt, in order to determine the cultural responsiveness of
the primary instructional materials for Algebra I for each school district. During this
step, the researcher held a virtual meeting with each district's building principal and
Algebra I teachers to explain the purpose of the Culturally Responsive STEAM
Scorecard and how to use it. The groups were then given time to review their primary
instructional materials, evaluating them through the scorecard parameters and
determining individual scores for each of the four categories: Representation, Social
Justice, Teacher’s Materials, and Materials & Resources. Each district group was
allowed to complete the scorecard in a group setting or individually. The researcher then
created an average score for each category based on the individual scores provided.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
64
Based on the average score, the researcher was able to recommend to each district the
cultural responsiveness of the primary instructional materials their district purchased for
Algebra I.
Once all steps were completed, the researcher looked at the historic Algebra I test
scores for each of the three participating school districts, their 2021 - 2022 demographic
data, and their Culturally Responsive STEAM Scorecard results for their primary
instructional materials for Algebra I. This was done to determine if there is a correlation
between low test scores and the use of instructional materials that do not reflect the
district's student population. This research plan is designed to help districts determine
their instructional materials' efficacy and impact on student achievement in Algebra I.
This research plan meets the needs of the research problem because the research
will investigate the possible correlation between the historically low Keystone Algebra I
scores in Westmoreland County and building principal and Algebra I teachers’
perceptions of student identification with examples in the word problems found within
their instructional materials in Algebra I classrooms in Westmoreland County.
There are minimal costs associated with this Doctoral Capstone Research Project.
The main cost associated with it is the time and effort of the researcher. This work aligns
with her current work for the Westmoreland Intermediate Unit. The budget below shows
an allotment of 30% of the researcher’s annual salary. Should the work result in the
recommendation for districts to adjust their instructional materials, there may be a shift in
district resources allotted to both instructional materials and professional development.
As a result, one year of funding for the implementation of this study will be
supported through the general operating budget of the Westmoreland Intermediate Unit
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
65
under the function of Instruction for Regular Programs. The funding source will allocate
dollars from the Administrative Budget, as all administrators within the Westmoreland
Intermediate Unit are not in this category and are not assigned to specific instructional
levels. The Westmoreland Intermediate Unit is classified as a non-instructional building.
Therefore all expenditures within this budget are not split by individual schools, as you
see within a school district. All project costs fall under Mathematics Instruction for the
Curriculum Specialist Assignment job description.
To effectively analyze each district’s instructional materials for Algebra I, the
researcher must dedicate time to thoroughly understanding the resources being used,
educator perspectives, and student achievement data. This budget allots 30% of the
researcher’s time to this specific project, as the researcher has additional duties and
responsibilities that must be completed throughout the year. Budget line items include
the Supervisor’s salary and benefits such as health care, insurance, retirement,
unemployment, and worker’s compensation.
The indirect costs associated with this project, external of the researcher’s time,
include the use of a laptop and iPad, access to a Westmoreland Intermediate Unit-issued
Google Account, and the cost of secure digital storage on the Westmoreland Intermediate
Unit’s Amazon Web Services (AWS) server. As the researcher also utilizes all of these
indirect cost items to complete the other assigned duties and responsibilities, the amount
listed reflects 30% of the annual usage of each of these items.
The immediate cost associated with the project on the part of the individual
districts is minimal as well, resulting in approximately 2 ½ hours in salary and benefits
per participant. There may be a future cost to each district. Should the project result in
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
66
providing districts with a recommendation for adjusting their instructional materials, the
researcher will collaborate with each district to determine the budgetary impact of
implementing the recommendation. This will help districts to project the sustainability of
the recommendations made by the researcher.
Research Methods & Data Collection
The researcher studied this topic by using a mixed-methods embedded design in
the form of a case study of three school districts that are representative of all seventeen
Westmoreland County public school districts. This mixed-methods research allowed the
researcher to utilize one form of data to support the other. As stated by Mertler when
discussing mixed-methods research, “Its focus must always be on integration - that is, on
the idea of integrating not only multiple types of data but also perhaps integrating
theoretical perspectives, research design, research results, etc. Integration is the central
concept in this methodological approach to conducting research.” (pg. 140) Utilizing a
mixed-method research approach allowed the researcher to efficiently collect qualitative
and quantitative data as she progressed through her research. Once all data was collected,
she could begin looking for potential correlations between instructional materials, student
achievement scores, perceptions of those resources, and reasons why teachers utilize
specific resources within their classrooms.
The researcher utilized qualitative data from district curriculum documents,
standard alignment maps, lesson plans, and instructional material reviews and policies to
determine the ways in which academic standards are addressed within the curriculum and
addressed through teacher instruction. The researcher also used this qualitative data to
support the quantitative data collected from student assessment scores, district budgets,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
67
and survey responses regarding instructional materials to determine the amount being
spent on instructional materials, their alignment to Pennsylvania Core Standard and their
cultural relevancy and to show whether or not the resources that are currently being
purchased and utilized within the classroom are effective in supporting student academic
understanding and achievement.
This design enhanced the study by giving the researcher the opportunity to more
thoroughly explain the research and data that was collected and interpreted. It also
helped the researcher to better manage her time to ensure that everything was complete
within the defined timeline as established within the research plan. Data that was
available via public, openly available records, such as budget documents, student
achievement scores, and district policies were able to be accessed at any time once
released online. Building administrator and teacher perception survey data was collected
via Google Form. The data from the Culturally Responsive STEAM Scorecard was
collected via Google Sheet. This ensured that each district team and the researcher had
instant access to the results of their team’s scorecard.
Data Collection Timeline and Plan Described
The first research question, What are the factors that impact student achievement
in Algebra I?, was investigated from November 2022 through February 2023. In order to
thoroughly investigate this question, the researcher analyzed Algebra I Keystone Exam
scores, available local assessment scores, Future Ready PA Index Information, student
attendance and discipline records, and the results of the Culturally Responsive STEAM
Scorecard. Each of these items provided historical data for the three school districts
being studied within this case study.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
68
Keystone Exam scores and local assessment scores were obtained from the
Pennsylvania Value-Added Assessment System (PVAAS) website and the eMetric
website. All Future Ready PA Index information was obtained from the Future Ready PA
Index website. Attendance and discipline data was collected from the individual school
districts. Teachers and administrators completed the Culturally Responsive STEAM
Scorecard digitally.
The second research question, What are the teacher and administrator
perceptions of the impact of instructional delivery on student achievement scores in
Algebra I?, was investigated from November 2022 through January 2023. Due to the
nature of this study, the researcher was able to investigate this question at the same time
as the first research question. In order to thoroughly investigate this question, the
researcher analyzed the results of the building administrator and teacher perception
surveys.
These two surveys helped the researcher to understand the perceptions of both
teachers and administrators regarding the use of instructional materials in Algebra I
classrooms, their cultural responsiveness, and their potential impact on student scores.
Surveys were shared electronically via email, and results were automatically delivered to
the researcher once participants completed the survey.
The final research, question What are the factors that impact instructional
delivery in Algebra I?, investigated from March 2023 through April 2023. In order to
thoroughly investigate this research question, the researcher analyzed budget documents,
available professional development plans, instructional material adoption policies,
available instructional material standards alignment documentation, available district
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
69
curriculum map, and available scopes and sequences of instruction. Each of these items
helped the researcher to understand the correlation between the intended curriculum, the
implemented curriculum, and the attained curriculum. All documents were obtained
from district websites, or were requested from the participating districts if they had been
developed.
The researcher submitted proposal #PW22-007 to the Institutional Review Board
(IRB) at the Pennsylvania Western University before initiating the outlined research plan.
The IRB granted approval effective 9/20/22 with an expiration date of 9/19/23 (Appendix
D).
The data collection tools utilized for this study were available online at no cost,
therefore the fiscal implications of conducting the study were minimal. Participants
completed perception surveys at times that were most convenient to their daily schedules,
and worked on completing the Culturally Responsive STEAM Scorecard during
established collaborative planning time during their workday, ensuring that districts
would not accrue additional costs for participating in the case study.
Validity
Hendricks (2017) describes the validity of action research as the “trustworthiness”
of the research. Relying heavily on the criteria established by Lincoln and Guba (1985),
Hendricks (2017) reminds researchers that they must consider both the credibility and the
validity of their work. The four types of validity established by Lincoln and Guba (1985)
are
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
70
● Credibility: the plausibility of the research findings for the context that was
studied.
● Transferability: the extent to which results of a study are applicable to other
contexts and other individuals
● Dependability: the degree to which research results would replicate with the same
or similar participants and/or contexts.
● Confirmability: Showing that results are an accurate representation of what
occurred rather than the results of the researcher’s bias, motivation, or interest.
The researcher designed the research plan to address the credibility of the study
by utilizing two specific strategies recommended by Hendricks (2017): triangulation and
accurate data recording. In order to thoroughly investigate this topic, the researcher
analyzed data from multiple sources over a span of 13 years. In order to ensure that data
was recorded accurately, the data files were downloaded and utilized to ensure an
accurate transfer of information.
Transferability was addressed by providing a detailed description of the setting
and the steps of the study. By using a mixed-methods embedded design, the researcher
had the ability to provide a rich description of the setting, participants, interventions, and
research methods employed in the study. This will allow future researchers to apply this
work to other contexts.
In order to address dependability, and determine the degree to which this research
would replicate with the same or similar participants and/or contexts, the researcher
relied upon triangulation and providing a detailed description of the setting and steps of
the study. The researcher also engaged in an ongoing investigation with different
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
71
participants by selecting three representative school districts from across Westmoreland
County to participate in the case study.
Confirmability was also addressed within the research plan through triangulation
and accurate data recording. The amount of data collected necessitated the accurate
recording of all information gathered. In addition, providing a thick description of the
setting and study allowed the researcher to show that the results of the study are an
accurate representation of what occurred during the course of the study rather than the
results of the researcher’s own bias, motivations, or interests (Hendricks 2017).
Summary
The researcher designed a plan to investigate the possible correlation between the
historically low Algebra I test scores in Westmoreland County and the culturally
responsive nature of the instructional materials currently being used in Algebra I
classrooms. In order to investigate this possible correlation, the researcher analyzed data
from multiple sources, including student achievement scores over a period of eight years,
district demographic data as reported to PDE, district budget documents, instructional
material standards alignment documentation, district curriculum maps, and scopes and
sequences of instruction. In addition, perception data was collected and analyzed from
budiling administrators and teachers. Finally, participants engaged in the completion of a
Culturally Responsive STEAM Scorecard in order to determine the responsiveness of
their Algebra I instructional materials.
Chapter III provided an overview of the research problem identified within this
case study. It then provided a detailed explanation of the purpose, setting, participants,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
72
and research plan for this study. Finally, Chapter III discussed the methods of data
collection and validity.
Chapter IV will provide a thorough report of the data analysis that was conducted.
It will also share the results of this study. Finally, Chapter IV will provide a detailed
discussion of the interpretation of those results.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
73
Chapter IV
Data Analysis and Results
This case study aimed to determine the possible correlation between the
historically low Keystone Algebra I Scores in middle school and building principal and
Algebra I teachers’ perceptions of student identification with examples in the word
problems found within their middle school instructional materials. The research plan for
this case study was designed to determine if a correlation existed by examining the
relationship between district curriculum documents, standard alignment maps, lesson
plans, and instructional material reviews and policies to determine how academic
standards are addressed within the curriculum and addressed through teacher instruction
and comparing those documents to student assessment scores, district budgets, and
survey responses regarding instructional materials to determine the amount spent on
them, their alignment with Pennsylvania Core Standard, and their cultural relevancy.
This chapter presents an analysis of the data collected to determine whether or not
a correlation between student test scores and the perceived cultural responsiveness of
instructional materials in middle school Algebra I courses exists through the framework
of the identified research questions. This chapter also provides a detailed discussion of
the interpretations of the data concerning the research questions for each school district
identified within the case study.
Research Questions
1. What are the factors that impact student achievement in Algebra I?
2. What are the factors that impact instructional delivery in Algebra I?
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
74
3. What are the teacher and administrator perceptions of the impact of instructional
delivery on student achievement scores in Algebra I?
Data Analysis
Participants
Eighteen schools are designated as middle schools, junior high schools, or
intermediate middle schools within the 17 public school districts in Westmoreland
County. The building administrators and Algebra I teachers within those 18 buildings
were invited to participate in the perception surveys for this case study. Eleven building
administrators agreed to participate in the Building Principal Perception Survey and
signed the established consent form before completing the survey. All of the building
administration participants remained in the study, resulting in the completion of the
survey by all eleven participants. Six Algebra I teachers agreed to participate in the
Teacher Perception Survey and signed the established consent form before completing
the survey. All of the teacher participants remained in the study, resulting in the
completion of the survey by all six participants.
In addition, building administrators and Algebra I teachers from three districts
agreed to complete the Culturally Responsive STEAM Scorecard to review their primary
middle school Algebra I instructional materials. To access the Culturally Responsive
STEAM Scorecard, building administrators and teachers first agreed to participate in the
study and sign the established consent form. While all participants agreed, the teachers
from one of the three districts did not return the signed consent forms and could not
participate in the study. Due to this, the study only reflects the Culturally Responsive
STEAM Scorecard responses for two of the three case study districts.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
75
The Building Principal Perception Survey and the Teacher Perception Surveys
were administered via Google Forms. Survey responses did not collect identifying
information, such as participants’ names, to ensure confidentiality. Their email addresses
were also not collected upon submission. Once all responses were collected, each
participant was assigned a letter of the alphabet and a corresponding number to associate
with the reported data.
The researcher speculates that the lack of participation by teachers is due to the
current political climate in education and the connection of this case study to the phrase
“culturally responsive teaching.” The researcher further speculates that external factors
impacting teachers in one of the case study districts, such as the unexpected death of a
student, staffing shortages, and changes in district administration, may have caused
teachers to struggle to find the time to complete the Culturally Responsive STEAM
Scorecard.
Data Analysis Procedure
The research aimed to investigate the correlation between historically low
Keystone Algebra I Scores in Westmoreland County middle schools, principals' and
teachers' perceptions of instructional materials alignment and efficacy, training and
support provided to teachers, and the cultural responsiveness of the instructional
materials being used in middle school Algebra I classrooms. Quantitative data included
exam scores, assessment data, budget information, and standards alignment from three
school districts. Data sources included eMetric, PVAAS, PDE, the Future Ready PA
Index, district and content vendor websites. Qualitative data included surveys and the
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
76
Culturally Responsive STEAM Scorecard. The data were analyzed to determine if a
correlational relationship existed.
The quantitative data that was gathered for this case study was compiled by the
researcher from multiple sources. This data includes overall student Algebra I Keystone
Exam scores for all 17 public school districts in Westmoreland County from 2012 - 2022,
countywide Algebra I Anchor Analysis based on student achievement scores from 2015 2021, local assessment data for Algebra I from seven of the 17 public school districts in
Westmoreland County from 2019 - 2022, Future Ready PA Index data for all public
school districts in Westmoreland County from 2001 - 2022, annual budget information
for all 17 public school districts in Westmoreland County from 2016 - 2023,
Comprehensive Plans for all 17 public school districts in Westmoreland County from
2016 - 2026, and Pennsylvania Common Core Standards alignment information for the
primary Algebra I instructional resources used by Greater Latrobe School District,
Monessen City School District, and Yough School District. This data was gathered from
the eMetric Data Interaction for Pennsylvania Student Assessments web-based system
(eMetric), the web-based Pennsylvania Value Added Assessment System (PVAAS), the
Pennsylvania Department of Education’s website (PDE), PDE’s web-based Future Ready
PA Index, individual district websites, and individual content vendor websites.
The quantitative data gathered for this case study was then organized by the
researcher for analysis according to the research plan outlined in Chapter III. This
allowed the researcher to discover patterns and trends in student achievement scores over
time, determine funds allocated to instructional materials, and determine whether the
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
77
materials used by Greater Latrobe School District, Monessen City School District, and
Yough School District are aligned with Pennsylvania Common Core Standards.
The eleven building administrators voluntarily submitted the first qualitative data
gathered for this case study through a Google Form. The six teachers voluntarily
submitted the second set of qualitative data for this case study through a Google Form.
This format helped the researcher to compile the data to facilitate analysis according to
the research plan outlined in Chapter III. Participant responses were used to support the
researcher's efforts to determine a potential correlation between instructional materials,
Culturally Responsive Teaching pedagogy, and increases in student achievement.
First, the researcher gathered the total number of students tested for each testing
window from the 2012 - 2013 school year through the 2021 - 2022 school year. Then the
researcher determined the percentage of students with scores in the Below Basic, Basic,
Average, and Proficient ranges. From there, the scores were separated into categories by
Algebra I Assessment Anchors separated the scores into categories to determine student
achievement within each Anchor. The change in these percentages by district is shown
below. Figure 17 shows the percentage of each district’s scores in each performance
category in the winter of 2012. Figure 18 shows the percentage of each district’s scores
in each performance category in the spring of 2022.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 17
Group Summary: Winter 2012 Performance Keystone, Algebra I
78
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
79
Figure 18
Group Summary: Spring 2022 Performance Keystone, Algebra I
The researcher next gathered demographic data from all 17 public school districts
in Westmoreland County for the 2021 - 2022 school year, as reported to PDE. This
demographic data allowed the researcher to separate each district’s total student
enrollments by race/ethnicity, as shown in Table 1 The researcher was also able to
separate each district’s enrollments by the following subgroups:
● Economically Disadvantaged, as shown in Figure 19
● English Language Learners (ELL), as shown in Figure 20
● Special Education, as shown in Figure 21
● Gender (male/female), as shown in Figure 22
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
80
Table 1
All Districts’ Total Student Enrollments by Race/Ethnicity
District
American
Indian /
Asian
Alaska
Native
Black
Native
Hawaiian
Two or
/ other Hispanic White More
Pacific
Races
Islander
Belle Vernon Area SD
0%
1.3%
3.2%
0%
2.9%
87.3%
5.2%
Burrell SD
0%
0.5%
1.9%
0.1%
1.5%
91.7%
4.4%
Derry Area SD
0.1%
0.5%
1.7%
0%
0.6%
93.6%
3.5%
Franklin Regional SD
0.1%
7.8%
1.8%
0.1%
2.5%
83.7%
4.1%
Greater Latrobe SD
0.2%
0.9%
1%
0%
1.8%
9%
3.6%
0%
0.3%
6.3%
0.2%
3.2%
80.3%
9.7%
Hempfield Area SD
0.1%
1.1%
1.2%
0.1%
1.8%
91.5%
4.3%
Jeannette City SD
0.7%
0%
14.2%
0%
3.1%
62.4%
19.6%
Kiski Area SD
0.1%
0.2%
6.3%
0%
1.5%
90.7%
1.1%
Ligonier Valley SD
0.1%
0.1%
0.6%
0.1%
1.4%
95.4%
2.3%
Monessen City SD
0%
0.7%
37.5%
0%
3.5%
37.2%
21.1%
Mount Pleasant Area SD
0%
0.1%
0.7%
0.1%
0.6%
95%
3.5%
New Kensington-Arnold SD
0.3%
0.4%
32.9%
0%
3.6%
43.7%
19.2%
Norwin SD
0.1%
1.5%
1%
0%
2%
91.4%
4%
0%
1.1%
1.8%
0.2%
1.1%
92.7%
3.1%
0.1%
0.4%
0.7%
0%
0.9%
94.3%
3.7%
0%
0.1%
1.4%
0%
0.9%
93.3%
4.3%
Greensburg Salem SD
Penn-Trafford SD
Southmoreland SD
Yough SD
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 19
% Enrollments Economically Disadvantaged by District
81
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 20
% Enrollments ELL by District
82
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Figure 21
% Enrollments Special Ed. by District
83
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
84
Figure 22
% Enrollments by Gender (M/F) by District
Next, the first group of participants submitted their qualitative data by accessing
the Building Principal Perception Survey via Google Forms. This allowed the researcher
to quickly compile and organize the data collected by the participants to determine their
perception of the instructional materials used in the Algebra I classrooms in their
buildings. Data was collected from this survey by utilizing Open-Ended Questions,
Intervals Scales, and Ordinal Scales. The questions from the Building Principal
Perception Survey are shown in Table 2 and in Appendix A.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
85
Table 2
Building Principal Perception Survey Questions
Building Principal Perception Survey Questions
To what degree are you aware of Open Educational Resources (OER)?
To what degree are teachers aware of Open Educational Resources (OER)?
How many administrators in your building feel adequately informed about OER?
How many teachers in your building feel adequately informed about OER?
What are the internal factors that cause teachers to not use OER?
What are the external factors that cause teachers to not use OER?
How regularly do teachers modify instructional resources to better meet their students’
needs?
How regularly do teachers develop their own original instructional resources to better
meet their students’ needs?
What percentage of purchased instructional materials are accurately aligned to
Pennsylvania Core Academic Standards?
How often do teachers evaluate the efficacy of their purchased instructional materials?
How often do administrators evaluate the efficacy of their purchased instructional
materials?
How much money is spent annually on purchased instructional materials?
How much money is spent annually on teacher professional development designed to
help teachers utilize purchased instructional materials in their classrooms?
How much money is spent annually on teacher professional development designed to
help teachers curate, evaluate, and implement OER in their classrooms?
The same process was used for the second group of participants as they submitted
their qualitative data by accessing the Teacher Perception Survey via Google Forms.
Again, this allowed the researcher to quickly compile and organize the data the
participants collected to determine their perception of the instructional materials being
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
86
used within their Algebra I classrooms. Data was collected from this survey by utilizing
Short Answer Questions and Likert Scale questions. The questions from the Teacher
Perception Survey are shown in Table 3 and Appendix B.
Table 3
Teacher Perception Survey Questions
Teacher Perception Survey Questions
Which best describes your education level?
In which District do you teach Algebra I?
In which building do you teach Algebra I?
Generally speaking, do you believe that the 7th or 8th grade mathematics curriculum
within your district is well-aligned to the PA Core Standards for Mathematics?
Generally speaking, do you believe that the 7th or 8th grade mathematics instructional
materials used within your district are missing critical standards addressed on the
Algebra I Keystone Exam?
I believe that the physical and digital instructional materials purchased and used with
our students were well researched and help us to prepare students for success on the
Algebra I Keystone Exam.
I have been given plenty of professional development focused on how to effectively
use the instructional materials provided to me with my students.
I have been given plenty of professional development focused on how to effectively
develop my own instructional materials with my students.
I believe that the instructional materials used in my classroom have a direct impact on
my students’ achievement scores.
In terms of the physical instructional materials used with students, how would you
describe the way in which the skills addressed prepare students for the Algebra I
Keystone Exam?
In terms of the digital instructional materials used with students, how would you
describe the way in which the skills addressed prepare students for the Algebra I
Keystone Exam?
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
87
I believe that the instructional materials used accurately reflect my student population.
I believe that my students can see themselves and their life experiences in their
instructional materials in my class.
I believe that when students are able to see themselves in their instructional materials
they will do better on assessments.
I have been adequately trained in understanding the culture and values of my student
population.
I have been adequately trained to evaluate instructional materials for cultural relevance.
I have been adequately trained to modify existing resources or develop new resources
that are culturally relevant to my student population.
I am aware of Open Educational Resources (OER) and use them within my learning
environment.
My colleagues are aware of Open Educational Resources (OER) and use them in their
learning environments.
My building administrators are aware of Open Educational Resources (OER) and
encourage their use.
The final data collection step was to have the middle school Algebra I teachers of
the three representative school districts in Westmoreland County complete the Culturally
Responsive STEAM Scorecard to evaluate their primary Algebra I instructional
materials. The Culturally Responsive STEAM Scorecard collects data utilizing a Likert
Scale. It evaluates teachers’ perceptions of the cultural responsiveness of their
instructional materials in four categories: Representation, Social Justice, Teachers’
Materials, and Materials & Resources. During this step, the researcher met with the
building principal and Algebra I teachers of Greater Latrobe School District, Monessen
City School District, and Yough School District to explain the components of the
Culturally Responsive STEAM Scorecard, how to use the Culturally Responsive STEAM
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
88
Scorecard to evaluate their primary Algebra I instructional materials, and to establish the
timeline for completion and submission of the scorecard. Once each district’s group of
teachers compiled the data, the researcher created an average score for each statement in
all four categories and a final score for each. The statements from the Culturally
Responsive STEAM Scorecard are shown in Table 4 and Appendix C.
Table 4
Culturally Responsive STEAM Scorecard
Representation Statements
The curriculum centers the origins of science, technology, engineering, arts and/or
math within BIPOC communities (Black, Indigenous, people of color).
The curriculum elevates mathematicians, artists, and/or scientists with historically
marginalized identities (i.e. non-binary or trans people, women, people of color, people
with disabilities, working class people, multilingual people) and their discoveries.
The curriculum acknowledges and/or incorporates the expertise of diverse
communities, their cultures, and their historical and/or contemporary experiences.
The curriculum has photos/pictures, names, scenarios, and text that reflect the
experiences and interests of students of color in your community.
The curriculum elevates not just “scientists, artists, and mathematicians”, but the
everyday users of math, science, arts, technology, and engineering.
The curriculum affirms the multiple forms of communication or language systems
during mathematical and scientific argumentation rooted in historically marginalized
cultures.
Social Justice Statements
Curriculum highlights and affirms the knowledge systems of Indigenous,
Black/African, Brown, and non-Western conceptions of science, technology,
engineering, arts, and math (such as interdependence, sustainability, and continual
change).
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
89
The curriculum and instructional activities promote or provoke critical questions about
science, technology, engineering, arts, and/or math and the societal status quo.
The curriculum and instructional activities present alternatives or allow students to
present alternatives about the role science, technology, engineering, arts, and/or math
plays in existing social structures.
The curriculum presents multiple understandings of a scientific, mathematical, or
artistic concept or theory, especially highlighting points of view from marginalized
people/communities.
The curriculum provides avenues for students to see STEAM as a way to understand
and improve their world, take actions that combat inequity or promote equity, and
connect learning to social, political, and/or environmental concerns.
The curriculum presents social situations and problems not as individual problems but
as embedded within a societal and/or systemic context.
The curriculum encourages students to critically examine dominant knowledge systems
as scientists, mathematicians, and artists.
Teachers’ Materials Statements
The authors of the teachers’ materials are people of diverse identities (race/ethnicity,
gender, other identities).
Guidance is provided to teachers on being aware of one’s biases, assumptions, and the
gaps between one’s own culture and students’ cultures (implicit bias).
The teachers’ materials ask teachers to reflect on their own practices and experiences
learning STEAM subjects and critique them through the lens of cultural
responsiveness.
Guidance is provided on combating the legacy of STEAM education related trauma
amongst historically marginalized communities and on designing healing and joyful
STEAM experiences.
Guidance is provided on using students’ everyday lives as the starting point for
learning, and making real-life connections between academic content and the local
neighborhoods, culture, environment, community issues, and current events.
Guidance is provided on creating opportunities to meaningfully engage students’
families to enhance lessons.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
90
Guidance is provided on appropriately using Indigenous and non-Western resources to
understand math and science, including oral histories, legends, and community
knowledge.
Guidance is provided on engaging students in culturally responsive experiential
learning activities.
Guidance includes, for applicable lessons, a range of possible student responses that
could all be valid, given the range of student experiences and perspectives.
Guidance is provided on how to design lessons or engage in conversations that use
STEAM to disrupt power inequities or create opportunities for students to practice
disruption.
Guidance is provided on planning opportunities for students to create and teach lessons
to their peers.
Guidance is provided to help teachers foster environments of collaboration that go
beyond group work by meshing individual's assets, resources, and strengths (such as
various ways of knowing, doing and being) to advance group and individual learning.
Materials & Resources Statements
The curriculum provides opportunities for students to use accessible technology and
manipulatives to explore STEAM concepts in ways that reflect a variety of ways of
“doing.”
Issues of social justice, equity, and Black, Indigenous, and people of color’s
experiences and contributions are reflected in homework/classroom assignments and
assessments.
The curriculum incorporates the arts as a tool to center experiences within STEM and
innovation.
Curriculum rigor is not dependent on access to resources, materials and technology that
students and schools may not have. In other words, the curriculum materials are fully
accessible; all resources, materials and technology options are rigorous and interesting.
(Ex. If students can engage curriculum materials with a computer or paper, the paper
materials should be just as rigorous, interesting, and engaging as using the computer).
The resources, materials and technology referenced in lessons include resources from
BIPOC communities (Black, Indigenous, people of color).
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
91
As stated in Chapter III, the researcher looked at the historic Algebra I test scores
for each of the three participating school districts, their 2021 - 2022 demographic data
and their Culturally Responsive STEAM Scorecard results to effectively answer the three
research questions established for this case study.
Results
Research Question 1
The first research question in this case study aims to answer “What are the factors
that impact student achievement in Algebra I?” After student assessment data was
compiled, scores were disaggregated into achievement categories and by Algebra I
Assessment Anchors. District demographic data were also disaggregated into the
subgroups that PDE uses for reporting. These subgroups are consistent with existing
Civil Rights Data Collection standards established by the United States Department of
Education.
To thoroughly investigate this question, student assessment scores for Greater
Latrobe School District, Monessen City School District, and Yough School District were
further disaggregated by combining two or more standard subgroups that PDE uses for
reporting to create unique subgroups. Those distinctive subgroups include:
● Ethnicity and Gender
● Ethnicity and Economically Disadvantaged
● Ethnicity and English Language Learners
● Ethnicity and IEP
The unique subgroups were then analyzed in comparison with the results of the
Culturally Responsive STEAM Scorecard for Greater Latrobe School District and Yough
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
92
School District. To determine a correlation between student achievement scores and the
results of the Culturally Responsive STEAM Scorecard, the researcher focused on
Algebra I Keystone performance levels for middle school students, disaggregated by
ethnicity, for Spring 2022. Table 5 shows those results below for Greater Latrobe School
District, Monessen City School District, and Yough School District.
Table 5
Group Summary, Performance Levels by Grade 08 and Ethnicity
Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022
Best Score
Below
Basic
Basic Proficient
Advanced
Number
Administration Tested
Mean
SS
Greater Latrobe SD
Spring 2022
383
1494.3
12.5
45.7
25.1
16.7
Grade 08
Spring 2022
111
1545.4
0
11.7
40.5
47.7
Monessen City SD
Spring 2022
49
1422.7
75.5
20.4
4.1
0
Grade 08
Spring 2022
11
1449.5
54.5
27.3
18.2
0
Yough SD
Spring 2022
183
1476.6
17.5
53
24.6
4.9
Grade 08
Spring 2022
41
1503
2.4
39
43.9
14.6
Group
%
%
%
%
Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022, Grade 08 and Black
Best Score
Group
Number
Administration Tested
Mean
SS
Below
Basic
%
Basic Proficient
%
%
Advanced
%
Greater Latrobe SD
Grade 08
Spring 2022
0
0
0
0
0
0
Monessen City SD
Grade 08
Spring 2022
3
1425.3
66.7
33.3
0
0
Yough SD Grade 08
Spring 2022
0
0
0
0
0
0
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
93
Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022, Grade 08 and Hispanic/Latino
Best Score
Group
Number
Administration Tested
Mean
SS
Below
Basic
%
Basic Proficient
%
Advanced
%
%
Greater Latrobe SD
Grade 08
Spring 2022
1
1591
0
0
0
100
Monessen City SD
Grade 08
Spring 2022
3
1435
100
0
0
0
Yough SD Grade 08
Spring 2022
0
0
0
0
0
0
Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022, Grade 08 and White
Best Score
Below
Basic
Basic Proficient
Advanced
Number
Administration Tested
Mean
SS
Greater Latrobe SD
Grade 08
Spring 2022
108
1544.4
0
12
41.7
46.3
Monessen City SD
Grade 08
Spring 2022
6
1469.2
33.3
33.3
33.3
0
Yough SD Grade 08
Spring 2022
39
1502.1
2.6
38.5
46.2
12.8
Group
%
%
%
%
The researcher compared this information to the Culturally Responsive STEAM
Scorecard results for Greater Latrobe School District. The primary Algebra I
instructional material utilized by the middle school Algebra I teachers is the 8th Grade
Algebra edition of Go Math by Houghton Mifflin Harcourt. Overall, teachers in the
Greater Latrobe School District felt satisfied that the curriculum of their primary Algebra
I instructional materials adequately represented diverse communities, including Black,
Indigenous, and People of Color (BIPOC), those with historically marginalized identities.
Furthermore, they are satisfied that the curriculum of their primary Algebra I
instructional materials incorporates the expertise of these diverse and historically
marginalized communities and elevates those communities throughout their
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
94
student-facing materials. Due to this, the total score for the Representation category was
2.
In reviewing the Social Justice category, teachers in the Greater Latrobe School
District were not satisfied with how their primary Algebra I instructional materials
addressed concepts such as interdependence and continual change. They were also not
satisfied that the materials’ instructional activities adequately presented alternatives or
allowed students to present alternatives about the role science, technology, engineering,
arts, and/or math plays in existing social structures. Due to this, the total score for the
Social Justice category was -5.
The only category where teacher opinions varied greatly was the Teachers’
Materials category. Some teachers were very satisfied that the teachers’ materials for
their primary Algebra I instructional materials adequately provided guidance to teachers
on being aware of one’s biases, assumptions, and the gaps between one’s own culture and
students’ cultures that the materials ask teachers to reflect on their own practices and
experiences learning STEAM subjects and critique them through the lens of cultural
responsiveness and that the materials provide guidance on using students’ everyday lives
as the starting point for learning and making real-life connections between academic
content and the local neighborhoods, culture, environment, community issues, and
current events. Others felt that the teachers’ materials were very lacking in these areas.
Due to this, the total score for the Teachers’ Materials category was -1.5.
Finally, the Greater Latrobe School District teachers had the most confidence in
the materials and resources associated with their primary Algebra I instructional
materials. Overall, they felt satisfied that the materials and resources provide
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
95
opportunities for students to use accessible technology and manipulatives to explore
STEAM concepts in ways that reflect a variety of ways of “doing,” that issues of social
justice, equity, and Black, Indigenous, and people of color’s experiences and
contributions are reflected in homework/classroom assignments and assessments, and that
the rigor of the curriculum is not dependent upon the access to resources, materials and
technology that students and schools may not have. Due to this, the total score for the
Materials & Resources category was 2.5, which was the highest score given by teachers
in the Greater Latrobe School District. Figure 23 shows the scorecard results for the
Greater Latrobe School District.
Figure 23
GLSD Culturally Responsive STEAM Scorecard Results
The researcher then compared the Algebra I Keystone performance levels for
middle school students, disaggregated by ethnicity, for Spring 2022 to the results of the
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
96
Culturally Responsive STEAM Scorecard for the Yough School District. The primary
Algebra I instructional material utilized by the middle school Algebra I teachers is a
district-specific, OER-based Algebra I Flexbook that was customized on CK-12’s digital
platform. Overall, teachers in the Yough School District felt satisfied that the curriculum
of their primary Algebra I instructional materials adequately represented diverse
communities, including Black, Indigenous, and People of Color (BIPOC), those with
historically marginalized identities. Furthermore, they are satisfied that the curriculum of
their primary Algebra I instructional materials incorporates the expertise of these diverse
and historically marginalized communities and elevates those communities throughout
their student-facing materials. Due to this, the total score for the Representation category
was 3.5, which was the highest score given by teachers in the Yough School District.
In reviewing the Social Justice category, teachers in the Yough School District
were less satisfied with how their primary Algebra I instructional materials addressed
concepts such as interdependence and continual change. It was unclear whether the
materials’ instructional activities adequately presented alternatives or allowed students to
present alternatives about the role science, technology, engineering, arts, and/or math
plays in existing social structures. Due to this, the total score for the Social Justice
category was -1.
As we also saw in the Greater Latrobe School District, the only category where
teacher opinions varied greatly in the Yough School District was the Teachers’ Materials
category. Some teachers were very satisfied that the teachers’ materials provided
guidance on using students’ everyday lives as the starting point for learning, and making
real-life connections between academic content and the local neighborhoods, culture,
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
97
environment, community issues, and current events, and to help teachers foster
environments of collaboration that go beyond group work by meshing individual's assets,
resources, and strengths to advance group and individual learning. However, other
teachers felt that the teachers’ materials were unclear in their ability to adequately
provide guidance to teachers on being aware of one’s biases, assumptions, and the gaps
between one’s own culture and students’ cultures that the materials ask teachers to reflect
on their own practices and experiences learning STEAM subjects and critique them
through the lens of cultural responsiveness, and in how to design lessons or engage in
conversations that use STEAM to disrupt power inequities or create opportunities for
students to practice disruption. Due to this, the total score for the Teachers’ Materials
category was 1.
Finally, the teachers in the Yough School District felt satisfied that the materials
and resources for their primary Algebra I instructional materials provide opportunities for
students to use accessible technology and manipulatives to explore STEAM concepts in
ways that reflect a variety of ways of “doing,” and that the rigor of the curriculum is not
dependent upon the access to resources, materials and technology that students and
schools may not have. They felt unclear that issues of social justice, equity, and Black,
Indigenous, and people of color’s experiences and contributions are reflected in
homework/classroom assignments and assessments. Due to this, the total score for the
Materials & Resources category was 2. Figure 24 shows the results of the scorecard for
the Yough School District.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
98
Figure 24
YSD Culturally Responsive STEAM Scorecard Results
The researcher had planned to review the Algebra I Keystone performance levels
for middle school students, disaggregated by ethnicity, for Spring 2022 to the results of
the Culturally Responsive STEAM Scorecard for the Monessen City School District.
Unfortunately, the teachers in that school district did not complete the scorecard. The
primary Algebra I instructional material utilized by the middle school Algebra I teachers
is the enVision - Algebra I online program by Savvas, formerly Pearson.
Next, the researcher completed a standards-analysis review for each of the
primary Algebra I instructional materials utilized by the middle school Algebra I teachers
in the Greater Latrobe School District, the Monessen City School District, and the Yough
School District. Since each resource was developed to be aligned to Common Core State
Standards, the researcher had to complete a Standards Crosswalk to align the materials to
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
99
Pennsylvania Core Standards and then determine which Standards, Assessment Anchors
and Eligible Content were present within each. Table 6 shows the number of Eligible
Content items present in each primary instructional material used by the Greater Latrobe
School District, Monessen City School District, and Yough School District.
Table 6
Number of Algebra I Eligible Content Items Present by Vendor
Algebra I Assessment Anchor
A1.1.1 - Operations With Real
Numbers and Expressions
# of Eligible Content # of Eligible # of Eligible # of Eligible
Items within Anchor in
Content
Content
Content
PA Core Standards For
Items
Items
Items
Mathematics
Addressed Addressed Addressed
Curriculum Framework
in HMH
in CK-12
in Savvas
Algebra 1
Resource
Resource
Resource
9
7
7
5
A1.1.2 - Linear Equations
19
7
13
14
A1.1.3 - Linear Inequalities
10
0
10
2
A1.2.1 - Functions
24
9
12
11
A1.2.2 - Coordinate Geometry
13
4
8
4
5
2
0
3
A1.2.3 - Data Analysis
When comparing Algebra I Keystone performance levels for middle school
students, disaggregated by ethnicity, for Spring 2022 to the results of the Standards
Crosswalk for all three primary Algebra I instructional materials, the findings indicate
that there may be an impact on student learning and achievement since the Algebra I
Assessment Anchors are not adequately addressed in each of the materials being used.
Go Math by covers 36.25% of the available Eligible Content items for Algebra I
within their curriculum. CK-12’s Algebra I flexbook covers 62.5% of the available
Eligible Content items for Algebra I within their curriculum in its original form before
being modified by users. Savvas’ enVision Algebra I program covers 48.75% of the
available Eligible Content items for Algebra I within their
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
100
curriculum. If teachers are not aware of the items that are not addressed within the
curriculum of the primary Algebra I instructional materials that their district has
approved, students may not be exposed to this content in the classroom, resulting in a
lack of understanding of concepts and mastery of skills needed to achieve higher
performance levels on the Algebra I Keystone Exam.
Finally, data were collected from the Teacher Perception Survey and compared
with historical student assessment scores in Algebra I and the results of the Culturally
Responsive STEAM Scorecard. Data was also collected from the Building Principal
Perception Survey and compared with the results of the Teacher Perception Survey,
historical student assessment scores in Algebra I, and the results of the Culturally
Responsive STEAM Scorecard.
In reviewing the results from the Teacher Perception Survey, teachers in
Westmoreland County overwhelmingly feel that their middle school mathematics
curriculum is well-aligned to Pennsylvania Core Standards for Mathematics, with 66.7%
of respondents reporting that their curriculum is mainly aligned to the standards.
However, their views began to vary as they began to evaluate the alignment of their
primary instructional resources to Pennsylvania Core Standards. Figure 25 shows the
differences in their perceptions of the alignment of the primary instructional materials
used in middle school Algebra I classes to Pennsylvania Core Standards for Mathematics.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
101
Figure 25
Teacher Perceptions of Standards Alignment of Instructional Materials
Of the teachers participating in the survey, 66.7% felt that their materials were not
missing critical standards addressed on the Algebra I Keystone Exam. In reviewing the
Building Principal Perception Survey results, most building principals participating in the
survey feel that at least 75% of their purchased instructional materials accurately align
with Pennsylvania Core Academic Standards. Only 27.3% of participants felt that 50%
of their purchased instructional materials were accurately aligned with Pennsylvania Core
Academic Standards. Figure 26 shows building principals' perceptions regarding the
accurate alignment to Pennsylvania Core Academic Standards for their purchased
instructional materials.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
102
Figure 26
Building Principals’ Perceptions of Instructional Materials Alignment to Standards
However, the results of the Standards Crosswalk completed by the researcher, as shown
above in Figure 30, show that resources being used within Westmoreland County are
missing substantial amounts of Eligible Content items directly connected to the
Pennsylvania Core Standards. The perception surveys' findings indicate that teachers and
building principals feel that the primary instructional materials used in middle school
Algebra I classrooms are more accurately aligned to Pennsylvania Core Standards and are
addressing Eligible Content items in Algebra I than they really are.
Research Question 2
The second research question in this case study aims to answer “What are the
factors that impact instructional delivery in Algebra I?” Data from both the Teacher
Perception Survey and the Building Principal Perception Survey were compiled and
analyzed to research this question effectively. Many of the questions presented to
teachers and building principals were related to their perceptions about the time and
funds spent on professional development for utilizing instructional materials.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
103
The researcher first focused on the results of the Teacher Perception Survey.
When asked if teachers had been given plenty of professional development focused on
using the instructional materials their district had purchased effectively, the answers
varied greatly. Some teachers strongly agreed they had been given plenty of time, while
others disagreed. Figure 27 shows the differences in teachers' perceptions regarding the
amount of time they feel that they have to use the instructional materials they have been
given effectively.
Figure 27
Teacher Perceptions of PD to Effectively Use Instructional Materials
The researcher next compared these results to data from the Building Principal
Perceptions Survey. When asked how much money is spent annually on teacher
professional development experiences designed to help teachers utilize their purchased
instructional materials in their classrooms, 81.8% of building principals who participated
in the survey said they spend between $0.00 and $25,000.00. Figure 28 shows the
breakdown of funds spent annually by districts on teacher professional development to
utilize instructional materials in their classroom.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
104
Figure 28
Money Spent Annually By Districts on Teacher PD to Utilize Instructional Materials
In comparing this data with the results of the Culturally Responsive STEAM
Scorecard, as shown in Figures 23 and 24 above, the findings indicate that teachers feel
they need more confidence in their ability to evaluate the instructional resources that their
districts have purchased. Teachers also need more professional development experiences
designed to help them utilize their primary instructional materials more effectively and
develop culturally responsive learning experiences to meet students' needs better. Each
of these factors directly impacts the instructional delivery of Algebra I instruction
occurring in middle school classrooms in Westmoreland County.
When comparing Algebra I Keystone performance levels for middle school
students, disaggregated by ethnicity, for Spring 2022 to the results of the Culturally
Responsive STEAM Scorecard, the findings indicate that the guidance provided to
teachers on how to use their instructional materials from the vendor that developed those
materials may have an impact on student learning and achievement. When teachers feel
that this guidance needs to be clarified, they may struggle to create instructional
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
105
experiences for students that reflect the lived experiences of the students in their
classroom and actively design instruction and instructional materials that reflect those
experiences.
The researcher speculates that this may be addressed by implementing the
Culturally Relevant and Sustaining Education Guidelines and Framework that the
Commonwealth of Pennsylvania has adopted. As of this writing, Pennsylvania school
districts have been directed to include Culturally Relevant and Sustaining Education as a
part of their continuing education plan for their educators starting with the 2023-2024
school year. Based on the competencies within the framework, this professional
development will assist teachers in developing instructional experiences for students that
reflect their lived experiences in their classroom and actively design instruction and
instructional materials that reflect those experiences.
Research Question 3
The third research question in this case study aims to answer “What are the
teacher and administrator perceptions of the impact of instructional delivery on student
achievement scores in Algebra I?” To investigate this question, the researcher analyzed
data from the Teacher and Building Principal Perception Surveys relating to instruction,
instructional materials, and student achievement scores. Overall, teachers believe that
their instructional materials directly impact student achievement scores, as shown in
Figure 29. These perceptions appear to correlate to the Standards Crosswalk results and
the historical student achievement scores in Algebra I.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
106
Figure 29
Teacher Perceptions on the Impact of Instructional Materials on Student Achievement
In examining the data further to determine the impact on instructional delivery,
teachers feel split in their opinions that the physical instructional materials used with
students adequately address the skills needed to help prepare students for the Algebra I
Keystone Exam. They feel less confident in the ability of their digital instructional
materials to address the skills required to help prepare students for the Algebra I
Keystone Exam. Figures 30 and 31 show the results of these questions from the Teacher
Perception Survey.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
107
Figure 30
Teacher Perceptions on Physical Materials Preparing Students for Keystone Exams
Figure 31
Teacher Perceptions on Digital Materials Preparing Students for Keystone Exams
Teachers overwhelmingly believe that students do better on assessments when
they are able to see themselves in their instructional materials, as seen in Figure 32
below. However, teachers acknowledge that their primary instructional materials are not
always culturally reflective of their entire student population. They must supplement their
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
108
instruction with Open Educational Resources or other materials to develop more
culturally relevant learning experiences.
Figure 32
Teachers' Perceptions of Student Reflections on Instructional Materials
Building principals also acknowledge that teachers need to modify resources to meet
students' cultural and academic needs. If teachers are unable to modify the instructional
materials that are available to them, they will develop their own original materials
instead. Figures 33 and 34 provide a breakdown of the perceptions that building
principals have regarding the frequency in which teachers modify resources to meet the
needs of students or develop their own resources to meet the needs of students.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
109
Figure 33
Building Principals’ Perceptions of Teacher Modifications to Resources
Figure 34
Building Principals’ Perceptions of Teacher Development of Original Resources
Although building principals acknowledge that teachers are engaging in the
modification of existing resources or the creation of original resources to meet the
cultural and academic needs of their students as a way to supplement the instruction that
is provided within their primary instructional materials for Algebra I, they also
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
110
acknowledge that there are internal and external factors that cause teachers to shy away
from taking these action. Building principals feel that the internal factors that cause
teachers not to use OER in their Algebra I classrooms include:
● “A perception that OER are not aligned to PA Core Standards”
● “Lack of knowledge and how they can be incorporated into what they do and a
perceived lack of time during the day”
● “Ignorance”
● “Lack of research and commitment to utilizing various resources that may be
different from what they are comfortable already using”
They also feel that the external factors that cause teachers not to use OER in their
Algebra I classrooms include:
● “Not explicitly approved as per the curriculum / textbook approval process.”
● “Potential pressure from community members”
● “Focus on school curriculum”
● “School board”
These findings suggest that the internal and external factors preventing the
broader adoption of Open Educational Resources in Algebra I classrooms in
Westmoreland County are rooted in a lack of understanding about OER and a fear of
judgment from others for utilizing OER.
Discussion
The research aimed to examine the possible correlation between the historically
low Keystone Algebra I Scores in middle schools in Westmoreland County, building
principals and Algebra I teachers’ perceptions of the standards-alignment and efficacy of
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
111
their primary instructional materials, the amount of training and support provided to
teachers to effectively utilize those materials in their Algebra I classrooms, and the
culturally responsive nature of their materials. As stated earlier in Chapter IV, the
quantitative data gathered by the researcher included overall student Algebra I Keystone
Exam scores, completed a countywide Algebra I Anchor Analysis, local assessment data
for Algebra I, Future Ready PA Index data, annual budget information relating to
instructional materials expenses, district Comprehensive Plans, and Pennsylvania
Common Core Standards alignment information for the primary Algebra I instructional
resources used by Greater Latrobe School District, Monessen City School District, and
Yough School District. This data was gathered from the eMetric Data Interaction for
Pennsylvania Student Assessments web-based system (eMetric), the web-based
Pennsylvania Value Added Assessment System (PVAAS), the Pennsylvania Department
of Education’s website (PDE), PDE’s web-based Future Ready PA Index, individual
district websites, and individual content vendor websites. The qualitative data gathered
by the researcher included Building Principal and Teacher Perception Surveys and the
completion of the Culturally Responsive STEAM Scorecard. The data were analyzed to
address all research questions to determine if a correlational relationship existed.
To analyze the first research question, “What are the factors that impact student
achievement in Algebra I?” student assessment data was compiled and then
disaggregated into achievement categories and by Algebra I Assessment Anchors.
District demographic data were also disaggregated into the subgroups that PDE uses for
reporting. Student achievement data was then compared to the results of the Culturally
Responsive STEAM Scorecard and the Standards Crosswalk for the instructional
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
112
materials. Analyses revealed that several factors may be impacting student achievement
in Algebra I.
The first factor revealed in the data analysis for this research question was that
middle school Algebra I teachers do not feel that their primary instructional materials for
Algebra I provide them with guidance and support in developing culturally responsive
learning experiences that meet the needs of their student populations. Disaggregating
district demographic data by ethnicity and comparing it to the results of the Culturally
Responsive STEAM Scorecard shows that culturally responsive instruction may impact
student achievement. Teachers believe this to be true but need more guidance and
support in creating those learning experiences for their students.
The second factor revealed in the data analysis for this research question was that
the primary instructional materials used by the three identified case study school districts
do not adequately address the content covered on the Algebra I Keystone Exam.
Moreover, teachers must be made aware that this content must be addressed in their
instructional materials. While teaching to the test is not recommended, reviewing
instructional materials to ensure that they adequately support instruction to cover
concepts and skills that will be assessed on the Algebra I Keystone Exam is a highly
recommended action by PDE. To thoroughly complete this work, teachers and
administrators cannot simply accept the supporting alignment documentation that content
vendors provide. Instead, they must conduct their own independent evaluation to
determine if all Assessment Anchors and Eligible Content items are present and can
support the instructional experiences that their teachers develop. If absent, teachers and
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
113
administrators will need a plan to supplement their materials and instruction to prepare
students for the Algebra I Keystone Exam.
To analyze the second research question, “What are the factors that impact
instructional delivery in Algebra I?” data from both the Teacher Perception Survey and
the Building Principal Perception Survey that related to their perceptions about time and
funds spent on professional development for utilizing instructional materials were
compiled and analyzed. This research question aimed to explore how teachers grow in
their knowledge of their student populations, the use of their instructional materials, and
how to design effective learning experiences that help to increase student achievement.
Unfortunately, there was not much teacher participation in the Teacher Perception
Survey. The researcher speculates that the current political climate and the connection to
the term “culturally responsive” may have caused teachers to hesitate to participate in the
survey. However, findings based on the received responses show that teachers feel they
are given enough time to learn how to effectively utilize their instructional resources
when districts spend money on professional development specifically designed to help
teachers use those materials in their classrooms.
The findings based on the Culturally Responsive STEAM Scorecard show that
teachers still feel uncertain about developing culturally responsive learning experiences
or identifying how resources may not be meeting the needs of their students from a
culturally responsive standpoint. While the low participation numbers on the Teacher
Perception Survey make it challenging to determine if a lack of culturally responsive
instruction impacts student achievement scores, teachers in Westmoreland County believe
there is a correlation between culturally responsive pedagogy and increased student
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
114
achievement. As districts begin to evaluate their instructional materials through a more
culturally responsive lens with the guidance of the Culturally Relevant and Sustaining
Education Guidelines and Framework developed by PDE, they will need to design
professional development experiences for teachers that guide them through the process of
how to balance the effective use of instructional materials to present concepts and skills,
and better understand the specific needs of their particular student populations from a
culturally responsive point of view and modify or supplement instruction with OER to
address those needs.
To analyze the third and final research question, “What are the teacher and
administrator perceptions of the impact of instructional delivery on student achievement
scores in Algebra I?” data was gathered from the Teacher and Building Principal
Perception Surveys relating to instruction, instructional materials, and student
achievement scores. This research question aimed to analyze whether or not teachers and
administrators believe that instructional experiences and resources directly impact student
achievement.
The findings indicate that while teachers and building principals recognize that
instruction and instructional materials will impact student achievement, they must learn
to analyze their resources from a standards-alignment and culturally responsive
perspective. They may also need more time to engage in observational activities that
allow them to see examples of standards-alignment activities and culturally responsive
instruction to learn how to develop and implement these skills.
The findings also indicate a significant disconnect in understanding Open
Educational Resources while teachers regularly engage in activities commonly associated
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
115
with their use. Survey results show that teachers regularly modify instructional materials
to meet the needs of their students, with respondents indicating that many teachers do this
at least weekly, if not daily, to ensure that students can successfully engage with their
instructional materials. The act of revising a resource to adapt or modify it to meet the
needs of students better is one of the main ways in which educators engage with Open
Educational Resources. Teachers across Westmoreland County engage in this critical
practice to support student academic success.
In summary, the most significant finding is a direct correlation between student
achievement scores and the amount of Assessment Anchors and Eligible Content items
present within the primary instructional materials that middle school Algebra I teachers
use. When instructional materials have a higher level of Assessment Anchors and
Eligible Content items present, students have a better chance of having higher
achievement scores on the Algebra I Keystone Exam. This shows that teachers strongly
align their instruction to the content in their primary instructional materials. Factors that
prevent them from deviating from their primary instructional materials include pressure
only to use their purchased materials, a lack of knowledge about the Assessment Anchors
and Eligible Content items that are present within their primary instructional materials,
and a lack of understanding of how to utilize Open Educational Resources effectively.
Summary
Chapter IV of the study introduced a mixed-methods embedded research design
aimed at investigating a potential correlation between the historically low Keystone
Algebra I Scores of middle school students in Westmoreland County and the perceptions
of building principals and Algebra I teachers regarding student identification with
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
116
examples in word problems found within their middle school instructional materials. The
chapter provided a comprehensive analysis of both quantitative and qualitative data to
address the three research questions formulated for this particular case study.
To start, the chapter offered a detailed explanation of the research methodology
used by the researcher, which combined quantitative and qualitative approaches to gain a
thorough understanding of the research problem. The mixed-methods design allowed the
researchers to gather numerical data through the collection of assessment scores while
also gathering subjective insights through perception surveys. This comprehensive
approach aimed to provide a more robust analysis and interpretation of the research
findings.
The chapter then delved into the data analysis process, describing the techniques
used to analyze both the quantitative and qualitative data. The quantitative analysis
sought to identify any correlations or patterns between the Keystone Algebra I Scores and
analyses of the middle school instructional materials. The qualitative data analysis
involved a thorough examination of surveys and the perceptions of principals and
teachers. This qualitative analysis aimed to provide a deeper understanding of the
perceptions and experiences of principals and teachers regarding student identification
within the word problems presented to students within their materials.
The chapter then presented the results obtained from both the quantitative and
qualitative approaches. The quantitative results provided evidence of any correlations
between low Keystone Algebra I Scores and the alignment of instructional materials to
PA Core Standards and Eligible Content Standards. In addition, the qualitative results
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
117
offered rich descriptions and interpretations of the identified themes and patterns within
the perceptions of principals and teachers.
Finally, Chapter IV engaged in a comprehensive discussion of the interpretation
of the results. It analyzed the implications of the findings, considering both the
quantitative and qualitative perspectives. The discussion aimed to identify the possible
reasons behind the findings, providing insights into the underlying factors influencing
student identification in the context of word problems.
Chapter IV serves as a bridge between data analysis and the upcoming
conclusions and recommendations of the research study, which will be presented in the
next chapter. Chapter V, which will focus on drawing conclusions and formulating
recommendations based on the interpretation of the results from the previous chapter. It
will also synthesize the findings, discuss their significance, and propose actionable
recommendations that can contribute to addressing the challenges identified in the study.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
118
Chapter V
Conclusion and Recommendations
Administrators and teachers in public school districts in Westmoreland County
have struggled to understand their students’ historically low middle school mathematics
test scores on PSSA and Keystone Exams. The Keystone Exam test scores of middle
school students have been particularly concerning since the inception of the Keystone
Algebra I Exam in 2010. To understand the root cause of these scores, building
administrators and Algebra I teachers have engaged in intensive data analysis of student
test scores over the years. This analysis has driven the adoption of instructional
materials, the refinement of scopes and sequence of instruction, the evaluation of and
alignment of curriculum maps to Pennsylvania Core State Standards, and the
implementation of new instructional initiatives designed to increase student achievement
scores.
The mixed-methods embedded design of the research set out to answer three
research questions, which sought to further determine the root cause of students’
historically low test scores by understanding the impact of the instructional materials
used in Algebra I classrooms on student achievement. They also sought to understand
building administrator and Algebra I teacher perceptions of instructional materials and
professional development time devoted to implementing instructional materials with
fidelity, as well as understanding the perceived cultural responsiveness of their primary
instructional materials for Algebra I in middle school classrooms.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
119
Eleven building principals and six middle school Algebra I teachers employed in
K-12 public schools in Westmoreland County, Pennsylvania, participated in the study.
The researcher collected data related to student Algebra I achievement scores, assessment
data, budget data, and standards alignment data to discover patterns and trends in student
achievement scores over time. The participants completed perception surveys and the
Culturally Responsive STEAM Scorecard to provide a potential correlation between
instructional materials, Culturally Responsive Teaching pedagogy, and increases in
student achievement.
Conclusion
The mixed-method embedded research design sought to determine if a correlation
exists between the historically low Keystone Algebra I Scores of middle school students
in Westmoreland County and the perceptions of building principals and Algebra I
teachers regarding student identification with examples in word problems found within
their middle school instructional materials. The researcher investigated this issue to
provide both building administrators and Algebra I teachers with guidance to help
support their efforts to understand the gaps between the intended curriculum better, as
identified by state standards and district curriculum maps, the implemented curriculum,
as observed through the alignment of resources to standards and culturally responsive
pedagogy, and the attained curriculum, as shown by student achievement scores. The
following sections discuss specific conclusions related to each of the established research
questions, explain how the findings support those conclusions, and describe how those
findings may be impacting current instructional practices in middle school Algebra I
classrooms.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
120
Research Question 1: “What are the factors that impact student achievement in
Algebra I?”
The first research question analyzed the factors that impact student achievement
in Algebra I by reviewing disaggregated, historic student achievement data. The
researcher then completed a standards-analysis review for each of the primary Algebra I
instructional materials utilized by the middle school Algebra I teachers in three case
study districts representing all of Westmoreland County: Greater Latrobe School District,
the Monessen City School District, and the Yough School District. Finally, the researcher
compared disaggregated test scores and the standards analysis with the results of the
Culturally Responsive STEAM Scorecard by participants from the case study school
districts. The study design aimed to determine if the instructional materials used in
Algebra I classrooms impacted student proficiency scores on the Algebra I Keystone
Exam.
By disaggregating student scores on the Algebra I Keystone Exam by race, the
researcher gained insight into the impact of the wording of the exam questions on student
subgroups. There was no significant difference in Algebra I Keystone Exam scores
between subgroups, regardless of gender, race, socioeconomic status, special education
status, or English language learner status. These factors alone do not have a significant
impact on student proficiency. This suggests that the wording used in the Algebra I
Keystone Exam questions is mainly free from bias.
Perception data showed that both building principals and Algebra I teachers have
high confidence levels in the accuracy of their primary instructional materials. They feel
strongly that their materials are closely aligned with Pennsylvania Core Academic
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
121
Standards and will support students as they prepare for the Algebra I Keystone Exam.
Each district’s process for adopting primary instructional materials includes, at a
minimum, a review of the standards alignment documentation provided by the content
vendor. Despite high levels of confidence from building principals and Algebra I
teachers that their primary instructional materials are aligned with Pennsylvania Core
Academic Standards, a comprehensive standards analysis shows that the primary
instructional materials used by the three case study districts are missing critical Eligible
Content Items that are addressed on the Algebra I Keystone Exam. The implication is
that if teachers rely heavily on instructional materials that do not address Eligible Content
Items that are presented to students on the Algebra I Keystone Exam, students may not
understand the content, contributing to their low test scores.
In evaluating materials with the Culturally Responsive STEAM Scorecard,
Algebra I teachers were able to quantify their perception of the cultural relevance of their
primary instructional material used for Algebra I. The results of the Culturally
Responsive Scorecard show that teachers are not fully confident in their abilities to
identify bias within their instructional materials accurately. The adoption of
Pennsylvania’s Culturally Relevant and Sustaining Education Framework, including its
standards and competencies, is designed to assist educators in completing this work to
help teachers critically examine their instructional materials to determine if students can
see themselves reflected in their instructional materials. The required training
components associated with the framework may assist teachers in building their
confidence in analyzing instructional materials through a culturally responsive lens.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
122
Research Question 2: “What are the factors that impact instructional delivery in
Algebra I?”
The second research question analyzed the time and funds spent on professional
development for utilizing instructional materials. The study design aimed to determine
whether or not time and funds for professional development related to using instructional
materials in the classroom impacted student achievement. It also sought to understand
the differences in perceptions regarding time and funds for professional development
from building administrators and Algebra I teachers.
There was a discrepancy in teacher opinions on the amount of time given for
professional development, with some strongly agreeing that they had been given plenty
of time and others disagreeing. This may indicate that the amount of time spent on
professional development varies across districts in Westmoreland County. Conversely, it
highlights teachers' different confidence levels in their abilities to utilize their
instructional materials without additional support effectively.
When asked how much money is spent annually on teacher professional
development experiences designed to help teachers utilize their purchased instructional
materials in their classrooms, 81.8% of building administrators who participated in the
survey said they spend up to $25,000.00 However, it is important to note that this is a
building-wide cost. It is not isolated to the funds spent only on professional development
to support implementing Algebra I instructional materials in classrooms.
As discussed in the conclusions for Research Question 1, the Culturally
Responsive STEAM Scorecard found that teachers in Westmoreland County need more
confidence in their ability to evaluate instructional resources. The scorecard also found
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
123
that they need more professional development on using their primary instructional
materials effectively and more support in developing culturally responsive learning
experiences, as teachers feel strongly that students better understand concepts and master
skills when they see themselves in their instructional materials. These factors directly
impact the instructional delivery of Algebra I in middle school classrooms. This may be
influenced positively by implementing the Culturally Relevant and Sustaining Education
framework.
Research Question 3: “What are the teacher and administrator perceptions of the
impact of instructional delivery on student achievement scores in Algebra I?”
The third research question analyzed Teacher and Building Principal Perception
Surveys data relating to instruction, instructional materials, and student achievement
scores. The study design aimed to determine whether there was a correlation between the
Standards Crosswalk results and the historical student achievement scores in Algebra I.
Teachers believe that their instructional materials directly impact student
achievement scores. Teachers are divided on whether the physical instructional materials
they use with students adequately address the skills needed to prepare students for the
Algebra I Keystone Exam. They are less confident that their digital instructional
materials can do so. These results indicate that teachers are not confident in effectively
analyzing instructional materials to identify standards and Eligible Content Items. It may
also suggest that they cannot verify the accuracy of the standards documentation provided
to them by content vendors. Teacher confidence levels may also vary across districts in
Westmoreland County and may be influenced by factors related to Research Question 2.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
124
Teachers overwhelmingly believe students do better on assessments when they
see themselves in their instructional materials. However, teachers acknowledge that their
primary instructional materials are only sometimes culturally reflective of their entire
student population. To address this, teachers must supplement their instruction with
resources that they create other purchased instructional materials that are more culturally
relevant. In doing so, teachers can add variety to how they introduce concepts and skills
to students through their materials. However, the need to supplement their instruction
with additional instructional materials means that teachers also need extra professional
development time to curate or develop those materials. Due to the additional time
required, teachers hesitate to implement supplemental instructional materials within their
classrooms.
Teachers may also utilize Open Educational Resources (OER) to develop
instructional experiences that are more culturally reflective of their student population.
Using OER in their classrooms enables teachers to modify resources to meet the needs of
their students better. However, as we saw with additional purchased instructional
materials, teachers are hesitant to implement OER in their classrooms. This is due to
several factors, including a need for more understanding about OER and a lack of time to
explore or have comprehensive professional development and ongoing support regarding
OER initiatives within their schools.
These findings suggest that the main reasons Open Educational Resources are not
widely adopted in Algebra I classrooms in Westmoreland County are a need for more
understanding about OER and a fear of being judged for using them. The current
political climate makes teachers hesitant to explore materials beyond their approved
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
125
primary materials. Building administrators are also reluctant to encourage their use due
to the current political environment. Both groups fear that the broad acceptance of OER
and open discussion of culturally responsive pedagogy will open them up to criticisms
from stakeholders and that these criticisms will detract from the positive benefits of both
for students. At this point, instead of working to educate stakeholders, most building
principals and teachers feel that it makes sense to find an amicable path forward to
support students without causing additional strife among their communities.
Overview of Results
The overall research design aimed to investigate the relationship between low
Keystone Algebra I Scores in middle schools in Westmoreland County, the perceptions of
building principals and Algebra I teachers regarding instructional materials, training and
support provided to teachers, and the cultural responsiveness of the materials. The
researcher collected data from various sources, such as student assessment scores, district
demographics, budget information, and surveys. The analysis revealed two key factors
that may impact student achievement in Algebra I. The first key factor was that teachers
felt their instructional materials lacked guidance and support for culturally responsive
learning experiences. The second key factor was that the primary materials used by
teachers did not adequately cover the content assessed in the Algebra I Keystone Exam,
requiring teachers to supplement their instruction with additional materials from sources,
including OER.
Regarding instructional delivery, the survey data showed that teachers believed
they had enough time and support to use the instructional resources when provided with
professional development effectively. However, teachers needed to feel more confident
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
126
about developing culturally responsive experiences and identifying resource gaps.
Regarding teacher and administrator perceptions, they recognized the impact of
instruction and materials on student achievement but needed to analyze resources from a
standards-alignment and culturally responsive perspective. There also needed to be a
greater understanding of OER despite teachers regularly modifying materials to meet
student needs.
The most significant finding was the correlation between student achievement
scores and the presence of Assessment Anchors and Eligible Content Items in the
instructional materials used by Algebra I teachers. The identified case study districts,
Greater Latrobe School District, Monessen City School District, and Yough School
District, revealed a more detailed look into the results of this study. The following
sections discuss specific results for each of the three case study districts,
recommendations, and fiscal implications for each.
Greater Latrobe School District
The Greater Latrobe School District has been using Go Math by Houghton
Mifflin Harcourt as their primary instructional resource for middle school Algebra I. A
comprehensive analysis of this resource reveals that only 36.25% of the Algebra I
Eligible Content Items are present in this textbook. With the use of this resource, plus the
addition of supplemental materials that teachers both developed and found from external
sources, 8th-grade student scores on the Algebra I Keystone Exam were in the following
proficiency categories:
● Below Basic: 0%
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
127
● Basic: 11.7%
● Proficient: 40.5%
● Advanced: 47.7%
The teachers within the Greater Latrobe School District expressed a strong sense
of satisfaction with the curriculum of their primary Algebra I instructional materials,
emphasizing its strong alignment with Pennsylvania Core Academic Standards. Teachers
were also content with the curriculum's rigor, feeling that the material presented to
students is appropriately demanding and helps them to apply concepts and skills to
further their understanding. They also feel that the materials help to prepare students for
the Algebra I Keystone Exam.
Teachers felt that their primary Algebra I instructional materials have the ability
to represent diverse communities effectively. Moreover, the teachers were pleased with
how the curriculum went beyond mere representation and actively incorporated the
expertise of these diverse and historically marginalized communities. By integrating
their knowledge and insights, the instructional materials provided a comprehensive
understanding of algebraic concepts and served as a platform to elevate the voices and
contributions of these communities. The teachers found this approach essential in
fostering a classroom environment that values diversity and promotes inclusivity.
Throughout the student-facing materials, the teachers felt that the curriculum
consistently highlighted these communities' achievements, struggles, and unique
perspectives. The teachers recognized the significance of this inclusive approach, as it
empowered students to embrace diversity and recognize the contributions of historically
marginalized people. Teachers expressed dissatisfaction with how their primary Algebra
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
128
I instructional materials approached essential concepts such as interdependence and
continual change. They felt that the topics were not adequately addressed within the
instructional materials, leaving gaps in students' understanding of the interconnectedness
of mathematical concepts and their relevance in real-world scenarios. The teachers
recognized the significance of teaching these concepts as they are crucial for developing
critical thinking skills and preparing students for the complexities of the modern world.
Teachers also found the instructional activities in the materials lacking when presenting
alternatives or allowing students to explore and offer their own alternatives regarding the
role of science, technology, engineering, arts, and math (STEAM) in existing social
structures.
A range of perspectives emerged among teachers regarding their satisfaction with
the teachers' primary Algebra I instruction materials in addressing crucial aspects such as
cultural responsiveness and real-life connections. While some teachers expressed high
satisfaction, others felt that the materials fell short in these areas. Teachers recognize the
importance of making real-life connections between academic content and the local
neighborhoods, culture, environment, community issues, and current events and feel that
their primary instructional materials do not give them the flexibility to do so. When they
have materials that provide them with this flexibility, they can better foster meaningful
engagement and relevance in the learning process, enabling students to see the direct
applicability of Algebra I concepts in their own lives.
Based on these results, the researcher recommends that the Greater Latrobe
School District adopt a new primary instructional resource for their middle school
Algebra I classrooms. During their resource evaluation process, the researcher suggests
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
129
Algebra I teachers and building administrators engage in a thorough standards-analysis
review to determine which Algebra I Eligible Content Items are present within each
resource being considered for adoption. In doing so, teachers will better understand the
items not present in their final decision. The fiscal implication of this recommendation is
that the district will need to incur the cost of purchasing a new textbook for their middle
school Algebra I classes, which may be more than expected if this action falls outside
their typical adoption cycle.
The researcher also recommends that the Greater Latrobe School District partner
with the Curriculum Services Team at the Westmoreland Intermediate Unit to engage in
ongoing professional development for all staff around the effective and ongoing use of
Open Educational Resources in their classrooms to support culturally responsive
pedagogy. This training will help teachers and building administrators learn how to
curate, modify, and remix resources to meet the needs of their student population. It will
also help them develop their instructional skills to effectively represent diverse
communities in their lessons and design lessons and materials that better represent the
students within their classrooms while also aligning to content standards and Eligible
Content Items. The fiscal implication of this recommendation is minimal. It may include
minor upgrades to the technical infrastructure of the district to ensure that teachers and
students can effectively utilize digital resources as needed.
Finally, the researcher recommends that the Greater Latrobe School District
utilizes a new primary Algebra I resource and supports the implementation of OER
through ongoing professional development for multiple years to track the efficacy of
these recommendations. The district will not see sustainable results with a one-year
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
130
implementation. Instead, they must commit to a systematic approach to professional
development that supports teachers in learning to use new resources, develop their own
resources, and grow their knowledge and skills. Adding an OER professional
development initiative to the district’s Comprehensive Plan will help the district to
commit to a long-term plan for growth and to see their efforts reflected in student test
scores.
Monessen City School District
The Monessen City School District has been using Savvas’ enVision Algebra I
program as their primary instructional resource for middle school Algebra I. A
comprehensive analysis of this resource reveals that only 48.75% of the Algebra I
Eligible Content Items are present in this textbook. With the use of this resource, plus the
addition of supplemental materials that teachers both developed and found from external
sources, 8th-grade student scores on the Algebra I Keystone Exam were in the following
proficiency categories:
● Below Basic: 54.5%
● Basic: 27.3%
● Proficient: 18.2%
● Advanced: 0%
Unfortunately, no perception or Culturally Responsive STEAM Scorecard data is
available from the Monessen City School District. Both building administrators and
Algebra I teachers did not complete the perception surveys or the scorecard. This means
that the researcher is unable to determine if the teachers in the district with the most
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
131
diverse population of the three case study districts feel that their instructional materials
are culturally responsive or if their instructional materials are doing an adequate job of
addressing the standards and preparing their students for the Algebra I Keystone Exam.
Based on the information that the researcher collected, the researcher can
recommend that the Monessen City School District adopt a new primary instructional
resource for their middle school Algebra I classrooms. The researcher recommends that
the district consider an OER-based primary resource to provide teachers with a higher
level of flexibility in addressing the wide variety of needs that the students within their
classrooms have. This will also allow the district to reallocate funds typically used for
resource adoption and instead dedicate them to additional professional development to
better support teachers in understanding how to utilize OER to meet the needs of students
effectively.
As was recommended for the Greater Latrobe School District, the researcher
recommends that the Monessen City School District partner with the Curriculum Services
Team at the Westmoreland Intermediate Unit to engage in ongoing professional
development for all staff around the effective and ongoing use of Open Educational
Resources in their classrooms to support culturally responsive pedagogy. This training
will help teachers and building administrators develop their instructional skills to
effectively represent diverse communities in their lessons and design lessons and
materials that better represent the students within their classrooms while also aligning to
content standards and Eligible Content Items. The fiscal implication of this
recommendation is minimal. It may include minor upgrades to the technical infrastructure
of the district to ensure that teachers and students can effectively utilize digital resources
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
132
as needed. The researcher also recommends that the district commits to ongoing
professional development for multiple years to track these recommendations' efficacy.
Adding an OER professional development initiative to the district’s Comprehensive Plan
will help the district to commit to a long-term plan for growth and to see their efforts
reflected in student test scores.
Yough School District
The Yough School District has been using CK-12’s Algebra I flexbook as their
primary instructional resource for middle school Algebra I. A comprehensive analysis of
this OER-based resource reveals that 62.5% of the available Eligible Content items for
Algebra I within their curriculum are in its original form before users modify it. With the
use of this resource, plus the addition of supplemental materials that teachers both
developed and found from external sources, 8th-grade student scores on the Algebra I
Keystone Exam were in the following proficiency categories:
● Below Basic: 2.4%
● Basic: 39%
● Proficient: 43.9%
● Advanced: 14.6%
Overall, the teachers' satisfaction with the curriculum of their primary Algebra I
instructional materials reflected a commitment to inclusivity and equity within the Yough
School District. By adequately representing and incorporating the expertise of diverse
and historically marginalized communities, the curriculum promoted a positive and
inclusive learning environment that valued the experiences and contributions of all
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
133
students. Teachers in the Yough School District expressed less satisfaction with how
their primary Algebra I instructional materials tackled key concepts such as
interdependence and continual change. They felt that these areas were not adequately
addressed, leaving gaps in students' understanding of the interconnected nature of these
concepts and their relevance in real-world contexts. The teachers believed that a
comprehensive understanding of interdependence and continual change is essential for
developing critical thinking skills and preparing students for the complexities of the
modern world.
Teachers were uncertain if the materials effectively encouraged students to
analyze and challenge how STEAM subjects impact society critically. However, they
acknowledge that they need more confidence to evaluate resources through this lens
effectively. The teachers emphasized the need for instructional materials to provide more
explicit guidance and resources for addressing interdependence, continual change, and
the societal implications of STEAM. They believed that by encouraging students to
explore alternatives and present their own ideas, the materials would foster a deeper
understanding of these concepts and promote critical thinking skills.
In the Yough School District, the Teachers' Materials category stood out as the
only area where teacher opinions varied greatly. Some teachers expressed high
satisfaction with the teachers' materials, appreciating their guidance on utilizing students'
everyday lives as a foundation for learning. These teachers found value in the materials'
emphasis on making real-life connections between academic content and various aspects
of the local community, including neighborhoods, culture, environment, community
issues, and current events. They also commended the materials for promoting
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
134
collaboration and leveraging individuals' unique assets, resources, and strengths to
enhance group and individual learning experiences.
Conversely, other teachers held reservations regarding Teachers' Materials. They
felt that the guidance provided for teachers to be aware of their biases, assumptions, and
cultural gaps between themselves and their students was unclear. These teachers believed
that the materials did not adequately prompt teachers to reflect on their own practices and
experiences in learning science, technology, engineering, arts, and math (STEAM)
subjects through the lens of cultural responsiveness. The contrasting perspectives among
teachers underscored the need for more explicit and comprehensive guidance within the
Teachers' Materials category.
Teachers in the Yough School District expressed satisfaction with the materials
and resources associated with their primary Algebra I instructional materials. They
appreciated that these materials offered students opportunities to engage with STEAM
concepts using accessible technology and manipulatives and were pleased that the rigor
of the curriculum was not contingent upon access to specific resources, materials, or
technology that students and schools may not have. This ensured that all students could
actively participate in and benefit from the curriculum's challenges and academic rigor
regardless of their available resources.
Based on these results, the researcher recommends that the Yough School District
continue utilizing the OER-based CK-12 Algebra I flexbook with their middle school
Algebra I classes. However, the researcher also suggests that the district leverage the
resource's flexible nature and modify the flexbook's structure to better align the book to
Pennsylvania Core Academic Standards by adding in the missing Eligible Content Items
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
135
from other instructional content within the CK-12 system. This will require teachers to
understand where to find those items within the CK-12 system and know how to add
them to their Algebra I flexbook to truly develop a personalized resource that matches the
scope and sequence of instruction that Yough School District has adopted, is aligned to
state standards, and covers all Eligible Content Items covered on the Algebra I Keystone
Exam. Since CK-12 is an OER-based resource, there will be no additional cost to the
district to enhance its already adopted resources. However, the researcher does
recommend that the district reallocate funds to increase the amount of professional
development and collaboration time for teachers to engage in this work.
As with the Greater Latrobe School District and the Monessen City School
District, the researcher also recommends that the Yough School District partner with the
Curriculum Services Team at the Westmoreland Intermediate Unit to engage in ongoing
professional development for all staff around the effective and ongoing use of Open
Educational Resources in their classrooms to support culturally responsive pedagogy.
This training will help teachers and building administrators learn how to curate, modify,
and remix resources to meet the needs of their student population beyond the CK-12
system and introduce teachers and building administrators to several reputable
repositories for OER. The fiscal implication of this recommendation is minimal. It may
include minor upgrades to the technical infrastructure of the district to ensure that
teachers and students can effectively utilize digital resources as needed.
The researcher also recommends that the Yough School District include these
recommendations in its Comprehensive Plan to engage in this work over multiple years.
In doing so, both building principals and teachers can commit to actively engaging in the
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
136
professional development and collaboration needed to implement OER within the
classroom in a sustainable manner effectively. This will also give the district time to see
their work reflected in student test scores.
Limitations
The established research design and implementation resulted in one significant
limitation, which impacted the results and findings. This significant limitation was the
low participation numbers by teachers across Westmoreland County. This particularly
impacted the results and findings for the Monessen City School District, as the researcher
could not gather any perception data for that school district. At the start of the study, four
Algebra I teachers from Monessen City School District indicated their willingness to
participate. However, none of the teachers took any steps to sign the Informed Consent
Letter, complete the Teacher Perception Survey, or the Culturally Responsive STEAM
Scorecard.
In addition, only six Algebra I teachers across the county were willing to engage
in any of the activities for this study. As the results suggested, teachers are concerned
about criticism from stakeholders regarding culturally responsive pedagogy. The
researcher feels that the lack of participation may be due to the topic of this study, the use
of the term culturally responsive, and fear over the current political climate in relation to
that term.
While not a limitation of the study itself, it is worth mentioning that the statewide
adoption and implementation of the Culturally Relevant and Sustaining Education
(CR-SE) guidelines and framework has been put in jeopardy during the study. Originally
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
137
passed into law on April 23, 2022, as part of Chapter 49, relating to the Certification of
Professional Personnel, the law has been challenged several times. Most recently, several
western Pennsylvania school districts have filed a lawsuit to prevent the implementation
of the CR-SE guidelines.
In addition, legislation was introduced to the House of Representatives Education
Committee on May 31, 2023, proposing the abrogation of Chapter 49 regulations related
to CR-SE. Should that legislation pass, it will go into effect immediately, eliminating the
CR-SE guidelines and framework, preventing their use, and any training for teachers
regarding culturally responsive pedagogy. While these developments did not directly
impact the study itself, they may have an impact on the researcher's recommendations.
Future Directions for Research
The findings of this research warrant further study to gain a better understanding
of the challenges building administrators and teachers face when working to bridge the
gaps between the intended curriculum, the implemented curriculum, and the attained
curriculum in order to positively influence student achievement. The researcher
recommends that future researchers continue this work and seek out larger participant
groups to gain a more comprehensive understanding of teachers' perceptions. Future
researchers can achieve this by expanding the work beyond middle school, looking at
math scores across transition years, and including all math teachers in the study. Doing
so will allow participants to work together more collaboratively and engage in
meaningful discussion while completing the Culturally Responsive STEAM Scorecard
rather than in isolation.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
138
The researcher recommends replicating this study with other public school
districts within Westmoreland County to understand better the resources being used
across the county. This will also provide county administrators and teachers with a
common critical analysis experience regarding their instructional materials and student
test scores and can serve as the starting point for a countywide Algebra I community of
practice to meet regularly to share resources, experiences, instructional practices, and to
engage in problem-solving around common concerns related to Algebra I test scores in
Westmoreland County.
In replicating the study, the researcher recommends also looking at the
assessments used by Algebra I teachers in their classrooms throughout the year, the
alignment of those assessments to the Eligible Content Items on the Algebra I Keystone
Exam, and the implementation of common assessments by teachers in the same districts.
This will give future researchers better insight into how Algebra I teachers regularly
assess students on their understanding and mastery of skills and concepts taught to
students with the help of the primary instructional materials used by teachers throughout
the school year as they prepare for the Algebra I Keystone Exam.
Finally, the researcher recommends expanding this study into other content areas
beyond mathematics. Doing so will test to see if the research design is effective and if
the study applies to other content areas beyond mathematics. With the new adoption of
the Science, Technology & Engineering, and Environmental Literacy & Sustainability
(STEELS) Standards and the current focus that many districts have on finding new
resources that align with new instructional strategies, this study may be well-suited to be
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
139
replicated for districts that are concerned with student test scores on the Biology
Keystone Exam next.
Summary
To understand the root cause of students’ historically low test scores on the
Algebra I Keystone Exam, the researcher developed a mixed-method embedded research
design that examined the impact of the instructional materials used in Algebra I
classrooms on student achievement. Chapter V presented the conclusion of this research
by providing a comprehensive analysis of the findings for each of the three identified
research questions. The chapter also included specific implications for the three case
study districts identified within the research study, providing each study district with
personalized recommendations and the fiscal implications for implementing those
recommendations. Finally, Chapter V discussed the limitations of the research and
recommendations for further study by future researchers in the hopes of continuing the
work to support building administrators and teachers as they work to meet the needs of
their individual student populations.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
140
References
Abdulrahim, N., & Orosco, M. J. (2019, April 5). Culturally responsive mathematics
teaching: A research synthesis. The Urban Review, 52(1), 1-25.
https://doi.org/10.1007/s11256-019-00509-2
Ayers, D. F. (2006, June). Culturally responsive pedagogy and online learning:
Implications for the globalized community college. Community College Journal
of Research and Practice, 30(5-6), 401-415.
https://doi.org/10.1080/10668920500442125
Brown, C., Boser, U., Sargrad, S., & Marchitello, M. (2016, January 29). Implementing
the Every Student Succeeds Act toward a coherent, aligned assessment system.
Center for American Progress.
https://www.americanprogress.org/article/implementing-the-every-student-succee
ds-act/
Brown, J. C., & Crippen, K. J. (2016, March 2). The Growing Awareness Inventory:
Building capacity for culturally responsive science and mathematics with a
structured observation protocol. School Science and Mathematics, 116(3),
127-138. https://doi.org/10.1111/ssm.12163
Chingos, M. M., & Whitehurst, G. J. ". (2012, April). Choosing blindly: Instructional
materials, teacher effectiveness, and the Common Core. Brookings.
https://www.brookings.edu/research/choosing-blindly-instructional-materials-teac
her-effectiveness-and-the-common-core/
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
141
Cole-Malott, D.-M., Parker Thompson, K., Peterson-Ansari, R., & Whitaker, II, R. W.
(2021, April). The Pennsylvania Culturally Relevant and Sustaining Education
Competencies.
https://s3.amazonaws.com/newamericadotorg/documents/The_Pennsylvania_Cult
urally_Relevant_and_Sustaining_Education_Competencies.pdf
Copur‑Gencturk, Y., Jacobson, E., & Rasiej, R. (2021, February 18). On the alignment of
teachers’ mathematical content knowledge assessments with the common core
state standards. Journal of Mathematics Teacher Education, 25, 267–291.
https://doi.org/10.1007/s10857-021-09486-4
Dorn, E., Hancock, B., Sarakatsannis, J., & Viruleg, E. (2020, June 1). COVID-19 and
student learning in the United States: The hurt could last a lifetime. McKinsey.
https://www.mckinsey.com/industries/education/our-insights/covid-19-and-studen
t-learning-in-the-united-states-the-hurt-could-last-a-lifetime
Elementary and Secondary Education Act. (1965, April 11). 20 U.S.C. § 6301.
https://www.congress.gov/89/statute/STATUTE-79/STATUTE-79-Pg27.pdf
eMetric. (2023). Data Interaction for Pennsylvania Student Assessments. eMetric Data
Interaction. https://pa.emetric.net
Every Student Succeeds Act. (2015, December 10). 20 U.S.C. § 6301.
https://www.congress.gov/114/plaws/publ95/PLAW-114publ95.pdf
Garland, S. (2020, March 30). Who was behind the Common Core math standards, and
will they survive? The Hechinger Report.
https://hechingerreport.org/behind-common-core-math-standards-will-survive/
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
142
Garth, R., & Sebastian, R. (2020, November 18). Using open educational resources to
create a culturally relevant classroom. Achieving the Dream.
https://achievingthedream.org/using-open-educational-resources-to-create-a-cultur
ally-relevant-classroom/
Gay, G. (2013, January). Teaching to and through cultural diversity. Curriculum Inquiry,
43(1), 48-70. https://doi.org/10.1111/curi
Greater Latrobe School District. (2022). District Information – About Us.
https://www.glsd.us/apps/pages/index.jsp?uREC_ID=1756118&type=d&pREC_I
D=1937431
Hamilton, L. S., Stecher, B. M., & Yuan, K. (2008, December). Standards-based reform
in the United States: History, research, and future directions. Institute of
Education Sciences.
https://www.rand.org/content/dam/rand/pubs/reprints/2009/RAND_RP1384.pdf
Harmon, D. A. (2012, Spring). Culturally responsive teaching though a historical lens:
Will history repeat itself? Interdisciplinary Journal of Teaching and Learning,
2(1), 12-22.
Heise, M. (2017). From No Child Left Behind to Every Student Succeeds: Back to a
future for education federalism. Columbia Law Review, 117(7), 1859-1896.
https://columbialawreview.org/content/from-no-child-left-behind-to-every-student
-succeeds-back-to-a-future-for-education-federalism/
Hendricks, C. (2017). Improving schools through action research: A reflective practice
approach. Pearson.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
143
Improving America's Schools Act. (1994, October 20). 20 U.S.C. § 6301.
https://www.congress.gov/103/statute/STATUTE-108/STATUTE-108-Pg3518.pdf
Jacques, I. (2022, October 24). NAEP math, reading scores: Report card shows how
school closures hurt. USA Today.
https://www.usatoday.com/story/opinion/columnist/2022/10/24/report-card-studen
t-test-scores-reveal-how-much-closing-schools-hurt/10551234002/
Johansson, M. (2003, November). Textbooks in mathematics education A study of
textbooks as the potentially implemented curriculum. Luleå University of
Technology.
Katz, V. J. (2009). A history of mathematics : an introduction. Addison-Wesley.
Kaufman, J. H., Doan, S., & Fernandez, M.-P. (2021). The rise of standards-aligned
instructional materials for U.S. k–12 mathematics and english language arts
instruction [Findings from the 2021 American Instructional Resources Survey].
RAND Corporation.
https://www.rand.org/pubs/research_reports/RRA134-11.html
Lincoln, Y. S., & Guga, E. (1985). Naturalistic inquiry. Sage.
Loeb, H., Knapp, M. S., Elfers, A. M., & University of Washington. (2008, April 21).
Teachers’ response to standards-based reform: Probing reform assumptions in
Washington state. Education Policy Analysis Archives, 16(8).
https://doi.org/10.14507/epaa.v16n9.2008
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
144
Matthews, J. S., & López, F. (2019). Speaking their language: The role of cultural content
integration and heritage language for academic achievement among Latino
children. Contemporary Educational Psychology.
https://doi.org/10.1016/j.cedpsych.2018.01.005
Mehta, J. (2015, Summer). Escaping the shadow: A nation at risk and its far-reaching
influence. American Educator, 39(2), 20-26.
Mertler, C. A. (2021). Introduction to educational research. SAGE Publications,
Incorporated.
Molchanow, K. (2022, April 23). Pennsylvania bulletin (17, 52nd ed.). PA Code and
Bulletin.
https://www.pacodeandbulletin.gov/secure/pabulletin/data/vol52/52-17/52-17.pdf
Mollenkamp, D. (2022, October 24). NAEP ‘nation’s report card’ shows steep fall in
math scores. EdSurge.
https://www.edsurge.com/news/2022-10-24-naep-nation-s-report-card-shows-stee
p-fall-in-math-scores
Monessen City School District. (2023). About Us – District – Monessen City SD.
https://www.monessenschooldistrict.com/apps/pages/index.jsp?uREC_ID=13007
04&type=d&pREC_ID=1504939
Muñiz, J. (2019, March 28). Culturally Responsive Teaching. New America.
https://www.newamerica.org/education-policy/reports/culturally-responsive-teach
ing/
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
145
National Assessment of Educational Progress. (2022). 2022 Mathematics and Reading
Report Cards at Grades 4 and 8. The Nation's Report Card.
https://www.nationsreportcard.gov/
National Center for Education Statistics. (1995). Pursuing excellence: A study of U.S.
fourth-grade mathematics and science achievement in international context.
https://nces.ed.gov/timss/results95.asp
National Commission on Excellence in Education. (1983). A nation at risk: The
imperative for educational reform. U.S. Government Printing Office.
National Governors Association Center for Best Practices & Council of Chief State
School Officers. (2022, October 31). Common Core State Standards for
Mathematics. Council of Chief State School Officers.
https://ccsso.org/resource-library/common-core-state-standards-mathematics
No Child Left Behind Act. (2002, January 8). 20 U.S.C. § 6301.
https://www.congress.gov/107/plaws/publ110/PLAW-107publ110.pdf
Owens, J. E. (1988, August 9). Curriculum and evaluation standards for school
mathematics: Report of the national council of teachers of mathematics'
commission on standards for school mathematics. Institute of Education Sciences.
Pennsylvania Association of Intermediate Units. (2022). Focus. PAIU.
https://paiu.org/Focus
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
146
Pennsylvania Department of Education. (2013). Fact sheet: PA Academic Standards and
the Common Core.
https://www.sasd.k12.pa.us/Downloads/FACT%20SHEET%20for%20PA%20Aca
demic%20Standards%20and%20the%20Common%20Core%20April%202013.pd
f
Pennsylvania Department of Education. (2022). Future Ready PA Index district fast facts
- Greater Latrobe SD.
https://futurereadypa.org/District/FastFacts?id=195212081008215029032253089
092020162089031176052
Pennsylvania Department of Education. (2023). Future Ready PA Index.
https://www.education.pa.gov/K-12/ESSA/FutureReady/Pages/default.aspx
Pennsylvania Department of Education, Pennsylvania Educator Diversity Consortium, &
U.S. Department of Education's Region 4 Comprehensive Center. (2021, April).
The Pennsylvania Culturally Relevant and Sustaining Education Competencies.
https://s3.amazonaws.com/newamericadotorg/documents/The_Pennsylvania_Cult
urally_Relevant_and_Sustaining_Education_Competencies.pdf
Pennsylvania Joint State Government Commission. (1997, December). Intermediate units
in Pennsylvania: The role of educational service agencies in promoting equity in
basic education.
Pennsylvania State Board of Education. (2010). Implementing the Common Core
Standards in Pennsylvania: Reflections from the field. Pennsylvania Department
of Education.
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
147
https://www.stateboard.education.pa.gov/Documents/Research%20Reports%20an
d%20Studies/Common%20Core%20White%20Paper.pdf
Petrin, K. (2022, October 23). NAEP scores show big drop in wake of COVID. Chalkbeat.
https://www.chalkbeat.org/2022/10/24/23417139/naep-test-scores-pandemic-scho
ol-reopening
Smith, D. R., & Ayers, D. F. (2006, June). Culturally responsive pedagogy and online
learning: Implications for the globalized community college. Community College
Journal of Research and Practice, 30, 401-415.
https://doi.org/10.1080/10668920500442125
Sparks, S. D., & Tomko, G. (2022, October 24). Two decades of progress, nearly gone:
National math, reading scores hit historic lows. Education Week.
https://www.edweek.org/leadership/two-decades-of-progress-nearly-gone-national
-math-reading-scores-hit-historic-lows/2022/10
Superfine, B. M. (2009). The courts and standards based reform. Oxford University
Press, USA.
Tampio, N. (2018). Common Core: National education standards and the threat to
democracy. Johns Hopkins University Press.
Tang, H. (2020, September). A qualitative inquiry of k–12 teachers’ experience with open
educational practices: Perceived benefits and barriers of implementing open
educational resources. International Review of Research in Open and Distributed
Learning, 21(3), 211-229. https://doi.org/10.19173/irrodl.v21i3.4750
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
148
Tang, H., Lin, Y.-J., & Qian, Y. (2020, April). Understanding K-12 teachers’ intention to
adopt open educational resources: A mixed methods inquiry. British Journal of
Educational Technology, 51(6), 2558–2572. https://doi.org/10.1111/bjet.12937
UNESCO. (n.d.). Open educational resources. UNESCO.org. Retrieved November 12,
2022, from
https://www.unesco.org/en/communication-information/open-solutions/open-educ
ational-resources
United States Department of Education. (2008). A nation accountable: Twenty-five years
after a nation at risk. U.S. Department of Education.
https://www2.ed.gov/rschstat/research/pubs/accountable/accountable.pdf
Watt, M. G. (2011). The Common Core State Standards Initiative: An Overview. Institute
of Education Services.
Westmoreland Intermediate Unit. (n.d.). Curriculum Services General Information Curriculum - Westmoreland IU 7. Westmoreland Intermediate Unit.
https://www.wiu7.org/apps/pages/index.jsp?uREC_ID=1559997&type=d&pREC
_ID=1605947
Xie, K., Nelson, M. J., Cheng, S.-L., & Jiang, Z. (2021, August). Examining changes in
teachers’ perceptions of external and internal barriers in their integration of
educational digital resources in K-12 classrooms. Journal of Research on
Technology in Education. https://doi.org/10.1080/15391523.2021.1951404
Yough School District. (2020). General Information/Alma Mater. Yough School District.
https://www.youghsd.net/general_informationalma_mater
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Appendix A
149
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
150
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
151
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
152
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Appendix B
153
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
154
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
155
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
156
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
157
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
158
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Appendix C
159
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
160
IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS
Appendix D
161