Understanding the Impact of Culturally Responsive Instructional Materials on Middle School Students’ Math Achievement Scores in Westmoreland County A Doctoral Capstone Project Submitted to the School of Graduate Studies and Research Department of Education In Partial Fulfillment of the Requirements for the Degree of Doctor of Education Rebecca M Henderson Pennsylvania Western University July 2023 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Rebecca M. Henderson July 2023 ii IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS iv Dedication To my children, Neil and Elizabeth. The most meaningful things we do are often the hardest things we do. It's a concept that may seem scary at first, but it's an essential truth that I want you to understand and embrace. During your lives, you will encounter situations that appear overwhelming, and it's normal to feel a sense of fear or doubt. Fear is a natural response, but it should not determine your actions or hinder your progress. Instead, let courage guide you and push through the fear. Trust in your abilities and believe in yourself. You are capable of incredible things. Remember, we do hard things. Love, Mom IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS v Acknowledgements The completion of this Capstone Research Project would not have been possible without the love, support, guidance, and encouragement that I received from family, friends, and colleagues during this process. I want to start by thanking my husband, Scott, and my children, Neil and Elizabeth. Thank you for believing in me and supporting me, even when it meant spending less time together so that I could complete this work. A special thank you to my parents, Jim and Betty, and the rest of my family for their continued love and support. To my friends who have supported me, and laughed with me, and have always kept me grounded. A special thank you to the following colleagues: Dr. Jason Stragand and Dr. Denise Lohr, for encouraging me to enter this program and going on this journey with me, Dr. Matt Thomas, Dr. Jason Boone, and Mr. Timothy Hammill, for their friendship, humor, and wisdom, and Mr. Blaine Howard, for suggesting the focus of my research. Thank you to Dr. J. Kevin Lordon, Faculty Capstone Committee Advisor, who encouraged me throughout the process. A very special thank you to Dr. Jason A. Conway, for your guidance, support, feedback, encouragement, and wisdom during this process. I deeply value the environment that you have established at the Westmoreland Intermediate Unit, where the focus on personal and professional growth is paramount and nurtured for every employee. And as always, thank you to Steve Perry and Corky St. Clair. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS vi List of Figures and Tables Figure 1: Westmoreland County Student Enrollments by Race / Ethnicity 7 Figure 2: Countywide Average by Anchor 2015 - 2019 8 Figure 3: What Culturally Responsive Competencies do State Teaching Standards Address? 31 Figure 4: Growing Awareness Inventory Theory of Action 33 Figure 5: Greater Latrobe School District Percent Enrollment by Race/Ethnicity 50 Figure 6: Greater Latrobe School District Percent Enrollment by Student Groups 50 Figure 7: Greater Latrobe School District Algebra I Proficiency Percentages 2013 - 2022 51 Figure 8: Greater Latrobe School District Total Number of Student Tested vs. Year Administered 52 Figure 9: Monessen City School District Percent Enrollment by Race/Ethnicity 53 Figure 10: Monessen City School District Percent Enrollment by Student Groups 54 Figure 11: Monessen City School District Algebra I Proficiency Percentages 2013 - 2022 55 Figure 12: Monessen City School District Total Number of Student Tested vs. Year Administered 56 Figure 13: Yough School District Percent Enrollment by Race/Ethnicity 57 Figure 14: Yough School District Percent Enrollment by Student Groups 58 Figure 15: Yough School District Algebra I Proficiency Percentages 2013 - 2022 59 Figure 16: Yough School District Total Number of Student Tested vs. Year Administered 60 Figure 17: Group Summary: Winter 2012 Performance Keystone, Algebra I 78 Figure 18: Group Summary: Spring 2022 Performance Keystone, Algebra I 79 Table 1: All Districts’ Total Student Enrollments by Race/Ethnicity 80 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS vii Figure 19: Percent Enrollments Economically Disadvantaged by District 81 Figure 20: Percent Enrollments ELL by District 82 Figure 21: Percent Enrollments Special Ed. by District 83 Figure 22: Enrollments by Gender (M/F) by District 84 Tabe 2: Building Principal Perception Survey Questions 85 Table 3 Teacher Perception Survey Questions 86 Table 4: Culturally Responsive STEAM Scorecard 88 Table 5: Group Summary, Performance Levels by Grade 08 and Ethnicity 92 Figure 23: GLSD Culturally Responsive STEAM Scorecard Results 95 Figure 24: YSD Culturally Responsive STEAM Scorecard Results 98 Table 6: Number of Algebra I Eligible Content Items Present by Vendor 99 Figure 25: Teacher Perceptions of Standards Alignment of Instructional Materials 101 Figure 26: Building Principals’ Perceptions of Instructional Materials Alignment to Standards 102 Figure 27: Teacher Perceptions of PD to Effectively Use Instructional Materials 103 Figure 28: Money Spent Annually By Districts on Teacher PD to Utilize Instructional Material 104 Figure 29: Teacher Perceptions on the Impact of Instructional Materials on Student Achievement 106 Figure 30: Teacher Perceptions on Physical Materials Preparing Students for Keystone Exams 107 Figure 31: Teacher Perceptions on Digital Materials Preparing Students for Keystone Exams 107 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS viii Figure 32: Teachers' Perceptions of Student Reflections on Instructional Materials 108 Figure 33: Building Principals’ Perceptions of Teacher Modifications to Resources 109 Figure 34: Building Principals’ Perceptions of Teacher Development of Original Resources 109 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS ix Table of Contents Dedication iv Acknowledgement v List of Figures vi Abstract xiii Chapter I. Introduction 1 Background of Study 1 Identification of Capstone Focus 3 Research Questions 4 Expected Outcomes 4 Fiscal Implications 4 Summary 5 Chapter II. Literature Review 6 Standards-Based Education Reform in the United States 8 The National Council of Teachers of Mathematics 10 National Education Goals 12 No Child Left Behind 14 Common Core State Standards 15 Pennsylvania Common Core Standards 16 The Every Student Succeeds Act 17 Publishers’ Responses to Standards-Based Education Reform 18 Textbook Content vs. Proficiency Exam Content 19 Textbook Structure 20 Standards Alignment 21 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Educators’ Response to Standards-Based Instructional Materials x 22 Teacher Professional Development 23 Process for Adopting and Implementing New Standards in PA 24 A Mindset Shift in Mathematics 26 History of Culturally Responsive Teaching in the United States 27 Culturally Responsive Teaching in Mathematics 28 Culturally Responsive Teaching in Pennsylvania 30 The Impact of Reform on Culturally Responsive Teaching 32 Culturally Responsive Teaching as a Pedagogical Practice 34 Implementing OER to Facilitate Culturally Responsive Teaching 35 Perceived Benefits of Using Open Educational Resources 36 Perceived Barriers to Increasing Student Achievement 37 Understanding the Impact of COVID-19 on Student Achievement 38 NAEP Results by Student Group 41 Steep Declines in Algebra 41 Summary 42 Chapter III. Methodology 44 Methodology 44 Purpose of Study 45 Setting 46 Participants 49 Research Plan 60 Research Methods & Data Collection 66 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS xi Data Collection Timeline and Plan Described 67 Validity 69 Summary 71 Chapter IV. Data Analysis and Results Data Analysis 73 74 Participants 74 Data Analysis Procedure 75 Results 91 Research Question 1 91 Research Question 2 102 Research Question 3 105 Discussion 110 Summary 115 Chapter V. Conclusion and Recommendations Conclusion 118 119 Research Question 1 120 Research Question 2 122 Research Question 3 123 Overview of Results 125 Limitations 136 Future Directions for Research 137 Summary 139 References 140 APPENDIX A. Building Principals Survey 149 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS xii APPENDIX B. Teacher Perception Survey 153 APPENDIX C. Culturally Responsive STEAM Scorecard 159 APPENDIX D. IRB Approval 161 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS xiii Abstract Building Administrators and Algebra I teachers in Westmoreland County have worked hard to determine the root causes of students’ historically low achievement scores on the Algebra I Keystone Exam over the past decade. Through intensive data analysis, they have investigated the alignment of their curriculum to Pennsylvania Core State Standards, their instructional practices, and the alignment of their in-class assessments to Eligible Content Items covered on the Algebra I Keystone Exam. Yet one area that has not been intensively investigated is the impact of culturally responsive instructional materials on student achievement scores. This research study focused on building administrator and teacher perceptions of their primary instructional materials used in Algebra I classrooms, as well as the amount of time and money dedicated to support educator professional development to successfully implement those resources in their classrooms. Participants completed perception surveys to share their thoughts about their instructional material, supplemental materials, Open Educational Resources, professional development, and time and funding spent by their district to support teachers. They also completed the Culturally Responsive STEAM Scorecard to evaluate the cultural responsiveness of their primary instructional materials. In addition, the researcher evaluated the presence of Eligible Content Items present on the Algebra I Keystone Exam on primary Algebra I instructional materials used by three case study districts representing Westmoreland County. While there was not a strong correlation between the culturally responsive nature of instructional materials and student achievement scores, there was evidence that shows that the lack of Eligible Content Items present in primary instructional materials negatively impacts student achievement scores. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 1 Chapter I Background Each year, the Curriculum Services Team of the Westmoreland Intermediate Unit (WIU) works with the seventeen public school districts within their geographic region to analyze student achievement data, evaluate their curriculum and instruction, and support student achievement and success. This work focuses more on standards alignment and state-mandated standardized test scores and less on instructional materials' impact on student achievement. Previous actions to complete this work include mapping standards to a district’s curriculum, supporting the development of a more thorough curriculum and concept maps that address tested content, and a deeper understanding of the state standards and how to align instruction to them. While there is less consideration given to the actual instructional materials used to support instruction, those materials make a significant impression on students and teachers and steer teaching and learning within the classroom. Districts rarely separate their curriculum discussions into defined categories. Instead, they use the term curriculum to refer to standards, the scope and sequence of content taught, the instruction delivered, the instructional materials used by teachers to engage students, and the assessments developed and used to determine student achievement. The researcher has been a part of the Curriculum Services Team at the WIU since August 2018. She has served as Distance Learning Supervisor and Curriculum Services Supervisor during this time. In this capacity, she has been responsible for supporting districts as they work to implement instructional initiatives and mandates as set forth by the Pennsylvania Department of Education (PDE) in both traditional and online IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 2 classroom environments. In serving as a liaison between PDE and the seventeen public school districts in Westmoreland County, the WIU’s Curriculum Services team utilizes innovative practices to advance the practice of teaching and learning across Westmoreland County. School Improvement and Accountability is a central focal point of this work for the WIU Curriculum Services Team, as the United States Department of Education (USDOE) has outlined annual requirements that both states and local school districts must meet to qualify for funding. As such, the WIUs Curriculum Services Team is committed to supporting and monitoring the school improvement efforts of districts. These efforts typically include curriculum writing, standards analysis and alignment, assessment data analysis, instructional strategy, and resource review and implementation. As part of this work, the WIU’s Curriculum Services Team regularly analyzes district-specific data on the annual Pennsylvania System of School Assessment (PSSA) and Keystone Exams. The PSSA is a standards-based assessment administered to every student in grades 3 through 8. Students are assessed in English Language Arts and Mathematics each year and in Science in grades 4 and 8. The Keystone Exams serve as end-of-course assessments determining a student’s proficiency in Algebra I, Literature, and Biology. These assessments meet the federal accountability criteria established in the Every Student Succeeds Act (ESSA) of 2015. Each year, PDE releases the scores of all students, all districts, and the statewide average score for each subject area. Historically, Algebra I scores across Westmoreland County’s seventeen public schools have been below the state average, which has been an ongoing concern for administrators. Administrators want to understand the root cause of IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 3 these county-wide historically low scores to support teachers and students better. They have asked the WIU Curriculum Services Team to help them analyze their specific curriculum, instruction, instructional materials, and assessment circumstances and provide consultative services that can promote increased student achievement. Identification of Capstone Focus This research will investigate the possible correlation between the historically low Keystone Algebra I Scores in middle school and building principal and Algebra I teachers’ perceptions of student identification with examples in the word problems found within their middle school instructional materials. There are many barriers to implementing Open Educational Resources (OER) and developing culturally-responsive instruction that reflects a district’s community, demographics, and values to establish equitable instructional practices. Despite the amount of funds made available to districts through the Elementary and Secondary School Emergency Relief (ESSER) Fund, and the resources compiled by the Pennsylvania Department of Education on their Equitable Practices Hub, there appears to be hesitancy to develop instructional materials that meet the needs of their individual student populations. The research is showing that when students see themselves reflected in their instructional content, they learn more and understand content and concepts on a deeper level. Yet, the conversations around school districts in Westmoreland County focus on purchasing technologies, tools, and programs developed by national vendors that do not reflect the students within the county. This study seeks to better understand the correlation between the instructional materials currently in use by 7th and 8th-grade mathematics teachers, their perceived cultural responsiveness by district building IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 4 principals, mathematics teachers, and students, and student achievement scores in Operations with Real Numbers and Expressions in Algebra I classes. Research Questions The following questions will guide this case study: 1. What are the factors that impact student achievement in Algebra I? 2. What are the factors that impact instructional delivery in Algebra I? 3. What are the teacher and administrator perceptions of the impact of instructional delivery on student achievement scores in Algebra I? Expected Outcomes There are many barriers to implementing Open Educational Resources (OER) and developing culturally-responsive instruction that reflects a district’s community, demographics, and values to develop equitable instructional practices. Despite the amount of funds made available to districts through the Elementary and Secondary School Emergency Relief (ESSER) Fund, and the resources compiled by the Pennsylvania Department of Education on their Equitable Practices Hub, there appears to be hesitancy to develop instructional materials that meet the needs of their individual student populations. The results of this case study will provide building administrators and Algebra I teachers with additional guidance and support when determining which instructional materials may significantly impact student achievement within their student populations. Fiscal Implications The fiscal implications of this capstone research project consist solely of the cost associated with the salaries and benefits of the participants for the time required to IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 5 complete the activities within this case study. Each participant will complete a Perception Survey at the start of the case study, which will take 20 minutes to complete. They will then complete the Culturally Responsive STEAM Scorecard, developed by NYU Steinhardt, to share their perception of the cultural relevance of their primary instructional material used for Algebra I as the second step of data collection. This activity will take approximately two hours to complete. Since the participants are employees at three representative districts across Westmoreland County, the cost will be distributed evenly among those districts. The researcher will utilize Google Forms to collect perception survey data at no additional cost to the school districts or the Westmoreland Intermediate Unit. The researcher has also been granted permission to fully use and publish components of the Culturally Responsive STEAM Scorecard, a freely available online resource, by NYU Steinhardt at no additional charge. All costs associated with this capstone research project are indirect, and the overall financial impact for an individual school district will be minimal. Summary Chapter I introduced the background context of this case study. It also explained why studying the impact of instructional materials on student achievement scores in Algebra I across Westmoreland County is essential for administrators and educators to develop culturally responsive instructional experiences that promote increased student achievement. Chapter II will provide a literature review that helps create a thorough understanding of the currently available research to help inform instructional practices. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 6 Chapter II Literature Review Since the early 1980s, Standards-Based Education Reform has driven public education in the United States. This type of education reform calls for the establishment of academic content standards, which are used to determine critical elements that students must know within each content area. Student achievement is also measured against these established standards, making high-stakes testing a key component to monitoring student progress annually. Historically, the individuals that determine the need for Standards-Based Education Reform are not the same individuals that are responsible for the implementation of the standards at the classroom level. While structures are in place to ensure that educators have access to high-quality, job-embedded professional development that is designed to help them implement newly adopted standards within the classroom, much of that professional development is focused on instructional and assessment strategies. Historically, little attention has been given to assisting educators in critically examining instructional materials in order to determine the accuracy of their standards alignment. Unfortunately, instructional materials are one component of a district’s implemented curriculum that has the greatest impact on student achievement and mastery of newly adopted standards. Westmoreland County, located in the Southwestern region of Pennsylvania, is home to approximately 43,710 students in grades Kindergarten through twelfth grade across 17 different public school districts. The smallest school district serves approximately 712 students, while the largest serves approximately 5,304 students. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Figure 1 shows student enrollments in Westmoreland County public school districts by Race / Ethnicity. Figure 1 Westmoreland County Student Enrollments by Race / Ethnicity While student populations across the county tend to be overwhelmingly white, each district has a unique culture and climate based on geographic location, population demographics, and societal norms. A commonality between all districts is their historically low scores in Algebra I on state standardized tests. Since 2015, countywide averages have only risen above a 60% proficiency rating one time. Figure 2 shows the countywide average scores by Algebra I Anchor Analysis from 2015 - 2019. 7 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 8 Figure 2 Countywide Average by Anchor 2015 - 2019 These scores have become a major concern for district administrators and building leaders across Westmoreland County who are analyzing their intended curriculum and implemented curriculum in order to determine how to better support students in the classroom and foster higher levels of student achievement. Standards-Based Education Reform in the United States Standards-based education reform in the United States of America has its roots in the 1983 publication titled, “A Nation At Risk: The Imperative for Educational Reform.” This report, authorized by President Ronald Reagan, was developed by an eighteen-member commission of individuals working within private companies, government agencies, and education organizations. The authors determined that declines IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 9 in the efficacy of public education were due to inconsistencies in four key areas of education: the content or curriculum being covered in classrooms, district expectations for students, the time that is dedicated to education over the course of the year, and the quality of programs designed to prepare candidates for the teaching profession as well as continuing professional development once teachers enter the profession. As the authors laid out their case for the need for a more rigorous approach to K-12 public education in the United States, and warned of an increase in mediocrity among students, the authors state, “Our goal must be to develop the talents of all to their fullest. Attaining that goal requires that we expect and assist all students to work to the limits of their capabilities. We should expect schools to have genuinely high standards rather than minimum ones, and parents to support and encourage their children to make the most of their talents and abilities” (National Commission on Excellence in Education, 1983, p. 13). Furthermore, the authors felt that there must be a focus on equity for students. They cautioned that sacrificing equity over education reform, or education reform over equity, would lead to potentially disastrous consequences. We do not believe that a public commitment to excellence and educational reform must be made at the expense of a strong public commitment to the equitable treatment of our diverse population. The twin goals of equity and high-quality schooling have profound and practical meaning for our economy and society, and we cannot permit one to yield to the other either in principle or in practice. To do so would deny young people their chance to learn and live according to their aspirations and abilities. It also would lead to a generalized accommodation to mediocrity in our society on the one hand or the creation of an undemocratic IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 10 elitism on the other (National Commission on Excellence in Education, 1983, p. 13). The strongest conclusion from this report was that, in order to effectively ensure that students would be able to productively contribute to society and that future generations would be able to compete on a global stage, we would need to develop a more rigorous and structured curriculum that would address the relevant concepts and skills that students would need to master prior to graduation in an equitable manner. Doing so would instill a lifelong love of learning that would encourage individuals to continue seeking learning opportunities throughout their adult lives. It would also continue to foster support for public education well into the future. The report was distinguished historically from others for its strikingly alarmist tone. Mehta (2015) concluded: Public confidence in schooling, which had already been falling as part of the post-Watergate decline of confidence in public institutions, hit a new low after the release of the report in 1983. States rushed to issue their own reports to evaluate whether they were falling short of the new measures of excellence specified by A Nation at Risk. Despite the claims of the critics, A Nation at Risk had been the rare report that galvanized not only debate but also action. (p. 22) The National Council of Teachers of Mathematics The action that resulted from A Nation at Risk is commonly referred to as Standards-Based Education Reform. One of the first recommendations supporting IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 11 Standards-Based Education reform came in 1988 from the National Council of Teachers of Mathematics (Owens, 1988), which released a report titled, “Curriculum and Evaluation Standards for School Mathematics: Report of the National Council of Teachers of Mathematics’ Commission on Standards for School Mathematics.” While there were other recommendations, this report focused heavily on the need for schools to “implement a core curriculum that allows all students the opportunity to study the important ideas and methods of mathematics.” (Owens, 1988, p. 2) In order to allow for time to address new concepts within the core curriculum, the authors recommended a de-emphasis on practicing computational skills with pencil and paper. Instead, they encouraged the use of technology such as calculators and computers to assist students in computation so that they were able to prioritize developing a solid conceptual foundation for skills. Computational skills are defined as more than mere arithmetical competencies. They include, at the secondary level, the extensive manipulation of expressions and equations that are central to current study in Algebra--a considerable portion of the current Algebra I curriculum, for example, is devoted to manipulation skills associated with solving equations, simplifying complex rational expressions, and factoring. As with arithmetic skills, these are not envisioned as unnecessary but as areas that should be de-emphasized; conceptual development of the ideas underlying such operations should take precedence, calculator and computer solutions are stressed, and applications serve as both motivators of operations and practice opportunities for developing skills. (Owens, 1988, p. 3) IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 12 This strategy of de-emphasizing paper and pencil computational skills was developed to better assist teachers in putting a priority on supporting students’ developmental readiness for instruction. Teachers would now have time to focus on instructional strategies designed to support students in developing their conceptual thinking and problem solving skills at various age levels. The end result would be students that can make real-world connections to the content, solve complex problems, make connections to the content year after year, and communicate about mathematical constructs. National Education Goals This was further reinforced in 1990 by President George H.W. Bush and the Nation’s Governors, who adopted six National Education Goals that were to be achieved within the next ten years. These goals formed the basic structure for Standards-Based Education Reform that are still present in the public education system of the United States today. The six National Education Goals establish that, by the year 2000: ● All children in America will start school ready to learn. ● The high school graduation rate will increase to at least 90 percent. ● American students will leave grades 4, 8, and 12 having demonstrated competency in challenging subject matter including English, mathematics, science, history, and geography; and every school in America will ensure that all students learn to use their minds well, so they may be prepared for responsible citizenship, IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 13 ● Further learning, and productive employment in our modern economy. ● United States students will be first in the world in science and mathematics achievement. ● Every adult American will be literate and will possess the knowledge and skills necessary to compete in a global economy and exercise the rights and responsibilities of citizenship. ● Every school in America will be free of drugs and violence and will offer a disciplined environment conducive to learning. (Educate America Act, 1994) These goals, intended to be seen as overlapping, intricate goals that complimented each other, were designed to provide a more structured framework for public education. Officially becoming known as the Neighborhood Schools Improvement Act, the 102d Congress debated the merits of the six goals, as well as the roles of individuals and organizations that would be involved in monitoring public schools as they worked towards achieving those goals. Ultimately, the 102d Congress did not formally pass the Neighborhood Schools Improvement Act and did not formally adopt the goals developed by President Bush and the Nation's Governors at this time. Instead, the previously adopted Elementary and Secondary Education Act of 1965 (ESEA) was reauthorized in 1994 under President Bill Clinton. This reauthorization, called the Improving America’s Schools Act of 1994, acknowledged the importance of the National Education Goals, and began to develop parameters in which schools would need to operate so that students could meet those IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 14 goals. As stated in the document on page 3, since 1988 the country has learned that “All children can master challenging content and complex problem-solving skills. Research clearly shows that children, including low-achieving children, can succeed when expectations are high and all children are given the opportunity to learn challenging material.” Again, there was a reiteration of the need for a focus on problem-solving skills over computational skills, and in setting higher expectations for students in order to push their understanding and mastery of a more challenging curriculum. No Child Left Behind This pattern of reauthorizing ESEA and renaming it would continue into both the George W. Bush and Barack Obama Administrations. During the George W. Bush Administration’s reauthorization, also known as the No Child Left Behind Act of 2001 (NCLB), we still see a strong preference for establishing high achievement standards, however now we see a shift away from a common set of national academic standards. Instead, states are directed to develop their own set of academic standards as well as accompanying assessments to measure student understanding and mastery of content. To stress the importance of this development, Title VI, referred to as Section 601 Flexibility and Accountability, laid forth several conditions for states to receive grant funding for the development of standards-aligned, state assessments designed to ensure validity and reliability by assessing student achievement through multiple measures (United States Congress, 2002). We now see a shift in control for K-12 public education. While accountability has been a central tenant of Standards-Based Education Reform since the publication of A IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 15 Nation at Risk, the control has largely laid at the feet of the individual states. With the inception of NCLB, control over accountability methods now shifted to the federal government. However, it would not be the last shift in control that we would see. Despite a tradition, or at least a conventional myth, of local control that dominated education federalism for decades, federalism boundaries are increasingly noted for their fluidity in the public education sector. Not only are federalism boundaries becoming increasingly fluid, but the pace of change to these boundaries has also quickened over time, no doubt a reflection of increasingly contested federalism claims. (Heise, 2017) The replacement for NCLB, the Every Student Succeeds Act of 2015 (ESSA), shifted that control back to the states again. Common Core State Standards In the midst of this back and forth regarding accountability oversight and control, the conversation regarding the establishment of a core curriculum continued. In 2008, the National Governors’ Association established a group of experts to develop a common set of core standards that could be utilized across all fifty states. In order to do so, this group of experts needed to analyze the current standards already in place in each state and determine whether or not those standards were high-quality, rigorous, and effective. “The common critique was that most American grade-level guidelines were “a mile wide and an inch deep,” in stark contrast to the fewer but more intense expectations in high-achieving countries like Japan and Singapore” (Garland, 2020). Out of their work, the Common Core State Standards Initiative (CCSS) was developed to address learning IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 16 standards in English Language Arts and Mathematics. “The Standards for Mathematical Practice describe varieties of expertise that mathematics educators at all levels should seek to develop in their students” (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2022). The focus of the mathematics standards became processes and proficiency in order to ensure that students were able to not only complete each step of a given problem, but also explain those steps as well as the ways in which the steps were connected to each other. This emphasis on processes and proficiency was a strategic design meant to support students and help to prepare them for their future endeavors. “The goal is not just to have students master calculations that they will perform only on standardized tests; the goal is to have them think in ways that will prepare them for college, careers, and life” (Tampio, 2018). Again, we see a prioritization of skills that will support students as they move beyond high school. Pennsylvania Common Core Standards While CCSS were developed to be relevant, rigorous, and applicable to students in all fifty states, not all fifty states adopted the standards as written. As the Pennsylvania Department of Education (PDE) publicized, “On July 1, 2010, the State Board of Education adopted improved standards in math and English to replace the original standards adopted in 1999. The standards were based off the Common Core initiative” (Pennsylvania Department of Education, 2013). PDE recognized the work that went into the development of CCSS, but acknowledged that the state-specific standards that had been developed and approved for Pennsylvania, referred to as Pennsylvania Common IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 17 Core (PA Core), were tailored to meet the specific needs of students within the Commonwealth. Furthermore, PDE took steps to ensure that the public understood that the standards were written to serve as guidelines that would assist local districts and their personnel in developing their curriculum. The Every Student Succeeds Act The most current reauthorization of ESEA, known as the Every Student Succeeds Act of 2015 (ESSA), was signed into law in 2015 by President Barack Obama. At this point, Standards-Based Education Reform in the United States was now forty years old. This latest reauthorization continued to emphasize the importance of increased rigor and accountability. In 2012, the Obama administration began granting flexibility to states regarding specific requirements of NCLB in exchange for rigorous and comprehensive state- developed plans designed to close achievement gaps, increase equity, improve the quality of instruction, and increase outcomes for all students. (USDOE, 2015) While NCLB was written in such a way that it did not prevent the federal government from promoting a specific set of standards to states, ESSA was written so that the federal government was unable to influence a state’s adoption of a specific set of standards in order to maintain a clear delineation between state and federal authority over K-12 public education (Every Student Succeeds Act, 2015). The flexibility of ESSA and the inherent shift back to state accountability and oversight was touted by many advocates as a key triumph of the new law. The Center for IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 18 American Progress (CAP) noted that this flexibility would be an opportunity to empower local school districts to become innovative in their approaches to accountability. They stressed that this could be achieved by collaborating with stakeholder groups to critically analyze their current testing structures. In doing so, this would allow them to design better-aligned assessment systems that “include the smart and strategic use of formative and interim tests that provide real-time feedback to inform instruction, as well as high-quality summative tests that measure critical thinking skills and student mastery of standards.” (Brown et al., 2016) Publishers’ Response to Standards-Based Education Reform To study the impact of Standards-Based Education Reform, the National Center for Education Statistics (NCES) established the Trends in International Mathematics and Science Study (TIMSS) in 1995. This study, which has been administered to students in grade 4 and grade 8 every four years, assesses and compares student achievement and educational experiences in math and science by collecting data from students, parents, educators, and administrators (NCES, 2022). The United States has been participating in this study since its inception, providing us with 24 years worth of data as of this writing. The TIMSS Framework helps us to critically examine instructional materials, instruction, assessment, and their alignment to each other in order to determine their impact on student understanding and achievement. In working within the TIMSS Framework, we focus on the following terms: IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 19 ● Intended Curriculum: National, social, and educational contexts such as standards documents, curriculum documents, vision statements, and educational objectives. ● Implemented Curriculum: Home, school, teacher, and classroom contexts such as classroom management, instruction, instructional materials used, and the ways in which content is interpreted by teachers. ● Attained Curriculum: Student achievement and attitudes, most commonly obtained through student test scores, and student belief measures. (NCES, 1995) Textbook Content vs. Proficiency Exam Content As Standards-Based Education Reform gained prominence in both federal and state education policy and practice, K-12 textbook and instructional content publishers took notice. Since textbooks were designed to be the instructional tools that help teachers bridge the gap between the intended curriculum of an educational system and the attained curriculum of an individual student, publishers recognized the importance that Standards-Based Education Reform would have, and marketed their products accordingly. In her analysis of mathematics textbooks utilized by countries all over the world, Johanssen (2003) observed that, while many of the textbooks used within the United States have a strong focus on contextual features and performance requirements, the content covered within the textbooks was disproportionately matched to the content that was on student proficiency exams, with the greatest areas of mismatch being in arithmetic, measure, and algebra (p. 28). This is of great concern, as Johanssen goes on to note that in the United States, teachers use textbooks in the least flexible manner, preferring to navigate through them as a part of their implemented curriculum in order, IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 20 lesson by lesson, using the information as presented, regardless as to whether that information aligns to their intended curriculum or locally developed assessment. Textbook Structure In analyzing the response to Standards-Based Education Reform made by textbook and instructional content publishers, an obvious data point comes to us in the number of pages within mathematics textbooks. Overwhelmingly, publishers in the United States include more pages within their mathematics textbooks than publishers from other countries (Johanssen, 2003). Even with this inclusion of additional pages, the structure of reform-oriented textbooks in the United States focused on including as much content as possible within the textbook itself. However, in comparing textbooks to one another, there is a great deal of variance in the specific content within each textbook (Davis et al., 2014). What this clearly shows is that while textbook and instructional content publishers understood the importance of Standards-Based Education Reform on their products, they were unable to develop an industry-wide approach to addressing the changes in content and instruction that came out of this reform initiative. However, similar marketing approaches were employed by publishers. Textbook and instructional content publishers focused on their marketing efforts, adjusting language to show that their products were aligned to CCSS or state-specific standards, however they made little changes to the content itself, continuing to perpetuate the gap between the intended curriculum and the attained curriculum. As Kaufman et al. (2021) acknowledge, “​​One critical reason for this disconnect between state standards and instructional practice is that teachers’ IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 21 curriculum materials—the textbooks and other materials that drive lesson content—are frequently not aligned with state standards (Kaufman et al., 2020; Polikoff et al., 2020). Standards Alignment “The influence of the idea of alignment is evident today in the large number of textbooks, assessment systems, and professional development materials that are explicitly marketed as being aligned with state standards” (Hamilton et al., 2008). They go on to note that while standards alignment is now the universally accepted measure of endorsement for use by educators, “there is no widely accepted method for aligning standards, curriculum, instruction, and assessments, nor is there a consensus on how to determine if existing components are aligned” (Hamilton et al., 2008). The implication is that while most textbook and instructional content publishers market their materials as being aligned to either CCSS, or state-specific standards, there is no mechanism in place for those publishers to determine the consistency and accuracy of their alignment. The consistency and accuracy of their alignment, instead, falls to individual districts as they review materials in consideration of their purchase and use within their classrooms. However, most educators are not provided with adequate training or professional development designed to help them analyze the consistency and accuracy of perceived alignments. Rather, they look to documentation provided to them from the publisher that explains their rationale for their alignment. Furthermore, there is a gap present in the development of math textbooks and instructional content regarding the connection between content standards, teacher’s content knowledge, and their pedagogical knowledge (Copur-Gencturk et al., 2022). This IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 22 gap makes the connection between the intended curriculum and the attained curriculum much harder to bridge if the materials used for the implemented curriculum are not designed to consider this connection. Educators’ Response to Standards-Based Instructional Materials As Loeb et al. (2008) point out, teachers’ responses to Standards-Based Education Reform and their actions in the classroom are central to students achieving the desired attained curriculum. As such, they must be provided with high-quality, job-embedded professional development that is designed to help them understand the intricacies of the reform initiatives, the ways in which student achievement is assessed, and the ways in which their instructional materials are aligned to reform initiatives. In doing so, we are able to address the assumptions about Standards-Based Education Reform and the ways in which teachers will respond in order to align and improve their instruction. Those assumptions include: ● Assumption One: Teachers will pay attention to the reform and become familiar with the standards and what they imply for practice. ● Assumption Two: Teachers will take the reform seriously, as will their supervisors and other local leaders, who will exhort teachers to meet the demands of the policy, and offer support, as needed. ● Assumption Three: Teachers will adjust their instruction to align with the standards and associated assessments, including preparation for assessment. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 23 ● Assumption Four: Teachers will expect all of their students to succeed—and believe that they are capable of succeeding. ● Assumption Five: Teachers will have access to appropriate professional learning opportunities. (Loeb et al., 2008, p.4) Teacher Professional Development The time that is needed for teachers to be adequately trained on the implementation of newly adopted standards is substantial. To be most effective, this professional development should be provided in an ongoing manner so that teachers can engage in new learning frequently, allowing them to have regular opportunities to collaborate with their peers. This collaboration should include rich discussion around designing instructional strategies that engage students and help them achieve mastery of new concepts and skills brought forth from new standards, and a critical review of instructional materials to ensure that they are aligned to the standards and will effectively support the implemented curriculum. This can become a daunting task, as it may require the adjustment of teacher schedules to ensure that sufficient time is dedicated towards providing them with meaningful professional development. There must be a clear delineation between understanding the impact of Standards-Based Education Reform and the intended curriculum on student achievement, the impact of teacher instruction and the implemented curriculum on student achievement, and the impact of the instructional materials used by teachers as a part of the implemented curriculum on student achievement. We have evidence that shows a clear connection between teacher professional development for instructional strategies IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 24 and an increase in student achievement. As Chingos and Whitehurst (2012) show, there is also evidence that shows that the instructional materials utilized by teachers as a part of the implemented curriculum have a large impact on student achievement and the attained curriculum. To drive this point home, they state: Student learning occurs primarily through interactions with people (teachers and peers) and instructional materials (textbooks, workbooks, instructional software, web-based content, homework, projects, quizzes, and tests). The contexts within which these interactions occur are surely important. The effectiveness of teachers, the behavior of peers, and the instructional materials with which students have the opportunity to interact are affected by layers of influences that range from circumstances in the home, to leadership in the school, to the international macro-economy and everything in between. But students do not engage in instructional interactions with the governance mechanism for their school or with state standards or with a school district’s evaluation system for teachers or with collective bargaining agreements or with the leadership qualities of their school principal - they learn by engaging in cognitive processes that are triggered and shaped by interactions with people and instructional materials. (Chingos & Whitehurst, 2012, p. 3) Process for Adopting and Implementing New Standards in PA In the Commonwealth of Pennsylvania, there is a formalized process for adopting and implementing new standards. Once the standards’ language has been approved by the Pennsylvania State Board of Education, the Pennsylvania Department of Education IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 25 (PDE), in collaboration with their 29 intermediate unit partners, will work together to present the new standards language to educators, and provide training for classroom implementation. This process ensures that all administrators and educators are aware of the new standards, the manner in which they will be assessed, and are well-equipped to provide high-quality instruction that addresses them within their classrooms. As new standards are adopted in the Commonwealth of Pennsylvania, there is a direct impact on the ways in which districts, school buildings and individual educators are held accountable for their implementation and student’s understanding and mastery of them. This occurs through a combination of educator evaluation tools, student assessment tools, and school progress tools. Educators are required to address the implementation of the newly adopted standards within the classrooms as the adoption is directly tied to their individual annual evaluations. As noted by Kaufman et al. (2021), “ By themselves, state academic standards do not lead to major shifts in what teachers do in their classrooms.” In order to adequately support teachers as they adjust their instruction in order to meet new standards, teachers need access to high-quality, standards-aligned instructional materials as well as engaging, job-embedded professional development that helps them to understand the most effective and efficient ways in which to use those materials in their classrooms in order to help all students achieve academic success. “This means that states and districts must continue working to push use of standards-aligned materials, along with curriculum-focused professional development that can improve use of those materials and learning outcomes” (Kaufman et al., 2021). Without this training, teachers will struggle to accurately address IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 26 the standards in a comprehensive manner as they select instructional materials for their implemented curriculum. A Mindset Shift in Mathematics The overarching disconnect between the intended curriculum, the implemented curriculum, and the attained curriculum comes down to the fact that instructional materials are not as flexible or as well-aligned to standards as teachers need them to be in order to ensure that students master the concepts and skills outlined within their adopted standards. It is at this point that we need to consider the goal of Standards-Based Education Reform, and teachers’ responses to it. A central tenet of Standards-Based Education Reform is that all students are capable of achieving mastery in difficult content areas, and that rigorous standards should be adopted in order to ensure that students do so. This is a relatively recent shift in mindset. Traditionally, more advanced content was taught only to students that needed to engage with that content, going all the way back to the Ancient Egyptian and Babylonian Eras. Katz (2009) points to this clearly with regards to Algebra instruction, “In fact, throughout most of recorded history, algebra was only taught to the “few”, not the “many”. So the concept of “algebra for all” is a very recent one and one that will take a lot of work to make into a reality.” While each ESEA reauthorization has attempted to ensure that instruction and learning would be equitable, the statewide adoption of common standards has decreased IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 27 opportunities for personalized instruction designed to meet the needs of students as teachers have felt the pressures of increased assessments for accountability purposes. While the implementation of common standards in education was to ensure that all students receive the same education regardless of where they live through the intended curriculum, their use within the implemented curriculum has decreased instances of personalized instruction tailored to students’ specific needs. This shift away from personalized instruction is not a new phenomenon in the history of education in the United States. It reflects a long-standing tension between the ideals of standardization and the need for culturally responsive teaching practices that recognize and value the diverse backgrounds and experiences of students. History of Culturally Responsive Teaching in the United States The roots of Culturally Responsive Teaching in the United States can be traced back to the 1970s, before the publication of A Nation At Risk. Also referred to as culturally appropriate, culturally relevant, culturally congruent, or culturally compatible teaching, this pedagogy is designed to be highly responsive to the backgrounds and identities of the individuals within a given classroom in order to tailor instructional materials, examples, and scenarios so that students can personalize them to their own life experiences. The most widely accepted formal definition of Culturally Responsive Teaching, defined by Gay (2010, p. 31) is “using the cultural knowledge, prior experiences, frames of reference and performance styles of ethnically diverse students to make learning encounters more relevant to and effective for them.” IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 28 As teachers have struggled to ensure that students meet mandated achievement benchmarks due to Standards-Based Education Reform, the gap between the intended curriculum and the attained curriculum has become more prevalent. In response, teachers have begun to recognize that students need instructional materials and instructional experiences that are reflective of their personal, cultural experiences for the implemented curriculum to be more effective in meeting their learning needs (Harmon, 2012). Engaging in Culturally Responsive Teaching has become an effective method for ensuring that students not only understand the concepts and skills being taught in the classroom, it helps to ensure that students are able to relate to those concepts and skills by introducing them to students through culturally familiar scenarios and explanations that allow them to engage in meaningful learning (Harmon, 2012). “It is a means for improving achievement by teaching diverse students through their own cultural filters” (Gay, 2010, p. 50). This pedagogical approach becomes a transformative experience for students and teachers by empowering them to engage in a personalized instructional experience during the implemented curriculum. Culturally Responsive Teaching in Mathematics Most educators are quick to see the connection between Culturally Responsive Teaching and literacy-centric subjects, such as English Language Arts. However Culturally Responsive Teaching is effective in all content areas, as demonstrated by Matthews and López (2019), who sought to reinforce this point as they centered their work on the importance of Culturally Responsive Teaching in mathematics. They found that initially most teachers did not recognize the importance of Culturally Responsive Teaching on academic goals, favoring the pedagogy to reinforce socio-engagement goals IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 29 for students. However, as teachers engaged more fully in Culturally Responsive Teaching within mathematics classrooms they noticed increased levels of student achievement among Latino students (Matthews & López, 2019). The emphasis on designing learning experiences that are reflective of students’ lived experiences provide them with additional conceptual resources that allow them to make stronger connections to the implemented curriculum, thus increasing the amount of attained curriculum that they experience. While mathematics is often viewed as a culturally neutral subject, it is deeply rooted in culture and history, making the need for culturally responsive teaching in mathematics crucial to creating inclusive and equitable learning environments for all students. By incorporating culturally relevant examples and activities into math instruction, teachers can help students see the relevance and applicability of math in their own lives and communities. Culturally responsive teaching in math can also help address equity gaps by ensuring that all students have access to rigorous and meaningful math instruction that is culturally relevant and responsive to their needs. Comparing the current demographics of our teaching force to the demographics of our student populations further highlights the need to engage in Culturally Responsive Teaching. The need for culturally responsive teaching is more pressing than ever before, especially when you consider the deep demographic gaps between teachers and students. A teaching workforce that remains overwhelmingly female, white, middle-class, and monolingual is increasingly likely to teach students who are of IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 30 a different race, ethnicity, socio-economic status, language group, and so on. (Muñiz, 2019, p. 6) This disparity between those delivering instruction and those receiving instruction has not gone unnoticed by policymakers. Culturally Responsive Teaching in Pennsylvania Many states are now recognizing the importance of providing teachers with access to ongoing, high-quality, and job-embedded professional development that will assist them in developing culturally responsive instructional experiences for students. In Pennsylvania, the Commonwealth uses the term Culturally Relevant and Sustaining Education (CR-SE) to refer to a set of agreed-upon standards and competencies that all Pennsylvania educators are accountable for when working with students (Cole-Malott et al., 2021). These competencies, built upon the 50-state survey of culturally responsive standards completed by New America, ask educators to reflect upon their own cultural lens, deepen their understanding of bias within the system as a whole, design educational experiences that are reflective of students’ lived cultural experiences, and provide all students with equitable and differentiated learning opportunities. Each competency is accompanied by a set of “Questions to Stimulate Reflexivity,” which educators can use to critically examine their current practices and approaches in order to identify areas in which they can improve their practices. These questions help educators to ensure that their instruction and instructional practices “connect academics to their learners' daily lives, cultural backgrounds, challenges and build upon the funds of knowledge of learners and their families” (Cole-Malott et al., IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 31 2021, p. 2). They are also designed to ask educators to critically examine the resources that they use to assist in their instructional delivery so that students see positive and accurate representations of themselves. In evaluating instruction and instructional resources through a CR-SE lens, Pennsylvania educators have a unique opportunity to help bridge the gap between the intended curriculum, the implemented curriculum and the attained curriculum for their students. In reviewing the standards of all 50 states, Muñiz (2019) observed that, while most state standards generally include some form of Culturally Responsive Teaching within their language, there are varying levels of depth and nuance to the language. Figure 3 shows the ways in which the New America Culturally Responsive Competencies are addressed within all 50 states: Figure 3 What Culturally Responsive Competencies do State Teaching Standards Address? Note: Culturally responsive competencies addressed in state teaching standards. (Muñiz, 2019) IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 32 The Impact of Reform on Culturally Responsive Teaching Abdulrahim and Orosco (2020) note that historically, the focus on mathematics instruction has not been to make a strong connection between students’ lived experiences and math activities in order to promote math achievement. They feel that this is due to the fact that the standardization of the curriculum has created an instructional environment that is more reflective of the teacher and less reflective of the individual learners. Abdulrahim and Orosco (2020, p. 2) state, Despite reform agendas calling for the improvement of mathematics education for all students, traditional approaches to instruction in favor of basic skills instruction continue to dominate practice (Ellis and Berry, 2005). The majority of mathematics education in the U.S. is based on standardized curricula, pedagogy, and testing that is not normed on CLD (Culturally Linguistic or Diverse) students (Bartell et al., 2017; Kress, 2005). This one-size-fits-all approach separates students’ cultural strengths and experiences from teaching and learning mathematics, which influences their response to mathematics instruction (Leonard, 2008). To assist in moving away from this standardized approach to instruction, Brown and Crippen (2016) detail the utilization of the Growing Awareness Inventory (GAIn) with preservice teachers. Designed as a structured observation protocol to assist in the development of Culturally Responsive Teaching, the GAIn helps educators to build their capacity to deliver Culturally Responsive Teaching by IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 33 engaging in continuous, reflective review of their instructional materials and practices. Figure 4 shows the GAIn review process. Figure 4 Growing Awareness Inventory Theory of Action Note: The Theory of Action cycle of implementation from GAIn (Brown & Crippen, 2016, p. 136) The GAIn is just one tool that can be utilized by educators to help them implement Culturally Responsive Teaching within their implemented curriculum so that their lessons and materials are reflective of the individuals that comprise the community in which they teach. As each community is unique in its structure, demographics, and culture (Smith & Ayers, 2006), so too should be a teacher’s approach to the implemented curriculum for that community. In order for a teacher to be able to provide instruction and utilize instructional materials that are reflective of their student population, they must engage in reflective practices in order to understand the impact that their instruction and instructional materials are having on student understanding and achievement. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 34 Culturally Responsive Teaching as a Pedagogical Practice Howard and Rodriguez-Minkoff (2017) caution that Culturally Responsive Teaching is not just a simple way of teaching or utilizing out-of-the-box units of instructional materials. Rather, it is a pedagogical practice that is rooted in deeply understanding the lived experiences of the students in a teacher’s classroom and actively designing instruction and instructional materials that reflect those experiences. In order to do so, Culturally Responsive Teaching must become a pedagogical practice that is taught extensively to both pre-service and in-service teachers. In order to ensure that all teachers are familiar with and actively include CR-SE as a core component of their implemented curriculum, the Pennsylvania State Board of Education has amended Chapter 49 of the Public School Code to include the mandatory inclusion of CR-SE training as a requirement for pre-service teachers, as a component of new teacher induction programs, and as a part of districts’ continuing education plans. This guidance, released in April 2022, states, The existing requirements of Chapter 49 further require that professional education plans must address training in meeting the needs of diverse learners (defined as students with limited English language proficiency or students with disabilities), improving language and literacy acquisition, and closing the achievement gap among students. (Molchanow, 2022, p. 117) In order to do so, in-service educators are required to engage in continuing professional education courses regarding CR-SE. The scope and depth of this training has not been IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 35 defined, nor have the strategies for resource evaluation to support Culturally Responsive Teaching within the implemented curriculum inside individual classrooms. Despite the growing awareness of the importance of culturally responsive teaching, the scope and depth of training for educators in this area have not been clearly defined. This lack of clarity can create challenges for teachers who want to implement culturally responsive teaching practices in their classrooms. One possible solution to this challenge is the use of open educational resources (OERs) that focus on culturally responsive teaching. OERs can provide teachers with a variety of resources, such as lesson plans, activities, and assessments, that have been designed and curated to support culturally responsive teaching. OERs can also help teachers access a wider range of perspectives and ideas, and allow for customization and adaptation to meet the needs of their specific students and communities. Implementing OER to Facilitate Culturally Responsive Teaching In order to support students as they address Culturally Responsive Teaching within their implemented curriculum, many teachers have begun to utilize Open Educational Resources (OER) as their preferred instructional materials. The term, first coined by UNESCO at their 2002 Forum on Open Courseware, is defined as, “learning, teaching and research materials in any format and medium that reside in the public domain or are under copyright that have been released under an open license, that permit no-cost access, re-use, re-purpose, adaptation and redistribution by others” (UNESCO, 2002). Due to their ability to be adapted, teachers are able to easily personalize these materials to better reflect the lived experiences of their students, making them more relevant and engaging. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 36 Perceived Benefits of Using Open Educational Resources When asked about the benefits of utilizing OER, teachers discussed many positive aspects of their use (Tang, 2020). From feeling empowered as instructional experts who had the capacity to deliver personalized instruction that was tailored to the specific needs of individual student populations, to being able to have access to a larger repository of materials with no impact to their classroom or district budgets, OER have become a powerful asset to teachers looking to increase student achievement through Culturally Responsive Teaching. Many see the implementation of OER as a transformative element of their instruction. Teachers now commonly utilize OER to create a culturally responsive classroom. Garth and Sebastian (2020) highlight the increased flexibility and options that teachers now have by using OER by stating: With an ever-expanding library of digital content at their disposal - including online publications, instruction materials, and institutional peer-reviewed assets instructors have countless pathways to embrace the entire community of learners by designing culturally relevant classrooms that invite and incorporate every student into the process. (Garth & Sebastian, 2020, para. 8) Furthermore, there is now evidence that student performance has remained unchanged or increased through the use of OER in the classroom (Tang et al., 2020). Multiple studies from 2014 - 2019 show that the use of OER in K-12 classrooms has allowed students of varying cultural backgrounds to maintain or increase their achievement scores. Moreover, these studies also showed that students were more IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 37 motivated and engaged in their learning when they interacted with OER (Tang et al., 2020). Perceived Barriers to Increasing Student Achievement While many teachers feel empowered to use OER as a component of their implemented curriculum, most teachers report multiple barriers to the effective use of OER in the classroom. These barriers, whether real or perceived, tend to outweigh the reported benefits of OER on student achievement. Unfortunately, they prevent teachers from engaging with instructional materials that may be able to be designed to be more culturally reflective of individual students and increase student achievement (Tang, 2020). The first identified barrier is related to adequate professional development regarding the understanding of and intricacies surrounding OER use in the classroom. Many teachers report that they only have a surface-level understanding of OER. Teachers are uncomfortable in actively engaging in the work that is related to finding, reviewing, and modifying OER to use in their classrooms (Tang, 2020). The lack of professional development provided to teachers in the implementation of OER also results in a lack of time to engage in a thorough review of the resources that are found. As stated earlier in this review, teachers need consistent opportunities to engage in professional development that helps them to understand the most effective and efficient ways in which to review and use instructional materials in their classrooms in order to increase the academic achievement of students. In order for OER use to become a sustainable component of Culturally Responsive Teaching designed to increase student IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 38 achievement in the implemented curriculum, districts must provide adequate time for professional development that is designed to assist teachers in learning the process of efficiently finding OER that relate to their students’ needs, critically reviewing those resources for rigor and quality, and strategically aligning those resources to the intended curriculum (Kaufman et al., 2021). Understanding the Impact of COVID-19 on Student Achievement As conversations regarding student achievement, Standards-Based Education Reform, and Culturally Responsive Teaching continue today, the discussion will naturally turn towards the impact of COVID-19. The National Center for Education Statistics (NCES) administers the National Assessment of Educational Progress (NAEP) every two years. As stated on their website, The National Assessment of Educational Progress (NAEP), first administered in 1969, is the largest continuing and nationally representative assessment of what our nation’s students know and can do in subjects such as mathematics, reading, science, and writing. Standard administration practices are implemented to provide a common measure of student achievement. Teachers, principals, parents, policymakers, and researchers all use NAEP results to assess progress and develop ways to improve education in the United States. The results of NAEP are released as The Nation’s Report Card, and are available for the nation, states, and in some cases, urban districts. NAEP is a congressionally mandated project administered by the National Center for Education Statistics (NCES), within the U.S. Department of Education and the Institute of Education Sciences (IES). (National Assessment of Educational Progress, 2022, para. 1) IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 39 Due to the COVID-19 pandemic, the NAEP was not administered in 2020. Because of this, the results of the 2022 NAEP administration were the first to show the impact of COVID-19 on student achievement in the United States. The results showed the “​largest score declines in NAEP mathematics at grades 4 and 8 since initial assessments in 1990” (National Assessment of Educational Progress, 2022). In comparing scores to 2019, student scores in grade 4 dropped an average of 5 points and student scores in grade 8 dropped an average of 8 points. A sample of the headlines that resulted from the report included: ● “Two Decades of Progress, Nearly Gone: National Math, Reading Scores Hit Historic Lows” (Sparks & Tomko, 2022) ● “Nation’s report card: Massive drop in math scores, slide in reading linked to COVID disruption” (Petrin, 2022) ● “NAEP ‘Nation’s Report Card’ Shows Steep Fall in Math Scores” (Mollenkamp, 2022) ● “Nation's Report Card shows how badly we failed kids during pandemic. Now what do we do?” (Jacques, 2022) The alarmist language that was so unique to A Nation At Risk appears to be used as common language to discuss student achievement scores in the wake of COVID-19. However, it is important to note that while the gap in student achievement has become larger, due in part to school closures and interruptions to consistent approaches to instructional delivery, there is compelling evidence that student achievement scores had been persistent well before the onset of the pandemic. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 40 While the pandemic has certainly exacerbated existing inequities and highlighted the deep-seated disparities in access to education and resources, the existence of the achievement gap is not a new phenomenon. The gap in academic achievement between students from different racial, ethnic, and socioeconomic backgrounds has persisted over time, even when controlling for factors such as parental education and income. The COVID-19 pandemic has certainly magnified the challenges faced by students and schools, but it is not the root cause of the achievement gap. Rather, it is a symptom of the systemic and structural inequities that have long plagued the US education system. The US academic-achievement gap was first identified in 1966. Its persistence is troubling. The possibility that COVID-19 could make it worse deserves focused attention. The achievement gap costs the United States hundreds of billions of dollars - and also exacts a long-term cost in social cohesion. This is a moment and a challenge - that calls for urgency and energy. (Dorn et al., 2020) This quote by Dorn et al. (2020) highlights the troubling persistence of the achievement gap in the US education system, and recognized the fact that the COVID-19 pandemic had the potential to exacerbate this long-standing issue. As previously mentioned, the achievement gap has been identified as early as 1966 and has persisted despite various efforts to address it. Dorn et al. (2020) take their research one step further by also addressing the cost associated with addressing the achievement gap, suggesting that it costs the US hundreds of billions of dollars and exacts a long-term cost in our ability to develop the skills needed to strengthen our communities. To address the achievement gap effectively, it is essential for us to examine student achievement scores by different student groups. Such analysis can reveal IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 41 disparities in performance and highlight the need for targeted interventions to support historically marginalized students. It is also crucial to recognize the intersectionality of identities and experiences that contribute to the achievement gap. By analyzing achievement data through an equity lens and developing evidence-based interventions, educators and policymakers can work towards creating a more equitable education system that supports all students in achieving their full potential. NAEP Results by Student Group In examining the 2022 results of the NAEP, most student groups had lower average scores than they did in 2019. American Indian / Alaska Native students’ average scores dropped by 4 points, Asian students’ average scores dropped by 7 points, Asian/Pacific Islander students’ average scores dropped by 6 points, Black students’ average scores dropped by 7 points, Hispanic students’ average scores dropped by 7 points, Native Hawaiian / Other Pacific Islander students’ average scores dropped by 2 points, students of Two or More Races average scores dropped by 10 points, and White students’ average scores dropped by 7 points (National Assessment of Educational Progress, 2022). Steep Declines in Algebra In the Algebra content area, the subtopic with the steepest decline in scores is Equations and Inequities, which dropped from 35% correct in 2019 to 26% correct in 2022. While NCTM recommended the prioritizing of developing a solid conceptual foundation for skills before moving into the utilization of calculators or computers to assist with the development of more involved problem-solving skills, (NCTM, 1989) as IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 42 was mentioned earlier, these results are indicative of the gap that exists between education policy that is created in the wake of Standards-Based Education Reform and the instructional practices that are adopted by teachers in the classroom. Summary Standards-Based Education Reform has become a deeply embedded reality of the K-12 public education system in the United States for the past 40 years. As we look at the history of Standards-Based Education Reform, there has been a clear response from both the publishers of instructional content and the consumers of instructional content in an effort to increase student achievement in order to meet reform expectations. Instructional content publishers have gone to great lengths to market their materials as being aligned to national and state content standards. Their documentation that details this alignment has become a trusted source of information that many education decision makers use to determine which materials to purchase for their students. In the same vein, administrators and teachers have gone to great lengths to understand new content standards and synthesize them into concepts and skills that can be taught in classrooms. They spend countless hours working to determine the best ways to present content to students from varied cultural backgrounds, and with varied lived experiences so that they can all meet standardized proficiency benchmarks. While this standardization makes monitoring student achievement easy to quantify, it highlights gaps that exist between the intended curriculum that is set forth by national or state standards and a district’s established curriculum, the implemented curriculum that is taught within individual teachers’ classrooms, and the attained IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 43 curriculum that is evident through student achievement scores. In order to bridge this gap between policy, practice, and performance, teachers look to pedagogical techniques such as Culturally Responsive Teaching and more flexible instructional materials such as Open Educational Resources to support the individual needs of their students. The inclusion of Culturally Responsive Teaching pedagogy is a critical element in all content areas, and not just those with a strong literature-based focus, such as English Language Arts and Social Studies. Teachers are looking at the ways in which they can develop culturally responsive instructional experiences for students in order to help them increase their understanding and mastery of skills and concepts in content areas such as Algebra I. In Westmoreland County, educators and administrators are curious to understand whether or not there is a correlation between instructional materials, Culturally Responsive Teaching pedagogy, and increases in student achievement. Due to the fact that Algebra I test scores have been historically low countywide, districts are looking at the ways in which their intended curriculum and implemented curriculum are reflecting their student populations in order to determine whether or not their Algebra I instructional materials and strategies are culturally responsive to their student populations and whether or not there is an impact on student achievement. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 44 Chapter III Methodology There is a need for school districts to separate their curriculum discussions into defined categories to understand better how standards, instructional materials, and the scope and sequence of instruction impact student achievement scores. In doing so, districts can engage in a more comprehensive analysis of the impact of their intended curriculum, implemented curriculum, and attained curriculum. When school districts take the time to understand the connection between state-mandated academic standards, the alignment and relevance of instructional materials, and the scope and sequence of the instruction that the teacher provides, they can better adjust their efforts to ensure that learning experiences are designed to be culturally responsive and personalized to the individual student populations of each classroom. The literature shows us that Standards-Based Education Reform was adopted in the early 1980s to provide broad, top-down guidance to K-12 public schools in the United States to increase academic rigor and develop a structured curriculum. While Standards-Based Education Reform was designed to impact all content areas, mathematics was a particular concern. Organizations such as the National Council of Teachers of Mathematics provided detailed recommendations to overhaul K-12 mathematics education in the United States, including the adoption of a common, core curriculum that would be utilized in all schools to ensure that all students would be introduced to the same skills and concepts in mathematics. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 45 Another critical recommendation was to focus on equity in education and not just on education reform. Researchers noted the importance of honoring the diverse populations of K-12 public school districts by developing diverse learning experiences. When educators create learning experiences that students feel reflect their unique cultures and backgrounds, they can quickly connect to the content. This connection can result in a deeper understanding and mastery of the concepts and skills presented within the implemented curriculum and increase the student’s attained curriculum. Purpose of Study This study investigated the possible correlation between the historically low Keystone Algebra I Scores and building principal and Algebra I teachers’ perceptions of student identification with examples in the word problems found within their instructional materials. To thoroughly investigate this topic, the researcher used a mixed-methods embedded design in the form of a case study of three school districts that represent Westmoreland County public schools. The following research questions guided this case study: 1. What are the factors that impact student achievement in Algebra I? 2. What are the factors that impact instructional delivery in Algebra I? 3. What are the teacher and administrator perceptions of the impact of instructional delivery on student achievement scores in Algebra I? Qualitative data has been generated from district curriculum documents, standard alignment maps, lesson plans, and instructional material reviews and policies to determine how academic standards are addressed within the curriculum and addressed through teacher instruction. Quantitative data has been generated from student IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 46 assessment scores, district budgets, and survey responses regarding instructional materials to determine the amount spent on them, their alignment with Pennsylvania Core Standard, and their cultural relevancy. The desired outcome of this case study is that the data will show whether or not the instructional resources currently being purchased and utilized within the classroom effectively support student academic understanding and achievement. The results of this case study will provide building administrators and Algebra I teachers with additional guidance and support when determining which instructional materials may significantly impact student achievement within their student populations. Setting The Pennsylvania General Assembly established intermediate units in 1970. As the report “Intermediate Units in Pennsylvania: The Role of Educational Service Agencies in Promoting Equity in Basic Education” states: In 1970, the General Assembly passed Act 102, creating a system of 29 intermediate units (Us), which replaced the county superintendent offices as of July 1, 1971. The lUs were mandated to develop a broad program of educational services to be offered to public and nonpublic schools, including curriculum development and instructional improvement services; educational planning services; instructional materials services; continuing professional education services; pupil personnel services; State and federal agency liaison services; and management services. The lUs were created as instruments of federal, State, and local education policies. They were empowered to create new services needed by IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 47 public and nonpublic schools. (Pennsylvania Joint State Government Commission, 1997). Each of the Commonwealth’s 29 intermediate units is now charged with providing cost-effective and management-efficient programs designed to support school districts, charter schools, and nonpublic schools and serves as the liaison between these organizations and the Pennsylvania Department of Education (PDE). Intermediate units have four significant areas of focus as service agencies. The first is Advancing Professionals through high-quality training and professional development. Through this area of focus, intermediate units work with over 52,000 educators each year in data analysis, curriculum development, technology integration, school safety, and educator certifications (Pennsylvania Association of Intermediate Units [PAIU], 2022). The second area of Safe and Healthy Students is an area of focus that supports schools in addressing individual students' behavioral, emotional, and mental health needs. These supports include services such as partial hospitalization programs, alternative education programs, student assistance programs, and student evaluations. These supports also extend to implementing services designed to prevent school threats, such as creating Crisis Response Teams (PAIU, 2022). The third area of Serving Students is broken into two categories: Early Childhood and Young Children and School Age Students. Through this service area, intermediate units can provide educational opportunities to students from birth to 18 years of age. These services include Early Intervention programs, online education programs, and special education services such as speech therapy, occupational therapy, and behavior intervention (PAIU, 2022). The fourth area of focus for intermediate units is Supporting IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 48 Schools. This service helps create purchasing consortiums so districts can take advantage of lower pricing for health insurance, energy purchases, transportation, and administrative services (PAIU, 2022). Each of these focus areas allows intermediate units to be responsive to the needs of the member districts within their geographic region. The Westmoreland Intermediate Unit is in Greensburg, Pennsylvania. Situated in southwestern Pennsylvania, the WIU serves seventeen public schools, three career and technology centers, one charter school, and forty-two nonpublic schools within Westmoreland County. The WIU provides Curriculum Services, Executive Services, Financial Services, Student Services, and Technology and Infrastructure Services to their member schools. These services are delivered both in-person and online and are designed to help districts to build their capacity to support their communities. “The Curriculum Services Division's primary goal is to advance the practice of teaching and learning to the schools and students we serve. Through innovative practice, we strive to be the premier leading experts in curriculum services, operating as an integral part of our local, regional, and state education community” (Westmoreland Intermediate Unit, 2023). The Curriculum Services Division comprises a team of one director, three supervisors, one support specialist, and one secretary. While this team is small, its capacity to support districts continues to grow through its increased involvement in local, state, and national educational initiatives. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 49 Participants Three public school districts participated in this case study. The first is the Greater Latrobe School District. Founded in 1960, the Greater Latrobe School District is a midsized district in Westmoreland County. As stated on their district website, The Greater Latrobe School District is in southwestern Pennsylvania at the foothills of the Laurel Highlands. It is approximately 50 miles east of downtown Pittsburgh and 30 miles southwest of Johnstown, Pennsylvania. U.S. Route 30 bisects much of the district, which encompasses the City of Latrobe, Youngstown Borough, and Unity Township, with approximately 30,000 residents. The Greater Latrobe area has diversified industries which afford job opportunities in specialty steel, plastics, molds and casting ceramics, and brewing beverages. The local hospital, four area colleges, the airport, and specialty stores and shopping areas provide many forms of employment. (Greater Latrobe School District, 2022) As of the 2021-2022 school year, Greater Latrobe School District reported a district enrollment of 3,512 students in grades K-12. Of these students, 51.8% are male, and 48.2% are female. The district states that 2.2% of their students are enrolled in their Gifted program, 7% in Eastern Westmoreland Career and Technical Center, and 3% in outside charter or cyber charter schools. Figure 5 shows the percentage of enrollments by race/ethnicity for the 2021-2022 school year. Figure 6 shows a demographic breakdown of the district’s student enrollments for the 2021-2022 school year. (Pennsylvania Department of Education, 2022) IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Figure 5 Greater Latrobe School District Percent Enrollment by Race/Ethnicity Note: 2021 - 2022 Greater Latrobe School District student enrollments by race/ethnicity. Data retrieved from Future Ready PA Index, December 2022 Figure 6 Greater Latrobe School District Percent Enrollment by Student Groups Note: 2021 - 2022 Greater Latrobe School District student enrollments by student group. Data retrieved from Future Ready PA Index, December 2022 50 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 51 The Greater Latrobe School District carefully reviews their Algebra I Keystone Exam student achievement scores to work toward meeting Pennsylvania’s statewide established proficiency goal. The Pennsylvania Department of Education has set the statewide goal of having 71.8% of all students reach proficiency in Algebra I by 2030. Districts work to increase their proficiency percentage annually incrementally to achieve this goal. Figure 7 shows the proficiency percentages of all Greater Latrobe students taking the Algebra I Keystone Exam from 2013 - 2022. Figure 7 Greater Latrobe School District Algebra I Proficiency Percentages 2013 - 2022 Since the 2013-2014 school year, the Greater Latrobe School District’s highest percentage area has been in the Basic category. There is a correlation between the number of students in the Basic category and the overall number of students tested each year. Figure 8 shows the total number of students tested annually from 2013 - 2022. The district’s goal is to review its instructional materials and strategies to determine strategies IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 52 to increase student achievement and move students from the Basic category into the Proficient category to achieve the 2030 statewide goal of 71.8% proficiency. Figure 8 Greater Latrobe School District Total Number of Student Tested vs. Year Administered The second district participating in this case study is the Monessen City School District. The Monessen City School District is a small, urban district in Westmoreland County. Serving the City of Monessen, the district spans approximately 2 square miles in total. As stated on their district website, “Monessen, a very caring and unique school district, relies on versatility in meeting the needs of our very special young people by assisting them to become productive citizens and confident individuals in our own and world community” (Monessen City School District, 2023). As of the 2021-2022 school year, Monessen City School District reported a district enrollment of 712 students in grades K-12. Of these students, 48.7% are male, and 51.3% are female. The district states that 1.5% of their students are enrolled in their IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 53 Gifted program, 4% are enrolled in Mon Valley Career and Technical Center, and 7% are enrolled in an outside charter or cyber charter schools. Figure 9 shows the percentage of enrollments by race/ethnicity for the 2021-2022 school year. Figure 10 shows a demographic breakdown of the district’s student enrollments for the 2021-2022 school year (Pennsylvania Department of Education, 2022). Figure 9 Monessen City School District Percent Enrollment by Race/Ethnicity Note: 2021 - 2022 Monessen City School District student enrollments by race/ethnicity. Data retrieved from Future Ready PA Index, December 2022 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 54 Figure 10 Monessen City School District Percent Enrollment by Student Groups Note: 2021 - 2022 Monessen City School District student enrollments by student group. Data retrieved from Future Ready PA Index, December 2022 The Monessen City School District also carefully reviews their Algebra I Keystone Exam student achievement scores to work toward meeting Pennsylvania’s statewide established proficiency goal. Figure 11 shows the proficiency percentages of all Monessen City students taking the Algebra I Keystone Exam from 2013 - 2022. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 55 Figure 11 Monessen City School District Algebra I Proficiency Percentages 2013 - 2022 Historically, Monessen City School District has not had any students in the Advanced category on the Algebra I Keystone Exam since 2013. Instead, most of their students were in the Basic category until the Spring 2018 administration of the Keystone Exam. At this point, we see the same percentage of students in the Basic and Below Basic categories. From this point on, we see more students in the Below Basic category. Figure 12 shows the total number of students tested yearly from 2013 - 2022. The district is concerned about the increasing number of students entering the Below Basic category and is investigating the reasons for this shift. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 56 Figure 12 Monessen City School District Total Number of Student Tested vs. Year Administered The third district participating in this case study is the Yough School District. The largest of the three districts participating, the Yough School District is a rural district that spans 77 square miles and serves the Arona Borough, Madison Borough, Sewickley Township, Smithton Borough, South Huntingdon Township, Sutersville Borough, and West Newton Borough. As stated on their district website, “The Yough School District is enthusiastic about the growing local industries, which is providing unlimited opportunities for our students and families to gain meaningful careers. Employment, coupled with local community and recreation organizations, allows individuals to be actively involved, while preserving hometown pride” (Yough School District, 2020). As of the 2021-2022 school year, Yough School District reported a district enrollment of 1,805 students in grades K-12. Of these students, 50.1% are male, and 49.9% are female. The district reports that 2.4% of their students are enrolled in their Gifted program, 6.9% are enrolled in Central Westmoreland Career and Technical Center, IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 57 and 4.5% are enrolled in outside charter or cyber charter schools. Figure 13 shows the percentage of enrollments by race/ethnicity for the 2021-2022 school year. Figure 14 shows a demographic breakdown of the district’s student enrollments for the 2021-2022 school year (Pennsylvania Department of Education, 2022). Figure 13 Yough School District Percent Enrollment by Race/Ethnicity Note: 2021 - 2022 Yough School District student enrollments by race/ethnicity. Data retrieved from Future Ready PA Index, December 2022 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 58 Figure 14 Yough School District Percent Enrollment by Student Groups Note: 2021 - 2022 Yough School District student enrollments by student group. Data retrieved from Future Ready PA Index, December 2022 The Yough School District also carefully reviews their Algebra I Keystone Exam student achievement scores to work toward meeting Pennsylvania’s statewide established proficiency goal. Figure 15 shows the proficiency percentages of all Yough students taking the Algebra I Keystone Exam from 2013 - 2022. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 59 Figure 15 Yough School District Algebra I Proficiency Percentages 2013 - 2022 Like the Greater Latrobe School District, Yough School District’s highest percentage area has historically been in the Basic category since 2013. However, unlike the Greater Latrobe School District, there is no direct correlation between the number of students taking the Algebra I Keystone Exam and the percentage of students in the Basic category. Figure 16 shows the total number of students tested yearly from 2013 - 2022. The district is concerned with the percentage of students in the Basic category. They have not been able to raise their Algebra I proficiency level beyond 33.9%, which occurred in 2015. In recent years, their highest proficiency level was 28.3% in Spring 2021. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 60 Figure 16 Yough School District Total Number of Student Tested vs. Year Administered Student demographic data, achievement score data, attendance data, and disciplinary data from each of the three participating school districts were collected to investigate this topic thoroughly. In addition, teacher and building administrator perception data were contributed by teachers currently instructing Algebra I classes within the three participating school districts and their building-level principals. All participants volunteered to participate in this study, completing and submitting a participant informed consent form. Research Plan As Standards-Based Educational Reform has become more prevalent in K-12 education in the United States, public school districts have relied on content publishers to align their instructional materials to educational standards to help them increase student achievement scores, investing large amounts of time and money into effectively using IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 61 these materials in classrooms. Districts have prioritized purchasing these materials in Mathematics, which has been one of the most globally competitive content areas since the 1980s. They hope these materials will help teachers better align their implemented curriculum with the intended curriculum of the district. Unfortunately, the literature shows us that, in many cases, content publishers are investing more effort into updating their marketing materials to show that their instructional materials are aligned to content standards instead of updating and aligning the instructional materials themselves. As districts struggle to increase student achievement scores in mathematics, they are now looking to supplement the instructional materials they purchase with open educational resources (OER) that are more reflective of their student population. They are also taking the time to evaluate the instructional materials they buy from content developers to determine whether they are culturally reflective of their student population and whether there is a correlation between culturally reflective instructional materials and student achievement scores. To explore this issue in Westmoreland County, the established research plan included independent actions by the researcher and voluntary actions by participants from districts within Westmoreland County. The researcher began by investigating the Algebra I test scores in all seventeen public school districts in Westmoreland County since 2013. This data was collected from the eMetric Data Interaction for Pennsylvania Student Assessments system and was generated as a Group Summary report showing the performance results of all students that took the Algebra I Keystone Exam during each administration period from 2013 through 2022. This resulted in a total of 322 unique reports showing the test results of 64,959 students. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 62 The researcher also analyzed current demographic data for all seventeen public school districts in Westmoreland County as reported to PDE for the 2021 - 2022 school year. This data was collected from the Future Ready PA Index, a website maintained by PDE annually. As stated on PDE’s website, “The Future Ready PA Index illustrates student and school success via three color-coded categories: academic performance, student progress, and college and career readiness, providing the public with a more comprehensive look at how Pennsylvania's schools are educating students” (Pennsylvania Department of Education, 2023). To gather perception data regarding the instructional materials being used in Algebra I classrooms in Westmoreland County, building principals and Algebra I teachers in Westmoreland County were given the opportunity to participate in a perception survey. The survey for building principals (Appendix A) was 14 questions in length and covered topics such as the amount spent on instructional materials annually, administrators’ awareness of OER, and their perception of the following: ● the alignment of purchased materials to PA Core Standards ● their teachers’ awareness of OER ● internal and external barriers toward the use of OER in the classroom ● the frequency in which teachers need to modify instructional materials to meet the needs of students ● the frequency in which teachers need to develop their own instructional materials to meet the needs of students IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 63 The survey for Algebra I teachers (Appendix B) was 20 questions in length and covered topics such as teachers’ academic background, instructional environments, and their perception of the multiple items relating to their instructional materials, including: ● the alignment of their 7th and 8th grade mathematics curriculum with PA Core Standards ● the alignment of instructional materials being used with PA Core Standards ● the amount of professional development given to effectively use the instructional materials being purchased by the district ● the amount of professional development given to effectively develop their own instructional materials ● the impact of instructional materials on student achievement scores The final step in the established research plan involved working with three selected districts to complete the Culturally Responsive STEAM Scorecard (Appendix C), developed by NYU Steinhardt, in order to determine the cultural responsiveness of the primary instructional materials for Algebra I for each school district. During this step, the researcher held a virtual meeting with each district's building principal and Algebra I teachers to explain the purpose of the Culturally Responsive STEAM Scorecard and how to use it. The groups were then given time to review their primary instructional materials, evaluating them through the scorecard parameters and determining individual scores for each of the four categories: Representation, Social Justice, Teacher’s Materials, and Materials & Resources. Each district group was allowed to complete the scorecard in a group setting or individually. The researcher then created an average score for each category based on the individual scores provided. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 64 Based on the average score, the researcher was able to recommend to each district the cultural responsiveness of the primary instructional materials their district purchased for Algebra I. Once all steps were completed, the researcher looked at the historic Algebra I test scores for each of the three participating school districts, their 2021 - 2022 demographic data, and their Culturally Responsive STEAM Scorecard results for their primary instructional materials for Algebra I. This was done to determine if there is a correlation between low test scores and the use of instructional materials that do not reflect the district's student population. This research plan is designed to help districts determine their instructional materials' efficacy and impact on student achievement in Algebra I. This research plan meets the needs of the research problem because the research will investigate the possible correlation between the historically low Keystone Algebra I scores in Westmoreland County and building principal and Algebra I teachers’ perceptions of student identification with examples in the word problems found within their instructional materials in Algebra I classrooms in Westmoreland County. There are minimal costs associated with this Doctoral Capstone Research Project. The main cost associated with it is the time and effort of the researcher. This work aligns with her current work for the Westmoreland Intermediate Unit. The budget below shows an allotment of 30% of the researcher’s annual salary. Should the work result in the recommendation for districts to adjust their instructional materials, there may be a shift in district resources allotted to both instructional materials and professional development. As a result, one year of funding for the implementation of this study will be supported through the general operating budget of the Westmoreland Intermediate Unit IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 65 under the function of Instruction for Regular Programs. The funding source will allocate dollars from the Administrative Budget, as all administrators within the Westmoreland Intermediate Unit are not in this category and are not assigned to specific instructional levels. The Westmoreland Intermediate Unit is classified as a non-instructional building. Therefore all expenditures within this budget are not split by individual schools, as you see within a school district. All project costs fall under Mathematics Instruction for the Curriculum Specialist Assignment job description. To effectively analyze each district’s instructional materials for Algebra I, the researcher must dedicate time to thoroughly understanding the resources being used, educator perspectives, and student achievement data. This budget allots 30% of the researcher’s time to this specific project, as the researcher has additional duties and responsibilities that must be completed throughout the year. Budget line items include the Supervisor’s salary and benefits such as health care, insurance, retirement, unemployment, and worker’s compensation. The indirect costs associated with this project, external of the researcher’s time, include the use of a laptop and iPad, access to a Westmoreland Intermediate Unit-issued Google Account, and the cost of secure digital storage on the Westmoreland Intermediate Unit’s Amazon Web Services (AWS) server. As the researcher also utilizes all of these indirect cost items to complete the other assigned duties and responsibilities, the amount listed reflects 30% of the annual usage of each of these items. The immediate cost associated with the project on the part of the individual districts is minimal as well, resulting in approximately 2 ½ hours in salary and benefits per participant. There may be a future cost to each district. Should the project result in IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 66 providing districts with a recommendation for adjusting their instructional materials, the researcher will collaborate with each district to determine the budgetary impact of implementing the recommendation. This will help districts to project the sustainability of the recommendations made by the researcher. Research Methods & Data Collection The researcher studied this topic by using a mixed-methods embedded design in the form of a case study of three school districts that are representative of all seventeen Westmoreland County public school districts. This mixed-methods research allowed the researcher to utilize one form of data to support the other. As stated by Mertler when discussing mixed-methods research, “Its focus must always be on integration - that is, on the idea of integrating not only multiple types of data but also perhaps integrating theoretical perspectives, research design, research results, etc. Integration is the central concept in this methodological approach to conducting research.” (pg. 140) Utilizing a mixed-method research approach allowed the researcher to efficiently collect qualitative and quantitative data as she progressed through her research. Once all data was collected, she could begin looking for potential correlations between instructional materials, student achievement scores, perceptions of those resources, and reasons why teachers utilize specific resources within their classrooms. The researcher utilized qualitative data from district curriculum documents, standard alignment maps, lesson plans, and instructional material reviews and policies to determine the ways in which academic standards are addressed within the curriculum and addressed through teacher instruction. The researcher also used this qualitative data to support the quantitative data collected from student assessment scores, district budgets, IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 67 and survey responses regarding instructional materials to determine the amount being spent on instructional materials, their alignment to Pennsylvania Core Standard and their cultural relevancy and to show whether or not the resources that are currently being purchased and utilized within the classroom are effective in supporting student academic understanding and achievement. This design enhanced the study by giving the researcher the opportunity to more thoroughly explain the research and data that was collected and interpreted. It also helped the researcher to better manage her time to ensure that everything was complete within the defined timeline as established within the research plan. Data that was available via public, openly available records, such as budget documents, student achievement scores, and district policies were able to be accessed at any time once released online. Building administrator and teacher perception survey data was collected via Google Form. The data from the Culturally Responsive STEAM Scorecard was collected via Google Sheet. This ensured that each district team and the researcher had instant access to the results of their team’s scorecard. Data Collection Timeline and Plan Described The first research question, What are the factors that impact student achievement in Algebra I?, was investigated from November 2022 through February 2023. In order to thoroughly investigate this question, the researcher analyzed Algebra I Keystone Exam scores, available local assessment scores, Future Ready PA Index Information, student attendance and discipline records, and the results of the Culturally Responsive STEAM Scorecard. Each of these items provided historical data for the three school districts being studied within this case study. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 68 Keystone Exam scores and local assessment scores were obtained from the Pennsylvania Value-Added Assessment System (PVAAS) website and the eMetric website. All Future Ready PA Index information was obtained from the Future Ready PA Index website. Attendance and discipline data was collected from the individual school districts. Teachers and administrators completed the Culturally Responsive STEAM Scorecard digitally. The second research question, What are the teacher and administrator perceptions of the impact of instructional delivery on student achievement scores in Algebra I?, was investigated from November 2022 through January 2023. Due to the nature of this study, the researcher was able to investigate this question at the same time as the first research question. In order to thoroughly investigate this question, the researcher analyzed the results of the building administrator and teacher perception surveys. These two surveys helped the researcher to understand the perceptions of both teachers and administrators regarding the use of instructional materials in Algebra I classrooms, their cultural responsiveness, and their potential impact on student scores. Surveys were shared electronically via email, and results were automatically delivered to the researcher once participants completed the survey. The final research, question What are the factors that impact instructional delivery in Algebra I?, investigated from March 2023 through April 2023. In order to thoroughly investigate this research question, the researcher analyzed budget documents, available professional development plans, instructional material adoption policies, available instructional material standards alignment documentation, available district IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 69 curriculum map, and available scopes and sequences of instruction. Each of these items helped the researcher to understand the correlation between the intended curriculum, the implemented curriculum, and the attained curriculum. All documents were obtained from district websites, or were requested from the participating districts if they had been developed. The researcher submitted proposal #PW22-007 to the Institutional Review Board (IRB) at the Pennsylvania Western University before initiating the outlined research plan. The IRB granted approval effective 9/20/22 with an expiration date of 9/19/23 (Appendix D). The data collection tools utilized for this study were available online at no cost, therefore the fiscal implications of conducting the study were minimal. Participants completed perception surveys at times that were most convenient to their daily schedules, and worked on completing the Culturally Responsive STEAM Scorecard during established collaborative planning time during their workday, ensuring that districts would not accrue additional costs for participating in the case study. Validity Hendricks (2017) describes the validity of action research as the “trustworthiness” of the research. Relying heavily on the criteria established by Lincoln and Guba (1985), Hendricks (2017) reminds researchers that they must consider both the credibility and the validity of their work. The four types of validity established by Lincoln and Guba (1985) are IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 70 ● Credibility: the plausibility of the research findings for the context that was studied. ● Transferability: the extent to which results of a study are applicable to other contexts and other individuals ● Dependability: the degree to which research results would replicate with the same or similar participants and/or contexts. ● Confirmability: Showing that results are an accurate representation of what occurred rather than the results of the researcher’s bias, motivation, or interest. The researcher designed the research plan to address the credibility of the study by utilizing two specific strategies recommended by Hendricks (2017): triangulation and accurate data recording. In order to thoroughly investigate this topic, the researcher analyzed data from multiple sources over a span of 13 years. In order to ensure that data was recorded accurately, the data files were downloaded and utilized to ensure an accurate transfer of information. Transferability was addressed by providing a detailed description of the setting and the steps of the study. By using a mixed-methods embedded design, the researcher had the ability to provide a rich description of the setting, participants, interventions, and research methods employed in the study. This will allow future researchers to apply this work to other contexts. In order to address dependability, and determine the degree to which this research would replicate with the same or similar participants and/or contexts, the researcher relied upon triangulation and providing a detailed description of the setting and steps of the study. The researcher also engaged in an ongoing investigation with different IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 71 participants by selecting three representative school districts from across Westmoreland County to participate in the case study. Confirmability was also addressed within the research plan through triangulation and accurate data recording. The amount of data collected necessitated the accurate recording of all information gathered. In addition, providing a thick description of the setting and study allowed the researcher to show that the results of the study are an accurate representation of what occurred during the course of the study rather than the results of the researcher’s own bias, motivations, or interests (Hendricks 2017). Summary The researcher designed a plan to investigate the possible correlation between the historically low Algebra I test scores in Westmoreland County and the culturally responsive nature of the instructional materials currently being used in Algebra I classrooms. In order to investigate this possible correlation, the researcher analyzed data from multiple sources, including student achievement scores over a period of eight years, district demographic data as reported to PDE, district budget documents, instructional material standards alignment documentation, district curriculum maps, and scopes and sequences of instruction. In addition, perception data was collected and analyzed from budiling administrators and teachers. Finally, participants engaged in the completion of a Culturally Responsive STEAM Scorecard in order to determine the responsiveness of their Algebra I instructional materials. Chapter III provided an overview of the research problem identified within this case study. It then provided a detailed explanation of the purpose, setting, participants, IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 72 and research plan for this study. Finally, Chapter III discussed the methods of data collection and validity. Chapter IV will provide a thorough report of the data analysis that was conducted. It will also share the results of this study. Finally, Chapter IV will provide a detailed discussion of the interpretation of those results. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 73 Chapter IV Data Analysis and Results This case study aimed to determine the possible correlation between the historically low Keystone Algebra I Scores in middle school and building principal and Algebra I teachers’ perceptions of student identification with examples in the word problems found within their middle school instructional materials. The research plan for this case study was designed to determine if a correlation existed by examining the relationship between district curriculum documents, standard alignment maps, lesson plans, and instructional material reviews and policies to determine how academic standards are addressed within the curriculum and addressed through teacher instruction and comparing those documents to student assessment scores, district budgets, and survey responses regarding instructional materials to determine the amount spent on them, their alignment with Pennsylvania Core Standard, and their cultural relevancy. This chapter presents an analysis of the data collected to determine whether or not a correlation between student test scores and the perceived cultural responsiveness of instructional materials in middle school Algebra I courses exists through the framework of the identified research questions. This chapter also provides a detailed discussion of the interpretations of the data concerning the research questions for each school district identified within the case study. Research Questions 1. What are the factors that impact student achievement in Algebra I? 2. What are the factors that impact instructional delivery in Algebra I? IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 74 3. What are the teacher and administrator perceptions of the impact of instructional delivery on student achievement scores in Algebra I? Data Analysis Participants Eighteen schools are designated as middle schools, junior high schools, or intermediate middle schools within the 17 public school districts in Westmoreland County. The building administrators and Algebra I teachers within those 18 buildings were invited to participate in the perception surveys for this case study. Eleven building administrators agreed to participate in the Building Principal Perception Survey and signed the established consent form before completing the survey. All of the building administration participants remained in the study, resulting in the completion of the survey by all eleven participants. Six Algebra I teachers agreed to participate in the Teacher Perception Survey and signed the established consent form before completing the survey. All of the teacher participants remained in the study, resulting in the completion of the survey by all six participants. In addition, building administrators and Algebra I teachers from three districts agreed to complete the Culturally Responsive STEAM Scorecard to review their primary middle school Algebra I instructional materials. To access the Culturally Responsive STEAM Scorecard, building administrators and teachers first agreed to participate in the study and sign the established consent form. While all participants agreed, the teachers from one of the three districts did not return the signed consent forms and could not participate in the study. Due to this, the study only reflects the Culturally Responsive STEAM Scorecard responses for two of the three case study districts. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 75 The Building Principal Perception Survey and the Teacher Perception Surveys were administered via Google Forms. Survey responses did not collect identifying information, such as participants’ names, to ensure confidentiality. Their email addresses were also not collected upon submission. Once all responses were collected, each participant was assigned a letter of the alphabet and a corresponding number to associate with the reported data. The researcher speculates that the lack of participation by teachers is due to the current political climate in education and the connection of this case study to the phrase “culturally responsive teaching.” The researcher further speculates that external factors impacting teachers in one of the case study districts, such as the unexpected death of a student, staffing shortages, and changes in district administration, may have caused teachers to struggle to find the time to complete the Culturally Responsive STEAM Scorecard. Data Analysis Procedure The research aimed to investigate the correlation between historically low Keystone Algebra I Scores in Westmoreland County middle schools, principals' and teachers' perceptions of instructional materials alignment and efficacy, training and support provided to teachers, and the cultural responsiveness of the instructional materials being used in middle school Algebra I classrooms. Quantitative data included exam scores, assessment data, budget information, and standards alignment from three school districts. Data sources included eMetric, PVAAS, PDE, the Future Ready PA Index, district and content vendor websites. Qualitative data included surveys and the IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 76 Culturally Responsive STEAM Scorecard. The data were analyzed to determine if a correlational relationship existed. The quantitative data that was gathered for this case study was compiled by the researcher from multiple sources. This data includes overall student Algebra I Keystone Exam scores for all 17 public school districts in Westmoreland County from 2012 - 2022, countywide Algebra I Anchor Analysis based on student achievement scores from 2015 2021, local assessment data for Algebra I from seven of the 17 public school districts in Westmoreland County from 2019 - 2022, Future Ready PA Index data for all public school districts in Westmoreland County from 2001 - 2022, annual budget information for all 17 public school districts in Westmoreland County from 2016 - 2023, Comprehensive Plans for all 17 public school districts in Westmoreland County from 2016 - 2026, and Pennsylvania Common Core Standards alignment information for the primary Algebra I instructional resources used by Greater Latrobe School District, Monessen City School District, and Yough School District. This data was gathered from the eMetric Data Interaction for Pennsylvania Student Assessments web-based system (eMetric), the web-based Pennsylvania Value Added Assessment System (PVAAS), the Pennsylvania Department of Education’s website (PDE), PDE’s web-based Future Ready PA Index, individual district websites, and individual content vendor websites. The quantitative data gathered for this case study was then organized by the researcher for analysis according to the research plan outlined in Chapter III. This allowed the researcher to discover patterns and trends in student achievement scores over time, determine funds allocated to instructional materials, and determine whether the IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 77 materials used by Greater Latrobe School District, Monessen City School District, and Yough School District are aligned with Pennsylvania Common Core Standards. The eleven building administrators voluntarily submitted the first qualitative data gathered for this case study through a Google Form. The six teachers voluntarily submitted the second set of qualitative data for this case study through a Google Form. This format helped the researcher to compile the data to facilitate analysis according to the research plan outlined in Chapter III. Participant responses were used to support the researcher's efforts to determine a potential correlation between instructional materials, Culturally Responsive Teaching pedagogy, and increases in student achievement. First, the researcher gathered the total number of students tested for each testing window from the 2012 - 2013 school year through the 2021 - 2022 school year. Then the researcher determined the percentage of students with scores in the Below Basic, Basic, Average, and Proficient ranges. From there, the scores were separated into categories by Algebra I Assessment Anchors separated the scores into categories to determine student achievement within each Anchor. The change in these percentages by district is shown below. Figure 17 shows the percentage of each district’s scores in each performance category in the winter of 2012. Figure 18 shows the percentage of each district’s scores in each performance category in the spring of 2022. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Figure 17 Group Summary: Winter 2012 Performance Keystone, Algebra I 78 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 79 Figure 18 Group Summary: Spring 2022 Performance Keystone, Algebra I The researcher next gathered demographic data from all 17 public school districts in Westmoreland County for the 2021 - 2022 school year, as reported to PDE. This demographic data allowed the researcher to separate each district’s total student enrollments by race/ethnicity, as shown in Table 1 The researcher was also able to separate each district’s enrollments by the following subgroups: ● Economically Disadvantaged, as shown in Figure 19 ● English Language Learners (ELL), as shown in Figure 20 ● Special Education, as shown in Figure 21 ● Gender (male/female), as shown in Figure 22 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 80 Table 1 All Districts’ Total Student Enrollments by Race/Ethnicity District American Indian / Asian Alaska Native Black Native Hawaiian Two or / other Hispanic White More Pacific Races Islander Belle Vernon Area SD 0% 1.3% 3.2% 0% 2.9% 87.3% 5.2% Burrell SD 0% 0.5% 1.9% 0.1% 1.5% 91.7% 4.4% Derry Area SD 0.1% 0.5% 1.7% 0% 0.6% 93.6% 3.5% Franklin Regional SD 0.1% 7.8% 1.8% 0.1% 2.5% 83.7% 4.1% Greater Latrobe SD 0.2% 0.9% 1% 0% 1.8% 9% 3.6% 0% 0.3% 6.3% 0.2% 3.2% 80.3% 9.7% Hempfield Area SD 0.1% 1.1% 1.2% 0.1% 1.8% 91.5% 4.3% Jeannette City SD 0.7% 0% 14.2% 0% 3.1% 62.4% 19.6% Kiski Area SD 0.1% 0.2% 6.3% 0% 1.5% 90.7% 1.1% Ligonier Valley SD 0.1% 0.1% 0.6% 0.1% 1.4% 95.4% 2.3% Monessen City SD 0% 0.7% 37.5% 0% 3.5% 37.2% 21.1% Mount Pleasant Area SD 0% 0.1% 0.7% 0.1% 0.6% 95% 3.5% New Kensington-Arnold SD 0.3% 0.4% 32.9% 0% 3.6% 43.7% 19.2% Norwin SD 0.1% 1.5% 1% 0% 2% 91.4% 4% 0% 1.1% 1.8% 0.2% 1.1% 92.7% 3.1% 0.1% 0.4% 0.7% 0% 0.9% 94.3% 3.7% 0% 0.1% 1.4% 0% 0.9% 93.3% 4.3% Greensburg Salem SD Penn-Trafford SD Southmoreland SD Yough SD IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Figure 19 % Enrollments Economically Disadvantaged by District 81 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Figure 20 % Enrollments ELL by District 82 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Figure 21 % Enrollments Special Ed. by District 83 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 84 Figure 22 % Enrollments by Gender (M/F) by District Next, the first group of participants submitted their qualitative data by accessing the Building Principal Perception Survey via Google Forms. This allowed the researcher to quickly compile and organize the data collected by the participants to determine their perception of the instructional materials used in the Algebra I classrooms in their buildings. Data was collected from this survey by utilizing Open-Ended Questions, Intervals Scales, and Ordinal Scales. The questions from the Building Principal Perception Survey are shown in Table 2 and in Appendix A. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 85 Table 2 Building Principal Perception Survey Questions Building Principal Perception Survey Questions To what degree are you aware of Open Educational Resources (OER)? To what degree are teachers aware of Open Educational Resources (OER)? How many administrators in your building feel adequately informed about OER? How many teachers in your building feel adequately informed about OER? What are the internal factors that cause teachers to not use OER? What are the external factors that cause teachers to not use OER? How regularly do teachers modify instructional resources to better meet their students’ needs? How regularly do teachers develop their own original instructional resources to better meet their students’ needs? What percentage of purchased instructional materials are accurately aligned to Pennsylvania Core Academic Standards? How often do teachers evaluate the efficacy of their purchased instructional materials? How often do administrators evaluate the efficacy of their purchased instructional materials? How much money is spent annually on purchased instructional materials? How much money is spent annually on teacher professional development designed to help teachers utilize purchased instructional materials in their classrooms? How much money is spent annually on teacher professional development designed to help teachers curate, evaluate, and implement OER in their classrooms? The same process was used for the second group of participants as they submitted their qualitative data by accessing the Teacher Perception Survey via Google Forms. Again, this allowed the researcher to quickly compile and organize the data the participants collected to determine their perception of the instructional materials being IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 86 used within their Algebra I classrooms. Data was collected from this survey by utilizing Short Answer Questions and Likert Scale questions. The questions from the Teacher Perception Survey are shown in Table 3 and Appendix B. Table 3 Teacher Perception Survey Questions Teacher Perception Survey Questions Which best describes your education level? In which District do you teach Algebra I? In which building do you teach Algebra I? Generally speaking, do you believe that the 7th or 8th grade mathematics curriculum within your district is well-aligned to the PA Core Standards for Mathematics? Generally speaking, do you believe that the 7th or 8th grade mathematics instructional materials used within your district are missing critical standards addressed on the Algebra I Keystone Exam? I believe that the physical and digital instructional materials purchased and used with our students were well researched and help us to prepare students for success on the Algebra I Keystone Exam. I have been given plenty of professional development focused on how to effectively use the instructional materials provided to me with my students. I have been given plenty of professional development focused on how to effectively develop my own instructional materials with my students. I believe that the instructional materials used in my classroom have a direct impact on my students’ achievement scores. In terms of the physical instructional materials used with students, how would you describe the way in which the skills addressed prepare students for the Algebra I Keystone Exam? In terms of the digital instructional materials used with students, how would you describe the way in which the skills addressed prepare students for the Algebra I Keystone Exam? IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 87 I believe that the instructional materials used accurately reflect my student population. I believe that my students can see themselves and their life experiences in their instructional materials in my class. I believe that when students are able to see themselves in their instructional materials they will do better on assessments. I have been adequately trained in understanding the culture and values of my student population. I have been adequately trained to evaluate instructional materials for cultural relevance. I have been adequately trained to modify existing resources or develop new resources that are culturally relevant to my student population. I am aware of Open Educational Resources (OER) and use them within my learning environment. My colleagues are aware of Open Educational Resources (OER) and use them in their learning environments. My building administrators are aware of Open Educational Resources (OER) and encourage their use. The final data collection step was to have the middle school Algebra I teachers of the three representative school districts in Westmoreland County complete the Culturally Responsive STEAM Scorecard to evaluate their primary Algebra I instructional materials. The Culturally Responsive STEAM Scorecard collects data utilizing a Likert Scale. It evaluates teachers’ perceptions of the cultural responsiveness of their instructional materials in four categories: Representation, Social Justice, Teachers’ Materials, and Materials & Resources. During this step, the researcher met with the building principal and Algebra I teachers of Greater Latrobe School District, Monessen City School District, and Yough School District to explain the components of the Culturally Responsive STEAM Scorecard, how to use the Culturally Responsive STEAM IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 88 Scorecard to evaluate their primary Algebra I instructional materials, and to establish the timeline for completion and submission of the scorecard. Once each district’s group of teachers compiled the data, the researcher created an average score for each statement in all four categories and a final score for each. The statements from the Culturally Responsive STEAM Scorecard are shown in Table 4 and Appendix C. Table 4 Culturally Responsive STEAM Scorecard Representation Statements The curriculum centers the origins of science, technology, engineering, arts and/or math within BIPOC communities (Black, Indigenous, people of color). The curriculum elevates mathematicians, artists, and/or scientists with historically marginalized identities (i.e. non-binary or trans people, women, people of color, people with disabilities, working class people, multilingual people) and their discoveries. The curriculum acknowledges and/or incorporates the expertise of diverse communities, their cultures, and their historical and/or contemporary experiences. The curriculum has photos/pictures, names, scenarios, and text that reflect the experiences and interests of students of color in your community. The curriculum elevates not just “scientists, artists, and mathematicians”, but the everyday users of math, science, arts, technology, and engineering. The curriculum affirms the multiple forms of communication or language systems during mathematical and scientific argumentation rooted in historically marginalized cultures. Social Justice Statements Curriculum highlights and affirms the knowledge systems of Indigenous, Black/African, Brown, and non-Western conceptions of science, technology, engineering, arts, and math (such as interdependence, sustainability, and continual change). IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 89 The curriculum and instructional activities promote or provoke critical questions about science, technology, engineering, arts, and/or math and the societal status quo. The curriculum and instructional activities present alternatives or allow students to present alternatives about the role science, technology, engineering, arts, and/or math plays in existing social structures. The curriculum presents multiple understandings of a scientific, mathematical, or artistic concept or theory, especially highlighting points of view from marginalized people/communities. The curriculum provides avenues for students to see STEAM as a way to understand and improve their world, take actions that combat inequity or promote equity, and connect learning to social, political, and/or environmental concerns. The curriculum presents social situations and problems not as individual problems but as embedded within a societal and/or systemic context. The curriculum encourages students to critically examine dominant knowledge systems as scientists, mathematicians, and artists. Teachers’ Materials Statements The authors of the teachers’ materials are people of diverse identities (race/ethnicity, gender, other identities). Guidance is provided to teachers on being aware of one’s biases, assumptions, and the gaps between one’s own culture and students’ cultures (implicit bias). The teachers’ materials ask teachers to reflect on their own practices and experiences learning STEAM subjects and critique them through the lens of cultural responsiveness. Guidance is provided on combating the legacy of STEAM education related trauma amongst historically marginalized communities and on designing healing and joyful STEAM experiences. Guidance is provided on using students’ everyday lives as the starting point for learning, and making real-life connections between academic content and the local neighborhoods, culture, environment, community issues, and current events. Guidance is provided on creating opportunities to meaningfully engage students’ families to enhance lessons. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 90 Guidance is provided on appropriately using Indigenous and non-Western resources to understand math and science, including oral histories, legends, and community knowledge. Guidance is provided on engaging students in culturally responsive experiential learning activities. Guidance includes, for applicable lessons, a range of possible student responses that could all be valid, given the range of student experiences and perspectives. Guidance is provided on how to design lessons or engage in conversations that use STEAM to disrupt power inequities or create opportunities for students to practice disruption. Guidance is provided on planning opportunities for students to create and teach lessons to their peers. Guidance is provided to help teachers foster environments of collaboration that go beyond group work by meshing individual's assets, resources, and strengths (such as various ways of knowing, doing and being) to advance group and individual learning. Materials & Resources Statements The curriculum provides opportunities for students to use accessible technology and manipulatives to explore STEAM concepts in ways that reflect a variety of ways of “doing.” Issues of social justice, equity, and Black, Indigenous, and people of color’s experiences and contributions are reflected in homework/classroom assignments and assessments. The curriculum incorporates the arts as a tool to center experiences within STEM and innovation. Curriculum rigor is not dependent on access to resources, materials and technology that students and schools may not have. In other words, the curriculum materials are fully accessible; all resources, materials and technology options are rigorous and interesting. (Ex. If students can engage curriculum materials with a computer or paper, the paper materials should be just as rigorous, interesting, and engaging as using the computer). The resources, materials and technology referenced in lessons include resources from BIPOC communities (Black, Indigenous, people of color). IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 91 As stated in Chapter III, the researcher looked at the historic Algebra I test scores for each of the three participating school districts, their 2021 - 2022 demographic data and their Culturally Responsive STEAM Scorecard results to effectively answer the three research questions established for this case study. Results Research Question 1 The first research question in this case study aims to answer “What are the factors that impact student achievement in Algebra I?” After student assessment data was compiled, scores were disaggregated into achievement categories and by Algebra I Assessment Anchors. District demographic data were also disaggregated into the subgroups that PDE uses for reporting. These subgroups are consistent with existing Civil Rights Data Collection standards established by the United States Department of Education. To thoroughly investigate this question, student assessment scores for Greater Latrobe School District, Monessen City School District, and Yough School District were further disaggregated by combining two or more standard subgroups that PDE uses for reporting to create unique subgroups. Those distinctive subgroups include: ● Ethnicity and Gender ● Ethnicity and Economically Disadvantaged ● Ethnicity and English Language Learners ● Ethnicity and IEP The unique subgroups were then analyzed in comparison with the results of the Culturally Responsive STEAM Scorecard for Greater Latrobe School District and Yough IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 92 School District. To determine a correlation between student achievement scores and the results of the Culturally Responsive STEAM Scorecard, the researcher focused on Algebra I Keystone performance levels for middle school students, disaggregated by ethnicity, for Spring 2022. Table 5 shows those results below for Greater Latrobe School District, Monessen City School District, and Yough School District. Table 5 Group Summary, Performance Levels by Grade 08 and Ethnicity Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022 Best Score Below Basic Basic Proficient Advanced Number Administration Tested Mean SS Greater Latrobe SD Spring 2022 383 1494.3 12.5 45.7 25.1 16.7 Grade 08 Spring 2022 111 1545.4 0 11.7 40.5 47.7 Monessen City SD Spring 2022 49 1422.7 75.5 20.4 4.1 0 Grade 08 Spring 2022 11 1449.5 54.5 27.3 18.2 0 Yough SD Spring 2022 183 1476.6 17.5 53 24.6 4.9 Grade 08 Spring 2022 41 1503 2.4 39 43.9 14.6 Group % % % % Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022, Grade 08 and Black Best Score Group Number Administration Tested Mean SS Below Basic % Basic Proficient % % Advanced % Greater Latrobe SD Grade 08 Spring 2022 0 0 0 0 0 0 Monessen City SD Grade 08 Spring 2022 3 1425.3 66.7 33.3 0 0 Yough SD Grade 08 Spring 2022 0 0 0 0 0 0 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 93 Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022, Grade 08 and Hispanic/Latino Best Score Group Number Administration Tested Mean SS Below Basic % Basic Proficient % Advanced % % Greater Latrobe SD Grade 08 Spring 2022 1 1591 0 0 0 100 Monessen City SD Grade 08 Spring 2022 3 1435 100 0 0 0 Yough SD Grade 08 Spring 2022 0 0 0 0 0 0 Group Summary: Performance Levels: Keystone, Algebra I, Spring 2022, Grade 08 and White Best Score Below Basic Basic Proficient Advanced Number Administration Tested Mean SS Greater Latrobe SD Grade 08 Spring 2022 108 1544.4 0 12 41.7 46.3 Monessen City SD Grade 08 Spring 2022 6 1469.2 33.3 33.3 33.3 0 Yough SD Grade 08 Spring 2022 39 1502.1 2.6 38.5 46.2 12.8 Group % % % % The researcher compared this information to the Culturally Responsive STEAM Scorecard results for Greater Latrobe School District. The primary Algebra I instructional material utilized by the middle school Algebra I teachers is the 8th Grade Algebra edition of Go Math by Houghton Mifflin Harcourt. Overall, teachers in the Greater Latrobe School District felt satisfied that the curriculum of their primary Algebra I instructional materials adequately represented diverse communities, including Black, Indigenous, and People of Color (BIPOC), those with historically marginalized identities. Furthermore, they are satisfied that the curriculum of their primary Algebra I instructional materials incorporates the expertise of these diverse and historically marginalized communities and elevates those communities throughout their IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 94 student-facing materials. Due to this, the total score for the Representation category was 2. In reviewing the Social Justice category, teachers in the Greater Latrobe School District were not satisfied with how their primary Algebra I instructional materials addressed concepts such as interdependence and continual change. They were also not satisfied that the materials’ instructional activities adequately presented alternatives or allowed students to present alternatives about the role science, technology, engineering, arts, and/or math plays in existing social structures. Due to this, the total score for the Social Justice category was -5. The only category where teacher opinions varied greatly was the Teachers’ Materials category. Some teachers were very satisfied that the teachers’ materials for their primary Algebra I instructional materials adequately provided guidance to teachers on being aware of one’s biases, assumptions, and the gaps between one’s own culture and students’ cultures that the materials ask teachers to reflect on their own practices and experiences learning STEAM subjects and critique them through the lens of cultural responsiveness and that the materials provide guidance on using students’ everyday lives as the starting point for learning and making real-life connections between academic content and the local neighborhoods, culture, environment, community issues, and current events. Others felt that the teachers’ materials were very lacking in these areas. Due to this, the total score for the Teachers’ Materials category was -1.5. Finally, the Greater Latrobe School District teachers had the most confidence in the materials and resources associated with their primary Algebra I instructional materials. Overall, they felt satisfied that the materials and resources provide IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 95 opportunities for students to use accessible technology and manipulatives to explore STEAM concepts in ways that reflect a variety of ways of “doing,” that issues of social justice, equity, and Black, Indigenous, and people of color’s experiences and contributions are reflected in homework/classroom assignments and assessments, and that the rigor of the curriculum is not dependent upon the access to resources, materials and technology that students and schools may not have. Due to this, the total score for the Materials & Resources category was 2.5, which was the highest score given by teachers in the Greater Latrobe School District. Figure 23 shows the scorecard results for the Greater Latrobe School District. Figure 23 GLSD Culturally Responsive STEAM Scorecard Results The researcher then compared the Algebra I Keystone performance levels for middle school students, disaggregated by ethnicity, for Spring 2022 to the results of the IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 96 Culturally Responsive STEAM Scorecard for the Yough School District. The primary Algebra I instructional material utilized by the middle school Algebra I teachers is a district-specific, OER-based Algebra I Flexbook that was customized on CK-12’s digital platform. Overall, teachers in the Yough School District felt satisfied that the curriculum of their primary Algebra I instructional materials adequately represented diverse communities, including Black, Indigenous, and People of Color (BIPOC), those with historically marginalized identities. Furthermore, they are satisfied that the curriculum of their primary Algebra I instructional materials incorporates the expertise of these diverse and historically marginalized communities and elevates those communities throughout their student-facing materials. Due to this, the total score for the Representation category was 3.5, which was the highest score given by teachers in the Yough School District. In reviewing the Social Justice category, teachers in the Yough School District were less satisfied with how their primary Algebra I instructional materials addressed concepts such as interdependence and continual change. It was unclear whether the materials’ instructional activities adequately presented alternatives or allowed students to present alternatives about the role science, technology, engineering, arts, and/or math plays in existing social structures. Due to this, the total score for the Social Justice category was -1. As we also saw in the Greater Latrobe School District, the only category where teacher opinions varied greatly in the Yough School District was the Teachers’ Materials category. Some teachers were very satisfied that the teachers’ materials provided guidance on using students’ everyday lives as the starting point for learning, and making real-life connections between academic content and the local neighborhoods, culture, IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 97 environment, community issues, and current events, and to help teachers foster environments of collaboration that go beyond group work by meshing individual's assets, resources, and strengths to advance group and individual learning. However, other teachers felt that the teachers’ materials were unclear in their ability to adequately provide guidance to teachers on being aware of one’s biases, assumptions, and the gaps between one’s own culture and students’ cultures that the materials ask teachers to reflect on their own practices and experiences learning STEAM subjects and critique them through the lens of cultural responsiveness, and in how to design lessons or engage in conversations that use STEAM to disrupt power inequities or create opportunities for students to practice disruption. Due to this, the total score for the Teachers’ Materials category was 1. Finally, the teachers in the Yough School District felt satisfied that the materials and resources for their primary Algebra I instructional materials provide opportunities for students to use accessible technology and manipulatives to explore STEAM concepts in ways that reflect a variety of ways of “doing,” and that the rigor of the curriculum is not dependent upon the access to resources, materials and technology that students and schools may not have. They felt unclear that issues of social justice, equity, and Black, Indigenous, and people of color’s experiences and contributions are reflected in homework/classroom assignments and assessments. Due to this, the total score for the Materials & Resources category was 2. Figure 24 shows the results of the scorecard for the Yough School District. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 98 Figure 24 YSD Culturally Responsive STEAM Scorecard Results The researcher had planned to review the Algebra I Keystone performance levels for middle school students, disaggregated by ethnicity, for Spring 2022 to the results of the Culturally Responsive STEAM Scorecard for the Monessen City School District. Unfortunately, the teachers in that school district did not complete the scorecard. The primary Algebra I instructional material utilized by the middle school Algebra I teachers is the enVision - Algebra I online program by Savvas, formerly Pearson. Next, the researcher completed a standards-analysis review for each of the primary Algebra I instructional materials utilized by the middle school Algebra I teachers in the Greater Latrobe School District, the Monessen City School District, and the Yough School District. Since each resource was developed to be aligned to Common Core State Standards, the researcher had to complete a Standards Crosswalk to align the materials to IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 99 Pennsylvania Core Standards and then determine which Standards, Assessment Anchors and Eligible Content were present within each. Table 6 shows the number of Eligible Content items present in each primary instructional material used by the Greater Latrobe School District, Monessen City School District, and Yough School District. Table 6 Number of Algebra I Eligible Content Items Present by Vendor Algebra I Assessment Anchor A1.1.1 - Operations With Real Numbers and Expressions # of Eligible Content # of Eligible # of Eligible # of Eligible Items within Anchor in Content Content Content PA Core Standards For Items Items Items Mathematics Addressed Addressed Addressed Curriculum Framework in HMH in CK-12 in Savvas Algebra 1 Resource Resource Resource 9 7 7 5 A1.1.2 - Linear Equations 19 7 13 14 A1.1.3 - Linear Inequalities 10 0 10 2 A1.2.1 - Functions 24 9 12 11 A1.2.2 - Coordinate Geometry 13 4 8 4 5 2 0 3 A1.2.3 - Data Analysis When comparing Algebra I Keystone performance levels for middle school students, disaggregated by ethnicity, for Spring 2022 to the results of the Standards Crosswalk for all three primary Algebra I instructional materials, the findings indicate that there may be an impact on student learning and achievement since the Algebra I Assessment Anchors are not adequately addressed in each of the materials being used. Go Math by covers 36.25% of the available Eligible Content items for Algebra I within their curriculum. CK-12’s Algebra I flexbook covers 62.5% of the available Eligible Content items for Algebra I within their curriculum in its original form before being modified by users. Savvas’ enVision Algebra I program covers 48.75% of the available Eligible Content items for Algebra I within their IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 100 curriculum. If teachers are not aware of the items that are not addressed within the curriculum of the primary Algebra I instructional materials that their district has approved, students may not be exposed to this content in the classroom, resulting in a lack of understanding of concepts and mastery of skills needed to achieve higher performance levels on the Algebra I Keystone Exam. Finally, data were collected from the Teacher Perception Survey and compared with historical student assessment scores in Algebra I and the results of the Culturally Responsive STEAM Scorecard. Data was also collected from the Building Principal Perception Survey and compared with the results of the Teacher Perception Survey, historical student assessment scores in Algebra I, and the results of the Culturally Responsive STEAM Scorecard. In reviewing the results from the Teacher Perception Survey, teachers in Westmoreland County overwhelmingly feel that their middle school mathematics curriculum is well-aligned to Pennsylvania Core Standards for Mathematics, with 66.7% of respondents reporting that their curriculum is mainly aligned to the standards. However, their views began to vary as they began to evaluate the alignment of their primary instructional resources to Pennsylvania Core Standards. Figure 25 shows the differences in their perceptions of the alignment of the primary instructional materials used in middle school Algebra I classes to Pennsylvania Core Standards for Mathematics. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 101 Figure 25 Teacher Perceptions of Standards Alignment of Instructional Materials Of the teachers participating in the survey, 66.7% felt that their materials were not missing critical standards addressed on the Algebra I Keystone Exam. In reviewing the Building Principal Perception Survey results, most building principals participating in the survey feel that at least 75% of their purchased instructional materials accurately align with Pennsylvania Core Academic Standards. Only 27.3% of participants felt that 50% of their purchased instructional materials were accurately aligned with Pennsylvania Core Academic Standards. Figure 26 shows building principals' perceptions regarding the accurate alignment to Pennsylvania Core Academic Standards for their purchased instructional materials. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 102 Figure 26 Building Principals’ Perceptions of Instructional Materials Alignment to Standards However, the results of the Standards Crosswalk completed by the researcher, as shown above in Figure 30, show that resources being used within Westmoreland County are missing substantial amounts of Eligible Content items directly connected to the Pennsylvania Core Standards. The perception surveys' findings indicate that teachers and building principals feel that the primary instructional materials used in middle school Algebra I classrooms are more accurately aligned to Pennsylvania Core Standards and are addressing Eligible Content items in Algebra I than they really are. Research Question 2 The second research question in this case study aims to answer “What are the factors that impact instructional delivery in Algebra I?” Data from both the Teacher Perception Survey and the Building Principal Perception Survey were compiled and analyzed to research this question effectively. Many of the questions presented to teachers and building principals were related to their perceptions about the time and funds spent on professional development for utilizing instructional materials. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 103 The researcher first focused on the results of the Teacher Perception Survey. When asked if teachers had been given plenty of professional development focused on using the instructional materials their district had purchased effectively, the answers varied greatly. Some teachers strongly agreed they had been given plenty of time, while others disagreed. Figure 27 shows the differences in teachers' perceptions regarding the amount of time they feel that they have to use the instructional materials they have been given effectively. Figure 27 Teacher Perceptions of PD to Effectively Use Instructional Materials The researcher next compared these results to data from the Building Principal Perceptions Survey. When asked how much money is spent annually on teacher professional development experiences designed to help teachers utilize their purchased instructional materials in their classrooms, 81.8% of building principals who participated in the survey said they spend between $0.00 and $25,000.00. Figure 28 shows the breakdown of funds spent annually by districts on teacher professional development to utilize instructional materials in their classroom. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 104 Figure 28 Money Spent Annually By Districts on Teacher PD to Utilize Instructional Materials In comparing this data with the results of the Culturally Responsive STEAM Scorecard, as shown in Figures 23 and 24 above, the findings indicate that teachers feel they need more confidence in their ability to evaluate the instructional resources that their districts have purchased. Teachers also need more professional development experiences designed to help them utilize their primary instructional materials more effectively and develop culturally responsive learning experiences to meet students' needs better. Each of these factors directly impacts the instructional delivery of Algebra I instruction occurring in middle school classrooms in Westmoreland County. When comparing Algebra I Keystone performance levels for middle school students, disaggregated by ethnicity, for Spring 2022 to the results of the Culturally Responsive STEAM Scorecard, the findings indicate that the guidance provided to teachers on how to use their instructional materials from the vendor that developed those materials may have an impact on student learning and achievement. When teachers feel that this guidance needs to be clarified, they may struggle to create instructional IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 105 experiences for students that reflect the lived experiences of the students in their classroom and actively design instruction and instructional materials that reflect those experiences. The researcher speculates that this may be addressed by implementing the Culturally Relevant and Sustaining Education Guidelines and Framework that the Commonwealth of Pennsylvania has adopted. As of this writing, Pennsylvania school districts have been directed to include Culturally Relevant and Sustaining Education as a part of their continuing education plan for their educators starting with the 2023-2024 school year. Based on the competencies within the framework, this professional development will assist teachers in developing instructional experiences for students that reflect their lived experiences in their classroom and actively design instruction and instructional materials that reflect those experiences. Research Question 3 The third research question in this case study aims to answer “What are the teacher and administrator perceptions of the impact of instructional delivery on student achievement scores in Algebra I?” To investigate this question, the researcher analyzed data from the Teacher and Building Principal Perception Surveys relating to instruction, instructional materials, and student achievement scores. Overall, teachers believe that their instructional materials directly impact student achievement scores, as shown in Figure 29. These perceptions appear to correlate to the Standards Crosswalk results and the historical student achievement scores in Algebra I. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 106 Figure 29 Teacher Perceptions on the Impact of Instructional Materials on Student Achievement In examining the data further to determine the impact on instructional delivery, teachers feel split in their opinions that the physical instructional materials used with students adequately address the skills needed to help prepare students for the Algebra I Keystone Exam. They feel less confident in the ability of their digital instructional materials to address the skills required to help prepare students for the Algebra I Keystone Exam. Figures 30 and 31 show the results of these questions from the Teacher Perception Survey. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 107 Figure 30 Teacher Perceptions on Physical Materials Preparing Students for Keystone Exams Figure 31 Teacher Perceptions on Digital Materials Preparing Students for Keystone Exams Teachers overwhelmingly believe that students do better on assessments when they are able to see themselves in their instructional materials, as seen in Figure 32 below. However, teachers acknowledge that their primary instructional materials are not always culturally reflective of their entire student population. They must supplement their IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 108 instruction with Open Educational Resources or other materials to develop more culturally relevant learning experiences. Figure 32 Teachers' Perceptions of Student Reflections on Instructional Materials Building principals also acknowledge that teachers need to modify resources to meet students' cultural and academic needs. If teachers are unable to modify the instructional materials that are available to them, they will develop their own original materials instead. Figures 33 and 34 provide a breakdown of the perceptions that building principals have regarding the frequency in which teachers modify resources to meet the needs of students or develop their own resources to meet the needs of students. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 109 Figure 33 Building Principals’ Perceptions of Teacher Modifications to Resources Figure 34 Building Principals’ Perceptions of Teacher Development of Original Resources Although building principals acknowledge that teachers are engaging in the modification of existing resources or the creation of original resources to meet the cultural and academic needs of their students as a way to supplement the instruction that is provided within their primary instructional materials for Algebra I, they also IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 110 acknowledge that there are internal and external factors that cause teachers to shy away from taking these action. Building principals feel that the internal factors that cause teachers not to use OER in their Algebra I classrooms include: ● “A perception that OER are not aligned to PA Core Standards” ● “Lack of knowledge and how they can be incorporated into what they do and a perceived lack of time during the day” ● “Ignorance” ● “Lack of research and commitment to utilizing various resources that may be different from what they are comfortable already using” They also feel that the external factors that cause teachers not to use OER in their Algebra I classrooms include: ● “Not explicitly approved as per the curriculum / textbook approval process.” ● “Potential pressure from community members” ● “Focus on school curriculum” ● “School board” These findings suggest that the internal and external factors preventing the broader adoption of Open Educational Resources in Algebra I classrooms in Westmoreland County are rooted in a lack of understanding about OER and a fear of judgment from others for utilizing OER. Discussion The research aimed to examine the possible correlation between the historically low Keystone Algebra I Scores in middle schools in Westmoreland County, building principals and Algebra I teachers’ perceptions of the standards-alignment and efficacy of IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 111 their primary instructional materials, the amount of training and support provided to teachers to effectively utilize those materials in their Algebra I classrooms, and the culturally responsive nature of their materials. As stated earlier in Chapter IV, the quantitative data gathered by the researcher included overall student Algebra I Keystone Exam scores, completed a countywide Algebra I Anchor Analysis, local assessment data for Algebra I, Future Ready PA Index data, annual budget information relating to instructional materials expenses, district Comprehensive Plans, and Pennsylvania Common Core Standards alignment information for the primary Algebra I instructional resources used by Greater Latrobe School District, Monessen City School District, and Yough School District. This data was gathered from the eMetric Data Interaction for Pennsylvania Student Assessments web-based system (eMetric), the web-based Pennsylvania Value Added Assessment System (PVAAS), the Pennsylvania Department of Education’s website (PDE), PDE’s web-based Future Ready PA Index, individual district websites, and individual content vendor websites. The qualitative data gathered by the researcher included Building Principal and Teacher Perception Surveys and the completion of the Culturally Responsive STEAM Scorecard. The data were analyzed to address all research questions to determine if a correlational relationship existed. To analyze the first research question, “What are the factors that impact student achievement in Algebra I?” student assessment data was compiled and then disaggregated into achievement categories and by Algebra I Assessment Anchors. District demographic data were also disaggregated into the subgroups that PDE uses for reporting. Student achievement data was then compared to the results of the Culturally Responsive STEAM Scorecard and the Standards Crosswalk for the instructional IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 112 materials. Analyses revealed that several factors may be impacting student achievement in Algebra I. The first factor revealed in the data analysis for this research question was that middle school Algebra I teachers do not feel that their primary instructional materials for Algebra I provide them with guidance and support in developing culturally responsive learning experiences that meet the needs of their student populations. Disaggregating district demographic data by ethnicity and comparing it to the results of the Culturally Responsive STEAM Scorecard shows that culturally responsive instruction may impact student achievement. Teachers believe this to be true but need more guidance and support in creating those learning experiences for their students. The second factor revealed in the data analysis for this research question was that the primary instructional materials used by the three identified case study school districts do not adequately address the content covered on the Algebra I Keystone Exam. Moreover, teachers must be made aware that this content must be addressed in their instructional materials. While teaching to the test is not recommended, reviewing instructional materials to ensure that they adequately support instruction to cover concepts and skills that will be assessed on the Algebra I Keystone Exam is a highly recommended action by PDE. To thoroughly complete this work, teachers and administrators cannot simply accept the supporting alignment documentation that content vendors provide. Instead, they must conduct their own independent evaluation to determine if all Assessment Anchors and Eligible Content items are present and can support the instructional experiences that their teachers develop. If absent, teachers and IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 113 administrators will need a plan to supplement their materials and instruction to prepare students for the Algebra I Keystone Exam. To analyze the second research question, “What are the factors that impact instructional delivery in Algebra I?” data from both the Teacher Perception Survey and the Building Principal Perception Survey that related to their perceptions about time and funds spent on professional development for utilizing instructional materials were compiled and analyzed. This research question aimed to explore how teachers grow in their knowledge of their student populations, the use of their instructional materials, and how to design effective learning experiences that help to increase student achievement. Unfortunately, there was not much teacher participation in the Teacher Perception Survey. The researcher speculates that the current political climate and the connection to the term “culturally responsive” may have caused teachers to hesitate to participate in the survey. However, findings based on the received responses show that teachers feel they are given enough time to learn how to effectively utilize their instructional resources when districts spend money on professional development specifically designed to help teachers use those materials in their classrooms. The findings based on the Culturally Responsive STEAM Scorecard show that teachers still feel uncertain about developing culturally responsive learning experiences or identifying how resources may not be meeting the needs of their students from a culturally responsive standpoint. While the low participation numbers on the Teacher Perception Survey make it challenging to determine if a lack of culturally responsive instruction impacts student achievement scores, teachers in Westmoreland County believe there is a correlation between culturally responsive pedagogy and increased student IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 114 achievement. As districts begin to evaluate their instructional materials through a more culturally responsive lens with the guidance of the Culturally Relevant and Sustaining Education Guidelines and Framework developed by PDE, they will need to design professional development experiences for teachers that guide them through the process of how to balance the effective use of instructional materials to present concepts and skills, and better understand the specific needs of their particular student populations from a culturally responsive point of view and modify or supplement instruction with OER to address those needs. To analyze the third and final research question, “What are the teacher and administrator perceptions of the impact of instructional delivery on student achievement scores in Algebra I?” data was gathered from the Teacher and Building Principal Perception Surveys relating to instruction, instructional materials, and student achievement scores. This research question aimed to analyze whether or not teachers and administrators believe that instructional experiences and resources directly impact student achievement. The findings indicate that while teachers and building principals recognize that instruction and instructional materials will impact student achievement, they must learn to analyze their resources from a standards-alignment and culturally responsive perspective. They may also need more time to engage in observational activities that allow them to see examples of standards-alignment activities and culturally responsive instruction to learn how to develop and implement these skills. The findings also indicate a significant disconnect in understanding Open Educational Resources while teachers regularly engage in activities commonly associated IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 115 with their use. Survey results show that teachers regularly modify instructional materials to meet the needs of their students, with respondents indicating that many teachers do this at least weekly, if not daily, to ensure that students can successfully engage with their instructional materials. The act of revising a resource to adapt or modify it to meet the needs of students better is one of the main ways in which educators engage with Open Educational Resources. Teachers across Westmoreland County engage in this critical practice to support student academic success. In summary, the most significant finding is a direct correlation between student achievement scores and the amount of Assessment Anchors and Eligible Content items present within the primary instructional materials that middle school Algebra I teachers use. When instructional materials have a higher level of Assessment Anchors and Eligible Content items present, students have a better chance of having higher achievement scores on the Algebra I Keystone Exam. This shows that teachers strongly align their instruction to the content in their primary instructional materials. Factors that prevent them from deviating from their primary instructional materials include pressure only to use their purchased materials, a lack of knowledge about the Assessment Anchors and Eligible Content items that are present within their primary instructional materials, and a lack of understanding of how to utilize Open Educational Resources effectively. Summary Chapter IV of the study introduced a mixed-methods embedded research design aimed at investigating a potential correlation between the historically low Keystone Algebra I Scores of middle school students in Westmoreland County and the perceptions of building principals and Algebra I teachers regarding student identification with IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 116 examples in word problems found within their middle school instructional materials. The chapter provided a comprehensive analysis of both quantitative and qualitative data to address the three research questions formulated for this particular case study. To start, the chapter offered a detailed explanation of the research methodology used by the researcher, which combined quantitative and qualitative approaches to gain a thorough understanding of the research problem. The mixed-methods design allowed the researchers to gather numerical data through the collection of assessment scores while also gathering subjective insights through perception surveys. This comprehensive approach aimed to provide a more robust analysis and interpretation of the research findings. The chapter then delved into the data analysis process, describing the techniques used to analyze both the quantitative and qualitative data. The quantitative analysis sought to identify any correlations or patterns between the Keystone Algebra I Scores and analyses of the middle school instructional materials. The qualitative data analysis involved a thorough examination of surveys and the perceptions of principals and teachers. This qualitative analysis aimed to provide a deeper understanding of the perceptions and experiences of principals and teachers regarding student identification within the word problems presented to students within their materials. The chapter then presented the results obtained from both the quantitative and qualitative approaches. The quantitative results provided evidence of any correlations between low Keystone Algebra I Scores and the alignment of instructional materials to PA Core Standards and Eligible Content Standards. In addition, the qualitative results IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 117 offered rich descriptions and interpretations of the identified themes and patterns within the perceptions of principals and teachers. Finally, Chapter IV engaged in a comprehensive discussion of the interpretation of the results. It analyzed the implications of the findings, considering both the quantitative and qualitative perspectives. The discussion aimed to identify the possible reasons behind the findings, providing insights into the underlying factors influencing student identification in the context of word problems. Chapter IV serves as a bridge between data analysis and the upcoming conclusions and recommendations of the research study, which will be presented in the next chapter. Chapter V, which will focus on drawing conclusions and formulating recommendations based on the interpretation of the results from the previous chapter. It will also synthesize the findings, discuss their significance, and propose actionable recommendations that can contribute to addressing the challenges identified in the study. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 118 Chapter V Conclusion and Recommendations Administrators and teachers in public school districts in Westmoreland County have struggled to understand their students’ historically low middle school mathematics test scores on PSSA and Keystone Exams. The Keystone Exam test scores of middle school students have been particularly concerning since the inception of the Keystone Algebra I Exam in 2010. To understand the root cause of these scores, building administrators and Algebra I teachers have engaged in intensive data analysis of student test scores over the years. This analysis has driven the adoption of instructional materials, the refinement of scopes and sequence of instruction, the evaluation of and alignment of curriculum maps to Pennsylvania Core State Standards, and the implementation of new instructional initiatives designed to increase student achievement scores. The mixed-methods embedded design of the research set out to answer three research questions, which sought to further determine the root cause of students’ historically low test scores by understanding the impact of the instructional materials used in Algebra I classrooms on student achievement. They also sought to understand building administrator and Algebra I teacher perceptions of instructional materials and professional development time devoted to implementing instructional materials with fidelity, as well as understanding the perceived cultural responsiveness of their primary instructional materials for Algebra I in middle school classrooms. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 119 Eleven building principals and six middle school Algebra I teachers employed in K-12 public schools in Westmoreland County, Pennsylvania, participated in the study. The researcher collected data related to student Algebra I achievement scores, assessment data, budget data, and standards alignment data to discover patterns and trends in student achievement scores over time. The participants completed perception surveys and the Culturally Responsive STEAM Scorecard to provide a potential correlation between instructional materials, Culturally Responsive Teaching pedagogy, and increases in student achievement. Conclusion The mixed-method embedded research design sought to determine if a correlation exists between the historically low Keystone Algebra I Scores of middle school students in Westmoreland County and the perceptions of building principals and Algebra I teachers regarding student identification with examples in word problems found within their middle school instructional materials. The researcher investigated this issue to provide both building administrators and Algebra I teachers with guidance to help support their efforts to understand the gaps between the intended curriculum better, as identified by state standards and district curriculum maps, the implemented curriculum, as observed through the alignment of resources to standards and culturally responsive pedagogy, and the attained curriculum, as shown by student achievement scores. The following sections discuss specific conclusions related to each of the established research questions, explain how the findings support those conclusions, and describe how those findings may be impacting current instructional practices in middle school Algebra I classrooms. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 120 Research Question 1: “What are the factors that impact student achievement in Algebra I?” The first research question analyzed the factors that impact student achievement in Algebra I by reviewing disaggregated, historic student achievement data. The researcher then completed a standards-analysis review for each of the primary Algebra I instructional materials utilized by the middle school Algebra I teachers in three case study districts representing all of Westmoreland County: Greater Latrobe School District, the Monessen City School District, and the Yough School District. Finally, the researcher compared disaggregated test scores and the standards analysis with the results of the Culturally Responsive STEAM Scorecard by participants from the case study school districts. The study design aimed to determine if the instructional materials used in Algebra I classrooms impacted student proficiency scores on the Algebra I Keystone Exam. By disaggregating student scores on the Algebra I Keystone Exam by race, the researcher gained insight into the impact of the wording of the exam questions on student subgroups. There was no significant difference in Algebra I Keystone Exam scores between subgroups, regardless of gender, race, socioeconomic status, special education status, or English language learner status. These factors alone do not have a significant impact on student proficiency. This suggests that the wording used in the Algebra I Keystone Exam questions is mainly free from bias. Perception data showed that both building principals and Algebra I teachers have high confidence levels in the accuracy of their primary instructional materials. They feel strongly that their materials are closely aligned with Pennsylvania Core Academic IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 121 Standards and will support students as they prepare for the Algebra I Keystone Exam. Each district’s process for adopting primary instructional materials includes, at a minimum, a review of the standards alignment documentation provided by the content vendor. Despite high levels of confidence from building principals and Algebra I teachers that their primary instructional materials are aligned with Pennsylvania Core Academic Standards, a comprehensive standards analysis shows that the primary instructional materials used by the three case study districts are missing critical Eligible Content Items that are addressed on the Algebra I Keystone Exam. The implication is that if teachers rely heavily on instructional materials that do not address Eligible Content Items that are presented to students on the Algebra I Keystone Exam, students may not understand the content, contributing to their low test scores. In evaluating materials with the Culturally Responsive STEAM Scorecard, Algebra I teachers were able to quantify their perception of the cultural relevance of their primary instructional material used for Algebra I. The results of the Culturally Responsive Scorecard show that teachers are not fully confident in their abilities to identify bias within their instructional materials accurately. The adoption of Pennsylvania’s Culturally Relevant and Sustaining Education Framework, including its standards and competencies, is designed to assist educators in completing this work to help teachers critically examine their instructional materials to determine if students can see themselves reflected in their instructional materials. The required training components associated with the framework may assist teachers in building their confidence in analyzing instructional materials through a culturally responsive lens. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 122 Research Question 2: “What are the factors that impact instructional delivery in Algebra I?” The second research question analyzed the time and funds spent on professional development for utilizing instructional materials. The study design aimed to determine whether or not time and funds for professional development related to using instructional materials in the classroom impacted student achievement. It also sought to understand the differences in perceptions regarding time and funds for professional development from building administrators and Algebra I teachers. There was a discrepancy in teacher opinions on the amount of time given for professional development, with some strongly agreeing that they had been given plenty of time and others disagreeing. This may indicate that the amount of time spent on professional development varies across districts in Westmoreland County. Conversely, it highlights teachers' different confidence levels in their abilities to utilize their instructional materials without additional support effectively. When asked how much money is spent annually on teacher professional development experiences designed to help teachers utilize their purchased instructional materials in their classrooms, 81.8% of building administrators who participated in the survey said they spend up to $25,000.00 However, it is important to note that this is a building-wide cost. It is not isolated to the funds spent only on professional development to support implementing Algebra I instructional materials in classrooms. As discussed in the conclusions for Research Question 1, the Culturally Responsive STEAM Scorecard found that teachers in Westmoreland County need more confidence in their ability to evaluate instructional resources. The scorecard also found IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 123 that they need more professional development on using their primary instructional materials effectively and more support in developing culturally responsive learning experiences, as teachers feel strongly that students better understand concepts and master skills when they see themselves in their instructional materials. These factors directly impact the instructional delivery of Algebra I in middle school classrooms. This may be influenced positively by implementing the Culturally Relevant and Sustaining Education framework. Research Question 3: “What are the teacher and administrator perceptions of the impact of instructional delivery on student achievement scores in Algebra I?” The third research question analyzed Teacher and Building Principal Perception Surveys data relating to instruction, instructional materials, and student achievement scores. The study design aimed to determine whether there was a correlation between the Standards Crosswalk results and the historical student achievement scores in Algebra I. Teachers believe that their instructional materials directly impact student achievement scores. Teachers are divided on whether the physical instructional materials they use with students adequately address the skills needed to prepare students for the Algebra I Keystone Exam. They are less confident that their digital instructional materials can do so. These results indicate that teachers are not confident in effectively analyzing instructional materials to identify standards and Eligible Content Items. It may also suggest that they cannot verify the accuracy of the standards documentation provided to them by content vendors. Teacher confidence levels may also vary across districts in Westmoreland County and may be influenced by factors related to Research Question 2. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 124 Teachers overwhelmingly believe students do better on assessments when they see themselves in their instructional materials. However, teachers acknowledge that their primary instructional materials are only sometimes culturally reflective of their entire student population. To address this, teachers must supplement their instruction with resources that they create other purchased instructional materials that are more culturally relevant. In doing so, teachers can add variety to how they introduce concepts and skills to students through their materials. However, the need to supplement their instruction with additional instructional materials means that teachers also need extra professional development time to curate or develop those materials. Due to the additional time required, teachers hesitate to implement supplemental instructional materials within their classrooms. Teachers may also utilize Open Educational Resources (OER) to develop instructional experiences that are more culturally reflective of their student population. Using OER in their classrooms enables teachers to modify resources to meet the needs of their students better. However, as we saw with additional purchased instructional materials, teachers are hesitant to implement OER in their classrooms. This is due to several factors, including a need for more understanding about OER and a lack of time to explore or have comprehensive professional development and ongoing support regarding OER initiatives within their schools. These findings suggest that the main reasons Open Educational Resources are not widely adopted in Algebra I classrooms in Westmoreland County are a need for more understanding about OER and a fear of being judged for using them. The current political climate makes teachers hesitant to explore materials beyond their approved IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 125 primary materials. Building administrators are also reluctant to encourage their use due to the current political environment. Both groups fear that the broad acceptance of OER and open discussion of culturally responsive pedagogy will open them up to criticisms from stakeholders and that these criticisms will detract from the positive benefits of both for students. At this point, instead of working to educate stakeholders, most building principals and teachers feel that it makes sense to find an amicable path forward to support students without causing additional strife among their communities. Overview of Results The overall research design aimed to investigate the relationship between low Keystone Algebra I Scores in middle schools in Westmoreland County, the perceptions of building principals and Algebra I teachers regarding instructional materials, training and support provided to teachers, and the cultural responsiveness of the materials. The researcher collected data from various sources, such as student assessment scores, district demographics, budget information, and surveys. The analysis revealed two key factors that may impact student achievement in Algebra I. The first key factor was that teachers felt their instructional materials lacked guidance and support for culturally responsive learning experiences. The second key factor was that the primary materials used by teachers did not adequately cover the content assessed in the Algebra I Keystone Exam, requiring teachers to supplement their instruction with additional materials from sources, including OER. Regarding instructional delivery, the survey data showed that teachers believed they had enough time and support to use the instructional resources when provided with professional development effectively. However, teachers needed to feel more confident IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 126 about developing culturally responsive experiences and identifying resource gaps. Regarding teacher and administrator perceptions, they recognized the impact of instruction and materials on student achievement but needed to analyze resources from a standards-alignment and culturally responsive perspective. There also needed to be a greater understanding of OER despite teachers regularly modifying materials to meet student needs. The most significant finding was the correlation between student achievement scores and the presence of Assessment Anchors and Eligible Content Items in the instructional materials used by Algebra I teachers. The identified case study districts, Greater Latrobe School District, Monessen City School District, and Yough School District, revealed a more detailed look into the results of this study. The following sections discuss specific results for each of the three case study districts, recommendations, and fiscal implications for each. Greater Latrobe School District The Greater Latrobe School District has been using Go Math by Houghton Mifflin Harcourt as their primary instructional resource for middle school Algebra I. A comprehensive analysis of this resource reveals that only 36.25% of the Algebra I Eligible Content Items are present in this textbook. With the use of this resource, plus the addition of supplemental materials that teachers both developed and found from external sources, 8th-grade student scores on the Algebra I Keystone Exam were in the following proficiency categories: ● Below Basic: 0% IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 127 ● Basic: 11.7% ● Proficient: 40.5% ● Advanced: 47.7% The teachers within the Greater Latrobe School District expressed a strong sense of satisfaction with the curriculum of their primary Algebra I instructional materials, emphasizing its strong alignment with Pennsylvania Core Academic Standards. Teachers were also content with the curriculum's rigor, feeling that the material presented to students is appropriately demanding and helps them to apply concepts and skills to further their understanding. They also feel that the materials help to prepare students for the Algebra I Keystone Exam. Teachers felt that their primary Algebra I instructional materials have the ability to represent diverse communities effectively. Moreover, the teachers were pleased with how the curriculum went beyond mere representation and actively incorporated the expertise of these diverse and historically marginalized communities. By integrating their knowledge and insights, the instructional materials provided a comprehensive understanding of algebraic concepts and served as a platform to elevate the voices and contributions of these communities. The teachers found this approach essential in fostering a classroom environment that values diversity and promotes inclusivity. Throughout the student-facing materials, the teachers felt that the curriculum consistently highlighted these communities' achievements, struggles, and unique perspectives. The teachers recognized the significance of this inclusive approach, as it empowered students to embrace diversity and recognize the contributions of historically marginalized people. Teachers expressed dissatisfaction with how their primary Algebra IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 128 I instructional materials approached essential concepts such as interdependence and continual change. They felt that the topics were not adequately addressed within the instructional materials, leaving gaps in students' understanding of the interconnectedness of mathematical concepts and their relevance in real-world scenarios. The teachers recognized the significance of teaching these concepts as they are crucial for developing critical thinking skills and preparing students for the complexities of the modern world. Teachers also found the instructional activities in the materials lacking when presenting alternatives or allowing students to explore and offer their own alternatives regarding the role of science, technology, engineering, arts, and math (STEAM) in existing social structures. A range of perspectives emerged among teachers regarding their satisfaction with the teachers' primary Algebra I instruction materials in addressing crucial aspects such as cultural responsiveness and real-life connections. While some teachers expressed high satisfaction, others felt that the materials fell short in these areas. Teachers recognize the importance of making real-life connections between academic content and the local neighborhoods, culture, environment, community issues, and current events and feel that their primary instructional materials do not give them the flexibility to do so. When they have materials that provide them with this flexibility, they can better foster meaningful engagement and relevance in the learning process, enabling students to see the direct applicability of Algebra I concepts in their own lives. Based on these results, the researcher recommends that the Greater Latrobe School District adopt a new primary instructional resource for their middle school Algebra I classrooms. During their resource evaluation process, the researcher suggests IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 129 Algebra I teachers and building administrators engage in a thorough standards-analysis review to determine which Algebra I Eligible Content Items are present within each resource being considered for adoption. In doing so, teachers will better understand the items not present in their final decision. The fiscal implication of this recommendation is that the district will need to incur the cost of purchasing a new textbook for their middle school Algebra I classes, which may be more than expected if this action falls outside their typical adoption cycle. The researcher also recommends that the Greater Latrobe School District partner with the Curriculum Services Team at the Westmoreland Intermediate Unit to engage in ongoing professional development for all staff around the effective and ongoing use of Open Educational Resources in their classrooms to support culturally responsive pedagogy. This training will help teachers and building administrators learn how to curate, modify, and remix resources to meet the needs of their student population. It will also help them develop their instructional skills to effectively represent diverse communities in their lessons and design lessons and materials that better represent the students within their classrooms while also aligning to content standards and Eligible Content Items. The fiscal implication of this recommendation is minimal. It may include minor upgrades to the technical infrastructure of the district to ensure that teachers and students can effectively utilize digital resources as needed. Finally, the researcher recommends that the Greater Latrobe School District utilizes a new primary Algebra I resource and supports the implementation of OER through ongoing professional development for multiple years to track the efficacy of these recommendations. The district will not see sustainable results with a one-year IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 130 implementation. Instead, they must commit to a systematic approach to professional development that supports teachers in learning to use new resources, develop their own resources, and grow their knowledge and skills. Adding an OER professional development initiative to the district’s Comprehensive Plan will help the district to commit to a long-term plan for growth and to see their efforts reflected in student test scores. Monessen City School District The Monessen City School District has been using Savvas’ enVision Algebra I program as their primary instructional resource for middle school Algebra I. A comprehensive analysis of this resource reveals that only 48.75% of the Algebra I Eligible Content Items are present in this textbook. With the use of this resource, plus the addition of supplemental materials that teachers both developed and found from external sources, 8th-grade student scores on the Algebra I Keystone Exam were in the following proficiency categories: ● Below Basic: 54.5% ● Basic: 27.3% ● Proficient: 18.2% ● Advanced: 0% Unfortunately, no perception or Culturally Responsive STEAM Scorecard data is available from the Monessen City School District. Both building administrators and Algebra I teachers did not complete the perception surveys or the scorecard. This means that the researcher is unable to determine if the teachers in the district with the most IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 131 diverse population of the three case study districts feel that their instructional materials are culturally responsive or if their instructional materials are doing an adequate job of addressing the standards and preparing their students for the Algebra I Keystone Exam. Based on the information that the researcher collected, the researcher can recommend that the Monessen City School District adopt a new primary instructional resource for their middle school Algebra I classrooms. The researcher recommends that the district consider an OER-based primary resource to provide teachers with a higher level of flexibility in addressing the wide variety of needs that the students within their classrooms have. This will also allow the district to reallocate funds typically used for resource adoption and instead dedicate them to additional professional development to better support teachers in understanding how to utilize OER to meet the needs of students effectively. As was recommended for the Greater Latrobe School District, the researcher recommends that the Monessen City School District partner with the Curriculum Services Team at the Westmoreland Intermediate Unit to engage in ongoing professional development for all staff around the effective and ongoing use of Open Educational Resources in their classrooms to support culturally responsive pedagogy. This training will help teachers and building administrators develop their instructional skills to effectively represent diverse communities in their lessons and design lessons and materials that better represent the students within their classrooms while also aligning to content standards and Eligible Content Items. The fiscal implication of this recommendation is minimal. It may include minor upgrades to the technical infrastructure of the district to ensure that teachers and students can effectively utilize digital resources IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 132 as needed. The researcher also recommends that the district commits to ongoing professional development for multiple years to track these recommendations' efficacy. Adding an OER professional development initiative to the district’s Comprehensive Plan will help the district to commit to a long-term plan for growth and to see their efforts reflected in student test scores. Yough School District The Yough School District has been using CK-12’s Algebra I flexbook as their primary instructional resource for middle school Algebra I. A comprehensive analysis of this OER-based resource reveals that 62.5% of the available Eligible Content items for Algebra I within their curriculum are in its original form before users modify it. With the use of this resource, plus the addition of supplemental materials that teachers both developed and found from external sources, 8th-grade student scores on the Algebra I Keystone Exam were in the following proficiency categories: ● Below Basic: 2.4% ● Basic: 39% ● Proficient: 43.9% ● Advanced: 14.6% Overall, the teachers' satisfaction with the curriculum of their primary Algebra I instructional materials reflected a commitment to inclusivity and equity within the Yough School District. By adequately representing and incorporating the expertise of diverse and historically marginalized communities, the curriculum promoted a positive and inclusive learning environment that valued the experiences and contributions of all IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 133 students. Teachers in the Yough School District expressed less satisfaction with how their primary Algebra I instructional materials tackled key concepts such as interdependence and continual change. They felt that these areas were not adequately addressed, leaving gaps in students' understanding of the interconnected nature of these concepts and their relevance in real-world contexts. The teachers believed that a comprehensive understanding of interdependence and continual change is essential for developing critical thinking skills and preparing students for the complexities of the modern world. Teachers were uncertain if the materials effectively encouraged students to analyze and challenge how STEAM subjects impact society critically. However, they acknowledge that they need more confidence to evaluate resources through this lens effectively. The teachers emphasized the need for instructional materials to provide more explicit guidance and resources for addressing interdependence, continual change, and the societal implications of STEAM. They believed that by encouraging students to explore alternatives and present their own ideas, the materials would foster a deeper understanding of these concepts and promote critical thinking skills. In the Yough School District, the Teachers' Materials category stood out as the only area where teacher opinions varied greatly. Some teachers expressed high satisfaction with the teachers' materials, appreciating their guidance on utilizing students' everyday lives as a foundation for learning. These teachers found value in the materials' emphasis on making real-life connections between academic content and various aspects of the local community, including neighborhoods, culture, environment, community issues, and current events. They also commended the materials for promoting IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 134 collaboration and leveraging individuals' unique assets, resources, and strengths to enhance group and individual learning experiences. Conversely, other teachers held reservations regarding Teachers' Materials. They felt that the guidance provided for teachers to be aware of their biases, assumptions, and cultural gaps between themselves and their students was unclear. These teachers believed that the materials did not adequately prompt teachers to reflect on their own practices and experiences in learning science, technology, engineering, arts, and math (STEAM) subjects through the lens of cultural responsiveness. The contrasting perspectives among teachers underscored the need for more explicit and comprehensive guidance within the Teachers' Materials category. Teachers in the Yough School District expressed satisfaction with the materials and resources associated with their primary Algebra I instructional materials. They appreciated that these materials offered students opportunities to engage with STEAM concepts using accessible technology and manipulatives and were pleased that the rigor of the curriculum was not contingent upon access to specific resources, materials, or technology that students and schools may not have. This ensured that all students could actively participate in and benefit from the curriculum's challenges and academic rigor regardless of their available resources. Based on these results, the researcher recommends that the Yough School District continue utilizing the OER-based CK-12 Algebra I flexbook with their middle school Algebra I classes. However, the researcher also suggests that the district leverage the resource's flexible nature and modify the flexbook's structure to better align the book to Pennsylvania Core Academic Standards by adding in the missing Eligible Content Items IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 135 from other instructional content within the CK-12 system. This will require teachers to understand where to find those items within the CK-12 system and know how to add them to their Algebra I flexbook to truly develop a personalized resource that matches the scope and sequence of instruction that Yough School District has adopted, is aligned to state standards, and covers all Eligible Content Items covered on the Algebra I Keystone Exam. Since CK-12 is an OER-based resource, there will be no additional cost to the district to enhance its already adopted resources. However, the researcher does recommend that the district reallocate funds to increase the amount of professional development and collaboration time for teachers to engage in this work. As with the Greater Latrobe School District and the Monessen City School District, the researcher also recommends that the Yough School District partner with the Curriculum Services Team at the Westmoreland Intermediate Unit to engage in ongoing professional development for all staff around the effective and ongoing use of Open Educational Resources in their classrooms to support culturally responsive pedagogy. This training will help teachers and building administrators learn how to curate, modify, and remix resources to meet the needs of their student population beyond the CK-12 system and introduce teachers and building administrators to several reputable repositories for OER. The fiscal implication of this recommendation is minimal. It may include minor upgrades to the technical infrastructure of the district to ensure that teachers and students can effectively utilize digital resources as needed. The researcher also recommends that the Yough School District include these recommendations in its Comprehensive Plan to engage in this work over multiple years. In doing so, both building principals and teachers can commit to actively engaging in the IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 136 professional development and collaboration needed to implement OER within the classroom in a sustainable manner effectively. This will also give the district time to see their work reflected in student test scores. Limitations The established research design and implementation resulted in one significant limitation, which impacted the results and findings. This significant limitation was the low participation numbers by teachers across Westmoreland County. This particularly impacted the results and findings for the Monessen City School District, as the researcher could not gather any perception data for that school district. At the start of the study, four Algebra I teachers from Monessen City School District indicated their willingness to participate. However, none of the teachers took any steps to sign the Informed Consent Letter, complete the Teacher Perception Survey, or the Culturally Responsive STEAM Scorecard. In addition, only six Algebra I teachers across the county were willing to engage in any of the activities for this study. As the results suggested, teachers are concerned about criticism from stakeholders regarding culturally responsive pedagogy. The researcher feels that the lack of participation may be due to the topic of this study, the use of the term culturally responsive, and fear over the current political climate in relation to that term. While not a limitation of the study itself, it is worth mentioning that the statewide adoption and implementation of the Culturally Relevant and Sustaining Education (CR-SE) guidelines and framework has been put in jeopardy during the study. Originally IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 137 passed into law on April 23, 2022, as part of Chapter 49, relating to the Certification of Professional Personnel, the law has been challenged several times. Most recently, several western Pennsylvania school districts have filed a lawsuit to prevent the implementation of the CR-SE guidelines. In addition, legislation was introduced to the House of Representatives Education Committee on May 31, 2023, proposing the abrogation of Chapter 49 regulations related to CR-SE. Should that legislation pass, it will go into effect immediately, eliminating the CR-SE guidelines and framework, preventing their use, and any training for teachers regarding culturally responsive pedagogy. While these developments did not directly impact the study itself, they may have an impact on the researcher's recommendations. Future Directions for Research The findings of this research warrant further study to gain a better understanding of the challenges building administrators and teachers face when working to bridge the gaps between the intended curriculum, the implemented curriculum, and the attained curriculum in order to positively influence student achievement. The researcher recommends that future researchers continue this work and seek out larger participant groups to gain a more comprehensive understanding of teachers' perceptions. Future researchers can achieve this by expanding the work beyond middle school, looking at math scores across transition years, and including all math teachers in the study. Doing so will allow participants to work together more collaboratively and engage in meaningful discussion while completing the Culturally Responsive STEAM Scorecard rather than in isolation. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 138 The researcher recommends replicating this study with other public school districts within Westmoreland County to understand better the resources being used across the county. This will also provide county administrators and teachers with a common critical analysis experience regarding their instructional materials and student test scores and can serve as the starting point for a countywide Algebra I community of practice to meet regularly to share resources, experiences, instructional practices, and to engage in problem-solving around common concerns related to Algebra I test scores in Westmoreland County. In replicating the study, the researcher recommends also looking at the assessments used by Algebra I teachers in their classrooms throughout the year, the alignment of those assessments to the Eligible Content Items on the Algebra I Keystone Exam, and the implementation of common assessments by teachers in the same districts. This will give future researchers better insight into how Algebra I teachers regularly assess students on their understanding and mastery of skills and concepts taught to students with the help of the primary instructional materials used by teachers throughout the school year as they prepare for the Algebra I Keystone Exam. Finally, the researcher recommends expanding this study into other content areas beyond mathematics. Doing so will test to see if the research design is effective and if the study applies to other content areas beyond mathematics. With the new adoption of the Science, Technology & Engineering, and Environmental Literacy & Sustainability (STEELS) Standards and the current focus that many districts have on finding new resources that align with new instructional strategies, this study may be well-suited to be IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 139 replicated for districts that are concerned with student test scores on the Biology Keystone Exam next. Summary To understand the root cause of students’ historically low test scores on the Algebra I Keystone Exam, the researcher developed a mixed-method embedded research design that examined the impact of the instructional materials used in Algebra I classrooms on student achievement. Chapter V presented the conclusion of this research by providing a comprehensive analysis of the findings for each of the three identified research questions. The chapter also included specific implications for the three case study districts identified within the research study, providing each study district with personalized recommendations and the fiscal implications for implementing those recommendations. Finally, Chapter V discussed the limitations of the research and recommendations for further study by future researchers in the hopes of continuing the work to support building administrators and teachers as they work to meet the needs of their individual student populations. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 140 References Abdulrahim, N., & Orosco, M. J. (2019, April 5). Culturally responsive mathematics teaching: A research synthesis. The Urban Review, 52(1), 1-25. https://doi.org/10.1007/s11256-019-00509-2 Ayers, D. F. (2006, June). Culturally responsive pedagogy and online learning: Implications for the globalized community college. Community College Journal of Research and Practice, 30(5-6), 401-415. https://doi.org/10.1080/10668920500442125 Brown, C., Boser, U., Sargrad, S., & Marchitello, M. (2016, January 29). Implementing the Every Student Succeeds Act toward a coherent, aligned assessment system. Center for American Progress. https://www.americanprogress.org/article/implementing-the-every-student-succee ds-act/ Brown, J. C., & Crippen, K. J. (2016, March 2). The Growing Awareness Inventory: Building capacity for culturally responsive science and mathematics with a structured observation protocol. School Science and Mathematics, 116(3), 127-138. https://doi.org/10.1111/ssm.12163 Chingos, M. M., & Whitehurst, G. J. ". (2012, April). Choosing blindly: Instructional materials, teacher effectiveness, and the Common Core. Brookings. https://www.brookings.edu/research/choosing-blindly-instructional-materials-teac her-effectiveness-and-the-common-core/ IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 141 Cole-Malott, D.-M., Parker Thompson, K., Peterson-Ansari, R., & Whitaker, II, R. W. (2021, April). The Pennsylvania Culturally Relevant and Sustaining Education Competencies. https://s3.amazonaws.com/newamericadotorg/documents/The_Pennsylvania_Cult urally_Relevant_and_Sustaining_Education_Competencies.pdf Copur‑Gencturk, Y., Jacobson, E., & Rasiej, R. (2021, February 18). On the alignment of teachers’ mathematical content knowledge assessments with the common core state standards. Journal of Mathematics Teacher Education, 25, 267–291. https://doi.org/10.1007/s10857-021-09486-4 Dorn, E., Hancock, B., Sarakatsannis, J., & Viruleg, E. (2020, June 1). COVID-19 and student learning in the United States: The hurt could last a lifetime. McKinsey. https://www.mckinsey.com/industries/education/our-insights/covid-19-and-studen t-learning-in-the-united-states-the-hurt-could-last-a-lifetime Elementary and Secondary Education Act. (1965, April 11). 20 U.S.C. § 6301. https://www.congress.gov/89/statute/STATUTE-79/STATUTE-79-Pg27.pdf eMetric. (2023). Data Interaction for Pennsylvania Student Assessments. eMetric Data Interaction. https://pa.emetric.net Every Student Succeeds Act. (2015, December 10). 20 U.S.C. § 6301. https://www.congress.gov/114/plaws/publ95/PLAW-114publ95.pdf Garland, S. (2020, March 30). Who was behind the Common Core math standards, and will they survive? The Hechinger Report. https://hechingerreport.org/behind-common-core-math-standards-will-survive/ IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 142 Garth, R., & Sebastian, R. (2020, November 18). Using open educational resources to create a culturally relevant classroom. Achieving the Dream. https://achievingthedream.org/using-open-educational-resources-to-create-a-cultur ally-relevant-classroom/ Gay, G. (2013, January). Teaching to and through cultural diversity. Curriculum Inquiry, 43(1), 48-70. https://doi.org/10.1111/curi Greater Latrobe School District. (2022). District Information – About Us. https://www.glsd.us/apps/pages/index.jsp?uREC_ID=1756118&type=d&pREC_I D=1937431 Hamilton, L. S., Stecher, B. M., & Yuan, K. (2008, December). Standards-based reform in the United States: History, research, and future directions. Institute of Education Sciences. https://www.rand.org/content/dam/rand/pubs/reprints/2009/RAND_RP1384.pdf Harmon, D. A. (2012, Spring). Culturally responsive teaching though a historical lens: Will history repeat itself? Interdisciplinary Journal of Teaching and Learning, 2(1), 12-22. Heise, M. (2017). From No Child Left Behind to Every Student Succeeds: Back to a future for education federalism. Columbia Law Review, 117(7), 1859-1896. https://columbialawreview.org/content/from-no-child-left-behind-to-every-student -succeeds-back-to-a-future-for-education-federalism/ Hendricks, C. (2017). Improving schools through action research: A reflective practice approach. Pearson. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 143 Improving America's Schools Act. (1994, October 20). 20 U.S.C. § 6301. https://www.congress.gov/103/statute/STATUTE-108/STATUTE-108-Pg3518.pdf Jacques, I. (2022, October 24). NAEP math, reading scores: Report card shows how school closures hurt. USA Today. https://www.usatoday.com/story/opinion/columnist/2022/10/24/report-card-studen t-test-scores-reveal-how-much-closing-schools-hurt/10551234002/ Johansson, M. (2003, November). Textbooks in mathematics education A study of textbooks as the potentially implemented curriculum. Luleå University of Technology. Katz, V. J. (2009). A history of mathematics : an introduction. Addison-Wesley. Kaufman, J. H., Doan, S., & Fernandez, M.-P. (2021). The rise of standards-aligned instructional materials for U.S. k–12 mathematics and english language arts instruction [Findings from the 2021 American Instructional Resources Survey]. RAND Corporation. https://www.rand.org/pubs/research_reports/RRA134-11.html Lincoln, Y. S., & Guga, E. (1985). Naturalistic inquiry. Sage. Loeb, H., Knapp, M. S., Elfers, A. M., & University of Washington. (2008, April 21). Teachers’ response to standards-based reform: Probing reform assumptions in Washington state. Education Policy Analysis Archives, 16(8). https://doi.org/10.14507/epaa.v16n9.2008 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 144 Matthews, J. S., & López, F. (2019). Speaking their language: The role of cultural content integration and heritage language for academic achievement among Latino children. Contemporary Educational Psychology. https://doi.org/10.1016/j.cedpsych.2018.01.005 Mehta, J. (2015, Summer). Escaping the shadow: A nation at risk and its far-reaching influence. American Educator, 39(2), 20-26. Mertler, C. A. (2021). Introduction to educational research. SAGE Publications, Incorporated. Molchanow, K. (2022, April 23). Pennsylvania bulletin (17, 52nd ed.). PA Code and Bulletin. https://www.pacodeandbulletin.gov/secure/pabulletin/data/vol52/52-17/52-17.pdf Mollenkamp, D. (2022, October 24). NAEP ‘nation’s report card’ shows steep fall in math scores. EdSurge. https://www.edsurge.com/news/2022-10-24-naep-nation-s-report-card-shows-stee p-fall-in-math-scores Monessen City School District. (2023). About Us – District – Monessen City SD. https://www.monessenschooldistrict.com/apps/pages/index.jsp?uREC_ID=13007 04&type=d&pREC_ID=1504939 Muñiz, J. (2019, March 28). Culturally Responsive Teaching. New America. https://www.newamerica.org/education-policy/reports/culturally-responsive-teach ing/ IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 145 National Assessment of Educational Progress. (2022). 2022 Mathematics and Reading Report Cards at Grades 4 and 8. The Nation's Report Card. https://www.nationsreportcard.gov/ National Center for Education Statistics. (1995). Pursuing excellence: A study of U.S. fourth-grade mathematics and science achievement in international context. https://nces.ed.gov/timss/results95.asp National Commission on Excellence in Education. (1983). A nation at risk: The imperative for educational reform. U.S. Government Printing Office. National Governors Association Center for Best Practices & Council of Chief State School Officers. (2022, October 31). Common Core State Standards for Mathematics. Council of Chief State School Officers. https://ccsso.org/resource-library/common-core-state-standards-mathematics No Child Left Behind Act. (2002, January 8). 20 U.S.C. § 6301. https://www.congress.gov/107/plaws/publ110/PLAW-107publ110.pdf Owens, J. E. (1988, August 9). Curriculum and evaluation standards for school mathematics: Report of the national council of teachers of mathematics' commission on standards for school mathematics. Institute of Education Sciences. Pennsylvania Association of Intermediate Units. (2022). Focus. PAIU. https://paiu.org/Focus IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 146 Pennsylvania Department of Education. (2013). Fact sheet: PA Academic Standards and the Common Core. https://www.sasd.k12.pa.us/Downloads/FACT%20SHEET%20for%20PA%20Aca demic%20Standards%20and%20the%20Common%20Core%20April%202013.pd f Pennsylvania Department of Education. (2022). Future Ready PA Index district fast facts - Greater Latrobe SD. https://futurereadypa.org/District/FastFacts?id=195212081008215029032253089 092020162089031176052 Pennsylvania Department of Education. (2023). Future Ready PA Index. https://www.education.pa.gov/K-12/ESSA/FutureReady/Pages/default.aspx Pennsylvania Department of Education, Pennsylvania Educator Diversity Consortium, & U.S. Department of Education's Region 4 Comprehensive Center. (2021, April). The Pennsylvania Culturally Relevant and Sustaining Education Competencies. https://s3.amazonaws.com/newamericadotorg/documents/The_Pennsylvania_Cult urally_Relevant_and_Sustaining_Education_Competencies.pdf Pennsylvania Joint State Government Commission. (1997, December). Intermediate units in Pennsylvania: The role of educational service agencies in promoting equity in basic education. Pennsylvania State Board of Education. (2010). Implementing the Common Core Standards in Pennsylvania: Reflections from the field. Pennsylvania Department of Education. IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 147 https://www.stateboard.education.pa.gov/Documents/Research%20Reports%20an d%20Studies/Common%20Core%20White%20Paper.pdf Petrin, K. (2022, October 23). NAEP scores show big drop in wake of COVID. Chalkbeat. https://www.chalkbeat.org/2022/10/24/23417139/naep-test-scores-pandemic-scho ol-reopening Smith, D. R., & Ayers, D. F. (2006, June). Culturally responsive pedagogy and online learning: Implications for the globalized community college. Community College Journal of Research and Practice, 30, 401-415. https://doi.org/10.1080/10668920500442125 Sparks, S. D., & Tomko, G. (2022, October 24). Two decades of progress, nearly gone: National math, reading scores hit historic lows. Education Week. https://www.edweek.org/leadership/two-decades-of-progress-nearly-gone-national -math-reading-scores-hit-historic-lows/2022/10 Superfine, B. M. (2009). The courts and standards based reform. Oxford University Press, USA. Tampio, N. (2018). Common Core: National education standards and the threat to democracy. Johns Hopkins University Press. Tang, H. (2020, September). A qualitative inquiry of k–12 teachers’ experience with open educational practices: Perceived benefits and barriers of implementing open educational resources. International Review of Research in Open and Distributed Learning, 21(3), 211-229. https://doi.org/10.19173/irrodl.v21i3.4750 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 148 Tang, H., Lin, Y.-J., & Qian, Y. (2020, April). Understanding K-12 teachers’ intention to adopt open educational resources: A mixed methods inquiry. British Journal of Educational Technology, 51(6), 2558–2572. https://doi.org/10.1111/bjet.12937 UNESCO. (n.d.). Open educational resources. UNESCO.org. Retrieved November 12, 2022, from https://www.unesco.org/en/communication-information/open-solutions/open-educ ational-resources United States Department of Education. (2008). A nation accountable: Twenty-five years after a nation at risk. U.S. Department of Education. https://www2.ed.gov/rschstat/research/pubs/accountable/accountable.pdf Watt, M. G. (2011). The Common Core State Standards Initiative: An Overview. Institute of Education Services. Westmoreland Intermediate Unit. (n.d.). Curriculum Services General Information Curriculum - Westmoreland IU 7. Westmoreland Intermediate Unit. https://www.wiu7.org/apps/pages/index.jsp?uREC_ID=1559997&type=d&pREC _ID=1605947 Xie, K., Nelson, M. J., Cheng, S.-L., & Jiang, Z. (2021, August). Examining changes in teachers’ perceptions of external and internal barriers in their integration of educational digital resources in K-12 classrooms. Journal of Research on Technology in Education. https://doi.org/10.1080/15391523.2021.1951404 Yough School District. (2020). General Information/Alma Mater. Yough School District. https://www.youghsd.net/general_informationalma_mater IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Appendix A 149 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 150 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 151 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 152 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Appendix B 153 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 154 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 155 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 156 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 157 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 158 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Appendix C 159 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS 160 IMPACT OF CULTURALLY RESPONSIVE INSTRUCTIONAL MATERIALS Appendix D 161