Reports 732-723

#732 – Issues in Assessing English Language Learners: English Language Proficiency Measures and Accommodation Uses--Practice Review
Mikyung Kim Wolf, Jenny Kao, Noelle Griffin, Joan L. Herman, Patina L. Bachman, Sandy M. Chang, Tim Farnsworth

Summary

The No Child Left Behind (NCLB) has had a great impact on states’ policies in assessing English language learner (ELL) students. The legislation requires states to develop or adopt sound assessments in order to validly measure the ELL students’ English language proficiency, as well as content knowledge and skills. While states have moved rapidly to meet these requirements, they face challenges to validate their current assessment and accountability systems for ELL students, partly due to the lack of resources. Considering the significant role of an assessment in guiding decisions about organizations and individuals, it is of paramount importance to establish a valid assessment system. In light of this, we reviewed the current literature and policy regarding ELL assessment in order to inform practitioners of the key issues to consider in their validation process. Drawn from our review of literature and practice, we developed a set of guidelines and recommendations for practitioners to use as a resource to improve their ELL assessment systems. We have compiled a series of three reports. The present report is the second component of the series, providing a comprehensive picture of states’ current policies related to ELL assessment. The areas reviewed include the procedures of ELL identification and redesignation, the characteristics of English language proficiency assessments, including validity information, and the use of accommodations in the assessment of content knowledge.

#731 – Issues in Assessing English Language Learners: English Language Proficiency Measures and Accommodation Uses--Literature Review
Mikyung Kim Wolf, Jenny Kao, Joan L. Herman, Lyle F. Bachman, Alison L. Bailey, Patina L. Bachman, Tim Farnsworth, Sandy M. Chang

Summary

The No Child Left Behind (NCLB) Act has made a great impact on states’ policies in assessing English language learner (ELL) students. The legislation requires states to develop or adopt sound assessments in order to validly measure the ELL students’ English language proficiency (ELP), as well as content knowledge and skills. Although states have moved rapidly to meet these requirements, they face challenges to validate their current assessment and accountability systems for ELL students, partly due to the lack of resources. Considering the significant role of assessments in guiding decisions about organizations and individuals, it is of paramount importance to establish a valid assessment system. In light of this, we reviewed the current literature and policy regarding ELL assessment in order to inform practitioners of the key issues to consider in their validation processes. Drawn from our review of literature and practice, we developed a set of guidelines and recommendations for practitioners to use as a resource to improve their ELL assessment systems. We have compiled a series of three reports. The present report is the first component of the series, containing pertinent literature related to assessing ELL students. The areas being reviewed include validity theory, the construct of ELP assessments, and the effects of accommodations in the assessment of ELL students’ content knowledge.

#730 – Creating Accurate Science Benchmark Assessments to Inform Instruction
Terry P. Vendlinski, Sam Nagashima, Joan L. Herman

Summary

Current educational policy highlights the important role that assessment can play in improving education. State standards and the assessments that are aligned with them establish targets for learning and promote school accountability for helping all students succeed; at the same time, feedback from assessment results is expected to provide districts, schools, and teachers with important information for guiding instructional planning and decision making. Yet even as No Child Left Behind (NCLB) and its requirements for adequate yearly progress put unprecedented emphasis on state tests, educators have discovered that annual state tests are too little and too late to guide teaching and learning. Recognizing the need for more frequent assessments to support student learning, many districts and schools have turned to benchmark testing—periodic assessments through which districts can monitor students’ progress, and schools and teachers can refine curriculum and teaching—to help students succeed. We report in this document a collaborative effort of teachers, district administrators, professional developers, and assessment researchers to develop benchmark assessments for elementary school science. In the sections which follow we provide the rationale for our work and its research question, describe our collaborative assessment development process and its results, and present conclusions.

#729 – Predictive Validity of an English Language Arts Performance Assessment
Jia Wang, David Niemi, Haiwen Wang

Summary

The main goal of this report is to present evidence on the predictive validity of an English language arts (ELA) performance assessment (PA) administered in Grades 2–9 in a large urban school district. To account for the hierarchical structure of the data (students are nested within schools), we employed hierarchical linear modeling (HLM) to distinguish individual and aggregated explanatory variables. Based on a sub-sample of 5,427 students, we found that students’ 2001 ELA PA scores were predictive of their probability of passing the California High School Exit Exam (CAHSEE). We also found a significant correlation between student performances on the ELA performance assessment and other standardized tests. We believe that the ELA PA may be a dependable and useful indicator to identify at-risk students.

#728 – Accountability and Assessment: Is Public Interest in K-12 Education Being Served?
Joan L. Herman

Summary

The reauthorization of No Child Left Behind (NCLB) makes this a good time to consider whether and how current accountability serves the public interest and whether and how it can better do so. This report explores these issues in the context of the current literature on the effects of accountability in K–12 education. It considers the meaning of “public interest” and offers a model of how public interest may be served through accountability to benefit student learning. The report considers how well the model fits available evidence by examining whether and how accountability assessment influences students’ learning opportunities and the relationship between accountability and learning.

#727 – Developing Academic English Language Proficiency Prototypes for 5th Grade Reading: Psychometric and Linguistic Profiles of Tasks
Alison L. Bailey, Becky H. Huang, Hye Won Shin, Tim Farnsworth, Frances A. Butler

Summary

Within an evidentiary framework for operationally defining academic English language proficiency (AELP), linguistic analyses of standards, classroom discourse, and textbooks have led to specifications for assessment of AELP. The test development process described here is novel due to the emphasis on using linguistic profiles to inform the creation of test specifications and guide the writing of draft tasks. In this report, we outline the test development process we have adopted and provide the results of studies designed to turn the drafted tasks into illustrative prototypes (i.e., tried out tasks) of AELP for the 5th grade. The tasks use the reading modality; however, they were drafted to measure the academic language construct and not reading comprehension per se. That is, the tasks isolate specific language features (e.g., vocabulary, grammar, language functions) occurring in different content areas (e.g., mathematics, science and social studies texts). Taken together these features are necessary for reading comprehension in the content areas. Indeed, students will need to control all these features in order to comprehend information presented in their textbooks. By focusing on the individual language features, rather than the subject matter or overall meaning of a text, the AELP tasks are designed to help determine whether a student has sufficient antecedent knowledge of English language features to be able to comprehend the content of a text.

#726 – The Role of Teacher Discourse in Effective Groupwork
Noreen M. Webb, Megan L. Franke, Marsha Ing, Angela Chan, Tondra De, Deanna Freund, Dan Battey

Summary

Prior research on collaborative learning identifies student behaviors that significantly predict student achievement, such as giving explanations of one’s thinking. Less often studied is how teachers’ instructional practices influence collaboration among students. This report investigates the extent to which teachers engage in practices that support students’ explanations of their thinking, and how these teacher practices influence the nature of explanations that students give when asked by the teacher to collaborate with each other. In this study, we videotaped and audiotaped teacher and student participation, and measured student achievement, in second- and third-grade mathematics classrooms working on algebraic concepts of equality and relational thinking. The teachers observed here, all of whom received specific instruction in eliciting the details of student thinking, varied significantly in the extent to which they asked students to elaborate on their suggestions. This variation corresponded strongly to variation across classrooms in the nature and extent of student explanations during collaborative conversations, and to differences in student achievement.

#725 – Eliciting Student Thinking in Elementary School Mathematics Classrooms
Megan L. Franke, Noreen M. Webb, Angela Chan, Dan Battey, Marsha Ing, Deanna Freund, Tondra De

Summary

The importance of student talk in mathematics classrooms figures prominently in curriculum and teaching standards. Student talk is a vehicle for increasing student learning and for helping teachers monitor student understanding and inform student instructional practices. Although researchers have begun to study the moves teachers may make to support students in making their mathematical thinking explicit, sharing out with others and using it as the basis of conversation, much remains to be known about the teacher practices that help students clarify and communicate their mathematical thinking. To learn more about these teacher practices, we look closely at what teachers say and do as they engage with their students in mathematical conversation and how students participate in relation to what teachers say and do. In this report we examine the questions teachers ask and how those questions support students to detail their mathematical thinking. Although all teachers in this study asked students to explain how they solved problems, an important teacher practice for encouraging further student elaboration and giving complete and correct explanations was asking further questions about specific aspects of students’ answers or explanations. We describe the variety of teacher questioning practices and the differences in patterns of student participation that emerged.

#724 – Recommendations for Building a Valid Benchmark Assessment System: Second Report to the Jackson Public Schools
David Niemi, Jia Wang, Haiwen Wang, Julia Vallone, Noelle Griffin

Summary

There are usually many testing activities going on in a school, with different tests serving different purposes, thus organization and planning are key in creating an efficient system in assessing the most important educational objectives. In the ideal case, an assessment system will be able to inform on student learning, instruction and curricula, and district and school administration, as well as providing information to identify and solve educational problems. This report represents the second of two deliverables provided to the Jackson Public Schools (JPS) and provides recommendations, based on ongoing discussions with the district and review of information from the benchmark assessments, on topics related to building a valid benchmark assessment system. We consider how item data can be used to improve benchmark tests over time and also cover what needs to be done to insure that results of a test are correctly interpreted, reported and used.

#723 – Recommendations for Building a Valid Benchmark Assessment System: Interim Report to the Jackson Public Schools
David Niemi, Julia Vallone, Jia Wang, Noelle Griffin

Summary

Many districts and schools across the U. S. have begun to develop and administer assessments to complement state testing systems and provide additional information to monitor curriculum, instruction and schools. In advance of this trend, the Jackson Public Schools (JPS) district has had a district benchmark testing system in place for many years. To complement and enhance the capabilities of district and school staff, the Stupski Foundation and CRESST (National Center for Research on Evaluation, Standards, and Student Testing at UCLA) worked out an agreement for CRESST to provide expert review and recommendations to improve the technical quality of the district’s benchmark tests. This report (which represents the first of two deliverables on this project) focuses on assessment development and is consistent with the district goal of increasing the predictive ability of the assessments for students’ state test performance, as well as secondary goals.

Like this Product? Get CRESST News Every Month!

Reports

Download Our Reports

Partners & Collaborators