National Center for Research on Evaluation, Standards and Student Testing

#541 – Short Circuits or Superconductors? Examining Factors That Encourage or Undermine Group Learning and Collaboration Among High-Ability Students
Noreen Webb, Kariane Mari Welner, and Stephen Zuniga

Summary

Peer-directed small-group collaboration is featured prominently in debates about good classroom instruction and in the promotion of school reform. Although many cooperative learning methods advocate grouping students heterogeneously in order to maximize the diversity of perspectives, skills, and backgrounds, past research shows that while heterogeneous grouping generally benefits low-ability students, it does not necessarily benefit high-ability students. This study investigates the effects of group ability composition (homogeneous vs. heterogeneous) on group processes and outcomes for high-ability students completing science performance assessments. The results show that group ability composition does not have straightforward effects on achievement. While high-ability students working in homogeneous groups uniformly performed well, high-ability students in some heterogeneous groups performed better than high-ability students in other heterogeneous groups. The quality of group functioning served as the strongest predictor of high-ability students' achievement. High-ability students in groups that were responsive to group members' need for help and did not engage in debilitating social behavior performed well, whereas high-ability students in groups with poorer functioning did not. Whereas homogeneous groups consistently showed beneficial group functioning, only some heterogeneous groups exhibited these traits. These results show that achievement of high-ability students cannot be predicted from a simple homogeneous-heterogeneous grouping contrast and that the level of groupfunctioning provides the key to understanding group performance.

#765 – Connecting Policy to Practice: Accommodations in States' Large-Scale Math Assessments for English Language Learners
Mikyung Kim Wolf, Noelle Griffin, Jenny C. Kao, Sandy M. Chang, Nichole M. Rivera

Summary

Accommodations have been widely utilized as a way of increasing the validity of content assessments for ELL students. However, concerns have also arisen regarding the validity of accommodation use, as well as accessibility and fairness. While many states have developed ELL-specific accommodation policies and guidelines, little research has been available on how the accommodation policies are carried out in practice. The present study investigated two states' accommodation policies, specifically for the states' respective large-scale Grade 8 math assessments, and conducted a case study to examine teachers' understanding of the policies and uses of accommodations in their respective schools. Results indicated a wide variation in applying the policies in practice, which raises a validity concern for providing accommodations and interpreting accommodated test results. Based on the findings, implications and recommendations for an appropriate use of accommodations are offered.

To cite from this report, please use the following as your APA reference:

Wolf, M. K., Griffin, N., Kao, J. C., Chang, S. M., & Rivera, N. M. (2009). Connecting policy to practice: Accommodations in states' large-scale math assessments for English language learners (CRESST Report 765). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

#786 – IES Teacher Assignment Final Report
Yael Silk, David Silver, Stephanie Amerian, Claire Nishimura, and Christy Boscardin

Summary

The goal of this study was to test the effectiveness of WestEd¹s Reading Apprenticeship (RA) professional development program on teacher practices and student learning. The professional development is designed to teach high school teachers how to integrate subject-specific literacy instruction into their regular curricula. The CRESST researchers found that history and biology treatment teachers significantly outscored control teachers on three dimensions (reading comprehension strategies, metacognitive processes, and collaborative meaning making). History treatment teachers outperformed control teachers on an additional three dimensions (reading opportunities, support for reading engagement, and student feedback); biology treatment teachers scored higher on the adjusting instruction measure as compared to control teachers.

#350 – The Vermont Portfolio Assessment Program: Interim Report on Implementation and Impact, 1991-1992 School Year
Daniel Koretz, Brian Stecher, Edward Deibert

Summary

Vermont is the first state to make portfolios the backbone of a statewide assessment system. Daniel Koretz, Brian Stecher, and Edward Deibert, the authors of this CRESST/RAND report, have been evaluating the Vermont portfolio program for almost two years. The researchers found that support for the Vermont portfolio program, despite tremendous demands on teacher time, is widespread. "Perhaps the most telling sign of support for the Vermont portfolio program," write the authors, "is that [even in the pilot year] the portfolio program had already been extended beyond the grades targeted by the state." An interesting instructional phenomenon was that over 80% of the surveyed teachers in the Vermont study indicated that they had changed their opinion of students' mathematical abilities based upon their students' portfolio work. In many cases, teachers noted that students did not perform as well on the portfolio tasks as on previous classroom work. This finding, supported by other performance assessment research, suggests that portfolios may give teachers another assessment tool that appears to broaden their understanding of student achievement.

#360 – Raising the Stakes of Test Administration: The Impact on Student Performance on NAEP
Vonda Kiplinger and Robert Linn

Summary

The National Assessment of Educational Progress (NAEP) test has been accused of underestimating student achievement because this "low-stakes" assessment has no consequences for students, their teachers, or their schools. In contrast, "high-stakes" tests--those assessments that have serious consequences for students, teachers, and schools--are assumed to motivate greater student performance because of the positive or negative consequences (such as college entrance) associated with student performance on the test. The purpose of this study was to investigate whether differences in test administration conditions and presumed levels of motivation created by the different testing environments affect student performance on the NAEP test. The testing conditions studied were the "low-stakes" environment of the current NAEP administration and a "higher-stakes" environment typified by many state assessment programs. The results of the study lead to the conclusion that estimates of achievement from NAEP would not be substantially higher if the stakes were increased to the level associated with a "higher-stakes" test.

#490 – Assessments and Accountability
Robert Linn

Summary

Uses of tests and assessments as key elements in five waves of educational reform during the past 50 years are reviewed. These waves include the role of tests in tracking and selection emphasized in the 1950s, the use of tests for program accountability in the 1960s, minimum competency testing programs of the 1970s, school and district accountability of the 1980s, and the standards-based accountability systems of the 1990s. Questions regarding the impact, validity, and generalizability of reported gains and the credibility of results in high-stakes accountability uses are discussed. Emphasis is given to three issues of currently popular accountability systems. These are (a) the role of content standards, (b) the dual goals of high performance standards and common standards for all students, and (c) the validity of accountability models. Some suggestions for dealing with the most severe limitations of accountability are provided.

#824 – Evaluation of Green Dot’s Locke Transformation Project: From the Perspective of Teachers and Administrators
Joan Herman, Jia Wang, Christine Ong, Rolf Straubhaar, Jon Schweig, and Vivian Hsu

Summary

In the fall of 2007, Alain Leroy Locke High School, historically one of California’s lowest performing secondary schools, underwent a transformation. Suffering from a history of extremely low academic performance, student unrest and even violence, the nonprofit charter organization Green Dot Public Schools was charged to transition Locke into a set of smaller charter academies, in partnership with the Los Angeles Unified School District (LAUSD).With a grant from the Bill and Melinda Gates Foundation, the National Center for Research on Evaluation, Standards and Student Testing (CRESST), was charged with monitoring the progress and effects of Green Dot Public Schools’ Locke transformation from 2007 to the present. Previous annual reports have presented findings related to the academic performance of Green Dot Locke (GDL) students. The primary focus of this current report, is to use both quantitative (including teachers’ value-added data based on state test scores) and qualitative data (interviews with 13 teachers and four administrators across GDL academies) to explore potential teacher factors influencing students’ academic progress since the transformation, particularly focusing on teacher recruitment/selection, retention, and support.

#686 – An Investigation of Language-Minority Children: Demographic Characteristics, Initial Performance, and Growth in Achievement
Douglas Ready and Gerald Tindal

Summary

Research on language-minority (LM) children has generally focused on language as the primary mediator of student achievement without considering other student demographic characteristics. This is unfortunate, as studies that approach languageminority children as a homogeneous group will misestimate relationships between language status and academic achievement. Moreover, extant research is often hampered by its lack of focus on language-minority students’ cognitive growth over time. Using data from the Early Childhood Longitudinal Study, Kindergarten Cohort of 1998-1999 (ECLS-K) and growth curve analyses within a three-level hierarchical framework, this report examines the academic skills of LM children as they enter kindergarten and progress through first grade. This report defines language-minority children as those having a primary home language other than English. A further distinction is made between language-minority children who are English proficient (LM/P) and those who are not proficient (LM/NP), although examinations of LM/NP children using the regular ECLS-K cognitive assessments are limited to mathematics and Hispanic (Spanish-speaking) LM/NP children. Descriptive findings stress the diverse socio-demographic and academic backgrounds of language-minority children. Many LM children have highly-educated and affluent parents, while others come from families with few social or economic resources. Some LM children enter school with well-developed literacy and numeracy skills, while others exhibit few initial academic competencies. Despite this considerable variability, one theme that permeates this report is the socioeconomic and academic disadvantage among Hispanic LM children. Unsurprisingly, Hispanic LM/P children enter kindergarten with fewer English skills than non-LM children. However, Hispanic LM/P children’s initial literacy skills also lag behind those of other LM/P children. Much of this initial disadvantage is explained by the relative socioeconomic disadvantage of Hispanic LM/P children. Not coincidentally, non-Hispanic LM/P students and their non-LM peers share similar 2 socioeconomic and academic backgrounds. In terms of literacy learning, during kindergarten, Asian LM/P children erase the small literacy gap that separates them from non-LM children. The Hispanic LM/P literacy disadvantage, however, remains relatively constant during kindergarten and first grade and actually increases during the intervening summer months. As with literacy, Hispanic LM/P children enter kindergarten with fewer mathematics skills than their non-LM peers. An even larger mathematics skills gap separates Hispanic LM/NP and non-LM children. For both groups, a substantial proportion of these initial achievement differences can be explained by their socioeconomic disadvantage compared to non-LM children. Although Hispanic LM/P children gain mathematics skills at rates comparable to Hispanic non-LM children, Hispanic LM/NP children fall even further behind during kindergarten (but learn at similar rates during the summer and first grade). Non-Hispanic LM/P children enter kindergarten and end first grade with mathematics skills equal to non-LM children.

#445 – Group Discussion and Large-Scale Language Arts Assessment: Effects on Students' Comprehension
Randy Fall, Noreen Webb, and Naomi Chudowsky

Summary

Large-scale assessment programs are beginning to design group assessment tasks in which small groups of students collaborate to solve problems or complete projects. Little is known, however, about the effects of collaboration on students' cognitive processes and performance on such tests. The present study compared student performance on language arts tests in which they either were or were not permitted to discuss the story they were required to read and interpret. The analyses compared the quality of student responses on test forms with and without collaboration, examined qualitative changes in students' responses before and after collaboration, and examined students' reflections about the impact of collaboration on their understanding of the story. The results show that a 10-minute discussion of the story in three-person groups had a substantial impact on student performance. Implications for the design and interpretation of large-scale testing with collaboration are discussed.

#371 – Can Portfolios Assess Student Performance and Influence Instruction? The 1991-92 Vermont Experience
Daniel Koretz, Brian Stecher, Stephen Klein, Daniel McCaffrey, and Edward Deibert

Summary

Vermont's statewide assessment initiative program has garnered widespread attention nationwide because of its reliance on portfolios of student work. This 145 page report describes the results of a multifaceted evaluation of the program and provides information about the implementation of the Vermont assessment; program effects on educational practice; reliability and validity of portfolio scores; and tensions that exist between assessment and instructional reform. "Findings from the evaluation," said the research team, "suggest that the assessment program resulted in changes in curriculum content and instructional style." Additionally, the researchers noted that the amount of classroom time devoted to problem solving increased, as did the amount of time students worked in small groups. Finally, portfolios seem to increase teachers' enthusiasm for their subjects and for teaching. While there was widespread support for the reform at the school level throughout the state--nearly one-half of the schools were voluntarily expanding the use of portfolios to other grade levels--substantial problems remained. The mathematics portfolio assessment created new burdens for principals, teachers and students; including demands on teachers' time and school resources. Over 80% of fourth-grade teachers and over 60% of eighth-grade teachers reported that they often had difficulty covering the required curriculum. Researchers anticipate that some of these demands are likely to decline with experience, although others represent continuing burdens. "The Vermont experience has important implications for reforms that are underway or under consideration in other jurisdictions," wrote the researchers, "but only time and careful scrutiny will show how fully the goals of the program--and of similar reform programs centered on performance assessment--can be met."

Like this Product? Get CRESST News Every Month!

Reports

Download Our Reports

Partners & Collaborators