Reports
Please note that CRESST reports were called "CSE Reports" or "CSE Technical Reports" prior to CRESST report 723.
#441 – Alternative Assessment Programs: What Are the True Costs? An Analysis of the Total Costs of Assessment in Kentucky and Vermont
Lawrence O. Picus and Alisha Tralli
Lawrence O. Picus and Alisha Tralli
CSE Report 441, 1998 
Summary
Summary
At the March 1996 education summit in New York, President Clinton, the nation's governors, and a select group of chief executive officers from large companies agreed that a critical component of education reform was the need to hold schools accountable for student learning. Although the exact methods for doing this were not specified, it seems certain that a major component of this effort will include some form of assessment. While it is doubtful that we will see a national standardized test, the importance of assessment programs will continue to grow in the foreseeable future.
The purpose of this report is to provide a first detailed analysis of the "economic" or opportunity costs of the testing systems in two states, Kentucky and Vermont. Using a framework developed by Picus (1994), this study looked closely at the amount of time local school officials spent supporting the assessment programs in their respective states in 1995-96, and estimates the value of that effort. As the results show, when the full "economic" costs of an assessment system are estimated, the costs of assessment programs are considerably higher than they appear when only state level appropriations are considered.
The purpose of this report is to provide a first detailed analysis of the "economic" or opportunity costs of the testing systems in two states, Kentucky and Vermont. Using a framework developed by Picus (1994), this study looked closely at the amount of time local school officials spent supporting the assessment programs in their respective states in 1995-96, and estimates the value of that effort. As the results show, when the full "economic" costs of an assessment system are estimated, the costs of assessment programs are considerably higher than they appear when only state level appropriations are considered.
#586 – Research-Supported Accommodation for English Language Learners in NAEP
Jamal Abedi, Mary Courtney, and Seth Leon
Jamal Abedi, Mary Courtney, and Seth Leon
CSE Report 586, 2003
Summary
Summary
With recent legislation calling for equal learning opportunity for all children—including English language learners (ELLs)—the issue of assessment and accommodation for ELLs is gaining more attention. The No Child Left Behind Act of 2001 (2002) asks for fair assessment for all children, including ELL students, and encourages experimentally controlled research to examine issues related to assessment and accommodation for ELL students. Considering the fast-growing nature of the ELL population, this study aims to address several important issues concerning the use of accommodation in NAEP. First, it is important to identify those accommodations that help ELL students perform better by reducing the language barriers in content-area assessments (i.e., accommodations that are effective). A second major task is to make sure that accommodations that are effective in increasing the performance of ELL students do not give them an unfair advantage over non-ELL students not receiving the accommodations (i.e., the accommodations should be valid). We test this by examining whether the accommodations seem to have a positive effect on the performance of non-ELL students. The task of finding effective and valid accommodations is complete with the testing of accommodation feasibility. Therefore, the main objective of this study is to identify accommodations that are effective, valid, and logistically feasible to implement.
#446 – Large-Scale Assessment in Support of School Reform: Lessons in the Search for Alternative Measures
Joan Herman
Joan Herman
CSE Report 446, 1997
Summary
Summary
Assessment has long been a cornerstone of educational reform in the United States, fueled by beliefs in meritocracy, accountability, and the value of programmatic efforts to improve teaching and learning. Thirty years ago, for example, the passage of the original Elementary and Secondary Education Act of 1965 brought the federal government into local schools for the first time in support of quality education for disadvantaged students, and with it the requirement that schools receiving this funding administer standardized tests to determine eligibility and to evaluate the effects of their programs.
Fifteen or so years ago, minimum competency testing enjoyed a groundswell of popularity across the country, mandated by states to assure that all students would attain minimum standards of competence. More recently, the Goals 2000 legislation (1994), advocated by the President and passed by Congress, encouraged states to set rigorous standards for student performance and to assess students' progress toward their attainment; and even more recently, a national summit of the nation's governors similarly affirmed the need for their states to establish high standards for and rigorous assessment of student accomplishment (National Governors' Association, 1996).
Fifteen or so years ago, minimum competency testing enjoyed a groundswell of popularity across the country, mandated by states to assure that all students would attain minimum standards of competence. More recently, the Goals 2000 legislation (1994), advocated by the President and passed by Congress, encouraged states to set rigorous standards for student performance and to assess students' progress toward their attainment; and even more recently, a national summit of the nation's governors similarly affirmed the need for their states to establish high standards for and rigorous assessment of student accomplishment (National Governors' Association, 1996).
#713 – The Practical Relevance of Accountability Systems for School Improvement: A Descriptive Analysis of California Schools
Heinrich Mintrop, Tina Trujillo
Heinrich Mintrop, Tina Trujillo
CSE Report 713, 2007
Summary
Summary
In search for the practical relevance of accountability systems for school improvement, we ask whether practitioners traveling between the worlds of system-designated highand low-performing schools would detect tangible differences by observing concrete behaviors, looking at student work, or inquiring about teacher, administrator, or student perceptions. Would they see real differences in educational quality? Would they find schools that are truly more effective? In this study, we compare nine exceptionally high and low performing urban middle schools within the California accountability system. Traversing the nine schools, our travelers would learn that schools that grew on the state performance indicator tended to generate internal commitment for the accountability system. They eschewed the coercive aspects of accountability, maintained a climate of open communication, and considered the system as an impetus for raising expectations and work standards. On the instructional side, this commitment translated into the forceful implementation of structured language arts and literacy programs that were aligned with the accountability system. If our travelers expected to encounter visible signs of an overall higher quality of students’ educational experience in the highperforming schools, they would be disappointed. Rather they would have to settle on a much narrower definition of quality that homes in on attitudes and behaviors that are quite proximate to the effective acquisition of standards-aligned and test-relevant knowledge.
#582 – Measuring Instructional Quality in Accountability Systems: Classroom Assignments and Student Achievement
Lindsay Clare Matsumura, Helen E. Garnier, Jenny Pascal, Rosa Valdes
Lindsay Clare Matsumura, Helen E. Garnier, Jenny Pascal, Rosa Valdes
CSE Report 582, 2002
Summary
Summary
This report describes the technical quality of a CRESST-developed measure of the quality of classroom assignments piloted in the LAUSD’s proposed new accountability system. For this study, 181 teachers were sampled from 35 schools selected at random. Participating teachers submitted three language arts assignments with samples of student work (N = 50). Results indicated a fair level of agreement among the raters who scored the assignments and a high level of internal consistency within four dimensions of assignment quality. The stability of the ratings and the number of assignments needed to yield a consistent estimate of quality differed by elementary and secondary school levels. As a group, secondary students who received higher quality assignments produced higher quality written work and scored higher on the reading and language portions of the Stanford Achievement Test, Ninth Edition, adjusted for student background and prior achievement.
#522 – Instructional Variation and Student Achievement in a Standards-Based Education District
Lauren Resnick and Michael Harwell
Lauren Resnick and Michael Harwell
CSE Report 522, 2000
Summary
Summary
This paper, part of a larger study of the links between instructional variation and variation in performance on standards-based assessment, reports on the relations between examination results and instructional variation in a diverse New York City school district. The district has put in place an educational improvement system founded on intensive, school-based professional development that is carefully related to a preferred framework for teaching literacy. This study provides strong evidence that the school district's school-based professional development program improves teaching quality for diverse schools in ways that affect students' achievement scores.
#457 – Equity Issues in Collaborative Group Assessment: Group Composition and Performance
Noreen Webb, Kariane Nemer, Alex Chizhik, and Brenda Sugrue
Noreen Webb, Kariane Nemer, Alex Chizhik, and Brenda Sugrue
CSE Report 457, 1997
Summary
Summary
Large-scale assessment programs increasingly use group assessment tasks, in which small groups of students collaborate to solve problems or complete projects, often in combination with tasks that students perform individually. Whether some group compositions may be more advantageous than others is an important equity issue.
The present study investigated the effects of group ability composition on group processes and outcomes in science performance assessments in which students worked on a series of assessments first individually, then in groups, and finally individually again. Group composition had a major impact on the quality of group discussion in group work and on students' achievement test scores, both during group work and on the subsequent individual test. Groups with above-average students produced more correct answers and generated a greater number of high-quality explanations of how to solve the test problems than did groups without above-average students.
The higher level of discussion translated into an advantage on the achievement tests for below-average students working in groups with above-average students compared to below-average students working in groups without above-average students: the former students performed better on the achievement test completed during group work and on the subsequent achievement test completed individually than did the latter students. High-ability students performed equally well in heterogeneous groups, homogeneous groups, and when they worked alone.
The present study investigated the effects of group ability composition on group processes and outcomes in science performance assessments in which students worked on a series of assessments first individually, then in groups, and finally individually again. Group composition had a major impact on the quality of group discussion in group work and on students' achievement test scores, both during group work and on the subsequent individual test. Groups with above-average students produced more correct answers and generated a greater number of high-quality explanations of how to solve the test problems than did groups without above-average students.
The higher level of discussion translated into an advantage on the achievement tests for below-average students working in groups with above-average students compared to below-average students working in groups without above-average students: the former students performed better on the achievement test completed during group work and on the subsequent achievement test completed individually than did the latter students. High-ability students performed equally well in heterogeneous groups, homogeneous groups, and when they worked alone.
#661 – Upgrading America’s Use of Information to Improve Student Performance
Margaret Heritage, John Lee, Eva Chen, and Debbie LaTorre
Margaret Heritage, John Lee, Eva Chen, and Debbie LaTorre
CSE Report 661, 2005
Summary
Summary
This report presents a description of web-based decision support tool, the Quality School Portfolio (QSP), developed at the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) at UCLA, a discussion of the professional development to support the implementation of QSP, findings from an evaluation research study of the implementation, and recommendations for a next generation of QSP.
#711 – Does Teacher Professional Development Affect Content and Pedagogical Knowledge: How Much and for How Long?
Pete Goldschmidt, Geoffrey Phelps
Pete Goldschmidt, Geoffrey Phelps
CSE Report 711, 2007
Summary
Summary
We examine the impact of teacher professional development on knowledge growth and subsequent knowledge retention. Specifically we use English Language Arts teacher content and pedagogy assessments to determine whether the California Professional Development Institutes significantly improve teacher content knowledge and whether teachers retain that knowledge six months after the institutes are completed. The results indicate that teachers vary significantly in pre-institute knowledge on the four assessed domains, demonstrate significant knowledge growth, but only retain about one half of what was gained during the institute. Further, pre-existing knowledge gaps are not systematically reduced and teacher perceptions of institute quality are not related to knowledge growth and knowledge retention.
#568 – Collaborative, School-Based Professional Development Settings for Teachers: Implementation and Links to Improving the Quality of Classroom Practice and Student Learning
Lindsay Clare Matsumura and Joan Rector Steinberg
Lindsay Clare Matsumura and Joan Rector Steinberg
CSE Report 568, 2002
Summary
Summary
This study examined how cognitive and motivational factors jointly contributed to science achievement, engagement, and choice of science-related majors and careers in a sample of 491 high school students. Students completed cognitive and motivational measures in three different sessions: (a) a survey of motivational processes, including competence beliefs, task values, and behavioral engagement in the science classroom; (b) assessments of fluid, crystallized, and spatial abilities; and (c) a science achievement test. Results of regression analyses showed that the inclusion of motivational variables enhanced the predictive validity for science achievement. General ability was the strongest predictor of achievement outcomes, whereas motivational variables were the strongest predictors of engagement and choice. General ability had a direct effect on achievement and an indirect effect through the mediation of competence beliefs. Competence beliefs and task values had direct effects on achievement and indirect effects through the mediation of engagement. The study highlights the differential predictive validity of cognitive and motivational factors for different types of outcome and corroborates the mediational pathways linking self-system processes, action, and outcomes.
