Reports
Please note that CRESST reports were called "CSE Reports" or "CSE Technical Reports" prior to CRESST report 723.
#741 – From Evidence to Action: A Seamless Process in Formative Assessment?
Margaret Heritage, Jinok Kim, Terry P. Vendlinski, Joan L. Herman
CRESST Report 741, 2008 
Summary
Based on the results of a generalizability study (G study) of measures of teacher knowledge for teaching mathematics developed at The National Center for Research, on Evaluation, Standards, and Student Testing (CRESST) at the University of California, Los Angeles, this report provides evidence that teachers are better at drawing reasonable inferences about student levels of understanding from assessment information than they are in deciding the next instructional steps. We discuss the implications of the results for effective formative assessment and end with considerations of how teachers can be supported to know what to teach next.
#809 – Relationships between Teacher Knowledge, Assessment Practice, and Learning- Chicken, Egg, or Omelet?
Joan Herman, Ellen Osmundson, Yunyun Dai, Cathy Ringstaff, and Mike Timms
CRESST Report 809, November 2011
Summary
Drawing from a large efficacy study in upper elementary science, this report
had three purposes: First to examine the quality of teachers'
content-pedagogical knowledge in upper elementary science; second, to
analyze the relationship between teacher knowledge and their assessment
practice; and third, to study the relationship between teacher knowledge,
assessment practice, and student learning. Based on data from 39 teachers,
CRESST researchers found that students whose teachers frequently analyzed
and provided feedback on student work had higher achievement than students
whose teachers spent less time on such activities. The findings support
other research indicating the power of well-implemented formative assessment
to improve learning.
#400 – Portfolio-Driven Reform: Vermont Teachers' Understanding of Mathematical Problem Solving and Related Changes in Classroom Practice
Brian M. Stecher and Karen J. Mitchell
CSE Report 400, 1995
Summary
The Vermont portfolio assessment program, conclude the authors of the report, has had substantial positive effects on fourth-grade teachers' perceptions and practices in mathematics.
"Through the Vermont state training materials and network meetings," say CRESST/RAND researchers Brian Stecher and Karen Mitchell, "teachers have incorporated problem solving into their curriculum and have gained a greater insight into teaching problem-solving skills."
However, the researchers add that teachers do not yet share a common understanding of mathematical problem solving and have not reached agreement on the most essential skills to be taught. The emphasis on the scoring rubrics, for example, has helped teachers focus on some of the important and observable aspects of students' problem solving, but may have caused teachers to neglect other important problem-solving skills not addressed in the scoring rubrics. Additionally, significant variability exists in teaching methods-some teachers "preteach" portfolio tasks by assigning similar, simpler problems prior to student work on portfolio pieces so that assessment problems are not overly novel or difficult for students. Differential help may threaten the validity of portfolio scores for comparison of students, classrooms or schools.
The Vermont Department of Education, conclude the authors, should orient existing professional teacher development programs towards increasing teachers' basic understanding of mathematical problem solving and related instructional practices.
Effects of Introducing Classroom Performance Assessments on Student Learning is one of the first empirical examinations of the link between student achievement and performance assessment. Achievement results were compared for both treatment and control schools, where the schools were matched on demographics and socioeconomic factors. Assessments from the Maryland State Department were selected as independent measures of student performance in both sets of classrooms because they are still relatively standardized test-like compared to many performance assessments, but markedly different from traditional standardized tests.
"Our concluding advice," write the researchers, "is that reformers take seriously the current rhetoric about `delivery standards' and the need for sustained professional development to implement a thinking curriculum. The changes that did occur...confirm our beliefs that many more students can develop conceptual understandings presently exhibited by only the most able students-if only they are exposed to relevant problems and given the opportunity-to-learn."
#760 – Third Year Report: Evaluation of the Artful Learning Program
Noelle C. Griffin, Judy N. Miyoshi
CRESST Report 760, 2009
Summary
The National Center for Research on Evaluation, Standards, and Student Testing (CRESST) at UCLA was contracted to undertake a three-year external evaluation of the Artful Learning program, an arts-based school improvement model developed from the work and philosophy of the late composer Leonard Bernstein. This is the third-year report of evaluation findings, with a primary focus on Artful Learning participants in the 2003–2004 school year. The purpose of this report is to provide information about the implementation and impact of the program at current participating school sites, as well as place these findings within the context of the overall findings from the three-year evaluation as a whole. Multiple quantitative and qualitative data collection methods were employed throughout this evaluation. Overall, the findings suggest that the Artful Learning program was a useful tool for teachers with a variety of previous teaching experience, district and state contextual demands, grade/content areas taught, and student populations. Teacher satisfaction with the professional development components of the program were high, although assessment was an area singled out as needing additional support. Recommendations, drawing from all three years of the evaluation, are also discussed.
#802 – Knowing and Doing: What Teachers Learn from Formative Assessment and How They Use the Information
Greta Frohbieter, Eric Greenwald, Brian Stecher and Heather Schwartz
CRESST Report 802, July 2011
Summary
This study analyzed three different middle school mathematics formative assessment programs, examining how features of each program were associated with the information they provided to teachers and the manner in which teachers used the information.
The research team found considerable variation in the information teachers obtained from each program and how they used it. They found that greater familiarity with the specific formative assessment system did seem to be accompanied by more integrated use during the school year. They also found that teachers seemed to find it easier to incorporate the systems that had pre-existing assessments than the system that put the burden for assessment design on their shoulders.
The results from this study can aide teachers, administrators and other education stakeholders in deciding which formative assessment systems to adopt, planning for the implementation of formative assessment and providing adequate training for teachers, designing formative assessment systems that better meet teachers' needs, setting realistic expectations for the impact of formative assessment systems on a large scale, and lastly, understanding the impact of formative assessment in a particular context.
#788 – IES Integrated Learning Assessment Final Report
David Silver, Mark Hansen, Joan Herman, Yael Silk, and Cynthia L. Greenleaf
CRESST Report 788, March 2011
Summary
The main purpose of this study was to examine the effects of the Reading Apprenticeship professional development program on several teacher and student outcomes, including effects on student learning. A key part of the study was the use of an enhanced performance assessment program, the Integrated Learning Assessment (ILA), to measure student content understanding. The ILA instruments included multiple components that assessed student content knowledge, reading comprehension, metacognition, use of reading strategies, and writing skills in applied knowledge. An analysis of student scores using the ILA found little or no significant effects from the Reading Apprenticeship program on class-level student outcomes. However, the researchers found a significant positive effect on teachers' literacy instruction.
#740 – Formative Assessment and the Improvement of Middle School Science Learning: The Role of Teacher Accuracy
Joan L. Herman, Kilchan Choi
CRESST Report 740, 2008
Summary
This article articulates a framework for examining the quality of formative assessment practice and provides empirical evidence in support of one of its components. Based on a study of middle school science, the study examines the accuracy of teachers' judgments of students’ understanding and the relationship of such accuracy to middle school students’ learning. Analyses within and between teachers show a consistent, positive relationship between teacher accuracy and student learning. Study results lend support for the power of assessment in improving student learning and also suggest some potential challenges in assuring quality formative assessment practice.
#725 – Eliciting Student Thinking in Elementary School Mathematics Classrooms
Megan L. Franke, Noreen M. Webb, Angela Chan, Dan Battey, Marsha Ing, Deanna Freund, Tondra De
CRESST Report 725, 2007
Summary
The importance of student talk in mathematics classrooms figures prominently in curriculum and teaching standards. Student talk is a vehicle for increasing student learning and for helping teachers monitor student understanding and inform student instructional practices. Although researchers have begun to study the moves teachers may make to support students in making their mathematical thinking explicit, sharing out with others and using it as the basis of conversation, much remains to be known about the teacher practices that help students clarify and communicate their mathematical thinking. To learn more about these teacher practices, we look closely at what teachers say and do as they engage with their students in mathematical conversation and how students participate in relation to what teachers say and do. In this report we examine the questions teachers ask and how those questions support students to detail their mathematical thinking. Although all teachers in this study asked students to explain how they solved problems, an important teacher practice for encouraging further student elaboration and giving complete and correct explanations was asking further questions about specific aspects of students’ answers or explanations. We describe the variety of teacher questioning practices and the differences in patterns of student participation that emerged.
#807 – How Middle School Mathematics Teachers Use Interim and Benchmark Assessment Data
Lorrie A. Shepard, Kristen L. Davidson and Richard Bowman
CRESST Report 807
Summary
In recent years, a large number of school districts have adopted interim or benchmark assessments to help inform instruction during the school year. In this study, researchers drew from interviews across seven school districts and analyzed: 1) teachers understandings of their school districts’ purposes for interim assessments; 2) professional development to support teachers effective use of assessment information; and 3) reported teacher use of the data to adjust instruction. The researchers found that although many teachers expressed an interest in using interim assessment results as intended, that they often received minimal professional development and frequently had a different understanding regarding the intended use of the assessments than did district leaders. The researchers concluded that, in general, the interim assessment data did not provide teachers with insights about what to do next to help students other than to reteach.
#736 – Assessment Portfolios as Opportunities for Teacher Learning
Maryl Gearhart, Ellen Osmundson
CRESST Report 736, 2008
Summary
This report is an analysis of the role of assessment portfolios in teacher learning. Over 18 months, 19 experienced science teachers worked in grade-level teams to design, implement, and evaluate assessments to track student learning throughout a curriculum unit, supported by semi-structured tasks and resources in assessment portfolios. Teachers had the opportunity to complete three assessment portfolios for two or three curriculum units. Evidence of teacher learning included (a) changes over time in the contents of 10 teachers' portfolios spanning Grades 1–9 and (b) the full cohort's self-reported learning in surveys and focus groups. Findings revealed that Academy teachers developed greater understanding of assessment planning, quality assessments and scoring guides, strategies for analysis of student understanding, and use of evidence to guide instruction. Evidence of broad impact on teacher learning was balanced by evidence of uneven growth, particularly with more advanced assessment concepts such as reliability and fairness as well as curriculum-specific methods for developing and using assessments and scoring guides. The findings point to a need for further research on ways to balance general approaches to professional development with content specific strategies to deepen teacher skill and knowledge.
