National Center for Research on Evaluation, Standards and Student Testing

#725 – Eliciting Student Thinking in Elementary School Mathematics Classrooms
Megan L. Franke, Noreen M. Webb, Angela Chan, Dan Battey, Marsha Ing, Deanna Freund, Tondra De

Summary

The importance of student talk in mathematics classrooms figures prominently in curriculum and teaching standards. Student talk is a vehicle for increasing student learning and for helping teachers monitor student understanding and inform student instructional practices. Although researchers have begun to study the moves teachers may make to support students in making their mathematical thinking explicit, sharing out with others and using it as the basis of conversation, much remains to be known about the teacher practices that help students clarify and communicate their mathematical thinking. To learn more about these teacher practices, we look closely at what teachers say and do as they engage with their students in mathematical conversation and how students participate in relation to what teachers say and do. In this report we examine the questions teachers ask and how those questions support students to detail their mathematical thinking. Although all teachers in this study asked students to explain how they solved problems, an important teacher practice for encouraging further student elaboration and giving complete and correct explanations was asking further questions about specific aspects of students’ answers or explanations. We describe the variety of teacher questioning practices and the differences in patterns of student participation that emerged.

#761 – Using Classroom Artifacts to Measure the Efficacy of Professional Development
Yael Silk, David Silver, Stephanie Amerian, Claire Nishimura, Christy Kim Boscardin

Summary

This report describes a classroom artifact measure and presents early findings from an efficacy study of WestEd's Reading Apprenticeship (RA) professional development program. The professional development is designed to teach high school teachers how to integrate subject-specific literacy instruction into their regular curricula. The current RA study is notable in that it is the first to include random assignment in its design. The National Center for Research on Evaluation, Standards, and Student Testing (CRESST) designed a teacher assignment instrument to address the question of whether treatment teachers demonstrate greater integration of literacy into their instructional practice than control teachers. Early findings based on preliminary data from participating history teachers indicate that treatment teachers outperformed control teachers on 6 out of 11 rubric dimensions. These dimensions address opportunities for reading in the assignment, the strategies in place to support successful reading, teacher support for reading engagement, and student feedback. Data collection will conclude at end of the 2008-2009 school year, followed by a final report.

#770 – Capturing Quality in Formative Assessment Practice: Measurement Challenges
Joan L. Herman, Ellen Osmundson, & David Silver

Summary

This study examines measures of formative assessment practice using data from a study of the implementation and effects of adding curriculum embedded measures to a hands-on science program for upper elementary school students. The authors present a unifying conception for measuring critical elements of formative assessment practice, illustrate common measures for doing so, and investigate the relationships among and between scores on these measures. Findings raise important issues with regard to both the challenge of obtaining valid measures of teachers’ assessment practice and the uneven quality nature of current teacher practice.

To cite from this report, please use the following as your APA reference:

Herman, J., L., Osmundson, E., & Silver, D. (2010). Capturing quality in formative assessment practice: Measurement challenges. (CRESST Report 770). Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

#788 – IES Integrated Learning Assessment Final Report
David Silver, Mark Hansen, Joan Herman, Yael Silk, and Cynthia L. Greenleaf

Summary

The main purpose of this study was to examine the effects of the Reading Apprenticeship professional development program on several teacher and student outcomes, including effects on student learning. A key part of the study was the use of an enhanced performance assessment program, the Integrated Learning Assessment (ILA), to measure student content understanding. The ILA instruments included multiple components that assessed student content knowledge, reading comprehension, metacognition, use of reading strategies, and writing skills in applied knowledge. An analysis of student scores using the ILA found little or no significant effects from the Reading Apprenticeship program on class-level student outcomes. However, the researchers found a significant positive effect on teachers' literacy instruction.

#562 – Looking Into Students' Science Notebooks: What Do Teachers Do With Them?
Maria Araceli Ruiz-Primo, Min Li, and Richard J. Shavelson

Summary

We propose the use of students' science notebooks as one possible unobtrusive method for examining some aspects of teaching quality. We used students' science notebooks to examine the nature of instructional activities they encountered in their science classes, the nature of their teachers' feedback, and how these two aspects of teaching were correlated with students' achievement. We examined the characteristics of students' science notebooks from 10 fifth-grade classrooms. Six students' notebooks in each classroom were randomly selected. Each entry of each student's science notebook was analyzed according to the characteristics of the activity, quality of student's performance as reflected by the notebook entry, and the teacher feedback in the notebook. Results indicated that (a) raters can consistently classify notebook entries despite the diversity of the forms of communication (written, schematic or pictorial). They can also consistently score the quality of a student's communication, conceptual and procedural understanding, and the quality of a teacher's feedback to the student. (b) The intellectual demands of the tasks required by the teachers were, in general, low. Teachers tended to ask students to record the results of an ,experiment or to copy definitions. (c) Low student performance scores across two curriculum units revealed that students' communication skills and understanding were far from the maximum score and did not improve over the course of instruction during the school year. And (d) teachers provided little, if any, feedback. Only 4 of the 10 teachers provided any feedback to students' notebook entries, and when feedback was provided, comments took the form of a grade, checkmark, or a code phrase. We concluded that the benefits of science notebooks as a learning tool for students and as a source of information for teachers were not exploited in the science classrooms studied.

#784 – Evaluation of the Artist Teacher Collaborative Program: Summary of Year 1 Findings
Noelle C. Griffin, Yael Silk, Kirby A. Chow, and Yourim Chai

Summary

The Artist-Teacher Collaborative (ATC) program is a partnership between the Armory Center for the Arts and Pasadena Unified School District (PUSD). A central goal of this program, which is supported by the U. S. Department of Education, is to provide sustained, rigorous professional development to 3rd through 5th grade PUSD teachers, instilling in them the skills and knowledge needed to deliver standards-based instruction in the visual arts. Using survey methodology, classroom observations, and quantitative techniques, CRESST evaluated the quality the program. This year-end report addresses the ongoing program¹s success and provides formative feedback on the strengths and weaknesses of program implementation.

#822 – The Impact of Short-Term Science Teacher Professional Development on the Evaluation of Student Understanding and Errors Related to Natural Selection
Rebecca E. Buschang

Summary

This study evaluated the effects of a short-term professional development session. Forty volunteer high school biology teachers were randomly assigned to one of two professional development conditions: (a) developing deep content knowledge (i.e., control condition) or (b) evaluating student errors and understanding in writing samples (i.e., experimental condition). A pretest of content knowledge was administered, and then the participants in both conditions watched two hours of online videos about natural selection and attended different types of professional development sessions lasting four hours. Significant differences between conditions in favor of the experimental condition were found on participant identification of critical elements of student understanding of natural selection and content knowledge related to natural selection. Results suggest that short-term professional development sessions focused on evaluating student errors and understanding can be effective at focusing a participant‟s evaluation of student work on particularly important elements of student understanding. Results have implications for understanding the types of knowledge necessary to effectively evaluate student work and for the design of professional development.

#711 – Does Teacher Professional Development Affect Content and Pedagogical Knowledge: How Much and for How Long?
Pete Goldschmidt, Geoffrey Phelps

Summary

We examine the impact of teacher professional development on knowledge growth and subsequent knowledge retention. Specifically we use English Language Arts teacher content and pedagogy assessments to determine whether the California Professional Development Institutes significantly improve teacher content knowledge and whether teachers retain that knowledge six months after the institutes are completed. The results indicate that teachers vary significantly in pre-institute knowledge on the four assessed domains, demonstrate significant knowledge growth, but only retain about one half of what was gained during the institute. Further, pre-existing knowledge gaps are not systematically reduced and teacher perceptions of institute quality are not related to knowledge growth and knowledge retention.

#423 – Teachers' Developing Ideas and Practices About Mathematics Performance Assessment: Successes, Stumbling Blocks, and Implications for Professional Development.
Hilda Borko, Vicky Mayfield, Scott Marion, Roberta Flexer, and Kate Cumbo

Summary

This study focuses on the change process experienced by a group of third-grade teachers as they implemented mathematics performance assessments in their classrooms. Based on workshop conversations and interviews between teachers and the rsearch/staff development team throughout a single school year, the team reached five major conclusions.

First, locating the change process in actual classrooms and school settings was an effective strategy for helping teachers change their assessment practice.

"The greatest changes occurred," said the researchers, "when teachers discussed ideas in workshops, attempted to implement them in their classroom, and then reflected or otherwise built upon their experiences in subsequent workshop sessions."

Other key findings from the research were that (a) group discussions were an effective tool for the social construction of new ideas; (b) staff development personnel facilitated change by introducing new ideas based on teachers' skills, interests, and understanding; (c) when teachers' beliefs incompatible with the intentions of the staff development team were not challenged, the teachers were likely either to ignore new ideas or to inappropriately assimilate them into existing practice; and (d) lack of time was a major obstacle to changing classroom practice.

#366 – Teachers' Ideas and Practices About Assessment and Instruction
Hilda Borko, Maurene Flory, and Kate Cumbo

Summary

Participants involved in this study were part of a year-long intervention designed to help teachers develop performance assessments in reading and mathematics. Seeking to evaluate teachers' knowledge, beliefs, and practices about assessment and instruction, the researchers also studied the changes that occurred to teachers during the first semester of the intervention program. Findings from the study indicated that the performance assessment development and implementation process led to teachers having better understandings and new insights into students' thinking and learning than when teachers relied exclusively on more traditional forms of assessment. However, it was not clear to what extent teachers changed their instructional programs to take advantage of their newly gained insights. Based on their observations so far, researchers feel confident that as the program continues, more extensive changes will occur.

Like this Product? Get CRESST News Every Month!

Reports

Download Our Reports

Partners & Collaborators