Reports
Please note that CRESST reports were called "CSE Reports" or "CSE Technical Reports" prior to CRESST report 723.
#796 – Games as Formative Assessment Environments: Examining the Impact of Explanations of Scoring and Incentives on Math Learning, Game Performance, and Help Seeking
Girlie C. Delacruz
Girlie C. Delacruz
CRESST Report 796, June 2011 
Summary
Summary
Due to their motivational nature, there is growing interest in the potential
of games to help teach academic content and skills. This report examines how
different levels of detail about a game's scoring rules affect math learning
and performance. Using random assignment, the author collected data from 164
students in the fourth to sixth grades. The treatment conditions included a
control group (played a different math game); three variations of scoring
explanations (elaborated, minimal, and no scoring information); and combined
elaborated scoring explanation with incentives to access additional
feedback.
The scoring explanation alone did not lead to better math learning. However, compared to the minimal-to-no scoring information variations, the combined treatment of the elaborated scoring explanation and incentive resulted in higher scores. The author discusses implications for learning games in mathematics.
The scoring explanation alone did not lead to better math learning. However, compared to the minimal-to-no scoring information variations, the combined treatment of the elaborated scoring explanation and incentive resulted in higher scores. The author discusses implications for learning games in mathematics.
#795 – Progress Report Year 4: National Center for Research on Evaluation, Standards,
and Student Testing (CRESST) The Development and Impact of POWERSOURCE©
Eva L. Baker
Eva L. Baker
CRESST Report 795, May 2011
Summary
Summary
The POWERSOURCE© intervention is intended to be a powerful formative
assessment strategy that can be integrated with any on-going mathematics
curriculum. The key goal is to to improve teachers' knowledge and practice
and, in turn, student learning in middle school mathematics. In this 4th
year study, the authors provide updated results from the 2006-07
experimental (randomized) field test of POWERSOURCE© and present findings
from the 2007-08 school year on student and teacher outcomes. The authors
found that a short targeted POWERSOURCE© intervention on key mathematical
principles had a positive student performance impact on a transfer measure
of related content. POWERSOURCE© had more impact on the higher-performing
students than the lower-performing students. They also found a stronger
effect on more difficult mathematics items.
#794 – Using Key Conceptual Ideas to Improve Teacher Use of Formative Assessment Data
Terry P. Vendlinski and Julia Phelan
Terry P. Vendlinski and Julia Phelan
CRESST Report 794, May 2011
Summary
Summary
This paper reports on the outcomes achieved from an innovative professional development program, POWERSOURCE©, that helps teachers apply key foundational concepts and formative assessment to math content studied in the 6th, 7th, and 8th grade. The authors found significant gains from students' pretests to students' posttests, attributed to POWERSOURCE©. The findings suggest that the gains were associated with changes in teacher thinking and that such change may be easier for more experienced math teachers in earlier rather than later middle school grades.
#793 – The Development and Impact of POWERSOURCE©: Year 3
Julia Phelan, Terry Vendlinski, Kilchan Choi, Joan Herman, and Eva L. Baker
Julia Phelan, Terry Vendlinski, Kilchan Choi, Joan Herman, and Eva L. Baker
CRESST Report 793, May 2011
Summary
Summary
The POWERSOURCE© intervention is intended as a powerful formative assessment-based strategy that can be integrated with any ongoing mathematics curriculum to improve middle school teachers' knowledge and practice and, in turn, student learning. In this Year 3 (of 5) study, researchers found significant differences between POWERSOURCE© and control students' performance on project-developed measures for all districts and domains.
#792 – The Development and Impact of POWERSOURCE©: Year 5
Julia Phelan, Terry Vendlinski, Kilchan Choi, Yunyun Dai, Joan Herman, and Eva L. Baker
Julia Phelan, Terry Vendlinski, Kilchan Choi, Yunyun Dai, Joan Herman, and Eva L. Baker
CRESST Report 792, May 2011
Summary
Summary
The POWERSOURCE© intervention is a formative assessment-based strategy that can be integrated with any ongoing mathematics curriculum to improve teachers' knowledge and practice and, in turn, student learning. POWERSOURCE© includes both a system of learning-based assessments and an infrastructure to support teachers' use of those assessments to improve student learning. The Year 5 study focused on middle school mathematics, starting in Grade 6. Of the two teacher knowledge measures used, the teacher knowledge maps provided clearer information about how the POWERSOURCE© intervention affected teacher knowledge. Our findings indicate that POWERSOURCE© professional development helps teachers become more expert-like in the way they think about math problems in relation to mathematical concepts.
#791 – Evaluation of the Enhanced Assessment Grants (EAGs) SimScientists Program: SITE VISIT FINDINGS
Joan Herman, Yunyun Dai, Aye Mon Htut, Marcela Martinez, and Nichole Rivera
Joan Herman, Yunyun Dai, Aye Mon Htut, Marcela Martinez, and Nichole Rivera
CRESST Report 791, May 2011
Summary
Summary
This evaluation report addresses the implementation, utility, and feasibility of simulation-based assessments for middle school science classrooms, with particular attention to the use of accommodations available in the program. The SimScientists program includes embedded, formative assessments; reflection activities designed to deepen student understanding of key ideas and processes; and benchmark assessments to gauge student learning at the end of the unit. While teachers and students alike were very positive about their experiences with SimScientists, the evaluation team offered several recommendations for improvementin particular, recommendations for refining the program¹s embedded and benchmark assessments and for increasing the feasibility of the reflection activities.
#790 – The Feasibility of Using Cluster Analysis to Examine Log Data From Educational Video Games
Deirdre Kerr, Gregory K. W. K. Chung, and Markus R. Iseli
Deirdre Kerr, Gregory K. W. K. Chung, and Markus R. Iseli
CRESST Report 790, April 2011
Summary
Summary
Analyzing log data from educational video games has proven to be a challenging endeavor. In this paper, we examine the feasibility of using cluster analysis to extract information from the log files that is interpretable in both the context of the game and the context of the subject area. If cluster analysis can be used to identify patterns of thought as students play through the game, this method may be able to provide the information necessary to diagnose mathematical misconceptions or to provide targeted remediation or tailored instruction.
#788 – IES Integrated Learning Assessment Final Report
David Silver, Mark Hansen, Joan Herman, Yael Silk, and Cynthia L. Greenleaf
David Silver, Mark Hansen, Joan Herman, Yael Silk, and Cynthia L. Greenleaf
CRESST Report 788, March 2011
Summary
Summary
The main purpose of this study was to examine the effects of the Reading Apprenticeship professional development program on several teacher and student outcomes, including effects on student learning. A key part of the study was the use of an enhanced performance assessment program, the Integrated Learning Assessment (ILA), to measure student content understanding. The ILA instruments included multiple components that assessed student content knowledge, reading comprehension, metacognition, use of reading strategies, and writing skills in applied knowledge. An analysis of student scores using the ILA found little or no significant effects from the Reading Apprenticeship program on class-level student outcomes. However, the researchers found a significant positive effect on teachers' literacy instruction.
#787 – The Effects of Practicing With a Virtual Ultrasound Trainer on FAST Window Identification, Acquisition, and Diagnosis
Gregory K. W. K. Chung, Ruth G. Gyllenhammer, and Eva L. Baker
Gregory K. W. K. Chung, Ruth G. Gyllenhammer, and Eva L. Baker
CRESST Report 787, February 2011
Summary
Summary
n this study, CRESST researchers used random assignment of medical and nursing school students to analyze the differences between "simulator-based virtual ultrasound scanning practice" to "classroom-based hands-on ultrasound scanning practice." The CRESST team found that classroom-based practice appeared to promote physical acquisition skills while simulator-based practice appeared to promote window interpretation skills. Participants responded favorably to the simulator, which the researchers conclude is a promising training tool.
#786 – IES Teacher Assignment Final Report
Yael Silk, David Silver, Stephanie Amerian, Claire Nishimura, and Christy Boscardin
Yael Silk, David Silver, Stephanie Amerian, Claire Nishimura, and Christy Boscardin
CRESST Report 786, February 2011
Summary
Summary
The goal of this study was to test the effectiveness of WestEd¹s Reading Apprenticeship (RA) professional development program on teacher practices and student learning. The professional development is designed to teach high school teachers how to integrate subject-specific literacy instruction into their regular curricula. The CRESST researchers found that history and biology treatment teachers significantly outscored control teachers on three dimensions (reading comprehension strategies, metacognitive processes, and collaborative meaning making). History treatment teachers outperformed control teachers on an additional three dimensions (reading opportunities, support for reading engagement, and student feedback); biology treatment teachers scored higher on the adjusting instruction measure as compared to control teachers.
