National Center for Research on Evaluation, Standards and Student Testing

#792 – The Development and Impact of POWERSOURCE©: Year 5
Julia Phelan, Terry Vendlinski, Kilchan Choi, Yunyun Dai, Joan Herman, and Eva L. Baker

Summary

The POWERSOURCE© intervention is a formative assessment-based strategy that can be integrated with any ongoing mathematics curriculum to improve teachers' knowledge and practice and, in turn, student learning. POWERSOURCE© includes both a system of learning-based assessments and an infrastructure to support teachers' use of those assessments to improve student learning. The Year 5 study focused on middle school mathematics, starting in Grade 6. Of the two teacher knowledge measures used, the teacher knowledge maps provided clearer information about how the POWERSOURCE© intervention affected teacher knowledge. Our findings indicate that POWERSOURCE© professional development helps teachers become more expert-like in the way they think about math problems in relation to mathematical concepts.

#793 – The Development and Impact of POWERSOURCE©: Year 3
Julia Phelan, Terry Vendlinski, Kilchan Choi, Joan Herman, and Eva L. Baker

Summary

The POWERSOURCE© intervention is intended as a powerful formative assessment-based strategy that can be integrated with any ongoing mathematics curriculum to improve middle school teachers' knowledge and practice and, in turn, student learning. In this Year 3 (of 5) study, researchers found significant differences between POWERSOURCE© and control students' performance on project-developed measures for all districts and domains.

#795 – Progress Report Year 4: National Center for Research on Evaluation, Standards, and Student Testing (CRESST) The Development and Impact of POWERSOURCE©
Eva L. Baker

Summary

The POWERSOURCE© intervention is intended to be a powerful formative assessment strategy that can be integrated with any on-going mathematics curriculum. The key goal is to to improve teachers' knowledge and practice and, in turn, student learning in middle school mathematics. In this 4th year study, the authors provide updated results from the 2006-07 experimental (randomized) field test of POWERSOURCE© and present findings from the 2007-08 school year on student and teacher outcomes. The authors found that a short targeted POWERSOURCE© intervention on key mathematical principles had a positive student performance impact on a transfer measure of related content. POWERSOURCE© had more impact on the higher-performing students than the lower-performing students. They also found a stronger effect on more difficult mathematics items.

#763 – The Effects of POWERSOURCE Intervention on Student Understanding of Basic Mathematical Principles
Julia Phelan, Kilchan Choi, Terry Vendlinski, Eva L. Baker, Joan L. Herman

Summary

This report describes results from field-testing of POWERSOURCE formative assessment alongside professional development and instructional resources. The researchers at the National Center for Research, on Evaluation, Standards, & Student Testing (CRESST) employed a randomized, controlled design to address the following question: Does the use of POWERSOURCE strategies improve 6th-grade student performance on assessments of the key mathematical ideas relative to the performance of a comparison group? Sixth-grade teachers were recruited from 7 districts and 25 middle schools. A total of 49 POWERSOURCE and 36 comparison group teachers and their students (2,338 POWERSOURCE, 1,753 comparison group students) were included in the study analyses. All students took a pretest of prerequisite knowledge and a transfer measure of tasks drawn from international tests at the end of the study year. Students in the POWERSOURCE group used sets of formative assessment tasks. POWERSOURCE teachers had exposure to professional development and instructional resources. Results indicated that students with higher pretest scores tended to benefit more from the treatment as compared to students with lower pretest scores. In addition, students in the POWERSOURCE group significantly outperformed control group students on distributive property items and the effect was larger as pretest scores increased. Results, limitations and future directions are discussed.

To cite from this report, please use the following as your APA reference:

Phelan, J., Choi, K., Vendlinski, T., Baker, E. L., & Herman, J. L. (2009). The effects of POWERSOURCE intervention on student understanding of basic mathematical principles (CRESST Report 763). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

#794 – Using Key Conceptual Ideas to Improve Teacher Use of Formative Assessment Data
Terry P. Vendlinski and Julia Phelan

Summary

This paper reports on the outcomes achieved from an innovative professional development program, POWERSOURCE©, that helps teachers apply key foundational concepts and formative assessment to math content studied in the 6th, 7th, and 8th grade. The authors found significant gains from students' pretests to students' posttests, attributed to POWERSOURCE©. The findings suggest that the gains were associated with changes in teacher thinking and that such change may be easier for more experienced math teachers in earlier rather than later middle school grades.

#734 – Using Data and Big Ideas: Teaching Distribution as an Instance of Repeated Addition
Terry P. Vendlinski, Keith E. Howard, Bryan C. Hemberg, Laura Vinyard, Annabel Martel, Elizabeth Kyriacou, Jennifer Casper, Yourim Chai, Julia C. Phelan, Eva L. Baker

Summary

The inability of students to become proficient in algebra seems to be widespread in American schools. One of the reasons often cited for this inability is that instruction seldom builds on prior knowledge. Research suggests that teacher effectiveness is the most critical controllable variable in improving student achievement. This report details a process of formative assessment and professional development (called PowerSource©), which is designed to improve teacher effectiveness and student achievement. We describe the process we used to develop a model of distribution over addition and subtraction, one of three big ideas developed during the year, and the interactions we had with teachers about teaching distribution in various ways. As a consequence of these interactions, we were able to test whether teaching distribution using the notion of multiplication as repeated addition (a concept which students had learned previously), using array or area models, or teaching it procedurally had the greatest effects on student learning. We found that the repeated addition model was not only less likely to create certain student misconceptions, but also found that students taught using the repeated addition model were more likely to correctly answer questions involving distribution than were their counterparts taught using either of the other methods. Teachers subsequently reported that they preferred teaching distribution as an instance of repeated addition than teaching it using other available methods.

#696 – Measuring Teachers' Mathematical Knowledge
Margaret Heritage and Terry Vendlinski

Summary

Teachers' knowledge of mathematics is pivotal to their capacity to provide effective mathematics instruction and to their ability to assess student learning (Ball, Hill, & Bass, 2005; Ma, 1999; Schifter, 1999). The National Council for the Teaching of Mathematics (NCTM, 2000) makes it clear that teachers need knowledge of the whole domain as well as knowledge about the important ideas that are central to their grade level. POWERSOURCE is expected, through professional development and job aids, to influence teachers' pedagogical content knowledge and assessment practices. To gauge such effects we have developed teacher measures that focus on three key mathematical principles that are central to POWERSOURCE: the distributive property, solving equations, and rational number equivalence.

#750 – Some Aspects of the Technical Quality of Formative Assessments in Middle School Mathematics
Julia Phelan, Taehoon Kang, David N. Niemi, Terry Vendlinski, Kilchan Choi

Summary

While research suggests that formative assessment can be a powerful tool to support teaching and learning, efforts to jump on the formative assessment bandwagon have been more widespread than those to assure the technical quality of the assessments. This report covers initial analyses of data bearing on the quality of formative assessments in middle school mathematics. Specifically, these data address the question of whether relatively short assessments can provide reliable and useful information on middle school students' understanding of conceptual domains in pre-algebra. Items and test forms were developed and tested in four domains (rational number equivalence, properties of arithmetic, principles for solving equations, and applications of these concepts to other domains), all of which are critical to eventual mastery of algebra. We tested the items with sixth-grade students in classrooms in four districts. We then pared down the items to create eight assessment forms that were further tested alongside instructional support materials and professional development. Results of this study suggest that relatively brief formative assessments focused on key conceptual domains can provide reliable and useful information on students’ levels of understanding and possible misunderstandings in the domain.

#697 – The Power of Big Ideas in Mathematics Education: Development and Pilot Testing of POWERSOURCE Assessments
David Niemi, Julia Vallone, and Terry Vendlinski

Summary

The characteristics of expert knowledge-interconnectedness, understanding, and ability to transfer-are inextricably linked, a point that is critically important for educators and constitutes a major theme of this paper. In this paper we explore how an analysis of the architecture of expert knowledge can inform the development of assessments to help teachers move students toward greater expertise in mathematics, and we present examples of such assessments. We also review student responses and preliminary results from pilot tests of assessments administered in sixth-grade classes in a large urban school district. Our preliminary analyses suggest that an assessment strategy based on the structure of mathematical knowledge can reveal deficiencies in student understanding of and ability to apply fundamental concepts of pre-algebra, and has the potential to help teachers remediate those deficiencies.

#698 – Celebrating 20 Years of Research on Educational Assessment: Proceedings of the 2005 CRESST Conference
Anne Lewis

Summary

The 2005 CRESST conference marked the 20th year of work on critically important accountability topics by the UCLA institution, "a tremendous accomplishment for a research center," according to Aimee Dorr, dean of the Graduate School of Education and Information Studies. In her welcoming remarks, Dean Dorr described why CRESST has achieved such longevity. The center is "independent, very lively, grounded in practice, and very forward looking, with many top accountability experts from around the nation," said Dorr, "interested in new technologies and helping to shape the future of education." She also noted that although it was "good fortune" for the center's senior partner to be located at UCLA, "it is a partnership throughout the country, and one that enriches us here as the partners do on the national scene."

The anniversary for CRESST was an opportunity for the conference program to focus on the achievements in the use of assessment to improve student learning. The two-day gathering described many of the lessons learned from a century of testing. The discussions also featured the newest CRESST initiative, known as POWERSOURCE, a $10 million grant from the Institute of Education Sciences at the U.S. Department of Education to develop formative mathematics assessments in the middle grades to improve student performance and learning.

Like this Product? Get CRESST News Every Month!

Reports

Download Our Reports

Partners & Collaborators