National Center for Research on Evaluation, Standards and Student Testing

#702 – English Language Learners and Math Achievement: A Study of Opportunity to Learn and Language Accommodation
Jamal Abedi, Mary Courtney, Seth Leon, Jenny Kao, and Tarek Azzam

Summary

This study investigated the interactive effects between students’ opportunity to learn (OTL) in the classroom, two language-related testing accommodations, and English language learner (ELL) students and other students of varying language proficiency, and how these variables impact mathematics performance. Hierarchical linear modeling was employed to investigate three class-level components of OTL, two language accommodations, and ELL status. The three class-level components of OTL were: (1) student report of content coverage; (2) teacher content knowledge; and (3) class prior math ability (as determined by an average of students’ Grade 7 math scores). A total of 2,321 Grade 8 students were administered one of three versions of an algebra test: a standard version with no accommodation, a dual-language (English and Spanish) test version accommodation, or a linguistically modified test version accommodation. These students’ teachers were administered a teacher content knowledge measure. Additionally, 369 of these students were observed for one class period for student-teacher interactions. Students’ scores from the prior year’s state mathematics and reading achievement tests, and other background information were also collected.

Results indicated that all three class-level components of OTL were significantly related to math performance, after controlling for prior math ability at the individual student level. Class prior math ability had the strongest effect on math performance. Results also indicated that teacher content knowledge had a significant differential effect on the math performance of students grouped by a quick reading proficiency measure, but not by students’ ELL status or by their reading achievement test percentile ranking. Results also indicated that the two language accommodations did not impact students’ math performance. Additionally, results suggested that, in general, ELL students reported less content coverage than their non-ELL peers, and they were in classes of overall lower math ability than their non-ELL peers.

While it is understandable why a student’s performance in seventh grade strongly determines the content she or he receives in eighth grade, there is some evidence in this study that students of lower language proficiency can learn algebra and demonstrate algebra knowledge and skills when they are provided with sufficient content and skills delivered by proficient math instructors in a classroom of students who are proficient in math.

#808 – Teaching Rational Number Addition Using Video Games: The Effects of Instructional Variation
Terry P. Vendlinski, Gregory K. W. K. Chung, Kevin R. Binning, and Rebecca E. Buschang

Summary

Understanding the meaning of rational numbers and how to perform mathematical operations seems to be a perennial problem in the United States for both adults and children. CRESST researchers hypothesized that giving students more time to practice using rational numbers in an environment that enticed them to apply their understanding might prove educationally beneficial. They developed a video game, based on two key ideas about addition and rational numbers, to investigate their hypothesis. in this initial study, they found that students using an appropriately design game increased their ability to add rational numbers even when playing the game for a relatively short period of time. The authors discuss implications for the larger efficacy study to follow.

#786 – IES Teacher Assignment Final Report
Yael Silk, David Silver, Stephanie Amerian, Claire Nishimura, and Christy Boscardin

Summary

The goal of this study was to test the effectiveness of WestEd¹s Reading Apprenticeship (RA) professional development program on teacher practices and student learning. The professional development is designed to teach high school teachers how to integrate subject-specific literacy instruction into their regular curricula. The CRESST researchers found that history and biology treatment teachers significantly outscored control teachers on three dimensions (reading comprehension strategies, metacognitive processes, and collaborative meaning making). History treatment teachers outperformed control teachers on an additional three dimensions (reading opportunities, support for reading engagement, and student feedback); biology treatment teachers scored higher on the adjusting instruction measure as compared to control teachers.

#507 – Knowledge Mapping in the Classroom: A Tool for Examining the Development of Students' Conceptual Understandings
Ellen Osmundson, Gregory Chung, Howard E. Herl, and Davina C. D. Klein

Summary

The objective of this study was to investigate how computer-based knowledge mapping could be used simultaneously as an instructional tool and an assessment tool in a classroom setting. Data are presented that demonstrate how knowledge mapping served as a tool to support, facilitate, promote, and evaluate students' development of understandings in science. In this study, knowledge mapping was (a) integrated into instruction, (b) employed as a repeated measure to capture the ongoing development of ideas, (c) used individually as well as collaboratively, (d) scored according to algorithms that emphasized the recursive and incremental nature of both learning and the development of scientific ideas, and (e) accessed online through the computer. A non-equivalent control group design was utilized. Fifty-two fourth- and fifth-grade students from two intact classrooms participated in this study. Both groups generated pretest and posttest knowledge maps of their understandings of the digestive, respiratory, and circulatory systems. Students in the experimental group created three additional (collaborative) maps during the course of instruction. Students in the control condition worked in small groups on three occasions to research the human body by using the Internet and related instructional materials. Results suggest that for students in the experimental group, repeated use of the mapping software supported and facilitated the development of scientific and principled understandings. Further, collaborative work with the mapper also afforded students the opportunity to establish connections between the systems in the human body to more fully develop their understandings of the domain--both integral components of learning and the development of scientific understandings.

#403 – Instructional Influences on Content Area Explanations and Representational Knowledge: Evidence for the Construct Validity of Measures of Principled Understanding
David Niemi

Summary

Recent calls for understanding-based mathematics instruction imply a need for alternative kinds of assessment. The purpose of this study was to determine if elementary students who received understanding-based mathematics instruction in a key domain (fractions), would outperform students who received more traditional instruction. The results showed that students who received understandings-based instruction on fraction principles performed better than students in a more traditionally instructed group on nearly all measures of principled understanding, and equally well on measures of computation. In response to understandings-based instruction, many students appeared to add new ideas about fractions to their existing knowledge, without discarding or reworking previously learned ideas. Results also showed the feasibility of assessing important aspects of mathematical understanding through students' use of mathematical representations and language. Further, students used their representational knowledge when they wrote justifications and explanations to the assessment.

#766 – Examining the Effectiveness and Validity of Glossary and Read-Aloud Accommodations for English Language Learners in a Math Assessment
Mikyung Kim Wolf, Jinok Kim, Jenny C. Kao, Nichole M. Rivera

Summary

Glossary and reading aloud test items are often listed as allowed in many states' accommodation policies for ELL students, when taking states' large-scale mathematics assessments. However, little empirical research has been conducted on the effects of these two accommodations on ELL students' test performance. Furthermore, no research is available to examine how students use the provided accommodations. The present study employed a randomized experimental design and a think-aloud procedure to delve into the effects of the two accommodations. A total of 605 ELL and non-ELL students from two states participated in the experimental component and a subset of 68 ELL students participated in the think-aloud component of the study. Results showed no significant effect of glossary, and mixed effects of read aloud on ELL students' performance. Read aloud was found to have a significant effect for the ELL sample in one state, but not the other. Significant interaction effects between students' prior content knowledge and accommodations were found, suggesting the given accommodation was effective for the students who had acquired content knowledge. During the think-aloud analysis, students did not actively utilize the provided glossary, indicating lack of familiarity with the accommodation. Implications for the effective use of accommodations and future research agendas are discussed.

To cite from this report, please use the following as your APA reference:

Wolf, M. K., Kim, J., Kao, J. C., & Rivera, N. M. (2009). Examining the effectiveness and validity of glossary and read-aloud accommodations for English language learners in a math assessment (CRESST Report 766). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

#735 – Templates and Objects in Authoring Problem-Solving Assessments
Terry P. Vendlinski, Eva L. Baker, David Niemi

Summary

Assessing whether students can both re-present a corpus of learned knowledge and also demonstrate that they can apply that knowledge to solve problems is key to assessing student understanding. This notion, in turn, impacts our thinking about what we assess, how we author such assessments, and how we interpret assessment results. The diffusion of technology into venues of learning offers new opportunities in the area of student assessment. Specifically, computer-based simulations seem to provide sufficiently rich environments and the tools necessary to allow us to infer accurately how well a student’s individual mental model of the world can accommodate, integrate, and be used to exploit concepts from a domain of interest. In this paper then, we first identify the characteristics of simulations that our experience suggests are necessary to make them appropriate for pedagogical and assessment purposes. Next, we discuss the models and frameworks (templates) we have used to ensure these characteristics are considered. Finally, we describe two computerized instantiations (objects) of these frameworks and implications for the follow-on design of simulations.

#385 – The Evolution of a Portfolio Program: The Impact and Quality of the Vermont Program in Its Second Year (1992-1993)
Daniel Koretz, Brian Stecher, Stephen Klein, and Daniel McCaffrey

Summary

Part of an ongoing evaluation of the Vermont portfolio assessment program by RAND/CRESST researchers, this reports presents recent analyses of the reliability of Vermont portfolio scores, and the results of school principal interviews and teacher questionnaires. The message, especially from Vermont teachers, say the researchers, remains mixed. Math teachers, for example, have modified their curricula and teaching practices to emphasize problem solving and mathematical communication skills, but many feel they are doing so at the expense of other areas of the curriculum. About one-half of the teachers report that student learning has improved, but an equal number feel that there has been no change. Additionally, teachers reported great variation in the implementation of portfolios into their classroom, including the amount of assistance provided to students. "One in four teachers," found the authors, "does not assist his or her own students in revisions, and a similar proportion does not permit students to help each other. Seventy percent of fourth-grade teachers and thirty-nine percent of eighth-grade teachers forbid parental or other outside assistance." Consequently, students who receive more support from teachers, parents and other students, may have a significant advantage over students who receive little or no outside help. Reliability problems continue. "The degree of agreement," wrote the authors, "among Vermont's portfolio raters was much lower than among raters in studies with other types of constructed response measures." The authors suggest that one cause of the low reliability was the diversity of tasks within each portfolio. Because teachers and students are free to select their own pieces, performance on the tasks is much more difficult to assess than if the work were standardized. Despite these problem areas, support for the portfolio program remains high. Teachers, for example, expressed strong support for expanding portfolios to all grade levels. Seventy percent of principals said that their schools had extended portfolio usage beyond the original Vermont state mandate.

#628 – Developing Assessments to Inform Teaching and Learning
Kristin M. Bass and Robert Glaser

Summary

The centrality of assessment for facilitating thinking, reasoning, and problem solving is well-documented and indisputable. Less apparent is how to create informative, yet practical measures for classroom use. Clearly, the changing of assessments alone will not in and of itself improve learning; teachers? beliefs and practices will need to be altered with various levels of support. The design of assessment situations can nevertheless have a substantial impact on the quality of information provided to teachers and students for instructional decision-making and meaningful learning. This report considers principles of informative assessments that improve teaching and learning by communicating learning goals, interpreting student performance, tracking progress over time, and suggesting appropriate corrective actions. In the report, we describe several properties of assessment design that enable teachers and students to describe progress in terms of cognitive features of performance, and then act on that information to improve learning. We review classroom assessment programs across subject matters and grade levels in order to suggest essential design elements for tasks, score forms, and interpretive materials that maximize the information provided by assessment of performance and competence. These principles are not intended to be comprehensive, but are meant to highlight some promising areas for informative assessment research.

#704 – Developing Expertise With Classroom Assessment in K-12 Science: Learning to Interpret Student Work
Interim Findings From a 2-Year Study
Maryl Gearhart, Sam Nagashima, Jennifer Pfotenhauer, Shaunna Clark, Cheryl Schwab, Terry Vendlinski, Ellen Osmundson, Joan Herman, Diana J. Bernbaum

Summary

This article reports findings on growth in three science teachers’ expertise with interpretation of student work over 1 year of participation in a program. The program was designed to strengthen classroom assessment in the middle grades. Using a framework for classroom assessment expertise, we analyzed patterns of teacher learning, and the roles of the professional program and the quality of the assessments provided with teachers’ instructional materials.

Like this Product? Get CRESST News Every Month!

Reports

Download Our Reports

Partners & Collaborators