National Center for Research on Evaluation, Standards and Student Testing

#602 – Teachers’ Assignments and Student Work: Opening a Window on Classroom Practice
Lindsay Clare Matsumura, Jenny Pascal

Summary

In this report, four years of CRESST's research is described developing indicators of classroom practice that have the potential to be used in large-scale settings and that draw attention to important aspects of standards-based learning and instruction. CRESST's method was based on the collection of teachers' assignments with student work. The assignments then were rated and results were summarized to create indicators of classroom practice. Results to date indicated an acceptable level of inter-rater reliability across study years. It likely would be necessary to collect as many as three or four assignments from teachers to obtain a stable estimate of quality. Additionally, this method was reliable when teachers created their own assignments, but not when teachers submitted assignments created by outside sources. The quality of classroom assignments was associated with the quality of observed instruction, as well as the quality of students' written work. Students who were exposed to teachers who created more cognitively challenging assignments and who had clearer grading criteria also made greater gains on the Stanford Test of Achievement, 9th Edition (Stanford 9). The quality of teachers' assignments submitted at each of the study years, however, tended to be of basic quality only. Teachers' reactions to the data collection and implications for the use of this method in collaborative professional development sessions also are discussed.

#768 – What Works? Common Practices in High Functioning Afterschool Programs Across the Nation in Math, Reading, Science, Arts, Technology, and Homework--A Study by the National Partnership
Denise Huang, Jamie Cho, Sima Mostafavi, Hannah H. Nam

Summary

In an effort to identify and incorporate exemplary practices into existing and future afterschool programs, the U.S. Department of Education commissioned a large-scale evaluation of the 21st Century Community Learning Center (CCLC) program. The purpose of this evaluation project was to develop resources and professional development that addresses issues relating to the establishment and sustainability of afterschool programs. Fifty-three high functioning programs representative across eight regional divisions of the nation, including rural and urban programs, community-based and school district related programs, were identified using rigorous methods. Exemplary practices in program organization, program structure, and especially in content delivery were studied. The findings were synthesized into the Afterschool Toolkit that was made available to programs nationwide via the world-wide-web. Professional development was conducted consistently and extensively throughout the nation.

To cite from this report, please use the following as your APA reference:

Huang, D., Cho, J., Mostafavi, S., Nam, H., H., Oh, C., Harven, A., & Leon, S. (2009). What works? Common practices in high functioning afterschool programs across the nation in math, reading, science, arts, technology, and homework—A study by the National Partnership. The afterschool program assessment guide (CRESST Report 768). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

#806 – District Adoption and Implementation of Interim and Benchmark Assessments
Kristen L. Davidson and Greta Frohbieter

Summary

In order to provide more frequent information about student progress during the year, many school districts have been implementing "interim" or "benchmark" assessment programs. To date, little research has examined the implementation of interim assessments or their effects on teaching and learning. This new CRESST report investigates purposes in adopting interim or benchmark assessments, ensuing implementation efforts, and actual assessment uses. The researchers found a number of substantial barriers to success including test questions that were predominantly multiple-choice, lack of professional development for teachers, and minimal coherence in shared understandings of assessment purposes and uses across district, school, and classroom levels. Based on the results, the researchers provide recommendations for a successful interim or benchmark assessment system.

#530 – Parents and Teachers Working Together to Support Third-Grade Achievement: Parents as Learning Partners (PLP) Findings
Denise D. Quigley

Summary

The Los Angeles Annenberg Metropolitan Project (LAAMP), in an effort to have schools work more collaboratively with parents surrounding academic issues, granted funds matched by the Weingart Foundation to 29 schools in three School Families in Los Angeles County to focus on three primary areas in which parents and teachers can work together to support children's academic progress: communication, parenting, and learning at home. This effort is entitled Parents as Learning Partners (PLP). The PLP initiative recognizes that when parents and teachers share common responsibility and expectations, assist each other in providing learning experiences, and establish two-way communication, students will develop bet

This report describes goals and context of the Parents as Learning Partners (PLP) evaluation as well as provides an overview of third-grade classrooms both in the schools in which PLP is operating and in similar schools selected as comparison schools. Specifically, this report describes parent involvement goals of the schools; student, teacher, and classroom characteristics; professional development activities and voicemail usage of teachers; teacher-interactions with their students' parents including communication patterns and parent involvement at the school and in the classroom; barriers to parent participation, particularly surrounding parent education workshops; access, usage, and satisfaction with parent education and other school services; parents' habits and structure at home regarding reading and homework; and the behavior and achievement of third graders. Furthermore, the report summarizes the findings and concludes with the limitations of the PLP program and programmatic recommendations.

#417 – Final Report on an Evaluation of the California Mathematics Diagnostic Testing Project
John R. Novak, Alexander Chizhik, and Susane Moran

Summary

In this report, a CSE evaluation team found that concerns that the results from the California Mathematics Diagnostic Testing Project were being improperly used as components of student grades were largely ungrounded. However, the researchers did find that the tests were often being used to make student placement decisions, a usage that is not recommended or intended by the creators of the tests. According to the researchers, ninety percent of the teachers reported use of test results to inform student placement decisions and 56% said that placement was the most important use of the test results at their site. The intended purpose of the tests was to inform teachers about student mathematics readiness prior to entering California universities.

"The magnitude of this usage," concluded the authors, "certainly indicates that further examination of the ways in which the test results are used is warranted."
In a second part of the evaluation, the researchers found that teacher classroom practices were not influenced by their use of the MDTP mathematics tests.

#637 – Children Left Behind In AYP and Non-AYP Schools: Using Student Progress and the Distribution Of Student Gains to Validate AYP
Kilchan Choi, Michael Seltzer, Joan Herman, and Kyo Yamashiro

Summary

The No Child Left Behind Act (NCLB, 2002) establishes ambitious goals for increasing student learning and attaining equity in the distribution of student performance. Schools must assure that all students, including all significant subgroups, show adequate yearly progress toward the goal of 100% proficiency by the year 2014. In this paper, we wish to illustrate an alternative way of measuring AYP that both emphasizes individual student growth over time, and focuses on the distribution of student growth between performance subgroups. We do so through analyses of a longitudinal dataset from an urban school district in Washington. We also examine what these patterns tell us about schools that meet their AYP targets and those that do not. This alternative way of measuring AYP helps bring to light potentially important aspects of school performance that might be masked if we limit our focus to classifying schools based only on current AYP criteria. In particular, we are able to identify some schools meeting the Washington state criteria for AYP, for example, that have above average students making substantial progress but below average students making little to no progress. In contrast, other schools making AYP have below average students making adequate progress but above average students making little gains. These contrasts raise questions about the meaning of "adequate" progress and to whom the notion of progress is referring. We believe that closely examining the distribution of student progress may provide an important supplementary or alternative measure of AYP.

#726 – The Role of Teacher Discourse in Effective Groupwork
Noreen M. Webb, Megan L. Franke, Marsha Ing, Angela Chan, Tondra De, Deanna Freund, Dan Battey

Summary

Prior research on collaborative learning identifies student behaviors that significantly predict student achievement, such as giving explanations of one’s thinking. Less often studied is how teachers’ instructional practices influence collaboration among students. This report investigates the extent to which teachers engage in practices that support students’ explanations of their thinking, and how these teacher practices influence the nature of explanations that students give when asked by the teacher to collaborate with each other. In this study, we videotaped and audiotaped teacher and student participation, and measured student achievement, in second- and third-grade mathematics classrooms working on algebraic concepts of equality and relational thinking. The teachers observed here, all of whom received specific instruction in eliciting the details of student thinking, varied significantly in the extent to which they asked students to elaborate on their suggestions. This variation corresponded strongly to variation across classrooms in the nature and extent of student explanations during collaborative conversations, and to differences in student achievement.

#575 – Knowledge Mapper
Authoring System Prototype
Gregory K. W. K. Chung, Eva L. Baker, and Alicia M. Cheak

Summary

We have developed a prototype authoring system for knowledge mapping that provides the capability to (a) specify map tasks (i.e., specify terms and links), (b) launch a knowledge map related to a task, (c) specify criterion or expert maps (for scoring purposes), (d) define groups of users and associate tasks to those groups, and (e) publish tasks for global access. The prototype interface is a simple HTML interface, with any destination function accessible within four mouse clicks. A usability study was conducted with one teacher and 62 fourth- and fifth-grade students. Students were considered beta testers, and the teacher, who had used our existing knowledge mapper, was considered an expert. The teacher reported using the authoring system to create pretest and posttest tasks for students and to access and evaluate student maps. Students used the authoring system to create knowledge maps of their group research projects. The teacher reported that the authoring system was easy to use and thought student authoring promoted ownership of work. Students using the authoring system created more sophisticated knowledge maps, especially in the links specified by students. This anecdotal report is bolstered by analyses of the links used across all student-authored maps. Thirty-eight percent of the links were causal (e.g., causes) or functional (e.g., protects), and 24% were part-whole (e.g., part of). Limited screen real estate, confusing map access, complicated login functionality, and system crashes limited usability. Student performance on a usability task showed that, in general, students were able to independently carry out major authoring and mapping tasks; however, a positive relationship was found between the amount of time a student was at the keyboard and the amount of help that student needed during the usability task, suggesting that practice using the authoring system improves fluency with the authoring system.

#371 – Can Portfolios Assess Student Performance and Influence Instruction? The 1991-92 Vermont Experience
Daniel Koretz, Brian Stecher, Stephen Klein, Daniel McCaffrey, and Edward Deibert

Summary

Vermont's statewide assessment initiative program has garnered widespread attention nationwide because of its reliance on portfolios of student work. This 145 page report describes the results of a multifaceted evaluation of the program and provides information about the implementation of the Vermont assessment; program effects on educational practice; reliability and validity of portfolio scores; and tensions that exist between assessment and instructional reform. "Findings from the evaluation," said the research team, "suggest that the assessment program resulted in changes in curriculum content and instructional style." Additionally, the researchers noted that the amount of classroom time devoted to problem solving increased, as did the amount of time students worked in small groups. Finally, portfolios seem to increase teachers' enthusiasm for their subjects and for teaching. While there was widespread support for the reform at the school level throughout the state--nearly one-half of the schools were voluntarily expanding the use of portfolios to other grade levels--substantial problems remained. The mathematics portfolio assessment created new burdens for principals, teachers and students; including demands on teachers' time and school resources. Over 80% of fourth-grade teachers and over 60% of eighth-grade teachers reported that they often had difficulty covering the required curriculum. Researchers anticipate that some of these demands are likely to decline with experience, although others represent continuing burdens. "The Vermont experience has important implications for reforms that are underway or under consideration in other jurisdictions," wrote the researchers, "but only time and careful scrutiny will show how fully the goals of the program--and of similar reform programs centered on performance assessment--can be met."

#633 – Issues in Assessing English Language Learners’ Opportunity to Learn Mathematics
Joan L. Herman and Jamal Abedi

Summary

The Annual Yearly Progress (AYP) requirements of No Child Left Behind (NCLB) underscore both the mandate and the challenge of assuring that English Language Learners (ELL) achieve the same high standards of performance that are expected of their native English speaking peers. The intent indeed is laudable: states, districts, schools, and teachers must be accountable for the learning of their ELLs as are the students themselves. ELLs can no longer be invisible in the educational system, their learning needs must be met, and they too must make steady progress the goal of all students being judged proficient based on statewide testing by the year 2014. Already, however, NCLB results suggest a different reality: ELL subgroups are being left behind and schools and districts serving significant proportions of ELLs are less likely to meet their AYP goals and more likely to be subject to corrective action. Fairness demands that ELLs have equitable opportunity to learn (OTL) that upon which they are assessed, especially if those assessments carry significant future consequences. Moreover, if NCLB goals are to be met and achievement gaps reduced, schools must move beyond the performance only orientation of AYP to understand why results are as they are and how to improve them. OTL data can help to provide guidance in these areas and to acknowledge the reality that ELLs’ learning is unlikely to improve unless and until students have more effective opportunities to attain expected performance standards. We view this study as an interesting beginning. It was conceived as a pilot, the results of which add fuel to the concern for and underscore some of the complexities of adequately measuring OTL, and we look forward to the full study involving a larger and more representative sample of teachers and classrooms and a more robust outcome measure.

Like this Product? Get CRESST News Every Month!

Reports

Download Our Reports

Partners & Collaborators