This study examined the relationship between the quality of classroom talk and academic rigor in reading comprehension lessons. In addition, the study aimed to characterize effective questions to support rigorous reading comprehension lessons. The data were collected as a part of the Instructional Quality Assessment (IQA) pilot. The IQA is a tool-kit being developed to evaluate the quality of instruction. As for the indicators of instructional quality, IQA included a set of rubrics to measure the extent to which the classroom talk is accountable (Accountable Talk rubrics), the lesson is rigorous (Academic Rigor rubrics), and the teacher’s expectations are communicated to the students (Clear Expectations rubrics). Specifically, Accountable Talk rubrics consist of seven dimensions in classroom talk: (1) participation, (2) teacher’s linking ideas, (3) students’ linking ideas, (4) asking for knowledge, (5) providing knowledge, (6) asking for rigorous thinking, (7) providing rigorous thinking. The data for this study included 21 reading comprehension lessons in several elementary and middle schools from three urban school districts. Quantitative analyses showed that the ratings on students’ providing knowledge and providing thinking rubrics had strong, positive relationships with the rating of academic rigor. These results suggest that students’ participation in classroom talk allows for a rigorous lesson. Qualitatively, the lesson transcripts were closely examined to find characteristics of teachers’ questions that engage students in high-level thinking. This study also discussed implications for effective questioning in classroom and effective indicators for instructional quality.

This report focuses on issues of validity and equity of assessments as they guide educational policies and practices for the education of limited English proficient students. Although estimates of the number of students who are English language learners (ELLs) vary, from self-reports in the 1990 census (U.S. Bureau of the Census, 1990) to surveys conducted of schools districts (Fleischman & Hopstock, 1993), their proportion is rising and may reach 10% by the end of the century. Although Spanish is the primary language for about three fourths of these students, Asian group languages - Vietnamese, Hmong, Cantonese, Cambodian, Tagalog, Laotian and Korean - are represented in large numbers. Navajo and Russian are also significantly represented.

The case of limited English proficient students is particularly instructive, for it illustrates the unprecedented challenge posed by the educational reforms of the 1990s: the simultaneous call for higher standards of performance in content areas and the inclusion of children of all backgrounds in the reform movement. Although this expanded set of requirements may be regarded by some as little more than optimistic rhetoric, state and federal legislation has been enacted to create policies and practices intended to raise the attainment of limited English proficient children. The challenge is twofold: to change the perceptions of the public and teaching personnel so that these goals may be accepted; and to achieve the twin goals of increased attainment and expanded participation. In the case of students who are not fluent in English, the situation is complicated by diverse public perceptions on the use of primary language in school. At the heart of much of the discussion is the role of language in student achievement and the expectation by a majority of the public that learning English should be a priority. Controversy exists, for instance, on the degree and length of time of maintenance of primary language in instruction. There is also a strong basic education movement in some sectors of the public, exemplified by the pressure for computationally oriented mathematics and phonics-based reading programs. These advocates take the position that the education system should demonstrate that it can teach children fundamentals before it tackles higher standards and more ambitious goals.

The great success of the American system, its retention of more students through high school, is also its downfall, for the lack of demonstrable skills for many of these students is unacceptable. As the proportion rises of students in school who have home languages other than English, pressure increases for better approaches to teach and assess their learning.

Drawing from a large efficacy study in upper elementary science, this report
had three purposes: First to examine the quality of teachers'
content-pedagogical knowledge in upper elementary science; second, to
analyze the relationship between teacher knowledge and their assessment
practice; and third, to study the relationship between teacher knowledge,
assessment practice, and student learning. Based on data from 39 teachers,
CRESST researchers found that students whose teachers frequently analyzed
and provided feedback on student work had higher achievement than students
whose teachers spent less time on such activities. The findings support
other research indicating the power of well-implemented formative assessment
to improve learning.

The development of an assessment tool to measure the quality of instruction is necessary to provide an informative accountability system in education. Such a tool should be capable of characterizing the quality of teaching and learning that occurs in actual classrooms, schools, or districts. The purpose of this paper is to describe the development of the Academic Rigor in Mathematics (AR-Math) rubrics of the Instructional Quality Assessment Toolkit and to share the findings from a small pilot study conducted in the Spring of 2003. The study described in this paper examined the instructional quality of mathematics programs in elementary classrooms in two urban school districts. The study assessed the reliability of the AR-Math rubrics, the ability of the AR-Math rubrics to distinguish important differences between districts, the relationships between rubric dimensions, and the generalizability of the assignment collection. Overall, exact reliability ranged from poor to fair, though 1-point reliability was excellent. Even with the small sample size, the rubrics were capable of detecting difference in students’ opportunities to learn mathematics in each district. The paper concludes by suggesting how the AR-Math rubrics might serve as professional development tools for mathematics teachers.

Partially funded by CRESST, the New Standards Project is an effort to create a state- and district-based assessment and professional development system that will serve as a catalyst for major educational reform. In 1992, as part of a professional development strategy tied to assessment, 114 teachers, curriculum supervisors, and assessment directors met to score student responses from a field test of mathematics and English language arts assessments. The results of that meeting, the Big Sky Scoring Conference, were used to analyze comparability across scorers and comparability across holistic and anaholistic scoring methods. Interscorer reliability estimates," wrote the researchers, "for reading and writing were in the moderate range, below levels achieved with the use of large-scale writing assessment or standardized tasks. Low reliability limits the use of [the] 1992 reading and writing scores for making judgments about student performance or educational programs," concluded the researchers. However, interscorer reliability estimates for math tasks were somewhat higher than for literacy. For six out of seven math tasks, reliability coefficients approached or exceeded acceptable levels. Use of anaholistic and holistic scoring methods resulted in different scores for the same student response. The findings suggest that the large number and varied nature of participants may have jeopardized the production of valid and reliable data. "Scorers reported feeling overwhelmed and overworked after four days of training and scoring," wrote the researchers. Despite these difficulties, evidence was provided that scoring of large-scale performance assessments can be achieved when ample time is provided for training, evaluation, feedback and discussion; clear definitions are given of performance levels and the distinctions between them; and well-chosen exemplars are used.

This study analyzed three different middle school mathematics formative assessment programs, examining how features of each program were associated with the information they provided to teachers and the manner in which teachers used the information.

The research team found considerable variation in the information teachers obtained from each program and how they used it. They found that greater familiarity with the specific formative assessment system did seem to be accompanied by more integrated use during the school year. They also found that teachers seemed to find it easier to incorporate the systems that had pre-existing assessments than the system that put the burden for assessment design on their shoulders.

The results from this study can aide teachers, administrators and other education stakeholders in deciding which formative assessment systems to adopt, planning for the implementation of formative assessment and providing adequate training for teachers, designing formative assessment systems that better meet teachers' needs, setting realistic expectations for the impact of formative assessment systems on a large scale, and lastly, understanding the impact of formative assessment in a particular context.

Two consortia, the Smarter Balanced Assessment Consortium (Smarter Balanced) and the Partnership for Assessment of Readiness for College and Careers (PARCC), are currently developing comprehensive, technology-based assessment systems to measure students’ attainment of the Common Core State Standards (CCSS). The consequences of the consortia assessments, slated for full operation in the 2014/15 school year, will be significant. The assessments themselves and their results will send powerful signals to schools about the meaning of the CCSS and what students know and are able to do. If history is a guide, educators will align curriculum and teaching to what is tested, and what is not assessed largely will be ignored. Those interested in promoting students’ deeper learning and development of 21st century skills thus have a large stake in trying to assure that consortium assessments represent these goals.

Funded by the William and Flora Hewlett Foundation, UCLA’s National Center for Research on Evaluation, Standards, and Student Testing (CRESST) is monitoring the extent to which the two consortia’s assessment development efforts are likely to produce tests that measure and support goals for deeper learning. This report summarizes CRESST findings thus far, describing the evidence- centered design framework guiding assessment development for both Smarter Balanced and PARCC as well as each consortia’s plans for system development and validation. This report also provides an initial evaluation of the status of deeper learning represented in both consortia’s plans.

Study results indicate that PARCC and Smarter Balanced summative assessments are likely to represent important goals for deeper learning, particularly those related to mastering and being able to apply core academic content and cognitive strategies related to complex thinking, communication, and problem solving. At the same time, the report points to the technical, fiscal, and political challenges that the consortia face in bringing their plans to fruition.

This is an evaluation report for Year 6 of the Math and Science Academy (MSA), an initiative of the Los Alamos National Laboratory. A brief overview of the project, with goals and framework is presented first, followed by a description of methods used for the evaluation. Next, findings from the Year 6 Evaluation are described, including program impact on students and teachers. The report concludes with recommendations for future years of the program.

More than fifteen years ago, a prominent national commission declared us a nation at educational risk, noting "a rising tide of mediocrity that threatens our very future as a nation" (National Commission on Excellence in Education, 1983). A decade later, California received its own special wake-up call when results from the 1990 and 1992 National Assessment of Educational Progress (NAEP) state-by-state comparisons revealed that California students were scoring near the bottom nationally in eighth-grade mathematics and fourth-grade reading. What of the situation today? How are California's students faring? Are our students making progress toward the rigorous standards that have been established for their performance? Are our schools improving? Are they better preparing our students for future success? As we strive toward excellence, who is being helped most and who not by California's educational system?

Answers to these seemingly simple, bottom-line questions are complex to formulate, made more so by the history and current status of the state's assessment system, the nature of other available indicators of educational quality, and the imprecision of all assessments. In this report, the authors provide a context for examining the progress of students and schools by reviewing California's recent testing history and the state's progress in creating a sound, standards-based assessment system. Then, they review available data about student performance, examining how schools are doing and the factors that most influence assessment results, and close with a discussion of the goals of accountability and standards by which such systems should be judged.

In this three-state study, the authors estimate the magnitudes of achievement gaps between EL students and their non-EL peers, while avoiding typical caveats in cross sectional studies. The authors further compare the observed achievement gaps across three distinct dimensions (content areas, grades, and states) and report patterns of EL and non-EL achievement gaps within and across states. The study findings suggest that linguistic barriers and long-term EL designation may contribute to the observed achievement gaps. The findings further suggest that the differences in the stringency of state reclassification criteria may influence the reported size of the EL and non-EL achievement gaps between states.


To cite from this report, please use the following as your APA reference:

Kim, J., & Herman, J. L. (2009). A three-state study of English learner progress (CRESST Report 764). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).