The Artist-Teacher Collaborative (ATC) program is a partnership between the Armory Center for the Arts and Pasadena Unified School District (PUSD). A central goal of this program, which is supported by the U. S. Department of Education, is to provide sustained, rigorous professional development to 3rd through 5th grade PUSD teachers, instilling in them the skills and knowledge needed to deliver standards-based instruction in the visual arts. Using survey methodology, classroom observations, and quantitative techniques, CRESST evaluated the quality the program. This year-end report addresses the ongoing program¹s success and provides formative feedback on the strengths and weaknesses of program implementation.

The main purpose of this study was to examine the effects of the Reading Apprenticeship professional development program on several teacher and student outcomes, including effects on student learning. A key part of the study was the use of an enhanced performance assessment program, the Integrated Learning Assessment (ILA), to measure student content understanding. The ILA instruments included multiple components that assessed student content knowledge, reading comprehension, metacognition, use of reading strategies, and writing skills in applied knowledge. An analysis of student scores using the ILA found little or no significant effects from the Reading Apprenticeship program on class-level student outcomes. However, the researchers found a significant positive effect on teachers' literacy instruction.

In the United States, child care arrangements serve increasingly linguistically diverse populations of children. However, little is known about patterns of childcare arrangements for language-minority children and the linguistic environment of child care arrangements. Using the Early Childhood Program Participation Survey of the 2001 National Household Education Surveys Program (ECPP-NHES, 2001), this report asks three research questions: (1) what are the patterns of child care arrangements for language-minority children?, (2) do child care arrangements differ by language use of the child?, and (3) what factors are associated with the child having a care provider who speaks a non-English language? The results show that after controlling for the characteristics of the child, children are less likely to be in nonparental care (i.e., center-based programs, and relative care or nonrelative care) when the child speaks a non-English language mostly. However, this association disappears when characteristics of parents and household are controlled. The results also suggest that language use of the child as well as parents are important factors associated with the care provider’s language use. These findings may suggest that an additional context, the linguistic environment of child care arrangements, contribute to a child’s non-English language use.

The present study piloted a survey-based measure of Opportunity to Learn (OTL) and Academic Language Exposure (ALE) in fourth grade science classrooms that sought to distinguish teacher practices with ELL (English language learner) and non-ELL students. In the survey, participant teachers reported on their instructional practices and the context in their science classrooms. A small sub-sample was also observed teaching a lesson in their classroom on two occasions. The pilot data were used to investigate basic psychometric properties of the survey: specifically (a) the dimensions underlying the survey items, in particular whether OTL and ALE are distinct or overlapping features or dimensions of science instruction and (b) the match between information reported by teachers in the survey, and that collected by classroom observers. Qualitative analyses of observation and teacher open ended responses in the survey informed the interpretation of the quantitative analysis results and provided useful insights for refining the survey instrument to better capture the classroom experiences of ELL students.


To cite from this report, please use the following as your APA reference:

Martinez, J. F., Bailey, A. L., Kerr, D., Huang, B. H., & Beauregard, S. (2010). Measuring Opportunity to Learn and Academic Language Exposure for English Language Learners in Elementary Science Classrooms (CRESST Report 767). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

In the latter part of the 1990s, education in California was caught in a whirlwind of change. Schools scrambled to find enough teachers and enough classroom space to fulfill state-mandated class-size reduction requirements. The voters eliminated bilingual education, leaving schools with no specific classroom tool for teaching English-language learners. Schools were required to administer a new standardized test each spring to Grades 2 through 11 that was not aligned to classroom work and yet carried great weight for both students and educators.

Amidst this upheaval, a major new school-reform initiative was trying to make headway in Los Angeles County. From 1994 through 2000, the Los Angeles Annenberg Metropolitan Project, or LAAMP, was one of 18 major school improvement initiatives across the country to be funded by the $1.1 billion Annenberg Challenge. Its centerpiece was a new educational structure known as the School Family, which brought together teachers, administrators, and parents from high schools and their feeder middle schools and elementary schools, plus others with an interest in education. LAAMP organizers hoped the School Families would create a stable learning environment for students by encouraging coordination among schools and between grade levels.

Today, the Annenberg Challenge has drawn to a close. A final report released in June called the national effort a partial success. The report credited the program with strengthening urban, rural, and arts education and with raising the quality of teaching. The report also found that school-reform programs must learn to deal with rapid leadership turnover, changes in direction, and other setbacks. And it found that the grant money, while generous, frequently was spread too thin over too many schools.

The national findings parallel conclusions drawn about the 6-year Los Angeles program, which received $53 million from the Annenberg Challenge in December 1994. LAAMP commissioned a group of education researchers from UCLA and USC to evaluate the local project. Known as the Los Angeles Consortium for Evaluation, or LACE, the researchers found that LAAMP accomplished some of what it set out to do. But for a variety of reasons, it did not attain its ultimate goal of improving student performance.

The tumultuous period of California history that suctioned off time, energy, and financial resources from schools and the people working in them bears much of the blame. Researchers found other explanations. Among them were:
• School Family teams of teachers, administrators, and parents needed more time than was anticipated to develop the group process skills necessary for success and spent much of their time trying to learn how to collaborate instead of instituting change.
• The teams needed time to learn about and understand the concepts of results- or standards-based school reform and to develop the skills required to analyze available data and use them in the planning process.
• There were insufficient resources to ensure adequate support for teachers attempting to implement programs devised by the School Families. There also was no mechanism for extending the reforms to teachers not directly involved in the reform project.

LACE also acknowledged that the research methodologies it used to evaluate LAAMP, although the best that were available, might not have presented a full and accurate picture of the effects the program had on its participating schools. In addition, researchers suggested the need for more sensitive gauges of student accomplishment that measure the actual curriculum taught. The primary measurement used—California’s Stanford 9 test—may not have been the best tool for detecting the effects of specific changes in teaching and learning.

Overall, the researchers found that the LAAMP reform can claim many achievements that have benefited K-12 education in Los Angeles County, including:
• Creation of the School Family concept, which in many cases was responsible for productive changes that could not have been realized by a single school working alone.
• Strengthening of schools’ acceptance of accountability, their focus on performance, and their capacity for self-evaluation especially in regard to accessing and using student-achievement data.
• Creation of valuable teacher professional development activities and access to new instructional programs, which were especially helpful for the many new and uncredentialed teachers who were hired to fulfill class-size reduction requirements.
• Encouragement of parental involvement in the schools and in children’s learning at home, which had demonstrable effects on student performance.
• Demonstration of the potential of stable learning communities for curing many of the ills facing urban schools.

Looking at test scores, LACE researchers saw improvement at LAAMP schools over the 3-year period from 1997-1998 to 2000-2001. However, there was no statistically significant difference between LAAMP schools and non-LAAMP schools with regard to student performance on the state’s Stanford 9 standardized test.

Researchers also found no indication that LAAMP has had a wide impact on classroom practices. In other words, its core school-reform principles have not yet permeated participating schools. However, researchers saw signs that LAAMP initiatives were starting to move into the classroom in the later years of the program after so much time and energy were spent initially on developing the School Family structure.

When Walter Annenberg issued his challenge in December 1993 by giving what at the time was the largest gift ever dedicated to improving public education, he called it a “crusade for the betterment of our country.” Nine years later, that crusade has made a difference. The public schools “in most major cities are still not doing the job they must,” the June report said, but they are “better today than they were a decade ago and teachers are better equipped to help children overcome obstacles and achieve higher standards.”

In this report, Lorraine McDonnell continues her synthesis of innovative state assessment programs in Kentucky (Kentucky Instructional Results Information System), California (California Learning Assessment System), and North Carolina. Using results from interviews with state policy makers and their staff, articles in major newspapers, and other relevant documentary materials, McDonnell analyzed the key factors in the enactment and initial implementation of each state's accountability program.

North Carolina assumed more modest instructional changes that were reflected in a continued dependency on multiple-choice items on the state's new assessment. "In contrast," said McDonnell, "what is remarkable about California and Kentucky is that not only was the state specifying a more precise curriculum through its framework and assessment than it had traditionally, but the curriculum was unfamiliar to most adults in the state." For a state accountability program to succeed, maintains McDonnell, the development of new curriculum standards and assessments cannot be solely a technical process with participation limited to experts.

In this study, Mary Lee Smith and other researchers focused on how schools changed as a result of state-mandated standards and assessments. Drawing data from both qualitative and quantitative methods, the researchers concluded that achievement of the policy goals of the Arizona Student Assessment Program (ASAP) was limited to those schools that had adequate resources to provide professional and curriculum development, and where school philosophies and practices were already consistent with the ASAP program. Inadequate state attention to capacity-building, to the technical qualities of the performance tests, and to issues of equity created barriers to implementation of the ASAP goals. For example, less than one fifth of the teachers surveyed believed that adequate professional development had been provided relative to ASAP. Most teachers distrusted the validity of the tests and believed that the performance assessments were no more fair than standardized tests to poor children, children of color, and language minority children.

The evidence of this study, concluded Smith, shows that the "opportunities of these children to learn, for example, higher order thinking skills, are considerably less than those of other children."

Glossary and reading aloud test items are often listed as allowed in many states' accommodation policies for ELL students, when taking states' large-scale mathematics assessments. However, little empirical research has been conducted on the effects of these two accommodations on ELL students' test performance. Furthermore, no research is available to examine how students use the provided accommodations. The present study employed a randomized experimental design and a think-aloud procedure to delve into the effects of the two accommodations. A total of 605 ELL and non-ELL students from two states participated in the experimental component and a subset of 68 ELL students participated in the think-aloud component of the study. Results showed no significant effect of glossary, and mixed effects of read aloud on ELL students' performance. Read aloud was found to have a significant effect for the ELL sample in one state, but not the other. Significant interaction effects between students' prior content knowledge and accommodations were found, suggesting the given accommodation was effective for the students who had acquired content knowledge. During the think-aloud analysis, students did not actively utilize the provided glossary, indicating lack of familiarity with the accommodation. Implications for the effective use of accommodations and future research agendas are discussed.


To cite from this report, please use the following as your APA reference:

Wolf, M. K., Kim, J., Kao, J. C., & Rivera, N. M. (2009). Examining the effectiveness and validity of glossary and read-aloud accommodations for English language learners in a math assessment (CRESST Report 766). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

In response to the growing achievement gap between English Learners (ELs) and non-ELs, standards-based instruction and assessment have been promulgated at the state and federal level. Yet, the consequences of standards-based assessment reforms for ELs have rarely been systematically studied. The work reported here represents the initial study of a 4-year research project with the purpose of investigating how the implementation of standards-based performance assessments and related instructional strategies influences the achievement of ELs. In this study, we were specifically interested in identifying the opportunity-to-learn (OTL) variables that positively impact student performance. We also investigated potential differences in the impact of OTL on performance between ELs and non-ELs. Our study suggested that there are several factors contributing to studentsð performance on the Language Arts Performance Assignment (LAPA). At the student level, the analysis suggested that the greatest contributors to individual studentsð LAPA scores were performance on the Stanford 9 Language test, ethnicity, gender, and language proficiency status. At the teacher level, we found that content coverage was significantly associated with student performance. The study showed that higher levels of content coverage in both writing and literary analyses were associated with higher performance for all students, including ELs. We also found differential impact of one OTL variable, content coverage-writing, on ELs performance. This finding indicates that the gap between ELs and non-ELs increases as teacher reports of content coverage-writing increase.

Educational reform leaders generally agree that professional development opportunities for teachers are crucial to the success of any effort to make meaningful, sustainable changes in educational practice. As Fullan (1991) explained, "Continuous development of all teachers is the cornerstone for meaning, improvement, and reform. Professional development and school development are inextricably linked" (p. 315). Kentucky Department of Education (KDE) personnel charged with the responsibility to operationalize the Kentucky Educational Reform Act (KERA) understood this link. They developed an extensive professional development (PD) program to help Kentucky educators achieve the ambitious KERA goals. In this paper we describe the Department's multi-faceted approach to professional development and provide evidence for its impact on schools' achievement of KERA goals. We draw upon data from the exemplary case study component of a larger research project, The Effects of Standards-Based Assessments on Schools and Classrooms.

Perhaps the biggest challenge that KDE faced in providing PD services was geography. Many of Kentucky's school districts are located in remote rural areas, accessible only by mountain roads which are particularly treacherous to travel during the winter months. To reach these districts, KDE relied on a system of nine regional service centers, which provided a wide variety of services to districts, schools, and individual teachers. However, as Ed Reidy, then-Deputy Commissioner of Education, explained, "We have a real commitment that what kids learn should not be a function of geography --You could draw a circle around [the regional service] centers. Most of our audited schools were outside those circles and most were poor." To supplement the work of the centers, KDE developed a variety of materials and activities specifically designed to meet emerging needs of teachers as they worked to achieve KERA goals. This paper focuses on the two major categories of services--school-based professional development and professional development for mathematics and writing portfolios.

All four case study schools exhibited a strong commitment to professional development and a belief in the importance of ongoing support for teacher learning. They used state PD resources to enhance their instructional programs in areas explicitly connected to KERA, such as curriculum alignment and development of materials and activities keyed to the core content standards. Further, teachers at each school served in leadership roles in the KDE Division of Portfolio Initiatives professional development activities. These teachers saw their leadership roles as benefiting their schools, their students, and their colleagues, as well as supporting their own professional growth. Thus, using state resources and opportunities, these four exemplary schools created extensive professional development programs to suit the specific needs of their teachers and students. Through their successful efforts, they provide an existence proof that Kentucky's approach to professional development can provide the resources needed to support statewide, standards-based educational reform. The paper concludes with recommendations for approaches to professional development that seem to hold promise for facilitating statewide standards-based educational reform efforts.