In this study we compared the effects of two math game designs on math and game performance, using discrete-time survival analysis (DTSA) to model players’ risk of not advancing to the next level in the game. 137 students were randomly assigned to two game conditions. The game covered the concept of a unit and the addition of like-sized fractional pieces. The math content in the baseline version of the game focused on procedures and did not elaborate on the math topics. The experimental version of the game provided more conceptual instruction by emphasizing the underlying concepts in fractional addition. Students played the game for 30 minutes. DTSA was used to examine student game performance, and whether and how game performance relates to students’ prior math knowledge and game experience. Students who played the experimental version of the game were less likely to fail in the game relative to students who played the baseline version of the game (odds ratio = 0.64). Students with higher prior knowledge of fractions also were less likely to fail in the game (odds ratio = 0.41), and students with more game experience were less likely to fail (odds ration = 0.58). The use of DTSA provided an analytical method to understand game performance and game process data. DTSA enabled examination of the game play progression of students with various characteristics over sequences of game levels.

A central challenge in efficacy studies centers on the issue of “fidelity of implementation,” that is, the extent to which participants use the curriculum specified by curriculum developers. In this study, we describe and discuss a “fidelity of implementation” model using multiple methods and instruments to compare two versions of a science curriculum and embedded assessment system. We present results from our validation study and discuss the challenges in determining the extent to which teachers use a curriculum as designed. We focus as well on the practical curriculum implementation issues amidst the needs and perspectives from different stakeholders.

This report examines enrollment history, achievement gaps, and persistence in school for ELL students and reclassified ELL students as compared to non-ELL students. The study uses statewide individual-level data sets merged from students’ entry to exit in a state public school system for graduate cohorts of 2006, 2007, and 2008. Analytic methods include multilevel logistic regression in which students are nested within districts to study correlates of dropouts. The results reconfirmed other literature showing large achievement and socio economic gaps between ELL and non-ELL students. Results also show that after accounting for academic achievement, behavioral issues, background, and district contexts, the longer a student is designated as an ELL, the more likely he or she is to drop out of school.

Drawing from a large efficacy study in upper elementary science, this report
had three purposes: First to examine the quality of teachers'
content-pedagogical knowledge in upper elementary science; second, to
analyze the relationship between teacher knowledge and their assessment
practice; and third, to study the relationship between teacher knowledge,
assessment practice, and student learning. Based on data from 39 teachers,
CRESST researchers found that students whose teachers frequently analyzed
and provided feedback on student work had higher achievement than students
whose teachers spent less time on such activities. The findings support
other research indicating the power of well-implemented formative assessment
to improve learning.

Understanding the meaning of rational numbers and how to perform mathematical operations seems to be a perennial problem in the United States for both adults and children. CRESST researchers hypothesized that giving students more time to practice using rational numbers in an environment that enticed them to apply their understanding might prove educationally beneficial. They developed a video game, based on two key ideas about addition and rational numbers, to investigate their hypothesis. in this initial study, they found that students using an appropriately design game increased their ability to add rational numbers even when playing the game for a relatively short period of time. The authors discuss implications for the larger efficacy study to follow.

In recent years, a large number of school districts have adopted interim or benchmark assessments to help inform instruction during the school year. In this study, researchers drew from interviews across seven school districts and analyzed: 1) teachers understandings of their school districts’ purposes for interim assessments; 2) professional development to support teachers effective use of assessment information; and 3) reported teacher use of the data to adjust instruction. The researchers found that although many teachers expressed an interest in using interim assessment results as intended, that they often received minimal professional development and frequently had a different understanding regarding the intended use of the assessments than did district leaders. The researchers concluded that, in general, the interim assessment data did not provide teachers with insights about what to do next to help students other than to reteach.

In order to provide more frequent information about student progress during the year, many school districts have been implementing "interim" or "benchmark" assessment programs. To date, little research has examined the implementation of interim assessments or their effects on teaching and learning. This new CRESST report investigates purposes in adopting interim or benchmark assessments, ensuing implementation efforts, and actual assessment uses. The researchers found a number of substantial barriers to success including test questions that were predominantly multiple-choice, lack of professional development for teachers, and minimal coherence in shared understandings of assessment purposes and uses across district, school, and classroom levels. Based on the results, the researchers provide recommendations for a successful interim or benchmark assessment system.

This paper introduces a new text mining framework using a tree-based Linguistic Query Language, called LQL. The framework generates more than one parse tree for each sentence using a probabilistic parser, and annotates each node of these parse trees with main-parts information. Using main-parts-annotated parse trees for a given textual dataset, the system can efficiently answer individual queries as well as mine the text for a given set of queries. The framework also has the ability to support grammatical ambiguity through probabilistic rules and linguistic exceptions in order to increase the quality of the extracted information.

Many gaming environments use narrative to communicate the background of a computer game, for example, who are the characters, what is their environment, what challenges await, and what resources are available to overcome those challenges. In this study, CRESST researchers analyzed the motivational benefits of narrative in a math game and the relationship to gender and learning outcomes. They found that the use of narrative increased student engagement and learning outcomes regardless of gender, but that learning gains only occurred for male students given the masculine narrative.

This study analyzed three different middle school mathematics formative assessment programs, examining how features of each program were associated with the information they provided to teachers and the manner in which teachers used the information.

The research team found considerable variation in the information teachers obtained from each program and how they used it. They found that greater familiarity with the specific formative assessment system did seem to be accompanied by more integrated use during the school year. They also found that teachers seemed to find it easier to incorporate the systems that had pre-existing assessments than the system that put the burden for assessment design on their shoulders.

The results from this study can aide teachers, administrators and other education stakeholders in deciding which formative assessment systems to adopt, planning for the implementation of formative assessment and providing adequate training for teachers, designing formative assessment systems that better meet teachers' needs, setting realistic expectations for the impact of formative assessment systems on a large scale, and lastly, understanding the impact of formative assessment in a particular context.