Comparison of IMP Students with Students Enrolled in Traditional Courses on Probability, Statistics, Problem Solving, and Reasoning

April 1997

Norman Webb
Maritza Dowling

Wisconsin Center for Education Research
School of Education
  University of Wisconsin-Madison

The Interactive Mathematics Program Evaluation Project operates under the auspices of the Wisconsin Center for Education Research and is funded under a contract with the San Francisco State University Foundation, Inc., with resources that are provided by the National Science Foundation (award number ESI-9255262). The opinions, findings, and conclusions that are expressed in this paper do not necessarily reflect those of the supporting agencies.

Table of Contents

Summary  

            A series of three studies compared the performance of students enrolled in the Interactive Mathematics Program (IMP) with students enrolled in the traditional algebra 1, geometry, and algebra 2 sequence. The studies, conducted at the end of the 1995-96 school year, compared the performance of IMP students with students enrolled in the traditional college-preparatory course sequence on activities using probability, statistics, quantitative reasoning, and problem solving. The studies were done independently of each other at three different schools, with ethnically diverse populations, different outcome measures, and different grade levels: grade 9, grade 10, and grade 11. 

            The grade 9 test consisted of all four statistics items released by the Second International Mathematics Study (SIMS). The multiple-choice items used by SIMS were modified to a constructed-response format. The results on these four items were analyzed for 115 students who were enrolled in either IMP Year 1 or algebra 1. The grade 10 test consisted of two performance assessment activities prepared for the Wisconsin Student Assessment System. Each activity required students to produce a response by solving a multi-step mathematics problem, making a generalization, and writing an explanation of the procedures used. One activity required some conceptual knowledge of probability. The other activity required some knowledge of combinatorics and pattern recognition. The test was administered to 184 students who were enrolled either in IMP Year 2 or geometry. The grade 11 test consisted of 10 multiple-choice items taken from a practice version of a quantitative reasoning test developed by a prestigious university to test the knowledge of entering students. This test required data and graph interpretation, and basic understanding of probability and statistics concepts such as the standard deviation and the mean of a distribution. The test was administered to 133 students enrolled in either IMP Year 3 or algebra 2.

              In each of the three studies, IMP students performed significantly better on tests compared to students in the traditional mathematics course sequence. These results were true even when accounting for differences between groups in prior mathematics achievement as measured by grade 8 standardized norm-referenced test scores. An analysis of covariance was performed to account for any differences in prior mathematics achievement. Matched-group analyses also were performed in each study as a second method to control for any differences due to prior mathematics achievement, ethnic background (when possible), and sex. 

            The results obtained across the three schools and three grade levels support the assertion that students who took IMP gained knowledge and reasoning skills beyond what students learned in the traditional mathematics sequence.

Comparison of IMP Students with Students Enrolled in Traditional Courses on Probability, Statistics, Problem Solving, and Reasoning  

Norman L. Webb and Maritza Dowling

Wisconsin Center for Education Research, School of Education,
University of Wisconsin-Madison
 

            An increasing number of studies have compared the mathematical knowledge of students enrolled in the Interactive Mathematics Program (IMP) with that of students enrolled in the traditional algebra 1, geometry, algebra 2 sequence. Accumulating evidence shows that IMP students perform as well as, if not better than, students taking the traditional curriculum, as measured by standardized norm-referenced tests such as the Scholastic Assessment Test (SAT), Pre-Scholastic Assessment Test (PSAT), and the Comprehensive Test of Basic Skills (Interactive Mathematics Program, 1995; Schoen, 1993; S. Chew, personal communication, November, 1996; Webb & Dowling, 1995a, 1995b, 1995c, 1996). These traditional instruments measure students' knowledge of very general mathematics skills and reasoning in mathematics. Probability, statistics, quantitative reasoning, and problem solving are given little or no attention on these instruments however. It is increasingly important for students to have knowledge in these areas. This has been recognized by the NCTM Curriculum and Evaluation Standards for School Mathematics (1989) and by various state standards such as South Carolina Mathematics Framework (1993), Illinois Academic Standards Project (1996), and New Jersey Mathematics Curriculum Framework (Rosenstein, Caldwell, & Crown, 1996). Therefore, we set out to examine the performance of IMP students on these critical criteria, which have not been focused upon by past research.

              A series of three studies was designed to produce information on how well IMP students perform on activities using probability, statistics, reasoning, and problem solving. The IMP students were compared to students who were enrolled in the traditional algebra 1 (Dowling & Webb, 1997a), geometry (Dowling & Webb, 1997b) and algebra 2 sequence (Dowling & Webb, 1997c). 

            The IMP curriculum is a four-year college-preparatory sequence of courses designed for grades 9 through 12. The IMP curriculum integrates traditional areas of mathematics such as algebra, geometry, and trigonometry with probability, statistics, discrete mathematics, and matrix algebra. Students are challenged to actively explore open-ended situations, in a way that closely resembles the inquiry methods used by mathematicians and scientists. IMP calls on students to experiment with examples, look for and articulate patterns, and make, test and prove conjectures. The problem-based curriculum is organized into five-to-eight week units, each centered on a central problem or theme. Students engage in solving both routine and non-routine problems, use graphing calculators, and are encouraged to work cooperatively.

            IMP, as a comprehensive college-preparatory four-year course sequence, incorporates mathematical ideas and concepts from the traditional high school curriculum of algebra, geometry, and trigonometry. Other subject areas, including probability and statistics, are incorporated throughout the program. Students study topics such as how to calculate simple probabilities, the distinction between theoretical and experimental probabilities, the meaning of a best-fitting line for a set of data, properties of normal and other distributions, the calculation and use of standard deviation, and the notion of testing a null hypothesis and methods for doing so. 

            Teaching techniques used in IMP are designed to help students gain deep understanding of mathematical ideas, reason mathematically, and apply mathematics to solve problems. For example, one night's homework involves four sets of data about which students are to answer a number of questions, including about the spread from the mean, the standard deviation, and the similarity among the data sets (Fendel, Resek, Alper, & Fraser, 1997, p. 343). Students also are assigned Problems of the Week (POWS) to work on for five or more days in addition to daily homework. Students' ability to reason is enhanced in many ways. For instance, students are asked to design experiments, to state their conclusions based on evidence and analyses, to compare mathematical ideas, and to explain general cases of specific situations. 

            Classroom experiences such as presentations, written explanations, and small group activities are structured for students to verbalize their thinking. This verbalization is designed to increase students' understanding and to improve their communications of mathematics. Students are to become more independent learners by using multiple sources of information including their teachers, the textbook, classmates, and other references. 

Design  

            To make valid comparisons between IMP and the traditional mathematics course sequence, specific controls were imposed on the design of these three studies. Each participating school had to provide grade 8 standardized norm-referenced mathematics test scores on a significant number of IMP and traditional mathematics students who were tested. An adequate number of students, as close to 50 as possible or more in each program, had to be available to be tested. This meant that there had to be more than one IMP class at the target grade level. One or more teachers of the traditional mathematics course taken by students at the target grade level had to agree to participate in the study. The tests had to come from a source independent of IMP and had to be easily administered under the same conditions to classes of students in both course sequences. The inconvenience to teachers and students had to be kept to a minimum. Students' knowledge on as wide a range of content as possible was sought. Regional and school factors had to be reduced as much as possible. Effects had to be attributed to the curriculum rather than other factors such as teacher, school, or region.

            Three studies were conducted. Each study was done in a different school at a different grade level with a different outcome measure. This design was chosen to meet the criteria above while maximizing the comparative information and minimizing the interference within any one school (Figure 1, p. 3). Three different instruments were used. The grade 9 test (Appendix A) was composed of all statistics items (a total of four) released from the Second International Mathematics Study (SIMS) (Crosswhite et al., 1986). These items were administered to grade 12 students in the 1981-82 school year. The items were modified from the multiple-choice format used in SIMS to an open-response format. The grade 10 test consisted of two performance assessment activities prepared for the Wisconsin Student Assessment System under the auspices of the Wisconsin Department of Public Instruction. Each activity requires students to construct a response by solving a multi-step mathematics problem, generalizing the results obtained, and writing an explanation of the reasoning and procedures used. One activity required some knowledge of probability and one activity required some knowledge of combinatorics. These activities are only available for field testing and cannot be released. The grade 11 test was 10 multiple-choice items taken from a practice version of a quantitative reasoning test (QRT) developed by a prestigious university for its first-year students. Students at this university are required to pass the QRT or a specified course as a graduation requirement. Permission was not received from the university to disclose its name or the test items. 

                                       Studies 

 

Grade 9
IMP Yr 1
Algebra 1

Grade 10
IMP Yr 2
Geometry

Grade 11
IMP Yr 3
Algebra 2

School 1
(western U.S.)

Statistics Test
(4 items: five points)

 

 

School 2
(north-central U.S.)

 

Problem- Solving Test (2 activities)

 

School 3
(eastern U. S.)

 

 

Quantitative Reasoning Test (10 items)

  Figure 1. General design of three studies.

                                                                                                                                        Sample  

            Three different schools were selected for each of the studies. Each school came from a different region of the country, was considered to be effectively implementing the IMP curriculum, and had an IMP coordinator who would attend to all of the controls on the study. All testing was done within the last six weeks of the 1995-96 school year. Some classes tested in these studies included students not in the target grade. Those students were not included in the analyses.

              School 1 is located in a city in the western United States, and serves a diverse group of students. Three Year 1 IMP classes were taught in School 1, all by the same teacher. All the students in these three classes were tested along with four algebra 1 classes taught by two other teachers. The grade 8 Comprehensive Test of Basic Skills (CTBS) scores were used as the measure of prior knowledge of mathematics. A total of 60 IMP Year 1 students and 55 algebra 1 students in grade 9 were included in the analysis.

              School 2 is a select public high school located in a large city in the north-central United States. The school serves an ethnically diverse student body who are required to meet minimal requirements to enroll in the school, including scoring at least in the 6th stanine on the grade 8 Iowa Test of Basic Skills (ITBS). Nearly all of the students will continue their education at universities or colleges. Students tested were in four IMP Year 2 classes, two taught by one teacher and two by another teacher, and in six geometry classes, three taught by one teacher and three by another teacher. The grade 8 ITBS scores were used as the measure of prior knowledge of mathematics. A total of 87 IMP Year 2 students and 97 geometry students in grade 10 were included in the analysis.

              School 3 is a college-preparatory magnet public high school located in a large city in the eastern United States. In 1994, the school enrolled over 2,000 students—35 percent black; 17 percent Asian, 4 percent Latino, and 44 percent white. A large number of students who spoke English as a second language were enrolled. Students had to meet minimum requirements to attend School 3. About one-third of the applicants were accepted. Nearly all, 98 percent, of the students continue their education at universities or colleges. Students who attended School 3 came from a variety of middle schools including public and private. Students tested were in four Year 3 IMP classes—two taught by two different teachers, one taught by these two teachers as a team, and one taught by one of these teachers and a third teacher—and three algebra 2 classes, all taught by the same teacher. Students had taken a variety of standardized tests in grade 8. The majority of the students had grade 8 scores on the California Achievement Test (CAT). Two analyses were performed. One analysis ignored the publisher of the grade 8 test taken, assuming that the national norm equivalent is generally comparable across tests. A second analysis used only students with grade 8 CAT scores. Altogether, a total of 93 IMP Year 3 students and 40 algebra 2 students in grade 11 were included in the analysis. 

            An ethnically diverse group of students participated in the study (Table 1, p. 16). A total of 240 students who were taking IMP were included in the analysis—47 percent white, 24 percent black, 11 percent Hispanic, 9 percent Asian/Pacific Islander, 1 percent American Indian, and 8 percent other or unknown. A total of 192 students who were taking the traditional curriculum were included in the analysis—33 percent black, 24 percent white, 18 percent Asian/Pacific Islander, 11 percent Hispanic, 1 percent American Indian, and 13 percent other or unknown. 

            A higher proportion of female students than male students participated in this study in both the IMP group and the traditional group—55 percent female for IMP and 61 percent female for the traditional curriculum (Table 2, p. 17). Because IMP and traditional groups varied some on their composition by ethnicity and sex at each of the three schools, analyses were performed to determined if differences associated with group characteristics were significant. 

            To test for statistical significance, the student was used as the unit of analysis rather than the teacher or the class. Except for grade 9, the tests measured knowledge and skills that were not specific to the current course and teacher but reflected the results of two or more years in the given program. In all three studies, analyses were performed to determine if findings were upheld for classes as well as students.

                                                                                                                                      Findings  

            At each of the three schools, IMP students obtained statistically significant higher test scores compared to students in the traditional mathematics course (Table 3, p. 18). These results were true even when accounting for any differences between groups in prior achievement as measured by grade 8 standardized norm-referenced test scores. An analysis of covariance was used to account for prior achievement for each of the studies. Matched-group analyses also were performed for each school to control for any differences due to prior achievement. 

School 1            

            Grade 9 Year 1 IMP students at school 1 outperformed those taking algebra 1 on the statistics items from the SIMS (Table 3, p. 18). On the average, IMP students attained three of the possible 5 points, compared to the algebra 1 students who attained one of the possible 5 points. The internal-consistency reliability estimate was .73, high for a test consisting of only five items. Eleven of the 60 IMP students (18 percent) attained a perfect score, compared to none of the algebra 1 students. A total of 24 students did not receive any points, 4 IMP students (7 percent) and 20 algebra 1 students (36 percent). No significant differences were found in the performance by sex or ethnicity. 

            SIMS items were originally administered in a multiple-choice format to precalculus students, as compared to the open-response format used in the current study. The open-response format is more difficult. The same percentage of correct answers on an open-response item, compared to the corresponding multiple-choice item, represents a greater degree of understanding. In analyzing results of the current study, performance of IMP students was compared to that of the SIMS precalculus students as well as to that of current grade 9 algebra 1 students. 

            On item 1 (requiring students to determine the approximate average weekly rainfall from a bar graph), IMP students did better than algebra 1 students (78 percent correct compared to 55 percent). The IMP students' performance matched that of the SIMS precalculus students, of whom 78 percent correctly answered this item, even though the IMP students had to produce the correct answer as an open response rather than selecting the correct response among five options. 

            Item 2 required the computation of a weighed average. On this item, 48 percent of IMP students found the correct answer, compared to 11 percent of the algebra 1 students. Only 22 percent of the SIMS precalculus students selected the right answer, a percentage that is only slightly better than randomly selecting from five choices. 

            Item 3 required students to analyze a linear transformation on the mean and standard deviation of a distribution. Concerning the effect on the mean, 87 percent of the IMP students got the right answer, compared with 33 percent of the algebra 1 students. Both groups had more difficulty with the item determining the effect on the standard deviation. The IMP students again did much better than the algebra 1 students (57 percent correct compared to 2 percent). In the 1982-83 SIMS, the two questions were combined as a single multiple-choice item, and only 24 percent of the precalculus students selected the correct choice. 

            Item 4 required the application of properties of the normal curve (identifying the proportion of the area under the curve related to +/- one standard deviation). On this item, 37 percent of the IMP students were able to produce the correct answer, while none of the algebra 1 group were able to do so. Only 22 percent of the precalculus students identified the correct answer from five choices. 

            The IMP curriculum exposes students to the properties of the normal curve of mean and standard deviation among other mathematical concepts and ideas. Not all IMP students are expected to master these ideas by the end of grade 9. More instruction is provided on these ideas later in the curriculum. These ideas are not included in most traditional algebra curricula so algebra 1 students would not be expected to know these ideas. The findings from this study identify at least some differences between what students learn from taking IMP compared to the traditional algebra course. The findings support that students who take IMP Year 1 gain knowledge about statistics and the normal distribution. The IMP students scored as well as, or better than, the SIMS precalculus students, who had taken the items 14 years before. Again, this finding denotes difference in curriculum coverage. 

            Although students were in three different IMP classes, they all had the same teacher. This teacher was considered to be using IMP as intended. The algebra 1 students were taught by two different teachers, two classes each. We have no special information about these teachers. There was a variation in class means both within IMP classes (2.25, 2.89, and 4.13) and within algebra 1 classes (.47, .90, 1.29, 1.31). The variation in class means indicates some differences in achievement by class. Of the possible 12 pairwise comparisons of an IMP class mean with an algebra 1 class mean, for 10 of the comparisons the IMP class mean was significantly higher than the algebra class mean. One IMP class (mean of 2.25) had a significantly higher mean than two of the algebra 1 classes (means of .47 and 1.29), but not significantly higher mean than the other two algebra 1 classes (means .90 and 1.31). An analysis of covariance, using the grade 8 CTBS score as the covariate, was used to make the pairwise comparison of class means. 

  School 2  

            Grade 10 IMP Year 2 students performed significantly better than geometry students on an instrument measuring students' mathematical reasoning and problem solving (Table 3, p. 18). The instrument was composed of two performance assessment activities, each designed to require students about 20 minutes to complete. Students' work on each activity was scored using a six-level rubric ("advanced response" (5), "proficient" (4), "nearly proficient" (3), "minimal" (2), "attempted" (1), and "not scorable" (0)). A "proficient" or "advanced" score on one activity, Connecting Nodes, indicates that the student demonstrated skills in solving problems, reasoning, developing and testing conjectures, writing clear and correct explanations, computing, and forming a generalization. A "proficient" or "advanced" score on the second activity, New Cubes, indicates that the student performed a number of skills. These included comparing probabilities between two or more sums when rolling two dice; computing all possibilities for an invented set of dice meeting specified conditions; determining the expected value for a large number of trials; and describing their reasoning in writing. Each student response was scored by at least two raters. A third rater scored the response if there was an unacceptable range in scores by the first two raters. Exact agreement by raters on scoring was 78 percent for Connecting Nodes and 80 percent for New Cubes. Acceptable inter-rater agreement—exact agreement or only varying by one level between "attempted" and "minimal" or "proficient" and "advanced"—on both activities was 87 percent. 

            IMP Year 2 students, compared to geometry students, had a significantly higher (p < .01) mean score on both activities—2.53 compared to 2.04 on Connecting Nodes and 3.91 compared to 2.52 on New Cubes. On Connecting Nodes, 21 percent of the IMP Year 2 students, compared to 5 percent of the geometry students, performed at a "proficient" or "advanced" level. On New Cubes, 64 percent of the IMP Year 2 students, compared to 16 percent of the geometry students, performed at a "proficient" or "advanced" level. These results were true even though the geometry students had a higher mean score on the grade 8 ITBS. A total of 15 IMP students (17 percent) achieved a "proficient" score or higher on both of the activities. Fifty-nine of the IMP students (69 percent) achieved a "proficient" score or higher on one of the two activities. This is compared to one geometry student (1 percent) who achieved at least a "proficient" score on both activities and 46 students (47 percent) who achieved at least a "proficient" score on one activity. The differences in the respective percentages are statistically significant (p < .01). Nearly all of the students participating in the study, 87 percent, were identified as black, Asian, or Hispanic. There were no significant differences by sex or ethnicity. 

            Four classes of IMP Year 2 students—two classes for each of two teachers—and six classes of geometry students—three classes for each of two teachers—were tested. Two of the geometry classes were honors classes. Mean scores among the four IMP classes and between the two teachers varied little (6.0 and 6.79 for one teacher and 6.16 and 6.75 for the other teacher). The mean scores among the six geometry classes differed significantly between honors and regular classes. The mean scores for the geometry honors classes, both taught by the same teacher, were 5.10 and 6.00. The mean scores for the regular geometry classes were 4.45 (taught by the same teacher as the honors courses) and 3.50, 4.07 and 4.63 (taught by the other geometry teacher). 

            Of the 24 possible pairwise comparisons of an IMP class with a geometry class, IMP classes had significantly higher means for 20 of the comparisons. One of the four IMP Year 2 classes (mean of 6.79) had a significantly higher mean than all six of the geometry classes. Two of the IMP Year 2 classes (means of 6.75 and 6.16) had significantly higher means than five of the six geometry classes. Only the honors geometry class, with a mean of 6.00, was not significantly different from the means of these two IMP classes. Only one IMP Year 2 class (mean of 6.00) was not significantly different from either of the two honors geometry classes (means of 6.00 and 5.10), but this class was significantly higher than all four of the regular geometry classes. An analysis of covariance, using grade 8 ITBS score as the covariate, was used to make the pairwise comparisons. 

            We did two analyses to compare grade 10 IMP Year 2 students with grade 10 geometry honors students. One analysis involved only those students for whom we had grade 8 ITBS‑math scores (71 out of 87 IMP Year 2 students, 20 out of 23 geometry honors students). The IMP students had a significantly lower mean score on grade 8 ITBS‑math (77.34 compared to 92.50; p < .01), but performed higher on the problem-solving test (mean of 6.35 compared to mean of 5.65). Using analysis of covariance to control for grade 8 scores, this difference on the problem-solving test is statistically significant (p < .01). In the second analysis, we compared scores on the problem-solving test, using all grade 10 IMP Year 2 students and all grade 10 geometry honors students (including those without grade 8 ITBS scores). Using analysis of variance, the IMP students performed significantly higher than the geometry honors students (p < .05). These two analyses suggest that the experiences students had in IMP raised their level of performance on these two activities above the highest achieving students from the traditional curriculum at this school. 

            Performance assessment is designed to measure students' performance on complex open-response situations with multiple parts. An important part of the IMP curriculum is for students to work complex problems, some over an extended period of time. This attention to complex problems solved over extended time periods differs greatly from the approach of most traditional curricula, which are designed for students to become accomplished in doing more limited tasks. The two performance activities used in this study required some knowledge of probability and computing combinations, two topics included in the IMP curriculum. Again, as in the first study, grade 10 IMP students out-performed students taking the traditional curriculum on activities aligned with the IMP curriculum. A higher percentage of IMP students were able to apply reasoning to analyze a situation using ideas from probability, and to make a generalization, compared to geometry students. The fact that the honors geometry classes outperformed the regular geometry classes suggests that general mathematics achievement is related to doing well on these performance assessment activities. All IMP classes did as well as, or better than, both of the two honors geometry classes. This was the case even though IMP students' prior achievement in grade 8 was even lower than geometry students in general. This suggests that the experiences students had in IMP had raised them to a level of performance on these two activities above the highest achieving students from the traditional curriculum at this school. 

School 3   

            Grade 11 IMP Year 3 students performed significantly better than grade 11 algebra 2 students on 10 items from a practice version of a quantitative reasoning test developed for a prestigious university to administer to entering first year students (Table 3, p. 18). In order to attain second year status, students are required to obtain a specific score on this test or to take a designated course. All of the items are multiple choice. University students are given 90 minutes to work 25 items, or 3.6 minutes per item. Ten items aligned with the IMP curriculum were selected from the practice test to be administered in one class period of nearly 50 minutes. Knowledge measured by these items include data and graph interpretation, probability of independent events, and statistical concepts of mean and standard deviation for a distribution. The measure of internal consistency for the test of 10 items was .65. 

            Students in all four IMP Year 3 classes and students in three algebra 2 classes were tested. IMP Year 3 students had a significantly higher mean score than the algebra 2 students, 5.04 compared to 2.40 (Table 3, p. 18). The difference in means was found to be statistically significant using an analysis of covariance to account for the difference in mean score on the grade 8 achievement measure. For the total group, IMP and algebra students combined, male students had a significantly higher (p < .05) mean than females. But no significant interactions by sex or ethnicity were found between the two curricula. Of the 12 possible pairwise comparisons of IMP class mean with algebra 2 class mean, on 11 comparisons IMP classes had significantly higher means. Only the IMP class with the lowest mean (4.44) was not significantly higher than the algebra 2 class with the highest mean (3.13). Means were compared using an analysis of variance. 

            The percentage of IMP students who correctly answered any one item ranged from 18 percent to 75 percent. The percentage of algebra 2 students who correctly answered any one item ranged from 3 percent to 53 percent. Two IMP students correctly answered all 10 times. The highest score obtained by an algebra 2 student (one student) was 8. Three students obtained a score of 0, one IMP student and two algebra 2 students. IMP students out-performed algebra 2 students on all ten items. The highest percentage of both groups answered correctly a probability question requiring students to determine the number of balls in a bag given the probability for drawing each of three of four colors of balls and the number of one color of balls. Of the IMP students, 75 percent correctly answered this question, compared to 53 percent of the algebra 2 students. On a second question based on the same situation, where students were to identify the number of balls of one color to be replaced by a second so that the probability to pick either color was the same, 67 percent of the IMP students answered correctly, compared to 38 percent of the algebra 2 students. 

            On another item, given graphs of growths in revenue for four divisions of a company, students were asked to select a true written description about the comparison of the divisions' rates of growth. Seventy-one percent of the IMP students answered this correctly, compared to 45 percent of the algebra 2 students. The most difficult item for all of the students was a statistical question. Students were asked to identify the number of people who were within a specific range of reading speeds on a reading test. The test item gave the total number of students who took the reading test, the fact that reading speeds were normally distributed, the mean, and the standard deviation. On this item, only 18 percent of the IMP students and 3 percent of the algebra 2 students gave a correct answer. Students in both groups performed better on the next item. Students were asked to use the mean and standard deviation of the distribution to approximate the reading speed for a student at the 80th percentile. Over half of the IMP students answered this correctly (54 percent), compared to 28 percent of the algebra 2 students. 

            None of the 10 items asked a question in the same form as it would appear in the IMP curriculum. Half of the 10 items required some understanding of probability and statistics, two areas given some emphasis in IMP. The other five items required knowledge of rate of change and the interpretation of slope of a graph, topics given as much attention in IMP as in the traditional curriculum. The inclusion of these items suggests that the university where the test was developed felt that it was important for entering college students to know what is measured by these items. The findings from this study indicated that IMP students performed significantly better on these items than students in the traditional algebra 2 course. Part of the differences in performance could be explained by the increased emphasis by IMP on probability and statistics. The other part related more to differences in general quantitative reasoning skills demonstrated by students in the two different curricula. 

Matched-Group Analyses  

            For each of the three studies, the IMP students and students in the traditional college preparatory mathematics courses had a different mean score on the grade 8 mathematics standardized norm-referenced test. For two schools, the mean for IMP students was significantly higher at the .05 level. For the third school, the mean for IMP students was lower than the mean for the traditional students, but not significantly so (Table 3, p. 18). An analysis of covariance was performed, using the grade 8 score as the covariate, to adjust for differences in prior achievement by group. Other factors, such as ethnicity and sex, were not controlled. We performed a matched-group analysis to consider the results in a second way and to control for prior achievement, ethnicity (when possible), and sex (Tables 4 and 5, pp. 19-20). For schools 1 and 2, the students within each school had scores from the same grade 8 test. School 3 students were from a large number of middle schools, both public and private. Thus, not all of the students had taken the same test in grade 8. In the matched-group analyses, only students with scores on the same grade 8 test (California Achievement Test) were included in the analysis. 

            For the matched-group analysis, we tried to generate two comparison groups for each school so that the IMP group and the traditional group had comparable statistical qualities on the independent variables. Student scores were not included in the analysis if a close match could not be found. As a result, the numbers of students included in the matched-group analyses are lower than the numbers included in the analyses of covariance. For all three schools, the matching process successfully produced very similar groups on grade 8 mathematics score and distribution of sex, but for school 1 the groups varied some in ethnic composition (Tables 4, 5 and 6, pp. 19-21). 

            As expected, the matched-group analysis produced the same results as the analysis of covariance. IMP students in each of the three schools, compared to those in the same grade in the traditional mathematics course, performed significantly higher on the given measure of mathematical knowledge (Table 6, p. 21; Figure 2, p. 23). Grade 9 IMP students had a significantly higher score on the SIMS statistics items than the algebra students. Grade 10 IMP students had a significantly higher score on the performance assessment instrument than the geometry students. Grade 11 IMP students had a significantly higher score on the 10-item quantitative reasoning test designed for entering university students. All results were statistically significant (p < .01). 

                                                                                                                                           Conclusions  

            This series of three studies provides evidence that IMP students in mathematics are learning beyond what students are learning in the traditional algebra 1, geometry, and algebra 2 course sequence. IMP students performed better than students from the same grades in the traditional mathematics courses on activities requiring knowledge of probability and statistics and skills in reasoning, problem solving, and forming generalizations. The measures used in these studies were developed independently of the IMP curriculum. One instrument was items from an international study, one instrument was activities developed for a state assessment, and one instrument was items developed by university faculty. Grade 8 mathematics test scores were obtained to control for differences in prior achievement upon entering high school. In all three studies, IMP achieved a mean score that was significantly higher using common statistical tests. 

            The number of students in this series of three studies is small. Schools were selected to participate in these studies because they were implementing IMP as designed. Two of the schools had enrollment criteria. The findings support that students taking IMP are learning mathematics beyond what students learn in the traditional curriculum. There is mounting evidence that IMP students do as well as students enrolled in the traditional college-preparatory mathematics course sequence on standardized norm-referenced tests such as the SAT. Combined, these studies indicate that IMP not only prepares students to perform well on traditional measures, but that the curriculum provides students knowledge and skills that are becoming increasingly important.

                                                                                                                                             References  

Crosswhite, F. J., Dossey, J. A., Swafford, J. O., McKnight, C. C., Cooney, T. J., Downs, F. L., Grouws, D. A., & Weinzweig, A. I. (1986). Second International Mathematics Study, detailed report for the United States. Champaign, IL: Stipes. 

Dowling, M. & Webb, N. L. (1997a). Comparison on statistics items of grade 9 Interactive Mathematics Program (IMP) students with algebra students at one high school. Project Report 97-2 from the Interactive Mathematics Program Evaluation Project. Madison: University of Wisconsin-Madison, Wisconsin Center for Education Research. 

Dowling, M. & Webb, N. L. (1997b). Comparison on problem solving and reasoning of grade 10 Interactive Mathematics Program (IMP) students with geometry students at one high school. Project Report 97-3 from the Interactive Mathematics Program Evaluation Project. Madison: University of Wisconsin-Madison, Wisconsin Center for Education Research. 

Dowling, M. & Webb, N. L. (1997c). Comparison on quantitative reasoning test of grade 11 Interactive Mathematics Program (IMP) students with algebra 2 students at one high school. Project Report 97-4 from the Interactive Mathematics Program Evaluation Project. Madison: University of Wisconsin-Madison, Wisconsin Center for Education Research. 

Illinois State Board of Education. (1996). Preliminary draft: Illinois academic standards. English language arts, mathematics Volume One: State goals 1-10. Springfield, IL: Illinois Academic Standards Project. 

Interactive Mathematics Program. (1995). Standardized tests: Highlights from current studies of IMP student performance. IMP Evaluation Update [Newsletter], 1, Spring, p. 2. 

National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author. 

Rosenstein, J. G., Caldwell, J. H., & Crown, W. D. (1996). New Jersey Mathematics Curriculum Framework. A collaborative effort of the New Jersey Mathematics Coalition and the New Jersey Department of Education. New Brunswick, NJ: The New Jersey Mathematics Coalition, Rutgers, The State University of New Jersey. 

Schoen, H. L. (1993). Report to the National Science Foundation on the impact of the Interactive Mathematics Project (IMP). In N. L. Webb, H. Schoen, & S. D. Whitehurst (Eds.), Dissemination of nine precollege mathematics instructional materials projects funded by the National Science Foundation, 1981-91. Madison, WI: University of Wisconsin-Madison, Wisconsin Center for Education Research. 

South Carolina State Department of Education. (1993). South Carolina Mathematics Framework. Columbia, SC: Author. 

Webb, N. L. & Dowling, M. (1995a). Impact of the Interactive Mathematics Program on the retention of underrepresented students: Class of 1993 transcript report for school 1: Brooks High School. Project Report 95-3 from the Interactive Mathematics Program Evaluation Project. Madison: University of Wisconsin-Madison, Wisconsin Center for Education Research. 

Webb, N. L. & Dowling, M. (1995b). Impact of the Interactive Mathematics Program on the retention of underrepresented students: Class of 1993 transcript report for school 2: Hill High School. Project Report 95-4 from the Interactive Mathematics Program Evaluation Project. Madison: University of Wisconsin-Madison, Wisconsin Center for Education Research. 

Webb, N. L. & Dowling, M. (1995c). Impact of the Interactive Mathematics Program on the retention of underrepresented students: Class of 1993 transcript report for school 3: Valley High School. Project Report 95-5 from the Interactive Mathematics Program Evaluation Project. Madison: University of Wisconsin-Madison, Wisconsin Center for Education Research. 

Webb, N. L. & Dowling, M. (1996). Impact of the Interactive Mathematics Program on the retention of underrepresented students: Cross-school analysis of transcripts for the class of 1993 for three high schools. Project Report 96-2. Madison: University of Wisconsin-Madison, Wisconsin Center for Education Research.

TABLES
                                                                

Table 1  

Students by Ethnicity by School and Curriculum 

         Ethnicity

School 1
Grade 9

School 2
Grade 10

School 3
Grade 11

Total

 

IMP Yr1

Alg 1 

IMP Yr2

Geom

IMP Yr3

Alg 2   

IMP

Trad

N N N N N N

 

 

 

 

 

 

 

N

%

N

%

American
Indian

0

0

2

2

0

0

2

1

2

1

Asian or Pacific Islander

10

16

6

14

6

5

22

9

35

18

Black-not of Hispanic Origin

1

0

39

52

19

12

59

24

64

33

Hispanic

5

8

18

12

3

1

26

11

21

11

White-not of Hispanic Origin

38

17

16

8

59

21

113

47

46

24

Other/Missing

6

14

6

9

6

1

18

8

24

13

Total

60

55

87

97

93

40

240

100

192

100

Table 2  

Students by Sex by School and Curriculum

               Sex

School 1
Grade 9

School 2
Grade 10

School 3
Grade 11

Total

 

IMP Yr1   

Alg 1   

IMP Yr2

Geom

IMP Yr3

Alg 2

IMP

Trad

N N N N N N

 

 

 

 

 

 

 

N

%

N

%

Female

34

28

53

63

46

26

133

55

117

61

Male

26

23

34

34

47

14

107

45

 71

37

Not Reported

 

4

 

 

 

 

 

 

   4

2

Total

60

55

87

97

93

40

240

100

192

100


 Table 3  

Pre- and Post-Test Scores for Students by School and Curriculum
Using an Analysis of Covariance to Adjust for Prior Differences

            

 

  Achievement
  Scores

School 1
Grade 9
(5 Points Possible)

School 2
Grade 10
(10 Points Possible)

School 3
Grade 11
(10 Points Possible)

 

IMP Yr1 

N=60

Alg 1 

N=55

IMP Yr2 

N=87

Geom 

N=97

IMP Yr3 

N=93

Alg 2 

N=40

Test 

 

 

 

 

 

 

   Mean

3.07a

1.00

6.44a

4.56

5.04a

2.40

   S.D.

1.41

.92

1.66

1.44

2.12

1.52

Grade 8

 

 

 

 

 

 

   Meanc

70.45b

62.40

77.3

79.53

94.53b

92.71

   S.D.

23.34

22.85

15.13

16.24

4.18

 4.67

a           Significantly higher mean score from traditional curriculum group using an analysis of covariance to account for any differences on grade 8 achievement, p < .01.

b           Significantly higher mean score from other group within school, p < .05. 

c           Mean national percentiles scores.


 
Table 4

   Students by Ethnicity by School and Curriculum
   Using a Matched-Group Analysis to Form Comparison Groups 

             Ethnicity

School 1
Grade 9

School 2
Grade 10

School 3
Grade 11

Total

 

IMP Yr1

Alg 1

IMP Yr2

Geom

   

IMP Yr3

Alg 2

IMP

Trad

N N N N N N

 

 

 

 

 

 

 

N

%

N

%

American
Indian

0

0

2

1

0

0

2

1

1

1

Asian or Pacific Islander

 7

15

5

8

1

3

13

9

26

17

Black-not of Hispanic Origin

1

0

32

39

 8

10

41

27

49

32

Hispanic

5

6

14

 9

1

1

20

13

16

11

White-not of Hispanic Origin

31

16

11

7

17

16

59

39

39

26

Other/Missing

6

13

6

6

4

1

16

11

20

13

Total

50

50

70

70

31

31

151

100

151

100


Table 5  

Students by Sex by School and Curriculum
Using a Matched-Group Analysis to Form Comparison Groups 

             Sex

School 1
Grade 9

School 2
Grade 10

School 3
Grade 11

Total

 

IMP Yr1

Alg 1

   

IMP Yr2

Geom

   

IMP Yr3

Alg 2

IMP

Trad

N N N N N N

 

 

 

 

 

 

 

N

%

N

%

Female

27

28

44

47

19

21

90

60

96

64

Male

23

18

26

23

12

10

61

40

 51

34

Not Reported

 

4

 

 

 

 

 

 

   4

2

Total

50

50

70

70

31

31

151

100

151

100


Table 6  

Pre- and Post-Test Scores for Students by School and Curriculum
Using a Matched-Group Analysis to Form Comparison Groups

            

 

  Achievement
  Scores

School 1
Grade 9
(5 Points Possible)

School 2
Grade 10
(10 Points Possible)

School 3
Grade 11
(10 Points Possible)

 

IMP Yr1

  N=50

Alg 1

  N=50

IMP Yr2

  N=70

Geom

  N=70

IMP Yr3

  N=31

Alg 2

  N=31

Test 

 

 

 

 

 

 

   Mean

3.06a

1.02

6.39b

4.50

5.42c

2.39

   S.D.

1.49

.94

1.60

1.25

2.20

1.61

Grade 8

 

 

 

 

 

 

   Mean

66.84

66.24

77.50

78.13

93.71

93.68

   S.D.

20.25

19.76

15.18

13.50

 3.75

 3.77

a           The z-statistic approximation for the large-sample Wilcoxon test was found statistically significant, Z = 5.38, p < .01. 

b           The z-statistic approximation for the large-sample Wilcoxon test was found statistically significant, Z = 5.07, p < .01. 

c           The z-statistic approximation for the large-sample Wilcoxon test was found statistically significant, Z = 4.58, p < .01.

        *  Mean scores for both groups in school 1 were doubled to have these scores on the same scale as the measures for the other two schools.

Figure 2. Test means across schools comparing IMP to traditional math students.

Appendix

Grade 9 Instrument
Modified SIMS Statistics I

 

                                                   SAMPLE COVER SHEET ONLY

Student's Name:                                                            Class Period:                 

Grade:                                                                              Teacher:                          

Mathematics
Course Title:                                            

Gender: (circle one)      F  

Optional--
Ethnicity: (circle one)

(1)  American Indian or
       Alaskan Native

(4) Hispanic
(2) Asian or Pacific
       Islander
(5) White
      (not of Hispanic origin)
(3) Black
      (not of Hispanic origin)
(6) Other

This test consists of four questions.  Write your name at the top of each page.  For each question, find the answer and then write your answer on the line provided.  Each question will be scored as right or wrong.  The total score for the four problems will be the total number of questions you answered correctly.  You are free to show your work and write on the pages even though only your answer will be scored.  You can use a calculator. 

You may answer the questions in any order.  If you have difficulty answering one question, go to the next question.  Go back to any unanswered question and do your best to determine an answer.  Please check all of your answers before returning the test booklet to your teacher. 

Please complete the information above before beginning.

Name                                                            

 

1.

                         

 In the graph, rainfall (in centimeters) is plotted for 13 weeks.  What was the approximate average weekly rainfall during the period? 

 

            Answer 1:                                

 

2.         The same test was given in two classes.  The first class, with 20 pupils, obtained an average score of 12.3 points.  The second class, which had 30 pupils, obtained an average score of 14.8 points.  What was the average score for the whole group of 50 pupils?

 

              Answer 2:                                

3.         The mean of a population is 5 and its standard deviation is 1.  If 10 is added to each element of the population, what will be the new mean and standard deviation?  

 

            Answer 3:

                        New mean:                                        

 

                        New standard deviation:                        

 

4.         A test is taken by all first year university students in a country.  The mean is 50 and the standard deviation 20.  Assuming the scores are normally distributed, approximately what percentage of students score more than 30?

 

 

            Answer 4:                                


                                                                     Scoring Key

                                                   Five Modified SIMS Statistics Items 

1.         Average weekly rainfall during the period:                     1.9 to 2.3 cm 

2.         Average score for the whole group of 50 ss:                  13.8 points 

3a.       New Mean:                                                                          15 

3b.       New Standard Deviation:                                                   

4.         Percentage of students scoring more than 30:                 83 to 85%