Highlights of evaluations with minority students:

Statistically significant (p<.05) higher aspirations for careers in IT in the intervention group (many tools), in comparison to our baseline data.

Statistically significant higher post-test scores (p < .05) in a class using the VBL, in comparision to a control group with ordinary instruction.

Statistically significant improvement in grades (p<.001) for a class using Graffiti Grapher, in comparison to the math grades of the same students in the previous year.

Details:

I. Attitudes towards careers in information technology: Our past NSF/ITWF grant evaluated the effect of CSDTs on student attitudes towards IT careers. Baseline data was generated by 175 randomly selected eighth grade students from low-income families completing two surveys: (1) The Workforce Survey questioned them about plans for taking mathematics courses in high school, ideas about future careers, their gender, age, ethnicity and parents' education. The Bath County Computer Attitudes Scale (Bear, Richards & Lancaster, 1987) surveyed computer use and attitudes, including work in mathematics on computers. Middle school minority students (24 total) from a low-income population participated in three workshops and were exposed to design tools for 20 hours over a two week period (during which they were paid an hourly minimum wage). Afterwards they completed the same surveys. We found statistically significant (p<.05) higher aspirations for careers in IT in comparison to our baseline data. A similar intervention with 90% white students did not show a statistically significant difference from baseline data.

II. Quasi-experimental evaluation in mathematics class: In a pilot study, subsequently accepted as a master’s thesis, a teacher at Torch middle school in California, compared the performance of two pre-algebra classes, one with the VBL, and one without, on a standards-based evaluation (this evaluation is available online under the “teaching materials” section of the VBL webpage). She found statistically significant higher scores (p < .05) in the class using the VBL. A second quasi-experimental test was conducted at another school in California using both the VBL and the new Graffiti Grapher design tool. Again there was a control group, and a class that used the tools, with each taking a pre-test/post-test evaluation. The class using the design tools had higher pre-test/post-test gains than the control group, although the difference was not statistically significant. However students using the design tools did show statistically significant increases in their course grades as compared to the previous year (p < .001). In a third evaluation (carried out as a recent doctoral dissertation), the Attitudes toward Mathematics Inventory (ATMI: Tapia and Marsh 2004) was administered to students in classes with and without CSDTs at several schools; the students with CSDTs were found to have statistically significant increases in their ATMI scores.

III. Pre-test/post-test evaluation: The goal of the evaluation was simply to determine whether mathematics performance on topics in the mathematics curriculum can be improved by the use of CSDTs. Tools were also used in art classes or other non-mathematics courses, since any improvement in mathematics performance outside of the math class is a positive gain. We have conducted one pilot test using the Cornrow Curves design tool in an art class at Troy high school in upstate NY, where the majority of the population is from low-income families, with 30% black and Latino, and found improvement (mean of 73.75% for the pre-test, and 84.58% for the post-test); not statistically significant, most likely due to flaws in the pre/post-test questions and a long delay between use and testing. Two teachers who conducted similar pre-test/post-test comparisons for students at Harmony Hill middle school in upstate NY (again a district with over 50% low-income families) without these problems did show statistically significant improvements; one class at  p< .01 and the other at p< .001.

VI. Grade improvement: High school teacher Linda Rodrigues, also in California, compared grade levels for two classes, one using the VBL and Graffiti Grapher, and one from the previous year without any design tools. She found statistically significant improvement in grades (p<.001) for the class using the design tools.

While not conclusive, these are solid preliminary indications that CSDTs may be able to raise both achievement and interest in mathematics for low-income minority students.