Section 3: Water Quality (WQ) Curriculum Quantitative Evaluation
For the WQ curriculum implementation, a special effort was made to engage alternative schools. From Phase I evaluation results, it was learned that alternative schools with populations of “at risk” students yield some of the most significant gains in terms of student outcomes, particularly in team-related activities where students are challenged to learn new cooperative working skills, as well as gaining experience with both written and oral presentation. For at risk students who may be on the margins of educational achievement, the development of self-efficacy and self-confidence with respect to working with others and in demonstrating their written and oral presentation abilities may be a significant step in promoting both their personal and career development.
The evaluation design for WQ organized participating schools into two program groups. The individual schools are shown in Table 5 with the number of completed pretest-posttest evaluation protocols for each school. (Click on the school name for a description of each pilot school.)
Table 5. Schools participating in the WQ evaluation.
| Mainstream Schools |
Alternative Schools |
| Westview HS (N=263) |
|
| Skyview HS (N=101) |
|
| Tigard HS (N=58) |
|
| Young Parent Program (N=18) |
|
| Woodburn Success Program (N=49) |
The total number of completed pre/post evaluation protocols for the mainstream schools was N=462, and for the alternative schools was N=150. (The actual N for a given analysis will vary due to missing data. These N’s indicated reflect an upper bound on the number of data points available. In actuality the N’s for a given analysis may be reduced as much as 5%.) Alternative schools tended to have considerably lower enrollment than do mainstream schools, posing significant challenges to obtaining comparable sample sizes with mainstream schools.
Scale Means by Program and All Schools Combined. Table 6 shows the scale means by mainstream schools, alternative schools and for all schools combined. For each of the program types (i.e., mainstream vs. alternative schools) pretest and posttest means are shown as well as the number of students with complete pre/post protocols (N). Differences scores are shown along with a test of statistical significance of the value of the difference. At the far right of the table a between-program significance test is shown. This is test between the two program types done on their respective pretest, posttest and difference scores.
The WQ implementation evaluation had a much larger sample size than that for IAQ (see previous section), resulting in a more powerful statistical analysis that demonstrated in greater detail the impact of the curriculum on student outcomes. Six of the evaluation scales for the all schools combined group exhibited highly significant pre/post change (p<.001 or greater). This same pattern of pre/post change was reflected as well in the mainstream schools, for which a relatively large sample size was also available. On a selected basis, alternative schools also exhibited highly significant changes on four of the evaluation scales (p<.01 or greater), and particularly for the PersInvolv scale (p<.0001). Given the relatively small sample size for the alternative schools group, these patterns of change are important and meaningful.
Overall, students demonstrated an increased appreciation for the value of science in daily life (ValueSci scale; p<.001 or greater) as well as a more positive regard for the importance of self-protection as well as increased sensitivity to environmental risks (SelfPro2 and EnviroRisk scales). For all the school groupings (i.e., all schools, mainstream schools, alternative schools) students were more inclined (from pretest to posttest) to view different kinds of environmental science-type information as important for solving an environmental health problem (InfoSeek scale; p<.01 or greater). We infer that this is reflective of their greater awareness of how to use environmental science information in the context of identifying and analyzing a problem that may have an environmental cause.
Students in all three groups were much more inclined toward taking a personally involving role in working with others to solve an environmental health problem (PersInvolv scale; p<.0001). This was the strongest effect statistically across all of the scales for the alternative schools group. Again, we see the effect of the activity-oriented curriculum on improving the base of student skills with regard to group work, written reporting, and oral presentation of environmental science principles.
Intuitive Toxicology (IntuiTox). A significant pre/post change was observed for the following two IntuiTox items:
“If a chemical is released into the environment, then everyone in that environment is exposed to the chemical.”
“If a person is exposed to a chemical that can cause cancer in humans, then that person will probably get cancer someday.”
Results for these two items are shown in Table 7. Students responded to each item on four-point categorical scale: “strongly disagree,” “disagree,” “agree,” and “strongly agree.”
These items are important for a number of reasons. First, they have been studied extensively in the context of risk perception where both toxicological experts and non-scientists (e.g., general public) have provided their attitudes concerning them (MacGregor, D. G., Slovic, P., & Malmfors, T. (1999). “How exposed is exposed enough? Lay inferences about chemical exposure.” Risk Analysis, 19(4), 649-659.) The general finding has been that they lay public tends to hold the attitude that, for example, a chemical release always leads to exposure and that mere exposure is sufficient to result in a health effect. Second, taken together these items embody two critical concepts necessary to appreciate the relationship between a hazard source and exposure, and between exposure and a health outcome. And, third, problem solving based on environmental science principles necessitates understanding and appreciating these distinctions, as well as knowing how to apply this knowledge as part of identifying a problem and tracing it to a health effect.
Table 7. Student pre/post difference scores for two concept items relating to chemical release, exposure and health effects.
| All Schools Combined |
Mainstream Schools |
Alternative Schools |
|
| “If a chemical is released . . .” |
Diff=-0.13 |
Diff=-0.12 |
Diff=-0.16 |
| "If a person is exposed . . .” |
Diff=-.08 |
Diff=-.09 |
Diff=-.08 |
For the all schools combined group, a significant change was exhibit from pretest to posttest with regard to both items: students’ responses moved in a direction more consistent with the principles of environmental science. Although there was less statistical power to detect effects for the alternative schools group, the direction of change was consistent with the mainstream schools group.
Taking Stock. The WQ implementation yielded a more powerful statistical evaluation than any other in the HCP. This provided an opportunity to view in greater depth and with great reliability the impact of the curriculum on student outcomes. Overall students gained significantly in terms of a number of important dimensions relevant to attaining environmental science principles, including the perception of environmental science risks, the importance of self-protective behaviors in the home, and the importance of environmental science information in problem solving. One of the strongest effects observed was that due to changes in students’ perceived ability to participate in group problem solving and to use environmental science concepts as part of their writing and speaking with others. In addition, students gained significant conceptual richness in how they view environmental science concepts that relate to the meaning of exposure and dose-response relationships. The gains achieved in self-efficacy provide a valuable adjunct to the attainment of concepts from the curriculum and provide a base from which students have the potential for greater engagement with science in society.
Back to Main Page - Quantitative Evaluation Scales and Results
