Section 2: Indoor Air Quality (IAQ) Curriculum Quantitative Evaluation - Summary of Results
The evaluation design organized participating schools into two program groups: single teacher schools and integrated team schools. The individual schools are shown in Table 3 along with the number of completed pretest-posttest evaluation protocols for each school. (Click on the school name for a description of each pilot school.)
Table 3. Schools participating in the IAQ evaluation.
| Single Teacher |
Integrated Team |
| Benson (N=133) |
Reynolds HS (N=51) |
| Westview HS (N=27) |
|
| Fir Ridge Campus (N=3) |
|
| Young Parent Program (N=24) |
|
| The Community Schoolhouse (N=39) |
The total number of completed pre/post evaluation protocols for the integrated team schools was N=145. The small N’s for the integrated team schools provided very low statistical power for significance tests conducted on the results of individual schools. The data for the five integrated team schools were combined to yield a larger group for comparison with the single teacher school.
Scale Means by Program and All Schools Combined. Table 4 shows the scale means by single teacher, integrated team and for all schools combined. For each of the program types (i.e., single teacher vs. integrated team) pretest and posttest means are shown as well as the number of students with complete pre/post protocols (N). Differences scores are shown along with a test of statistical significance of the value of the difference. At the far right of the table a between-program significance test is shown. This is test between the two program types done on their respective pretest, posttest and difference scores.
Of the nine scales shown in Table 4 three evidenced significant pre/post changes for the Integrated Team schools. These were for the scales PersInvolv, ValueSci, and EnviroRisk. For the Single Teacher Program, only the SelfPro2 scale resulted in significant pre/post change.
The largest (and most statistically significant) pre/post change was for the scale PersInvolv, but only for the Integrated Team schools. From pretest to posttest, students were more likely to express positive attitudes about their capability to participate in a variety of activities relating to discovering the causes of health effects in the hypothetical community of Mayville.
A moderately significant effect was obtained on the EnviroRisk scale for students in the Integrated Team schools. This three-item subscale of the Perception of Risk scale assessed students’ perceptions of risks from environmental allergens. Change scores were in the direction of greater perceived risk in the posttest than in the pretest assessment.
A significant result was obtained for pre/post change scores on the SelfPro2 scale for the Single Teacher Program. This result was largely due to responses to a single item: “For most of the chemicals I am exposed to in daily life, including chemicals in the environment, I feel I know how to protect my health and safety.” Pre/post change scores in response to this item were in the direction of more positive agreement, suggesting that students were more likely to see themselves as knowledgeable about self-protection with respect to chemical risks. This change was significant for both the Single Teacher and Integrated Team Program at the p<.05 level, and was significant at the p<.01 level for All Schools Combined, owing to the larger sample size and greater statistical power.
A significant result was obtained for the Explain scale for All Schools Combined. Here again, the greater statistical power obtained from combining all schools in the analysis of pre/post change scores contributed to the observation of a significant result overall that was not observed for either of the two school groups.
Looking at the pattern of result overall we see that student change over the course of the IAQ curriculum implementation has some important dimensions. Students tended to be more confident taking on a personal role in group activities relating to identifying and solving a problem relating to an environmental issue. This was the strongest outcome of the IAQ evaluation. In addition, students tended to be more sensitive to environmental risk issue, particular relating to curriculum components, suggesting that students did attend and assimilate important environmental science concepts through the IAQ curriculum implementation. Students were also more responsive to the need for self-protection with regard to environmental health and safety risks, and gained an additional appreciation for the value of science, both in general and in their personal lives.
Intuitive Toxicology (IntuiTox). An important concept in the evaluation of student outcomes is based on the notion that interaction with scientific and technical experts (either personally or through the curriculum) produces student change in the direction of attitudes and intuitions about exposure to environmental hazards that are more consistent with those of the environmental science community. We called this concept “Intuitive Toxicology” (IntuiTox) to reflect the intuitive models and concepts students have regarding the relationship between sources of hazards, exposure to hazards, dose-response relationships, and health-related outcomes. During Phase I of the evaluation (i.e., Pesticide Spill and Mysterious Illness Outbreak ) a multiple-item scale was developed to assess change in the concept. Although important changes did occur in the direction hypothesized, the scale itself tended to exhibit weak psychometric properties with a low and marginal internal consistency index (Cronbach’s alpha < .50). This low alpha argues against combining the individual items into a single scale (For specific items used to measure IntuiTox see the Dictionary of Evaluation Scales) Instead, the individual items were analyzed separately. All were rated on a scale from “1” (Strongly Disagree) to “4” (Strongly Agree) scale. A significant pre/post change was observed for the following IntuiTox item:
“If a person is exposed to a chemical that can cause cancer in humans, then that person will probably get cancer some day.”
Students in the Integrated Team schools tended to express greater disagreement with the statement in the posttest than the pretest (p<.01), an attitude reflecting greater awareness of the distinction between exposure and the dose-response relationship required to lead to a health outcome. This perspective is more consistent with how toxicological experts would respond. Students in the Single Teacher Program showed no significant pre/post change for this item.
Taking Stock. Looking across all of the student outcome results for the IAQ implementation one of the first things we notice is the limited statistical power available for pre/post significance tests. Despite this limitation in statistical power, some very important student outcomes were observed. Significant gains were made in students’ self-efficacy and confidence in their personal involvement with others working in a group to identify and analyze a community problem involving a potential environmentally-related health effect. These activities involved information seeking, interviewing community members and environmental experts, as well as preparing written and oral reports. The scenario aspect of the curriculum directly exposed students to the complexities of solving environmental problems through work with others and gave them an opportunity to learn and implement writing and oral presentation skills. These elements of the curriculum made a distinct and measurable impact on students and are important contributors to their ability to use environmental science concepts in the future.
Students also made gains in terms of their value for science and its role in their personal life, including self-protective behaviors in the home. Their increased sensitivity to environmental risk issues coupled with their increased ability to recognize how to use environmental science principles as part of everyday problem solving contributes to their overall skill set as effective decision makers.
Back to Main Page - Quantitative Evaluation Scales and Results
