Section 1: Introduction - Quantitative Evaluation Scales and Results
INTRODUCTION
The Hydroville Curriculum Project (HCP) spanned a 7-year time frame, during which curricula were developed for four content areas. Each content area emphasized a distinct aspect of environmental health science (Click on the curriculum name in Table 1 for a complete description of the curricular modules). Table 1 indicates each of the curriculum content areas along with the school year in which the curriculum was implemented. No curriculum was implemented during the 2003-2004 school year to allow for review and evaluation of the previous two implementations before continuing with the program.
Table 1. Curriculum implementations by school year.
| School-Year Implemented |
|
| Pesticide Spill (PS) |
2001-2002 |
| Mysterious Illness Outbreak (MIO) |
2003-2003 |
| Indoor Air Quality (IAQ) |
2004-2005 |
| Water Quality (WQ) |
2005-2006 |
EVALUATION SCALES
A set of seven key concepts formed the basis for the evaluation of student outcomes. Each concept was used to develop a measurement scale comprised of a set of individual pretest/posttest items. The evaluation of the HCP with respect to student outcomes was broken down into two phases. The first phase corresponded to the first two implementation years (i.e., Pesticide Spill, Mysterious Illness Outbreak). During this phase the evaluation underwent development and testing that led to changes to the evaluation approach and the evaluation instrument. Although the same concepts were examined in both years, the number of evaluation items was significantly different. The second phase corresponded to the second two implementation years (i.e., Indoor Air Quality, Water Quality). The evaluation in each of these two implementations was the same. Each scale is identified and explained below, and summarized in Table 2. The scales form the basis for evaluation in all four implementations, but the specifics of each scales are those used in the second phase of the evaluation. A dictionary of evaluation scales and samples of the pre/post test documents can be found on this website (See 4b. Evaluation Instruments).
Quality of Explanations (Explain). An important component of problem solving using environmental health science principles is the ability to distinguish between the quality of various explanations that could be hypothesized to account for an observed health effect. In daily life, health effects are often reported in newspaper or media accounts. The 10-point Explain scale uses a media-style reporting of a health effect in the fictitious town of “Mayville” as the background for an evaluation exercise in which students judge the relative quality of each of a number of possible explanations. The set of explanations is based on factors that relate to differences in community contexts, potential exposures to environmental contaminants, and individual susceptibilities to health effects. For each explanation, the student provides one of five categorical responses ranging from “not a good explanation” to “slightly good . . .,” “moderately good . . .,”, “very good . . .,” and “don’t know.”
Information Seeking (InfoSeek). Problem solving using environmental health science principles requires the collection and interpretation of information. In doing so, students demonstrate their acquisition of knowledge about environmental health science concepts. The 8-point InfoSeek scale builds on the “Mayville” story problem by posing the context of a community problem solving committee that has an opportunity to collect information. Students judge how informative each of a number of activities are for helping the committee accomplish its purpose. The activities include interviewing local residents, having soil and water tests done, consulting the Internet for more information, and examining local buildings where people have become ill. A higher scale score indicates a more positive evaluation of how informative an activity would be for “helping solve the problem” of why the community of Mayville has higher incidence of serious illness.
Personal Involvement (PersInvolv). Problem solving in the context of environmental health sciences issues often requires social participation, such as public meetings or committees. For students to utilize the environmental health science knowledge gained through the Hydroville Curriculum, they must also have acquired a base of teamwork skills upon which they can draw, and the self-efficacy to utilize those skills in participation with others. The 6-point PersInvolv scale uses self-evaluation to assess students’ competence and capability with regard to a number of participation activities in the context of group problem solving. These include working with others, talking with community members, written and oral reporting, and interviewing a scientist. Students rate each item on a “Not . . . ,” “Slightly . . . ,” “Moderately . . . ,” “Very Qualified and Capable” scale. A higher scale score indicates a more positive self-evaluation of how “qualified and capable” students view themselves with respect to a number of different roles and activities that are part of a hypothetical committee problem-solving exercise.
Intuitive Toxicology (IntuiTox). A core objective of the Hydroville Curriculum is to improve the quality of students’ problem solving abilities through interaction with experts in environmental health science. Many of the concepts in environmental health science are complex and some are counterintuitive. A goal of student interaction with curriculum elements and with subject-matter experts is to increase their awareness and understanding of environmental health science and to learn to apply the same reasoning principles as do environmental health scientists to solve problems. The 7-point IntuiTox scale assesses student change with respect to a set of concepts in environmental health risk assessment and provides a basis for evaluating the degree to which students develop attitudes about environmental health science that are more consistent with those of science professionals. The scale items include perceptions of chemical exposure, the meaning of exposure, and the relationship between exposure and health-related outcomes. Students rate each item on a four-point “Strongly Agree/Disagree” scale. A higher scale score indicates perceptions and attitudes about chemicals and chemical risks that are more consistent with those of environmental science professionals.
Self Protection (SelfPro1 & SelfPro2). Key objectives of the Hydroville Curriculum are for students to develop appropriate increases in self-protective behavior with respect to environmental health and safety risks. The 10-point SelfPro1 scale assesses student change with respect to self-protective behavior associated with a number of activities in the home, including washing fresh food, reading directions and warning labels, using self-protective equipment, and testing environmental components (e.g., air, water). Students rate each of the items on a five-point scale ranging from “not important,” “seldom,” “important once in a while,” “ . . . most of the time,” and “always important to me or my family.” A higher scale score indicates a more positive attitude toward self-protective behaviors with respect to sources of environmental health and safety risk.
The 3-point SelfPro2 scale assesses students’ attitudes toward environmental health and safety in general. Students evaluate a set of attitude statements regarding their perceived ability to recognize and manage health and safety risks in the home, including their ability to read and understand warning labels. Each item is rated on a four-point “Strongly Agree/Disagree” scale. A higher scale score indicates a more positive attitude toward self-protection in the home.
Value For Science (ValueSci). Acquiring and using environmental health science knowledge depends upon and contributes to a positive view of science in daily life, and a positive view of science as a social enterprise. The 7-pointValueSci scale assesses students’ attitudes toward science in general, environmental health science in particular, and the value of science for making personal health and safety decisions. Items include interest in science, confidence in talking about science with others, and views about science as a possible career direction. Students rate each item on a four-point “Strongly Agree/Disagree” scale. A higher scale score indicates a more positive value for science in daily life.
Perception of Risk (PercRisk). As students develop greater knowledge skills and conceptual awareness of environmental health science, their perception of environmental health science risks should change with respect to other risks and hazards. The 10-point RiskPerc scale assesses change in students’ perceptions of a broad range of health and safety risks (e.g., smoking, alcohol use, food additives), including those identified by environmental health science (e.g., air pollutants, water contaminants, environmental allergens). Scale items are rated on a five-category scale ranging from “no risk,” slight risk,” “moderate risk,” high risk,” and “don’t know.” A higher scale score indicates greater perceived risk.
Two derived scales are based on a subset of the 10 items comprising the overall PercRisk scale: ChemRisk, a 3-item scale based on the chemical risk items (i.e., second hand cigarette smoke, household cleaning products, food additives), and EnviroRisk, a 3-item scale based on the environmental risk items (i.e., indoor air pollutants, water contaminants, environmental allergens.)
Table 2. Hydroville Student Evaluation Scales 2004/2005 & 2005/2006 Implementation Years (Indoor Air Quality & Water Quality Curricula)
| Number of Items |
|||
| Quality of Explanations |
10 |
Higher score indicates higher quality of explanations. |
|
| Information Seeking |
8 |
Higher score reflects greater informativeness of items. |
|
| Personal Involvement |
6 |
Higher score reflects more positive self-evaluation of capabilities. |
|
| Self-Protection 1 |
SelfPro1 |
10 |
Higher score reflects more positive attitude toward self-protective behaviors. |
| Self-Protection 2 |
SelfPro2 |
3 |
Higher scores reflects more positive attitude toward self-protection |
| Value for Science |
ValueSci |
7 |
Higher score reflects a more positive value for science in daily life. |
| Perception of Risk |
PercRisk |
10 |
Higher score reflects greater perceived risk. |
| Perception of Chemical Risks (subscale) |
ChemRisk |
3 |
Higher score reflects greater perceived risk from chemicals. |
| Perception of Environmental Risks (subscale) |
EnviroRisk |
3 |
Higher score reflects greater perceived risk from environmental pollutants. |
Back to Main Page - Quantitative Evaluation Scales and Results
