We describe a National Science Foundation-funded project called ‘Evolution Readiness’ that used computer-based interactive models as well as hands-on activities to help fourth grade students learn Darwin's model of natural selection as the process primarily responsible for evolution. The inclusion of ‘readiness’ in the title is important to keep in mind. A full understanding of evolution would require the acquisition of a detailed model of how information is encoded in DNA, interpreted in cells, and manifested in organisms and species. To understand the evidence presented by the fossil record and its implications for evolutionary theory would require an appreciation of the immensity of geologic time as well as a substantive introduction to geology and paleontology. These topics are not easily accessible to ten-year-olds, but we have found that children can successfully perform virtual experiments that explore the connection between the interdependence of species and their remarkable adaptations and recognize the latter as arising gradually from small variations that affect reproductive success. Working in three school districts, located in Texas, Missouri, and Massachusetts, we implemented a curriculum unit covering 16 class periods. In each state the elementary science standards include all the concepts we cover, but traditional curricula do not attempt to integrate these concepts or to use them to explain observations of the natural world. We compared students who had used our materials to a baseline cohort taught by the same teachers but exposed only to the traditional curriculum. The treatment students outscored the baseline students, demonstrating the feasibility of teaching young students the fundamental concepts behind the theory of evolution and thus preparing them to deepen their understanding when they next encounter the topic.
That the wondrous interdependency and exquisite adaptations of living organisms could have evolved by natural causes is arguably the most far-reaching and powerful idea in all of science. It is also, perhaps, the least intuitive. At first blush, in fact, it seems incredible to suppose that the complex and seemingly purposefully designed natural world could possibly have evolved by natural causes. Not only is the concept counterintuitive, the evidence for it is mostly indirect and cannot be appreciated without prior knowledge of seemingly unrelated sciences, from biochemistry and genetics to geology and paleontology. In some circles, particularly in the United States, the theory of evolution is in conflict with firmly held religious convictions (Verhey 2005; Sinatra and Nadelson 2010; Scott 2004). For these and other reasons, it is no wonder that evolution is so hard to teach.
On the bright side, evolution would seem to be ideally suited to teaching via computer simulations, which can transcend space and time constraints to model processes that take place on scales from molecules to ecosystems and over times ranging from milliseconds to billions of years (Horwitz et al. 1996; Wilensky and Novak 2010; Rosca et al 2010). It is not possible, for example, to demonstrate evolution in the classroom because macroscopic organisms do not run through enough generations, over reasonable times, to show measurable evolutionary effectsa. Consequently, students cannot watch a population of plants or animals adapt to changes in their environment through selective pressure but they can observe and even manipulate such effects in a simulated population running on a computer. In a recent project, funded by the US National Science Foundation, we have offered them the opportunity to do that.
Our project, entitled ‘Evolution Readiness,’ was conceived as the first stage of a three-stage learning progression that would eventually encompass the elementary, middle school and high school years. Our target audience was students in the fourth grade, approximately ten years of age. Our goal was to teach these students the fundamental concepts that underlie Darwin’s model of evolution, leaving out details that could be taught at a later stage—to make them, in other words, ready to learn the full theory of evolution. We worked in three school districts, located in Massachusetts, Missouri, and Texas. The project lasted three years. In the first year we developed and validated several assessment instruments. Toward the end of the school year we used these instruments to assess the learning of all the fourth graders in each district, prior to our treatment. These baseline students had been taught in traditional fashion, using textbooks and other materials aligned to the state science standards, which include the concepts we target, though they do not require, or even suggest, that they be integrated within an evolutionary framework. In each of the following two years of the project, we implemented, in the same schools, a treatment that consisted of a mix of computer-based and hands-on activities designed to encourage and reinforce that integration. The student cohorts in the treatment years were drawn from the same population as the baseline (year 1) cohort and they were taught by many of the same teachers who taught the baseline cohort (there was some turnover and some teachers who had been left out of the treatment group, based on enthusiastic reports from their peers, insisted on implementing the treatment on their own). Learning results of the treatment groups were measured by the same assessment instruments as those used for the baseline, administered at the same time of year. Content learning of both treatment groups exceeded that of the baseline with an effect size of .3, significant at the .001 level. Differences between the two treatment cohorts were non-significant.
One of the first tasks of the project was to clarify and refine what we meant by evolution ‘readiness.’ From the start we were keenly aware of the difficulty we would face in trying to get ten-year-old children, for whom the time to their next birthday is likely to seem forever, to appreciate the immense stretches of time over which evolutionary processes take place. Accordingly, we decided that although even preschool children are often fascinated by extinct creatures, such as dinosaurs, we would not attempt to teach our students about ‘deep time.’ Reasoning along the same lines, we also eliminated any mention of processes that take place at a cellular or molecular level. Thus, ‘evolution readiness’ came to comprise for us those aspects of Darwin’s theory that are ‘human sized’ and can be modeled using processes that operate over relatively few generations. Not coincidentally, these processes are included in all of the state education standards we have examined, and in particular those of the states in which we worked: Massachusetts, Texas, and Missouri.
Specifically, state elementary life science standards include the notion that different animal and plant species differ from one another and that many of those differences can be ascribed to adaptation of the organisms to the particular environments in which they live. Thus, polar bears have long fur, lions have sharp teeth, fish are streamlined, and birds have wings so that they can survive in the cold, kill their prey, swim fast and fly, respectively. So far, so good.
The standards also include the concept of variation within species, for example, the fact that although all polar bear babies grow into adult polar bears, rather than, say, lions, not every polar bear looks the same, any more than human babies—unless they are identical twins—look exactly like their brothers or sisters. Finally, the idea of inheritance of various traits is included in the elementary science standards. Just as humans may inherit red hair or blue eyes from their parents so, too, lion cubs born of parents with particularly sharp teeth will tend to have sharp teeth. If sharp teeth give them a competitive advantage, such lions will tend to live longer and have more babies than other lions, and over time more and more lions will have sharp teeth. The last sentence is the key, of course, to an understanding of evolution by natural selection. It is the central insight that Darwin (and Wallace) used to explain the elaborate adaptations they and many others had observed in living creatures. Even though all the necessary pieces of the puzzle are taught, this unifying concept is all too often missing from the science standards, not only in elementary school, but even in the later grades.
The central question that we posed to ourselves on this project was: can we successfully teach this concept to fourth graders? To address this question, elementary students were introduced to the basic concept of natural selection as an explanation for the observed adaptations and evolution of organisms in nature through computer-based models that are linked to off-line learning activities. For example, in one computer activity, described below, students were able to discover through experimentation that a population of virtual plants was able to adapt, over many generations, to gradual changes in their environment eventually evolving into very different-looking organisms. This activity was reinforced by an offline one involving fanciful structures created out of Lego™ blocks that ‘evolved’ into separate populations descended from a common ancestor. Teachers were provided with a step-by-step classroom implementation guide, kits that contained hands-on classroom activities for students, as well as online and face-to-face professional development designed to prepare them to use the computer-based and off-line learning activities in their classrooms.
The secondary, but no less important, question was: how will we know whether they have understood it? To address this question, we developed a concept inventory that provides specific and detailed information about students’ understanding of the concepts introduced (Adams and Wieman 2011).
Learning activity development
The Evolution Readiness project developed ten computer-based learning activities and complemented these with five offline activities. The latter were adapted from existing sources, mostly to make them age-appropriate. The computer-based activities present themselves to students in the form of educational games based on a manipulable model that represents organisms that have both morphological and behavioral traits that make them more or less fit in different environments. The learning activities have definite goals and provide context-sensitive scaffolding in the form of helpful hints and congratulatory messages when the goal state is attained. Many offer real-world examples linked to the students’ explorations of the interactive model. All the activities keep track of everything the students do, including their answers to embedded questions, and report back to the teachers, as well as to the researchers. All of the computer-based activities are available free of charge online at http://concord.org/stem-resources/projects/evolution-readiness webcite.
Below, we give a brief description of the online and offline activities.
We made our plants annuals and had them die at the end of each virtual ‘year.’ This was important pedagogically, in that it reinforced the notion that individual plants do not evolve to adapt to changes in their environment; rather, it is the entire population of plants that is able to evolve over many generations, due to the variability of offspring in each generation and its effect on reproductive fitness.
In many of our learning activities we presented model environments in which critical features, such as sunlight or water, varied continuously as a function of position. Since the model plants cannot move in order to find a favorable environment, in these activities the plant population automatically distributes itself (via the dispersion of seeds) so that plants with different characteristics grow in different places. The effect is visually salient and serves to reinforce the concept.
We created a sequence consisting of five plant activities.
The virtual greenhouse
The goal of this activity is to teach students that plants with different types of leaves are adapted to different amounts of light. The students are given three different types of seeds and are challenged to determine by experimentation in which of five virtual flowerboxes—differing in the amount of light they receive—each type grows best (Figure 1). Students may keep track of their data by taking snapshots of each experiment and saving them in an online laboratory notebook that is incorporated into the program. The activity also introduces a bar graph that shows how many plants of each type have produced flowers, indicating that they are healthy and their environment is optimal for them.
Figure 1. The virtual greenhouse. The bars are color-coded to match the colors of the flowers.
The virtual field
In this activity, students plant seeds in a field with a gradient of illumination. Plants at the top of the field receive less light than those at the bottom. (Note that the direction of the gradient is reversed from that in the flowerbox arrangement of the Virtual Greenhouse activity, so that students do not confuse location with the critical environmental factor, light.) As in the flowerbox environment, plants with big leaves can only live where the light is least, while those with the smallest leaves must be planted in the part of the field that receives the most light if they are to survive, produce a flower, and drop seeds. The students discover this by experimenting with the same three types of seeds as before. If they plant their seeds in the wrong place the plants will wither or die and fail to produce seeds. This activity also introduces the plant life cycle. ‘Winter’ arrives at regular intervals and all the plants in the field die and disappear. Their seeds, if any, survive the winter and grow into plants the following spring. This feature of the model is pedagogically important because it reinforces the point that the evolutionary changes the students will observe as they progress take place over many generations and affect the population of plants rather than individuals.
Initially, all the ‘offspring’ plants are identical to the ‘parent’ plant—no new types appear and after many generations the field is populated by three distinct rows of plants, corresponding to the three types of seeds the student was able to plant (Figure 2). The activity ends with a ‘zoomed in’ simulation of a single plant that produces exactly six seeds—two of which grow into plants that are slightly different from those of the parent plant. These ‘mutant’ plants wilt and do not produce seeds in the environment into which they were born, but the student can pick them up and move them to a slightly different environment where they will thrive.
Figure 2. The virtual field. Note that without variation the three types of plants occupy distinct regions in the field, due to the gradient of light across it. The bar graph shows only those three types.
Mystery plant adaptation
The third activity revisits the zoomed-in scenario of inheritance with variation that ended the previous activity, using different varieties of plant. It then returns to the same field as before, with the ambient light level varying smoothly from top to bottom. This time the students are given only a single type of seed to plant: the type that grows best in the center of the field. But the model has been altered to include a critically important feature: variation. A small fraction of the seeds produced by each plant will grow into new plants that differ ever so slightly from the parent and are adapted to the light level just above the parent plant’s or just beneath it. Since each plant scatters its seeds randomly, occasionally one of these mutant seeds will fall in a location where the light level is just right for it. When this happens the seed will grow into a healthy plant that will produce seeds of its own. In this way, the single type of plant, which could only live in a particular horizontal slice of the field, eventually evolves into a full spectrum of different varieties capable of living and reproducing in every area of the virtual field. The subtle source of variation introduced in this activity thus has quite a dramatic effect in the long term, as shown in Figure 3.
Figure 3. Mystery plant adaptation. Note the bar graph, which indicates that every type of plant is present in the population.
Changes in the environment
The Mystery Plant Adaptation activity described above let the students observe the dramatic effect of introducing inherited variation into a model with a spatially inhomogeneous environment. The next activity in the sequence helps them develop that observation into an understanding of evolution by natural selection, by enabling them to experiment directly with the environment. The field starts off with a uniform light level midway between the maximum and the minimum, and thus capable of growing plants with medium-sized leaves. Students can alter the environment by ‘growing’ a chain of mountains of variable height right down the middle of it. In the presence of these mountains, depending on their height, the light level increases by 1 to 4 units on one side and decreases by the same amount on the other side. Students are challenged to grow the mountains to their maximum height (corresponding to the maximum change in light level) while maintaining a viable population of plants on each side. If they make the changes too abruptly the plant population will not have time to adjust and all the plants will die. However, if they change the environment one step at a time, waiting before making each change until there are sufficient numbers of mutant plants, then the normal plants will die, but roughly half of the mutants on each side will survive and constitute the basis for the next generation.
Mystery plant mystery
The final plant activity is intended to assess what the students have learned in the first four. In previous research (Horwitz and Christie 2000; Horwitz et al. 2010from computer-based manipulatives to hypermodels’), we have found quite often that students who are taught with game-like activities may become proficient at the game yet fail to learn the science concepts that underlie it. To test whether this was happening, we introduced a new environmental variable (water level) and added ten new varieties of plants with different root types, ranging continuously from deep to shallow, adapted to different water levels. (Plants with long ‘tap roots’ are adapted to dry conditions; those with shallow, wide-spreading roots need lots of water.) Using these plants, we constructed an activity to use as a transfer exercise and a test of whether or not a student has really understood the target concepts. The new activity involves the same concepts of reproduction with variation, natural selection, and adaptation but uses a water-to-root mapping, rather than a light-to-leaf mapping. This is a significant change, particularly since the roots of the plants are not normally visible: they can only be seen if the student ‘uproots’ the plant or observes it closely using specialized tools. By monitoring students’ use of these tools we can gauge their understanding of the importance of roots as factors affecting each plant’s fitness.
The activity starts with five flowerboxes, as in ‘The Virtual Greenhouse,’ and three types of seed. The flowerboxes differ in the amount of water they receive, and the challenge, as before, is to discover which seeds thrive in which environment. This time, though, the plants all look the same above the ground (they all have medium-sized leaves and pink flowers). Beneath the surface, however, their roots are different. Once the students have discovered this, they are presented with a field where the water level varies continuously from left to right, from one end to the other. They are provided with a packet of seeds, all of which grow the same type of plants. The seeds cost virtual money and the challenge to the students is to spend as little as possible on seeds but still produce a bumper crop of plants that can grow everywhere in the field, taking advantage of a small variation in root type from one generation to the next.
From a pedagogical point of view, the main difference between plants and animals is that, in our model at least, plants depend only on abiotic (non-living) factors, such as light and water, while animals consume other living things—plants and other animals. So, by bringing animals into our model world we were able to introduce the concept of a food chain and the fact that the interdependence of species at each level of the food chain implies that the environment of each species comprises, in part, all the other species with which it interacts. Thus, evolutionary changes in one species will affect others and vice versa, resulting in a sort of ‘adaptation arms race’ qualitatively different from the one-way response of the plant population to external changes in a non-living environment.
The animal activities were not completed until year three of the project, so they were used only by the second treatment cohort. There was a total of five of them, as described below.
Activity 6: the virtual ecosystem
With this activity we introduce students to the idea that all living organisms must compete for food with other living organisms (Figure 4). We do this interactively by having students take on the role of a rabbit in a field with edible plants. The students can control their rabbit, using the arrow keys to move it from one plant to another. When a rabbit moves onto a plant it ‘eats’ it, the plant’s icon disappears, and the rabbit’s hunger level is decreased. At first the student’s rabbit is alone in the field, but then other, computer-controlled, rabbits appear one by one. As the competition mounts, it becomes harder and harder for the students to keep their rabbit aliveb. Even if their particular rabbit starves, however, the population of rabbits survives and, from the evolutionary point of view, that is all that matters. Accordingly, an important goal of this activity is to encourage students to think globally: shifting from a focus on individual organisms to a concern for the well-being of the population as a whole.
Figure 4. The virtual ecosystem. The student controls a single rabbit, feeding it by moving it over grass in the field. When other rabbits enter the field, it becomes harder to get enough food to survive.
Activity 7: variations and adaptations
This activity introduces three varieties of plant, tall, medium, and short and lets students experiment to determine how climate can affect ecosystems. First, they investigate the effect of rainfall on the plants and discover that the larger plants can live in near-drought conditions, while the smaller ones perish. Next, we introduce variation in the rabbit population and challenge the students to figure out which variety of rabbit eats which kind of plant. The students are encouraged to make the connection between rainfall amount and the rabbit population’s ability to survive. So, they must think first about rainfall and plants, and then about plants and rabbits to infer that when certain plants cannot grow and reproduce, the rabbits that eat those plants will not have enough food to survive. In this way, students are introduced to the concept of interdependence in an ecosystem and its effect at the population level.
Activity 8: natural selection
In the third activity of the animal sequence, students explore how changes in the environment affect both the plants and the animals in a simple ecosystem with just two species living in it: grass and rabbits. They build a dam in the middle of the field, dividing the ecosystem in half (Figure 5). The area below the dam gradually dries out, which affects both the grass and the rabbit populations in that region. As the smaller plants die out, the rabbits that eat them soon follow suit. Once the students have observed this progression and entered data into their virtual laboratory notebooks, they remove the dam and observe as the ecosystem slowly returns to its original state. This is the first example they have encountered of an ecological ‘chain reaction,’ in which a change that directly affects one species has an indirect effect on another species–a simplified version of the subtle but significant and often unanticipated–cascades that have been observed in many ecosystems.
Figure 5. Natural selection. Students build a dam in the middle of the environment, which changes the water level and thus affects both the plants that can survive and the rabbits that eat those plants.
Activity 9: predators and prey
This activity uses a model of the Virtual Ecosystem with three species in it: grass, rabbits, and hawks, enabling the students to explore the effect of predation on the prey population. At first they ‘become’ a hawk and try to catch and eat brown and white rabbits on a snowy field. The latter blend into the background and are harder to see, which gives them a selective advantage. Having discovered through personal experience the reason for this selective advantage, the students proceed to explore an environment that changes over time, starting out white and gradually turning brown as the snow melts (Figure 6). A line graph shows plainly the shifting of the relative proportions of white and brown rabbits in response to this environmental change.
Figure 6. Predators and prey. As the environment changes, brown rabbits have a selective advantage.
Activity 10: experiment with ecosystems
This is the most open-ended of all the Evolution Readiness activities and, perhaps, the most challenging for students. The goal is to give the students the opportunity to ‘think like a scientist,’ making hypotheses, doing experiments, observing what happens, and analyzing and thinking about data. Students are encouraged to construct and conduct their own experiments with ecosystems comprising grass, rabbits, and up to two predator species: hawks and foxes. First, they are prompted to come up with a hypothesis for a particular question—for example, What will happen to the hawk population if the grass is removed? Then they are challenged to experiment with the model ecosystem in a way that allows them to test their hypothesis.
Offline activities and teacher support
We supplemented the computer-based activities described above with offline activities involving objects of various kinds (fourth graders are accustomed to hands-on materials). These activities were borrowed or adapted from existing curricula. All materials were supplied by the project to the participating teachers, including:
• several books about evolution written for children;
• an 18-foot-long vinyl timeline with graphics and text depicting the evolution of life over the past 600 million years (three games use the timeline);
• a set of Fast Plantsc together with a simple lighting and watering system, designed by the project, to facilitate their maintenance, so that students could experience the plant life cycle;
• a game called the ‘Lego Tree of Life’ that illustrates phylogenetic trees; materials included sets of large Lego pieces and special-purpose plastic laminated cards;
• another game called ‘Clip Birds’ that illustrates selective pressure by challenging students to pick up three different sizes of ‘seeds’ using three different kinds of clips;
• an activity that introduces the interdependence of species in an ecosystem by having students literally construct a ‘food web’ by unrolling and passing a ball of yarn between them to represent interactions between different trophic levels.
Teacher professional development
We anticipated that both the scientific content and the model-based pedagogy embodied in our approach would be novel to most teachers; accordingly, we provided extensive professional development in order to prepare elementary school teachers to help their students achieve state and national standards in the life sciences. Each of the participating teachers completed more than 50 hours of professional development, including a three-day (20-hour) summer workshop held at the Concord Consortium, where they were introduced to the Evolution Readiness software and the science behind the models. They installed and ran the software and participated in a wide complement of supplementary activities. Following the summer workshop, teachers participated in an online course, offered in Moodle, that engaged them in a variety of activities, including participating in discussions, watching videos, using interactive Web-based resources, and reading online and print materials. The online course consisted of the following six modules:
• Evolution 101 – Teachers gained a broad perspective on the central ideas of biological evolution, starting with an exploration of the life and work of Charles Darwin.
• Adaptation, Heritability, and Variation – Teachers learned that any model that incorporates adaptation, inheritance, and variation will evolve. They researched local examples of adaptation, heritability, and variation for their students.
• Common Descent – Teachers learned to classify organisms into species and learned to interpret evolutionary trees by looking for common traits. They explored the concept of evolution as descent with modification.
• The Mechanism for Evolution – Teachers learned how selective pressure can cause species to evolve over many generations and adapt to their environment.
• Nature of Science and the ‘E’ Word in the Classroom – Teachers learned about the nature of science, and specifically that an explanation is not scientific if it cannot, in principle, be falsified. They read the NSTA position statement about evolution and developed strategies for addressing possible controversy in the classroom regarding the teaching of evolution.
• Developing Pedagogical Content Knowledge – Teachers met in local study groups to develop a plan for delivering content materials to their students.
In addition to professional development offered to teachers, we created for each activity a teacher guide that included links to the learning goals, instructions for the use of interactive models and a suggested lesson plan with proposed discussion starters as well as answers to each of the questions embedded within the activity.
In the first year of the project, we began the development of the plant activities and tested them with volunteer students not enrolled in any of the three participating school districts. Some students planted their virtual seeds haphazardly; others, systematically. In the latter case it was not always easy to discern the pattern: one girl, for instance, planted her seeds in a seemingly random pattern that grew into a smiley face. We learned that we would not be able to rely solely on activity log data for assessing student understanding. We would need to scaffold the activities, add formative questions and provide the students with feedback.
The Massachusetts district was the first to use the Evolution Readiness curriculum. Two teachers at one elementary school implemented the curriculum. The school serves a diverse population where 51% of the students are eligible for free or reduced lunch. One of the participating teachers identified herself as a novice teacher while the other was more experienced. These teachers worked closely together throughout the year. Project team members observed every class.
The teachers had little support from the administration and had difficulty scheduling time in the computer lab, which is shared by the entire school. The lab itself was also problematic. The computers were relatively old and slow and the lab shared space within the school library. The librarian often ran a concurrent class, which resulted in a loud and distracting environment. The teachers were often unable to hold effective class discussions in the lab. In the end, however, they were able to schedule an adequate number of lab sessions to complete four activities, which were all that were available at the time.
From our observation of these teachers, we concluded that they required more support in understanding the content as well as additional help with pedagogy. We therefore offered a refresher workshop to the teachers in Texas and Missouri immediately before the start of their classroom implementations. We also redesigned the teacher guides, for instance using ‘stop-sign’ icons to mark places in the activities where teachers could bring the class together for discussions.
In Year 2, the Texas and Missouri implementations ran concurrently in the winter. Several of the Texas classrooms were bilingual, English-Spanish; accordingly, we produced Spanish language versions of all the computer activities and students were allowed to use the Spanish version if they preferred. (They could even toggle back and forth, as needed.) We developed a classroom observation protocol and trained one observer from each district in its use. Their classroom reports enabled the project team to follow the classroom work even though we were unable to attend the classes in person. Each of the computer activities monitored students’ actions and produced a log file of them. The content of these files was not reported to the teacher, but the information was available to us for research purposes and enabled us to track, for instance, how many times the students ran the models, whether they were successful in accomplishing their assigned task, how often their plants or animals ‘went extinct’ and so forth.
Class schedules at the Texas school were frequently rearranged and teachers were sometimes absent. When this happened, the computer lab teacher led the classes. Although he had attended the first teacher workshop and was familiar with the technology, he did not have a solid understanding of the content. We also noted that teachers did not always follow the teacher guide. For example, the ‘Fast Plants’ activity called for one class period where students would learn about the plant life cycle and plant their own Fast Plant seeds. Each student was to receive a container in which to plant his or her Fast Plant seeds. The class was then to discuss the needs of healthy plants. Instead of following this protocol, the teachers had a small group of students plant all the seeds in a single cup. (We sent new seeds and had them start over.) Despite these and other problems with the implementation, however, one member of the project team visited the school and found that both teachers and students were very engaged in the curriculum.
In Missouri, three teachers implemented the curriculum in two different schools. In one of these, the teacher was a ‘science specialist’ and ran the project with three different classes, one of which was his regular home room class. Computers were available in the back of each classroom, but there were not enough of them for students to work independently and many students had to be paired up. In two of the classrooms, the computers were newer and worked well, but in the third the computers were old and had small monitors with low screen resolution so students could not see the entire screen without scrolling both horizontally and vertically—a serious problem.
At the other Missouri school, two teachers implemented the curriculum with their students. The computer lab was located in the library, but the library was quiet and the computers were modern. Although one of the teachers was slightly technophobic, she became more comfortable teaching with technology as time went on. The two teachers worked together in a successful partnership, sharing their supplemental materials and engaging in regular discussion throughout the implementation.
Unfortunately, several implementation issues arose with each of our participating schools over the summer before the third and final year of the project. The principal at the Massachusetts school left and a new principal was assigned to the school. At the same time, the site coordinator retired abruptly due to health problems and was not able to help us transition between principals.
In Missouri, the science specialist teacher was re-assigned to teaching gifted students in a different school. In order to maintain a sufficient number of experimental subjects we recruited two new teachers, provided face-to-face professional development and requested that they complete the online course. The principal at this school also changed over the summer and the site coordinator was promoted and had much less time to devote to the project than she had had in the previous year.
But the Texas school district posed the most challenging problem. In the third year, the school was used to house new students who were supposed to be taught at another school nearby. That school was still under construction, however, and remained so until halfway into the school year. Obviously, this resulted in serious overcrowding at the school where we were hoping to collect data. The computer lab was transformed into classroom space and the computers were locked away in storage and did not become available until the spring semester. This school also had a new principal, and the site coordinator was moved to a middle school and was no longer onsite. One of the original teachers moved to a different grade level and left the project and three new teachers were recruited and trained.
In the final year of the project, we offered all participating teachers a second professional development workshop before they began using the software. Again, the Massachusetts school was the first to implement the intervention and, thus, the first to use the animal activities. The research plan for this final year dictated that there be no observers in the classrooms and that the teachers would submit an online survey after each lesson. However, since this was the first time the animal activities were piloted in real classrooms, project staff did observe those class sessions.
The two Massachusetts teachers reported that their year 3 students were at a very low level academically, significantly lower than the year 2 cohort. Over half the students in one class were on an individualized education program (IEP). In addition, both teachers were absent from their classes on the day of the post-test, because they had to attend a special meeting called by the principal. The substitute teachers had very little control over the classes, and even though a project team member was present to help administer the post-test, students would not follow instructions or sit quietly during the test period. The majority of students were unable to complete the entire test booklet, and this was reflected in their low scores.
The implementations in Missouri went more smoothly with both the new and returning teachers. Teachers reported that the students were engaged or very engaged in 85% of the activities (both computer-based and offline). We were particularly interested in the students’ experience with the new animal activities. Surveys completed after each activity were encouraging. One teacher said that students ’reatly enjoyed working with the animal models—they had more of a personal connection than with the plants.’ Students also made connections between the new animal supplemental materials and the computer activities. A teacher reported, ‘Students have made references to the activity as we have looked at other environmental changes.’
Due to the extreme overcrowding mentioned above, the Texas implementation got off to a very late start. Teachers had to rush through the activities and were forced to suspend the implementation for two weeks in the middle in order to prepare for and administer required state assessments. Nevertheless, the teachers gave the activities rave reviews and reported that their students were engaged or very engaged in 96% of the activities, both computer-based and offlined.
Assessment instrument development
In the process of refining exactly what we meant by ‘Evolution Readiness,’ we came up with a list of 11 ‘big ideas’ that we thought students in the fourth grade ought to know.
1. Basic needs of organisms
2. Life cycle—birth and death
3. Organisms and their environments
4. Classification of organisms
5. Inter-specific differences
6. Interactions between species
7. Intra-specific differences
9. Heritability of traits
11. Descent with modification
These ideas are based on the National Science Education Standards and are in the draft of the emerging common core standards, but they extend them in a crucial way. The national standards, in turn, are reflected in those for the various states, including the three (Massachusetts, Missouri, and Texas) where we did our research.
In order to evaluate the students’ learning, we developed a ‘Concept Inventory for Evolution Readiness’ (CIER). We developed and analyzed the CIER using Rasch principles (Rasch, 1961; Wright and Stone, 1979) a one-parameter Item Response Theory approach, because it allowed us: (1) to estimate the students’ ability independent of the item characteristics, (2) anchor the item characteristics across cohorts; and (3) use the item maps (Rasch maps) to go beyond the total score on the test to look at how students responded to the items that were aligned to the 11 ‘big ideas’ central to understanding evolution and descent with modification. The CIER is a 40-item (61-prompt) concept inventory divided between two test sessions to avoid fatigue effects. The CIER includes 32 multiple-choice, 5 short-answer, and 24 open-response prompts. Scorers for the CIER were trained to identify subtle variations in students’ responses that are indicative of teleological/Lamarckian and essentialist preconceptions. Inter-rater reliability of 0.85 was deemed acceptable for pairs of scorers. Rasch analyses conducted to examine the psychometric properties of the CIER concept inventory show that it is reliable and that the results are replicable; the person reliability is 0.88 and the item reliability is 0.97. The item (or Rasch) map and the person and item separation indices indicate a good fit between the test difficulty and elementary student ability. In addition, all items presented well-ordered thresholds indicating that, as expected, it was more difficult to obtain a higher score than it was to obtain a lower score on all items. Based on our psychometric analysis of the CIER, we concluded that the test is matched to elementary students’ ability and is a valid measure for examining students’ understanding of the concepts introduced by the Evolution Readiness activities.
We used a cohort design to collect pre/post-implementation data from the students of nine participating Grade 4 teachers. In all, we collected pre-implementation baseline data from 132 Grade 4e students (Cohort 1) who covered the same life science content but who had not been exposed to the Evolution Readiness intervention. The following year, a second cohort of 186 Grade 4 students (Cohort 2), made up of students from the same schools and the same teachers as Cohort 1, was exposed to the intervention. This was followed by another intervention with slightly modified activities in the third year of the project, using a cohort of 188 Grade 4 students (Cohort 3), again from the same group of schools and teachers. To avoid unintentional bias, trained scorers combined and scored all the cohorts simultaneously and so were blind as to whether students’ responses were from the pre- or post-implementation cohort.
Comparison of item maps across cohorts revealed that Cohorts 2 and 3 had a more complex understanding of evolution (Big Idea 8) than the pre-implementation cohort. Specifically, they understood that: species are adapted to their environments; if the environment changes only certain species survive; organisms with traits best suited to their environment have better chances of survival; species adapt to changes in their environment; the organisms carrying traits that are better suited for a particular environment will have more offspring; and selection pressure could lead to a change in the characteristics of a population. Similarly, Cohorts 2 and 3 outperformed the pre-implementation cohort on questions relating to descent with modification (Big Idea 11): they understood that different species could arise from one species if different groups had different selection pressures.
Overall, the mean for the pre-implementation Cohort 1 was 531.45 (s.d. = 68.40), the mean for the post-implementation Cohort 2 was 566.14 (s.d.= 80.07), and the mean for post-implementation Cohort 3 was 555.35 (s.d. = 76.78). An analysis of variance (ANOVA) showed that there were statistically significant differences among the cohort means (df = 2,503, F = 8.19, P <001). Post hoc tests showed that students in Cohort 2 performed statistically significantly higher on the CIER than students in pre-implementation Cohort 1 (P <.001). The effect size difference between Cohort 2 and Cohort 1 was 0.46 standard deviations. Similarly, the post hoc tests showed that Cohort 3 performed statistically significantly higher on the CIER than students in pre-implementation Cohort 1 (P <.05), and the effect size difference was 0.33 standard deviations. These effect sizes exceed the WWC guideline for minimal practical significance (US DoE, 2008). There was no significant difference between the scores for the two cohorts that received the intervention, Cohort 2 and Cohort 3 (P = .356, effect size =.13).
So, what did we learn from this educational experiment? The CIER results obtained in Year 2 of the Evolution Readiness project, and replicated the following year, demonstrate that the intervention succeeded in teaching some difficult concepts to very young students, in the context of their regular school-based science curriculum. This result was by no means certain at the outset. Evolution by natural selection is an archetypal example of an emergent behavior in which macro-level properties emerge as the result of micro-level interactions between system components. Systems that exhibit such behavior are notoriously difficult to teach, even to students considerably older than the ones in our study (Penner 2000), and it was by no means obvious that we would succeed in leading ten-year-olds to an understanding of evolutionary mechanisms. Accordingly, we were pleased to see that so many of our young students improved in their understanding of this topic, and not at all surprised that some of our big ideas remained beyond the reach of many of them.
We attribute the success of this project at least in part to the fact that we were able to use the computer to create a sort of ‘virtual laboratory’ within which students could experiment with systems that evolved over time periods short enough to be observed. In previous research at the middle and high school levels (Buckley et al. 2010Neumann and Schwartz 1996), we have used this approach to teach genetics and its connection to molecular biology (DNA and proteins). A logical next step, then, would be to expand this work to the upper grades by including those more advanced topics, creating in effect a new curriculum – ‘Biology Through an Evolutionary Lens’ – supported by virtual experiments based on a unifying multi-level computational model. We view the Evolution Readiness project as a first step toward accomplishing this goal.
a It is true, of course, that small, so-called ‘micro-evolutionary’ effects are observable in times shorter than a human lifespan; however, in most organisms these changes take place too incrementally to be seen in a classroom.
b It turns out, in fact, that the only way to stay alive for the 100 seconds required to win the game is not to eat if you are not hungry, thereby conserving resources that you are going to need later on when more and more rabbits arrive—a useful lesson even without evolution.
d ‘One boy in the first class had a huge aha moment when he talked about both plants on the different sides of the mountains having a common ancestor. That was a great comment.’ – Missouri teacher
e Although we worked with 4th grade students, the Evolution Readiness System covers material from the life science standards for grades 4 and 5.
The author declares that they have no competing interests.
Paul Horwitz was the Principal Investigator (PI) for the project. Carolyn Staudt and Laura O'Dwyer were co-PIs, respectively, for Concord Consortium and Boston College. Dr. O'Dwyer was specifically responsible for the research. Cynthia McIntyre and Trudi Lord developed the curriculum. All authors read and approved the final manuscript.
Adams, WK, & Wieman, CE (2011). Development and validation of instruments to measure learning of expert-like thinking. International Journal of Science Education, 33(9), 1289–1312. Publisher Full Text
Buckley, B, et al. (2010). Looking inside the Black Box: assessing model-based learning and inquiry in biologica. International Journal of Learning Technologies, 5(2), 166–190. Publisher Full Text
Horwitz, P, & Christie, MA In Michael J (Ed.) (2000). Computer-based manipulatives for teaching scientific reasoning: an example. Innovations in science and mathematics education: advanced designs for technologies of learning Lawrence Erlbaum Assoc.
Horwitz, P, et al. In Jacobson MJ, Reimann P (Eds.) (2009). Learning genetics with dragons: from computer-based manipulatives to hypermodels. Designs for learning environments of the future: international perspectives from the learning sciences (p. 2009). New York: Springer.
Horwitz, P, et al. In Jacobson M, Reimann P (Eds.) (2010). Learning genetics from dragons: from computer-based manipulatives to hypermodels. Designs for learning environments of the future New York: Springer.
Penner, DE (2000). Explaining systems: investigating middle school students' understanding of emergent phenomena. Journal of Research in Science Teaching, 37(8), 784–806. Publisher Full Text
Rasch, G (1961). On general laws and the meaning of measurement in psychology, pp. 321–334 in Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, IV. Berkeley, California: University of California Press.
Verhey, SD (2005). The effect of engaging prior learning on student attitudes toward creationism and evolution. Bioscience, 55(11), 996–1003. Publisher Full Text
Wilenski, U, & Novak, M In Taylor R, Ferrari M (Eds.) (2010). Teaching and learning evolution as an emergent process: the Beagle Project. Epistemology and science dducation New York and London: Routledge.