Making Room for Creativity in Science Class

How many times have you had a student ask, “Will this be on the test?” I often did as a teacher. They fully expect the test questions to have one right answer, and I would often give them those answers over the course of a unit. Students might have to plug in different numbers, but there wasn’t a ton of critical thinking in the end.

I have wondered, “How could I have made more room for creativity in my science class?” I have a few ideas and welcome yours in the comments!

First, create a culture where students ask questions. When considering student questions, I think teachers worry about tangents that will distract from concepts they “need” to cover. At least, I worried about that. In my experience, when a unit focuses on students figuring out a key phenomenon, the questions they ask tend to be naturally answered in the planned unit anyway. The students, however, at least had a chance at creativity when asking them, and they buy in as they see them answered. For questions that aren’t part of the unit, let students go on a research tangent or test different variables, and give them credit for doing so. Of course, that sounds a bit like project-based learning, which is an obvious pathway here too and enables students to research/experiment based on their own interests. While I didn’t connect them as well I could have to their learning, I honestly loved crazy questions from students, so I appreciate the work of Randall Munroe in xkcd with "what if" questions like, “What if I had a mole of moles?”

Second, find ways to get rid of one right answer tasks. I thought web searches were going to destroy these questions, but now I see that AI is going to obliterate them. Teachers tell me they need to know whether students know definitions. The challenge is that they don’t remember them anyway. I had amazing students who would come back the next school year, and I’d ask, “What’s a proton?” They seriously had no idea. Testing definitions is pointless in the long run, and even in the short term, students often hide behind an illusion of understanding by being able to spout the right terms. They have to be asked to apply ideas to their worlds, to things that they’re interested in, and to current contexts. Plugging numbers into formulas is similarly problematic. Testing on conceptual understanding of what a formula means and how it’s used requires more critical thinking (and is more likely to support remembering it). To help make this change, students could even create the tasks (or questions) that they find meaningful.

Third, push for originality in scientific modeling. Modeling can also become a process of looking for what the teacher wants, but that defeats the whole purpose of modeling! We want students to make connections among ideas in their world and through the lens of their background understanding, not replicate diagrams from a book. In a workshop, I have used a phenomenon of cup phones several times and asked educators to model how they work, as well as why they work sometimes and not others. Once a teacher took the blank paper I handed out and folded it up like an accordion. When asked to share her model, she showed the folds bumping into each other and transferring energy. Brilliant! If I had specifically asked them to draw something or given a starting diagram to add to, I never would have seen that creativity shine.

Notably, I supported a research project where we asked students in grades K-8 to make a model of blowing a crumpled-up piece of paper across a desk. Younger students were more likely to include themselves in their model. Some 8th graders did too, but they were more likely to only draw a mouth or maybe a mouth to lungs system. We interpreted that as younger kids being more likely to see themselves as part of the scientific process, while older kids abstracted science to something outside of who they are. We should encourage kids to make the models personally relevant, meaning giving them phenomena where that will be possible, rather than stressing a system that doesn’t include them in the picture.

Finally, celebrate your students’ ideas! Whenever they get excited, get excited with them, even if it’s hard to do. I had a student who was excited to bring in articles that argued against human-caused climate change. It tended to make me grumpy, as they didn’t represent quality science. But, he wanted to talk science! I really should have celebrated his interest in pursuing science topics outside of school. I should have called my excitement out to the class! Because that’s what science should be, full of wonder and questions and creative expression, even in the face of the competing priorities and obligations of being a teacher.

Creating NGSS-Aligned Performance Tasks – Part 1

Whether you’re at the end of the unit or want to check for understanding earlier, performance tasks provide a way to gauge students’ abilities to engage in scientific thinking and use their content knowledge. It’s difficult to truly determine their depth of understanding of a concept or their ability to create scientific models and explanations through multiple choice or brief-response questions. As seen in the image to the right, people training to be astronauts don’t just answer multiple-choice questions! Performance tasks have the potential to provide more meaningful information to guide instruction and to frame feedback for students. But how do you create high-quality, NGSS-aligned tasks? Here’s one idea for a process to do so, and my next blog post will detail an example of going through this process.

1. Determine a phenomenon – considering the current unit, what relevant phenomenon would make students go “hmmmm”? One new resource I found that includes some fabulous phenomena comes from the California Academy of Sciences, called BioGraphic. Generally, phenomena don’t need to be earth-shattering ideas. I like having an interesting question to guide a unit, then connect that to large- and/or small-scale experiences and engaging stories. For example, that could be declining bat populations or dropping a bowling ball and a feather in a vacuum. A task with your selected phenomenon as a context could frame a performance task at the beginning, middle, or end of the unit. 


2. Work with practices – After determining a relevant phenomenon, I consider which science and engineering practice (SEP) would bring it to life and which SEP my students need more work with. It would be great if I was collaboratively working on a particular SEP with my department, making that a natural choice. Considering practices, I would not try to assess a practice as a whole, such as analyzing and interpreting data. It’s more useful to focus on a particular subskill in order to design the task, clearly determine students’ abilities, and provide specific feedback. Handy ideas for subskills can be found within Appendix F, the progression of SEPs, and the NGSS Evidence Statements, which break down each performance expectation by subskills of each practice. 


3. Form a learning target – My primary learning target would be having students use a subskill of a science practice to work with a specific disciplinary core idea (DCI). To achieve three-dimensionality, a crosscutting concept (CCC) might be an implicit part of this learning target. Once I start framing learning targets that are three-dimensional, I start stuttering in the process of rubric creation (as noted in the last blog post). Instead, I often use two learning targets: one that connects practice and content and a second that connects content and big ideas (CCCs).


4. Flesh out the scenario – With the goals and context of the task in mind, I begin to craft the story and related questions. Which part of the story are students exploring in this task? How does it fit into the overall storyline of the unit. My task might begin a unit, such as engaging students in data that describe concentrations of various chemicals in a nearby lake over the past 50 years. Students would go on to explore ecosystems, water chemistry, and human impacts. Crosscutting concepts are a wonderful resource for creating questions for the task, as each can be transformed into an authentic scientific question. For example, “What is the scale of the agricultural runoff problem?” Or, “What are the important inputs and outputs to consider in the sturgeons’ ecosystem?” This could be an opportunity for students to ask their own questions. Another great resource for framing questions based on the practices is the NGSS Task Formats from the Research + Practice Collaboratory. It provides a series of question templates that can be adapted to wide-ranging contexts. In the end, you’ll want to consider whether the question or series of questions in the task will be moving them further toward expertise in relation to performance expectations (PEs)—not that you’d have a goal of checking off proficiency in relation to PEs, more that you’d consider building student progress toward them through multiple authentic tasks. 


5. Create a vision of proficiency – I outline my main ideas on proficiency in my previous blog post on rubrics. For proficiency with explanations, I also like the “What, How, Why” rubric by Thompson et al.. Notably, expectations for proficiency may start out a bit vague – having sample student work will help clarify what proficiency looks like, and further rubrics will improve over time. It’s a process! Also, I believe that teachers should reflect on whether or not these individual pieces of proficiency will add up to an assessment of your overall vision for students’ learning in science. Additionally, it’s important to consider whether you want individual proficiency or if you can glean important information to guide instruction from group work. Or, can students’ self- and peer-assessment provide the critical learning at this point? Rubrics or other proficiency guidance should be accessible to students. 


6. Reflect – Both students and teachers should take time to reflect on the task. Teachers would reflect on evidence of student learning and how the task performed. Did it provide the information wanted in relation to the practice and content? Was it clear to students? Students should receive feedback sufficient to understand where they’re at in their learning in relationship to the goals put forth. That reflection can be supported by personal, peer, or teacher feedback. A key question with all assessment is: How are you giving feedback to students and how are they acting on that feedback? Honestly, I wouldn’t do in-depth reflection with every task; that could quickly become overwhelming. I’d recommend at least once per unit, with a range of practices throughout the year. Teachers will need at least a few common tasks and rubrics to use collaboratively through the year and discuss. 

As noted above, my next blog post will provide an example of going through this process to create a performance task.

Image courtesy of NASA: https://www.nasa.gov/feature/simulators-give-astronauts-glimpse-of-future-flights

Creating Rubrics for Performance Tasks Aligned to NGSS – Part 2

I created the three-dimensional rubric below in an attempt to help get the ball rolling. I have honestly not yet seen a rubric where the creator claims it is three-dimensional. I’m not sure I’m there yet, so critique away! Most rubrics I’ve found only focus on the practices, which I agree is a good place to start (see the resource list at the end of this post). I would, however, like to see practices and crosscutting concepts linked to content within a rubric, so I attempted to do that here. Importantly, column three represents where a proficient student should be, while four provides ideas for more advanced studies.

Some background on this unit of study and the related performance task:

• High school biology students are investigating ecosystems (LS2.C), human impacts on those ecosystems (LS4.D), and related pollution chemistry (PS1.B).
• Imagining I’m still teaching… I engage the class in this unit by having them walk over to a nearby lake to make observations, ask questions, and take multiple water samples, highlighting the presence of large amounts of algae if students don’t bring it up. We meet the regional limnologist there and she briefly shares some information about pollution in the lake system and is on hand for questions (could alternatively Skype w/ a scientist or even watch a short watching a short video detailing pollution challenges – such as this news story). 
• The next day students discuss their observations and consider how and why the ecosystem in their local lake may be changing. They model the ecosystem of the lake, detailing relationships within and across biotic and abiotic elements, including what might be causing ecosystem changes. The models provide a formative assessment on students’ modeling ability and their background understanding of ecosystems generally, but also within the lake context. After completion, class sharing and discussion of those models serves to build common background knowledge about topics such as farm runoff and other pollutants affecting the lake.
• I want to know where students are at in their ability to ask testable questions in an ecosystem modeling framework (Practice - Asking Questions; Crosscutting Concept – Systems and System Models). So, toward the end of that class I ask them to individually develop questions for studying changes to the lake ecosystem, framing those questions with the lens of the full system and available data on lake chemistry (e.g. data like this). I use the following rubric to score students’ individual responses before having them revise their questions in groups the next day. 

[click here for a full-sized pdf of the rubric]

Here are some of my considerations in crafting this rubric:

• I developed goals for the unit first and then created the rubric in conjunction with creating the investigations within the unit. I want multiple opportunities to assess student learning in a more formal way through a unit, and this performance task and rubric flowed out of the progression being built. So, the goals for learning represented in the rubric were in mind throughout the process, not an afterthought.
• Our state vision for science learning in Wisconsin comes from page one of the summary of the NRC Science Education Framework. I’d want my assessment to provide information as to whether students are progressing toward that vision as well as through the NGSS progression we’d laid out for the year. The goals of this lesson, students being able to ask meaningful questions about local water pollution and the chemical impact on ecosystems, do fit within those broader goals.
• Possibly the most important resource for designing the rubric was Appendix F, the progression document for the practices. The progression detailed for grades K-2, 3-5, 6-8, and 9-12 for asking questions provided ideas for where students should be and where they’re coming from, supporting the development of the columns within the rubric. They provide ideas for a developmental progression of learning without resorting to terms like never, somewhat, and always. Specifically, based on the progressions of the asking questions practice, I included having students connect questions to an analysis of data and systems.
• Another important resource for designing the rubric was the NGSS Evidence Statements document. The evidence statements provide a concrete way to break down a practice into specific subskills, which is very useful in articulating the multiple rows of a rubric. In my case, they were most useful in suggesting that the question needs to be practicably testable (in the classroom) and relate to cause and effect.
• Finally, I also used Appendix G, the progression document detailing the crosscutting concept of systems and system models. From this progression, I pulled ideas of inputs and outputs within the system, understanding the boundaries of the system to better formulate the question.  So, the rubric pushes students to consider how timeframes and a narrowed focus on particle chemicals and lake inputs could lead to a better question.
• The specific NGSS components targeted here are: SEP Ask Questions, CCC Systems and System Models, and DCIs HS-LS2.C, HS-LS4.D, and HS-PS1.B. 
• I also wanted to focus on questioning as the NGSS performance expectations (PEs) have limited connections to the questioning practice (only two in middle school and two in high school). Because teachers make the mistake of using the PEs to design their instruction, I worry students won’t have as many opportunities as they should to ask questions.
• I used the idea of “with guidance” as part of the progression. It was a tough decision to include that. I felt that if we’re talking about a true developmental progression, the first step is often being able to do it with some help. Some students need scaffolding to get going with a skill, and they’re not going to be independent at first. So, I reflected that within this rubric.
• Additionally, I’d want to have student responses to the performance task to serve as examples (anchors) of the varying levels within the rubric. I didn’t feel I could meaningfully create those on my own, so I hope to get some teachers to try this rubric, or something similar, and share anonymized samples of student work.

For the best outcomes, teachers should collaboratively create these rubrics or collaboratively refine and revise an existing rubric to meet their needs/vision. To improve instruction for all students, it’s also essential that they collaboratively review student work in light of the rubric. It won’t be perfect the first time! Teachers will have to improve the rubric over time along with other elements of their instruction based on formal and informal assessment data.

My next blog post will discuss strategies for developing NGSS-based performance tasks.

Annotated links to other resources w/ rubrics – please, add a link to yours in the comments!

• “Collaborative Inquiry into Students’ Evidence-based Explanations: How Groups of Science Teachers Can Improve Teaching and Learning” is article by Jessica Thompson, Melissa Braaten, Mark Windschitl, et al. This article provides details on how to create rubrics that detail learning progressions in terms of the what, how, and why of explanations. A sample rubric with embedded anchors of explanations, shows what student reasoning might look like, is provided.
•  The Design-BasedImplementation Research team created a first draft of a rubric on the practiceof scientific modeling. It provides super useful details on what constitutes effective modeling. A problem is that it’s a bit long to be useful, though perhaps portions of it could be pulled out to assess subskills. I also don’t think progressions of ability using language such as “does not,” “some,” and “all” is as straightforward as denoting what students at different levels can do. 
• The Instructional Leadership for Science Practices group provides a series of rubrics based on each practice that can be used to evaluate student performance. Or, there’s another version of the rubrics that could be used by an observer to provide teachers feedback on how the practices are being used in his/her classroom. Though, both versions tend to focus more on what students have the opportunity to do than what they have the capacity to do.
• Wisconsin's Marshall High School has been working on standards-based grading and created a rubric based on the practices and life sciences DCIs. 
• Arapahoe Elementary in the Adams County Five Star School District provides standards-basedgrading rubrics linked to NGSS – It gives a generic rubric template you’d use to plug in specifics for each particular CCC or SEP or DCI, but it might not provide sufficient information or nuances for individual SEPs, CCCs.
• Edutopia provides a rubric for science projects, which has some good ideas for progressions of abilities, but remains fairly traditional - built from “scientific method” steps.
• And, thanks to Cathy Boland, @MsBolandSci, for sharing a rubric for explanations through Twitter - I hope others will share too! 

Creating Rubrics for Performances Tasks Aligned to NGSS – Part 1

Evaluating current efforts to move science education forward, such as that framed by the Next Generation Science Standards, requires “assessments that are significantly different from those in current use” (National Academies report, Developing Assessments for the Next Generation Science Standards, 2014). Performance tasks in particular offer significant insights into what students know and are able to do. “Through the use of rubrics [for such tasks] … students can receive feedback” that provides them a “much better idea of what they can do differently next time” (Conley and Darling-Hammond, 2013). Learning is enhanced! Building from a vision of the skills and knowledge of a science literate student, rubrics can allow students (and other educators) to see a clearer path toward that literacy.

Of course, using rubrics with performance tasks is generally a more time-intensive process than creating a multiple-choice or fill-in-the-blank exam. In order to be used as part of a standardized testing system or as reliable common assessments, scoring these types of tasks requires more technical considerations:

• Tasks must include a clear idea for what proficient and non-proficiency looks like; 
• Scoring must involve multiple scorers who all have a clear understanding of the criteria in the rubric; and, 
• Designers need to develop clear rubric descriptors and gather multiple student anchor responses at each level for reference.

Often, when teachers grade student projects or other performance tasks on a rubric, it looks something like this (click for larger image/pdf):

While this example comes from a 4^th grade classroom, secondary rubrics often have similar characteristics.

Considering alignment to the NGSS, and effective rubric qualities in general, there are several changes I’d make:

1. What’s the science learning involved? They appear to be drawing or making a model. About what? What understanding would a proficient model of that phenomenon display?
2. What would mistakes look like in a model? I’m a little worried that the “model” here is just memorizing and recreating a diagram from another source. Students’ models should look different. There might be a mistake in not including a key element of a model to describe the phenomenon, or not noting a relationship between two of those elements. Types of mistakes indicating students aren’t proficient should be detailed. 
3. Neatness and organization are important, but I question the use of those terms as their own category. I would connect that idea to the practice of scientific communication. Does the student clearly and accurately communicate his or her ideas? Do they provide the necessary personal or research evidence to support their ideas? The same is true within the data category. I’m more concerned about whether the students can display the data accurately and explain what the data means, than whether they use pen, markers, and rulers to make their graphs…
4. It’s not a bad idea to connect to English Language Arts (ELA) standards—done here with the “project well-written” category. At the elementary level in particular that makes sense; however, I’d want to ensure that I’m connecting to more, actual ELA standards, such as the CCSS ELA anchor standards for writing, which emphasize ideas like using relevant and sufficient evidence. At upper grades, I’d also want to emphasize disciplinary literacy in science (e.g., how do scientists write?) over general literacy skills.
5. This rubric actually does better than many at focusing on student capacity rather than behaviors. I see many rubrics that score responsibility and on-task time, rather than scientific skills and understanding. Check out Rick Wormeli’s ideas. 
6. What does “somewhat” really suggest? Is the different between two mistakes and three mistakes really a critical learning boundary? I see a lot of rubrics that substitute always, sometimes, and never for a true progression of what students should know and be able to do. I also see many rubrics that differentiate rankings by saying things like no more than two errors, three to five, errors, more than five errors. What do we really know from that? What types of errors are made? Is it the same error multiple times? What exactly can’t the student do in one proficiency category vs. the next? I really can’t tell by just saying two vs. four errors.
7. Use anchors for clarity – this rubric notes that models are “self-explanatory” and that sentences have “good structure.” Do students really know what that means? Have they seen and discussed a self-explanatory model vs. one that is not self-explanatory? If there is space, an example of a model or sentence meeting the standard could be embedded write into the rubric in the appropriate column. If there isn’t a space, a rubric on a Google doc could link to those types of examples.

Looking around, really looking, I have found very few rubrics that make an attempt to align to the NGSS. I suspect some people are still nervous about sharing. In my next blog post, I’ll share an NGSS-aligned, three-dimensional rubric I created and detail the process involved. I’ll also share some resources from other groups tackling this work. 

Formative Assessment and the NGSS – Part 2

A new colleague here at the Wisconsin Department of Public Instruction, Lauren Zellmer, read through my last blog on formative assessments. Her question was, “What specifically do teachers do with the information once they’ve conducted these formative assessments?” Great question! I decided to write a Part 2 of the formative assessment blog, where I’ll share a few more details for possible instructional next steps based on hypothetical results.

First teacher – Rubric on modeling

In the first example, the teacher collected whole-class and individual information using a modeling practice rubric, as he walked around asking probing questions and jotting down student names across the rubric continuum. After some reflection in pairs, he had a few students share their models in order to highlight key aspects of the practice, which will help build capacity in all students. Depending on students’ level of understanding, further support might include the following:

• If he found that most students did not fully understand this element of modeling (mostly 1’s and 2’s), he could provide scaffolded modeling instruction in the next part of the lesson requiring modeling. He would prepare a modeling handout which lists possible elements of the model and requires students to note whether to include those aspects and why. Students already proficient would complete models without that scaffold. 
•  If he found that understanding is fairly varied (largely 2’s and 3’s, with some 1’s and 4’s), the teacher could provide further time for group reflection and sharing. That further reflection would best happen immediately—after a few, selected groups shared elements of their models, the class could get back in pairs to improve their models based on those ideas. And, next time modeling occurs in a lesson, the teacher could repeat a similar in-depth process, like the first time, to continue to provide significant support. 
•  If he found that most students proficiently performed this aspect of the practice (largely 3’s, with some 2’s and 4’s), he could make a note of which students are still struggling. The next time modeling happens, instead of moving around the class generally to assess where students are at, he could narrowly focus his support and questioning on those students. He might provide them some in-depth small group help, with scaffolds provided. He could also pair them with proficient students where he knows they won’t just be given answers, but meaningfully supported in their learning.

Second teacher – Testable questions
In the second example, the teacher had students write questions about biodiversity while on a nature walk, thus collecting whole-class and individual information on whether students could write testable questions. The next day, with the whole class, she shared useful “yellows,” where students’ questions needed more work, and “greens,” where students’ questions were testable. She also made some notes in a file as to where the class seemed to be overall with this skill. Depending on students’ level of understanding, further support might include the following:

• If she found that most students could not write a testable question (lots of “yellow”), she should do more than read through notable yellows and greens. After that review, students could receive more practice in a guided, whole class discussion, evaluating a series of questions, noting whether or not they’re testable, and fixing them to make them testable. Then, she could ask students in small groups to collaboratively revise their questions to make them testable. These groups would include a student who did proficiently demonstrate this skill where possible. 
• If she found that about half could write testable questions and half could not, I would again suggest she have students rewrite their questions in small groups. For students still struggling after a round-two attempt, she could support them in a small group with an activity like that noted above (evaluating examples together). The remainder of the students might begin some independent research or brainstorming on the design of the investigation to answer their questions. 
• If most could write testable questions, she might provide individual help to students still struggling with their questions after the discussion of the greens and yellows. Those students could use the reviewed questions as models to revise their own, with the teacher ensuring they can explain why their original ideas were not testable.

Formative assessment is “assessment for learning”. Teachers need specific ideas as to what strategies they’re going to implement depending on the results of the formative assessment. They also need some way to record progress, not only relying on “gut feelings” and memory (my brain and gut, at least, aren’t that reliable). Having some class assessment notes on a page or a rubric can be a quick means to do so.  Finally, coming together with peers to discuss the data and possible strategies is critical to moving forward as a science department (and community).

Using Surveys as Part of the Evaluation of School Science Programs

Surveys of students, teachers, and community members will provide critical information in the process of determining whether changes made to your science education program improve desired outcomes. Many important questions cannot be answered through typical science assessments. While it’s clearly essential that students understand and can do science, do they sincerely believe that someone like themselves could be a scientist? Further, are you changing not just knowledge of, but beliefs about, science? Do students see how science relates to their lives? Is it meaningful for them? Or, do they see a need to question “scientific evidence” within popular media?

A recent article in National Geographic noted that solid, research-based science often faces organized and angry opposition. We don’t want students leaving school doubting the consensus of the scientific community (unless they somehow have sufficient, valid evidence to doubt a claim). They can understand how vaccines work and still decide not to have their children vaccinated. It’s unfortunate that our society believes in science, but not its findings.

Furthermore, do students understand who scientists are and what they do? While it was created as a tool for K-5, the “Draw-a-Scientist” test (DAST) could be done at secondary levels as well. My 8^th graders certainly held onto stereotypes of scientists. We want students to see science as including a wide-range of tasks by a wide-range of people, particularly people who look like them and have interests similar to their own.

Here are a few sample surveys of student attitudes: 

• A wide range of examples are provided on this site from Pearson   
• Survey on student attitudes toward science and other STEM subjects (Faber, et al., 2013). This paper describes use and development of the survey, with the actual tool at the end. 
• Changes in Attitudes about the Relevance of Science (CARS) – with three versions of the instrument, it could be given at the beginning, middle of a school year, instead of being more spread out. 
• Test of Science Related Attitudes (TORSA) – see  final page for items and scoring information.

These surveys can provide teachers with data to evaluate their individual courses and the science program more generally.

While surveys of student outcomes are critical within a system of assessments, it’s also important to understand the views of parents/community members and teachers during the change process. Surveying parents and other community members can help ensure they’re aware of and meaningfully connecting to the school science vision and students’ science learning. Teacher surveys can ensure they’re comfortable teaching their content and the practices of science in accordance with the vision for science learning. Within results, you can look at trends by demographics, such as race and ethnicity, or differences between new and veteran teachers.

In a survey of parents and community members, you probably don’t want to get into the content being taught. Hearing about personal views of evolution and climate change isn’t necessary for these purposes. Questions could have a Likert-scale format, with selections from strongly agree to strongly disagree. Some examples include:

• Through the science courses, I believe my student is becoming a better scientific thinker (for the broader community that would be rephrased as “students are becoming”).  
• I am familiar with the district vision for science education. 
• I believe my student is receiving a quality foundation in his/her science classes to pursue science careers in the future. 
• I believe my student is being well-prepared for science classes at the college or university level.

There should also be an open-ended text box, asking survey takers to please share any comments or questions about the science education program at their school. Of course, even with community input, you’re not going to resort to poor instructional practice that isn’t research based, like lecture. Educators are the professionals in this setting. You may, however, decide to make more career linkages in your courses or bring in more guest scientists.

It’s also important to know where teachers are at in the change process. Are they getting the support they need in teaching science? Tools like the Survey of Enacted Curriculum (SEC) can also let educators and administrators know whether what they’re doing actually lines up with the intentions of the instructional program. It’s not a “gotcha” system, but an approach like Lesson Study that can lead to tremendous, collaborative professional learning.

A brief endnote… While it is true that for statistically-validated studies surveys need to undergo extensive testing, everyday school surveys can provide a useful piece of information for guiding instructional programs. Surveys linked above have largely undergone testing and include multiple item constructs, so using them or learning from them is a good step.

Some other tips for creating quality surveys include: 

• Use multiple questions to measure each idea or topic. Looking at several questions together provides a more valid picture of what people really think.  
• Have a student, parent, etc. verbally talk through their thoughts on the survey with you. They think aloud as they read and answer the questions. Are they understanding the questions in the way that was intended? Is there some confusing wording? Having people of different backgrounds do so helps ensure the questions are similarly interpreted by people. 
• A focus group, with a neutral facilitator (i.e., not your boss), can provide a different perspective and bring out ideas that a survey cannot. It can also inform survey development.  
• Pilot the survey before sending it out broadly. 
• Here are a few further tips for online surveys.

And, yes, it takes extra time and effort to know whether you’re actually making a long-term difference for your students and whether the large-scale changes improve classroom practice, but it’s worth it. 

Science Program Evaluation and a System of Science Assessment

Assuming your district/school has established a vision for science education and large-scale, specific goals aligned to that vision, you will next need to determine a system of assessments for evaluating progress toward those goals. As mentioned in my last post, many districts are working to adopt and implement new science standards. Strategically assessing science-related outcomes at multiple levels will provide ongoing evidence of effective change – after all, why make changes if you don’t know whether they actually make any difference?

While it might be obvious, an evaluation of a science program based on these goals will take more than one assessment! In other words, the annual state standardized test, often the only systematic science test used by a school, will not measure the full range of outcomes related to a meaningful vision for science education. That requires leaders to strategically implement a system of assessments. The Wisconsin DPI has a chart that illustrates some components of such a system, including formative, interim, and summative elements.

The majority of assessment will happen formatively at the classroom level. This level is where teachers see the day-to-day use of scientific practices by their students as they investigate, communicate, and ask questions about science. It will be critical for teachers to have the structures to discuss what they’re observing from their students, collaboratively determining next steps. Processes of informal formative assessment should drive instructional practice. If schools are moving toward the NGSS or NRC Science Framework, formative, as well as all levels of assessment, should be three-dimensional.

 Common, interim assessments and rubrics across classrooms and grade-levels can support collaborative understanding of students’ abilities. These types of assessments can provide a more formal view into student growth in relation to science content knowledge and practice. Quality performance tasks can potentially provide the clearest information for collaborative groups of teachers to reflect on progress toward their goals. They need to be implemented well, however, in order to be useful. Teachers must have the time to score papers together and come to an agreement on how particular examples of student work meet the rubric criteria.

Large-scale district summative tests (or state level tests) often afford the least amount of data for specific instructional guidance. They might, however, suggest areas for professional development or foci for revised student project rubrics. For example, a set of district end-of-course exams might all show that students across the district struggle with using data effectively. Often these types of tests are multiple-choice, which provide limited information in relation to authentic science practice, but they can be effectively paired with open-ended opportunities for students to describe their reasoning.

An often forgotten element in such an assessment system is an evaluation of student attitudes about science and their general scientific literacy. Do they see how science relates to their lives? Can they make sense of scientific evidence within popular media? Is science meaningful for them?

In summary, schools and districts reviewing and attempting to improve their science programs will have unclear success in that process if they haven’t defined what outcomes they want and how to measure them. A meaningful and strategic system of science assessment will be an essential part of this process. 

The next series of blog posts will discuss formative, interim, and summative assessments in more depth, as well as effective surveys of student attitudes. Each will provide examples of these assessment types and suggestions for classroom or school use.