Conducting a Science Program Audit

With a mission and vision in place for the science program, teachers will need to personally and collaboratively decide whether or not they have a sufficient understanding of what instructional practice looks like under that vision. In future blog posts, I’ll be providing suggestions for professional development on research-based science instruction. For this post, I’ll be assuming (always a risky plan!) that teachers have the skills and understanding necessary to evaluate their work in relation to their vision.

Therefore, to determine progress towards a vision, a school or district will first need to determine where they’re at now. A science program “audit” is a strategy to do that. It asks, “How well does our instruction align with this mission? How well are we accomplishing our vision? How do we know?”

Forming a Committee

Again having an audit committee will be a valuable guide for this process. While there could be some overlap with the leadership committee that crafted the mission and vision statements, the auditors should consist of outsiders, not district employees. A school will need constructive, impartial outsiders to give an unbiased perspective. Some suggestions for members of this committee:

1. A recent graduate of the district currently studying science in college
2. Community members working in science-related fields (healthcare, high tech, university research, etc.)
3. Science education professors or coordinators (college, university, teacher professional organizations)
4. State education department or regional education service agency science education leaders
5. Parents
6. Educators from neighboring school districts (would be great to have districts across a region support each other in this audit process)

The size of the committee needed depends on the size of the district. The committee would ideally be able to visit every school in the district and interview a representative sample of teachers and students across the district that includes some from every school. One strategy would be to have two auditors work together, visiting one school in the morning and one in the afternoon. Thus, in a district with 40 schools, 20 auditors would then be needed to complete all of the visits and interviews in one day. Of course, fewer auditors would be needed if the process was spread over several days. More in-depth audits could be conducted by one or two auditors over the course of several months, though an audit can be a unique opportunity to engage a broad range of community members.

Data Gathering

Before the committee comes to the school, administrators and teachers should collect and share useful data and information about the school/district science program. This data should be broken down by all appropriate subgroups and could include:

• Assessment data: standardized test scores, district or grade level common assessments, classroom level assessment examples;
• Descriptions of courses taught and typical student pathways: for example, it will be important to note if students from particular backgrounds tend to be in “honors” classes to a lesser extent;
• Teachers’ license and longevity information;
• Postsecondary pursuits: career or college/university pathways;
• Any documentation of science program mission/vision and links to science department or teacher websites.

Information gathered before the auditing day(s) should also include broad-scale surveys to gather quantitative data on the science program. These could easily be created and conducted through Google forms or another free survey tool. These surveys could alternatively be developed, conducted, and analyzed by the auditor(s).

Student Surveys Example Questions

• Elementary: at lower elementary the teacher would need to read and explain the questions. A simple yes/no might be most appropriate K-2. 

1. I enjoy science class. [A lot, kind of, not really]
2. We do interesting experiments and investigations in science [A lot, sometimes, not much] 
3. We learn about scientists in class that look like me [yes, no]
4. We study science outside [A lot, sometimes, not much]
5. I get to study my own questions in science class [yes, no]
6. I learn about things in science class that I wonder about in my life.

• Secondary:

1. I enjoy science class. [A lot, sometimes, not really]
2. We do interesting experiments and investigations in science [A lot, sometimes, not much]
3. We learn about scientists in class that look like me [yes, no]
4. We study science outside [A lot, sometimes, not much]
5. I get to study my own questions in science class [A lot, sometimes, not much]
6. I learn about things in science class that I wonder about in my life or the world around me [A lot, sometimes, not much]

Parent/Community Survey Example Questions

1. I am satisfied with the science education that my child (or our community’s children) is receiving [yes, somewhat yes, somewhat no, no]   Explain: [open-ended]
2. Students are involved in science learning relevant to needs and issues in our community [A lot, sometimes, not enough]
3. Students are learning relevant 21st century skills through science classes, such as analyzing scientific studies and evidence, communicating technical information, conducting investigations, and collaborating with peers [A lot, somewhat, not adequately]
4. Students are adequately prepared for postsecondary careers or educational pathways [A lot, somewhat, not adequately] 
5. Community members and experts are invited into science classrooms and to support science projects [A lot, sometimes, not enough]

Teacher Survey Example Questions

• Select how much you agree with the sentence.

1. I am comfortable teaching science. [Yes; somewhat yes; somewhat no; no]
2. I have sufficient collaboration time with other teachers focused on science. [Yes; somewhat yes; somewhat no; no]
3. I receive sufficient science-related professional development from my district/school. [Yes; somewhat yes; somewhat no; no]
4. Our current curricular materials effectively support science instruction. [Yes; somewhat yes; somewhat no; no]
5. I have adequate lab and related science equipment for effectively teaching science. [Yes; somewhat yes; somewhat no; no]

• The next set of questions asks about frequency of various actions.

1. I take advantage of outside professional development opportunities (not school or district sponsored). [At least monthly, five to ten times per year, two to four times per year, about once per year, once every few years, never]
2. Students have opportunities to investigate their own questions in my science class [At least monthly, five to ten times per year, two to four times per year, about once per year, never].
3. Our class goes outside or to other relevant sites in the community (or beyond) for science learning and investigation [At least monthly, five to ten times per year, two to four times per year, about once per year, never].

To better understand the “why” for the survey answers, auditors should also conduct interviews and focus groups of students and teachers based on these questions. This back-and-forth will allow for follow up questions and clarification. Focus groups will allow for greater synergy of respondents and will typically be more appropriate for work with students, where one-on-one interviews are not advisable for student safety and comfort reasons.

Observations

As part of the day(s) that the audit committee is in schools, they should be observing classrooms. Ideally, this observation will happen naturally in the flow of instruction, not just be scheduled for the best lab of the year. The auditors need an authentic perspective on what is happening in the day-to-day work of the classroom. Notably, auditors without an education background will need further guidance on what they’ll be looking for in classrooms. In addition to general, objective observations of what happens in the class, some specific prompts/questions that auditors could be answering include:

1. Describe the classroom. How are desks arranged? What materials and space are available? 
2. What happens during the class? What is the duration of each segment of the class? [lecture, independent work, collaborative work, and whole-class discussion]
3. What types of questions is the teacher asking the students? What questions are students asking of each other and of the teacher? [Yes/no, clarification, one-right answer, logistical, deeper though required, etc.] 
4. What type of work are students doing? [step-by-step lab, worksheet w/ one right answer, open-ended problems, peer discussion, arguing with evidence, modeling or investigating a phenomena, 
5. How does the teacher establish whether or not the students understand the material? 
6. How does the teacher ensure all students are equitably engaging in the instruction?

Bringing It All Together

At the beginning of the day (week, month, or year) the audit committee would come together with administrators and science teacher leaders to establish their roles and the plan of attack. They would receive guidance on what’s expected of them and a schedule, maps, and other needed support in accomplishing those tasks. Ideally, they would have tasty snacks (I prefer cake donuts and seasonal fruit, in case you’re wondering) and lunch available during the day. At the end of the allotted timeframe, the auditors will come together to discuss the data and observations as a group and collaboratively review the day. They should have feedback templates where they can create and share consensus comments of the group and their individual feedback on the day, though individual feedback could come back a few days/weeks later after auditors have had a chance to fully analyze the data provided from the school, their observations, and the observations of their fellow auditors. Templates would include the specific questions that the school/district wants feedback on from the auditors, as well as space for open reflections on the wide-ranging pieces of data reviewed.

After conferring as a group, the auditors should present their initial findings to teachers and administrators, with the opportunity for auditors and district educators to ask clarifying questions of each other. This discussion will need an able facilitator, as it is a time for teachers to reflect on the observations shared by the auditors, not get defensive about perceived negative feedback.

Once final feedback is received by individual auditors, either a primary auditor or a school/ district leader should combine the data and feedback into a final report. Several science department and leadership committee meetings will need to be devoted to reviewing the feedback and determining the next steps in moving toward the vision and mission for science education. 

*Special thanks to Judy Singletary of Stoughton School District for ideas on this audit process.