As one of my professional goals this year, I decided to use protocols with my teaching teams to help us analyse student data to improve instruction. So far I have only had the opportunity to use it with one teaching team and we employed it to look at our student’s writing data.
Examining assessment data can be scary. We worry whether our students won’t measure up, or whether our teaching will be reduced to a set of numbers. This is precisely where using a protocol can help. A protocol is a procedure used to structure professional dialogue. They determine who speaks, for how long, and about what. They are norm driven, making it easier to discuss some of the more challenging aspects of teaching, such as assessment data. “Because feedback must follow the ground rules of the protocol, teachers are insulated from attack or undue criticism. Instead, teachers get honest but respectful feedback about their work or the work of their students” (Venables, 2015).
Protocols also insist we dig deeper, which provides more effective feedback to our colleagues. Using a protocol we cannot get away with saying that assessment data ‘looks good’ or ‘is fine’. The protocols push us to work past the polite and into the productive.
There are many protocols you can use with teachers to examine student data. I used the following criteria to select one to use with my teams. I wanted a protocol that:
- Was easy enough to implement with limited assessment capabilities,
- Did not require extensive training to use the protocol,
- Could be completed within our planning time (80 minutes),
- Could be used to look at formative and summative task data,
- Involved making instructional decisions as a result of the analysis, and
- Would highlight issues of equity. (I wanted to be able to look for trends that would indicate whether all students had an equal chance of being successful.)
As a result, I chose the Student Work Analysis Protocol.
I used the protocol with one of my upper primary teaching teams, whose membership is three classroom teachers, a learning support teacher who works with this team, a teacher trainee, and me, the PYP coordinator. As for our data source, we had students write an initial diary entry. Teachers prepared the students by reading their class one or two samples of diary entries aloud (Diary of a Worm, Diary of a Fly, etc… by Doreen Cronin) and then had the students help them highlight important features of diary writing (first person, time connectives, simple past tense, describes an event or day). Each student had 15 minutes to plan and 40 minutes to write a diary entry. Teachers tracked the names of any students that required extra assistance and what type of assistance was given. For example, one of our students wrote his diary with one on one support from the learning support teacher, who scribed for him.
Once we had all our samples, we met to use the student work analysis protocol. Our goal was to identify what aspects of writing our students needed to improve as a cohort and identify instructional strategies that would help us teach these needs. We also wanted to determine what feedback we needed to give individual students about their writing.
One of the biggest benefits of using the protocol was how it focused our attention on what we had evidence of as opposed to what we felt about our students’ writing. Boudett & City (2013) stated that this was one of the important learnings from their many iterations of the Data Wise Project. Without a relentless focus on what the evidence actually shows, teachers were prone to “leap to conclusions” (2). For example, one of our team members was concerned about student spelling, yet when we really looked at the samples, we found that most students were spelling the vast majority of words correctly. We were then able to identify who really was struggling with spelling and what types of words they were struggling with, and that teacher has taken this information away to consider how to address this issue in her class. Ideally we would have done this in the meeting or at another meeting, but time constraints made that impossible this round.
We all enjoyed that the protocol started with identifying the things our students were doing well. This helped make the process feel more manageable. We didn’t feel pressure to fix every aspect of their writing at once. The other part we found useful was reviewing the learning outcomes to determine which ones were integral to the unit and how we could collect additional information about student’s understanding of these outcomes during the writing unit.
The most challenging part of the protocol was dividing the students’ work into categories, as the line between consolidating and developing was initially quite grey. We had to keep going back to our scope and sequence documents to clarify which elements of the work were expected. By the time we finished the process, we felt clearer about the distinctions, but there were still a couple of pieces we struggled to classify.
After analysing our students’ work, we identified three main items to focus on in their writing and then collected ideas of how to do this. We also found strong samples that could be used as exemplars to help all students understand what we are looking for in their writing. We created a google doc of instructional strategies which allowed teachers to choose ones that they hadn’t already used as well as improve their knowledge of different strategies for teaching these outcomes. We redesigned our rubric to ensure it met the learning outcomes we had identified and teachers shared the rubric with their students to guide their work for the remainder of the unit.
The protocol allowed us to focus only on the work of the student and not on judgements about what had or hadn’t been taught in the past. It helped us use our time effectively and we were able to accomplish 95% of the work within the 80 minutes.
One of the implementation challenges, according to the teachers on the team, was to maintain focus on only these three writing features and not get distracted by other things the students are not doing, such as grammatical or punctuation errors. However, we all agreed that to help our students improve, we had to be clear about our goals and to focus on them. We will have the opportunity to examine the other issues in the next iteration.
All in all, the student work analysis protocol was a positive and practical initial foray into using data with teaching teams. We are a long way from using student data on a routine basis to inform our teaching, but the experience highlighted how useful looking at student data is for planning instruction. I am curious as to how this protocol will work with our early years teachers as their assessment data looks very different; perhaps I need to consider a different protocol or approach for their work.
My next post: A collection of data questions and concerns
Boudett, K, & Clay, E. (2013). Lessons from the Data Wise Project: Three habits of mind for building a collaborative culture. Harvard Education Letter, 29(3). Retrieved from http://hepg.org/hel-home/issues/29_3/helarticle/lessons-from-the-data-wise-project_567#home.
Learning Forward Ontario (2014). The Power of Protocols. Retrieved from https://www.uen.org/literacyresources/downloads/Learning_Forward_Power_of_Protocols.pdf
Rhode Island Department of Education & National Center for the Improvement of Educational Assessment (n.d.). Student Work Analysis Protocol. Retrieved from http://www.ride.ri.gov/Portals/0/Uploads/Documents/Teachers-and-Administrators-Excellent-Educators/Educator-Evaluation/Online-Modules/Student_Work_Analysis_Protocol.pdf
Venables, D. (2015). The Case for Protocols. Educational Leadership, 72(7). Retrieved from http://www.ascd.org/publications/educational-leadership/apr15/vol72/num07/The-Case-for-Protocols.aspx