A collection of data questions and concerns

I feel I have made a great deal of progress in understanding how to use data with our teaching teams to improve learning for our students. At this point, I would like to explore some of the questions and concerns that I previously raised, determine what new questions I have, and clarify what potential barriers we still have to overcome.

Data use and teachers:

On the positive side, I think once teachers have developed at least rudimentary assessment capabilities, they will be able to ‘see’ their students’ learning more clearly and gain more knowledge about their teaching practice. I suspect that this will be motivating for many teachers as they will be able to identify the direct connection between their efforts and their students’ success. Over time, I am hopeful that they will also be able to include their students more meaningfully in the assessment process.

Datnow & Hubbard’s (2015) statement, “it is essential that teachers receive professional development on assessment and how to translate assessment data into information that can inform instructional planning” was a consistent recommendation in the literature I read (Kanjee & Molio, 2014; Ebbeler, Poortman, Schildkamp & Pieters, 2017; Kippers, Wolterinck, Schildkamp, Poortman & Visscher, 2018; Marsh, 2012; Hoover & Abrams, 2013; Datnow & Hubbard, 2015). For many teachers, using assessment data will require them to learn new terminology, new concepts, and new skills. For some, it will also mean examining their beliefs about assessment and its importance in daily teaching. As previously highlighted, assessment is an area of low confidence for many teachers and we have to understand how this is going to influence any plans that are made to help teachers develop this capability.

Additionally, as a school in its PYP infancy, having just achieved verification and working towards our first authorisation, how do we manage all the competing priorities without overwhelming our teachers?

How do we help teachers want to become assessment literate, what strategies have been tried by others, and which were most successful?

Data use and students:

A number of the assessment literacy capabilities made connections to helping students develop assessment literacy and being sensitive to the impact assessment has on student motivation. For all of my PYP teaching years, assessment has been an embedded practice. We used the students’ responses to daily discussions, their questions, their work products, their self-assessments, etc… to help inform us about their progress. Rarely however was this evidence shared with the student directly.

Given the potentially negative impact assessment data can have on students’ motivation and self-efficacy, how do we use it to help them improve their learning and develop greater agency? How do we help students develop assessment literacy? Which key competencies do they need to develop?

Data use and school leadership:

Very little of what I’ve read so far talks about the role of school leadership in assessment literacy, other than the recommendation that they will need to have a more sophisticated understanding of the key concepts and support the development of assessment capabilities. It seems to me that for a school to be successful in developing an assessment literate staff, there will need to be some significant investments from leadership and perhaps even a reshaping of school culture, school policies, and school practices.

What is the role of school leadership in developing an assessment capable staff and school culture?

Data use and pedagogy:

In a few of the studies that I’ve read which explored teacher’s use of assessment data (Kanjee & Molio, 2014; Herman, Wardrip, Hall & Chimino, 2012; Volante, 2004) particularly standardised or testing data, a common effect was that teachers started teaching to the test. For example, Kanjee and Molio (2014) found that of the South African teachers who were able to make sense of the Annual National Assessments data, access to the tests resulted in those teachers using the test format and questions to guide what they taught the subsequent year (103). In Herman et al, teachers gave students weekly 40 minute multiple choice math and reading tests on Monday that mimicked the questions on the annual assessment, and then used this data to inform teaching for the subsequent week. The students’ grades improved significantly, but at what cost? What underlying message have these students received about the value of learning?

Teaching to the test is not valued by PYP schools. We believe learning should be guided by the needs and interests of the student, not the demands of an external test. We want to raise self-regulating lifelong learners who use a variety of strategies to learn about their world. We want them to explore significant concepts and ideas that cannot be easily summed up in multiple-choice questions. We also want them to spend most of their time in the learning zone, not the performance zone.


Assessment absolutely needs to hold a central role in PYP schools, but how do we ensure this focus on assessment does not result in changes to our pedagogical beliefs and practices?

In short, I think we need to look beyond the use of test data to help us improve teaching and learning. PYP: From Principles into Practice (2018) states:

  • Both learning outcomes and the learning process are assessed, and
  • Assessment design is both backward and forward looking (1).

The PYP Standards and Practices (2018) states, “The school uses assessment methods that are varied and fit-for-purpose for the curriculum and stated learning outcomes and objectives” (8). This means we need to design a variety of assessment tasks that will inform teachers and students about the process students are using to learn, as well as how successfully they’ve learned the outcomes. Additionally, our assessments must be designed to assess not only the intended learning outcomes, but also unintended ones, and “soft” skills such as attitudes, the learner profile, and self-regulation.

What assessment strategies are most effective for providing teachers and students information about the learning process, learning outcomes, and soft skills? Do our current assessment practices meet these requirements? What role do reliability and validity play in the design of these assessment tasks? And what do we do with our standardised assessment data?

Data use and school culture:

One message that came across clearly in the literature I reviewed is the need to develop a school culture that values assessment. Kippers, Wolterinck, Schildkamp, Poortman, and Visscher (2018) theorised that for assessment data to be used effectively in the classroom “presupposes a positive attitude towards AfL (assessment for learning) and DBDM (data-based decision making), and a school culture in which assessment is seen as a way to improve student learning” (209).

The PYP: From Principles into Practices (2018) identified these practices as necessary to developing a culture of assessment:

– developing assessment capability within the learning community,

– developing a comprehensive assessment policy that emphasizes assessment integration,

– creating opportunities for teachers to plan, reflect and moderate assessment collaboratively,

– providing school-wide professional development opportunities around integrating effective assessment,

– reinforcing the role assessment plays in finding out what students know and can do, and in identifying the next steps for their learning, and

– reinforcing the links between monitoring, documenting, measuring and reporting of learning (5).

As a school, we already have some of these practices in place, although they could all use further development. For example, our school assessment policy highlights the relationship between assessment, student motivation and self-efficacy, as well as the role of the student as an active participant in the assessment process. It also states,

“the primary purpose of assessment is to provide students with feedback on the learning process so that they can improve. It is used by students and their teachers to decide where the students are at in their learning, where they need to go, and how best to get there. This data then informs the teacher’s planning so as to best support the needs of the learner. As the student engages in the next round of learning, the process is repeated, leading to on-going student growth” (6).

Our School Assessment Policy, August 2017

What I’d like to explore further are ways of supporting teachers to develop assessment capabilities and engage in effective data use.

As is typical in an inquiry, I think I now have more questions that I did when I began, but some things are becoming clearer. Before we can use data effectively to improve learning for our students, we need to solve the following problems:

My next post: Using assessment data with teachers: what works?


References:

Briceno, E. [TEDxManhattenBeach]. (2016, November). How to get better at the things you care about. [Video File]. Retrieved from https://www.ted.com/talks/eduardo_briceno_how_to_get_better_at_the_things_you_care_about#t-1878.

Datnow, A., & Hubbard, L. (2015). Teachers’ Use of Assessment Data to Inform Instruction: Lessons from the Past and Prospects for the Future. Teachers College Record, 117, 1-48.

Ebbeler, J., Poortman, C., Schildkamp, K., & Pieters, J. (2017). The effects of data use intervention on educators’ satisfaction and data literacy. Education Assessment Evaluation Association, 29, 83-105. DOI 10.1007/s11092-016-9251-z.

Herman, P., Wardrip, P., Hall, A., & Chimino, A. (2012). Teachers Harness the Power of Assessment: Collaborative use of student data gauges performance and guides instruction. Journal of Staff Development, 33(4), 26-29.

Hoover, N., & Abrams, L., (2013). Teachers’ Instructional Use of Summative Assessment Data. Applied Measurement in Education, 26, 219-231.

International Baccalaureate Organisation (2018). Assessment in the Primary Years. In PYP: From Principles into Practice. Retrieved from https://resources.ibo.org//data/pyp_11162-51465-en_id-e433028b-b1a6-4020-bfe3-855927995871.pdf.

International Baccalaureate Organisation (2018). Programme Standards and Practices. Retrieved from https://resources.ibo.org/data/programme-standards-and-practices_f719525e-3b15-4fbd-96e5-e12719ad78b7/PRC-Programme_standards_and_practices-WEB_c4c2d5c9-8328-45a9-945e-da951600651f.pdf

Kanjee, A., & Molio, Q. (2014). South African teachers’ use of national assessment data. South African Journal of Childhood Education, 4(2), 90-113.

Kippers, W., Wolterinck, C., Schildkamp, K., Poortman, C., & Visscher, A. (2018). Teachers’ views on the use of assessment for learning and data-based decision making in classroom practice. Teaching and Teacher Education, 75, 199-213.

Marsh, J. (2012). Interventions Promoting Educators’ Use of Data: Research Insights and Gaps. Teachers College Record, 114, 1-48.

Volante, V. (2004). Teaching to the Test: What Every Educator and Policy Maker Should Know. Canadian Journal of Educational Administration and Policy, 35.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s