A number of times, after sitting through a moderation session with a group of teachers, I’ve asked myself, “Did our rubric really measure what our students are capable of doing? Did we interpret the data correctly? Are there factors at play that we missed?” These questions prevented me from feeling fully confident in our findings.
It turns out that what I’m missing is called ‘assessment literacy’ and it is a common problem faced by educators. According to Brookhart (2011) and Popham (2018), teacher trainees, particularly those trained in Anglophone countries (DeLuca & Johnson, 2017), do not receive enough instruction in assessment and rarely is this gap filled intentionally through professional learning in schools and districts.
In the assessment section of PYP: From Principles into Practice (2018), the International Baccalaureate describes the assessment capable teacher in the graphic below. It is now an expectation of PYP schools that we ensure all of our teachers are assessment capable, also referred to as being assessment literate.
What then does it mean to be assessment capable/literate?
Here are some definitions that I found useful in helping me identify key concepts:
“Assessment literacy is the possession of knowledge about the basic principles of sound assessment practice, including its terminology, the development and use of assessment methodologies and techniques, and familiarity with standards of quality in assessment.”New Zealand Ministry of Education, n.d.
“Assessment literacy consists of an individual’s understanding of the fundamental assessment concepts and procedures deemed likely to influence educational decisions.”Popham, 2018, 4
“One becomes assessment literate by mastering basic principles of sound assessment practice, coming to believe strongly in their consistent, high-quality application in order to meet the diverse needs of all students, and acting assertively based on those values.”National Task Force on Assessment Education, 2017
Thus to be assessment literate, teachers must:
- have knowledge of basic assessment principles,
- understand specific assessment terminology, and
- know how to apply their understanding consistently.
I then went on to compare the PYP definition with definitions and explanations from four other sources:
- Assessment Literacy for Educators in a Hurry by W. James Popham
- New Zealand Ministry of Education’s Assessment Literacy website
- National Task Force on Assessment Education
- Susan Brookhart’s article Educational Assessment Knowledge and Skills for Teachers
Here are the commonalities:
- Purpose of assessment: Assessment literate teachers are clear about who needs the assessment data (student, teacher, school leader, district, etc…), what they intend to use the data for (what question they want answered), and from there determine what data needs to be collected. This answers the question, “why assess?”
- Select, design and use/administer appropriate tools and strategies: Assessment literate teachers know how to select an assessment strategy that matches the learning outcome being taught. They know how to choose or design an assessment tool which will allow them to measure the data collected. They are also aware of the cognitive, affective, and psychomotor demands the assessment places on the student and use this information to help determine its appropriateness.
- Give feedback to improve student learning: Assessment literate teachers are able to analyse the data they have collected about the student’s learning and use it to provide feedback to the student to answer the three big questions of learning: “Where am I going? (What are the goals?), How am I going? (What progress is being made toward the goal?), and Where to next? (What activities need to be undertaken to make better progress?)” (Hattie & Timperley, 2007, 86). The PYP calls this giving feedback to feedforward.
- Help students development assessment literacy: Assessment literate teachers know how to involve students in developing, using and analysing assessment data to increase motivation, to help students understand how to improve their learning, and to develop an anatomy of quality. The New Zealand Ministry of Education insists that students should always be included in assessment, from decisions on what, when, and how they should be assessed to how the data should be used. Professor Sally Brown from Anglia Ruskin University (2016) says that since students will be the ones who will learn from assessment, it is necessary that they “understand what the whole process is about” (02:15). An assessment literate student would know what good quality work looked like and when they had achieved it.
- Reflect on data to improve teaching practices: Assessment literate teachers know how to use the assessment data they have gathered to make instructional decisions, such as determining which strategies were effective, which concepts or content need reteaching, which students need a differentiated task and how to differentiate it for them.
- Standards of assessment quality: Assessment literate teachers understand the meaning of key assessment quality concepts, such as validity, reliability, and fairness, and are aware of the complexity of assessment. I will delve deeper into this topic in a future post as this is essential to being assessment literate and has a significant impact on teacher’s assessment confidence.
- Communicate findings meaningfully to stakeholders: Finally, assessment literate teachers are able to explain the inferences they’ve drawn from assessment data competently to the different stakeholders who need it. For example, they can support the student in understanding what the results on a test mean and how to improve, and they can explain to a parent the meaning of standardised assessment results and what it means about their child’s progress.
There were a few notable differences between the sources that are worth consideration:
- Motivation: Assessment literate teachers understand the relationship between motivation and assessment. The National Task Force (2017) states, “They [assessment literate people] understand that, for teacher and student alike, fear, vulnerability, and anxiety are the enemies of learning success, while a sense of self-efficacy, confidence, and accomplishment promote ongoing academic success.” The impact of assessment data on student self-efficacy and motivation will be discussed in an upcoming post, and is essential to helping students develop assessment literacy, but the idea of how assessment data impacts on teachers is also an important consideration. Brown (2016) from Anglia Ruskin University says teacher confidence with assessment is a major obstacle and one of the reasons we must address assessment literacy (02:58). The impact of data on teacher confidence and motivation is a major reason why our school leadership has been hesitant to share and analyse standardised assessment data with our staff. Some teachers have consistently better results than others, but we are unable to explain why this is – the data does not allow for that level of analysis. However, without appropriate levels of assessment literacy, it is too easy for people to interpret that it means one teacher is better than the other.
- Statistical knowledge: According to the New Zealand Ministry of Education (n.d.), assessment literate teachers “understand and can explain all relevant statistical terms; for example, norm, mean, standard deviation, and stanine.” Just seeing this raised my anxiety, so I will consider devoting a future post to this skill as I suspect they are essential to being able to understand standardised data and perhaps other sources of quantitative data that can tell us something useful. In the meantime, feel free to check out Khan Academy!
- Understand how aggregated data is used at different levels: Again, the New Zealand ministry thinks assessment literate teachers should understand how the collection of data we have about students and learning is used by different groups – classroom teachers, school leadership, district leadership, etc… This is an important consideration because school data is still used to determine important outcomes for students, such as certification, placement, and promotion. It should also help teachers understand why new assessment initiatives are proposed and implemented and what it will mean for them, their students, and their practice.
- Legal and ethical responsibilities: Assessment literate teachers understand their legal and ethical responsibilities associated with using assessment. Brookhart (2011) states, “Areas of understanding include, but are not limited to, test preparation, confidentiality of information, opportunity to learn, and due process” (10). This is an important consideration because, as indicated above, assessment does have a significant impact on a student’s opportunities for success and their view of themselves as successful learners. We also need to ensure that our assessments are fair and give equal opportunity for success. We must be able to determine if an assessment is biased for language, gender, culture, intelligence, etc… and either select a different tool or redesign the assessment tool or process to eliminate these biases.
- Content knowledge: Finally, assessment literate teachers have a deep understanding of the subject content they teach as well as the disciplinary pedagogical approaches needed for students to learn it. Brookhart (2011) states, “In order to be able to assess students well and to make sound decisions based on the results, teachers must understand general principles about how students learn, and they must understand deeply the content area(s) they teach” (6). Without this knowledge, teachers are unable to design appropriate assessment tasks, nor can they interpret them to provide useful feedback that will guide the student in their next steps.
So there we have it, the 12 essential key competencies of assessment literacy.
What this means for me:
I am going to use the Compass Points Visible Thinking Routine to summarise and clarify what I think this will mean for me personally and for my school.
E = Excited
What excites you about this idea or proposition? What’s the upside?
I am much clearer about what it means to be assessment literate. I think if all of our staff was assessment literate it could have the power to transform learning at our school, particularly by involving our students more fully in the assessment process and using assessment to help them develop learner agency. Teachers would have more confidence in dealing with assessment and would be able to use it to make instructional decisions that resulted in better learning for their students.
Personally, I think if I am able to develop all the skills and knowledge identified here I will be able to make use of the collection of data we have as a school. It will help inform our decision making as a leadership team as we will be working from a position of knowing, as opposed to a position of best guessing.
I am also looking forward to learning more about how to design effective assessments, as this is something I enjoy and want to improve.
W = Worrisome
What do you find worrisome about this idea or proposition? What’s the downside?
The how is my concern. How do we help teachers develop this literacy? We are a relatively small school on an isolated island. I cannot currently think of anyone who can come and support our teachers with this endeavour, yet it is essential that we all become assessment literate.
We will also need to consider how we will help teachers want to become assessment literate. It is one thing to develop a learning plan, but we will first have to address teachers beliefs about assessment and help them overcome any resistance, anxiety, and/or lack of confidence.
N = Need to Know
What else do you need to know or find out about this idea or proposition? What additional information would help you to evaluate things?
I still need to understand what Popham (2018) calls the measurement notions of assessment – validity, reliability and fairness and how these concepts impact on assessment design and interpretation. I also need to develop the statistical knowledge needed to analyse and understand standardised assessment data reports.
I would like to find out how other schools have helped their teachers develop assessment literacy and which strategies were effective and which were not. This could help us design a learning plan with our staff.
S = Stance or Suggestion for Moving Forward
What is your current stance or opinion on the idea or proposition? How might you move forward in your evaluation of this idea or proposition?
I agree, in part, with Popham’s (2018) contention that “the single most cost-effective way to improve our nation’s schools is to increase educators’ assessment literacy” (1). Certainly the potential impact of formative assessment on improving student learning and developing learner agency is well researched and reflected in the PYP’s updated documentation. Assessment is a core competency in teaching and one that research (and personal experience) claims we have not helped teachers acquire fully.
However I still think there are a number of potential pitfalls that we need to consider, such as how this focus on assessment will impact on students, our school’s culture, our values, our pedagogy, and staff morale. I will try to address these pitfalls in future posts.
My next post: The Assessment Capable Teacher, Part 2
Brown, S. & Price, M. [Anglia Ruskin University]. (2016, June 17). What is Assessment Literacy? [Video file]. Retrieved from https://myplayer.anglia.ac.uk/Play/4823.
Brookhart, S. (2011). Educational Assessment Knowledge and Skills for Teachers. Educational Measurement: Issues and Practice. 30(1), 3-12.
International Baccalaureate Organisation (2018). Assessment in the Primary Years. In PYP: From Principles into Practice. Retrieved from: https://resources.ibo.org//data/pyp_11162-51465-en_id-e433028b-b1a6-4020-bfe3-855927995871.pdf.
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112.
National Task Force on Assessment Education (2017). Assessment Literacy Defined. [PDF]. Retrieved from https://www.assessmenteducation.org/wp-content/uploads/Assessment-Literacy-Definition.2017.01.25.pdf
New Zealand Ministry of Education (n.d.). Assessment Literacy. [Website]. Retrieved from http://assessment.tki.org.nz/Assessment-for-learning/Assessment-for-learning-in-practice/Assessment-literacy
Popham, W.J. (2018). Assessment Literacy for Educators in a Hurry. Alexandria, VA: ASCD.