Developing a programme evaluation

Background: In 2015, our school purchased a new mathematics programme, called Oxford International Math for implementation in the 2015-16 academic year. Teachers quickly complained about the sequencing in the programme. It seemed to jump too quickly between difficult concepts without allowing time for concept building, consolidation and practice. It was workbook based as well, so it was challenging to differentiate for the needs of all our students. The process for selecting this resource remains unclear and was purchased prior to the new administrative team arriving.

Part way through that academic year, it was collectively decided that there was a need for a new programme. We wanted a program that supported all of our learners and matched our philosophy of how children learn. It needed to:

  • Help us achieve our school mission and vision
    • Inspiring our learners to thrive in a world of change (vision)
    • Inspire, empower, lead (mission)
      • Inspire our learning community through an inquiry curriculum and commitment to on-going improvement.
      • Empower our learning community to find solutions to real-life challenges through creative and critical thinking.
      • Lead our learning community to promote a spirit of internationalism by thinking globally and acting locally.
  • Match our philosophy of how students learn math (constructivist and inquiry-based)
  • Help students develop robust mental models of mathematical concepts and procedures,
  • Provide open-ended questions and opportunities for students to inquire about mathematical concepts and procedures,
  • Allow students to apply their learning to real-life problems,
  • Support students who needed additional accommodation, due to EAL or SEN issues (remediation and extension),
  • Provide sufficient support for teachers implementing the programme (day-to-day planning, long term planning and mathematical knowledge support)
  • Support children in developing methods for recording their own thinking (not workbook based)
  • Align with our school mathematics scope and sequence which is based on the Primary Years Programme Mathematics scope and sequence (conceptual) and the British National Curriculum (benchmarks)
  • Be appropriate to use with our youngest and oldest students (3 year olds and 12 year olds)
  • Provide a consistent approach to mathematics teaching, while still allowing for the magic of the teacher to emerge
  • Improve articulation between and within year levels
  • Reassure parents that math is being taught rigorously

After a review of a few programmes, it was decided by the Head of School that the Numicon programme met all of these criteria. It was originally designed to help SEN students understand mathematical concepts and relationships using a variety of manipulatives and learning engagements. As such, it was multi-sensory and provided resources for all year levels in our school. We reviewed documentation evaluating its effectiveness in early years classrooms as well as case studies at a variety of primary schools in the UK. It also provided teachers with support for planning, instruction, assessment, and knowledge building.

numicon_400x400

Before implementing the programme, we had a Numicon trainer come from the UK to spend two days with teachers explaining the components of the programme and how the materials are used as these were new to all of our teachers.

In October, all teachers involved in teaching the programme met in mixed groups and shared their successes and frustrations, which we documented. We also looked at examples of lessons from each year level to see how the number concepts build upon each other.

The programme has been in place for 6 months now. We have our first set of standardised data from the upper primary, which indicates that overall students are doing well in number, but not well in problem solving. Additionally, there are significant inconsistencies between classes within the same grade level.  I also have some anecdotal data from teachers that leads me to think there are some issues with the implementation of the programme.

Evaluation

Purpose of the evaluation: The purpose of this evaluation is to provide the leadership team with feedback to determine how to support teachers in implementing the programme.

Intended users of evaluation findings: We anticipate using the findings from the evaluation by the following stakeholders:

  1. The school administration: 
    1. provide an understanding of how well implementation is going so the school can determine how to improve implementation,
    2. provide an understanding of the effectiveness of the programme, and
    3. potentially request more funding if needed to improve implementation.
  2. The teaching staff: help teachers ascertain how effectively they are implementing the programme and how they could improve, as well as what they should continue to do.
  3. The school board: to justify the investment.

Evaluation questions: There are three significant questions that we need answered:

  1. How are teacher perceptions about the programme influencing how they implement it?
  2. What barriers to implementation are teachers facing?
  3. Are students achieving better on this programme than on the past programmes, and if so, how much better?

Teacher perceptions of the programme may impact how they implement the programme. As stated by Turner et al, 2009, “teachers’ beliefs about what is successful teaching and learning may guide practices that focus primarily on implementing lessons” (362). For example, if a teacher does not believe that the Numicon shapes can help students develop mental models and help teachers identify student misconceptions, then s/he may choose not to use them.

Additionally, if a teacher does not understand the mathematical concepts underlying a lesson well enough, s/he may choose not to teach it or teach it incorrectly. A lack of subject knowledge would prove to be a barrier for implementation.

Finally, if our students are not achieving better using this programme than they did on our previous programme, we will need to identify why not.

Size of staff involved in implementing the programme:

  • Classroom teachers: 19
  • Assistant teachers: 2
  • Class assistants: 8
  • Special Education Needs (SEN) teachers: 3.5
  • English as an Additional Language (EAL) teacher: 1
  • Curriculum coordinator: 1

Resources expended and available (time, people, money):

  • Teaching and assessment guides for each classroom teacher
  • Complete box of manipulatives to support the implementation of the programme (enough for each child, plus a few extras)
  • Time to discuss math teaching in year groups during weekly planning meetings
  • Time to reflect on implementation and vertical review as a staff (those teaching the programme)
  • A classroom teacher for each group of 14 to 24 students
  • An assistant teacher/classroom assistant in 12 of the classrooms
  • 3.5 SEN teachers who provide remediation and learning support to struggling students
  • 1 EAL teacher who helps teachers plan lessons that support EAL learners and supports some EAL students through language acquisition lessons.

Community demographics:

  • Private, independent, non-profit, co-educational, primary day school (ages 3-12) of 350 students
  • The school was founded in 1989.
  • The school is run by an elected community board.
  • The pedagogical leadership team comprises a Head of School, a Deputy Head of School, and a curriculum coordinator. They have the authority to make decisions on all academic issues.

Here are some charts showing additional data which may impact on implementation:

screen-shot-2017-02-04-at-14-20-35

We recognise that a student’s command of the language of instruction, in our case English, can have a profound impact on their ability to achieve concept attainment, access the lessons, and express their understanding in mathematics, despite its universality as a language.

We also acknowledge that our teacher’s educational backgrounds may impact on their ability to understand the mathematical jargon used within the teacher’s resource guides, as it is written for a largely UK audience. Here is a breakdown of our teacher’s nationalities.

screen-shot-2017-02-04-at-14-19-45


References:

Turner, Julianne, Debra K. Meyer, Andrea Christensen, and A.G. Dworkin. “Teachers’ Beliefs about Student Learning and Motivation.” Ed. L.J. Saha. International Handbook of Research on Teachers and Teaching. N.p.: Springer Science Business Media, 2009. 361-72. ResearchGate. Web. 10 Mar. 2017. <https://www.researchgate.net/publication/226604918_Teachers’_Beliefs_about_Student_Learning_and_Motivation&gt;.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s