Teacher Evaluation ... Training the Referees

Imagine watching this football game:

The score is tied at 0-0, even though the game is half over.
You soon realize that the reason the score is still 0-0 is because every time the offense gets close to the end zone, the quarterback throws the ball through the uprights.  The referees award 0 points and give the ball to the opposing team, which promptly does the same thing at the other end of the field. This is repeated over and over, up and down the field - until one player finally kicks the ball through the uprights out of frustration, at which point the referees award his team 3 points.
Everyone wonders: "Why didn't the referees tell us that was how to score points?"

Why in the world would we only train the referees in the rules of a game and make the players figure it out for themselves?  Makes no sense at all, right?  But that is exactly what many have done with teacher evaluation systems in recent years.

The impact of NCLB and Race to the Top funding caused states to focus on educator effectiveness in a very systematic fashion.  Many states now mandate the use of a particular framework.  In order to establish validity and reliability, administrators went through extensive training on the chosen framework and needed to pass tests to ensure inter-rater reliability. But nothing mandated that the teachers become educated on the framework, and the "trickle-down" varies on a school-by-school basis.  In the push for teacher accountability and effectiveness, we continue to train the referees and expect the score of the "game" to increase.

And we're not talking about a game here - we're talking about the real lives of children and the future of a nation!

I advocate that every teacher be trained in the same way administrators are trained, in regard to the use and understanding of teacher frameworks.  Level the playing field so that the teams on the field - the teachers - have the greatest chance of success for students.  Make sure the referees, coaches, and players share a common language for how the game is played.   Make sure the referees, coaches, and players have a common vision of how to succeed together.   Make sure everyone know what expertise looks like in order to maximize impact.

Principals must take it upon themselves to delve into teacher frameworks with staff.  Create conditions for rich discussion about what those frameworks contain.  Create cognitive dissonance in the minds of teachers who recognize current practice vs. expertise.  Create a shared language to talk about high-quality instruction.  The various teacher frameworks (Danielson, Stronge, Marzano, etc.) are rich with possibilities, and Frontier & Mielke in their new book Making Teachers Better, Not Bitter say "We've seen tremendous professional growth among teachers who have used frameworks as the starting point for a comprehensive self-assessment process" and that frameworks "clarify how [teachers] should invest their efforts" (19).

So - stop exclusively training the referees.  We need the referees and coaches to provide direction and create conditions for success.  But at some point, the players need to know the rules if everyone is going to achieve success together.

More Than Just Evaluation

Your principal walks into the room while you are teaching.
Here are some of the possibilities that go through your head while you continue to teach:
• Does she need one of the students for something?
• Does she need me for something?
• Is this just a quick 1-minute pop-in visit?
• Is this one of those 20-minute long "informal observations" that is actually formal ... because there are going to be forms involved?
• What am I supposed to be teaching right now?

I recently finished Tony Frontier & Paul Mielke's new book Making Teachers Better, Not Bitter (ASCD, 2016).  The title speaks for itself: are you developing processes that grow teacher expertise, or are you just developing processes?

The authors contend that evaluation, supervision, and reflection are three sides of a triangle that build teacher expertise.  All three are necessary if growth is your expected outcome, but pull on one side of the triangle too hard, and the entire system gets out of balance.  This is what happens when we put all our eggs in the "evaluation" basket - we end up with bitterness instead of betterment.

Frontier & Mielke clearly define their three components and offer protocols that can be implemented to improve practice in each area.  Evaluation is what you'd expect it to be - valid and reliable rating of performance - which is what teachers hopefully do with student work at the end of a term.  The problem is that most teachers (and principals?) do not know when the administrator is functioning as an "evaluator" and when he/she is functioning in a different capacity.  Was that walk-through part of an evaluation or something else?  Was that quick hallway conversation about what was seen in your room an evaluatory or a coaching conversation?  Trust is built upon transparency.

System out of balance

Supervision means the way a supervisor "creates conditions for teachers to uncover and close the gap between their current performance and the next level of performance" (72).  The authors make an analogy between evaluation and supervision this way:
   • Evaluation is the state accountability assessment
   • Supervision is the curriculum that students engage in order to do well on the state assessment
What conditions does the principal create between evaluation cycles in order that growth might occur and expertise develop?  This takes trust, feedback, and deep knowledge of the chosen teaching framework (Danielson, Stronge, Marzano, etc.) by both the supervisor and the teacher.

Reflection seems self-explanatory, but the bigger question the authors offer is this: "When everyone reflects in a different way, how do you leverage reflection for school-wide improvement?" Reflection is about creating an internal dialog that drives your own improvement, and it is perhaps the biggest motivator of continuous improvement towards expertise.

The question we must ask ourselves as educators and leaders is this:
Are we trying to measure effectiveness or are we trying to improve effectiveness (160)?
There is a time and a place for measuring effectiveness - that's what evaluation seeks to do.
But if we are trying to improve effectiveness, we need the other two sides of the triangle - supervision and reflection - to create conditions that will drive true growth.

A New View of Growing Teacher Expertise

Recently, I finished Susan Brookhart's book How to Make Decisions with Different Kinds of Student Assessment Data (ASCD, 2016).  We have so much student data coming at us as educators - it is important to remember that not all data can be treated the same, and it is not all meant for the same people to use.  

Brookhart creates four quadrants by placing assessments on a spectrum of "formative vs. summative" and "large scale vs. classroom-focused" (yellow boxes below). But all this data is meant for different people, so I have added the red boxes to represent who are the prime users of each type of assessment data (see also Stiggins & Chappuis for more on this).

But as I was reading, I couldn't help but think about how we use data from teacher evaluations. (Or rather, how we do not use data from evaluations.) What different kinds of data do we collect?  Who are the prime users of this data?  Our quadrants might look like the chart below at first.  Note that the users of the data do not change, but the yellow boxes are empty.  In essence, we are asking, "What kind of data would be most valuable to each user?"  This also leads to dangerous discussions such as "What kinds of data promote expertise?"

I'm guessing that the majority of the evidence that you collect (if you are an evalutor) or that is collected about your practice (if you are a teacher) is on the "summative" end of the spectrum.  And since many states are systematizing the collection of teacher practice and SLO data, you could make an argument that most evaluation data is in the upper-right quadrant.  Look whom that data is meant for - district leaders, boards, and state officials.  Perhaps if you have an evaluator who makes more time for it, or if you have an observation cycle where not every year is a summative year, you might get to the lower-right quadrant.  But the left side is largely empty.  What to do?  Especially if these are the areas that benefit practice and innovation.

Here is one possible view of a "balanced" system to growing teacher expertise.  I have adapted it from Brookhart's quadrants.  

In the upper-left quadrant, we have schools studying themselves using school-chosen, educator-gathered, non-judgmental, iterative data.  The principal works with teachers to answer the question "What practices do we want to see and study?"  And then someone gathers that data and reports back to the staff on a regular basis to see if it is happening and with what level of quality. What to measure?  Consider what the instructional vision for your school is.  Or consider some top-tier Hattie strategies.  

In the lower-left quadrant, we have teachers studying themselves.  This is where instructional coaching and reflection come into play.  I often wonder: some doctors are "hospitalists" who specialize at overseeing the care of a vast array of inpatients.  Principals should develop the capacity to be seen as "instructionalists".  Leaders who are experts at strategies, coaching, and spurring reflection.  Teacher reflection is perhaps the strongest of the motivators towards innovation and expertise. 

I had these thoughts while reading Brookhart's book, but the next book I am starting is Frontier & Mielke's book about redefining the process of teacher evaluation and growth.  I think there will be many more connections in it to the quadrants above.  

Getting Better at Innovation

I recently wrote four posts about innovation in education, all inspired by the book "Better" by Atul Gawande.  The bottom line is this: Innovation does not need to be a disruptive practice, but it certainly doesn't happen by accident.  Improving existing practice is innovative, even when it happens incrementally.  Here are links to the four posts I wrote.

Better (link to post 1) - A doctor writes about innovation in the medical world.  Small innovations over the years from around the world have made us the healthy society we are today.  You can replace "doctor" with "educator" throughout this book for takeaways of your own.

Educators - Innovators or Researchers? (link to post 2) - Innovators must be researchers. The focus on PLN's, SLO's, data based decision making, and the "Plan-Do-Study-Act" cycle are all ways of saying teachers must reclaim their role as researchers.

Innovation: Start with the Basics (link to post 3) - You can't innovate unless you know your impact. And as a teacher, you can't know your impact unless you have the basics of lesson design down. Leaders who leverage a common definition of "learning target" and "success criteria" create measurable clarity across a school.

Leading Innovation (link to post 4) - Innovation that isn't shared is wasted.  Sharing your innovations multiplies your impact.  Leaders must expect the sharing of innovation across a school, and educators must help each other realize their strengths.

Thanks for reading,
Chris

Leading Innovation

Last in a series about improving practice and creating a culture of innovation

The three qualities I said were necessary for fully innovative schools:
1.  Teachers claiming their roles as researchers, since research leads to innovation.
2.  Teachers knowing their impact, since professionals seek to increase their impact.

And, for this blog post:
3.  Innovation loves company: Leaders must expect transparent innovation across the school rather than supporting pockets of innovation.


At Google, employees are expected to share their passion projects.  This transparency eliminates silos and leverages the power of group-think to solve problems.  A recent article on IBM's Almaden Lab, one of the last corporate pure research labs, talks about how the researchers have switched from working in isolation to projects that co-evolve.

We need to develop schools in which teacher-researchers freely share their innovative projects with one another.  And leaders need to set the tone by sharing their own passion projects and expect the sharing of innovation across the board.  Remember: innovation does not have to be a disruptive practice, and sometimes we happen upon it by accident.  New educators may need assistance from others to discover what their passion is.  As we learned in the book Better (prior blog post), making daily practice better with consistent small innovations can be just as effective as searching for the miracle cure.

Impact is a measure of innovation.  If principals want to increase impact, we need to first ask how we are increasing innovation.

So what is YOUR innovation?  Your passion project?
Math talk ~ Personalized learning ~ Increasing feedback ~ Technology for achievement ~
Character ed ~ Building relationships with special ed students ~
Using achievement data better ~ Service learning ~ Parent relationships ~
Goal setting ~ Coaching other educators ~ ???

The list doesn't end.  Because innovation doesn't end.
Because getting better can't end.

Innovation: Start with the Basics

2nd in a series about teacher innovation

As an 18 year educator, I've been through many evaluations.  Do you know what point in every evaluation cycle I dread?
The point when you sit down with your evaluator after a formal observation and he/she asks "How do you think that lesson went?"  I always feel like I'm sitting at a car dealership negotiating a price: sound confident, but don't overdo it.

Here's the question I want evaluators to ask instead:
"What was your impact in that lesson?"
Who cares how I thought the lesson went?
What I care about is this: Do we know if meaningful student learning happened?

And this is where innovation comes into play.
You can't innovate unless you clearly know what your impact is on a lesson-by-lesson basis.
John Hattie's mantra of "Know Thy Impact" must be pervasive throughout a school in order for teachers to measure their impact, increase impact, and make their practice better.

If a teacher can't tell you what his/her impact in a lesson was, that's not a problem.
It's a starting point!  Let's help you find ways to measure your impact.
How about learning targets that are more focused? How about trying some exit tickets?
Let's do some meaningful group discussion.  Let's look at some student work.
Once you know what your impact is, the natural goal of a professional is to increase that impact.
And THAT takes innovation!

My two favorite resources on this topic are Formative Classroom Walkthroughs (Brookhart & Moss) and Leading with Focus (Mike Schmoker).  Although Formative Classroom Walkthroughs sounds like a book for principals, it is a great book for teachers to really rethink the basics of lesson design and evaluate impact.  "Learning target" has a different meaning to everyone, but if a principal can leverage a single definition of learning target and success criteria, a school can increase achievement. Remember that "teacher clarity" is a top-tier Hattie strategy.
The book Leading with Focus reminds principals that focusing on the basics of solid lesson design will always produce achievement.  But we need to get into classrooms often enough to see impact and coach impact.  No matter what other initiatives come along over the years, solid lesson design that measures daily impact will keep the ship steady.

You can't get to your destination unless you know where you are.
And you can't innovate unless you know your impact.
Sometimes innovation isn't about the latest, greatest thing.

Sometimes innovation is just about getting the basics right so we can get ... better.

Educators: Innovators or Researchers?

Do you consider yourself an innovator?
Do you consider yourself a researcher?

In my last post, I mentioned three conditions that are necessary for educators to embrace innovation:

• Educators must claim their role as researchers, because research is the foundation of innovation.
• Educators must refocus on the basics of solid lesson design, because increasing impact drives innovation.
• Leaders must expect transparent innovation from everyone, because innovation loves company.

I want to expand on all three in separate posts.  First, teachers need to recognize and honor their role as researchers.  The research cycle happens so quickly in a classroom that it often goes unnoticed, especially by the teacher him/herself. Here's what it looks like in slow motion:

Step 1 - The teacher is curious as to why a group of students did/thought _________.  
     Perhaps a group of students performed beyond expectations; or maybe a group did poorly.
     No matter what, the teacher is curious about performance and how to improve it.
     Innovation starts with curiosity.
Step 2 - The teacher gathers some background knowledge.
     The teacher talks to the students about their performance.
     The teacher talks to other teachers (including online PLC's) or does some reading.
Step 3 - The teacher creates a hypothesis.
     The teachers makes a guess as to what will improve this group's achievement.
     This is where innovation starts to come into play.
Step 4 - The teacher tries something different with the students.
     This would be called an "application".   But there is no control group, because
     it would be unethical to withhold help from a group of students that could benefit.
Step 5 - Look at the results.
     The teacher takes a step back and assesses.  Was the hypothesis valid?
     Did the application work?  Or do we need to cycle back and try something else?
Step 6 - Report the results.
     Reflect with the students on what happened.  Reflect with other teachers, coaches, and
     leaders about what happened.  Blog online.
     No matter if it worked or not - help others learn!

A professional educator goes through this cycle every day, and sometimes even "mini cycles" several times within a lesson.  Constantly listening to students, making a guess, applying a new strategy, measuring the response, over and over.

When we talk about improving practice - making practice better - it is a result of research and reflection.
This is why I believe the most important professional development that educators do is the daily interaction they have with students.  Research, reflection, and innovation become the daily norm for educators who use their classroom as a laboratory to study students. Professional development is no longer a stand-alone event; it becomes the norm.  Those half-days or whole days once a month become a time to consult the experts or do some deeper analysis with colleagues.

And it doesn't stop there:
Teachers study students.
Principals study schools.
Superintendents study districts.

So if an educator says "I'm not an innovator" I would ask, "Are you a researcher?" because one flows from the other.

   

Better

"Perfect is the enemy of good" - or - Instead of waiting for the miracle cure, what if we focused on consistent improvements in our daily practice?

This is the problem Atul Gawande presents in Better: A Surgeon's Notes on Performance.  There are so many times in the book when you could substitute "educator" for "doctor".  Medicine, like education, is ultimately a system of human contact and human decisions, and every decision makes an impact on the life of another.  Two professions that have become increasingly standardized, filled with bureaucracy, and influenced by profits.  Yet both are still practiced at a personal level - doctor to patient - teacher to student - with traditions and knowledge handed down over centuries to make the next generation ... better.

Gawande presents stories from the world of medicine to illustrate his point that continuously improving our practice can be nearly as effective as seeking the miracle cure.  For example, the clinic that has cystic fibrosis patients living nearly as long as their peers simply by innovating and holding themselves (and their patients) to a higher standard. Or how convincing health care workers in the the mid-1800's to simply wash their hands saved scores of lives.

How many of us in education are seeking the "miracle cure" today?  Flipped classrooms, interactive whiteboards, zero tolerance policies, small class sizes, new standards, experiments in school governance, standardized testing - the list goes on and on.  Gawande's question for us would be "Why throw out everything you've learned as a profession over the years?"  Improvement lies within your own power to control, and you can improve the lives of the people under you right now instead of waiting for the miracle cure to help somebody else down the road.  Improving practice works at the strategy & self levers (Frontier & Rickabaugh) as opposed to so many other innovations that sound better to the public but don't move achievement as far.

The chapter on innovation in childbirth translates well to education.  You've heard of Apgar scores for newborns?  Virginia Apgar's 1952 innovation allowed doctors and entire hospitals to be compared, driving standardization across the nation. Previously, OB's used various techniques to aid in childbirth, such as forceps deliveries, depending on the position of the baby. But since forceps deliveries and these other techniques are difficult to teach young doctors, C-sections became the industry standard.  C-sections are easier to teach, answer nearly every complication that could arise (as opposed to the myriad of techniques OB's had to master), and are convenient to schedule. Likewise, standardized testing in education has given rise to pre-packaged curricula and pacing guides.  Students spending weeks alone on devices drilling math facts.  Students expected to all know the same thing at the same time because their age says they should.

So what is the solution?
1.  Teachers must claim their role as researchers, because curiosity and research lead to innovation.  The most important professional development teachers engage in is their daily work in the classroom.  "I wonder why this group of students thought X?"  Curiosity leads to a theory, an application ("I'll try this intervention"), gathering data to see if it worked, and then reporting the result to help others.  That's research, and teachers do it over and over every day - we need to recognize and honor it because research is the foundation of innovation.

2.  Teachers must refocus on basics of lesson design, because increasing impact is how our practice gets better.  What do I want students to learn?  How will students prove they learned it? What impact did I, as a teacher, have on this lesson?  How do I know? Teaching = effectiveness. Learning = impact.  There's a difference.  (This book by Mike Schmoker and this book by Moss & Brookhart are great resources here.)

3.  Innovation needs to be the norm in school culture.  Administrators should do more than create a culture that merely "supports" pockets of innovation.  A culture that expects transparent innovation as part of professional responsibility is what will set individual schools and systems apart.

It's easy to wait for the miracle cure - mostly because it's somebody else's job to discover it and tell you how to use it effectively.  But getting better is a self-driven process that can make an impact now!