How should teachers be measured?

image from kevindooley’s site on flickr.com/creativecommons

Earlier this week, an Education Report reader sent me a link to a recent piece by New York Times columnist Nicholas Kristof, titled, “Our Greatest National Shame.” He was referring, of course, to education in the United States.

The reader said he found most of the column “unremarkable” (especially, I’m sure, when compared to the lofty prose spewing from this keyboard), but Kristof cited two studies that piqued his interest. The reports were about teacher effectiveness, and they made this reader wonder how teacher talent should be measured in public schools, including those in OUSD.

Parent evaluations? Student test score gains? Classroom observations?

For all of the debate about teacher preparation and proper credentialing, some researchers say that those things seem to make little difference. In other words, that it’s hard to tell who’s going to be a good teacher until they start teaching.

One of the reports Kristof cited in his column was released this month by the U.S. Department of Education’s Institute of Education Sciences. It compared two groups of relatively new teachers: those with alternative certification, such as Teach for America, and those who took the traditional route. Researchers analyzed the teaching practices and the student reading and math scores of 87 pairs of teachers in 63 schools in 20 school districts. Each pair taught in the same grade level, at the same school, with randomly assigned students.

The conclusion? It didn’t matter which path the teachers chose, or the number of hours required by their respective certification programs, or even the content of those programs:

This study found no benefit, on average, to student achievement from placing an (alternative route to certification) teacher in the classroom when the alternative was a (traditional route to certification) teacher, but there was no evidence of harm, either. In addition, the experimental and nonexperimental findings together indicate that although individual teachers appear to have an effect on students’ achievement, we could not identify what it is about a teacher that affects student achievement. Variation in student achievement was not strongly linked to the teachers’ chosen preparation route or to other measured teacher characteristics.

The other report Kristof cited was a bit older. The Brookings Institution published it in 2006. Its authors recommended that the system make it easier for people to enter into the teaching profession, but harder for teachers to be promoted, or to earn tenure. The Brookings report recommended that teacher performance be measured in a number of ways, including parent evaluations, classroom observations, teacher attendance, and gains in students’ test scores.

I imagine some of these ideas wouldn’t be very popular in Oakland or elsewhere, but maybe I’m wrong. How effectively are teachers’ abilities evaluated now, and how do you think they should be measured? Is there too much emphasis on evaluating teachers, or not enough? Or not the right kind?

Katy Murphy

Education reporter for the Oakland Tribune. Contact me at kmurphy@bayareanewsgroup.com.

  • Steven Weinberg

    Evaluating teachers is immensely complex. Using increases in test scores sounds like a good way to do it, but it has many problems. If matched CST test scores were used it would give an unfair advantage to 4th and 7th grade teachers because students in those grades tend to score higher on the CST than 3rd and 6th grade students do. Students just learning English make substantial point gains on the CSTs, but if you measure by the percentage of students achieving Proficient or above, they do very poorly.
    Furthermore, not every good teacher would be good with every group of students. Some who do a great job with average to high ability students are unable to help students who are behind. Some teachers who are less skilled at working with advanced students do an excellent job of motivating students who usually don’t do well in school. Some who are excellent in presenting the curriculum do not form close ties to their students, while others who are do not organize their lessons as well may be exceptional in reaching out to students who need help and supporting them.
    The best way to evaluate teachers is to give good principals as free a hand as possible to make the evaluations. The best way to improve the quality of teachers is to increase teacher pay so that there will be a larger pool of applicants to select from, and so many good teachers will not leave after a few years.

  • Nextset

    S Weinberg: I agree that the stats and scores can’t be directly used to evaluate teachers. That measures mainly the students.

    Giving the principals a free hand to evaluate allows them to play favorites for particular no reason if that’s the kind of person the principal is. It is the most flexible and maybe this is the best way.

    Then we get rid of the principals whenever we lose confidence in that principal’s production.

    Sounds like old school. I approve.

  • Ms. J.

    A few thoughts occur to me:

    Of course, just as with assessment of student progress, assessment of teachers has to have a purpose, and that often seems to be lacking. I don’t mean teacher rewards or sanctions, I mean building on what teachers do well and helping them to improve in the areas where they are weak. My uncle, a bank vice president, was nonplussed when I told him that no such system was in place in the school where I taught.

    There’s emphasis on judging teachers (and finding us lacking) but there’s never been much real evaluation of me or my colleagues going on in my ten years of teaching (nine in Oakland).

    Principals have way too much on their plates to be able to give teachers the attention they need, and there are so many pressing, moment to moment emergencies at schools that classroom observations are given very low priority (my principal actually missed scheduled observations of me more than once).

    To really do a good job of evaluation, there should be people with time dedicated solely to this purpose. Of course this is expensive, but I think that would be a better use of money than paying teachers more.

    Once evaluated, a teacher should come away with specific ideas for how to improve her practice, and benchmarks to measure her growth. In Obama’s education plan there is talk of creating a class of mentor/master teachers who do not leave the classroom entirely but are released and compensated to spend time with novice teachers. Programs with this idea exist but are not fully realized, and most people I know who’ve experienced BTSA have felt it comprised a lot of busy work and was not very helpful.

    There was an article on this topic in the New Yorker a few issues back, discussing the findings that it’s very hard to predict who will make a good teacher. I always thought that my teacher credentialing program was trying too hard to be academic with what is essentially a craft–learning how to respond to kids and leverage what they do know to help them know more.

    The majority of teachers I know feel very anxious about having anyone evaluating them. I think this is because the public dialogue is so critical of teachers and much of the time teacher evaluation (certainly if it comes down solely to the test scores) is more like teacher criticism. It’s also because teachers aren’t used to having their work seen–the results, maybe, but what we do day to day is so private (in front of the classful of kids).

    If we were more used to having visitors in our classrooms (and if we could be assured that these visitors were looking to catch us being good, as we do our students, rather than to catch us making mistakes), we would not feel so nervous at the prospect.

    Peer evaluation in which colleagues are released to observe each other and make constructive comments is a good beginning.

  • aly

    i agree with ms. j that we need to be observed by someone whose sole job is to evaluate us. principals are simply overwhelmed and don’t have the time. perhaps the solution is a sort of principal on special assignment who is not involved in discipline, scheduling, athletics, and all the rest, but ONLY evaluates, coaches and works with teachers. it would be critical for this person to be someone with tact that teachers can trust but who also doesn’t let slackers stick around.

    i also like the idea that tenure is less easy to obtain and easier to bypass in order to let someone go. i don’t think my job should be jeopardized because i talk about a controversial issue with my kids or we read a book that isn’t on the reading list, but if i ever stop teaching them and start showing movies all day, i want someone to kick my butt out! there are so many people in this profession who abuse their tenure it is shameful.

    as far as being observed, i can’t shake the feeling that you only ought to be afraid if you know you’re doing something wrong or you’re not a good teacher. and though all humans need some sugar with their salt, to not want criticism is detrimental to you and your students. if no one tells me what i’m doing wrong, it’s almost impossible to fix it!

  • ex-ousd staff

    I disagree with Aly – the principal’s first responsibility should be head teacher, the person whose primary, but probably not sole job, is leading the teachers, which includes evaluation. The principal on special assignment is the one who should be doing all the chores that don’t require such deep experience and long term relationship with the site. The evaluator has to be someone who is intimately involved with the school community and knows how each teacher fits into that community. Bringing in outsiders to do this job often produces useless commentary.
    Test scores can be used diagnostically but have no meaning without awareness of context. It takes someone who is familiar with the teacher and the students to draw conclusions from the standardized tests.

  • Catherine

    Evaluation should be in a combination of five areas:

    1. All children in a class need to progress one full school year for each year they attend school. If you are a fourth grade teacher, the first week of school every child is pretested. Those children who begin at 3rd grade 3rd month level should end at least 4th grade 3rd month level. Those who entered at 5th grade 5th month level should end at 6th grade 5th month level. Pre and post tests per student. Achievement for all. 60%

    2. Student evaluation. Every student has an opportunity to grade every teacher from K – 12. This should be administered in the classroom with the teacher absent. 10%

    3. Parent evaluation of the teacher. Teachers would send evaluations home with the students. Minimum of 75% need to be returned. 10%

    4. Principal evaluations of the teacher. This would be a minimum of one classroom visit per week in high school and middle school and two classroom visits per week in elementary school. Feedback in written form is given within two days of each visit. 15%

    5. Teacher participation in the school community. Oratoricals, study hall, small reading groups, walk-a-thon, science fair, spelling bee, community fairs, etc. 5%

    All stakeholders should have a say in the quality of the teacher. Every child working at the very bottom, the very top or all places in between has the right to progress one full school year in education from where they began.

  • TheTruthHurts

    On one hand, I agree that evaluation must be challenging and someone dedicated to the task would be great. Heck we could have video of classrooms streamed to a remote evaluator (a master teacher) who could review the “tapes” at her/his leisure and then conference with the teacher. This is the kind of innovation that happens in universities, why not here.

    On the other hand, evaluating teachers is EXTREMELY easy. Parents and students do it all the time. Now that we have charters, even poor families can vote with their FEET. They not only evaluate the teacher, but the principal, the overall administration and even their peers. As families get more options, there will be more evaluation going on. Right now, Oakland is not fairing so good either. A friend told me that we have more charters schools than anywhere in CA other than LA. Given we are much smaller that’s saying something — not good.

  • aly

    kudos, ex-ousd. my idea was focused on making sure there was a person on each campus dedicated to evaluation; i got a little myopic and didn’t even realize i was backwards in who should do what. you are absolutely right that it should be the job of the head teacher and i feel kind of silly for not recognizing that to begin with. comment revised :-)

    catherine: i like the process you outlined, but i disagree with parent evals requiring a 75% return rate and counting for 10%. i make 5-10 home contacts a week, but parents rarely reach out to our school. we don’t have parents who call to ask how their child is doing or respond to report cards. they won’t come to school site council meetings or even return my calls sometimes. in a lot of poorer cities this is the case; parents are too busy trying to make sure there is a roof and food to focus on school. i am uncomfortable then putting my career in the hands of people that don’t really know what i’ve done with their children for 9 months; even if their feedback were to be positive, it would carry less weight with me because they weren’t involved enough to know what was happening in my classroom for the last year.

    until communities can be supported so that parents can be freed up to participate in their childrens’ educations, it doesn’t strike me as a source that should be relied upon for evaluation.

  • Sharon

    I found it bizarre that a set of administrators with no personal experience with teaching an academic class at the secondary level were in charge of doing the teacher evaluations at my daughter’s middle school. The Principal was former PE teacher turned administrator, Asst. Principal #1 was also a former PE teacher turned administrator, and Asst. Principal #2 was a former 5th grade teacher.

    How much of this goes on in OUSD?

  • Donna

    Kids are also able to identify top teachers. Those who are not effective do not get their respect, and often, their attention. It can be a vicious cycle.

    Some kids from high SES families will score high despite poor or mediocre teachers. I have seen parents hire tutors and enroll them in tutoring schools if they feel their kids aren’t doing well enough in school. And some kids, especially if they are from challenging backgrounds, will need enormous resources including the best teachers for them to score as well.

    Teaching is an art and a craft and less a science. I am saddened and alarmed when I have seen so many experienced, gifted teachers leave the classroom in disgust over NCLB and other testing requirements which crowd out actual time they could be involved in teaching.

  • John

    Sharon writes: “I found it bizarre that a set of administrators with no personal experience with teaching an academic class at the secondary level were in charge of doing the teacher evaluations… How much of this goes on in OUSD?”

    Actually Sharon it’s NOT just OUSD. These non-experienced teacher evaluators are doing precisely what they’re exclusively permitted to do in accordance with the rights and privileges of their CA Administrative Credentials. These folks possess the same Adm. Credential as ‘Superintendent’ Dennis Chaconas who presided over the OUSD’s ‘03 fiscal catastrophe resulting in state receivership, and in his defense claimed he was focused on district curriculum issues, NOT the district budget. The CA administrative credential is a magical thing as it automatically qualifies folks to oversee & evaluate personnel they’re NOT experience or academically qualified to supervise & evaluate. Collective bargaining restrictions only permit (credentialed) administrators to evaluate teachers (& school district finance accountability personnel, etc.), and make retention and termination decisions. It’s just one of those well established sad but true facts of life in California’s mega lobbist represented education system.

    ALY writes: “until communities can be supported so that parents can be freed up to participate in their childrens’ educations, it doesn’t strike me as a source that should be relied upon for evaluation.”

    And they’ll “be freed up” once there’s been a redistribution of national wealth right ally? Freed up in accordance with a failed social theory that went down with the Berlin wall and now feverishly being resurrected by Obama and his senate and congressional comrads.

    So ALY, grab your prayer rug, get on your knees, face your computer and click on your new (President Lincoln Logs) national anthem:


    Great Smiley face ALY! Keem em coming, smiling, & drooling.

  • cranky teacher

    To me, this is one of those endless education debates that completely ignores the reality on the ground: The ratio of teachers to administrators is WAY too high to have the kind of in-class observation, thoughtful evaluation, and constructive coaching necessary to change anything in the status quo.

    When I worked in a corporate setting, most supervisors had 6-12 folks reporting to them.

    In most schools in most districts, supervisors (administrators) will usually have 30-50 employees reporting to them.

    When I worked in a corporate setting, my boss sat all day long 25-feet away from me, albeit behind glass. We had formal meetings 1-2 times a day and informal contact 2-3 more.

    In my school, no administrator has stepped in my room this year. Last year, i was visited twice, for about 20 minutes each time. We see administrators in the hall, in the office and at professional development.

    When I worked in corporate environment, evals were quarterly and the human resources department made sure all deadlines were met.

    In OUSD, some teachers have told me they have not been evaluated in 7-8 years.

    Moreover, evaluations are usually way too easy or way too hard, depending on what kind of reputation the administrator is trying to cultivate with those above and below them.

    The reason even reasonable teachers fear the end of tenure or the rise of “merit pay” is we know nobody gets systematically and fairly evaluated in the schools — and that it in the current system it is well-nigh impossible to do so.

    Teachers who wish to improve welcome real evaluations — but don’t want a drive-by eval based on a 20 minute visit by an administrator that’s been at the school for a few months.

  • cranky teacher

    Forgot to mention: Department chairs are ostensibly the “middle management” in a school, but they have no real eval authority and are usually little more than ad hoc communication channels from the top down.

  • AGCaraulia

    I have trained educators for over twenty years, been a professional teacher prior to that, and since my retirement doing analysis of interactive processes for personal “enjoyment”. (I have developed a allergy for “work” and institutional structure. ~:) ) The seminars I wrote and conducted a course, ‘Nonviolent Crisis Intervention’, to manage aggressive and assaultive students non-harmfully with therapeutic resolutions.

    I mention this to point out that the following are from first hand observations, not just unsupported musing.

    First, it is with regret that my experience with teachers has been 20% fantastic, creative, and thoroughly excellent professionals and then…the rest… the “ok” to “should take another job” (staj). The sad part is when criticism is raised, those who are “staj” are THE most angry and defensive; who seem to mask and forget their usual classroom behavior. They fail to use this criticism as a review process, helpful to identify how their performance is adjudicated by those who are responsible for their pay, and to treat as “real evaluations”. It is stunning that educated persons think that huge public opinion is not “real evaluation”. Question: are these criticism made-up, have no evidence, is from the “ignorant public” who do not know anything? Wow, one group is not very aware???

    Hoping I have not been too offensive,

    What has been written so far is not a negative piling-on, but a prequel to some fundamental (and not necessarily conclusive) thoughts that have occurred during my exploration of “what, how, why, and the best way to exchange information between beings”. There are some propositions challenging current approaches of teaching Teachers, but need a bit more supporting research. AGC

  • aly

    cranky: that is a really good point- we are definitely not served well by the overwhelming teacher:evaluator ratio. i guess that was one of the advantages of having the evaluator be someone other than a principal in my first comment. i get a lot of good things from BTSA and New Teacher Support and Development that make me wonder if people with that type of training would be the best to evaluate us? as it is now, BTSA observations cannot be used as an official evaluative tool. i used to like that, but the more i experience teaching, the more i realize that if a coach sees their charge is someone who should not be in charge of a classroom, they ought to be able to do something about it.

    i also appreciate the fact that because administrators are NOT present in teacher’s lives, the “drive-by” eval is unfair. 20 minutes in my room twice a year doesn’t tell you much about how i am doing my job. i have gotten spoiled by my smaller school and am used to having my principal in my room once or twice a week, so when it comes to a formal eval, it’s something i trust.

    using department chairs makes me anxious just because there is so much political nonsense that goes on and has nothing to do with quality teachers. i fear there would be alliances and bias in evaluative decisions made by some department heads.

    sharon and john: despite john’s unrelated attack on me, i agree that there is way too much gray area in what an admin credential allows. educators need powerful unions and representation because we do get a raw deal much of the time; however, it saddens me when we use our influence selfishly instead of to improve the lives of those we serve. it is difficult to be a quality administrator to a population you really don’t understand or have experience with.

  • nvt

    As a counselor for 40 years, mostly in a suburban school district I was amazed when I spent the first year of my retirement helping out at an Oakland high school. I saw some wonderful, dedicated teachers, perhaps an even higher percentage in OUSD, but the “system” did not support or validate them.

    To have valid means of evaluation you must have shared goals and trust in each other. I think that often the various stakeholders have different goals so unless goals are agreed upon, the evaluations of each group will be different. The second major piece is trust. It doesn’t matter if the evaluator is an administrator or a peer, the trust factor has to be established. In my former district a percentage of us (teachers) could opt for peer evaluations. I found that very gratifying and helpful. A tragedy for Oakland is the turnover of administrators so that the rules change annually and neither shared goals or trust is possible (see Skyline High!)

  • http://jeanswatercolors.blogspot.com Jean Womack

    Having a principal or even a Chief of Police is like choosing a husband or a wife. You want someone who is going to stay there for a long time. So you go out with a few people before you choose the one you want to be with for a long time. Maybe you don’t get married to the first one you go out with. Hopefully, by the time you decide on your Mr. or Ms. Wonderful, you’ll be willing to give him or her the support they need to continue in that job.

  • cranky teacher

    agc, your overall point is unclear to me. “huge public opinion” that teachers generally suck? Or parent/student gossip about which teachers are good?

    Perhaps you’re trainees were defensive because your message is garbled?

    I will agree, though, that Oakland special ed teachers — many of them young and idealistic — who are in lots of classrooms everyday generally endorse your breakdown of great teachers vs. OK and terrible.

    But they also have high standards, unrealistic in the current milieu. Many teachers are in the classrooms because they are survivors willing to put up with crap pay and marginal conditions and bullying from above and below. The status quo system selects for these folks more than it does “fantastic, creative and thoroughly excellent.”

  • Isabel Rodriguez-Vega

    I’m not sure if anyone mentioned this point, but why not let students evaluate teachers? After all, they are the ones who would know the best. They spend an hour a day, five days a week around each teacher and they are the ones learning (or not learning) from these teachers.

    As a student I know I always evaluate teachers, sometimes subconsciously. I can tell if a teacher knows what they are doing, is engaging, is effectively putting knowledge into our heads, etc. based on my observations of them and the students. If students are disrespectful towards the teacher, rarely come to class, rarely do the homework, or are uninterested in the material the chances are its a bad teacher.

    I don’t know why everyone thinks its necessary to put together a whole administrative team of professionals to evaluate every teacher on a regular basis. This is costly, time consuming, and probably near impossible. It would be simple, free, and quick to just hand out little teacher evaluation forms to the students every month or so, and this alternative would be much more accurate than any evaluation done by an administrator.

  • Sharon

    At Skyline in 2002 at my daughter had a 9th grade English teacher who she thought was great. Our family totally appreciated her because she had high expectations, was rigorous, and was very skilled with handling difficult students.

    At that time I worked at another school. The daughter of a school secretary was in the same class. They both HATED the teacher, big time. “She’s way too hard,” the secretary angrily told me one day.

    So here are two students, and two families, with completely opposing opinions about one teacher. My daughter ended up as a successful student. The secretary’s daughter eventually dropped out of Skyline and went to work at Macy’s.

    I’m not sure the opinions of students, or their parents, would be the best measure, unless of course, I’m the one being asked.

  • Nextset

    Isabel: Students should be surveyed in some fashion when administration is evalutating staff. But that survey can be sub rosa as well as review of performance stats. To openly allow students to “rate” a teacher upsets the positions of both. You don’t have enlisted men evaluate army officers for the same reason. But you do consider a sgt-major’s appraisal of a young Lt. There are ways of doing these things.

    It is far more important to have opinions and appraisals from other teachers about a subject teacher than student input. You see, when you teach a student, it’s purely optional if the like you. In many cases good teachers – the ones who make a huge difference for the better in a student – were seen as not being a nice person (ie annoying, cantankerous, old fashioned and a lot of trouble in general).

    That’s my experience and that of friends and relatives. The teachers from a lifetime ago we still remember and talk about and maybe emulate were not sunny days in the park. From Sr Mary Immaculate to certain college and law school instuctors, the best teachers were often but not always very strong and abrupt characters.

  • Isabel Rodriguez-Vega

    Maybe most students don’t think this way, but I always associated difficult teachers with good teachers, regardless of if I like them or not. If its easy they must be doing something wrong.

  • mike

    I am an excellent teacher in the classroom with 16 years experience along with 10 years related industrial experience. however with all the educational gimicks constantly thrown at me, I had a major bought of depression, to the point of 3 suicide attempts in 2 weeks. I haven’t been to work for 3 months and am working on long term disability. The criteria of this paperwork fiasco was so overwhelming it made me think that everything I’ve done is wrong and I’m not a qualified teacher based on the paperwork presented. I am not allowed to fail a students. If I do,it’s my fault even if the student was a no-show, It’s my responsibility to provide them some form to pass my class. Tons of interventions and processes that need to be done which take over 20 labor hours to go thru the process. 1% of my students requires 99% of my time and labor.