Test score gains: more knowledge or better prep?

Between 2008 and 2009, 80 percent of Oakland’s elementary schools improved their scores in math AND in English language arts, according to a school district analysis. (A list of the most-improved schools is posted below.)

Oakland’s not alone in its upward trend. On the page 4 and 7 of this news release, you’ll see increases in English and math scores, statewide, especially in the early grades.

John Boivin, who administers the STAR Program Office at the California Department of Education, said there were no major changes this year in the test, itself, or in the scoring of it. He said his team hadn’t yet drawn any conclusions about why the scores went up.

Boivin did say, though, that the law only requires the state to change half of the questions on each test from one year to the next. In other words, experienced teachers have a pretty good idea of what’s going to be on it.

Terry Vendlinski, a senior researcher at UCLA’s National Center for Research on Evaluation, Standards, and Student Testing (CRESST), said Bob Linn at the University of Colorado at Boulder has found that scores typically continue to rise steadily until a state changes its test.

“If the test doesn’t change substantially, year over year, the teachers begin to learn how to teach the material that’s tested,” said Vendlinski, a former teacher.

Nevertheless, some Oakland public schools outpaced the average improvements, statewide. Below is a list of most-improved schools and their percentage-point gains in proficiency, provided by the school district.

Translation: If 40 percent of students at a school scored “proficient” or better in 2008, and 50 percent did so in 2009, that would be an improvement of 10 percentage points.

Bridges at Melrose – an 11.1 percentage-point gain in Math, 11.4 in ELA
Burckhalter Elementary – 16.3 in Math, 19.5 in ELA
Community United – 12.2 in ELA
East Oakland Pride at Webster – 15.3 in Math, 15.6 in ELA
Emerson Elementary – 15.3 in Math
EnCompass Academy – 11.3 in Math, 17.0 in ELA
Fred T. Korematsu – 19.3 in Math
Futures – 15.5 in Math
Global Family School – 11.4 in Math
Greenleaf Elementary – 16.9 in Math, 13.9 in ELA
Hoover Elementary – 10.8 in Math
International Comm. Elementary – 12.6 in ELA
Laurel Elementary – 10.8 in ELA
Lazear Elementary – 11.5 in ELA
Learning Without Limits Elementary – 12.0 in ELA
Maxwell Park Elementary – 11.9 in Math, 10.8 in ELA
New Highland Academy – 13.6 in Math, 10.1 in ELA
Parker Elementary – 10.2 in ELA
Redwood Heights Elementary – 14.1 in ELA
Sankofa Academy – 13.7 in Math, 12.9 in ELA
SEED Elementary – 13.4 in Math, 21.6 in ELA
Sequoia Elementary – 11.0 in Math, 14.6 in ELA
Sobrante Park Elementary – 10.0 in Math, 12.1 in ELA
Think College Now – 17.6 in Math, 12.2 in ELA
Webster Academy – 11.7 in ELA
Whittier – 12.3 in Math

Claremont Middle School – 14.8 in Math, 10.2 in ELA
Cole Middle School – 24 in Math
Peralta Creek Middle School – 34.2 in Math
Urban Promise Academy – 11.1 in Math, 13.7 in ELA
West Oakland Middle School – 31.5 in Math, 17.0 in ELA

file photo by Cindi Christie/Contra Costa Times

Katy Murphy

Education reporter for the Oakland Tribune. Contact me at kmurphy@bayareanewsgroup.com.

  • Local teacher

    I’m disappointed by this report. Why do district and school gains need to be presented with the idea that perhaps we did better prep? Or that because the tests haven’t changed much, experienced teachers have a better idea of what to teach?

    Teachers SHOULD know what to teach – they should know exactly what the appropriate level of rigor is for the standards they are teaching. The way teachers know this is by using CST release questions and items to guide their instruction and to have a clear picture of what it means for a third grader to find the main idea of a passage.

    Regardless of all this, I’m offended and disappointed by this report because it gives the impression that the hard work happening in Oakland schools by our kids, families, and staffs is NOT the key factor in improving our scores and schools. Rather, the lack of changes to the test makes it easier. This belittles our work, our district, and most of all, our kids.

  • TheTruthHurts

    I have to agree. Although it is a natural question about whether it’s learning or prep, I’m still disappointed because I know they’re not saying that in Palo Alto or probably even San Jose.

    Frankly, our kids need to know how to take tests to succeed in this world. It may hurt, but it’s THE TRUTH.

  • Ms. J.

    I think your penultimate paragraph refutes your point, doesn’t it? If teachers are using release test questions, isn’t that a form of effective test prep?

    And in response to the idea that hard work has made improvements in Oakland–do you really think that teachers are working harder now? Or just harder on test readiness? As a teacher going into my tenth year, I am offended by the idea that I and my colleagues are only now working hard.

    I’m sorry to be cynical. I would love to agree with your optimistic view. But I would like to see some other metric than test scores. Everything I’ve ever read regarding these sorts of tests reports that the test scores go up a few years after they are first given, only to plateau–we’ll see what happens in a year two. As noted in the article, it wasn’t just in Oakland, but statewide, that test scores improved.

    Given the unholy alliance between publishers of tests, scripted curricula, and charter proponents, I wouldn’t be surprised to find out that this improvement in test scores were something the test makers plan for–to get us all hooked.

    And I’m sorry that you feel this belittles your (our) work etc. I think that this blog is one of the most uplifting things in the community of Oakland schools.

  • Ms. J.

    I meant Local Teacher’s penultimate paragraph–TruthHurts wasn’t up when I was writing.

    So in Palo Alto or San Jose, TruthHurts, you think Katy’s counterpart is trumpeting the success unanalytically? Speaking of truth, isn’t it better that she speak it (or hint it)?

    And the world isn’t one way or the other without our influence. The world is as we have made it and as we make it. If we subscribe to the notion of standardized tests as the only way to measure growth then you’re right.

    By uncritically celebrating improvement on such tests (while not acknowledging or emphasizing so many other kinds of growth) we are establishing a world in which all that matters are these tests.

  • Katy Murphy

    I have to counter the suggestion made (I think) by Local Teacher and TheTruthHurts that I saw the huge test score gains in Oakland and wondered why.

    The test prep question occurred to me because of the state trends — not what was happening in Oakland. Take a look at the overall increases in proficiency statewide since 2003, and then look at the change from last year to this year, particularly in the lower grades. It’s big.

    At the same time, I noted that Oakland’s gains exceeded state averages.

    Just trying to add a little context to the discussion…

  • Doctor J

    Check out the MONSTER gains posted by Delta View Elementary TWO YEARS in a row 63 points in 08 and 85 points in 09 — and with large groups of minority and low income kids. WOW ! And the state is bragging over 5 points ??? Delta View must be doing something different down in Baypoint (Pittsburg) in Mt. Diablo USD. What is it ??? Why aren’t the rest of the schools doing it ???

  • http://perimeterprimate.blogspot.com Sharon

    It’s important that we take a look at the “Achievement Gap” when the test scores are annually revealed. API’s are being calculated in Sacramento at this very minute, and the online reports will be released Sept. 2. From my recollection, the API’s for specific schools in OUSD are usually featured in the Trib, but information about the progression of the gap is never to be found.

    Calculating the gap is not hard to do. Here’s what I came up with last year, using the API as the most accurate measure (as suggested by the-very-savvy Steve Weinberg).

    API scores from 2002 to 2008
    — African American students: 539, 559, 562, 587, 604, 602, 609
    — Asian students: 684, 708, 718, 749, 768, 778, 801
    — Latino students: 494, 542, 559, 592, 609, 616, 642
    — White students: 806, 829, 847, 859, 884, 882, 890

    Achievement Gap trends from 2002-2008 (API of subgroup X minus API of subgroup Y)
    — Between White and African American students: 267, 270, 285, 272, 280, 280 and 281. This gap increased by 14 points since 2002.
    — Between Asian and African American students: 145, 149, 156, 162, 164, 176 and 192. This gap increased by 47 points since 2002.
    — Between White and Latino students: 312, 287, 288, 267, 275, 266 and 248. This gap decreased by 64 points since 2002.
    — Between Asian and Latino students: 190, 166, 159, 157, 159, 162 and 159. This gap decreased by 31 points since 2002.
    — Between Latino and African American students (a comparison not typically made): -45, -17, -3, 5, 5, 14 and 33. This gap increased by 78 points since 2002.

    In the past several years during which the focus has been on test scores as the single measure of school accountability, the achievement gap between African American students and the three other major subgroups in OUSD has actually increased. After all the hard work, I find it astonishing that any gap is increasing. At the best the curve should at least stay flat!

    Katy, perhaps the Trib could print a graph of this time span information when the API’s are released this year? As hard as it would be to look at, the information would definitely help with extending the conversation.

    Research needs to be done to find out what is going on. Perhaps OUSD AA demographics are changing. For instance, maybe the stronger AA families are moving out of Oakland or switching to charter schools, and our schools are now dealing with a population of the most challenging students who have been left behind. Or is the single-minded focus on test scores actually turning off AA students somehow?

  • Katy Murphy

    That’s a good point, Sharon. While I often highlight the achievement gap in my test score stories, I haven’t paid enough attention to the widening or narrowing of the gap over time.

    That’s a good idea, and one I will definitely consider for the API score release on Sept. 2.

  • http://techforeducators.com Matt Spergel

    To be honest, the workforce doesn’t need better test scores. What they need is greater creativity and innovation to solve the problems in the world.

    But what I don’t understand is how businesses expect schools to do better, but they themselves don’t get involved with their local schools. For instance, if Google needs more computer scientists, they should be more involved in the Mountain View school district. They have $20 billion in cash, but they don’t build a computer wing for Mountain View High School? Instead, they’ll hire more people in India. It makes no sense and shows an expectation of corporate irresponsibility which I find hard to fathom.

  • Nextset

    (Sigh) Sharon, thanks for the post above.

    Here we go again. The “Gap”.

    Research which I find compelling indicates that the “Gap” we speak of is firmly rooted in avg IQ levels of the populations we speak of. It can be expressed in brain processing speeds. The different groups have very specific averages – individuals within each group have their own operating speeds based on their individual circumstances. For the most part the IQ levels are strongly inherited. Nutrition is also a factor in this with such things as group practices of breast feeding being reflected in scores. The effect of the different scores are most dramatic after puberty with the more intelligent showing the greatest differences with age. The most obvious difference is in abstract thinking. The lower the score the more concrete the thinking and the less able to handle higher math and verbal demands and the more present oriented. Higher levels tend to have certain psych issues the lowers don’t. Essentially every racial group has a different average IQ as does most national averages.

    So 4th grade and 6th grade scores will not show gaps as much as 10th grade, age 22 and age 26 comparisons of group accomplishments. This may be why public school systems like to talk about the earlier tests and comparisons, falsely claiming the Gap doesn’t exist or is being closed.

    Which brings us back to OUSD which does not and is not going to feature high IQ groups. In the Brave New World the higher IQ groups – who take care of themself – have gone elsewhere.

    OUSD is dominated by groups of people who have not arranged their lives so they can live and school with those who go to Piedmont, Belmont, or some other “mont”.

    And you don’t have to be but so smart to make a living in the USA and stay out of Prison, off welfare and a without premature death. Lots of immigrants from 3rd world countries do it, not to mention the trailer park crowd.

    To the extent OUSD prattles on about sending the chillun to college they are no different than the politicians imposing college prep requirements as graduation requirements. They are deliberately funnelling the proleitariat children of Oakland – Black and Brown – into a meat grinder.

    Look at the math and verbal scores for the OUSD Black Children. By and large they are never going to do college (although we need a campus for those who do have a shot and want to go). OUSD exists to find a place in society for prole children, no matter how screwed up their families are.

    OUSD is supposed to bring the majority of it’s (black) students to the point where they can seriously expect to get a job at age 18, get into a training program for a career job, get accepted into the military – or at least (for the girls?) get into a decent marriage with a man who will provide for them and their children.

    Fix that Gap.

    Forget the college Gap. It’s more important that the black students not have a gap on entry level jobs, military acceptance and the other things in life that in 1960 Blacks took for granted. The public schools have thrown away all the safety nets they provided generations ago and just blame all the failures on unspecified teachers and the chillun themselves – for the fact that the kids are turned out so bereft.

    That means robbed of something useful, lacking in something expected.

    OUSD is NEVER going to change ghetto blacks into Rhodes Scholars. But OUSD used to train them to be fit for employment. Not anymore. Does anyone have OUSD black scores in math and verbal from 1960 to compare with the present? Do we have graphs of the reading scores over the generations?

    I think that school policy has resulted in deteriorating performance over the generations. Am I wrong? Or am I wrong in thinking that the black scores are worse than when I was in school.

  • Nextset

    A Postscript on Gap remediation. Efforts to close the (academic) Gap will be received differently by the different groups. It is likely that an intense remediation effort will (at the same time) lift the scores of other groups greater than the scores of the lowest group – so district-wide remediation may actually make the Gap worse. Especially if a large effort is being made to take the few to college level.

  • Cranky researcher

    Once again, Nextset: If IQ was largely inherited, we wouldn’t see a steady, strong climb in national IQ averages over the century, but we do. IQ is highly malleable. You can increase your own IQ significantly across your lifetime through…education! I already cited all the research last time you posted this bogus argument (Charles Murray’s weak IQ work has been refuted by good data and analysis).

    About the OUSD black-white achievement gap: apples and oranges. OUSD’s white students are a tiny affluent elite living in the hills. Of course they score higher than the poor majority. The relevant gap is between black and white student performance under relatively similar circumstances. Even affluent black students perform lower than their white peers. No, black OUSD students for the most part are not going to top colleges like white OUSD students do. But there is no reason why a majority of black and brown Oakland students couldn’t attend community college or state school and make it to graduation. Except low expectations, poor investments in teaching, etc. Three consecutive years with an excellent teacher can erase racial achievement gaps in elementary school, acc to some research.
    Finally, it is true that these tests do not measure everything and are not necessarily predictive of success. Persistence (“resilience”) is more highly correlated with college success than SAT scores. We need to teach the whole student, but we need test scores to ensure that basic educational needs are being met. Without them, schools pass along illiterate kids and there is no way to prove it and do something about it. The old school way.

  • Debora

    Cranky Researcher:

    How do you account for black students adopted into white middle-class to upper middle class families at birth who are still struggling and are performing at below basic and far below basic? I happen to personally know four such families.

    None of these children were born with drug or alcohol addicted. Some birth mothers had good nutrition and medical care throughout pregnancy, some did not.

    I struggle with this question often as these children are all in hills schools and three of four are now using IEPs to manage their public education.

  • oak261

    Cranky researcher: To say IQ is highly malleable is very misleading. You refer to the “Flynn effect” and the increase in test scores over time. Yes the Flynn effect is real, but it doesn’t debunk the meaning or significance of IQ. You can watch a debate between Charles Murray and James Flynn online, and what is remarkable is how much they agree.

    There is research supporting Debora’s observations that are not easily dismissed.

    Its important not to dismiss facts or hard research just because we don’t like the results.

  • Alice

    It is nobel to point out such tremendous gains in elementary schools here in Oakland, however are there any gains in Middle or High Schools? Pay attention to the “Small Autonomous Schools” created during the last few years. The scores are dismal, non the less the achievement gap.

  • Nextset

    Alice; Remember, IQ difference are most noticeable after puberty. Every time you see an educator using elementary school scores to discuss “The Gap” he is deliberately deceiving you about his program’s success and relevance.

    If the lower IQ students aren’t given needed training at the high school level the difference in standard of living by the 10 year reunion is stark indeed. The dull kids can’t “make up” the training deficits after they leave high school. This is why I feel schools such as OUSD are killing the black kids with kindness (if you call this “kind”). With schools such as OUSD neglecting the deportment and basic skill indoctrination of the larger black population this group is ready made for being supplanted by the 3rd world immigrants the government is importing as fast as it can.

    This was not true several generations ago where OUSD graduates went to service work and to the military upon leaving school. They were largely acceptable candidates then. Now they are unwelcome job applicants (“bilingual preferred” is code for “we don’t use colored people” and military rejects (due to failing the math/verbal computer-administered/time pressure tests).

    I was talking to a law grad today about some of this. When I was at Oakland Tech for the summer programs they had a basement cafeteria where large black women baked fresh hot cinnamon rolls doused with butter in the morning and cafeteria lunches – Mac & Cheese & Burgers and whatever else Cooked from Scratch with hot fresh cornbread, etc. (Cheap, too!) We went upstairs to Typing classrooms outfitted with 50 or so huge electric typewriters, Chemistry Classes with individual bunsen burners and sinks, biology classrooms with microscopes for every student… And NO ONE ever talked back to a teacher. Classroom doors were locked at the tardy bell. No one was in the halls during class. Miss class 3 days and you’re out. We got to school on time in the AM for the cinnamon rolls.

    Now I don’t know any urban school with that kind of food, that kind of equipment, and that kind of discipline. Maybe the kids would do better if we had this again.

  • Alice

    Nextset, That was my experience too with the exception of the large Black Lady in the kitchen. Discipline is up to the person heading the ckassroom. It is my experience if you present yourself as an adult, nit a “friend” the respect factor is very different. There needs to be some sort of expectation and experienced teachers in a high school classroom. That is not the place for on the job experience. This is just my opinion

  • Steven Weinberg

    Cole Middle School and Peralta Creek Middle School, both of which appear to be being phased out, had very large test score gains. That also happened when Lowell Middle School was closed. Is there anyone familiar with those schools who can explain the gains made by schools that the district decided needed to be closed?

  • Nextset

    Alice: Those cinnamon rolls were so large two people could share them. They were hot out of the oven at 7am and dripping with butter. I’m sure they served other things in the morning but all I can remember was the cinnamon rolls. My Biology teacher was black also, she was a postgrad (MS/PhD?) student from UC Berkeley. Very glamorous and tougher than we were used to in our regular school. Absolutely no false praise from her, no unearned anything. (We worked hard to get recognized..) After all-white teachers for 10 years I was really pretty happy with the food and the teaching staff at Oakland Tech. Nobody was goofing off even in the typing classes (it was seriously discouraged).

    I wish Oakland Tech was still operating all this today – maybe they are, I don’t know. I doubt it. I will remember those summer school classes as being what a large Urban School should be.

  • http://perimeterprimate.blogspot.com Sharon

    Nextset: You’ll appreciate hearing that students who attended Bret Harte Junior High in the late 1960’s also wax poetic about OUSD’s cinnamon rolls. A few years ago, I was involved with hosting that school’s 75th anniversary and heard all about those rolls from a number of alumni. Wish I had been there.

  • Pingback: Innovations in Education - Knowledge Management | 6 World Education Blog()