Comments on: The elephant in the race

By: Sharon

Sharon — Wed, 21 May 2008 20:08:14 +0000

Thank you, Steve. I am going to study your posting and hope that others will, too.

By: Steven Weinberg

Steven Weinberg — Wed, 21 May 2008 05:36:12 +0000

Thank you for the compliment, Sharon. I have been watching these test scores from several different positions over a number of years. One thing I’ve learned is that there is rarely a one-to-one correlation between education program changes and test scores. Just when you think you have a pattern figured out, the next year’s data doesn’t support your hypothesis. Fair measurement is also made more difficult because the tests are not the same every year. Last year for example, seventh grade writing test scores shot up throughout the state. It took me months to discover that the state had changed the weighting it gave to mechanical errors in scoring the tests, so the improvement was not a sign of writing improving, it was a sign of the scoring getting easier.

The easiest way to improve test scores is to stop doing something that you have been doing that artifically depressed the test scores in previous years–things like telling students that the tests don’t matter or having students test too long each day. I think that state-wide people have become convinced of the importance of these tests, and scores have improved somewhat because of that. Teachers are also more familiar with the format and content of the tests, and that helps them prepare the students better. Textbooks are also better aligned to the tests, which helps some.

Just using the STAR test results it is impossible to tell whether there are any real improvements in students’ abilities, or if it is just higher scores on this one test. I think that most of the improvement in test scores is artifical, based on changes in how components of the tests are weighted and not real improvements in students’ knowlege, but I think the jury is still out on the question of whether some of it reflects real improvement.

On your question about the achievement gap, I have two observations. First, looking at the AMO only counts the number of students Proficient and above, so if a group had lots of students just below the Proficient level, that group would be more likely to show improvement than a group where most of the students are Below Basic, so I don’t think it is surprising that better performing groups would show a bigger increase in the percent of students Proficient than lower achieving groups. For that reason, I would rather look at the API, which measures all levels, rather than AMO. API improvements since 2002 base are +90 for the entire district, +63 for African-Americans, +94 for Asians, +122 for Hispanics, and +76 for whites. I think some of the gains for Hispanics and Asians represent a decrease in the percentage of students in those groups who are recent immigrants. But we are still left with the question of why African-American students are showing smaller gains than any other group. One thing I noticed is that the biggest drop in enrollment for any group between 2002 and 2007 was for African-Americans. Based on my observations at my school and conversations with African-Americans who work there, Oakland is being seen as more and more dangerous for African-American young people, and those families who can afford it are (or were until the mortgage meltdown) leaving Oakland for Tracy and points east. If you remove some of the highest performers from any group, that group’s scores will suffer.
Those are my best guesses as to what is going on, but I’m open to other interpretations. There are too many variables to be certain about any of this.

By: Sharon

Sharon — Tue, 20 May 2008 23:17:10 +0000

Steve: As far as I am aware, you are the sharpest person in OUSD about testing and the data. What is your opinion about OUSD’s increased test scores, API’s, etc.? To me they seem to be a result of “standardization” methodologies imposed by the state, and then compliance in the classroom. Has all the “reform” stuff in OUSD made much of a difference at all?

Test scores have gone up, but more for the higher performing subgroups than the lower ones. As a result, OUSD’s achievement gap has actually INCREASED over the past five years!

What’s your take on this?

Subject: English Language Arts AMO’s

1. The gap between Asian and African American students increased 13.1 percentage points. The gap was 16.5 percentage points in 2002 and 29.6 percentage points in 2007.

2. The gap between White and African American students increased 2.4 percentage points. The gap was 51.8 percentage points in 2002 and 54.2 percentage points in 2007.

3. The gap between Asian and Latino students increased 12.9 percentage points. The gap was 21.6 percentage points in 2002 and 34.5 percentage points in 2007.

4. The gap between White and Latino students increased 2.2 percentage points. The gap was 56.9 in 2002 and 59.1 in 2007.

Subject: Mathematics AMO’s

1. The gap between Asian and African American increased 11.0 percentage points. The gap was 30.2 in 2002 and 41.2 in 2007.

2. The gap between White and African American students increased 4.6 percentage points. The gap was 48.6 in 2002 and 53.2 in 2007.

3. The gap between Asian and Latino students increased 6.1 percentage points. The gap was 30.6 in 2002 and 36.7 in 2007.

4. The gap between White and Latino students decreased 0.5 percentage points. The gap was 49.2 in 2002 and 48.7 in 2007.

So rather than the achievement gap showing any bit of closure, it has increased by the average of 6.5 percentage points over the past five years.

By: Holly at Oakland Small Schools Foundation

Holly at Oakland Small Schools Foundation — Tue, 20 May 2008 19:12:12 +0000

I apologize for my mis-statement about OUSD being the only large urban district to show academic growth three years in a row. Indeed, most of the state’s large urban districts have improved their API scores over the last three years. Of the all districts with greater than 20,000 students, though, OUSD has improved the most over the last three years. (See the chart in the 2007 OUSD annual report – front page of district website — that illustrates this.

By: Sharon

Sharon — Sun, 18 May 2008 16:56:03 +0000

I, for one, am truly grateful for any format (particularly the internet) which allows members of the community to learn more about political candidates and to share their concerns about those candidates with each other.

Mr. Spees’ unfortunate use of the use of the word “chatter” to indirectly refer to those who are expressing their deep concerns about the qualifications and motives of a candidate, in this case the one he supports, is simply insulting.

Chatter, by definition, means “idle, trivial talk.”

Yes, there have been expressions of concern about this candidate who was for years an outsider to the public school system and is now a newcomer in the public school community, who has been working for the past few years (by way of personal connections) in the inner depths of a state-appointed administration which has excluded most public input for the past five years, who is well-connected to that leadership and others because of family wealth and power, and who now has collected endorsements from members of a tier of power players (politicians and businessmen) who most assuredly associate privately with one another to discuss how they want the public schools should be run.

By what Mr. Spees calls “chattering” about all of this, only means that the smart Oakland electorate is being incredibly responsible in their role as citizens in a democratic nation, and are just doing their job.

By: Caroline

Caroline — Sun, 18 May 2008 15:46:38 +0000

Still wondering where Holly’s inaccurate claim about Oakland test scores came from. As a media critic as well as an education wonk, I am interested in tracking the spread of misinformation. If Holly’s not following this thread, I’ll give her a call tomorrow.

By: Steven Weinberg

Steven Weinberg — Sat, 17 May 2008 21:55:25 +0000

On the topic of the June election, I just received a very unfair hit piece against Loni Hancock. I’m not sure for whom I’m going to vote in that race; I admired Wilma Chan when she was on the school board, but I still wanted to explain how misleading the anti-Hancock flyer is.
The flyer accuses Hancock of lower expectations for California students because she favored changing the score required to be considered Proficient on California STAR tests. To evaluate her position one needs to understand how the Proficient level was originally set, and how it is now being used under No Child Left Behind.
When California began its STAR testing program, in 1999, the state used the word Proficient for the second highest band of test scores, roughly equivalent to the 66% to 90% percentile. If it makes any sense to assign the term “grade level” to any band of test scores, it would have been the middle band, 33% to 66%, which is called Basic. (The highest band is Advanced, and the two lower bands are Below Basic and Far Below Basic.) California set the goal of bringing all schools up to an average score of Proficient, with a yearly goal of raising scores by 5% of the difference between the current score and the goal at each school. The state has supplied extra funds to schools which are required to make the greatest gains, and does impose extra control over schools that do not make yearly progress. The high level set for the Proficient band made some sense because it was a long-term goal, and incremental progress was acceptable.
When No Child Left Behind passed in 2001 it said that all students needed to score Proficient by 2014 and that states would set their Proficiency levels and timetables to reach that goal. Most states set their Proficiency levels at something that would have fallen within our Basic range. Texas, which was the starting point for No Child, set it at about the bottom of our Basic range. To keep from having to redefine Proficiency, California kept its high standard (unreasonably high for No Child purposes) and set a timeline that required a very low percentage of Proficient students in the earliest years, but then increased its demands sharply from 2008 on. With these new higher requirements, so many schools will be classified as underperforming under No Child that the state will no longer be able to distinguish a school making reasonable progress from one that is truly failing.
It is not reasonable to expect every student to perform at a level that only one-third of students could perform at a few years ago. Bringing California into line with most other states is a reasonable step to take until the No Child law is reformed. Hancock should be commended for trying to do what is right.

Does anyone know anything about the group that put out the mailer: Education Leaders for High Standards Independent Expenditure Committee.

By: Dick Spees

Dick Spees — Sat, 17 May 2008 18:25:41 +0000

Dear Katy,

Since you referenced me, I would like to point out that I was elected six times to the Oakland City Council, first time at-large, by never less than 68% of the vote. I believe it is an example that party has little to do with the election to a non-partisan office. The Oakland electorate is very smart and will vote for the candidate who can get things done rather than those who politize and divide.

Brian Rogers is a teacher and a business trained leader who will work collaboratively to bring passion,innovation and creativity to Oakland Public Schools. I have enthusiastically endorsed Brian Rogers, not because the other candidates are not worthy, but because I know we have an historic opportunity to elect someone who will bring the resources, experience and personality to build a world class school system.

It is my fervant hope that voters will disregard the irrelevant and inaccurate chatter and vote for Brian Rogers June 3. He is the candidate who will truly make a difference.

Dick Spees
Oakland City Councilmember 1979 to 2003

By: Caroline

Caroline — Fri, 16 May 2008 19:45:24 +0000

Holly, I’m curious where that claim about Oakland came from. I’m sure it wasn’t a deliberate falsehood, but what gives?

By: Caroline

Caroline — Fri, 16 May 2008 05:39:30 +0000

Holly says: “OUSD is the only large district in CA that has had overall student improvement for three years in a row)”

San Francisco Unified School District
District growth API 2007 763
2006 755
2005 745
2004 724
2003 712