Now that standardized testing season is upon us, retired OUSD teacher Steven Weinberg offers another critique of the California Standards Tests.
Many people wonder why teachers object so strongly to the use of standardized test results to evaluate their teaching. After all, some would argue, student learning is the whole purpose of education, and these tests are supposed to measure student learning.
One reason teachers object to these tests is that they know what a poor job the tests do in actually measuring student learning. They have seen the tests and know how flawed and arbitrary they can be. It is hard to share this information with the public because teachers are forbidden from divulging the contents of the test to anyone, and they can be penalized severely for doing so.
The only time the public can see any items from the California Standards Tests (CSTs) is when the state releases a few questions that are no longer being used and posts those items on its website. Generally the state is very careful not to release any flawed items, but occasionally one slips by, and those items are worth examining because they show clearly what is wrong with these tests and why it is unfair to evaluate teachers based upon them.
The example I will cite today is from the eighth grade language arts test. According to the chart that accompanies the released items, it is supposed to be measuring how well students learned (and, by implication, how well teachers taught) Reading Standard 3.6 “Identify significant literary devices (e.g., metaphor, symbolism, dialect, irony) that define a writer’s style and use those elements to interpret the work.”
Imagine you are a teacher and have spent weeks teaching significant literary devices such as metaphor, symbolism, dialect, and irony, and then you find your students faced with this question on the test:
In the green field stand the scattered sheep,
and the Shepherd standing
just beyond the field—
and at the Shepherd’s feet, poised,
the rough-coat collie dog, with one thought only.
It is the woolies.
Her eyes, one blue, one brown
never leave them.
Here is the question:
44. Why is line 7 (“It is the woolies.”) in “Sheepdog” italicized?
A. to show this is the dog’s appearance
B. to give the text of the poem variety
C. to indicate that is the dog’s thought
D. to indicate a change in the tone
This question isn’t about a significant literary device such as metaphor, symbolism, dialect, or irony. It is not a question based on any of the eighth grade standards at all. It is, instead, about a rather obscure convention for using italics to show the thoughts of a character. This convention is so unimportant that it is not even mentioned in any of the state adopted texts for middle schools or high schools. (By the way, none of the rules for using italics that are in those books are tested.) No teacher who was teaching the California standards would cover this use of italics, yet it is tested on the California Standards Test.
This question is not unique. There are other flawed items on these tests. In fact, there were two other questions on this same test about this poem that were even worse. I am not allowed to reveal them, because they have not been “released” by the state, but one had two possible correct answers (only one of which was counted as correct), and the other had, as its correct answer, a phrase that made no sense at all (imagine something like “the oxygen-breathing era of United States history”).
How do flawed questions get on such important tests? The tests used in California (and most other states) are produced by private, profit-making companies which are seeking to construct the tests as quickly and cheaply as possible. They have huge data-banks of questions they have created for various purposes over the years, and it is far cheaper to use one of those old questions than create a new one, even if the old question doesn’t really fit the standard that is supposed to be tested.
Teachers have every right to insist that they not be judged by tests that do not match the standards they are charged with teaching. They have every right to demand the same high standards for our tests as we have for our students.