Can we stop the hysteria about testing, as if asking kids to take tests is inherently harmful and unfair? Perhaps not, but let me try by posing a more focused question in this plea for reason. Can we please stop the bogus logic that says that the growing demand of tests requires that the local response be massive ‘test prep’?
Put a tad sarcastically: would somebody please show me the research that says that the only way to RAISE test scores – and keep them rising – is via mind-numbing bad-teaching ‘test prep’? Would you please point me to any research that says that the best or only way to raise test scores is to teach worse?
I didn’t think so.
Look, accountability is a serious business. And the stakes have gotten much higher over the past 20 years. (Be careful what you wish for: in my day the lament was that nobody cared in the least about education and schools and it was mostly true.) But it is simply false to say that asking students to take tests is essentially wrong. The challenge is to offer kids a fabulous education; then, the test results take care of themselves, as they are supposed to.
A thought experiment. Hold your immediate disagreement; let’s try a simple thought experiment. Close your eyes and visualize the classrooms in the best schools and districts in your state – especially in those schools that are outliers in their demographic. What do you see? Do you see endless and grim test prep regimen, with horrible Gradgrind teachers? Or do you see far better teaching than is found in the low-performing schools, whose only arrow in the quiver is – more worksheets? You know the answer: in the best schools in America, private as well as public, we see more high-level questioning, more intriguing assignments, more constructed response tasks on local tests, and more higher-order instructional approaches than in low-performing schools. And we see high, not low, test scores. How could it be otherwise? That’s validity 101.
In fact, by definition in the best schools in America, local assessments are more rigorous than state tests. The most recent review of the NY State Regents exams only underscores what many of us have known (or should have known) for decades: state tests are not very hard, and there is going to be hell to pay when the new common assessments are rolled out (as NAEP scores have told us for decades.)
Can we consider common sense for a spell, then, without the posturing? The crude response of test prep is understandable in light of local fears, and the harm it does to students and teachers is real. But test prep exists not because there are tests but because educators in weak schools have utterly lost their way. Test prep, in other words, is a compensatory strategy by educators who don’t seem to know what high-quality teaching and learning actually look like, and/or they don’t have faith in the power of a great education to cause good test results.
What we should then be freaking out about is that so many educators still don’t seem to know or believe in ‘best practice’, have no access to it in their work, and are under no accountability pressures to get that expertise in their jobs. (My electrician and carpenter work to higher standards: they have to be re-certified every few years on current code and expert knowledge.) Alas, awareness and use of best practice is optional in almost every school in America. Would those of you who are quick to disagree with me here want your child to be taught by someone who prized their freedom to do whatever, over their obligation to find out and do what works best – when current approaches are not clearly working? Would you go to doctors who had such an attitude?
The tests themselves: nowhere near as bad as people seem to believe. I have written various pieces over the last few years on what I have learned from looking at released test items. I refer you to those pieces as a start, in hopes of having a more rational discussion of the proper role, strengths and weaknesses of external testing in each school. If you do what I did – look at all the released tests in those states that do so, such as Florida and Massachusetts – I think you’ll come away thinking pretty much what I did: most test questions in the core subjects of reading and math are fair and appropriate, given the standards. (I confess I am less enthusiastic about some tests in other subjects, but those haven’t been tied to NCLB and other high stakes in most places.)
Here’s the epiphany that I had after reviewing dozens of the most challenging questions (as judged by the patterns of scores): the questions that are most difficult ironically demand transfer of learning, not rote learning – the very aim we all prize. This becomes especially obvious in ELA: the student gets a brand-new reading (or writing prompt) and has to make sense of it, using whatever strategies and skills they have learned, without being told explicitly what to do or without hints from teachers. The questions that are difficult for our kids in reading are inevitably questions at the heart of instruction: ‘main idea’ and ‘author purpose’. (see gallery, below)
Why would teachers who had taught her kids to read well fear such a test? Why are local administrators so intellectually bankrupt in weak schools that all they can counsel is test prep, in light of what the tests actually demand? Why isn’t there a local plan to hammer away at the big ideas of reading across all grade levels, as a team effort, given that the results on these targets have been dismal for years?
Same in math: I didn’t find one question that I thought was unfair or pointless, given the Standards. The questions may not be brilliant, but after a while, they start to seem pretty straightforward and even predictable, and the poor student results start to seem really weird. For example, when there is a triangle and some missing info about sides and angles, you can almost bet before looking at the numbers and the figure that it is either about 180-degrees in a triangle or the Pythagorean Theorem. In each case, this is core content. Yet once again, the results are surprisingly poor.
Here are many instructive examples in ELA and math:
The 64-million dollar question, then is: why are our low-performing kids not taking content we know they were taught and tested on and using it to address a test question that demands it – if you think about the question? The difficulty, in other words, is not in the content complexity but in the thinking demand: do the students know which content to apply, when -in a question that is not so dumbed-down as to basically tell you what to recall and use? That is the point of math: to see if, when faced with novel problems, students can solve them.
The weak results speak volumes to the weakness of instruction and (especially) assessment locally: in their classes kids get few real problems, just low-level ‘plug and chug’ as if superficial drill can bypass the need to think.
In over 30 years of work in schools I have rarely seen solid school-wide and district-wide tests. Rather, in typical local common assessments they often mimic the format of state tests without having the rigor of state tests. In the best schools, however, many individual teacher assessments are often creative, funny, challenging. I challenge you, therefore, to audit your local assessments using these audit materials. ELA blank audit assessment Math blank audit assessment Audit of Algebra I Test. Included is my audit of a good district’s exam in Algebra; you may want to compare it to your own. (I will provide an audit of a fully-released state tests in math and ELA in my follow-up post.)
In that light, it’s worth remembering that Jaime Escalante had no interest in suing ETS or the College Board over his students’ scores on the AP being challenged because he felt that the external tests were key to the raising of the bar at Garfield HS (A point I made via sports in a past blog.) It is also worth remembering that once Escalante proved what was possible, AP scores from many other teachers at Garfield also climbed – even in non-math courses.
You don’t have to agree with me or like what I am saying. But please suspend disbelief until you have investigated all the released state test items, gone to the best schools in your state to see what they do, and (especially) audited local assessment. Then, see if you don’t feel as I do: that the problem is not testing and accountability per se but our unthinking response to testing and accountability in low-performing schools.
I am not criticizing the hard work or intentions of teachers in test-prep schools. I am criticizing the fact that though the methods are ineffective and doomed to be so by simple logic, test prep is all they keep doing. it’s wrong for kids and it’s a thoughtless response to the challenge. Someone locally needs to say that the test-prep emperor has no clothes.
Nor am I saying that good scores = good schools; that reverses what I said. I said that in really good schools it happens that test scores tend to be high – as they should be, if the tests are valid. Nor am I saying that any school with only high SES is a good school. There are countless schools in the suburbs that are not very good at all, i.e. they provide little or no value added, and the work is incredibly dreary – kids only persist out of extrinsic motivation. Such districts receive able kids, they graduate able kids, and not much vital learning happens in between.
Good schools and teachers, and our obligation to learn from them. What, then, do I mean by ‘good schools’ that are about something more than test prep, even in the face of tests? Nothing fancy. I mean the obvious: schools that any of us would want our kids to go to. Highly-qualified staff; a caring environment; where they really know your kid, and play to his/her interests/strengths; where they use endlessly engaging ways of getting kids hooked on a subject; where there is a challenging, yet stimulating curriculum, with worthy assignments; where they demand much of students but give much in the way of support; and whose graduates, regardless of GPAs, go on to make it in the world at something they care about – without being shocked and dismayed to learn that school standards were so low that they are unprepared for anything beyond school. In short, staff are mindful of the tests but not in a panic about them.
Many of you might not so much disagree with me as be disappointed that in this political war we find ourselves in that I would provide aid and comfort to the crazies who would destroy public education as we know it.
But I know this deep in my bones: if we who care about public education keep avoiding reasoned challenges to our beliefs from friends, if we keep dragging our heels on reform and accountability, and if we keep falling back on ad hominem attacks against everyone who disagrees with us, then we are no better than our enemies. And we disgrace ourselves as educators who, more than any other group in society, have the obligation to keep learning and questioning – including the questioning of thoughtless or demonstrably ineffective approaches such as ‘coverage’, and ‘test prep’.
PS: A week later this piece in Edutopia proved my point beautifully.