How well are we doing in comprehension of text as a nation? You know the answer. We are doing poorly when it comes to genuine comprehension:Screen Shot 2015-02-11 at 9.16.48 AM

And look at math vs. reading:

Screen Shot 2015-02-09 at 5.44.24 PMAnd this, from a Christian Science Monitor article on 12th grade NAEP results in reading:

In many ways, the 2013 reading scores for 12th-graders were even more discouraging [than the lack of progress in math]. While the average score of 288 was unchanged from 2009 – and two points higher than in 2005, which represented a nadir for the reading score – it was lower than the average of 292 back in 1992. 

A full 25 percent of 12th-graders in 2013 scored below basic, compared with 20 percent in 1992, and just 37 percent scored at or above proficient, compared with 40 percent in 1992. Those scoring at the proficient level could answer questions requiring them to recognize the paraphrase of an idea from a historical speech and the interpretation of a paragraph in such a speech.

Screen Shot 2015-02-11 at 11.42.06 AM

[I later added the NAEP chart on 12th grade reading, just above, since my focus here is on older students, and the trend is so dismal].

What should we infer from the data?

Numerous causes and their implied solutions, as readers know, have been proposed for flat reading scores: poverty, low expectations, inadequate background knowledge, an anti-boy bias in schools (especially in terms of book selection), IQ links to reading ability, computer games, TV, etc. etc.

The utterly flat national trend line, over decades, says to me that none of these theories holds up well, no matter how plausible each may seem to its proponents. Perhaps it’s time to explore a more radical but common sense notion: maybe we don’t yet understand reading comprehension and how it develops over time.

Maybe we have jumped to solutions before understanding the problems of naïve and superficial comprehension. Maybe we still haven’t specified, in diagnostic detail, what real readers do when they supposedly read books and articles and try to comprehend – regardless of what “good readers” supposedly do.

At the very least: it is a good time to question the premise that we understand the problem.

The black box. Reading is the hardest thing in the world to teach and assess because the reading mind is a black box: we cannot see inside the mind to see what people are doing when they read. We can only infer what readers are doing from what they tell us, write us, and show us. But what they tell, write, or show is neither direct nor necessarily valid evidence. (Maybe they read well but write or speak poorly, for example). Self-reporting of mental states is often inaccurate; expert readers may have forgotten how they came to learn to read for understanding; young readers may lack metacognitive ability and language to describe their reading as it unfolds.

In later posts I plan to share the surprisingly thin evidence about how readers actually read when they try to comprehend text. For now, I merely ask you to keep an open mind.

My own trials and tribulations with reading. I have always been puzzled by the idea of what it means to read because I was a poor reader in school – without realizing it! For a long time I got by. My analytic skills and basic smarts hid the fact, from me as well as my teachers in school, that I could not do what we now call close reading and core comprehension acts, such as summarize an author’s point, state the “moral of the story,” or identify the key “moves” in an argument (and pose questions about those moves). There were hints: an English teacher who said I was “too literal” a reader (without telling me how to be otherwise); a teacher who said I “didn’t seem to be thinking” about what I was reading.

It wasn’t until much later – in college – that I became more aware of my failures as a reader and more self-conscious of how limited my take on “reading” really was. The “reality therapy” feedback became unavoidable: I was put on probation at the end of my sophomore year. (A sad irony in that I was attending St. John’s College, the so-called Great Books school.) Yet, I knew from seminar discussions and feedback on papers that my performance was weaker than it should have been. But I still didn’t understand exactly why or how. As in most high schools and all colleges, alas, I was not taught to read hard books, I was merely assigned them.

I didn’t realize that I wasn’t reading properly until my junior year when I serendipitously spotted a book on a friend’s shelf in his dorm room: How to Read a Book by Adler and Van Doren. Within a few minutes of skimming the first chapters I was bolted intellectually awake: the authors were vividly describing my bad habits as a reader and offering some clear and easy-to-follow tips on how to remedy them when faced with challenging texts. I was helped to realize that when I “read” I merely scanned words passively; I took no steps to converse with the text. Slowly but surely my reading improved by following their advice, the gist of which was to force oneself to ask and answer certain probing questions of the text, in writing, in the margins. To comprehend better is, in part, to force oneself to think more effectively.

Vital hints in the literature on literacy. Thus, as my wife puts it, when is the problem of incomplete comprehension one of reading and when is it one of thinking? What is well-intentioned but ineffective reading/thinking, and how can one recognize the problem as such? This passage from Kylene Beers really resonated with me when I read it last year as part of my multi-year action research project on this topic:

I once thought that if my students could make an inference, any inference, then my teaching woes would be over… The problem with comprehension, it appeared, was that kids couldn’t make an inference.

I shared this frustration with Anne [the Principal]…. I stood leaning against her office door, complaining that that the kids she had given me that year could not make an inference. She simply replied: “Well, teach them.”

“Teach them what?”

“Inferencing. Teach them how to make an inference.”

“You can’t teach someone how to make an inference. It’s inferential. It’s just something you can or can’t do,” I said, beginning to mumble.

“Tell me you don’t really believe that,” she said….

I began to wonder just how I would teach inferencing. It took years for me to get a handle on that one.

As a teacher I, too, saw that some of my bright students could not read for meaning, although I often couldn’t figure out their problems. I could only say “Re-read it carefully and take notes” or some other advice. They, like I had, protested that they had “re-read” and “done the readings.” Yet, even after re-reading and taking notes they were often unable to draw critical inferences about the text as a whole. What, then, were they doing when “reading”?

Oscar’s desire to drop my elective answered the question for at least one student – in a shocking way. When I asked him what his troubles were as a reader, he replied: “I just cannot memorize all the pages you are assigning!”

Ouch. He really did believe “reading” was “scanning and memorizing for later recall.” But wasn’t that, in its own way, what I had done much of my academic life? He at least worked hard to memorize it all! I hadn’t even done that.

Chris Tovani offers a snippet of dialogue that underscores our need to better understand readers trying to understand:

“Luke, why don’t you try to get unstuck.”

“Nothing helps me. Re-reading is a waste of time.”

“Try another fix-up strategy, then.”

“What’s a fix-up strategy?…When I was younger I tried sounding out word out but that didn’t really help.”

“Did you learn to do anything else?”

“No, not really.”

“Does anyone else have a strategy he or she can suggest?”

“I don’t do anything,” brags Kayla.

“You don’t do anything?” I asked.

“Nope. I keep reading and hope it makes sense when I’m done.”

“And what if it doesn’t?”

“Then, oh well.”

I would venture to hypothesize that for many HS students their reading strategy is “Read on, then, oh well.” But let’s find out.

The “re-reading” strategy is a perfect example of our failure to understand the problem. Why would “re-reading” a passage, by itself, clear up what was confused in the first place? All the re-readers are doing is – re-reading. They aren’t thinking differently about what they are re-reading. As Tovani says, telling someone to “think harder” is useless advice. Yet, “Re-read!” is the same unhelpful advice if we don’t know how to re-read or whether we are re-reading “properly.” Too much of the reading-strategy literature amounts to such glib advice.

My first foray in writing about some of this was in a blog entry two years ago about the reading strategies. Boy, did I get a ton of hate mail from people who thought I had gone to the dark side and allied myself with the non-progressive camp. I had done no such thing, even if my prose was a bit dramatic. I had mostly raised these questions (and tried to clarify terms).

Indeed, a few of the more mainstream and thoughtful writers on literacy have made the very point that I was criticized for, such as Barnhouse & Vinton:

As we pondered what was happening with our strategy instruction, we came to several conclusions. The first was the discomfiting realization that while we were grounding our lessons in real literature… we were, in effect, using those books to practice strategies in isolation…. As it was, most of the students’ connections stayed on the surface level… This led us to the conclusion that some of the so-called comprehension strategies – especially visualizing, predicting, connecting, and questioning – seemed aimed more at helping students develop the habits of active and engaged readers rather than to specifically help them comprehend more than they might have… We would need, in effect, to find strategies for the strategies to ensure that they were used as meaning-making tools, not as end products…

So, let’s go slowly. When you as a teacher of older students tell students to “read” and “re-read” a challenging text, what exactly are you assuming that they should be doing in their heads? What do they assume you are asking them to do when you ask them to read or re-read? And – most importantly – what do you think they will actually do when they get stuck? Is Tovani correct that, regardless of training, adolescent readers will have little or no intelligent approach? (I am not asking what they “should” do but what they will likely do.) Do they use the reading strategies or do they forget all about them or approach them randomly or revert to some other bad habit or naïve approach?

Monitor yourself as a reader. What, in fact, do you do when you read challenging text? What do you do when you do not understand on first pass? Try a little test, below: “read” the following paragraphs, then post a comment about what you were doing as you were “reading.” Do not tell us what you think the passage means; tell us metacognitively, as best you can, what you believe you were doing with your eyes and thinking with your mind to try to determine the meaning of the text.

Unlike the famous ambiguous passage about “piles of things” developed by Bransford and Johnson for use in cognitive research (which we analyzed in Understanding by Design), this is a real text: the very first pages of the Introduction to Kant’s Critique of Pure Reason. What is Kant saying here?

THERE can be no doubt that all our knowledge begins with experience. For how should our faculty of knowledge be awakened into action did not objects affecting our senses partly of themselves produce representations, partly arouse the activity of our understanding to compare these representations, and, by combining or separating them, work up the raw material of the sensible impressions into that knowledge of objects which is entitled experience? In the order of time, therefore, we have no knowledge antecedent to experience, and with experience all our knowledge begins.

But though all our knowledge begins with experience, it does not follow that it all arises out of experience. For it may well be that even our empirical knowledge is made up of what we receive through impressions and of what our own faculty of knowledge (sensible impressions serving merely as the occasion) supplies from itself. If our faculty of knowledge makes any such addition, it may be that we are not in a position to distinguish it from the raw material, until with long practice of attention we have become skilled in separating it.

This, then, is a question which at least calls for closer examination, and does not allow of any off-hand answer: whether there is any knowledge that is thus independent of experience and even of all impressions of the senses. Such knowledge is entitled a priori, and distinguished from the empirical which has its sources a posteriori, that is, in experience.

The expression ‘a priori‘ does not, however, indicate with sufficient precision the full meaning of our question. For it has been customary to say, even of much knowledge that is derived from empirical sources, that we have it or are capable of having it a priori, meaning thereby that we do not derive it immediately from experience, but from a universal rule — a rule which is itself, however, borrowed by us from experience. Thus we would say of a man who undermined the foundations of his house, that he might have known a priori that it would fall, that is, that he need not have waited for the experience of its actual falling. But still he could not know this completely a priori. For he had first to learn through experience that bodies are heavy, and therefore fall when their supports are withdrawn.

In what follows, therefore, we shall understand by a priori knowledge, not knowledge independent of this or that experience, but knowledge absolutely independent of all experience. Opposed to it is empirical knowledge, which is knowledge possible only a posteriori, that is, through experience. A priori modes of knowledge are entitled pure when there is no admixture of anything empirical. Thus, for instance, the proposition, ‘every alteration has its cause’, while an a priori proposition, is not a pure proposition, because alteration is a concept which can be derived only from experience.

After reading the passage, consider the following questions:

  • Using arrows and labels, what was the visual “itinerary” of your reading. Where did your eyes go to, when, and why, as you read?
  • What questions, if any, did you ask yourself as you read? Mark the places on the text.
  • Where did you get stuck, if any place? (Mark the text) What, then, did you do, if you got stuck?
  • If you could not “unstuck” yourself at each moment of being stuck, what did you do next – and why?
  • Which “reading strategies” did you use when (without my having prompted you to use any)? If not, why not, do you think? If so, which ones did you consciously choose and why?
  • Where /when did you start feeling dumb/frustrated if at all. What did you do/feel about it if you did? If you quit, say where and why.
  • On a scale of 1-4, how confident are you of your understanding of Kant’s opening setup of his inquiry?

After this self-monitoring, take this formative quiz:

  • Circle the 1-2 key sentences in this selection, and be ready to explain why you are confident that they are key even if you are not sure what they mean.
  • Title this reading and be ready to explain why you gave it that title
  • In a sentence, state what Kant intends to explore. And speculate as to why he might want to explore such a question.

Here is one commenter’s stab at the exercise: Kant Reading Exercise

As I have long said, we give far too little feedback and too much advice. Beers, for example, has a nice chart in which she describes in general form what non-comprehenders do, but it is very brief and general, e.g. “can read all the words but consistently has difficulty asking questions, creating questions, discussing the text, doesn’t “see” anything in his mind while reading, thinking beyond literal questions…” and the book focuses on advice. I am calling for a far more intense look at what non-comprehenders do. Otherwise feedback and advice are easily too generic.

But to give feedback you have to somehow “see” what the reader is doing – which we said above is very hard. It is time, however, to pause in offering non-comprehending readers so much advice and to spend more time in trying to figure out what readers are actually doing in their heads when they supposedly “read.” We might then, like good coaches, offer highly specific feedback based upon what was working and what wasn’t in the readers’ attempts; and only offer specific advice about what to do, based on the specific attempt and the feedback. We would thus want to do far more ungraded comprehension and self-reporting quizzes than we now do.

In follow-up posts, I plan to review briefly the literature from the past 30 years on what readers do (including look at some of the eye-tracking research) and explain why I believe a core premise behind the teaching of the reading strategies is flawed: just because “good readers” do certain things, doesn’t mean we understand how to improve “weak” readers. The strategies – e.g. visualize, predict, connect, re-read, infer, etc. – may only be correlated, not causal. (And, as I will again argue, some of the so-called strategies simply do not pass muster.) So it should not surprise us that reading scores do not improve much if the strategies are taught and learned. Finally, I will have some reports from teachers who have volunteered to give the previous survey or another like it to their students to see what we can learn from just studying kids trying to understand.

If you teach English, try out some of the self-monitoring questions I proposed that you consider, above; and report back to us!


In response to the first few comments and emails:

PS: Yes, I actually do understand the Kant passage – after dozens of readings and multiple under-graduate and graduate classes on Kant. I didn’t really understand the passage – even after a dozen or more re-readings! – until I was helped to understand what the point of the Critique of Pure Reason was, and how Kant was arguing with Hume’s view about what we can understand. That is the paradox of reading, to me: you cannot understand the part until you understand the whole and the “great conversation”; but you cannot understand the whole unless you understand the parts through close reading.

What, then, reading and English teachers? How do we better help students understand what they do and do not understand? How can they better self-assess their degree of understanding – and use that feedback to better understand?


PPS: There are some fabulous suggestions in the comments thus far for how to address the problem of poor comprehension. But note my caution in reply and the focus of this post: what unsuccessful readers actually do is what we need to understand better, first. Then, we’ll be in a far better position to weigh and propose solutions that are valid and personalized for different readers.