In a previous post I argued that the dreary and un-improving results on tests of reading comprehension mean we need to take a radical look at what we are and are not doing, especially in middle and high school. I argued in the first post that we know far too little about what real readers actually do when they face challenging text – a predictable problem of great importance in light of Common Core and college.

Here, in Part 2 of a series, I summarize the research on comprehension.

I spent three weeks reading almost everything of note written on the subject since 2000, plus seminal pieces from the 80’s. My chief reference was The Handbook of Research on Reading Comprehension (2009) in which most of the top researchers in literacy summarize the state of the art of current and past research on comprehension.

Except for the bracketed comments by me, every paragraph below is a direct quote from the Handbook (sometimes shortened for readability).

What we know

Skilled comprehenders use metacognitive strategies significantly more often than less skilled readers.

Less skilled comprehenders were significantly less likely to make inferences from text even with the equal background knowledge… This supports the notion that comprehension requires flexible simultaneous consideration of multiple elements.

Younger readers have little awareness that they must attempt to make sense of text; they think that reading is decoding – reading as word understanding. Older readers     [5th and 7th graders] knew what to do but they did not do it. Older students were no less likely to classify their difficulties at the word level than younger students [as opposed to the sentence or paragraph level].

The extent to which children slow down their reading on encountering inconsistent information is a significant predictor of comprehension.

Comprehension monitoring accounted for unique variance, once working memory and other background factors were controlled.

Rather than teaching students how to become self regulated learners, teachers seem to expect behaviors would naturally developed through prompted questions. There is of course no evidence that such prompting leads to anything like active self-regulated use of comprehension strategies.

Improving performance is possible. However there is less evidence that comprehension focused interventions produce either autonomous use of comprehension strategies or longer-term improvements in comprehension proficiencies.

The instructional frameworks described (Question the Author, Reciprocal Teaching, Concept oriented Reading Instruction, Collaborative Strategic Reading, Transactional Strategy Instruction) have demonstrated great potential for developing higher-order comprehension. That said, research continues to describe few classes where children are benefiting from these or similar approaches…. While the research evidence in favor of comprehension instruction piles up, the gap between research and practice remains stubbornly wide.

It is difficult for many teachers to understand the necessity of keeping the content of the text at the forefront while teaching strategies… This [problem] occurs, for example, when teachers only ask students questions about which strategies they used and why, instead of asking questions about the content of the selection.

Decades later Pressley et al. (2006) found essentially the same thing [re: Durkin’s famous study in 1979 that, despite teacher enthusiasm for developing real understanding of texts, no instruction in comprehension was occurring.]. Despite these well-understood findings about the value of teaching strategies, comprehension instruction continues to receive less attention than other skills or content.

[Grant’s note: there is little justification in the Handbook for the most common long lists of Strategies I critiqued in an earlier post. Almost all the Landmark Studies discussed in the research focus on only 3-4 strategies to be taught and used simultaneously, in order to develop a self-regulated repertoire via transfer of learning thru gradual release: Preview, monitor comprehension for breakdown at more than the word level and re-read as needed, draw inferences across the text, identify the main idea of the text, ask questions of the text, summarize.

Note that these are precisely the abilities students do poorly on in tests of reading comprehension.]

What we don’t know

We lack research demonstrating how the ‘automatic pilot’ [of student internalization and self-regulation of comprehension monitoring] develops.

Remarkably few studies focus on typically developing middle-grades readers. Given the unique challenges presented by young adolescent readers and their teachers, there is a great deal of work to be done to further our understanding of these students.

Research in content area literacy/Adolescent literacy has rarely if ever addressed comprehension strategies. Previous research on comprehension strategies has been limited by its focus on younger readers with only very simple tasks.

Too few studies involve classroom teachers providing comprehension instruction, especially over longer periods of time. As Guthrie (2004) has noted, we have many, many studies involving a small number of children being taught a single strategy over a few days or weeks… But only one or two studies are available where public school teachers have provided instruction to large numbers of students on multiple comprehension strategies across 1 or more years.

In our view there is too little research to provide useful guidance for developing complex proficiencies in struggling readers across the school career.

Reading comprehension is an acquisition that involves many components and develops over an extended period of time, with the case made that a comprehensive theory of reading development must be one that includes many more components and many more years than the theories that we currently have. Relatively little work has investigated how expert reading processes develop.

The lack of evidence [about whether strategy instruction transfers] stems from the heavy reliance on smaller sample sizes and shorter-term intervention designs as well as limited attention to a ‘gold standard’ of transfer of training to autonomous use.

Too often traditional accounts assume students want to be higher or high achievers…. The most obvious direction needed is a solid base of empirical studies on self regulated comprehension. Contrary to what many believe, there is not a solid research based now.

3rd and 5th grade students relied almost exclusively on word-level criteria for evaluating their understanding, replicating the findings of Baker more than 20 years earlier. Similarly, Eme et al. found that third graders’ conception of a good reader was one who reads quickly without making mistakes, replicating the findings of Myers and Paris 30 years earlier. [Thus] research findings still look a great deal like they did originally which is quite troubling. Change is slow.

Some thoughts of mine, based on these findings:

1. There is no mention in any of the research to support E D Hirsch’s claims about the key to reading comprehension being better background content knowledge. Indeed, his name is not mentioned in this 600+ page reference book.

2. The lack of transfer of learning of the strategies seems to be a function of inadequate teaching for transfer, and a failure to understand the principles behind the strategies.

3. There is little or no teaching of comprehension/metacognition strategies in 6th through 12th grade because curriculum is written around books to be assigned and the ASSUMPTION that all a teacher needs to do is assign and assess, not teach reading. I once had a high school teacher say to me bluntly – after he complained about poor literacy levels – It’s not my problem: I teach English, not reading.

4. There is really only one piece of research that frames the entire debate about what is possible in adolescent literacy: the work of Palinscar & Brown in the mid-80s on so-called Reciprocal Teaching. This is the most-referenced study, and all of the highly-successful research projects of the past 20 years derive their approach from that study. Every teacher of reading and English should read it carefully, therefore. You can find it here. I also highly recommend Questioning the Author by Beck et al – one of the top-rated approaches – because it provides the most accessible and practical way of showing how to teach students to read more carefully and critically.

5. It seems reasonable to say, from the data and the research, that the reason SES correlates highly with reading scores AND that reading scores are too low and flat over time, is that we are not teaching people to read effectively. The data are clear: except in small research studies and outlier classrooms, all the time we put into literacy does not result in highly-literate students – in spite of what research DOES reveal about how to improve comprehension. The research could not be more consistent on one basic theme: comprehension and self-regulation CAN be improved in ALL learners of ALL backgrounds. That reading results haven’t improved much beyond decoding says clearly, however, that we have not designed ‘backward’ from in-depth comprehension, using what research tells us will work.