[UPDATE 11/2014: There have been recent reports suggesting that some of Hattie’s math is flawed. I am not in a position yet to judge the validity of the critique nor, perhaps more importantly, to determine the impact of that critique on the overall findings. The author of the statistics critique in fact suggests that the math used to explain effect size may not impact the conclusions. I will keep abreast of the issue and update this post as warranted.]
I have been a fan of John Hattie’s work ever since I encountered Visible Learning. Hattie has done the most exhaustive meta-analysis in education. Thanks to him, we can gauge not only the relative effectiveness of almost every educational intervention under the sun but we can compare these interventions on an absolute scale of effect size.
Perhaps most importantly, Hattie was able to identify a ‘hinge point’ (as he calls it) from exhaustively comparing everything: the effect size of .40. Anything above such an effect size has more of an impact than just a typical year of academic experience and student growth. And an effect size of 1.0 or better is equivalent to advancing the student’s achievement level by approximately a full grade.
The caveat in any meta-anlysis, of course, is that we have little idea as to the validity of the underlying research. In a summary of all research we are agnostic as to how ‘good’ the research is. (For a good critique of Hattie’s approach in particular and meta-analysis in education in general, read this. [alas, the post was removed and is now only available for a fee. Here is some background from NZ on the controversy and the rationale for the critique. Here is a different critique.
Fans of the book may be unaware that a brand new Hattie book has just been released entitled Visible Learning for Teachers: Maximizing Impact on Learning. In this slim but jam-packed book, Hattie takes us through the planning and teaching process, based on what works according to research. He provides a comprehensive set of checklists that reflect what best practice tells us we should consider in planning and teaching. And in an Appendix he provides a simple way for all teachers to gauge effect size of their teaching. Alas, the text is a bit too dense for the average teacher-reader, I think. But there are countless good pieces of advice, if one persists through the tiny print, lack of white space, and lots of data.(You can also hear and see Hattie discussing his research and its import here.)
As in Visible Learning, the (updated) rank order of those factors that have the greatest effect size in student achievement will be of interest to every teacher, administrator, and education professor.
Here is the rank-ordered list of the top effect sizes, with a half-dozen removed by me because they either refer to programs unknown outside of Australia & New Zealand – Hattie’s home base – or they refer to sub-sets of students (e.g. the learning disabled). And I am going to provide a bit of suspense with this list. I want you to guess which two factors come next after what is listed below; you’ll see why I wanted to add a bit of intrigue by the end. (I have also starred the factors that have an effect size of .7 or greater since these are significant gains):
- Student self-assessment/self-grading*
- Response to intervention*
- Teacher credibility*
- Providing formative assessments*
- Classroom discussion*
- Teacher clarity*
- Reciprocal teaching*
- Teacher-student relationships fostered*
- Spaced vs. mass practice*
- Meta-cognitive strategies taught and used
- Classroom behavioral techniques
- Vocabulary programs
- Repeated reading programs
- Creativity programs
- Student prior achievement
- Self-questioning by students
- Study skills
- Problem-solving teaching
- Not labeling students
- Concept mapping
- Cooperative vs individualistic learning
- Direct instruction
- Tactile stimulation programs
- Mastery learning
- Worked examples
- Visual-perception programs
- Peer tutoring
- Cooperative vs competitive learning
- Phonics instruction
- Student-centered teaching
- Classroom cohesion
- Pre-term birth weight
- Peer influences
- Classroom management techniques
- Outdoor-adventure programs
Can you guess the next two items on the rank order list?
“Home environment” and “socio-economic status.”
In other words, everything on the list has a greater effect on student achievement than the student’s background – despite the endless fatalism of so many teachers on this point (especially in the upper grades).
Co-incidentally, Jay Matthews in a recent Washington Post article discusses the remarkable gains in Arlington, VA, in which the achievement gap was greatly narrowed by sustained focused effort by district leaders. And the Gates Foundation released a preliminary report on its Measures of Effective teaching project that shows convincingly what any of us who have worked in schools for years knows: good teachers make a considerable value-added difference.
It is thus high time that we call teacher fatalism about their ability to achieve gains with poor or unmotivated students what it is: unprofessional, passive, and cynical thinking that has no place in school. It is a form of prejudice that becomes a self-fulfilling prophecy.