Is Assessing Reading Comprehension Even Possible?
All across the English-speaking world, legislation has demanded a greater emphasis on assessment. Specifically, predictive reading screeners. A worthy endeavor. After all, early intervention is key to avoiding reading failure, and we need quick and reliable ways to assess students in order to prevent reading struggles. But these assessments only help us determine areas of need with part of the Reading Rope–word recognition. What about the top part of the rope–language comprehension? What reliable reading comprehension tests are available to us?
Not many.
Why?
Because when it comes to assessing reading comprehension, there are so many factors at play.
There are just so many variables that go into reading comprehension. The background knowledge a reader brings with them, as well as their vocabulary level, are major factors that can either make or break understanding a text. But of course that isn’t all that goes into it. Ability to phrase and comprehend complex sentence structures, understand pronoun referents, and “book language” that isn’t part of everyday language also matter a lot. And, of course, interest, attention, and motivation play a big role, too.
In other words…it’s a complicated business. It also means that the quick screeners we so often use are never going to tell us this information. Decoding is crucial, of course, but it’s not everything. In fact, many students may show up as “in the green” on these measures but actually struggle to comprehend what they read. These students may well fly under the radar until classroom performance nosedives.
The good news? There are assessments available that aim to measure comprehension. The bad news? Many of them have drawbacks.
Let’s talk about it.
Oral reading fluency measures are correlated to reading comprehension, but don’t measure it. Because students only read for one minute, there’s no comprehension component at all; it’s just a prediction of relative ability to comprehend. But saying words doesn’t mean the words are understood. We have all known word callers who can read anything you put in front of them but don’t understand a lick. Most of the time, unless a teacher takes the extra small step to conduct a running record while a child reads, these only tell us whether a child does or doesn’t struggle with decoding–something we most likely already knew.
There’s no other helpful information, such as how a child phrases sentences, attends to punctuation, or reads with expression. Without the extra step to note reading behaviors, we have no idea what kind of words may have stymied a child–multisyllabic? Unfamiliar vocabulary? Names? A different animal than simple decoding issues.
Then there are MAZE tasks.
These purport to measure comprehension, but they’re little more than an assessment of a child’s proficiency with English syntax. Here’s just one example from the third grade version:
They were going to ride (close, farmer, the) train to the city and visit (he, rock, their) grandparents.
The answer choices are all incredibly obvious, and almost all of the time, just have to fit grammatically. There’s very little comprehension work required. It’s mostly a “what sounds grammatically correct” test.
There are a couple of reading tests that look at vocabulary level.
The CORE vocabulary assessment offers a quick look at this, and I do recommend it. The Fountas and Pinnell Benchmark Assessment (BAS) has a variety of tools to assess vocabulary. Few teachers are aware that every benchmark text includes an optional (and very helpful) vocabulary component, and they have a large selection of additional vocabulary assessments in their optional tools. Concept words, synonyms (just like CORE vocabulary does), homophones, and Greek and Latin roots are just some of the available assessments. They also, for the record, have a lot of phonological awareness and phonics assessments in their kit.
While we’re on the topic of F & P, no, we really can’t and shouldn’t place kids in specific text levels, their assessment adds a deep comprehension component after the running record portion. It’s the part of the test that asks literal, inferential, and evaluative questions. Some of the questions are eyebrow-raisingly difficult, but it’s at least a look at comprehension. I talk much more about what’s both good and bad about this assessment here.
Pioneer Valley has created their own version of a leveled text assessment, which, like the BAS, includes a variety of comprehension questions. Jennifer Serravallo, too, has her own version of this sort of assessment, with a big variety of comprehension questions, including vocabulary. This one also uses authentic, whole books, which is not something the others do, but of course this means it’s a very long assessment. Teacher’s College Advancing Literacy has a similar assessment that gets into the layers of comprehension. The questions are not as deep, but it’s also not as time-consuming, either. And it’s free.
Additional reading comprehension tests:
Online assessments like MAP, STAR, iReady, etc. provide comprehension scores in different areas of reading, but these are often done though a predictive algorithm and not actual performance. Plus, they tell us nothing about why a student may have missed something. We usually don’t even know what what questions were asked, how they were asked, or students’ responses.
There are several Informal Reading Inventories on the market. These are pretty similar to the F & P or Teacher’s College style of assessment, but they ask more literal than inferential questions. The one I used to use at the beginning of my career a million years ago, the QRI, also asks background knowledge questions before the student reads the text, which is not a feature of the F & P or Teacher’s College versions. These kinds of assessments, though, also have drawbacks.
There’s a lot to consider when it comes to reading comprehension!
Looking back at the Reading Rope’s top strands, we can see that many factors go into what makes reading comprehension possible. Reading comprehension, after all, is language comprehension. Which is why it completely depends on the reader and the text. There’s a lot of gray area here, making it extremely difficult to assess, whereas the bottom strands of the rope are pretty black and white–and easy to assess.
Difficult, but not impossible!
There is currently one assessment I know of that every teacher has access to that does give us information about language/reading comprehension, and I cannot sing its praises enough. It’s quick to give, easy to score (once you understand it), and provides information for the entire language comprehension part of the rope. It’s also completely free.
If you haven’t heard of the CUBED-3 assessment, I urge you to learn about it. Learning how to use it and how it’s structured is key to getting the most from it, but it’s an invaluable tool. It’s a narrative language assessment, but includes expository information that most children would not have prior knowledge about. It gives teachers so much insight into a child’s vocabulary, understanding of syntax, story structure knowledge, and both literal and inferential comprehension.
The developers offer quite a lot of training for it, both free and paid (totally recommend the in-depth, paid version), and this training is essential. I’m not affiliated with the company in any way, I just find the tool to be extremely helpful.
The take-away?
Assessing reading comprehension has always been and always will be a challenge because it’s so nuanced. Like an onion, there are many layers to it. Every child brings different levels of understanding (or lack thereof) to every text. But understanding what impacts a child’s reading comprehension isn’t a totally impossible task, and there are tools at the ready to guide us. Hopefully I’ve offered you some resources you can use, though, to give you insight. Combined with the “bottom rope” assessments so many of us are already using, we really can get a comprehensive view of our readers.
Was this post helpful? Subscribe here to be the first to see new posts that will make an impact on your teaching!
For ongoing, quick support, about writing and ALL things literacy, join my private FB group! It’s a community of educators just like you to offer support right when you need it.
Who is Coach from the Couch?I’m Michelle Ruhe, a 25+ year veteran educator, currently a K-5 literacy coach. I continue to learn alongside teachers in classrooms each and every day, and it’s my mission to support as many teachers as I can. Because no one can do this work alone. I’m available to you, too, through virtual coaching calls!
Add A Comment