How to Test Reading Skills in Any Language – Part II: Fluency and Comprehension

This is Part II of a three-part series on developing tests for reading skills in non-major languages. Be sure to check out Part I: Decoding, Part III: Assessment Start to Finish and How to Determine Text Difficulty in Any Language.

In this series, our goal is to give educators and NGOs working in non-major world languages the ability to test the literacy skills of their students. In Part I, we discussed the skill of decoding and how to design a simple spelling test that can measure a student’s level in that skill. In Part II, we’ll discuss the two more commonly recognized skills: Fluency and Comprehension.

While we can’t have one assessment that works universally, we can identify basic rules that make it possible to design reading assessments for any context. After reading this post, you should have a basic idea of how to create and use assessments that accurately measure the skills of your students.

For both of these assessments, you’ll need texts that are at the appropriate level for the readers you’re testing. If you don’t have any leveled texts, or fear that they may be off the mark, check out this article on measuring text readability in any language.

Fluency

Once a student has mastered decoding and can mentally transform characters on a page into the sounds that make up a word, the next step to becoming a strong reader is to do this quickly and effortlessly. Fluent readers do not treat every new word as a puzzle, painstakingly dissecting it into consonant and vowel sounds. Instead they are able to automatically decode a word without any conscious effort.

Fluency is a skill, like any other, that is developed slowly over time. It often follows right on the heels of decoding, although many an argument has been waged over how naturally one follows from the other and how specifically fluency as a skill needs to be taught.

Testing fluency is actually surprisingly easy. You’ll need an “on level” text (a piece of writing you would expect most students of a particular level to be able to read successfully), a watch, and a place to record the students’ scores. That’s it. Sit down with the students in a quiet place and ask them to read the test aloud for 60 seconds. If they struggle with a word for more than three or four seconds, it’s fine to feed them the word and let them continue, marking the word as an error rather than allowing it to stop them in their tracks.

Their fluency score is the number of words they were able to read per minute minus any errors. Errors include misread words, incomplete words, skipped words, or repeated words. If you feel a student may finish the text before the minute is up, have them start again from the beginning, adding the second reading to the score as well.

It, of course, helps to have a copy of the text for the teacher to read along to with word counts next to each line, as well as a place to mark down the number of mistakes and to record scores. Ideally, this is out of sight of the student (a clipboard does the trick) to avoid making them uneasy during the reading.

Comprehension

So, they can fluently and quickly sound out words, but that’s only half the battle. Do they understand what they’re reading? Without comprehension, reading becomes a parlor trick–something that can be done on command if asked, but not a useful life skill.

Comprehension can be tested with a pen-and-paper test quite easily. In fact, most literacy assessments out there focus largely on comprehension, assuming decoding and fluency is already present. While this is fine for adult or advanced learners (who are expected to already have mastered decoding and fluency) young learners are building all three skills at the same time.

When designing a comprehension test, it’s important to remember that reading narrative and expository texts often calls upon different skills, so including one of each is likely a good idea. Sequential written instructions are also a beast unto their own, and testing a student’s ability to follow and understand basic written instructions is never a bad idea.

Of course, if time is more abundant than physical resources, it’s absolutely possible to test students using a single copy of the text and an oral interview rather than a written test.

Types of Questions

Comprehension tests generally have three types of questions:

1. Cloze

Cloze questions are more commonly known as “fill in the blank.” These sorts of questions are often best used for simple comprehension questions with easily identified answers. A word or answer bank makes this a bit easier, depending on the task at hand.

2. Multiple choice

Multiple choice questions allow you to ask a wider variety of questions without leading students to give wildly divergent or unrelated answers and are also fast and easy to grade.

3. Written/Open answer

While written answer questions can most accurately assess how deeply a reader understands a given text, it can also be very difficult to score. Any time you use a question that includes a written response, it’s best to also have a clear and well-defined rubric that you will use to score them.

Difficulty of Questions

For comprehension assessment, I would recommend leveling the test with three basic types of questions. Three of each of the three types, for a total of nine, should give you a basic idea of a student’s comprehension ability.

Level 1: Search and Fill 

This is the most basic level of comprehension. Students can search through the text for basic, simple bits of information. Questions should be worded similarly to how they are presented in the text. Essentially, these questions let you know that students can understand the most basic and obvious facts from a piece of text.

For example, if a text says:

”Mangoes are one of the oldest domesticated fruits; they were first cultivated in South Asia as early as the 5th century BC”

Questions might include:

1. What century were mangoes first cultivated in South Asia?

2. ________ are one of the oldest domesticated fruits.

Level 2: Understanding

These questions reveal a more complete level of comprehension. Students finish the text with an understanding of its content and meaning, without having to search for specific answers using clues from question wording. Although still clearly found within the text, information asked for takes a different form from how it appears in the provided passage.

For example, if a text reads:

”The mango is generally sweet, although the taste and texture of the flesh varies across cultivars, some having a soft, pulpy texture similar to an overripe plum, while the flesh of others is firmer, like a cantaloupe or avocado, or may have a fibrous texture. For consumption of unripe, pickled or cooked fruit, the mango skin may be consumed comfortably, but has potential to cause contact dermatitis of the lips, gingiva or tongue in susceptible people (see above). Under-ripe mangoes can be ripened by placing them in brown paper bags. They will then keep in a plastic bag in the refrigerator for about four or five days. In ripe fruits which are commonly eaten fresh, the skin may be thicker and bitter tasting, so is typically not eaten.”

-From Wikipedia

Questions might include:

1. Based on the given description, mangoes would make a good ___________.

A. meal

B. dessert

2. How can you ripen mangoes more quickly?

Level 3: Drawing Inferences

This level of comprehension reveals a truly complex analysis of the text. The reader is able to guess the author’s intention, unspoken biases, and different possible points of view that can be taken when reading the test.

For example, if a text reads:

“Aussie Mangoes are the flavour of summer with a range of varieties for you to enjoy all season long, and we’re here to help you make the most of this delicious little fruit. Mangoes are quick and easy to prepare and are bursting with nutritional value.

But best of all, mangoes are one of the most versatile fruits on the market; delicious as a healthy snack on their own, a perfect addition to your favourite chicken and prawn dishes, light summer salads, cocktails and smoothies and sweet desserts.”

-From https://www.mangoes.net.au/

Questions might include:

1. The author’s main goal is to:

A. Inform the reader of the benefits of Mangoes

B. Warn the reader of potential dangers of eating Mangoes

C. Encourage the reader to buy Aussie Mangoes

2. The author is most likely (an employee of / not an employee of) Aussie Mangoes.

Creative Commons Love: Chrissam42, Khedmati, and Bethan on Flickr.com

 

Written by Michael Jones