Thursday, April 12, 2012

Keeping Our Eyes on the Testing Prize

By Richard Firsten
Retired ESOL Teacher, Teacher-Trainer, Columnist, Author

All during our years of education and career training, one of the most important lessons driven home was that we need to stay focused, we need to keep our attention on what we’re doing, and we need to make sure not to get distracted. Nobody would disagree with any of this, right? Well, sometimes it’s not so easy to keep our eyes on that prize when it comes to testing.

One area where this became obvious to me in my early years of teaching ESOL was in testing listening comprehension. I realized that the tests I was given to use, created at various US universities and sold to schools like the ones I taught at, weren’t exclusively testing what they were supposed to be testing. Those tests almost always ended up inadvertently testing reading comprehension along with listening comprehension. Was that fair to my students? Not at all! If I want to test reading comp., I’ll test reading comp. But if I want to test listening comp., well, I shouldn’t be making my students read three or four sentences quickly and decide which written item reflects something they’ve just heard. I’m sure you see my point. For example . . .

The students hear: “There are quite a few students who have won scholarships this year.”

Then they quickly have to read the following and choose which sentence reflects what they’ve just heard:

  1. Many students won scholarships this year.
  2. Few students won scholarships this year.
  3. A relatively large number of students won scholarships this year.
  4. A few students got one scholarship this year.

The correct answer is 3, but it’s obvious that it wasn’t just listening comprehension that’s been tested; reading comprehension has been tested as well – and that’s just not fair.

So how do we overcome this problem of inadvertently testing reading when we really just want to test listening? It’s actually quite a challenge to accomplish because we need to rely on visuals, not reading, to avoid the problem. We need to come up with drawings, photos, and other kinds of graphic material that will reflect and not reflect each item that the students have heard. Here’s an example of what I mean.













The students hear the sentence and then look at the two pictures. They decide which one reflects accurately what they’ve just heard and that’s how they choose their answer. No need to inadvertently test reading comprehension, right?

Here are six more examples that I’d like to share with you.



















































So if you do test listening comprehension, do your best to find test items that will not accidentally test  reading as well. True, sometimes a small amount of reading can’t be avoided, such as in the “How much does it cost?” example  above, but I know you understand my point, and I hope you’ll be able to give your students fair listening comprehension tests if you decide to test them on this skill.

One other thing to keep in mind is the complexities of grading your students’ work in a writing class. How do you decide beforehand on how you’ll grade their work? Do you base your grading solely on their skill with the writing form you’ve just taught, e.g., a business letter or a basic composition, or do you grade them at the same time on the mechanics, such as punctuation and capitalization? And what about their grammar? Is it appropriate to judge them on their grammar in a writing class that focuses on forms of writing? Not such easy questions to answer, are they?

Well, I hope I’ve given you some food for thought. Let’s keep our eyes on the testing prize.


Comment from Rami
May 3, 2012 at 5:10 am

Mr. Firsten,
Thanks for the informative post.
I couldn’t agree more on the “unfairness” of testing the students on the reading comp. while testing their listening comp.
However, I think using the visuals you suggested wouldn’t offer THE solution. It might be a part OF it, though.
This is to raise the point about the “abstract” & how it will be tested.
I think, a lot of factors are at play here. Time itself is one.
Test designers have to keep all these in mind.
Pointing it out is a huge step, i think, in raising awareness about the whole matter.
Thanks again.

Comment from Richard Firsten
May 3, 2012 at 10:20 am

Thank you for your comments, Rami.

I’d love to hear any suggestions you have on how to make testing listening comprehension without interference by testing other language skills. You state that visuals aren’t the total answer. I hear you. What other means do you think would help?

Comment from regim hotelier brasov
May 29, 2012 at 10:33 am

I just bookmarked » Blog Archive » DSC00247 on Digg, hope this helps get you more visits. I’m all over this using this type of write-up, I truly believe this site requires more consideration. I’ll likely be returning again to see a great deal more, thanks for the blog post it is superb

Comment from Richard Firsten
May 29, 2012 at 2:43 pm

And thank YOU very much for your enthusiasm and interest in “Teacher Talk”! The more visitors, the merrier! 🙂

Comment from regim hotelier iasi
June 7, 2012 at 3:23 pm

Hi there, I discovered your website by means of Google while searching for a related subject, your site got here up, it appears to be like good. I have bookmarked it in my google bookmarks.

Comment from Richard Firsten
June 7, 2012 at 4:43 pm

Well, welcome to and to “Teacher Talk”! We all hope you’ll visit us again and again. : )

Comment from bradvines
June 19, 2012 at 1:08 pm

Your: “There are quite a few students who have won scholarships this year”, should read, students who WON scholarships, without ‘have’ in front of the past tense ‘won’.
(Not shouting; can’t underline.)

Comment from Richard Firsten
June 19, 2012 at 2:14 pm

You’re absolutely right! The tense or aspect of the verb in the sentence that the students hear should agree with the distractors and the correct answer. Unfortunately, that’s the way this item was published in a standardized so-called listening test.

It’s interesting to note that using both the present perfect (have won) and the simple past (won) is fine. The present perfect communicates that the year isn’t over yet and more students may win scholarships, while the simple past technically makes us think that the year is ended or that no more scholarships are available this year.

Of course, the focus is on the phrase “quite a few students” and what that means. Thanks for noticing that goof. I did too, but didn’t want to alter the original.

Leave a comment on this post