Designing a Test that Works

While the Internet has made it easier than ever to share resources with other teachers and find pre-made tests online, it has also made it easier than ever for our students to find these same tests (or worse, answer keys or Quizlets with answers directly from the test). Alternatively, as we try to engage our students and bring more diverse voices into our curriculum, we sometimes end up teaching newer texts that don’t come with pre-made tests. After a few bad experiences with Internet tests, we have largely written our own, and through years of trial and error, we’ve developed a set of guidelines for designing a test that helps us create fair tests that measure student understanding.

Designing a Test: 11 Top Tips

Create different sections.

When designing a test for skills-based units, we usually include vocabulary (or literary terms), grammar, reading comprehension, and literary analysis questions. For novel-based units, we include a character identification section, a quotation identification section (especially for honors students), and a reading comprehension section with a few literary analysis questions mixed in. Not only does this lead to pretty comprehensive assessments, but it also balances easier questions with more challenging questions, which gives all students some sections to feel confident about.

Pro-Tip: When choosing quotations, try to find ones you have either focused on in class or that reflect character traits students should recognize. The section holds students accountable for reading the actual text rather than a summary, but it should still be doable: we don’t memorize quotations from the texts we read (at least not intentionally), so we shouldn’t expect our students to do so.

Cover the entire text or unit evenly.

Sometimes, it’s easy to write so many questions about the beginning that we only have room for a couple about the end. When we decide how many questions we will write about each part of the text before we start, we tend to create a more comprehensive test.

Balance familiar and less familiar questions.

We like to include some questions from the student assignments and class discussions, which rewards those students who have done their work and participated, but we also like to include a few less familiar ones to hold students accountable for reading the entire text. We do, however, try to avoid focusing on nitpicky details in these less familiar questions.

Let the computer create your answer key.

We use a random list generator to create our answer key to ensure there is no pattern students could guess while avoiding the headache of trying to make sure we have sufficiently mixed up the answer choices on our own. This is especially helpful if you’re making multiple versions.

Use student responses throughout the unit to guide your assessment.

The first time we write a unit, we usually don’t write the test until the end. We like to pay attention to students’ written work and class discussions to get a feel for common areas of misunderstanding. This holds students accountable for paying attention to class discussions and the feedback you provide on assignments, and it helps you write detractors that are “tricky” but not unfair. 

Make sure students read instead of watch.

If there is a movie (or, increasingly, a television series) based on the text, we try to include a couple questions about aspects of the book that are different from the movie, just to ensure students are held accountable for reading, not merely watching. This is especially helpful when you have shown the movie in class.

Keep the “gimmes” to a minimum.

While it’s nice to throw in a couple easy questions for students, we try not to include more than a couple obvious questions. We want our students to succeed, but we also want to make sure they have to think a little bit and aren’t able to get by on an Internet summary and class discussion.

Draw students’ attention to important words.

It’s amazing how often students overlook words like “not,” “except,” and “select all that apply.” Following the lead of test creators like the College Board, we make a point of bolding these words when designing a test to help students avoid test-taking mistakes. Since our goal is to assess student understanding rather than test-taking strategy, we usually tell students how many correct responses to look for unless the question is exceptionally straightforward or something we focused on extensively in class.

Be mindful when you’re chunking a matching test.

There are definitely times when we have wanted to ensure students are aware of subtle differences between words, but there are also times when that’s just unnecessary. Pay attention to terms with nearly identical meanings when you’re chunking a matching test. If it’s not absolutely necessary for students to notice subtle differences, give them a break and put those words in different sections.

Write detractors that are fair but do actually detract.

One of the hardest (but strangely most fun) parts of designing a test is writing detractors. When writing detractors, do your best to make sure there is something clearly wrong about each one but that there is only one detractor that is obviously wrong. We try to include possible misinterpretations of the text or easy “skimming” mistakes as detractors.

Entertain yourself, but don’t be self-indulgent.

Designing a test is tough work, and we’ve written our fair share of tests about coffee and included detractors that make us chuckle, but it’s easy to get carried away or focus on amusing yourself or your fellow teachers who will be using the test rather than the students who will be taking it. Creating a challenging, yet fair test for students should be your top concern.

What tips and tricks have you found useful as you design multiple-choice assessments to use in your classroom? Let us know what we’ve missed or haven’t thought of yet by emailing us at [email protected] or DMing us @threeheads.works on Instagram! Consider creating a list of guidelines for use in your PLC so you can share the load while maintaining a consistent style. If you’re interested in hearing more about all things classroom assessment, be sure to check out our podcast episode, “Navigating Assessments in the ELA Classroom,” and our YouTube video, “Helping Students Study for Your ELA Classroom Assessments.”