Should your online course include quizzes? What learning research says about testing students

Yes, almost every online course should include short quizzes after its lessons.

The research on what psychologists call retrieval practice is some of the most consistent in education: pulling an answer out of memory makes people remember it longer than watching or reading the material again, and it helps with applying ideas as well as recalling facts. The version that works is low stakes, placed after the lesson, retryable, and paired with an explanation of the right answer. One caveat runs through all of it. Nearly every study was done in schools and universities, not in paid courses for adults, so treat the size of the effect as a direction rather than a promise.

Why rewatching a lesson feels better than it works

The study most people cite comes from Washington University. Students read a short science passage and then either reread it or took recall tests on it. Five minutes later, the rereaders recalled more. A week later the result flipped: the group that studied the passage once and took three recall tests remembered 61% of it, against 40% for the group that read it four times. The rereaders also felt more confident about how much they'd remember, and that's the detail course creators should care about most.

An online course is built almost entirely around the rereading condition. Students watch, rewatch and see the progress bar move. Every signal says they're learning. A review of ten common study techniques led by John Dunlosky gave practice testing its highest utility rating and rated rereading low, along with highlighting and summarizing, which happen to be the other things course students do with their time.

A quiz is the cheapest way to put the testing condition inside your course.

How strong the evidence is, and where it stops

The largest review of classroom research, published in Psychological Bulletin, pooled 222 studies covering 48,478 students and found that quizzing raised achievement by about half a standard deviation. The authors translate that as moving a student from the 50th to the 69th percentile. 82.9% of the effects were positive, the benefit held for facts, concepts and solving new problems, and it showed up across 18 subject categories.

The limits matter, though, especially if your course is digital-only. The review only included courses taught in a classroom and left distance education out. Only six effects came from continuing education, too few to conclude anything, and that's the category closest to an adult buying your course on a Tuesday night. Your students also aren't sitting an exam at the end, so part of what makes quizzes work at school, knowing a test is coming, is weaker in a self-paced course.

That doesn't cancel the finding: the memory effect in the Washington University study has nothing to do with whether someone paid. It does mean you shouldn't promise buyers a percentage, and you should design quizzes to be worth taking without a grade attached.

Put the quiz after the lesson and keep the stakes low

Several of the review's findings translate straight into course design. The numbers below are effect sizes, where 0.5 is the half standard deviation described above:

Timing matters. Quizzes given after a lesson produced 0.536, against 0.186 for quizzes given before it. A quick check before a lesson can prime attention, but the quiz that does the work comes once the material has been taught.
Stakes don't. Quizzes that counted toward a grade and quizzes that didn't produced statistically indistinguishable benefits, 0.441 and 0.477. You don't need to pass or fail anyone, which is convenient, since an adult paying for your course doesn't want to be graded anyway. On Sherpo the pass mark is a field you set yourself, and if you leave it empty, every finished attempt passes.
Simple formats are fine. Recognition questions like multiple choice worked as well overall as recall questions where students produce the answer themselves, 0.518 against 0.520. You can skip hand-marking essays and keep most of the benefit, as long as the wrong options are mistakes people actually make.

Let students try again, and explain the right answer

Two settings make a measurable difference:

The first is feedback: quizzes followed by the correct answers produced 0.537, against 0.374 without them. On Sherpo, each question can carry a hint students can reveal while they answer, and an explanation that appears next to the right answer once an attempt ends.
The second is repetition. Material quizzed once came in at 0.444, twice at 0.601, three or more times at 0.642, and quizzes that allowed unlimited attempts reached 0.762. On Sherpo, you can leave the number of sittings empty for unlimited attempts or cap it at a specific number.

In practice, all of this means retries switched on and an explanation under every question, written the way you'd say it on a call: why the right answer is right, why the tempting wrong one is wrong, and which lesson covers it. Reusing an earlier module's question in a later quiz counts as repetition too.

One trade-off to think about. With answers shown after every attempt, a student can retake the quiz by remembering which option was correct rather than why. Explanations that walk through the reasoning make that second attempt worth something, and so does using weights to make the questions that test understanding count more than the ones that test recall.

Quizzes give students a reason to come back

The benefit also grew the longer quizzing went on. When it happened in a single class session, quizzed students scored 0.385 standard deviations higher than unquizzed ones on the final exam, roughly the 65th percentile instead of the 50th. When it ran for more than a semester, the gap reached 0.624, closer to the 73rd percentile. The authors put part of this down to motivation. One study they cite compared two sections of the same course that both started the term at about 90% attendance: the one with a short quiz every day stayed near 90% all semester, while the one without slid to about 66%.

A self-paced course has the same problem with fewer safeguards. Nobody takes a register, and members who drift away usually do it in the first weeks. A short quiz at the end of the first module gives a new student a small task they can finish and an early sign that the course is working for them. A certificate gives the whole course a finish line. On Sherpo, you can tie each certificate to a quiz and require a passing score before issuing it. The certificate can also carry a QR code linking to a verification page, useful if students show it to an employer or add it to their LinkedIn.

Your quiz results grade the course too

A quiz tells you as much about your teaching as about your students. If most people miss the same question, the likely problem is the lesson. Maybe the explanation skipped a step, or the example didn't match the situation your audience is in, which is the gap between what you taught and the job they were trying to get done.

Sherpo shows average score, completion time and pass rate for each quiz, and lets you read every student's answers filtered by correct or wrong. Because quizzes, lessons and customers sit in one dashboard rather than a fragmented stack, you can find the question that trips people up and fix the lesson behind it, or have your AI assistant pull those numbers for you through Sherpo's MCP.

You can also end a quiz with an ungraded free-text question asking students what was unclear, which gets you feedback on the lesson in their own words.

How to write quiz questions worth answering

Three to five questions per lesson is plenty. Ask students to use an idea rather than repeat a definition: give them a scenario and ask which approach fits, or show a worked example with a mistake in it and ask where it goes wrong. Build the wrong answers from errors you see in comments, emails and calls, since those are the misconceptions your explanations need to fix. And quiz earlier material again later, because a question about module two inside the module five quiz is the cheapest repetition you'll ever add.

On Sherpo there are three ways to build one. You can add blocks by hand: multiple choice, true or false, sliders, open-ended answers checked against yours, and ungraded free text, with static text and section breaks to split the quiz into steps. You can import from Markdown, where checklists become multiple choice or true/false and nested bullets become accepted answers, so an assistant can draft the whole quiz in that format from your lesson. Or you can ask an AI assistant connected through Sherpo's MCP to create the quiz and link it to the right lesson for you. Hints, explanations and weights don't come through the Markdown import, which is no loss, since the explanations are the part you should write yourself. An assistant writes plausible questions about a topic, while you know which mistakes your students actually make, and that knowledge is what makes a quiz teach. It's the same reason you can't outsource your understanding.

Why this is worth an afternoon

Quizzes are cheap to build, and the evidence behind them is unusually solid, even if most of it comes from classrooms rather than creator courses. Students who remember what you taught get results, and results are where testimonials, referrals and the next sale come from. A course people watch and forget is much harder to sell a second time, and it makes it harder to upsell those same students on new modules or a higher tier, or to cross-sell them your next product.

Ready to build a course people remember? Get started with Sherpo and add quizzes, explanations and certificates to the lessons you already have.