Letting AI write your quiz: what works, and what to reread

Writing twenty assessment questions from your own material takes one to two hours. A language model does it in a minute from the same material. The gain is real, and it comes with a cost few people plan for: the reread.

What AI does well

• Form. Four plausible options, one correct, even lengths. Mechanical and tedious by hand.
• Coverage. It does not skip part 3 because it is late, which happens to a human writing twenty questions in a row.
• Rephrasing. Asking the same idea twice from different angles, which helps separate understanding from remembering a wording.

The three faults, always the same

1. A question about the material rather than the subject

"According to slide 14, how many steps does the process have?" That tests whether somebody looked at a slide, not whether they understood the process. It is easy to generate because the answer is literally in the document supplied.

2. The correct answer twice as long as the others

A model writes the right answer carefully and rushes the distractors. The result is a question you can pass knowing nothing about the subject, by picking the most detailed option. It is the most common bias and the most invisible to a quick reread.

3. The out-of-scope question

A true statement, correctly worded, about something not taught that day. The model knows the subject beyond your material and fills the gaps. In the room this produces a question nobody can answer, and makes the group doubt their own attention.

A correct question about something never taught does not measure the training. It measures what people already knew.

Reread in three passes

1. Length. Scan only the options and check you cannot guess the answer from its size. Fastest pass, best return.
2. Scope. For each question ask: did I actually say this? If not, cut it, even if it is good.
3. Wording. Replace any reference to the material ("according to the document", "in the diagram") with the idea itself.

Those three passes take about ten minutes for twenty questions: against one to two hours to write them.

What cannot be delegated

Tagging questions to competencies remains a judgement: it depends on what you want to measure, not only on the content. It is also what makes the report readable afterwards; see measuring what was learned.

In practice

SkillLens drafts the quiz from your material, drops out-of-scope questions before showing them to you, and flags the ones it doubts with the reason. Every question stays editable, and the material is deleted after drafting: it is sent to the model provider for the duration of that call, which is what makes the questions about your content.

Try it free