Measuring what was learned, competency by competency
One overall mark hides which idea landed and which did not. How to tag questions with competencies and read progress you can act on.
"The group is at 74%" supports no decision at all. That number is the average of at least two opposite realities: ideas fully understood and ideas not understood at all. The average erases them precisely when you need to tell them apart.
The problem with a single mark
Take a course in four parts, assessed by twenty questions. A group scores 74%. That result is consistent with two very different situations:
- the group is around 74% across all four parts: understood evenly, improvable everywhere;
- the group is at 95% on three parts and 15% on the fourth: an entire part was missed.
The second case calls for a specific and urgent action. The first calls for none. The mark cannot tell them apart, and the mark is the only number most evaluations produce.
Tag questions with competencies
The fix is to label each question with the competency it tests, then read results per label rather than in aggregate. Three questions on regulatory compliance give a compliance rate; four on negotiation give a negotiation rate.
The extra work is real but small: one label per question, applied once, when the question is written. What you get in return is a map instead of a number.
A mark says whether the group passed. A per-competency reading says what to reteach on Monday.
Compare the same axis at two moments
The per-competency reading earns its keep when it is repeated. The same competency measured before and after gives progress; measured again six weeks later, it gives retention.
For each competency you then have three points: where they started, where they got to, and what remained. That is an object you can hand to whoever paid without commentary, because it reads on its own. The three moments are covered in how to evaluate a training course.
What this makes visible
- Never acquired: low before, low after. It was not taught, or not intelligibly.
- Already known: high before, high after. It cost session time for nothing.
- Genuinely learned: low before, high after. This is what the training produced.
- Learned but not kept: high right after, low six weeks later. It needs a reminder, not a reteach.
Those four cases call for four different decisions. None of them is visible in an average.
In practice
In SkillLens a question can carry a competency, and the session report shows the success rate per competency and per phase, with a before/after radar once at least three competencies are tagged.