Evaluation Toolkit

This is the programme’s only evaluation design. Other pages link here rather than repeating it. It uses the four levels of the Kirkpatrick model, and it is deliberately small: a handful of questions you will actually send, and evidence the sessions already produce.

Overview: the design at a glance

Level	What you collect	When
1 Reaction	One short form (6 questions)	End of Session 3. Optional one-question pulse at the end of Sessions 1 and 2
2 Learning	The Session 2 Kahoot results, and each participant’s completed canvas checked against a rubric	During Sessions 2 and 3
3 Behaviour	Three questions to each participant	30 and 90 days after Session 3
4 Results	Not claimed

Level 1 has a narrow use. It tells you about logistics, room and pace: the cold room, the venue nobody could find, the session that felt rushed. It is not evidence that anyone learned anything.

Level 4 is not claimed. The programme cannot attribute patient or service outcomes to itself. Too much else changes at the same time. Say so plainly when you report.

Plan this before Session 1. Decide who sends the 30- and 90-day checks, and how. If you might move on before 90 days, hand the job to a named colleague now. Evaluation designed afterwards rarely happens.

Governance

Before you collect anything, check these with your organisation:

• Register it as a service evaluation with your local audit, quality improvement or clinical effectiveness team. The HRA’s Is my study research? tool will confirm it is not research.
• Add a short privacy statement to every form: what you collect, why, who sees it, how long you keep it.
• Collect the minimum identifiable data. The Level 1 form is anonymous. Keep follow-up contact details in a separate list, and delete them after the 90-day check.
• Use Microsoft Forms in your organisation’s tenant (NHSmail or local Microsoft 365), not Google Forms or personal accounts.
• No patient information on free tools. Menti, Kahoot and similar tools are for teaching content only. Never enter patient details, and ask participants not to either.
• Check local information governance before using any free-tier tool, and before photographing participants’ canvases.

Level 1: end-of-programme form

End of Session 3, before people leave. Paper (printable below), two per A4 sheet. Anonymous. About 3 minutes.

One prompt per Kirkpatrick level, tagged with its level in small grey type, so the form models what Session 3 teaches. Be honest about what each one can show:

Level	Prompt	What it really tells you
1	How happy were you with the sessions overall (organisation, room, pace)? 1–5	Reaction and logistics
1	What one thing would you change? (free text)	What to fix for the next cohort
2	What was the most useful thing you learned? (free text)	Self-reported learning (and one last retrieval)
3	What’s the most useful change you could show someone you can do right now? (free text)	Readiness and capability, not yet changed behaviour
4	When you evaluate your own teaching and see a difference because of this, tell us: [facilitator email]	An invitation, not data: impact can only come later
—	Anything else you want to tell us? (optional)

Levels 2–4 here are self-report or invitation. Behaviour and impact still come from the 30- and 90-day check (Level 3, below). The Level 4 prompt gives an email address rather than asking for a name, so the form stays anonymous.

Optional pulse at the end of Sessions 1 and 2: “One thing we should change for next week?” on a sticky note by the door. Read them before the next session and say what you changed.

Level 2: evidence from the sessions

There is no separate knowledge test. The sessions already produce two pieces of evidence.

The Session 2 Kahoot. It retrieves Session 1 content a week later. Download the Kahoot report afterwards and note which questions most people got right and which they missed. Ask participants to play under a first name or nickname so the report holds no full names.

The Session 3 canvas. With permission, photograph each completed canvas (or collect a copy). Check it against the rubric below. Do this with a colleague if you can, and agree the judgements together.

Canvas rubric

Criterion	Clear	Not yet
Outcomes are observable	Each outcome uses a verb you could watch or check: demonstrate, explain to a patient, choose, calculate	Outcomes say know, understand, be aware of, appreciate
Evidence matches the verb	The evidence sits at the right level of Miller’s pyramid: a “shows how” outcome is checked by watching someone do it, not by a quiz	Evidence is a quiz or attendance for a doing outcome, or there is no evidence
Activities line up	Each activity gives learners practice at what the outcome asks them to do	Activities are mainly listening or reading when the outcome is about doing
Learners and need are specific	Names who the learners are, their setting, and the problem the teaching should fix	”Staff” or “everyone”; the need is a topic rather than a problem
Follow-through is planned	States what happens at 30/60/90 days, how, who will hold them to it, and what support they’ve agreed	No follow-through, or “send the slides afterwards”

Report counts, not scores: for example, “8 of 10 canvases had at least one observable outcome with matching evidence.”

Level 3: the 30- and 90-day check

Send 30 and 90 days after Session 3, by Microsoft Forms or a short email. Not anonymous, because you need to follow up. Five minutes.

1. What did you change in your teaching since the programme?
2. What happened when you did?
3. What got in the way?

The same three questions work as a 10-minute phone or Teams conversation if people don’t reply to forms.

The named follow-up person. At the end of Session 3, each participant agrees with one person in their own world (often their manager) who will hold them to their canvas at 30 days, and the support they need from them (time, release, a slot at the team meeting). Record only that a name was given, not the name. At 30 days, ask: “Did anyone ask you about it? Did you get the support you agreed?”

Optional corroboration. Invite participants to send one of: a before-and-after slide or session plan, or a short comment from a learner or peer. One artefact per person is plenty.

The self-report caveat

Most of this Level 3 evidence is self-report. People overstate change, and the ones who reply are often the ones who changed most. To strengthen it:

• Ask about specific actions (“what did you change?”), not ratings or satisfaction.
• Ask for one artefact: a before-and-after slide or session plan.
• Ask for a comment from someone who saw the change: a learner, peer or the named follow-up person.
• Report the response rate alongside the findings.

Using the data

When someone says “nothing changed”. Thank them: it is the most useful answer you can get. Ask what got in the way, and listen for the type of barrier. System (no time, no support), confidence (knew what to do, didn’t feel ready) or competing priorities. Ask: “What would need to be true for this to be a yes in four weeks?” Collect the barriers. They are evidence for leaders about what the programme alone cannot fix.

Reporting. A one-page summary is enough:

• Number who started and finished.
• Level 1: main themes, and what you changed in response.
• Level 2: Kahoot questions most and least often answered correctly; canvas rubric counts.
• Level 3: response rate at 30 and 90 days; the most common changes, with one or two examples; the most common barriers.
• A plain statement that Level 4 is not claimed.

Be honest about who took part. Early cohorts are often enthusiasts. If yours were, say that the results may flatter the programme.