Draft interview questions about checking AI-assisted work

Example result.

Criterion: Verification
Question: Which figure in the summary did you check, and against what?
Evidence recorded: Recounted the orders using the definition; found 8 of 10, not 9.
Still unestablished: whether they'd check a larger dataset the same way.

The assistant helps draft and improve questions before any interview. The panel assesses candidates. AI doesn’t analyse answers, score people or decide who progresses.

1. Choose the work and the three criteria

Pick a task where checking matters, such as a supplier summary, a customer reply or a project update. Then define what you’ll look for:

• Verification: did they check claims against the source? “I reviewed it carefully” isn’t evidence. “I pulled the three figures it quoted back to the spreadsheet and one was from the wrong column” is.
• Scope judgement: do they know which parts of the job they wouldn’t hand to an assistant, and why?
• Ownership: do they treat the final output as theirs, including what they left out or flagged?

Don’t equate tool access with ability. A candidate may have worked somewhere that banned AI. Decide how the same criteria can be shown without paid tools or experience of a particular product, and tell candidates the exercise rules in advance.

Why this works: criteria about the work give the panel something to observe. “Good with AI” doesn’t.

What you should see: a task, three criteria and a one-line rule on whether AI is allowed in the exercise.

2. Ask AI to draft the segment from the role

Draft a 15-minute interview segment for [role] about checking assisted work.
Use only these role tasks and assessment criteria: [tasks and criteria].
The exercise is: [fictional exercise], and the agreed AI-use rule is: [rule].

Write five core questions in a fixed order. For each, state the criterion it explores and the concrete evidence a panel could record. Add neutral follow-ups about sources, corrections and uncertainty. Offer an equivalent route for someone who hasn't used AI, using a supplied fictional draft instead.

Do not ask for confidential past-employer material, protected characteristics, health or personal circumstances. Do not create a detector test, ranking, score or hiring recommendation. Flag anything the hiring owner needs to review.

[role] is the job title. [tasks and criteria] are the tasks and three criteria from step 1. [fictional exercise] and [rule] describe the exercise everyone gets and whether AI is allowed.

Why this works: the role tasks prevent a generic technology quiz, and the equivalent route stops previous access becoming a hidden requirement.

What you should see: five questions, each labelled with a criterion and the evidence it could produce.

3. Review every question before anyone is interviewed

Ask of each question: what evidence would an answer give the panel? A useful sequence is:

1. What did the task ask for, and what source did you use?
2. Which parts of the first draft did you change, and why?
3. Which claims or calculations did you check against the source?
4. Was there anything it produced that you decided not to use, and why? Rejecting output means having judged it.
5. Is there a part of this job you wouldn’t use an assistant for?

Keep follow-ups neutral. A vague answer calls for a request for a specific example, not an automatic negative judgement. The hiring owner approves the final wording.

Why this works: every question points at something the candidate did, so the panel records actions rather than impressions.

What you should see: five questions you could ask any candidate, whether or not they’ve used AI.

4. Make the human evidence sheet

Role and exercise version:
Interview date and panel:
Application reference:
Criterion:
Question asked:
Evidence recorded, with exercise location:
Clarification asked and answer:
What remains unestablished:
Panel assessment under the agreed framework:

Keep observations apart from conclusions. “Recalculated the rate using the definition” is an observation. “Strong AI judgement” is a conclusion that needs a defined basis. Acas advises asking each applicant the same questions where possible, having more than one person interview where possible, and storing notes securely. Follow your normal arrangements for reasonable adjustments.

Why this works: a sheet with an “unestablished” line stops a good conversation turning into a claim the evidence doesn’t support.

What you should see: one sheet per criterion, ready for the panel to fill in during the interview.

Check the result

• Every question explores one of your three criteria.
• No question requires prior AI access, a paid tool or material from another employer.
• No question touches protected characteristics, health or personal circumstances.
• Nothing asks the AI to evaluate a real candidate.

The usual failure: the draft quietly tests tool familiarity. Questions like “Which AI tools are you most comfortable with?” or “Tell us about a time you used AI at your last job” reward access, not checking. Spot them by asking whether a candidate whose last employer banned AI could answer equally well.

Revise question [number] so it examines the same job-related criterion without requiring prior AI access or a paid tool. Use the supplied fictional draft as the alternative. Keep comparable difficulty and flag any difference the hiring owner needs to review. Do not assess candidates.

[number] is the question that failed.

Fictional example. Every candidate gets the same dataset and draft, and can discuss it without AI access:

Definition: an order is on time only when delivered in full by its due date.
Ten orders: eight in full by the due date. One partial by the due date, completed
two days later. One in full, one day late.
Draft summary: "Nine of ten orders were on time, a 90% on-time rate. This shows
a seasonal improvement in supplier performance."

The first AI draft of the segment opened with “Which AI tools are you most comfortable with?” and “Tell us about a time you used AI at your last job”.

• The failure. Both questions reward previous access, and the second invites material from another employer. Neither produces evidence about checking.
• The repair. After the revise prompt, they became “Which figure in the summary did you check, and against what?” and “Was there anything in the draft you decided not to use?” Both work with or without AI.
• What the panel can record. Under the definition, only eight orders qualify: 8 ÷ 10 = 80%. Ten orders from one period say nothing about seasons. A candidate who finds either issue gives the panel concrete evidence for Verification.

Task kit for this job: Download (ZIP) · Reusable instructions

Go deeper

Review this proposed exercise against its assessment purpose. Flag anything that needs paid access, confidential material or experience with a specific product. Suggest alternatives that keep the same job-related skill. Do not change the hiring policy.

Sources

• Acas: Interviewing job applicants