Why structure beats a good conversation
Most technical interviews are still a good conversation between two engineers. The interviewer picks questions that interest them that day, forms an impression in the first ten minutes and writes notes after the candidate leaves. It feels like judgement. The research says it is closer to noise.
The largest recent re-analysis of hiring methods, by Sackett and colleagues in 2022, puts the validity of structured interviews at 0.42 and unstructured ones at 0.19. In plain words: the structured version predicts job performance about twice as well. It was the strongest single predictor in their review.
There is a second reason to care in 2026. If your hiring uses any software that screens, ranks or scores candidates, the EU AI Act treats it as high-risk, with obligations from December 2, 2027. A structured process gives you most of the paper trail those obligations will expect. This article is written for engineering managers and recruiters who run technical loops.
What "structured" means in a technical loop
Not a script read in a monotone. Five rules, and you can adopt them one at a time.
Three or four competencies per interview
Derived from the job, written down before the posting goes live. "Debugging under pressure", "API design", "explaining trade-offs". Not "culture fit".
Same core questions for every candidate
Follow-ups are allowed if they probe the same competency. Swapping the question for a candidate you like is not.
Anchored scores
Each score from 1 to 4 has a written example of what that answer sounds like. Interviewers match evidence to anchors instead of inventing their own scale.
Notes and scores before the debrief
Everyone submits independently first. Otherwise the most senior person speaks and the rest adjust.
Calibrate twice a year
Three interviewers score the same recorded answer. If their numbers differ by more than one point, the anchor is unclear. Rewrite it.
An example scorecard row
Competency: debugging a production incident. Question: "Tell me about the last incident you owned from alert to fix."
| Score | What the answer contains |
|---|---|
| 1 | Describes the incident; others found the cause. No mention of what they personally checked. |
| 2 | Followed a runbook. Can name the fix but not how the cause was isolated. |
| 3 | Formed a hypothesis, checked it with logs or metrics, ruled out one alternative, shipped the fix. |
| 4 | All of 3, plus changed something so the class of incident cannot recur: alert, test, or design change, with a result. |
Where bias gets in, and which rule stops it
| Bias | How it shows up | Rule that limits it |
|---|---|---|
| First impression | Decision in the first 10 minutes, rest of the hour confirms it | Score each competency separately, at its own question |
| Similar-to-me | Same university, same stack, same jokes | Anchors describe answers, not people |
| Contrast | Average candidate after a weak one looks strong | Compare to anchors, not to the previous candidate |
| Groupthink | Debrief follows the loudest voice | Independent scores submitted before discussion |
| Question drift | Harder questions for some candidates | Fixed core questions per role |
The AI Act part, without the panic
A short, practical reading. It is not legal advice; your counsel decides how it applies to your tools.
- What is high-risk. Annex III, point 4: AI systems used to recruit or select people, in particular to place targeted job ads, analyse and filter applications, and evaluate candidates. A video call with human notes is not an AI system. An ATS that ranks CVs with a model is.
- When. The Digital Omnibus, in force since July 27, 2026, moved the high-risk obligations for these systems from August 2, 2026 to December 2, 2027.
- What already applies. Emotion recognition of candidates and workers has been prohibited since February 2, 2025. Transparency duties for systems that interact with people, such as a recruiting chatbot, apply from August 2026.
- What deployers will need. Use the system as its provider instructs, assign trained people to oversee it, keep its logs, monitor it and tell candidates when such a system is used on them.
Here is the useful overlap. Written competencies, fixed questions, anchored scores and independent notes are exactly the kind of documented, human-reviewed process that makes oversight possible. If a tool scores candidates, you can compare its output to your human scorecards and see where it disagrees. Without a scorecard there is nothing to compare against. In New York City, Local Law 144 already requires a yearly bias audit for automated employment decision tools; similar checks are reasonable to expect elsewhere.
Check your current loop
Rolling it out in four weeks
| Week | Do |
|---|---|
| 1 | Pick one role. Write competencies and two questions per competency with the hiring manager. |
| 2 | Write anchors. Run one calibration: three interviewers, one recorded answer, compare scores. |
| 3 | Use it on real candidates. Scores go into the hiring funnel before anyone talks. |
| 4 | Review: which anchors split interviewers, which questions everyone scored 3. Rewrite those two. |
Hiring for a technical role now? You can post the vacancy on IT Career Gym and browse the candidates section. More on the employer side: for employers.
The short version
Same questions, written anchors, independent scores, calibration twice a year. It roughly doubles how well your interviews predict performance and leaves the documentation that the AI Act will expect from teams using screening tools by December 2027.
Related: common interviewer mistakes, topgrading, AI in interviews.
Questions readers ask
-
Does a structured interview mean reading questions off a script?
The questions are fixed, the conversation is not. Follow-ups are allowed if they probe the same competency, and you write down which ones you asked. What you cannot do is swap the question for a candidate you like.
Link to this answer -
How many competencies should one interview score?
Three or four. Past five, interviewers start averaging gut feeling across boxes. Split the rest across other loop stages.
Link to this answer -
We do not use AI in hiring. Does the AI Act matter to us?
Only if a tool screens, ranks or scores candidates, including the ATS feature someone switched on last year. Plain video calls and human notes are not covered. Check vendor settings before December 2, 2027.
Link to this answer -
Is this legal advice?
No. It is an engineering manager's reading of public texts. Your counsel decides whether a specific tool is high-risk under Annex III.
Link to this answer -
Will structure make the interview feel cold?
Not if you tell candidates the format up front and leave five minutes for their questions. Warmth comes from the interviewer, not from improvised questions.
Link to this answer -
Can we practise the format before rolling it out?
Yes. Run two calibration mocks: three interviewers score the same recorded answer, compare numbers, rewrite the anchors that split them. Pair mocks on IT Career Gym work for this if you want outside candidates.
Link to this answer
Sources
- Sackett, Zhang, Berry, Lievens (2022), Revisiting meta-analytic estimates of validity in personnel selection
- Regulation (EU) 2024/1689 (AI Act), Annex III point 4
- Gibson Dunn: EU AI Act Omnibus agreement, postponed high-risk deadlines
- Google re:Work: Use structured interviewing
- NYC Local Law 144 (automated employment decision tools)
Figures were checked against these pages in October 2026. Pay and dates change; the links show where to recheck.