Skip to content

BlogCraft

Interview scoring grid: building a structured interview

A structured interview with a scoring grid predicts performance twice as well as a free-form one (0.42 against 0.19). Method, rating scale and legal limits.

9 min read

An interview scoring grid predicts performance on one condition: the questions are written before the interview from the real need, asked of every candidate in the same order, and scored answer by answer on a scale where each level describes a behaviour. That is what is called a structured interview, and it is the best-ranked selection method in the most recent research.

The gap is anything but marginal. In the revision Paul Sackett and three co-authors published in 2022, the structured interview has a validity of 0.42 and the free-form interview 0.19, so the same conversation between two people, held with or without preparation, doubles in value. Most of the firms we know run the second while believing they run the first, because they have a list of questions and no grid.

Why does a structured interview predict performance better?

Because it measures the same thing in every candidate, which a free-form interview never does. Two candidates seen by the same person on the same day were not asked about the same topics, and the comparison made afterwards is as much about the conversation as about them.

For a long time the reference figures were those of Frank Schmidt and John Hunter, whose 1998 meta-analysis is quoted in every recruitment textbook. It gave 0.51 to the structured interview, 0.38 to the unstructured interview and 0.51 to general mental ability tests, then seen as the best single predictor. A validity is a correlation between what a method forecasts and the performance observed afterwards: 0 predicts nothing, 1 predicts everything.

In 2022, Sackett, Zhang, Berry and Lievens showed that the statistical correction applied to those estimates, meant to compensate for the fact that only hired candidates have an observed performance, inflated almost all of them. Their revision lowers most methods by 0.10 to 0.20 points. Cognitive ability falls to 0.31, job knowledge tests come in at 0.40, work samples at 0.33, and the structured interview, at 0.42, moves to the top of the ranking.

The most useful detail for a recruiter lies elsewhere. The structured interview loses less than a fifth of its value in the revision, while the free-form interview loses half. What looked like a respectable method, the conversation run on instinct by someone experienced, turns out to be barely better than a reference check whose validity the research can no longer even measure.

What makes an interview genuinely structured?

Four properties, and it takes at least three for the word to be earned. Most so-called structured interviews have only one, the list of questions prepared in advance, and it is the least decisive of the four.

The first is where the questions come from: the need, not a generic template. For an IT services firm, that need was put into words during the brief with the client, which is why the intake meeting already decides the placement: a grid built on a vague need scores, with great precision, criteria that do not matter.

The second is equal treatment: the same questions, in the same order, for every candidate for the same role, with follow-ups planned rather than improvised. The third is scoring as the interview goes, question by question, rather than an overall impression written up after the candidate has left. The fourth is the independence of interviewers when there are several: each scores alone before hearing the others, or the most senior person’s score becomes everyone’s.

How do you build a scoring grid for a consultant or a developer?

Start from four to six competencies taken from the need, and write two questions and a scale for each before you see the first candidate. Beyond six, the interviewer no longer has distinct answers to score and copies the same impression into every box.

Take a backend developer placed on time-and-materials with a banking client, on an existing application. The competencies that decide whether such an assignment succeeds are not those of an in-house role: reading and evolving code you did not write, working under security constraints you do not choose, reporting to a client who is not your employer, and keeping delivery commitments. Command of the language comes fifth, because a technical test checks it better than an oral question.

Each competency gets two questions. The behavioural question is about a past situation: “Tell me about the last time you had to change a module nobody on the team understood any more.” The situational question sets a hypothetical case drawn from the assignment: “The client asks you to ship on Friday a change you estimate at eight days. What do you do on Monday morning?” The first says what the person did, the second how they reason when they lack the experience.

For a functional consultant or a project manager, the method is the same and the competencies change: running a workshop with reluctant users, arbitrating between two people on the client side, writing a specification a developer can follow. What does not change is that a competency absent from the need has no place in the grid, however much the company values it.

What does an anchored rating scale look like?

It is a scale where each level describes what you heard, rather than a judgement about what you think of it. “1: poor, 5: excellent” is not a scale but an invitation to score your impression, and two interviewers who heard the same answer will give it different scores.

For the impossible-deadline question, an anchored scale looks like this. Level 1: accepts the deadline without discussion, or refuses it without offering an alternative. Level 3: flags the gap to the client with an estimate, but without proposing how to split the work. Level 5: flags the gap the same day, proposes what can ship on Friday and what cannot, and warns their manager at the services firm before the client calls them. Levels 2 and 4 remain available for answers in between.

Anchoring has an effect interviewers notice from the second interview: it forces them to write down what the candidate said. You cannot give a 5 without being able to quote the answer that matches level 5, and that written trace is what later lets you compare two candidates on facts rather than memories.

It is also what separates this score from a match score that does not say what it measured. A score on an anchored grid can be refuted: a colleague can reread the recorded answer and say it belongs at level 3, not 5. A number with no answer next to it lacks that property, whether it comes from software or from a manager in a hurry.

How does the grid reduce bias, and where does it stop?

It reduces the biases born of improvisation: the impression of the first few minutes that steers every following question, the halo effect of a single brilliant answer, the warmth towards the candidate who resembles us. Once the questions and the scale are fixed before the interview, those biases no longer shape what is asked, only the score, and the score has to be justified by an answer.

The Défenseur des droits, France’s anti-discrimination authority, recommends exactly this in its guide to recruiting without discrimination: assessment grids and interview grids, “so that the same questions are asked of each person interviewed”. It also gives the legal reason, which concerns any managing director directly: a formalised, traceable procedure lets you provide justification if discrimination is alleged.

The limit deserves the same clarity. A grid built on a discriminatory criterion discriminates consistently, and the guide points out that you can be convicted without having intended it, with fines of up to €45,000 for an individual and €225,000 for a legal entity. Demanding a “total availability” no assignment requires, or scoring a candidate’s “ease” without tying it to a task, turns the grid into a well-kept instrument of exclusion.

What may you ask in an interview in France?

Only what has a direct and necessary link with the job on offer or with the assessment of professional aptitude. That is article L1221-6 of the Labour Code, and it applies to any information requested “in whatever form”, so to an oral question as much as to a form.

The Défenseur des droits gives a list of examples to avoid that deserves to be pinned up in every interview room: relationship status and the partner’s job, children and plans to have them, place of birth, the origin of a first name, the parents’ occupation, place of residence, how someone voted and which newspaper they read, weight, height, age and health. The candidate is entitled to refuse to answer, or even to lie to protect themselves, and information they volunteer must not enter the assessment.

The same guide spells out what remains legitimate: hours, pace, travel, pay, prospects and contract type. For an IT services firm, geographic mobility is therefore handled as a condition of the assignment (“the assignment requires two days a week in Lyon, does that work for you?”) and never as a question about home or family. The candidate must also have been told in advance which methods will be used to assess them, as article L1221-8 requires and a single sentence in the invitation email settles.

That leaves retention. The Défenseur des droits advises keeping the records of a recruitment for up to five years, the limitation period for discrimination, which matches how long a rejected candidate’s file may be archived. A completed grid is precisely the document you will want to find that day.

What an agent prepares, and what the interviewer scores

An agent can do most of the work that makes a structured interview sustainable day to day, and that work is exactly what teams skip for lack of time. It drafts a first version of the grid from the brief’s write-up, proposes two questions and an anchored scale per competency, and removes from the outset any wording that touches on private life.

After the interview, it files the interviewer’s notes against each question, flags a competency left without an answer and lines up the grids of several candidates side by side. It does not give the score. A score is a decision about a person, and whoever gave it must be able to justify it to a candidate, a client or a judge: that is how we built Balt, which prepares the grid and the write-up and leaves the scoring to the interviewer.

Should everything be structured?

No, and the concession is real. A fully timed interview is unpleasant on both sides, the most sought-after candidates sometimes find it schoolish, and part of what decides whether they join is played out in how you answer their own questions. Leaving ten minutes of free conversation at the end, unscored and announced as such, costs little in validity and far less than a candidate who turns the offer down.

The other limit belongs to the trade. In an IT services firm, the interview that decides is often the client’s, which you do not control. The grid cannot be imposed on the client, but it can be shared: a write-up that says, competency by competency, what the consultant answered prepares their presentation better than “a very strong profile”, and it turns your assessment into an argument rather than a formality.

Frequently asked questions

  • What is a structured interview?

    An interview where the questions are written in advance from the job requirements, asked in the same order of every candidate, and scored answer by answer on a scale defined before the first interview. Scoring during the interview, not the question list alone, is what makes the difference.

  • How many criteria should an interview scoring grid have?

    Four to six. Fewer and the grid does not cover the role; more and the interviewer can no longer score each criterion on a distinct answer, so they end up copying their overall impression into every box. Each criterion gets two questions, one about a past situation and one about a hypothetical one.

  • Which interview questions are prohibited in France?

    Any question without a direct and necessary link to the job or to the assessment of professional aptitude (Labour Code, article L1221-6). The Défenseur des droits lists relationship status, children, place of birth, the origin of a first name, place of residence, political views, weight, age and health. The candidate is entitled to refuse to answer, or even to lie.

  • Can an AI score a structured interview?

    It can prepare the grid from the brief, find in the notes what the candidate said to each question and flag an empty box. The score itself is a decision about a person, and whoever gave it must be able to justify it: it stays with the interviewer.

Sources

  1. Sackett, Zhang, Berry and Lievens, Revisiting Meta-Analytic Estimates of Validity in Personnel Selection, Journal of Applied Psychology, 2022doi.org
  2. Schmidt and Hunter, The Validity and Utility of Selection Methods in Personnel Psychology, Psychological Bulletin, 1998doi.org
  3. O’Shea and Luscombe, Is Cognitive Ability the Best Predictor of Job Performance?, SIOP, The Industrial-Organizational Psychologistsiop.org
  4. French Labour Code, article L1221-6 (Légifrance)legifrance.gouv.fr
  5. Défenseur des droits, Pour un recrutement sans discrimination, guide, 2019 (in French)defenseurdesdroits.fr

Balt does this work

Read next

Give Balt its first job.

Thirty seconds to add it to Teams and get to work.

Try for free