A Databricks hire is not one skill, it is four. Unity Catalog, Delta Live Tables, Spark tuning and cost governance each carry roughly a quarter of the value — and in most loops, only about half of each gets tested. The half you can test live is design reasoning and reading skill: can this person look at a plan, a pipeline graph or a catalog hierarchy and tell you what is wrong with it. The half you cannot test is ownership under pressure — whether they actually held the line on cluster policies in month nine, or quietly let the bill climb. That half belongs in the reference check, and most hiring managers never ask for it.
What follows is the matrix we hand to employers before they scope a Databricks role: four tracks, broken into what a 90-minute interview can prove and what only production exposes.
Why a generic screen misses all four tracks
Databricks skills are thin on the ground in the general developer population. In the 2026 Stack Overflow Developer Survey, just 4.2% of respondents — 510 of 12,061 — reported working with Databricks SQL. That number matters because of what it rules out: a strong SQL screen and a Python pairing exercise will surface competent engineers, but they will not tell you whether someone has ever reasoned about a streaming table’s checkpoint, a catalog-level grant or a photon-eligible join. The platform has to be tested as the platform.
It also means the loop needs to be fast. The U.S. Bureau of Labor Statistics projects employment of data scientists to increase 33.5 percent between 2024 and 2034, and the surrounding lakehouse talent market tightens with it. A five-stage unscored loop does not filter better — it just loses the shortlist to an employer who decided in a week.
Rarity is not the problem. Testing for the wrong thing in the time you have is the problem.
The matrix at a glance
| Track | What you can prove in a 90-minute interview | What only shows up in production | Weight for a senior IC |
|---|---|---|---|
| Unity Catalog | Ownership modelling, catalog/schema design reasoning, grant inheritance, migration sequencing from Hive metastore | Whether they maintained the model after the first org reorg; lineage hygiene at 18 months; tolerance for one-off exceptions | 30% |
| Delta Live Tables | Streaming vs. materialised view choice, expectations and quarantine design, idempotency, backfill reasoning | Pipeline reliability over quarter-end loads; whether expectations got loosened rather than fixed | 25% |
| Spark tuning | Reading a query plan or Spark UI live, diagnosing skew and spill, partitioning and file-size reasoning | Discipline to tune the top five jobs monthly instead of the one job someone complained about | 25% |
| Cost governance | Cluster policy design, job vs. all-purpose compute reasoning, naming the levers that move a bill | Whether the bill actually went down and stayed down; willingness to say no to a team | 20% |
The weights shift by role. A platform or governance lead flips Unity Catalog and cost governance up to 60% combined; a pipeline engineer loads DLT and Spark. But every Databricks role touches all four, which is why a loop that only tests Spark produces hires who are expensive in a different way.
Unity Catalog: an ownership-modelling skill, not a feature-recall one
Unity Catalog is no longer an optional layer to have opinions about. Databricks reports that more than 14,000 organizations now govern their data and AI on Unity Catalog. A candidate whose entire career sits in Hive metastore estates is a gap to close, not a stylistic preference — and the close takes months, not weeks.
But the skill underneath is organisational, not technical. The dbt Labs 2026 State of Analytics Engineering Report found that ambiguous data ownership remains a challenge for 41% of respondents, effectively unchanged year over year. You cannot grant your way out of that. So test the modelling, not the syntax.
What to ask
Give the candidate a real org shape — three business units, a shared customer dimension, one regulated dataset, two acquisitions that arrived with their own metastores — and ask them to draw the catalog and schema layout, then defend it. Strong answers argue about blast radius and who gets paged, not about whether to use catalogs or schemas. Then ask what breaks when the org reorganises in a year. Candidates who have only read the docs answer with features. Candidates who have run it answer with migrations they regretted.
The business case for weighting this track is Databricks’ own customer data: companies that implemented AI governance pushed 12x more projects to production. Governance maturity is not a tax on shipping — in that data it correlates with shipping more.
What you cannot test
Whether they kept the model clean. Every Unity Catalog estate we see degrades the same way: one urgent exception, then a second, then a schema nobody owns. Ask the reference directly — did this person ever refuse a grant request, and what happened.
Delta Live Tables: design reasoning is the most testable thing you have
The DLT track is the one where a single design choice moves the number by an order of magnitude. In Databricks’ own TPC-DI benchmark, switching full recomputes to materialized views at the serving layer delivered 17x speedup. That is the whole interview in one statistic: the candidate who reaches for incremental materialisation by default and the candidate who rebuilds the serving layer nightly will produce pipelines that differ by more than an order of magnitude in cost and runtime, on identical hardware.
| DLT sub-skill | Testable live? | How to test it |
|---|---|---|
| Streaming table vs. materialised view choice | Yes | Give three source profiles — append-only clickstream, slowly changing dimension, late-arriving fact — and ask for the choice plus the reason |
| Expectations and quarantine design | Yes | Ask where a failing row should go and who finds out |
| Idempotency and replay | Yes | “The pipeline ran twice on Tuesday. What does the gold table look like?” |
| Backfill strategy | Partly | They can describe it; you learn whether it worked from references |
| Long-run reliability | No | Reference check only |
| Resisting loosened expectations | No | Reference check only |
The last two rows are where DLT hires actually fail. Expectations are easy to write and easier to relax. A pipeline with twelve expectations, nine of them set to warn, is a pipeline nobody trusts. No interview surfaces that habit; a reference call does, if you ask what happened the first time the pipeline blocked a release.
Spark tuning: test reading, not recall
Spark tuning is the track most hiring managers over-test and under-score. Trivia about spark.sql.shuffle.partitions tells you someone read a blog post in 2021. What you want is reading skill — hand over a physical plan or a Spark UI screenshot from your own estate, with the names changed, and ask what is wrong.
Good candidates narrate in a predictable order: stage durations, task skew within the longest stage, spill, then join strategy, then file layout. They ask what the data looks like before they prescribe anything. Weak candidates start prescribing — broadcast it, repartition it, cache it — without asking a question.
Testable live
- Diagnosing skew from a task duration distribution
- Spotting a broadcast that should not be one, and saying why
- Small-file reasoning: why 400,000 files under a partition is a planning problem before it is an IO problem
- Knowing when Photon helps and when it is irrelevant
- Explaining a decision to a non-engineer in one sentence
Not testable live
- Whether they tune proactively or only when someone escalates
- Whether they tune the five jobs that cost the most or the one job their loudest stakeholder noticed
- Whether they leave the cluster config better than they found it
Our 90-minute Databricks vetting structure puts the plan-reading exercise in the middle third for this reason — it is the single highest-signal 20 minutes in the loop, and it is the part candidates cannot prepare for.
Cost governance: the track that belongs in the interview, not the admin’s job description
This is the track most often left off the matrix entirely, on the theory that the platform admin owns the bill. The spend data says otherwise. In the 2026 dbt Labs report, a majority of respondents (57%) report increased warehouse and compute spend, while only 13% report decreases. If most teams are spending more year over year, cost reasoning is a line-engineer competency, not a quarterly review item.
What a live test looks like
Ask the candidate to spend your money. Give them a workload mix — six scheduled ETL jobs, two ad-hoc analyst warehouses, one ML training loop — and a budget, and ask them to set cluster policies and compute types. Listen for: job compute for scheduled work, autoscaling bounds with reasons attached, spot instances where interruption is tolerable and nowhere else, serverless where cold start costs less than idle time. Then ask the uncomfortable one — which of these three teams do you take capacity from first, and how do you tell them.
What only production shows
Whether the bill went down. Cost governance is the easiest track to perform in an interview and the hardest to sustain, because every saving is a conversation someone did not want to have. Ask references for a number: what was monthly spend when they arrived, what was it when they left, and what did they change. A candidate who owned cost will have that number. A candidate who talked about cost will not.
If you are modelling the cost of the hire itself alongside the cost of the platform, our contractor versus full-time comparison breaks down the fully loaded numbers.
Turning the matrix into a scored loop
Four tracks, two columns each, one score sheet. The practical version:
| Stage | Length | Tracks covered | Output |
|---|---|---|---|
| Structured screen | 30 min | All four, shallow | Hive-only vs. Unity Catalog exposure; spend ownership yes/no |
| Technical deep-dive | 90 min | UC design, DLT choice, plan reading | Scored 1–4 per track, written same day |
| Cost exercise | 30 min | Cost governance | Policy sketch plus the hard conversation |
| Reference calls (2) | 30 min each | The production half of all four tracks | Spend delta, ownership behaviour, expectation hygiene |
Four touchpoints, under four hours of candidate time, and a decision inside a week. The reference calls are not a formality at the end — they are the only instrument you have for half the matrix, so brief them with specific questions rather than “would you work with them again.”
Scope the role against the matrix before you write the job description, and weight the tracks to the work you actually have. For the longer version of the role definition and market context, see how to hire Databricks developers in 2026, or browse anonymized specialists on the data engineering network.