20 Databricks interview questions on Delta Lake, Spark tuning, streaming and Unity Catalog, with short, clear answers and a Free Edition practice lab.
You walk into a Databricks interview. The first question is easy. The second one is not.
"Your pipeline ran in 20 minutes yesterday. Today it took two hours. Nothing changed. What do you check first?"
This is how Databricks interviews feel in 2026. Interviewers care less about syntax. They want to see how you think when a real system misbehaves.
This guide gives you 20 questions that come up again and again, with short answers in simple English. Read the answer, then practice it out loud.
A strong answer usually has four parts.
You do not need to say all four every time. But if you can, you will sound like someone who has run real pipelines.
flowchart LR
A[Concept] --> B[Failure mode]
B --> C[Practical fix]
C --> D[Trade-off]It is a folder called _delta_log next to your data files. Every write adds a new JSON commit file listing which Parquet files were added or removed.
Readers look at the log to know which files make up the current table. That is how Delta gives you ACID transactions and time travel on plain files.
Delta uses optimistic concurrency. Both jobs write their files, then try to commit. If the second job touched the same data the first job changed, it fails with a concurrent modification error.
The fix is to avoid overlapping writes. Use replaceWhere on separate date ranges, or run conflicting jobs one after another.
For new tables in 2026, start with liquid clustering. You pick clustering keys and Delta organizes files for you. You can change the keys later without rewriting the table.
Partitioning still works for very large tables with a clear, low-cardinality date column. The common mistake is partitioning on a column with thousands of values, which creates many tiny files.
VACUUM deletes old data files that the table no longer uses. It saves storage.
The danger is retention. If you vacuum with a very short window, time travel breaks and long-running readers can fail because their files are gone. The default seven days exists for a reason.
A shallow clone copies only the transaction log. It is fast and cheap, great for testing. A deep clone copies the data files too, so it is fully independent, good for backups.
The trap: a shallow clone still points at the source files. If someone vacuums the source, the clone can break.
That is data skew. One key, often a null or a default value, holds far more rows than the others. One core does all the work while the rest wait.
Check the Spark UI stage page for one task with huge input. Fix it with Adaptive Query Execution, a broadcast join, filtering nulls early, or salting the hot key.
Spark ran out of execution memory for a shuffle, sort, or join, so it wrote data to disk and read it back. That is slow.
You see it as "Spill (Disk)" in the Spark UI. Common fixes are more shuffle partitions, less skew, or selecting fewer columns before the heavy step.
Broadcasting sends a small table to every executor so the big table does not need to shuffle. It is very fast when the small table is truly small.
It goes wrong when the "small" table grows. The driver must collect it first, so a large broadcast can crash the driver with an out of memory error.
AQE changes the plan while the query runs, using real statistics. It merges tiny shuffle partitions, splits skewed join partitions, and can switch to a broadcast join when one side turns out small.
It does not fix everything. A heavy group by on one hot key, or a badly written query, still needs your help.
No. Caching helps when you reuse the same DataFrame several times. It costs memory, and if you cache something used once, you just slow the job down and push other data out of memory.
Cache on purpose, then unpersist when done.
Auto Loader tracks which files it already processed, so each run only picks up new files. It also handles schema changes and can rescue unexpected columns.
Reading a folder again and again means listing everything and risking duplicates.
In Lakeflow Declarative Pipelines, expectations are data quality rules. Each rule can warn and keep the row, drop the bad row, or fail the update.
A good answer explains where you place each one. Warn in Bronze, drop in Silver, fail only on rules that must never break, like a missing primary key in Gold.
It stores progress in a checkpoint folder. On restart, it resumes from the last committed offset. With an idempotent sink like Delta, you get exactly once results.
Never share one checkpoint between two queries, and never delete it casually.
Not always. If the business needs data every hour, use trigger(availableNow=True) and run on a schedule. You process everything new, then the compute shuts down.
Always on streams cost money even when nothing arrives.
catalog.schema.table. The catalog is the top container, often per environment or business area. Schemas group related tables. Permissions flow down from catalog to schema to table.
Use row filters and column masks. A row filter shows each user only the rows they may see. A column mask returns a masked value, like the last four digits of a card, for users without access.
One table, many safe views of it.
Volumes store files, like CSVs, images, or PDFs, under Unity Catalog governance. Paths look like /Volumes/workspace/default/book_data/. You control access with the same grants you use for tables.
Land raw changes in Bronze with Auto Loader. In Silver, apply the changes with MERGE or Auto CDC, keyed on the primary key and ordered by a sequence column. Build Gold tables for reporting.
Mention idempotency: running the same batch twice should give the same result.
Start with what changed. Did input volume jump? Did a new value create skew? Did small files pile up? Open the Spark UI, find the slowest stage, and look at task times, spill, and shuffle size.
Interviewers love a calm, step by step answer here more than a clever guess.
Right size clusters, use auto termination, prefer scheduled batch over always on streams, run OPTIMIZE on hot tables, and read only the columns you need.
Then measure. Cost work without measurement is guessing.
Reading answers is not enough. Run this in Databricks Free Edition and look at the transaction log yourself. It turns question 1 into something you have seen with your own eyes.
from pyspark.sql import functions as F
orders_df = spark.range(0, 1000).withColumn("amount", F.col("id") * 10)
orders_df.write.mode("overwrite").saveAsTable("workspace.default.interview_orders")
# Make a second commit
spark.sql("UPDATE workspace.default.interview_orders SET amount = 0 WHERE id < 10")
# Every write is a version in the log
display(spark.sql("DESCRIBE HISTORY workspace.default.interview_orders"))SELECT COUNT(*) AS old_rows_with_zero
FROM workspace.default.interview_orders VERSION AS OF 0
WHERE amount = 0Version 0 shows one zero row. The current version shows ten. Now you can explain time travel from experience, not memory.
Good interviews are not about knowing everything. They are about showing how you think. Practice that, and the rest follows.