Performance articles

Every BricksNotes article on Performance: 12 practical pieces on Databricks and data engineering, newest first.

The checkpoint trap is finally fixable: resize stateful Spark streams without starting over
You partitioned the table. It got slower. Partitioning and liquid clustering, explained.
The small files problem in Spark and Databricks, explained simply
The table had 40,000 tiny files. The small file problem in Databricks, explained
Same result, 3 seconds or 30 minutes. Query optimization in Databricks
The join that ran for seven hours. Broadcast and shuffle joins in Databricks
One task ran for forty minutes. Data skew in Databricks, explained
Z-ORDER or liquid clustering? How to lay out a Delta table in 2026
From 47 Minutes to 5 Minutes: Fixing the Small File Problem on a 100 GB Delta Table
Databricks Photon Engine: What It Actually Does and When It Matters
All BricksNotes articles