What is New in Databricks: April and May 2026 for Data Engineers

Disabling tasks in Lakeflow Jobs, Lakeflow Connect Free Tier, Lakebase, Genie Code, and what they mean for learners

2026 has been a quietly important year for Databricks.

Not because of one giant launch, but because the small changes add up. Free Edition keeps getting more capable. Lakeflow keeps getting easier. New ideas like Lakebase and Genie One are shifting how data engineering and AI agents work together.

If you have been heads down in pipelines, you may have missed a few things. This post pulls the most useful updates from April and early May 2026, explains what they mean for working data engineers, and points to the BricksNotes lessons that already cover the underlying ideas.

Disabling tasks in Lakeflow Jobs is now GA

Released April 30, 2026.

You can now disable an individual task inside a Lakeflow Job without deleting it. The task keeps its configuration. The Job simply skips it at run time.

Why this matters in real life:

You can pause a flaky downstream task without rewriting the whole Job.
You can do partial reruns by disabling tasks that already produced their output.
You can turn off a non-critical step during an incident, then re-enable it later.

If you have ever copy-pasted a Job to "temporarily" remove a task, this is for you.

Disable now means a real product feature, not a comment on a YAML file you forgot to revert.

The Workflows lesson on BricksNotes explains how Lakeflow Jobs are built, how Runs behave, and how this fits into the wider picture of orchestrating Pipelines.

Lakeflow Connect Free Tier

Lakeflow Connect now has a Free Tier. It includes built-in connectors for more than 30 SaaS applications and databases, with a point-and-click UI, a simple API, and unified governance via Unity Catalog. Lakeflow is now generally available as a unified ingestion and orchestration platform.

Before this, ingestion was the messy part of any new project. You either wrote your own custom readers or paid for a separate ingestion tool. Now the most common sources are first-class citizens of the lakehouse.

What this changes for learners:

You can practice end-to-end pipelines without writing connector code.
Your Bronze layer becomes less of a glue exercise and more of a design exercise.
The "ingest, transform, govern" loop is finally visible in one platform.

The Data Sources lesson walks through how ingestion fits into a real pipeline, and the Medallion Architecture lesson shows where that data should land.

Lakeflow Declarative Pipelines: 2026 channel improvements

Lakeflow Declarative Pipelines (the modern name for what some still call Delta Live Tables or DLT) saw a steady stream of improvements through the 2026 release channel.

The headline ideas to remember:

Declarative is the default. You describe what you want. The platform manages dependencies, retries, and incremental logic.
The same pipeline runs in both batch and streaming modes when you switch the trigger.
Expectations (data quality rules) are first-class, not bolted on.

If you are preparing for the Data Engineer Associate or Professional exam, this is one of the highest-value areas to understand deeply. The Incremental Processing lesson is the right place to start, followed by the Data Quality lesson.

Lakebase and Genie One: agentic data engineering, in plain English

Two of the bigger 2026 ideas come from the recent announcements:

Lakebase is generally available. It is a managed Postgres database in the lakehouse. Practically, it gives agents an operational, low-latency database that lives next to your lakehouse, governed by the same Unity Catalog.
Genie One is the agentic coworker for data engineering and data science. Instead of asking Genie a question and reading the answer, you can ask it to write and run code on your behalf, with humans still in the loop.

The honest take for working engineers:

These features do not remove the need to understand pipelines. They make it more important.
An agent that writes a Lakeflow Declarative Pipeline still needs you to read it, review the data contract, and own the on-call rotation.
The fundamentals (Delta, Unity Catalog, Lakeflow, Structured Streaming) are still the things that separate "I can prompt an agent" from "I can ship reliable data".

The Unity Catalog lesson covers the governance side, and the Medallion Architecture lesson covers the structural side. Both will keep mattering as agents get more involved.

Free Edition keeps quietly getting better

Free Edition was launched in mid-2025 to give learners and small projects free access to the Databricks Data Intelligence Platform. Since then, more features that used to be paid-only have moved into Free Edition, and Databricks Academy made all self-paced courses free.

For someone learning today, this is the cheapest moment in the platform's history to get serious. You can:

Spin up a workspace in Free Edition.
Use Unity Catalog Volumes (the standard path is /Volumes/workspace/default/book_data/).
Build Lakeflow Declarative Pipelines.
Practice on real datasets without a credit card.

If you have not set Free Edition up yet, the Workspace Essentials lesson walks you through it without skipping the parts that usually trip people up.

What this means for the DE Associate and Professional exams

Two practical things to remember.

First, the terminology has settled. Use "Lakeflow Declarative Pipelines", "Jobs and Pipelines", "Runs", and "Free Edition". The exam questions and the docs both use this language now. Older blog posts still say DLT and Workflows, but the platform language has moved. Note that Delta Sharing has evolved into OpenSharing, an open protocol for sharing data and AI assets.

Second, the Professional exam in particular leans into the production side: monitoring, partial reruns, observability, performance. The "disable task" feature, the Lakeflow Connect Free Tier, and the deeper Structured Streaming patterns all live in that area.

If you want to check where you stand:

Each one is free, scored honestly at 70 percent, and gives you a per-section breakdown plus a follow-up email pointing to the lesson on your weakest area.

A calm reading path through the 2026 changes

If you have an hour:

If you have an evening:

Medallion Architecture
Incremental Processing (now including Iceberg v3 support)
Workflows

If you have a weekend:

These nine lessons cover the ground that today's announcements actually sit on top of. New product names will keep showing up. The fundamentals will not change much.

Closing thought

2026 is not the year Databricks got fancier. It is the year Databricks got cheaper to learn on and harder to use lazily. The platform is doing more for you. The expectation is that you understand more of what it is doing.

That is also the bet behind BricksNotes. Calm, simple, practice-first lessons that respect your time and assume you want to think, not just memorize.

If you are ready to go deeper, the full book is here: Thinking in Data Engineering with Databricks.