Building one agent is easy. Running a whole fleet of them securely is the hard part.
It started with one agent.
A data engineer built it on a Friday afternoon. It read a support ticket, looked up the customer order, and drafted a reply. The demo was great. Leadership loved it.
By March there were forty agents.
Finance had one for reconciliations. Marketing had three for campaign reports. The data platform team had one that checked pipeline failures. Each team picked its own model, its own vector store, its own prompt library and its own way of storing API keys.
Then a column in the orders table was renamed.
Seven agents started giving wrong answers. Nobody knew which seven. Nobody knew which tables they read. The monthly model bill had doubled, and nobody could say which agent spent it.
This is the moment every company reaches. Building one agent is easy. Running a fleet of them securely is the hard part.
A single agent is a script. A fleet is infrastructure.
When we treat each agent as its own project, every team rebuilds the same things. Authentication. Tool access. Logging. Cost limits. Data lookups. That is weeks of work per agent, and most of it is the same work done slightly differently.
Data engineers have seen this before. Before the lakehouse, every team built its own data mart with its own copy of the truth. We fixed that with shared storage, a shared catalog and shared rules.
Agent fleets need the same fix. The question is not "how do I build a better agent?" The question is "what shared platform should every agent stand on?"
In The Context Advantage, we describe that platform with four ideas: Context, Control, Cost and Choice. For agent fleets, three of them carry most of the weight.
Models change every few months. A model that is the best today may be the expensive option next quarter.
If every agent calls one vendor's API directly, switching means rewriting forty agents. That is not choice. That is lock-in with extra steps.
Choice means the agent asks a shared serving layer for a model by name, and the platform decides where that call goes. On Databricks this is the role of Model Serving and the AI Gateway. You can route a simple classification agent to a small, cheap model and a complex planning agent to a larger one, without changing either agent's code.
The engineering habit is simple. Keep the agent definition separate from the model it uses. Treat the model as configuration, not as code.
Most agent failures are not model failures. They are context failures.
The model was smart enough. It simply read the wrong table, an old copy of the data, or a column whose meaning had changed. As we wrote in Context, Not Intelligence, enterprise AI is stuck on context far more than on raw model power.
For a fleet, context must be shared and governed:
When context lives in one place, the renamed column breaks one lineage graph you can see. It does not silently break seven agents you cannot see.
This is the part that keeps security teams awake.
An agent that can call tools can change things. It can query salaries, send emails or update records. If its credentials live in an environment variable, it has whatever power that key has, for everyone, forever.
Control means three rules:
refund_order is granted to a small group, not to every agent.The Model Context Protocol helps here because it gives agents one standard way to reach tools. But a protocol is only a doorway. The catalog decides who walks through it.
flowchart TB
subgraph Fleet["Agent fleet"]
A1["Support agent"]
A2["Finance agent"]
A3["Pipeline agent"]
end
subgraph Platform["Shared platform"]
G["Model gateway (Choice)"]
UC["Unity Catalog: tables, tools, grants (Control)"]
CX["Delta tables + Vector Search (Context)"]
AU["Audit table"]
end
A1 --> G
A2 --> G
A3 --> G
A1 --> UC
A2 --> UC
A3 --> UC
UC --> CX
G --> AU
UC --> AUNotice what is missing. No agent talks to storage directly. No agent holds its own keys. Every new agent plugs into the same three foundations on day one.
You can practice the control idea today in Databricks Free Edition. We will simulate tool calls from three agents, store them in a Delta table, and answer the questions that leadership asked in our story.
from pyspark.sql import functions as F
# Simulated tool calls from three agents
call_rows = [
("support_agent", "lookup_order", "orders", "ana@corp.com", 420, "ok"),
("support_agent", "lookup_order", "orders", "ravi@corp.com", 380, "ok"),
("finance_agent", "read_ledger", "ledger", "lee@corp.com", 2100, "ok"),
("finance_agent", "refund_order", "orders", "lee@corp.com", 650, "denied"),
("pipeline_agent","read_job_runs", "job_runs", "sam@corp.com", 900, "ok"),
("support_agent", "lookup_order", "orders", "ana@corp.com", 510, "error"),
]
columns = ["agent_name", "tool_name", "table_read", "on_behalf_of", "tokens_used", "status"]
calls_df = spark.createDataFrame(call_rows, columns)
calls_with_time_df = calls_df.withColumn("called_at", F.current_timestamp())
calls_with_time_df.write.mode("overwrite").saveAsTable("workspace.default.agent_tool_calls")Now ask the fleet questions in SQL.
-- Which agents spend the most tokens?
SELECT
agent_name,
COUNT(*) AS call_count,
SUM(tokens_used) AS total_tokens
FROM workspace.default.agent_tool_calls
GROUP BY agent_name
ORDER BY total_tokens DESC;
-- Which agents depend on the orders table?
SELECT DISTINCT agent_name
FROM workspace.default.agent_tool_calls
WHERE table_read = 'orders';
-- Which calls were blocked or failed?
SELECT agent_name, tool_name, on_behalf_of, status
FROM workspace.default.agent_tool_calls
WHERE status <> 'ok';Look at the second query. That is the answer to "which seven agents broke when the column was renamed?" One query, because every call landed in one place.
In a paid workspace, Databricks can capture much of this automatically through inference tables and system tables. The concept is the same. Free Edition lets you understand it by building it yourself.
Companies will not hire people to build the fortieth chatbot. They will hire people who can make forty agents safe, cheap and correct.
That work is data engineering. Catalogs, lineage, fresh tables, access rules, audit logs and cost reports. The skills you already practice are exactly the skills an agent fleet needs.
BricksNotes teaches how Databricks behaves underneath: tables, pipelines, governance and performance.
The Context Advantage goes one level up. It asks how organizations turn that foundation into AI that can be trusted. Choice, so you are never stuck with one vendor. Context, so agents know what your business knows. Control, so every action is owned and visible. Cost, so growth does not become a surprise bill.
Read them together and the picture becomes clear. The agent is the easy part. The platform underneath is the advantage.
Build the platform once, and every new agent becomes a small, safe step instead of another stack to rebuild.
Team BricksNotes