| → / Space | Next slide |
| ← | Previous slide |
| Home | First slide |
| End | Last slide |
| Swipe | Touch navigation |
Builds on Identity & Auth Foundations and Authorization by Resource Type — this deck is about what happens when agents and tools compose.
1. The four pillars of AI orchestration and where governance is enforced in each
2. Agent Bricks supervisor/sub-agent identity propagation, and the three agent auth methods
3. MCP's three server types and UC Connections as the outbound governance primitive
4. Unity AI Gateway as the single control surface for LLMs, MCPs, and Model Serving
5. Token federation for automation and external callers
6. Configuration steps, common gotchas, and a production checklist
7. Building new agents in 2026: Genie Agents, custom agents on Databricks Apps, and the Supervisor API deprecation
How identity propagates through Model Serving, MCP, and Databricks Apps; where AI Gateway fits (and where it doesn't); and the concrete steps to keep every hop auditable.
AI Gateway layers onto the LLM and MCP traffic within this stack; UC Connections govern the exit to external services. Same Unity Catalog enforcement regardless of which pillar the request passes through.
Supervisor auto-forwards the user's token to all sub-agents. current_user() = human email from the supervisor down to the deepest tool call — row filters, Genie Agents RLS, and function auth all see the actual human.
genie and dashboards.genie scopes, or a 403Simplest — Databricks manages everything.
| How | Declare resource dependencies at log time |
| Identity | System-generated SP, least privilege |
| Best for | No per-user access needed |
Agent runs as the calling user.
| How | Initialize in predict() with user credentials |
| Identity | current_user() = human email |
| Best for | Per-user access control + audit |
Explicit credentials, maximum flexibility.
| How | SP OAuth (recommended) via UC secret scopes |
| Identity | Service principal |
| Best for | External resources, prompt registry |
Key constraint: OBO's calling identity is only known at query time — resources must be initialized inside predict(), not __init__().
Pre-built connectors: Genie Agents, Vector Search, UC Functions, Databricks SQL. No code, no infra. Auth: automatic, OBO, or PAT.
Your code, deployed as a Databricks App. Full control over tools and auth — OBO, M2M, or mixed. Auth is OAuth only, PAT not used.
MCP Services (recommended): Seven built-in services system.ai.slack, system.ai.github, system.ai.atlassian, system.ai.google_drive, system.ai.google_calendar, system.ai.gmail, system.ai.microsoft_365 — zero setup, grant EXECUTE. Governed through Unity AI Gateway with tool filtering, service policies (allow, deny, or require approval), rate limits, and networking controls. OAuth tokens calling MCP Services require the ai-gateway scope. Legacy approach: UC HTTP Connections for custom external MCP servers.
Tool selection priority on name collision: Managed (highest trust) → Custom (explicit business logic) → External (third-party). Configure explicit tool routing in the supervisor when ambiguity exists.
| Network (SNP) | Workspace-level FQDN allowlist — anything not listed is unreachable |
| Credential (UC) | USE CONNECTION grant controls which SP can authenticate, enforced before any network traffic |
-- Grant access GRANT USE CONNECTION ON CONNECTION github_api TO `sp-appeals`; -- Instant revocation, no redeploy REVOKE USE CONNECTION ON CONNECTION github_api FROM `sp-billing`;
Four auth methods, only two give per-user identity externally: Bearer and OAuth M2M share one identity with the external service; OAuth U2M Per User and Managed OAuth give each user their own token on both sides.
| Pattern | Caller → target | Governance layer |
|---|---|---|
| 1 | Agent to Databricks services (Genie Agents, FM API, UC) | No gateway. Direct OBO — UC governs at the data plane |
| 2 | Any caller to an LLM endpoint | Databricks AI Gateway. Rate limits, guardrails, usage tracking, fallback routing |
| 3 | Agent to external services (GitHub, Slack, etc.) | UC Connections + SNP. Proxy model, REVOKE = instant removal |
| 4 | External clients to Databricks | External API Gateway. Auth translation, per-tenant rate limiting, versioning |
Why pattern 1 skips the gateway: re-issuing a token at a gateway makes current_user() resolve to the gateway service, not the human — row filters then fire as the gateway. UC enforcement happens at the data plane; a gateway can observe HTTP traffic but not which rows were filtered.
Agents, LLM endpoints, and coding agents (Cursor, Codex CLI, Claude Code) route through gateway endpoints with native multi-provider APIs.
External, custom, and Databricks-managed MCP servers governed under the same permissions, audit, and access controls.
Gateway features layered onto existing endpoints — safety, PII, and keyword guardrails, plus the established system-table observability surface.
One control point: permissions · rate limits · guardrails · usage tracking · inference logging · cost attribution · fallback routing · traffic splitting — applied consistently across every pillar. Coding-agent calls and deployed-agent calls land in the same system.ai_gateway.usage table.
Rate limits per user, group, or SP cap a runaway agent before it drives an unexpected model bill. system.ai_gateway.usage joined with system.billing.usage attributes spend by team, app, or project.
REVOKE USE CONNECTION is a runtime kill switch — no redeploy needed to cut off a compromised or misbehaving tool. Guardrails catch PII and unsafe content before it reaches a model or a caller.
One audit trail across coding agents, deployed agents, and tool calls means one place to answer “who called what, and what did it return” — instead of stitching together per-server logs.
Before this unification, MCP governance was ad hoc: UC handled data access, but request-level audit and rate limiting were per-server. Now LLM calls and tool invocations share one permissions model and one configuration surface.
| 1 | App authenticates with your IdP, receives a JWT |
| 2 | POST the JWT to /oidc/v1/token with grant_type=urn:ietf:params:oauth:grant-type:token-exchange |
| 3 | Databricks validates the JWT against the federation policy, returns a scoped OAuth token |
| 4 | Use the Databricks token to call APIs: Authorization: Bearer <token> |
| Feature | Account Federation | WIF |
|---|---|---|
| Applies to | Users & SPs | Workload → SP |
| SCIM required | Yes | No |
| Issuer limit | 5 | Unlimited |
| Refresh tokens | Yes (U2M) | No |
AWS IAM and GitHub Actions OIDC: WIF only. Token TTL copies the IdP's exp claim verbatim — plan refresh in client code.
| Symptom | Root cause | Fix |
|---|---|---|
| Genie Agents returns wrong rows, no error | is_member() in a row filter | Replace with current_user() — is_member() evaluates the execution service under Genie Agents/Agent Bricks OBO |
| User identity lost in App A → App B | Two-proxy stripping | Use X-Forwarded-Email (proxy-signed, survives hops) + M2M SQL with an email-based filter |
| Sub-agent 403 on a Genie Agent tool | Scope not granted on the sub-agent's own SP | Grant UC permissions and OAuth scopes per sub-agent — nothing cascades from the supervisor |
| External tool stops working post-deploy | Connection ownership tied to the old SP | Set connection ownership to the app SP; re-grant USE CONNECTION after every redeploy |
Scope ceiling: a token without serving, genie + dashboards.genie, or mcp.* cannot call that resource, even if UC grants would otherwise allow it. Request only what the agent needs.
Natural-language “talk to your data” for business users and supported low-code orchestration over structured data, document search, UC functions, and MCP tools. Governed by Unity Catalog, with OBO carrying the human identity through every sub-agent.
The recommended target for new code-based and multi-agent systems, built with LangGraph, the OpenAI Agents SDK, or a custom loop. Git packaging, app or on-behalf-of auth, MCP tools, Unity Gateway model access, and MLflow tracing and evaluation. You own the agent loop, state, retries, and tool authorization.
Managed, custom, and external MCP servers remain the preferred way to expose governed tools, and existing Unity Catalog function tools are not deprecated. MLflow Tracing gives one execution trace across model and tool calls.
Migrate Supervisor API workloads to a custom agent loop on Databricks Apps. The replacement hands you the agent loop, state, retries, and deployment lifecycle, so plan an application change rather than a URL swap. The managed Supervisor Agent and Genie Agents orchestration paths shown earlier are separate products and are unaffected by this change.
Source: Databricks docs — Build a custom agent using the Supervisor API (Beta) (deprecated), docs.databricks.com/aws/en/agents/custom-agents/supervisor-api-app