Identity & Auth Foundations

One IdP layer, three token paths, and the authentication/authorization split that governs every downstream AI resource on Databricks.

Foundational reference for platform architects and engineers building governed AI applications — internal and external identity worlds, both covered.

Agenda

What this deck covers

1

Native IdP support

How Databricks already integrates with your cloud identity provider, and when you need Federation instead.

2

Authentication vs. authorization

Two different questions, enforced at two different layers — tokens prove identity, Unity Catalog enforces access.

3

Three token paths

OBO, M2M, and Federation — how each carries identity, and when each is the right choice.

4

Service principal identifiers

Account-level vs. workspace-level identifiers, and how each maps into Unity Catalog grants.

5

Configuration & gotchas

Concrete steps to configure OBO and Federation, and the mistakes that fail silently.

The issue

Every AI app has to answer one question before anything else: who is asking?

Get the token path wrong and you either lose per-user identity — row filters silently return the wrong data — or you introduce a shared credential where none was needed.

3

token paths, one governance model

2

identity worlds: internal & external

Internal teams are the easy half

Employees already authenticate through your workspace IdP. But the harder ask is already on the table: partners, customers, and regulated subsidiaries need the same governed access through their own identity providers — without ever getting a Databricks account.

What this deck answers

Which token path fits which identity world, how authentication and authorization stay separately enforced, and the concrete configuration steps for each path.

What — Native IdP support

Databricks already supports your cloud IdP. Federation is only for identities it doesn't know about.

Azure

Entra ID (your tenant) — default

Users authenticate via Entra. SSO, MFA, Conditional Access all work out of the box.

GCP

Cloud Identity

Google Workspace or Cloud Identity federation. Native SSO.

AWS

Bring your own IdP

Okta, Entra, Ping, or any SAML/OIDC provider via account-level SSO configuration.

Apps ON Databricks → OBO

Users are already authenticated via the workspace IdP. The Databricks Apps proxy injects the user's OBO token automatically. current_user() returns the human email. No extra IdP integration needed.

Apps OUTSIDE Databricks → Federation

An external app — a partner portal, a customer-facing SaaS, an internal tool on separate infrastructure — authenticates users with its own IdP. Those users don't have Databricks accounts. Federation bridges the external IdP via token exchange (RFC 8693).

What — Foundations

Authentication and authorization are two different questions, enforced at two different layers.

Authentication (AuthN) — Who are you?

Proven by OAuth tokens: OBO, M2M, or Federation exchange. The token carries the caller's identity through every layer of the stack.

OBO	User's OAuth token via the Apps proxy
M2M	Service principal client-credentials grant
Federation	IdP JWT exchanged for a Databricks token

Authorization (AuthZ) — What can you do?

Enforced by Unity Catalog at the SQL engine level: row filters, column masks, USE CONNECTION, EXECUTE on functions. Application code cannot bypass these controls.

Scopes	Limit what the token can do
UC grants	Limit what the identity can access
Both	Enforce independently; either can deny

What — Identity models

Three token paths. Who authenticates, who is credited, and which path applies.

OBO

On-Behalf-Of. The human authenticates; the app forwards their token.

Auth	Human via workspace IdP
Identity	`current_user()` = human email
Audit	Attributed to the human

When: Databricks Apps, custom tools, any flow where per-user data access matters.

M2M

Machine-to-Machine. No human involved. App SP authenticates with its own credentials.

Auth	SP client credentials (auto-injected by platform)
Identity	`current_user()` = SP application UUID
Audit	Attributed to the SP only

When: pipelines, shared knowledge bases, Vector Search, any operation without a user in the loop.

Federation

External IdP JWT exchanged for Databricks tokens via RFC 8693 token exchange.

Auth	External IdP JWT → Databricks token
Identity	Role-mapped SP or user role
Audit	SP-level; app server verifies the external user

When: external users, partner apps, any identity from an IdP outside Databricks.

What — OBO in practice

OBO: the user's identity, unchanged, all the way to Unity Catalog.

Userbrowser OAuth via workspace IdP

→

Databricks Appsauthenticates & forwards

→

Your Codereads x-forwarded-access-token

↓

token forwarded as Authorization: Bearer

Unity Catalogfires as the human

↓

current_user() = the human's email at every hop. Row filters and column masks fire automatically, with no app code changes. The app never handles login flows, passwords, or OAuth redirects — the platform does all of it.

What — Federation in practice

Federation: an external IdP JWT bridged to a scoped Databricks token.

External UserAuth0 / Okta / Entra tenant

→

Your App Serververifies JWT, maps role

→

/oidc/v1/tokenRFC 8693 exchange

↓

Databricks validates iss / aud / sub against the federation policy

Unity Catalogfires as the role-mapped SP

↓

Your app server is the trust boundary. It verifies the external JWT independently (JWKS signature check) and performs the token exchange server-side — Databricks never trusts the external IdP directly.

What — Service principals

Two identifiers for the same service principal. They are not interchangeable.

Account-level SP

Created at the Databricks account level. Can be assigned to multiple workspaces. Has a globally unique application_id (UUID).

application_id	UUID, used in `current_user()` and UC GRANTs
display_name	Human-readable label

Workspace-level identity

When assigned to a workspace, the SP gets a workspace-specific principal_id (integer), used for workspace-level ACLs.

principal_id	Integer, workspace-specific (e.g. CAN USE on a SQL warehouse)
Groups	Added to workspace groups for UC governance

Common confusion: application_id is what current_user() returns and what UC GRANTs reference. principal_id is for workspace-level permissions only. In a Federation pattern, each external role maps to a dedicated SP — e.g. sales_west role → fed-exchange-sp-sales-west → sales_west UC group.

Why it matters

The identity model is a governance decision, not just a plumbing detail.

One governance model

Row filters, column masks, and connections enforce identically whether the caller arrived via OBO or Federation. No duplicate engineering effort per identity world.

Audit & compliance

OBO gives full per-user attribution in system.access.audit. That's the difference between an audit log that names a human and one that only names a service principal.

Secretless architecture

Federation and Workload Identity Federation mean no Databricks credentials live in external infrastructure or CI/CD pipelines — fewer secrets to leak, rotate, or forget about.

IdP-agnostic by design: Auth0, Okta, Entra, or any OIDC provider — swap the federation policy config, keep the same architecture and the same audit trail.

Why it matters

Picking the wrong token path fails silently, not loudly.

Wrong path chosen

✕

M2M where OBO was needed. Row filters evaluate against the SP's identity, not the human's — every user sees the same rows. No error, just wrong data.

✕

Databricks credentials in external infra. Every secret checked into a partner's CI/CD config is a credential you now have to rotate, audit, and hope never leaks.

✕

No per-user audit trail. A security review asks “who accessed this row last quarter” and the honest answer is a service principal UUID, not a name.

Right path chosen

✓

OBO for per-user access. Every row filter, column mask, and audit entry names the actual human, automatically.

✓

Federation for external identities. Zero Databricks secrets outside the platform — token exchange is the only bridge, and it's short-lived.

✓

M2M for automation. Background jobs and shared knowledge bases run with fixed, auditable SP permissions — exactly what they should have.

How — Decision guide

Which path, for which situation.

Situation	Path	Why
Internal Databricks users	OBO	`current_user()` = human, per-user audit
External users with their own IdP	Federation	IdP JWT → scoped Databricks token
No user present (automation, CI/CD)	M2M	SP client credentials, no human in the loop
Role-based access across 100+ external users	Federation	Role → SP → UC group mapping scales
Both internal and external users on the same tools	OBO + Federation	Same UC governance, same MCP tools, same audit table

Key insight: all three patterns share the same UC governance, tools, and audit infrastructure. The only difference is how the token reaches your app.

How — Configuring OBO

Three requirements: read the proxy header, add required scopes, configure User Authorization.

Step 1: Read the forwarded token (Streamlit)

import streamlit as st

# Use st.context.headers, not request.headers
token = st.context.headers.get(
  "x-forwarded-access-token", ""
)
host = st.context.headers.get(
  "x-forwarded-host", ""
)
# current_user() returns the human

Step 2: Enable User Authorization + scopes

Account Console → App integrations → enable User Authorization → select scopes:

`sql`	SQL statement execution
`genie` + `dashboards.genie`	Genie Agents API (both required)
`serving.serving-endpoints`	Model Serving / Agent Bricks
`catalog.connections`	External MCP (UC HTTP)
`unity-catalog`	UC governance operations

UI User Authorization (not CLI) produces real OBO tokens with effective scopes. Users consent on next login.

How — M2M OAuth & Federation

M2M uses two token endpoints: workspace-level and account-level. Federation uses token exchange (RFC 8693).

M2M OAuth token endpoints

# Workspace-level (typical)
POST https://<workspace-host>
  /oidc/v1/token

# Account-level (for account ops)
POST https://accounts.cloud
  .databricks.com/oidc/accounts
  /<account-id>/v1/token

grant_type=client_credentials
&client_id=<app-uuid>
&client_secret=<secret>
&scope=all-apis  /* or: sql, jobs, clusters */

M2M service principals can now request scoped OAuth secrets that restrict a token to specific API scopes (for example sql, jobs, clusters) instead of blanket all-apis access, reducing blast radius if a secret is compromised.

Federation token exchange (RFC 8693)

POST https://<workspace-host>
  /oidc/v1/token

grant_type=urn:ietf:params:
  oauth:grant-type:token-exchange
&subject_token=<external_jwt>
&subject_token_type=
  urn:ietf:params:oauth:token-type:jwt
&scope=sql genie serving

# Databricks validates iss/aud/sub

How — Gotchas

Silent failures. These break with no error or confusing 403.

Issue	Detail
`genie` scope not shown in UI	Genie Agents API requires both `genie` AND `dashboards.genie` scopes. The UI only shows `dashboards.genie`. Add `genie` via API: account custom-app-integration update.
`unity-catalog` scope for External MCP	The External MCP proxy checks `USE CONNECTION` privilege and requires the `unity-catalog` scope. Missing it returns `403: "does not have required scopes: unity-catalog"`. Not in docs.
`st.context.headers` not `request.headers`	In Streamlit, `request.headers` (Flask pattern) does not have the proxy headers. Always use `st.context.headers`.
`application_id` vs `principal_id`	UC GRANTs reference `application_id` (UUID). Workspace ACLs use `principal_id` (integer). Using one where the other is needed returns cryptic errors.
`iss` trailing-slash must match exactly	Federation validates the IdP issuer URL exactly. A trailing slash mismatch fails validation with a 401, not a helpful error message.

ONE IDP. THREE PATHS. ONE MODEL.

Authentication proves who's asking. Authorization decides what they see. Neither substitutes for the other.

Databricks Applied AI Governance Series · Identity & Auth Foundations

→ / Space	Next slide
←	Previous slide
Home	First slide
End	Last slide
Swipe	Touch navigation

Navigation

Identity & Auth Foundations

What this deck covers

Every AI app has to answer one question before anything else: who is asking?

Internal teams are the easy half

What this deck answers

Databricks already supports your cloud IdP. Federation is only for identities it doesn't know about.

Azure

GCP

AWS

Apps ON Databricks → OBO

Apps OUTSIDE Databricks → Federation

Authentication and authorization are two different questions, enforced at two different layers.

Authentication (AuthN) — Who are you?

Authorization (AuthZ) — What can you do?

Three token paths. Who authenticates, who is credited, and which path applies.

OBO: the user's identity, unchanged, all the way to Unity Catalog.

Federation: an external IdP JWT bridged to a scoped Databricks token.

Two identifiers for the same service principal. They are not interchangeable.

Account-level SP

Workspace-level identity

The identity model is a governance decision, not just a plumbing detail.

One governance model

Audit & compliance

Secretless architecture

Picking the wrong token path fails silently, not loudly.

Which path, for which situation.

Three requirements: read the proxy header, add required scopes, configure User Authorization.

M2M uses two token endpoints: workspace-level and account-level. Federation uses token exchange (RFC 8693).

Silent failures. These break with no error or confusing 403.