Federation Exchange
Under the Hood

IDP-Agnostic Identity Chain · Governance Enforcement · Error Reference

Works with Okta · Microsoft Entra ID · any OIDC-compliant IdP

The Decision

Two Paths for External Users, and Why SPs Win

Your channel partners, external auditors, and consultants need governed access to Databricks. They have their own IDP. They are not in your directory. Two architectural options:

Option A: On-Behalf-Of (OBO)

Each external user is provisioned as a Databricks identity

How it works: External user authenticates → their token is exchanged for a Databricks token that represents them as an individual. current_user() returns their email. Row filters can use current_user() = email.

Pros

Individual audit, current_user() = real person
Per-user row filters possible (WHERE owner = current_user())
Native Databricks identity lifecycle

Blockers

User must exist in Databricks (synced via SCIM from your corporate IDP)
External users from different orgs can't be SCIM-synced, they're not in your directory
Guest account provisioning for every external user, lifecycle management nightmare
Cross-tenant IDP federation required at the directory level
Doesn't scale: 500 partner reps = 500 Databricks identities to manage

Option B: Role-Based Service Principals

N external users map to O(roles) SPs, no Databricks provisioning

How it works: External user authenticates with their IDP → JWT exchanged for Databricks token scoped to a role-SP. Governance fires via is_member('group'). Real identity preserved in JWT claims + audit.

Why this wins

No Databricks user provisioning, external users never touch your directory
Scales with roles, not users: 500 reps → 2 SPs (west + east)
Works with any OIDC IDP, no cross-tenant federation needed
UC governance fires natively via is_member() on SP groups
Identity preserved, JWT claims → audit table + MLflow traces

Trade-off

current_user() returns SP UUID, not individual email
Per-user row filters need is_member() groups, not current_user() = email
Individual attribution requires app-level audit (solved by JWT-tagged audit table)

The SP handles authorization. The JWT handles attribution. Together they give you governed access + full audit without provisioning a single Databricks user.

The Mental Model

Three Actors Make Federation Work

Every OIDC IDP has these three components. The names differ, the roles don't.

Actor 1: The API Declaration

"Databricks exists at this address"

Declares the audience, the workspace URL. Goes into the JWT aud claim.

Generic	API / resource server (audience)
Okta	Authorization Server
Entra	App Reg → "Expose an API"

// IDP config
{
  "identifier": "https://"
}

// JWT claim produced
"aud": "https://"

// Databricks federation policy
"audiences": ["https://"]

Actor 2: The M2M App

"The machine identity of your app server"

App server uses client_credentials grant. client_id becomes JWT sub claim.

Generic	`sub` = client id (format varies by IdP)
Okta	`sub` = `{client_id}`
Entra	`sub` = `{app_id}`

// App server token request
POST /oauth/token
{
  "client_id": "your-m2m-client-id",
  "client_secret": "***",
  "audience": "https://",
  "grant_type": "client_credentials"
}

// JWT sub claim produced
"sub": "your-m2m-client-id@clients"

// Databricks federation policy
"subject": "your-m2m-client-id@clients"

Actor 3: The SPA App

"Identifies the human at the keyboard"

Browser PKCE login. IDP injects role, email, groups into ID token. Never talks to Databricks.

// You write this inside your IDP:
//   OIDC IdP: post-login / claims hook
//   Okta  → Token Inline Hook
//   Entra → Claims Mapping Policy
// It runs at token issuance time.

token.claims["/role"] = user.role;
  // → "west_sales"
token.claims["/email"] = user.email;
  // → "user@partner.example"

// ID token the browser receives
{
  "email": "user@partner.example",
  "https://fed/role": "west_sales",
  "https://fed/groups": ["sales"]
}

// Your app reads role → picks SP
"west_sales" → SP sp-west-sales-id

The handshake: IDP produces a JWT with iss, aud, sub. Databricks federation policy validates all three. They're mirror images, if any field doesn't match: 401 / 403.

The Architecture

External User → IDP Token → Databricks Token

The external user authenticates with their own IDP. Their JWT is exchanged for a Databricks OAuth token scoped to a role-specific Service Principal. UC row filters, column masks, and connection grants enforce governance. The real external identity is preserved end-to-end for audit.

What Changes Per IDP

Token endpoint URL
JWKS endpoint URL
Custom claims injection method
M2M / client_credentials flow setup
Token dialect / claim namespace

What Stays the Same

Databricks /oidc/v1/token exchange endpoint
Federation policy on each SP (issuer + audience + subject)
Role → SP mapping
UC row filters, column masks, connection grants
Audit table, MLflow tracing

End-to-End Sequence

From Login to Governed Data Access

1

User authenticates with IDP

PKCE / Authorization Code flow

2

IDP issues JWT + custom claims

role, email, groups

3

App resolves role → SP

"west_sales" → sp-west-sales-id

4

Token Exchange

IDP JWT → scoped Databricks SP token

5

App calls MCP server

DB token + IDP JWT + role header

6

MCP verifies JWT independently

JWKS check → server-side exchange

7

UC governance fires

Row filter + Column mask + USE CONNECTION

8

Audit with external identity

Email, role, tool, latency recorded

The token at each stage

Stage 1 · IdP JWT

Signed by the IdP (RS256). Carries iss, aud, sub, and custom groups / role / email claims. Verifiable via the IdP's JWKS.

Stage 2 · Exchange

POST /oidc/v1/token with the IdP JWT as subject_token and the role SP as client_id. The federation policy validates iss + aud + sub.

Stage 3 · Databricks token

Opaque Bearer token scoped to the role SP. current_user() is the SP; is_account_group_member() reflects the SP's groups, so UC row filters and column masks fire.

The eight steps on the left trace one request from login to governed data access. The panel on the right shows what the token looks like at each of the three stages.

Token Anatomy

What's in Each Token at Each Stage

Stage 1: IDP JWT

Issued by external IDP

{
  "iss": "https://{idp-domain}/",
  "aud": "https://",
  "sub": "{client_id}@clients",
  "exp": 1710612000,
  // Custom claims (namespace varies by IDP)
  "https://ns.example.com/groups": ["sales-west"],
  "https://ns.example.com/role": "sales-west",
  "https://ns.example.com/email": "user@partner.example"
}

Algorithm: RS256 · Signed by IDP's private key · Verifiable via JWKS

Stage 2: Databricks Token

From /oidc/v1/token exchange

{
  "access_token": "eyJhbG...",
  "token_type": "Bearer",
  "expires_in": 3600
}
// The access_token is a Databricks
// OAuth token scoped to the role-SP.
// current_user() = SP application_id
// is_member() = SP's group membership

Opaque to the caller · Encodes SP identity · UC enforcement via SP's groups

Stage 3: Governance Context

Derived from SP identity at query time

-- Row filter function
SELECT *
FROM sales.opportunities
-- UC auto-applies:
WHERE is_member('west_sales')
  AND opp_region = 'WEST'

-- Column mask function
CASE WHEN is_member('finance')
  THEN margin_pct
  ELSE NULL END

Fires automatically · No application code needed · Can't be bypassed

Governance Enforcement

7 Enforcement Points in the Chain

#	Enforcement Point	Layer	What Fires	Failure Mode
1	IDP Authentication	External IDP	PKCE / password / SSO → JWT issued	Login failed → no JWT issued
2	Claims Injection	IDP Action/Hook	Groups, role, email injected into JWT	Missing claims → role resolution fails at step 5
3	Federation Policy Validation	Databricks /oidc/v1/token	Issuer + audience + subject checked against SP's policy	`400/401/403` → no Databricks token
4	JWT Signature Verification	API Server (Databricks Apps)	JWKS-based RS256 verification + audience + issuer	`401` → request rejected
5	Role → SP Mapping	API Server (Databricks Apps)	JWT role claim → SP application_id lookup	`403` → unknown role
6	Tool Access Control	API Server (Databricks Apps)	TOOL_ACCESS dict: role ∈ allowed_roles?	ACCESS_DENIED logged to audit
7	UC Governance	Databricks SQL Engine	Row filters (`is_member()`), column masks, USE CONNECTION grants	Filtered/masked silently, never errors, just restricts

Key Insight: Silent vs. Loud Enforcement

Points 1-5 are loud, they return HTTP errors and block the request. Point 6 is semi-loud, it returns a structured ACCESS_DENIED response and logs to audit. Point 7 is silent, UC row filters and column masks never error; they just restrict what data you see. A query that returns 0 rows is governance working, not a bug.

Error Reference

What Goes Wrong and What It Means

HTTP	Source	Error	Cause	Fix
`400`	App Server (customer)	Missing tool or role	Request body missing `tool` or `role` field	Include both fields in POST body
`400`	App Server (customer)	Unknown role	Role not in ROLE_SP_MAP	Check role key matches config exactly
`400`	/oidc/v1/token	invalid_request	Malformed JWT, missing fields	Verify JWT is well-formed, all params present
`401`	/oidc/v1/token	invalid_grant	JWT signature invalid, issuer mismatch	Check JWKS keys, issuer URL (trailing slash matters!)
`401`	API Server (Databricks)	Invalid IDP token	JWKS verification failed (expired, wrong key, tampered)	Check IDP JWKS endpoint, token expiry, algorithm
`403`	/oidc/v1/token	OIDC policy validation failed	No federation policy on SP, or subject claim mismatch	Create/update federation policy: issuer + audience + subject
`403`	API Server (Databricks)	No role determined	JWT has no role claim, no X-Federation-Role header	Add role to JWT claims or pass header
`403`	API Server (Databricks)	Unknown role: '{role}'	Role not in server's ROLE_SP_MAP	Add role to ROLE_SP_MAP env var
`429`	Federation Policy API	Rate limited	Too many policy creation requests	Exponential backoff: 2ⁿ⁺¹ seconds
`502`	App Server / API Server	Token exchange failed	Databricks rejected the exchange	Check federation policy, SP status, JWT claims
`200`	MCP Tool	ACCESS_DENIED	Role not in TOOL_ACCESS for this tool	Expected behavior, logged to audit table
`200`	SQL Engine	0 rows returned	UC row filter excluded all rows for this SP's groups	Expected behavior, governance working correctly
`200`	SQL Engine	NULL in column	UC column mask returned NULL for non-privileged group	Expected behavior, column masking active

Troubleshooting

Decision Tree: Where Is It Failing?

IDP & Token Exchange

?

Can you get an IDP JWT?

Test: curl POST {idp}/oauth/token

N

→ IDP config issue

Check: client_id, client_secret, audience, grant_type

?

Does JWT have custom claims?

Test: echo $JWT | cut -d. -f2 | base64 -d | jq

N

→ Claims injection issue

Check IDP's claim injection mechanism: Action, Hook, or Policy

?

Does token exchange succeed?

Test: curl POST {workspace}/oidc/v1/token

N

→ Federation policy issue

Check: policy exists on SP, issuer matches (trailing slash!), audience = workspace URL, subject matches sub claim

API Server & Governance

?

Does API server accept the call?

Check: HTTP status from API endpoint

401

→ JWT verification failed

Check: JWKS endpoint accessible, RS256 algorithm, audience + issuer match server config

403

→ Role mapping failed

Check: JWT has role claim, role key exists in ROLE_SP_MAP config

?

Data returned but empty or masked?

OK

→ Governance is working correctly

0 rows = row filter active
NULL values = column mask active
ACCESS_DENIED = tool access control

Verify: SELECT is_member('group') as the SP

Setup Guide

What to Configure, In Order

Identity Provider Side

1Create an API / Resource Server

Register a custom API with audience = Databricks workspace URL. This tells the IDP: "JWTs for this audience are meant for Databricks."

audience: https://

2Create an M2M / Service App

For the external app server to get JWTs via client_credentials grant. This app's client_id becomes the sub claim in the JWT.

3Create a SPA App (if browser login needed)

For end-users to authenticate via PKCE in the browser. Redirect URI = your app's callback URL.

4Configure Users & Groups

Users typically already exist in the IDP. Assign them to groups that map to Databricks roles. Group membership drives which SP the user's token exchanges for.

Groups are native objects in most IdPs (Okta, Entra). Some IdPs keep group membership in user metadata that a claims hook copies into the token.

5Add Claims Injection Logic

Inject groups, role, email into the access token. This is the IDP-specific part:

Generic	Claims hook at token issuance
Okta	Authorization Server custom claims
Entra	Claims Mapping Policy + App Roles

Databricks Side

6Create Role-Based Service Principals

One SP per role (not per user). Each SP gets a unique application_id and numeric_id.

sp-west-sales, sp-finance, etc.

7Add SPs to Workspace Groups

Create workspace groups (e.g., west_sales) and add each SP. These groups are what is_member() checks in row filters and column masks.

8Create Federation Policies

On each SP, register the IDP's issuer, audience, and subject. This is what Databricks checks when validating the JWT during token exchange.

Account-level API, requires Databricks account admin.

9Configure UC Governance

Set up row filters (is_member('group')), column masks, and GRANT USE CONNECTION on tables. Grant SPs SELECT on data tables and SQL warehouse access.

10Deploy API Server + External App

API server on Databricks Apps (JWT verification + server-side exchange). External app anywhere (your infra). Configure both with the IDP's domain, client_id, JWKS URI, and the role → SP mapping.

The Connection

How IDP Config Maps to Databricks Config

IDP produces → Federation Policy consumes

IDP Config	Produces	Databricks Field
IDP Tenant / Domain	`iss` claim	Policy `issuer`
API / Resource Server	`aud` claim	Policy `audiences`
M2M App client_id	`sub` claim	Policy `subject`
Claims injection	`role`, `groups`	App-level ROLE_SP_MAP

Create the policy (one API call per SP)

// Databricks Account API
POST accounts/{account_id}
  /servicePrincipals/{sp_id}
  /federationPolicies

{
  "oidc_policy": {
    "issuer":    "https://{idp-domain}/",
    "audiences": ["https://"],
    "subject":   "{m2m_client_id}@clients"
  }
}

// Same 3 fields on all role SPs.
// Swap IDP → update these 3 → done.

The federation policy is the only config that connects the IDP to Databricks. Everything else (SPs, groups, UC governance) is IDP-independent.

Runtime Config

Role → SP Mapping + Grants Checklist

How the app server knows which SP to target

Static config map - created once when you set up the role SPs. Lives in the app server's env config.

// App server config (env var)
ROLE_SP_MAP = {
  "west_sales":  "sp-west-sales-id",
  "east_sales":  "sp-east-sales-id",
  "managers":    "sp-managers-id",
  "executive":   "sp-executive-id",
  "finance":     "sp-finance-id",
  "admin":       "sp-admin-id"
}

// Runtime flow:
// 1. SPA login → IDP token: role="west_sales"
// 2. App looks up ROLE_SP_MAP["west_sales"]
// 3. Token exchange: client_id=sp-west-sales-id

These are Databricks SP application_ids - not IDP client_ids. Created during setup, static thereafter. Only changes when you add/remove roles.

Databricks grants checklist

Grant	On	To
Federation Policy	Each role-SP	,
CAN_USE	SQL Warehouse	All SPs
USE CATALOG	Data catalog	All SPs
USE SCHEMA	Data schema(s)	All SPs
SELECT	Data tables	Role SPs
SELECT	VS index	App SP
MODIFY + SELECT	Audit table	App SP only
USE CONNECTION	external_api_conn	Exec + Admin
CAN_QUERY	Serving endpoint	Exec + Admin
CAN_RUN	Genie Agent	All role SPs
Workspace group	Group per role	Each role SP

Scopes (Token Exchange)

Currently: scope=all-apis. Minimum required per tool:

Role SPs	`sql` + `genie` + `serving`
App SP	`sql` + `vector-search`

The Databricks Side

Federation Policy, Two Levels

SP-Level Policy Recommended for Federation Exchange

Policy is scoped to a specific Service Principal. Only that SP can exchange tokens from this IDP.

POST /accounts/{account_id}
     /servicePrincipals/{sp_numeric_id}
     /federationPolicies

When to use:

Federation Exchange, each role-SP trusts one IDP with one subject
Per-partner access, Partner A's IDP can only exchange for Partner A's SP
Least privilege, tight blast radius if credentials leak

Visibility: Account Console → Service principals → {SP} → Credentials & secrets → Federation policies.

Account-Level Policy

Policy applies to any identity in the account. Any SP or user can exchange tokens from this IDP.

POST /accounts/{account_id}
     /federationPolicies

When to use:

CI/CD, GitHub Actions, GitLab CI, Azure DevOps OIDC tokens
Workload identity, cloud-native services (EKS, GKE) with service account tokens
Broad automation, any SP should be able to use this IDP

Visibility: Visible in Account Console → Security → Authentication → Federation policies.

Both levels validate the same 3 fields against the JWT:

Policy Fields (Same for Both Levels)

{
  "oidc_policy": {
    "issuer":    "https://{idp-domain}/",
    "audiences": ["https://"],
    "subject":   "{jwt-sub-claim}"
  }
}

Common Gotcha: Trailing Slash

The issuer URL must exactly match the iss claim in the JWT. Most IDPs include a trailing slash: https://login.example.com/. Omitting it = policy validation failure.

Subject Claim by IDP

OIDC IdP (varies)	`{client_id}` or `{client_id}@clients`
Okta Service	`{client_id}`
Entra App	`{app_id}` (GUID)
GitHub Actions	`repo:{org}/{repo}:ref:refs/heads/main`

Token Exchange Parameters (IDP-Agnostic)

grant_type=urn:ietf:params:oauth:grant-type:token-exchange
subject_token={IDP_JWT}
subject_token_type=urn:ietf:params:oauth:token-type:jwt
client_id={SP_APPLICATION_ID}
scope=all-apis

IdP: The Generic Pattern

Configuring Any OIDC IdP

Endpoints (from the IdP's discovery document)
Token	`https://{idp-domain}/oauth/token`
JWKS	`https://{idp-domain}/.well-known/jwks.json`
Issuer	`https://{idp-domain}/` (match the `iss` claim exactly)
Subject (M2M)	the machine client's id (format varies by IdP)

Claims Injection (at token issuance)

Every IdP has a hook that runs when it issues a token. Use it to copy the user's group membership and role into the access token.

// pseudocode, IdP-specific hook
const groups = user.groups;
token.setClaim("groups", groups);
token.setClaim("role", resolveRole(groups));

Common Gotchas

Claim namespace: some IdPs require a non-vendor namespace or the custom claims are silently dropped
Token content: some IdPs need an explicit setting to include authorization claims in the access token
Issuer string: must match the iss claim exactly, trailing slash included
Subject format: confirm the exact sub string before writing the federation policy

Apps You Register in the IdP

M2M	client_credentials flow for the external app server
SPA	PKCE flow for browser login
ADMIN	management API access (automation only, optional)

What to Configure

API / resource server	audience = workspace URL
Claims hook	inject groups, role, email
Users and groups	assign users to groups that map to roles
Connection	the IdP's user directory or SSO source

Federation Policy Subject

"subject": "{M2M_CLIENT_ID}"
// exact format is IdP-specific

OAuth Scopes at Each Stage

SPA Login	`openid profile email` Standard OIDC scopes → ID token + user info
M2M Token	audience = workspace URL Some IdPs use an audience parameter rather than explicit scopes; the token carries the permissions granted to the M2M app.
Token Exchange	`scope=all-apis` Databricks-side scope. UC governance restricts at the data layer regardless.

IDP: Okta

Okta Configuration

Endpoints
Token	`https://{org}.okta.com/oauth2/{auth_server_id}/v1/token`
JWKS	`https://{org}.okta.com/oauth2/{auth_server_id}/v1/keys`
Issuer	`https://{org}.okta.com/oauth2/{auth_server_id}`
Subject (M2M)	`{client_id}` (no @clients suffix)

Claims Injection: Authorization Server Claims

Admin → Security → API → Authorization Server → Claims tab

// Custom claim: "groups"
Name:       groups
Value:      user.getGroups().filter(
              g, g.name.startsWith("fed_"))
Include in: Access Token
Scope:      Any scope

Okta-Specific Gotchas

Authorization Server: Use custom (not "default" org server) for custom claims
Issuer URL: Includes /oauth2/{server_id} - NOT just the org URL
No trailing slash on issuer (some IdPs include one, some do not)
M2M sub claim: Just {client_id} (no @clients)
Group filter: Use Okta Expression Language in claim value

Required Okta Apps

Service	OAuth 2.0 Service app (client_credentials)
SPA	OIDC SPA app (PKCE, authorization code)

Required Okta Config

Authorization Server	Custom server with audience = workspace URL
Scopes	Add custom scope if needed
Claims	groups, role, email (custom claims on access token)
Access Policies	Rule for service app + SPA app
Groups	Assign users to Okta groups (mapped to Databricks roles)

Federation Policy Subject

"subject": "{SERVICE_APP_CLIENT_ID}"
// No @clients suffix for Okta

OAuth Scopes at Each Stage

SPA Login	`openid profile email` + custom scopes Custom scopes gate which claims are included in token
Service App	`scope={custom_scope}` on Auth Server Okta requires at least one scope. Create a custom scope on the Authorization Server (e.g., `federation`). Access policy rules filter by scope.
Token Exchange	`scope=all-apis` Same Databricks-side scope regardless of IDP

IDP: Microsoft Entra ID

Entra ID Configuration

Endpoints
Token	`https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token`
JWKS	`https://login.microsoftonline.com/{tenant_id}/discovery/v2.0/keys`
Issuer	`https://login.microsoftonline.com/{tenant_id}/v2.0`
Subject (M2M)	`{app_id}` (GUID of the App Registration)

Claims Injection: Claims Mapping Policy

PowerShell / MS Graph API → Claims Mapping Policy

// Claims Mapping Policy (JSON)
{
  "ClaimsMappingPolicy": {
    "Version": 1,
    "IncludeBasicClaimSet": "true",
    "ClaimsSchema": [{
      "Source": "user",
      "ID": "groups",
      "JwtClaimType": "groups"
    }]
  }
}

Entra-Specific Gotchas

Issuer format: .../v2.0 (v2 endpoints) vs .../ (v1), must match exactly
App vs Service Principal: App Registration creates both, use App ID for subject
Groups claim: Returns GUIDs by default, not names. Use groupMembershipClaims: "SecurityGroup" in manifest
Token audience: Set via "Expose an API" → Application ID URI = workspace URL
Client secret: Expires (max 2 years). Use certificates for production

Required Entra Config

App Reg (M2M)	Client credentials with secret/cert
App Reg (SPA)	PKCE redirect, SPA platform
API Scope	Expose an API → Application ID URI = workspace URL
Claims Policy	Assign to Service Principal
Security Groups	Map to Databricks roles

Federation Policy Subject

"subject": "{APP_REGISTRATION_APP_ID}"
// The Application (client) ID GUID
// NOT the Object ID

OAuth Scopes at Each Stage

SPA Login	`openid profile email User.Read` MS Graph delegated permissions for user info
M2M (App)	`scope={app_id_uri}/.default` Entra uses `.default` scope pattern. Requests all permissions pre-consented for the app. The `app_id_uri` = workspace URL.
Token Exchange	`scope=all-apis` Same Databricks-side scope regardless of IDP

Role Resolution Strategy

Option A: Map Entra Security Group GUIDs to roles in app config
Option B: Use App Roles (defined in App Registration manifest), appear as roles claim
Option C: Use Claims Mapping to emit group names instead of GUIDs

Comparison

OIDC IdP Configuration, Side by Side

Configuration	OIDC IdP (generic)	Okta	Entra ID
Issuer URL	`https://{idp-domain}/` trailing slash	`https://{org}.okta.com/oauth2/{server_id}` no trailing slash	`https://login.microsoftonline.com/{tenant}/v2.0` v2.0 suffix
JWKS URL	`.../.well-known/jwks.json`	`.../v1/keys`	`.../discovery/v2.0/keys`
M2M Sub Claim	`{client_id}@clients`	`{client_id}`	`{app_id}` (GUID)
Claims Injection	Token / claims hook	Authorization Server custom claims	Claims Mapping Policy + manifest
Namespace Required?	Sometimes Some IdPs need a non-vendor namespace	No Top-level claims OK	No Top-level claims OK
Token Dialect Fix?	Yes `access_token_authz`	No Custom claims included by default	No Claims policy handles it
Group Format	String names (from app_metadata)	String names (Okta groups)	GUIDs (Security Groups) by default
M2M App Type	Machine-to-Machine Application	OAuth 2.0 Service App	App Registration + Client Secret/Cert
SPA App Type	Single Page Application	OIDC SPA App	App Registration (SPA platform)
Automation	Management API v2	Okta Admin API	MS Graph API
SPA Scopes	`openid profile email`	`openid profile email` + custom	`openid profile email User.Read`
M2M Scopes	Implicit via `audience` No explicit scope param	`scope={custom_scope}` Must create on Auth Server	`scope={app_id_uri}/.default` .default = all pre-consented
Exchange Scope	`scope=all-apis` ← SAME for all IDPs (Databricks-side)
Free Tier	Varies by IdP and plan	100 MAU (Workforce), limited M2M	Part of Azure AD (included with M365)

Migration

Swapping IDP, Checklist

What You Change

1

IDP Apps

Create M2M + SPA apps in new IDP

2

Claims Injection

Configure groups/role/email claims in new IDP's mechanism

3

Federation Policies

Update issuer and subject on each SP's policy

4

External App Config

Update IDP domain, client_id, JWKS URI, token endpoint, claims namespace

5

API Server Config

Update IDP_DOMAIN, IDP_AUDIENCE, IDP_ISSUER, JWKS_URI env vars

6

Test Users

Create test users with groups/roles in new IDP

What You Keep

✓

Databricks Service Principals

Same SPs, same application_ids, same group membership

✓

UC Row Filters & Column Masks

Same is_member() functions, same tables

✓

UC Connection Grants

Same GRANT/REVOKE USE CONNECTION SQL

✓

Role → SP Mapping

Same ROLE_SP_MAP config (roles are app-defined, not IDP-defined)

✓

Token Exchange Endpoint

Same /oidc/v1/token with same grant_type

✓

Audit Table + MLflow Traces

Same schema, same tags, same correlation

Takeaways

The Identity Chain Is the Architecture

1. IDP Is Swappable

The IDP provides the JWT. Databricks validates it via federation policy. Swap the JWT issuer, update the policy, everything downstream stays the same. The governance layer (UC) is completely IDP-agnostic.

2. Two Token Exchanges, One Identity

External app exchanges IDP JWT → DB token (for proxy auth). MCP server independently exchanges the same IDP JWT → DB token (for SQL execution). Both resolve to the same role-SP. The external user's identity is preserved in JWT claims, audit records, and MLflow traces.

3. Governance Is Native

UC row filters and column masks fire at the SQL engine level. They can't be bypassed by changing the application, the IDP, or the query. is_member() checks the SP's group membership, not the caller's headers.

4. Silent vs. Loud Failures

Auth failures (401/403) are loud and visible. Governance enforcement (0 rows, NULL columns) is silent and correct. Know which one you're looking at before debugging.

5. The Subject Claim Is the Linchpin

90% of federation failures are a mismatch between the JWT's sub claim and the federation policy's subject field. Every IdP formats this differently: {client_id} or {client_id}@clients for OAuth apps, {app_id} (GUID) for Entra. Decode the JWT first, then write the policy.

6. Test the Chain, Not the Parts

A working IDP token + a working federation policy can still fail if the subject claim doesn't match. Test the full chain: get IDP JWT → exchange → call /api/2.0/preview/scim/v2/Me → verify SP identity.

Role Reference

Roles and Tools, Access Matrix

Role label	Role	Group	Regions	Margins	Identity	Regions	Sales	Genie Agents	KB	Supervisor	GitHub	Ext API	Audit
West Territory Rep	west_sales	west_sales	WEST only	NULL	Y	Y	Y	Y	Y	-	-	-	-
East Territory Rep	east_sales	east_sales	EAST only	NULL	Y	Y	Y	Y	Y	-	-	-	-
Manager	managers	managers	ALL	NULL	Y	Y	Y	Y	Y	-	-	-	-
Executive	executive	executives	ALL	Real	Y	Y	Y	Y	Y	Y	Y	Y	Y
External Auditor	finance	finance	ALL	Real	Y	Y	Y	Y	Y	-	-	-	Y
Platform Admin	admin	admin	ALL	Real	Y	Y	Y	Y	Y	Y	Y	Y	Y

Governance Layers Demonstrated

Row filter: a west rep sees only WEST rows, an east rep only EAST, an executive sees all regions
Column mask: margin_pct → NULL for sales/managers, real values for finance/exec/admin
Tool access: Supervisor + GitHub + External API restricted to executive + admin
UC Connection: GRANT/REVOKE USE CONNECTION, instant, no deploy
Audit isolation: role-SPs have zero write access to audit table

Example Flows

Data Isolation: A west rep asks Genie Agents for top deals and gets WEST only; an east rep gets EAST only.
Column Masking: a rep sees masked margins on the sales data; the auditor sees real values.
AI Governance: A west rep sees WEST pipeline; an executive sees all regions.
UC Connections: An executive can reach the external connection; a rep gets ACCESS_DENIED.
Audit Trail: The admin reads the audit log and sees every action with the external user identity.

Impact

External users get governed access.
Without ever touching Databricks.

0

Databricks users provisioned

N:1

Users to SPs ratio per role

1

SQL statement to toggle access

For the business: Partners, auditors, and consultants get self-service access to exactly the data they need, no IT tickets, no shared credentials, no waiting for provisioning.

For security: Every query runs through UC row filters and column masks. Every action is audited with the real external identity. Access is revocable with one SQL statement.

For engineering: Swap the IDP by updating the federation policy, no code changes. Add a new role by creating one SP and one workspace group. The architecture scales horizontally.

For compliance: External identity preserved end-to-end. Correlatable audit trail across custom tables, MLflow traces, and system.access.audit. No identity gaps.

Federation ExchangeUnder the Hood

Two Paths for External Users, and Why SPs Win

Option A: On-Behalf-Of (OBO)

Option B: Role-Based Service Principals

Three Actors Make Federation Work

Actor 1: The API Declaration

Actor 2: The M2M App

Actor 3: The SPA App

External User → IDP Token → Databricks Token

What Changes Per IDP

What Stays the Same

From Login to Governed Data Access

The token at each stage

What's in Each Token at Each Stage

Stage 1: IDP JWT

Stage 2: Databricks Token

Stage 3: Governance Context

7 Enforcement Points in the Chain

Key Insight: Silent vs. Loud Enforcement

What Goes Wrong and What It Means

Decision Tree: Where Is It Failing?

What to Configure, In Order

1Create an API / Resource Server

2Create an M2M / Service App

3Create a SPA App (if browser login needed)

4Configure Users & Groups

5Add Claims Injection Logic

6Create Role-Based Service Principals

7Add SPs to Workspace Groups

8Create Federation Policies

9Configure UC Governance

10Deploy API Server + External App

How IDP Config Maps to Databricks Config

Role → SP Mapping + Grants Checklist

Scopes (Token Exchange)

Federation Policy, Two Levels

SP-Level Policy Recommended for Federation Exchange

Account-Level Policy

Policy Fields (Same for Both Levels)

Common Gotcha: Trailing Slash

Subject Claim by IDP

Token Exchange Parameters (IDP-Agnostic)

Configuring Any OIDC IdP

Claims Injection (at token issuance)

Common Gotchas

Apps You Register in the IdP

What to Configure

Federation Policy Subject

OAuth Scopes at Each Stage

Okta Configuration

Claims Injection: Authorization Server Claims

Okta-Specific Gotchas

Required Okta Apps

Required Okta Config

Federation Policy Subject

OAuth Scopes at Each Stage

Entra ID Configuration

Claims Injection: Claims Mapping Policy

Entra-Specific Gotchas

Required Entra Config

Federation Policy Subject

OAuth Scopes at Each Stage

Role Resolution Strategy

OIDC IdP Configuration, Side by Side

Swapping IDP, Checklist

What You Change

What You Keep

The Identity Chain Is the Architecture

1. IDP Is Swappable

2. Two Token Exchanges, One Identity

3. Governance Is Native

4. Silent vs. Loud Failures

5. The Subject Claim Is the Linchpin

6. Test the Chain, Not the Parts

Roles and Tools, Access Matrix

Governance Layers Demonstrated

Example Flows

External users get governed access.Without ever touching Databricks.

Federation Exchange
Under the Hood

External users get governed access.
Without ever touching Databricks.