ZTE-Lightweight — Project Summary

Lightweight Zero Trust Environment (MVP) Demonstrating AI-driven development: Gemini as Architect, Claude as Engineer.

For what the product does — every capability, how they depend on each other, and how mature each one is — see the Feature Catalogue. For how it is built (architecture, component specs, data model, API reference, risk register, roadmap), see docs/SPECS.md. This README stays a quick-start summary.

A minimal, runnable Zero Trust microservices stack built entirely in Java, showing how every trust decision is made explicit in code — no service mesh, no "implicit trust once inside the network." Every request must prove:

1. Who is the user? — Keycloak JWT (RS256, validated at the gateway)
2. Is the user allowed? — YAML-defined access policy (zte-policies.yaml, ADR-011/ADR-012)
3. Who is the internal caller? — mTLS client certificate (signed by ZTE CA)
4. On whose behalf? — Signed OBO token (X-ZTE-User-Context, HMAC-SHA256, 30s TTL)

┌─────────┐  Keycloak JWT (RS256)   ┌──────────────────────────────────────────┐
│  User   │ ──────────────────────► │              ZTE Gateway                 │
└─────────┘                         │  ① Spring Security validates JWT sig     │
                                    │  ② ZteAuthorizationFilter: YAML policy   │
                                    │     role matches zte-policies.yaml?      │
                                    │     else 403 (ADR-012: YAML-only)        │
                                    │  ③ UserContextPropagationFilter:         │
                                    │     creates X-ZTE-User-Context (OBO JWT) │
                                    └──────────────┬───────────────────────────┘
                                                   │ HTTPS + mTLS
                                                   │ client.p12 (ZTE CA)
                                                   │ X-ZTE-User-Context: <OBO>
                                                   ▼
                                    ┌──────────────────────────────────────────┐
                                    │              Service A                   │
                                    │  ④ TLS handshake: client cert verified   │
                                    │     against ZTE CA (or reject)           │
                                    │  ⑤ Forwards X-ZTE-User-Context unchanged │
                                    │     (delegation — not re-issuance)       │
                                    └──────────────┬───────────────────────────┘
                                                   │ HTTPS + mTLS
                                                   │ client.p12 (ZTE CA)
                                                   │ X-ZTE-User-Context: <same OBO>
                                                   ▼
                                    ┌──────────────────────────────────────────┐
                                    │              Service B                   │
                                    │  ⑥ TLS handshake: client cert verified   │
                                    │  ⑦ UserContextController validates HMAC  │
                                    │     signature + expiry of OBO token      │
                                    │  ⑧ Returns: sub, roles, trustBasis       │
                                    └──────────────────────────────────────────┘

Trust at each hop:

zte-lightweight/
├── auth-library/          Shared security utilities (no main class)
│   ├── SecurityConfig     Default WebFlux security (JWT, deny-by-default)
│   ├── ZteAuditLogger     Structured [ZTE-AUDIT] log events (static utility)
│   ├── ReloadableSslContextFactory   Netty client SslContext with AtomicRef hot-swap
│   └── UserContextTokenService       HMAC-SHA256 OBO token create/validate
│
├── gateway-service/       ZTE entry point — port 8080 (HTTPS, client-auth: want — ADR-018)
│   ├── ZteAuthorizationFilter        users2service: YAML-only, deny-by-default (HIGHEST_PRECEDENCE+100, ADR-012)
│   ├── ServiceToServiceAuthorizationFilter  service2service: YAML-only, default-deny (HIGHEST_PRECEDENCE+150)
│   ├── UserContextPropagationFilter  OBO token injection (HIGHEST_PRECEDENCE+200)
│   ├── MtlsHttpClientConfig         Netty HttpClient with client.p12
│   ├── policy/def/                   YAML policy engine (see ADR-011/ADR-012): PolicyDefinitionStore,
│   │                                 PolicyMatcher, PolicyValidator, YamlPolicyFileLoader, RealmRoles
│   ├── GatewayRouteConfig            Routes: /api/v1/service-a/**, /api/v1/service-b/**
│   ├── InternalPolicyController      GET /api/v1/internal/policies (agent data provider, YAML-backed)
│   ├── PolicyReloadController        POST /api/v1/internal/policies/reload (no-downtime reload)
│   ├── admin/                        Admin Console API (see ADR-012, ADR-013)
│   │   ├── AdminPolicyController     GET /api/v1/admin/policies, POST .../reload (ADMIN JWT required)
│   │   ├── AdminAuditLogController   GET /api/v1/admin/audit-logs (latest 100, ADMIN JWT required)
│   │   ├── AdminAuthorizationFilter  Plain WebFilter enforcing users2service on /api/v1/admin/**
│   │   │                             (ZteAuthorizationFilter's GlobalFilter type doesn't run for
│   │   │                             non-Gateway-routed local controllers — see its Javadoc)
│   │   └── AdminUiConfig             permitAll + static resource serving for /admin/**
│   ├── audit/                        Async request audit trail (see ADR-013)
│   │   ├── RequestLog                R2DBC record (request_logs table)
│   │   ├── RequestLogRepository      ReactiveCrudRepository, findTop100ByOrderByTimestampDesc
│   │   └── RequestLogAuditService    Non-blocking Sinks.Many sink → boundedElastic writer,
│   │                                 SLF4J fallback on DB failure (mirrors LoggingMcpAuditService)
│   ├── RequestAuditFilter            Plain WebFilter (not GlobalFilter, ADR-013): X-Request-Id
│   │                                 resolve/forward, X-User-Id trust boundary, async audit write
│   │                                 via doFinally (LOWEST_PRECEDENCE-100)
│   └── mcp/                          MCP proxy — GET /sse, POST /message (see ADR-009)
│       ├── McpProxyHandler           Policy check, deny-via-SSE, backend forward
│       ├── McpSessionManager         sessionId → SSE sink (cross-request injection)
│       ├── McpBackendClient          WebClient forward to mcp-backend.uri
│       ├── policy/YamlMcpPolicyEngine    YAML-backed per-agent tool grants (see ADR-011)
│       ├── audit/LoggingMcpAuditService  Non-blocking audit sink (TSDB-ready stub)
│       └── mask/NoOpDataMaskingFilter    PII masking stub
│
├── service-a/             Protected downstream — port 8081 (HTTPS/mTLS), 9081 (mgmt)
│   ├── HelloController    Calls service-b, returns combined response
│   └── ServiceBClientConfig          mTLS WebClient for outbound calls
│
├── service-b/             Deep downstream — port 8082 (HTTPS/mTLS), 9082 (mgmt)
│   └── UserContextController         Validates OBO token, returns user context
│
├── zt-agents/             AI security copilot — port 8083 (Kotlin Spring Boot WebFlux)
│   ├── PolicyAuditorService          Fetches policies → prompts Claude → returns Markdown
│   ├── AnthropicClient               WebClient wrapper for Anthropic Messages API
│   └── GatewayClient                 Fetches /api/v1/internal/policies from gateway
│
├── approver/              Approval Center API + hosting (see ADR-026)
│   ├── ApproverApprovalsController  /api/v1/approver/approvals[/{id}/approve|reject]
│   │                                (USER or ADMIN role; same PendingApprovalService
│   │                                as the admin tab — one decision path, one audit trail)
│   ├── ApproverUiConfig             permitAll + static serving for /approver/**
│   └── UiConfigController           GET /ui-config.js — runtime OIDC authority for both SPAs
│
├── zt-approver-ui/        Approval Center SPA (Vite/TS/MUI) — see ADR-026
│
├── zt-admin-ui/           React Admin Console (Vite/TS/MUI) — see ADR-012
│   ├── src/App.tsx        Login gate (react-oidc-context) + dashboard shell
│   ├── src/PolicyDashboard.tsx  Fetches/renders policies, "Reload Policies" button
│   └── (plain npm project, not a Gradle subproject — built by gateway-service's
│        build.gradle.kts via the node-gradle plugin, packaged into its jar)
│
├── certs/
│   └── generate-certs.sh  Generates ZTE-CA, client.p12/.pem, service-a.p12, service-b.p12, gateway.p12
│                          (re-run safe: reuses an existing CA, reissues leaf certs;
│                           ZTE_REGENERATE_CA=1 forces a new CA, GATEWAY_EXTRA_SANS adds SANs)
│
├── scripts/
│   └── install-hooks.sh   Installs .githooks/pre-commit into local .git/hooks/
│
├── deploy/azure/          Azure Container Apps deployment (see ADR-027, ADR-028)
│   ├── deploy.sh          Two-phase provisioning (env + VNET, apps, certs share, phase-2 origin)
│   ├── push-images.sh     Builds and pushes every image to GHCR (private)
│   ├── bind-custom-domain.sh  Prints the required DNS records; binds domain + managed cert
│   ├── power.sh           stop / start / status — park the whole stack overnight
│   │                      (stop backs the database up first, and aborts if that fails)
│   ├── db-backup-job.sh   The pg_dump-to-share job the stop depends on (ADR-033)
│   ├── attach-volume.sh   Attach/detach an Azure Files share on an app (raw ARM PUT)
│   ├── make-cloud-realm.py    Cloud realm import (multi-origin redirect URIs)
│   └── Dockerfile.keycloak    Keycloak image with that realm baked in
│
├── docker-compose.cloud.yml   Local mirror of the cloud topology (single external origin)
│
├── .claude/commands/
│   └── pre-commit-docs.md /pre-commit-docs slash command — docs guardian pre-commit check
│
└── docs/adr/              Architectural Decision Records

gateway-service fronts Model Context Protocol (MCP) traffic over HTTP+SSE, intercepting every tool call for a policy decision before it reaches the backend MCP server. See ADR-009 for why this is a plain WebFlux router rather than a Gateway route.

Endpoints: GET /sse (open a session), POST /message?sessionId=<id> (send a JSON-RPC tools/call). Both require a JWT, same as every other route.

Flow:

1. Client opens GET /sse. The gateway generates a sessionId and immediately pushes an endpoint event: data: /message?sessionId=<id> (standard MCP HTTP+SSE handshake).
2. Client sends a JSON-RPC call to POST /message?sessionId=<id>. clientId comes from the JWT azp claim (falling back to sub); the tool name and arguments come from params.name / params.arguments.
3. As of Stage 10 (ADR-011), the gateway enforces real per-agent policy: YamlMcpPolicyEngine checks the agentMcpToolCalls rules in zte-policies.yaml (deny always wins, no match → default-deny). A denial is injected as a JSON-RPC envelope with result.isError=true, naming the matched rule — the backend is never called. An allow forwards the call to McpForwardService (backend + masking), and injects the result into the SSE stream. The POST still always returns 202 Accepted regardless — the transport contract from Stage 8 is unchanged, only what gets computed. This supersedes Stage 9's (ADR-010) deliberate dead-end stub — see ADR-011 for the full reasoning.
4. As of Stage 19 (ADR-019), a third outcome — Hold — routes a call to a human instead of allowing or denying it outright: an otherwise-allowed call additionally matching an agentMcpToolHolds rule is parked in pending_approvals (durable — a held item may be reviewed well after this SSE session closes) and a status: "held" envelope is injected instead. A human then calls POST /api/v1/admin/approvals/{id}/approve (the call is put to the policy engine again — ADR-043 — and forwarded through the same McpForwardService path only if policy still permits it; 409 if the agent was suspended or the grant narrowed while it waited) or /reject (an honest denial) — see the Admin Console's Approvals tab, below. 4a. As of Stage 20 (ADR-020), an agent with an ACAP profile gets a further, argument-aware tightening pass on top of steps 3–4's decision: a call step 3 would otherwise ALLOW (or hold) can still be denied based on its arguments — wrong territory, a disallowed field, a write attempt from a read-only agent, or a bulk/export-shaped tool — checks the coarse, tool-name-only rules above have no way to express. This is why read_contacts(territory=EMEA) and read_contacts(territory=NA) — the same tool, different arguments — can (and for the demo agent, do) produce different decisions. Additive and opt-in: an agent with no ACAP profile file is unaffected; a deny from step 3 is never reconsidered. See YAML Policy Engine's "ACAP scope profiles" note below and docs/policy-schema.md.
5. Every call is recorded asynchronously via LoggingMcpAuditService (non-blocking sink, logs today, TSDB-ready) with status "ALLOWED"/"DENIED"/"HELD"/"APPROVED"/"REJECTED", and synchronously via the unified ZteAuditLogger.policyDecision(...) structured log line shared with the gateway's REST policy filters.

Tested by: McpProxyIT (gateway-service/src/it) — full GET /sse → POST /message → SSE-injection round trip against a real running gateway (Testcontainers + WireMock), using Agent A/Agent B's (and, as of ADR-019, the demo crm-account-health-emea-01 agent's) actual client-credentials tokens; covers an allowed call forwarded to the backend, a tool with no grant, a destructive-shaped tool caught by the deny-list, a held call followed by an admin approval, and (ADR-020) the same read_contacts tool call allowed/denied by territory, (ADR-042) answered with a disallowed field stripped from the forwarded call and named in the result — and a wrong territory still refused outright even when fields are also named — plus a coarsely-granted-but-ACAP-denied update_deal, plus 401-without-token and unknown-sessionId 400. Also McpProxySecurityWebFluxTest (src/test) — a fast @WebFluxTest slice covering the same allow/deny/hold paths without Docker. Run with ./gradlew test integrationTest.

Admin Console Approvals tab: every call currently parked by an agentMcpToolHolds rule — agent, tool, arguments, and why it was held — with Approve/Reject buttons. Approve reconstructs and forwards the exact original call; Reject sends back an honest denial. Either way the decision is durable and audited even if the agent's SSE session has since closed; only pushing the result into that specific live connection is best-effort (see ADR-019's Self-Criticism). The demo crm-account-health-emea-01 agent (zte-policies.yaml) holds on send_email/draft_followup — try it: open an SSE session as that agent, call one of those tools, then approve or reject it from this tab.

A second, separate UI surface over the same hold queue, for business approvers who shouldn't get the full Admin Console: https://localhost:8080/approver/index.html — its own SPA (zt-approver-ui/), its own Keycloak client (zte-approver-ui, PKCE), its own login screen, one card per held call with Approve/Decline.

Open to any authenticated interactive user (USER or ADMIN realm role — u2s-approver-api-user/-admin rules on the new /api/v1/approver/** API), deliberately not source: "*": an agent's client-credentials JWT carries no realm role, so an agent can never approve its own held call. Decisions go through the exact same PendingApprovalService as the Admin Console tab — one decision path, one audit trail (decided_by = the approver's username).

Routing and expiry (ADR-034). A hold rule may name who owns the decision with routeTo: "role:APPROVER" (or user:<name>); both surfaces then show the item to everyone but only let a matching viewer decide it — anyone else sees the buttons disabled with the reason, and a direct POST .../approve gets 403 with the same sentence. Rules without routeTo stay open to any interactive user. In the dev realm the APPROVER role is held by zte-admin and zte-dpo and deliberately not by zte-test-user, and the demo's send_email hold is the one routed rule — log in as each to see the difference. Every held call also carries a deadline (ZTE_APPROVALS_TTL_MINUTES, default 24h) with a countdown on the card; past it the item turns EXPIRED (swept every ZTE_APPROVALS_SWEEP_INTERVAL_MS, default 60s), stays visible greyed out, and is audited as a DENY/408 — a late approval is refused with 409.

Notifications (ADR-035). Who may decide and who gets told are different questions: an unrouted call stays decidable by anyone, but it is addressed to zte.approvals.default-notify (role:APPROVER by default), resolved to real usernames from the identity cache. Both surfaces show an "N for you" badge and offer an opt-in desktop notification (asked for behind a button, never on load). Set ZTE_APPROVALS_WEBHOOK_URL to a Slack/Teams incoming webhook — or any JSON endpoint — and each held call also posts there; ZTE_APPROVALS_LINK_BASE adds a link back. The payload names the agent, tool, audience and deadline and deliberately omits the call's arguments, which are the sensitive part and stay behind a login. Every attempt is recorded in approval_notifications as SENT/FAILED/SKIPPED and shown per item, so "was anyone told?" is answerable — including when the answer is "no channel is configured".

An undecided item is nudged again as its deadline nears (ADR-036): ZTE_APPROVALS_REMINDER_FRACTIONS (default 0.5) lists fractions of each item's own lifetime — a fixed "an hour before" would be meaningless at a one-minute TTL — and both consoles show whether the last contact was the original announcement or a reminder. Both gateway apps run this sweep; each stage is claimed in the database before the message goes out, so they produce one reminder rather than two.

https://demo.zteasy.tech/chat/index.html (locally https://localhost:8080/chat/index.html) — a third SPA with its own Keycloak client zte-chat-ui and its own realm role CHAT_USER, held in the dev realm by zte-admin, zte-test-user and zte-ceo.

Left panel: a chat assistant with the CRM tools the demo's agent has. Right panel: every decision the gateway made about you, polled live, scoped to you in SQL. The header carries your own token spend for the day.

What makes it more than a chat window:

• The decision is about the person. An MCP call from here is evaluated against user:<name> and role:<r> (ADR-039), and tightened by the ACAP profile keyed to the role — role:CHAT_USER: EMEA only, a named field list, no writes. The same shape the CRM agent gets; nothing is softer because a human is typing.
• The model is reached through the gateway, which holds the vendor key, meters the tokens out of the vendor's own response and bills them to the person. The chat backend holds no model credential at all.
• The tool list is deliberately unfiltered, so the assistant can try something it may not do — and be refused in front of the person who asked.

Three things to try, in order:

# 1. "Show me our EMEA contacts with their phone numbers."
#    The model picks the fields it wants. Those on the profile's list are
#    answered; the rest are stripped from the call before it is forwarded and
#    named in the result, so the assistant reports what was withheld (ADR-042).
# 2. "List EMEA contacts showing only company and lifecyclestage."
#    Allowed — the data comes back. The field names are the CRM's own property
#    names; invented ones ("name", "lifecycle_stage") are withheld like any
#    other field outside the profile. Full walkthrough: docs/demo-chat-console.md
# 3. "Email rep@nordwind.example about the renewal."
#    Held, routed to role:APPROVER, waiting in the Approval Center. Sign in there
#    as zte-dpo to decide it.

Run it locally with ./gradlew :zt-chat:bootRun alongside the gateway; the gateway needs ZTE_LLM_API_KEY set to an Anthropic key (it, not the chat backend, is what talks to the vendor).

# Try it: log in as the USER-role account (scripts/set-keycloak-password.sh
# sets the local dev passwords; cloud credentials live only in the gitignored
# deploy/azure/out/cloud-credentials.env) and decide a held send_email.
open https://localhost:8080/approver/index.html

Both SPAs now load their OIDC authority at runtime from GET /ui-config.js (zte.ui.oidc-authority, env ZTE_UI_OIDC_AUTHORITY) — the same built bundle works against local Keycloak (localhost:8180, the default) or a reverse-proxied /auth deployment (ADR-027).

Admin Console Governance tab — the historical/reporting half (Approvals above is the live, actionable half): read-only over the same request_logs audit trail every prior stage already writes to, no new table.

• Agent Activity — per-agent ALLOW/HOLD/DENY counts over a selectable window (last hour / 24 hours / 7 days), plus each agent's most recent activity timestamp.
• Out-of-Policy Attempts — the latest 50 denied MCP-agent calls, newest first (agent, tool, reason) — deliberately narrower than the Audit Trail tab (MCP-agent traffic only, not REST); a human's post-hold rejection shows up here too, since it's audited as a DENY (ADR-019).
• Export Report — downloads both views as one JSON snapshot for the selected window.
• ACAP Profiles (Stage 22, ADR-022) — one card per loaded ACAP profile: owner, deployment date, a red "OVERDUE" badge once re-authorization is due, EU AI Act risk tier, and live used/limit chips for any usage thresholds — see below.

curl -sk "https://localhost:8080/api/v1/admin/governance/agent-activity?hours=24" -H "Authorization: Bearer $ADMIN_TOKEN"
curl -sk https://localhost:8080/api/v1/admin/governance/out-of-policy -H "Authorization: Bearer $ADMIN_TOKEN"

An agent authenticates to this gateway via OAuth2 Client Credentials — a service authenticating as itself, not on behalf of a user. See ADR-010.

As of ADR-041 the demo itself no longer ships synthetic agents — a person in the Chat Console exercises the same tools under the same engine. One Client-Credentials client remains in the realm, crm-account-health-emea-01, because the integration suite has no browser and must authenticate as something; the machine-identity flow below is unchanged and is what a real agent would use.

TOKEN=$(curl -s -X POST http://localhost:8180/realms/zte-realm/protocol/openid-connect/token \
  -d "grant_type=client_credentials&client_id=crm-account-health-emea-01&client_secret=$ZTE_SECRET_CRM_ACCOUNT_HEALTH_EMEA_01" \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")

# As of ADR-018, /sse is a protected path: a JWT alone is no longer enough, a client
# certificate signed by the ZTE-CA is also required (server.ssl.client-auth: want +
# MtlsEnforcementWebFilter) — -k accepts the dev self-signed CA on the gateway's own cert.
curl -sk --cert certs/client.p12:$ZTE_KEY_PASSWORD --cert-type P12 \
  -N -H "Authorization: Bearer $TOKEN" https://localhost:8080/sse
# → data: /message?sessionId=<id>  (open a second terminal to POST while this stays open)

Or run the full round trip via hubspot-mcp/run_agents.sh (see that repo's README) — it does the token fetch, GET /sse handshake, and POST /message for both agents automatically.

A single YAML file (gateway-service/src/main/resources/zte-policies.yaml, path configurable via zte.policy.file) defines allow/deny rules for four relationship categories, loaded and validated at startup and hot-swappable at runtime, and is the sole source of truth for all four (no DB fallback anywhere, as of ADR-012 — see ADR-011, ADR-012, and docs/policy-schema.md for the full schema and precedence rules.

Deny always overrides allow, regardless of priority or declaration order. Full schema reference (all fields, precedence, validation rules): docs/policy-schema.md. Full worked example: docs/examples/zte-policies-example.yaml.

schemaVersion: 1        # required, must be exactly 1
users2service: [ ... ]  # list of rules, may be empty/omitted
service2service: [ ... ]
agentMcpToolCalls: [ ... ]
agentMcpToolHolds: [ ... ]   # ADR-019 — route a call to a human instead of forwarding it

Every rule, in every category, shares one shape:

1. Edit gateway-service/src/main/resources/zte-policies.yaml (or whatever file zte.policy.file points at) and add a rule to the relevant category, e.g. to let USERs POST to service-a:

   users2service:
     - id: u2s-user-write-service-a
       effect: ALLOW
       source: USER
       target: service-a
       pathPattern: "/api/v1/service-a/**"
       methods: "POST"

2. Restart the gateway, or reload without downtime — either the unauthenticated ops/tooling endpoint, or the ADMIN-JWT-gated one the Admin Console UI uses (see Admin Console below):

   curl -sk -X POST https://localhost:8080/api/v1/internal/policies/reload | python3 -m json.tool

   An invalid file (bad schema, duplicate id, etc.) fails validation and the previously active policy set stays in effect — the reload response reports the errors.

3. Check the gateway log for the [ZTE-AUDIT] POLICY_ALLOW/POLICY_DENY line naming the matched rule id to confirm it's taking effect.

A separate, additive, opt-in layer on top of the four categories above — one YAML file per agent under gateway-service/src/main/resources/acap-profiles/ (path via zte.acap.profiles-location), consulted only for agents that have one. Where zte-policies.yaml only ever sees a tool name, an ACAP profile lets the gate look at the call's arguments too:

agentId: crm-account-health-emea-01
territory: EMEA
scope:
  read:
    - resource: contacts
      fields: [company, lifecyclestage, createdate, hs_object_id]
  writeAllowed: false

What each tool is comes from acap-tool-shapes.yaml (zte.acap.tool-shapes-location, ADR-045), not from its name: a read is checked against territory and fields; a write is denied unless writeAllowed, and is bound to the territory too; an export is always denied; an action (an email, a draft) has no scope and is left to the rules and holds. A tool the file does not list is refused to any caller that has a profile, and the gateway warns at startup ([ZTE-ACAP-SHAPES]) about every rule granting one — so adding a tool to the backend means adding it there too. It can only tighten a coarse ALLOW/HOLD, never loosen one — a caller with no profile file is governed by the rules alone (zte.acap.unprofiled-callers: rules-only, the default; deny refuses them). A wrong territory, a write or an export is refused outright; a read that names a field outside fields proceeds without that field — it is stripped from the request before forwarding and named in the result, and the audit row is an ALLOWED with the withholding as its reason (ADR-042). onDisallowedField: deny on the profile restores the outright refusal, hold routes the call to a human. Full schema and the ACAP-to-ZTeasy field mapping: docs/policy-schema.md. List what's loaded (each entry includes its current threshold usage, see below) / trigger a reload:

curl -sk https://localhost:8080/api/v1/admin/acap-profiles -H "Authorization: Bearer $ADMIN_TOKEN"
curl -sk -X POST https://localhost:8080/api/v1/admin/acap-profiles/reload -H "Authorization: Bearer $ADMIN_TOKEN"

Two more, purely additive, optional blocks on the same per-agent profile file:

agent:
  name: Account-Health Assistant
  client: Nordwind Components
  owner:
    name: Sales Operations Lead
    email: sales-ops@nordwind.example
  deploymentDate: "2026-08-01"
  reauthDue: "2026-02-01"
risk:
  euAiActClass: limited
  internalTier: 2
thresholds:
  - metric: followup_drafts_per_day
    toolName: draft_followup
    limit: 30
    onExceed: hold

agent/risk are display-only — shown in the Admin Console's Governance tab, new "ACAP Profiles" section (one card per agent: owner, deployment date, a red "OVERDUE" badge once reauthDue is in the past, risk tier) — no enforcement is tied to either, matching the demo's own framing ("re-authorization is a human process, not a technical gate").

thresholds is enforced, one step further than scope tightening: once a call gets past territory/field/write/bulk checks without being denied, AcapThresholdTracker (in-memory, resets daily) increments the matching metric's counter, and — only if the count now exceeds limit and the call was a plain ALLOW — escalates it to HOLD, same as an agentMcpToolHolds rule would. toolName (not part of the source ACAP schema) says explicitly which tool the metric counts, since a name like followup_drafts_per_day doesn't mechanically imply draft_followup by any convention worth trusting. Each profile card in the Governance tab shows live used/limit chips for its thresholds.

A React/Vite/TypeScript SPA (zt-admin-ui/), built by gateway-service's own Gradle build and served statically at https://localhost:8080/admin/index.html — no separate process to run. Tabs: Policies (the full active YAML policy set, all four categories as of ADR-019, and a "Reload Policies" button), Audit Trail, Identities, Registry, Approvals (ADR-019 — the 🟡 hold queue, see MCP Proxy above), Governance (ADR-021 — see below), and Documentation (ADR-025 — the gateway's own OpenAPI reference, see below).

No client certificate needed here: /admin/** and /api/v1/admin/** are both in MtlsEnforcementWebFilter's excluded-path list (ADR-018) — the gateway is HTTPS-only as of that ADR, but this is plain server-authenticated TLS (your browser will warn about the dev self-signed ZTE-CA cert; accept it to proceed, same as any local HTTPS dev setup).

Login flow: the SPA itself is served unauthenticated (/admin/**, permitAll) — it redirects to Keycloak (react-oidc-context, client zte-admin-ui, PKCE) and only then calls the gateway's JSON API with a bearer token. The API (/api/v1/admin/**) requires a valid JWT and the ADMIN realm role (enforced by the u2s-admin-console-api YAML rule in zte-policies.yaml) — a USER-role login gets 403 from the API even though the page itself loads.

# 1. Build + start the gateway (npm install/build runs automatically as part of the Gradle build)
./gradlew :gateway-service:bootRun

# 2. Open the console
open https://localhost:8080/admin/index.html   # or just visit it in a browser

# 3. Log in as zte-admin (password from scripts/set-keycloak-password.sh —
#    local dev only; never reuse it for a deployment reachable from anywhere else)

Building without Node/npm installed: ./gradlew build -x :gateway-service:buildAdminUi skips the React build (mirrors the existing -x :zt-agents:compileKotlin escape hatch for the no-API-key case) — the gateway still builds and runs, just without /admin/** content.

Tabs: "Policies", "Audit Trail", "Identities", and "Registry" (all below) — each fetches independently on load. The Policies tab cross-references the Identities cache to flag users2service rules whose source doesn't resolve to any synced identity (see IdP Identity Sync).

The Documentation tab renders the gateway's own OpenAPI spec (GET /v3/api-docs, auto-generated by springdoc-openapi from every @RestController under com.zte.gateway — covers /api/v1/admin/** and /api/v1/internal/**; not the MCP proxy's /sse//message routes or the proxy routes to service-a/service-b) via the same swagger-ui-react component SchemaDrawer.tsx already uses for discovered downstream schemas. Unlike every other tab, it needs no access token — /v3/api-docs and /swagger-ui/** are permitAll (ApiDocsSecurityConfig), since the spec describes route shapes, not data; every actual /api/v1/admin/** call underneath stays exactly as protected as before. Also reachable directly, outside the Admin Console, at https://localhost:8080/swagger-ui.html (springdoc's own bundled standalone UI).

Every zero-trust-relevant gateway request — a proxied REST call or MCP tool call, allowed or denied by policy — is written asynchronously to a request_logs table in the same Postgres instance already used for Flyway migrations (via R2DBC), in addition to the existing synchronous [ZTE-AUDIT] SLF4J line. The write never blocks the response: RequestAuditFilter fires the DB write from a doFinally callback into a non-blocking sink (mirrors LoggingMcpAuditService's pattern); a DB outage degrades to an SLF4J warning instead of losing the event or slowing requests down.

Excluded from the audit trail (configurable, zte.audit.excluded-path-prefixes in application.yml — deliberately separate from zte-policies.yaml, not hardcoded): the Admin Console's own traffic (/admin/** static assets, /api/v1/admin/** API — otherwise viewing the audit trail would generate more audit trail), /api/v1/internal/**, and /actuator/** health checks. X-Request-Id/X-User-Id handling below still applies to every request regardless of this list — only the request_logs row and the sync requestLog SLF4J line are scoped down.

Distributed tracing: every request gets an X-Request-Id — the caller's own value if present, otherwise a new UUID minted by the gateway — which is then guaranteed to be forwarded to service-a/service-b, so a single request can be correlated across the whole chain. Known gap: a request with no Authorization header at all (true 401) is rejected by Spring Security before reaching this filter, so it isn't written to request_logs — every denial scenario tested in this codebase uses a present-but-wrong-role JWT (403), not a missing token, so this doesn't affect the common case. See ADR-013 for the full reasoning.

# Read the latest 100 rows via the admin API (ADMIN JWT required)
curl -sk -H "Authorization: Bearer $TOKEN" https://localhost:8080/api/v1/admin/audit-logs | python3 -m json.tool

# ...or query Postgres directly
docker exec zte-postgres psql -U zte_user -d zte_db \
  -c "SELECT trace_id, client_ip, path, status_code FROM request_logs ORDER BY timestamp DESC LIMIT 10;"

Also visible in the Admin Console's "Audit Trail" tab (Timestamp, Trace ID, Client IP, Agent ID, Initiator/OBO User, Method, Path, Target, Tool, Status, Decision, Trace). For an MCP tool call the Trace chip opens the decision trace (ADR-044): every check the policy pipeline can run, in order, marked MATCHED / PASSED / SKIPPED with the reason — which is how an allowed call explains itself, and how "this caller has no scope profile" or "a switched-off DENY rule would have matched" shows up on the row instead of in nobody's head. The chip turns amber when a check was skipped for a call that went on. Operator-only: the person a decision was about never sees it. Hovering a Tool name shows a tooltip with the MCP session id, deny reason (if any), and the actual tools/call arguments sent — "Agent ID" is MCP-only (agentId, null for REST rows); REST/interactive-user identity lives in the adjacent "Initiator/OBO User" column instead.

Unified with MCP audit (ADR-017, further enriched in a same-area follow-up). REST and MCP traffic share this one table — LoggingMcpAuditService (see MCP Proxy below) writes a row per MCP event (a session opening, or a tool call's outcome), populating agent_id/tool_name (always blank for REST rows, and vice versa) and carrying the same full HTTP context (trace_id/client_ip/user_agent) REST rows always had — no more separate, thinner MCP-only row. Every row also carries initiator_client (the calling service/agent's JWT azp, blank for a plain interactive user), original_user_obo (the caller's preferred_username, falling back to JWT sub — display-only; the OBO token the gateway actually mints for downstream propagation always uses the raw sub, read independently and unaffected by this display choice), target_service, http_method, and decision_effect (ALLOW/DENY/ERROR, derived from the final status code — see ADR-017).

Policy rules can now target a synced IdP identity instead of a bare role name or client id — zte-gateway's service account (realm-management's view-users/view-realm/ view-clients roles, granted in keycloak/realm-export.json) periodically pulls Keycloak's users, groups, roles, and OIDC clients (ADR-015 — machine identities like Agent A/B and zte-gateway itself) into a local idp_identities Postgres cache, via an IdpClient adapter interface (KeycloakIdpAdapter today; a future Azure Entra ID/AWS IAM adapter is a drop-in). No sensitive IdP data (passwords, secrets, tokens) is ever cached — only id/type/name. fetchClients() excludes Keycloak's own realm-builtin clients (account, broker, realm-management, admin-cli, security-admin-console, and their account-/broker--prefixed satellite clients) — only real business clients/agents land in the cache.

URN sources: role:<name>/user:<preferred_username>/group:<name> for users2service, client:<clientId> for service2service/agentMcpToolCalls — see the Format table above. A bare name (no prefix) still means the category's implied form exactly as before either ADR — fully backward compatible. PolicyMatcher itself is unchanged; the enriched sources list is built at the filter/engine call sites (ZteAuthorizationFilter/AdminAuthorizationFilter via IdentitySources.enrich; ServiceToServiceAuthorizationFilter/YamlMcpPolicyEngine via IdentitySources.enrichClient).

Relations (idp_identity_relations, synced the same cycle): a User's group memberships and realm-role assignments, and a Client's realm-role assignments (via its service-account user) are cached too, resolved to idp_identities' internal ids with zero extra Keycloak calls beyond the sync itself. GET /api/v1/admin/identities/{id}/relations reads only local Postgres — no live Keycloak dependency on that request path, ever.

# Manual sync (also runs automatically every 15 min, zte.idp.sync-interval-ms)
curl -sk -X POST -H "Authorization: Bearer $TOKEN" https://localhost:8080/api/v1/admin/identities/sync

# Search the cache (used by the Admin Console's autocomplete / Identities tab)
curl -sk -H "Authorization: Bearer $TOKEN" "https://localhost:8080/api/v1/admin/identities/search?type=CLIENT" | python3 -m json.tool

# Roles/groups related to a given Actor identity (cached only, no Keycloak call)
curl -sk -H "Authorization: Bearer $TOKEN" "https://localhost:8080/api/v1/admin/identities/<id>/relations" | python3 -m json.tool

Orphaned rules: a rule in any category whose source doesn't resolve to any cached identity logs an SLF4J WARN "ORPHANED RULE: ..." line (checked at startup and on every policy reload) and is highlighted in the Admin Console's Policies tab — never rejected or deleted. A transient false-positive is possible on a cold start, before the first sync has run; see ADR-014/ADR-015 Self-Critique.

Identities tab layout: split into "Actors" (Users, Clients) and "Access Containers" (Groups, Roles), each type its own MUI Accordion (expanded by default if non-empty), with a client-side "Quick search" filter by name and an "info" button on User/Client rows opening a Drawer with that identity's cached Roles/Groups — see the Identities UI ADR.

Also visible in the Admin Console's "Identities" tab (Type, Name, Display Name, Last Synced) with a "Sync Now" button.

A central registry (inventory_services/health_metrics) of REST services and MCP agents this gateway fronts — onboarded manually via the Admin Console's "Registry" tab, auto-discovered, and health-monitored, both actively (periodic ping) and passively (real routed traffic).

Routing is 100% driven by this registry (ADR-017). There are no hardcoded Gateway routes — InventoryRouteDefinitionLocator builds /api/v1/{name}/** → base_url for every ACTIVE/WARNING REST-type entry, refreshed immediately on any onboard/edit/ delete and periodically otherwise (zte.routing.refresh-interval-ms, 30s default) to pick up status changes from discovery/health-polling. Onboarding a new REST service via the Admin Console makes it reachable at /api/v1/<name>/** immediately — no redeploy. service-a/service-b are seeded into the registry automatically at gateway startup (InventoryBootstrapSeeder, from the service-a.uri/service-b.uri properties) so a fresh docker compose up still routes them out of the box.

Onboarding → auto-discovery: POST /api/v1/admin/inventory persists a PENDING row and returns immediately — AutoDiscoveryWorker probes the service in the background (never delaying the response): GET {base_url}/v3/api-docs for REST (or an explicit docs_url override — a full absolute URL, for a target whose OpenAPI document doesn't live at that conventional path; REST-only, ADR-016 amendment), a stateless POST {base_url}/message JSON-RPC tools/list call for MCP. Success → ACTIVE; failure or timeout → WARNING ("reachable enough to route, but its schema/tool list couldn't be confirmed" — a degraded state that requires manual attention, not a hard failure). Note that status == ACTIVE alone doesn't guarantee a schema was captured — a 2xx response with an empty or non-JSON body still reaches ACTIVE but captures nothing; use hasSchema (below) to check.

Health polling: every 60s (zte.inventory.health-poll-interval-ms), HealthPollingService pings every ACTIVE/WARNING/DOWN service's /actuator/health and records last_ping_ms/actuator_status. A failed ping flips ACTIVE→DOWN; a successful one flips DOWN→ACTIVE — self-healing. WARNING is never touched by this job: a successful raw health ping doesn't mean the service's actual API/tool contract works, so it must not silently clear a discovery failure.

Registering service-a/service-b themselves: AutoDiscoveryWorker/HealthPollingService inject the application's default WebClient.Builder, which already carries the gateway's ZTE mTLS client certificate whenever zte.mtls.enabled=true — the same builder every other outbound gateway component uses, confirmed both by inspecting Spring Boot's ClientHttpConnectorAutoConfiguration and live (see ADR-016's amendment section). Register base_url=https://localhost:8081 (or :8082) so discovery exercises the real mTLS API port — both services expose /v3/api-docs (springdoc-openapi-starter-webflux-ui), so this correctly reaches ACTIVE. Health polling pings a different endpoint (/actuator/health), which service-a/service-b only expose on their separate plain-HTTP management port, not the mTLS API port — set the optional management_url field (http://localhost:9081 / :9082) at onboarding time so health polling targets the right port instead of base_url; leaving it blank falls back to base_url, which will show DOWN for these two services specifically (ADR-016 amendment, 2026-08-11).

Passive telemetry: RequestAuditFilter fires an async, non-blocking update (mirrors RequestLogAuditService's fire-and-forget architecture, ADR-013) on every 2xx routed response, setting health_metrics.last_successful_call for the matching inventory entry (matched by name — must equal the path segment RequestTargetResolver derives, e.g. service-a). Never blocks the request thread; a target name with no matching registry row is a harmless no-op.

Discovered schema (API Catalog): a successful discovery probe now captures the raw response body — the OpenAPI document for REST, the JSON-RPC tools/list response for MCP — into inventory_services.discovered_schema. Fetch it on demand via GET /api/v1/admin/inventory/{id}/schema (404 if nothing's been captured yet), or click a row's "View Schema" (📄) button in the Admin Console: REST targets render in an embedded Swagger UI, MCP targets as a plain tool name/description list. Deliberately excluded from the main registry list/CRUD payload so viewing the registry table stays light regardless of how large a target's schema is (ADR-016 amendment, 2026-08-12). The Admin Console gates "View Schema" on the list response's hasSchema field, not status — see the note above on why those two aren't equivalent.

Synchronous fetch: POST /api/v1/admin/inventory/{id}/schema/fetch (or the "Fetch" 🔄 button on a Registry row) runs discovery immediately and waits for the result, instead of the passive background trigger onboarding uses. 200 on success; 404 if id doesn't exist; 502 Bad Gateway if the target was unreachable, timed out, or returned no valid JSON — deliberately stricter here than the background worker's ACTIVE-on-any-2xx tolerance, since a human just clicked "Fetch" and needs a real yes/no answer, with an error message the UI shows directly (ADR-016 amendment, 2026-08-12, second). It's a true refetch — clicking it on a service that already has a captured schema overwrites the old one unconditionally, no special handling needed.

Editing a registration: the "Edit" (✏️) button on a Registry row opens the same onboarding dialog pre-filled with that row's data and saves via PUT /api/v1/admin/inventory/{id} instead of POST — every onboarding field (base_url, docs_url, management_url) is editable this way. Like onboarding, saving an edit always resets status to PENDING and re-triggers discovery (ADR-016 amendment, 2026-08-12, third). PUT returns 409 if the new name collides with a different existing entry (renaming without changing the name never false-positives against itself), and 404 if id doesn't exist (ADR-016 amendment, 2026-08-12, fourth).

# Onboard a service
curl -sk -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  https://localhost:8080/api/v1/admin/inventory \
  -d '{"name":"hubspot-mcp","targetType":"MCP","baseUrl":"http://localhost:9090"}'

# Onboard a REST service whose OpenAPI docs live somewhere other than /v3/api-docs
curl -sk -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  https://localhost:8080/api/v1/admin/inventory \
  -d '{"name":"legacy-api","targetType":"REST","baseUrl":"https://legacy.example.com","docsUrl":"https://legacy.example.com/swagger.json"}'

# List the registry (includes current health snapshot)
curl -sk -H "Authorization: Bearer $TOKEN" https://localhost:8080/api/v1/admin/inventory | python3 -m json.tool

# Fetch a service's captured schema (once discovery has succeeded at least once)
curl -sk -H "Authorization: Bearer $TOKEN" https://localhost:8080/api/v1/admin/inventory/<id>/schema | python3 -m json.tool

# Trigger discovery synchronously (e.g. after fixing a docs_url typo) and wait for the result
curl -sk -X POST -H "Authorization: Bearer $TOKEN" -w "\n%{http_code}\n" \
  https://localhost:8080/api/v1/admin/inventory/<id>/schema/fetch

See ADR-016 for the full design, including the MCP discovery assumption and the WARNING-is-sticky decision.

The zt-agents module adds an AI-native security copilot to the ZTE stack.

Fetches all access policies from the gateway and sends them to Claude for a zero-trust compliance analysis. Returns a structured Markdown security report.

Endpoint: POST /api/v1/agents/auditor/run (port 8083)

Prerequisites: Set ANTHROPIC_API_KEY before starting zt-agents (see ADR-008).

# 1. Copy the .env template and fill in your Anthropic API key
cp .env.example .env
# edit .env and set ANTHROPIC_API_KEY=sk-ant-...

# 2. Start the gateway (must be running — zt-agents fetches policies from it)
./gradlew :gateway-service:bootRun

# 3. Start zt-agents
./gradlew :zt-agents:bootRun

# 4. Run the policy audit (expect 10–60 s while Claude analyses the policies)
curl -s --max-time 150 -X POST http://localhost:8083/api/v1/agents/auditor/run | python3 -m json.tool

Expected response shape:

{
    "report": "## Executive Summary\n...\n## Risk Findings\n..."
}

Security notes:

• The gateway's internal endpoint (GET /api/v1/internal/policies) is network-restricted (Docker bridge) and requires no JWT for MVP. See ADR-007 for the production upgrade path.
• Never commit ANTHROPIC_API_KEY. Use .env files (gitignored) or Vault in production.

The stack also runs as containers in Azure Container Apps — full plan, topology, live-run gotchas and the security review: docs/azure-deployment-plan.md.

One front door (ADR-040): a single gateway app on demo.zteasy.tech serving browsers and agents alike. Azure's HTTP ingress requests a client certificate (clientCertificateMode: Accept) and relays it to the gateway, which validates it against our own CA — the edge accepts any certificate, so that check is not optional. ADR-028 originally split this into two apps because TCP passthrough was the only way to keep a client certificate; that split, and the two copies of every in-memory thing it produced, is gone.

# Provision from scratch (needs GHCR_USER/GHCR_PAT, HUBSPOT_TOKEN; see the plan)
./deploy/azure/deploy.sh

# Park it overnight / bring it back / see what's running.
# `stop` dumps the database to the pgbackup share first and refuses to stop if
# that fails; `start` restores it automatically (ADR-033), so the audit trail,
# approval queue, policy toggles and ACAP lifecycle are still there tomorrow.
./deploy/azure/power.sh stop
./deploy/azure/power.sh start
./deploy/azure/power.sh status

# Dump the database without stopping anything (before a risky change, say)
az containerapp job start -n db-backup -g zteasy-demo-rg

# Attach a custom domain + free managed certificate (prints the DNS records first)
./deploy/azure/bind-custom-domain.sh
./deploy/azure/bind-custom-domain.sh demo.zteasy.tech

Deployment-only settings worth knowing: ZTE_INTERNAL_API_KEY (shared secret for /api/v1/internal/** — mandatory once the gateway has a public ingress), ZTE_AUTH_PROXY_ENABLED (serve Keycloak's login under /auth), ZTE_UI_OIDC_AUTHORITY (what both SPAs get from /ui-config.js), and MANAGEMENT_PORT set equal to each service's API port (a Container App publishes only one port). Three more exist only because a container is not a developer machine (ADR-033): ZT_AGENTS_URI and ZTE_GATEWAY_CA_CERT — without which the gateway calls itself for policy audits, and zt-agents cannot speak TLS back for token metering — and ZTE_POLICY_FILE, which puts the policy document on the file share so that "Reload Policies" re-reads something an operator can actually edit.

Editing policy on a running deployment (this is what makes the AI auditor's "Modify policy" suggestion applicable, ADR-031):

az storage file upload --account-name <acct> --share-name certs \
   --source gateway-service/src/main/resources/zte-policies.yaml --path zte-policies.yaml
# then press "Reload Policies" in the console, or:
curl -X POST https://demo.zteasy.tech/api/v1/admin/policies/reload -H "Authorization: Bearer $TOKEN"

# 1. Prerequisites: Java 21, Docker Desktop, openssl, keytool, Node.js + npm
#    (Node/npm build the Admin Console — see "Building without Node/npm installed" above
#    if you want to skip it)

# 2. Generate local secrets into a gitignored .env (ADR-037 — the repository
#    carries no values, not even "dev-only" ones, because a committed default
#    is inherited by whatever forgets to override it, and this project shipped
#    one to a live deployment). Idempotent: re-running tops the file up.
./scripts/generate-dev-secrets.sh

# 3. Generate development certificates (keystore password comes from .env)
chmod +x certs/generate-certs.sh && set -a && . ./.env && set +a && ./certs/generate-certs.sh

# 4. Generate the Keycloak realm — keycloak/realm-export.json is a template whose
#    client secrets are placeholders; this substitutes the ones from .env into
#    keycloak/local/realm.json, which is what compose imports.
python3 deploy/azure/make-cloud-realm.py --local

# 5. Start infrastructure (PostgreSQL + Keycloak). Compose reads the same .env,
#    so a missing value stops it here with the variable's name rather than
#    starting something with a published password.
docker compose up -d

# 6. (Only if you need to reset a password without re-importing the realm —
#    step 4 already baked every demo user's password in from .env)
#    ./scripts/set-keycloak-password.sh

# 7. Start services (each in a separate terminal). They read .env through
#    spring-dotenv; without it they fail at startup naming the missing property.
./gradlew :gateway-service:bootRun
./gradlew :service-a:bootRun
./gradlew :service-b:bootRun

# 8. Get an ADMIN token
TOKEN=$(curl -s -X POST http://localhost:8180/realms/zte-realm/protocol/openid-connect/token \
  -d "grant_type=password&client_id=zte-gateway&client_secret=$ZTE_SECRET_ZTE_GATEWAY" \
  -d "username=zte-admin&password=$ZTE_LOCAL_ADMIN_PASSWORD" | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")

# 7. Call the full chain: User → Gateway → Service A → Service B
# As of ADR-018, /api/v1/** (proxied REST traffic) requires a client certificate in
# addition to the JWT — server.ssl.client-auth: want + MtlsEnforcementWebFilter. This
# applies uniformly to this general REST-proxy surface, not just agent/MCP traffic, so
# even this interactive-user demo call needs --cert now. -k accepts the dev self-signed CA.
curl -sk --cert certs/client.p12:$ZTE_KEY_PASSWORD --cert-type P12 \
  -H "Authorization: Bearer $TOKEN" https://localhost:8080/api/v1/service-a/hello | python3 -m json.tool

Expected response:

{
    "service": "service-a",
    "caller": "<keycloak-user-uuid>",
    "message": "Hello from Protected Service A",
    "service-b": "{\"service\":\"service-b\",\"sub\":\"...\",\"roles\":[\"ADMIN\"],\"trustBasis\":\"mTLS (ZTE-CA) + HMAC-SHA256 OBO token\"}"
}