Multi-Tenant Core API
The middleware every utility-customer portal in the fleet stands on - two generations of it, and the migration from one to the other.
- Role
- Primary author of both generations, architecture and security owner
- Period
- 2023 - present
- Ownership
- 58% of v1 · 76% of v2
- Stack
- Fastify 5 · TypeScript (strict) · Zod · MongoDB · AWS SDK v3 · ECS Fargate · Vitest
Problem
A fleet of utility companies each needed a customer portal talking to Oracle Utilities CCS, the billing system of record. The v1 server I inherited when I joined in 2023 - an Express codebase that predates me, 58% mine by commit today - had authorization gaps and no clean way to vary behavior per tenant. Every new client meant forked code and re-audited security.
The v2 rebuild, started in early 2026, had one mandate: a single API where a new tenant is configuration, not code, and where authorization failure is hard to ship by accident. Six of the seven launched tenants run production on v1 today; the seventh launched directly on v2 in June 2026. The first v1-to-v2 migration runs v2 beside its v1 service, certified, with the cutover held. The fleet moves one tenant at a time.
Constraints
Utility customers pay bills through this thing - it has to be up and it has to be right. Oracle CCS is reached over OAuth 2.0 with per-tenant credentials. The team is small, so the architecture had to make the secure path the easy path, and the migration cannot pause billing.
Decisions
- Both generations run side by side: the portals carry interceptors for either backend, and a tenant moves only when its parity checks pass.
- Tenant behavior is composed from a base configuration plus per-tenant overrides, assembled by strategy factories - the request path never branches on tenant name.
- Authorization is enforced as a route-level gate: every account-scoped endpoint proves the caller owns the account it touches before any data access, which is what closes IDOR at the route layer.
Architecture
Fastify 5 on Node 22, ESM-only, TypeScript strict with no any types. Zod validates every input at the boundary; responses use one fixed envelope so clients never parse ad-hoc shapes.
On v2, admin reads and impersonated sessions are audited as well as writes, attributed to the acting admin, with field-level capture of what a write actually changed and, on the v2 capture path, deep redaction of tokens, passwords and identifiers. Admin writes are kept for a year, admin reads for ninety days.
Where the billing system of record and the payment provider disagree, a reconciler surfaces the drift first; repairs run guarded, in shadow before they write, and the account-linking path guards against duplicate registrations and signup conflicts. Enforcing the ten-digit account number on one tenant's sign-in forms targeted about 215 CCS errors a day that partial or oversized numbers had been generating.
Typecheck, lint, tests and a Snyk gate run in CI; Bitbucket Pipelines builds to ECR and deploys to ECS Fargate behind a shared ALB.
Outcome
- On v2 a tenant is config plus provisioning, not a fork; one launched tenant runs production on v2, the first migration runs v2 beside its v1, certified, with the cutover held, and v1 stays patched and audited until the last one moves.
- Authorization is structural, not reviewed-in: the route contract enforces ownership checks on every account-scoped endpoint.
- Billing drift surfaces through a reconciler first; repairs run guarded, in shadow before they write.