Zero-Downtime Migrations in CD

By Michael Samuel Naeem · January 6, 2026 · 5 min read

The deploy failed at 2 AM because someone added NOT NULL to a column the old API version still writes without. Postgres locked the table for four minutes. Checkout returned 503. The migration ran in the same pipeline stage as the container push — "it's automated" — with no backward compatibility check.

Zero-downtime migrations aren't about faster tools. They're about ordering: schema changes, app deploys, and schema cleanup as separate releases.

The expand-contract lifecycle

Every breaking schema change splits into three releases:

Release N — Expand

• Add new column/table/index
• Keep old path working
• Migration is additive only

Release N+1 — Migrate

• Deploy app that dual-writes or reads new column
• Backfill data asynchronously
• Old app version still runs if rollback needed

Release N+2 — Contract

• Remove old column after confirming zero reads/writes
• Drop deprecated indexes

Example: rename email to email_address.

-- Release N: expand (run BEFORE new app deploy)
ALTER TABLE users ADD COLUMN email_address TEXT;

-- Release N+1: backfill (job, not blocking deploy)
UPDATE users SET email_address = email WHERE email_address IS NULL;

-- App now writes to both columns (deploy new version)

-- Release N+2: contract (run AFTER old app fully retired)
ALTER TABLE users DROP COLUMN email;

Never ALTER TABLE users RENAME COLUMN email TO email_address in a single deploy. Old pods break instantly.

Migration ordering in the pipeline

Our GitHub Actions pipeline stages:

1. lint + unit tests
2. integration tests (current schema)
3. migration dry-run against ephemeral DB
4. apply migration to staging
5. integration tests (old app vs new schema)  ← catches compatibility breaks
6. deploy new app to staging
7. smoke tests
8. (manual promote) migration + deploy to prod

# Simplified pipeline stage
migrate-staging:
  steps:
    - run: flyway migrate -url=$STAGING_DB_URL
    - run: ./scripts/test-old-app-against-new-schema.sh
    - run: kubectl rollout status deployment/api

test-old-app-against-new-schema.sh runs the previous release's Docker image against the migrated database. If it fails, the pipeline stops before prod.

Flyway and Liquibase in CD

Flyway — SQL-first, versioned files (V001__add_users.sql). Simple, reviewable in PRs.

-- V042__add_email_address.sql
ALTER TABLE users ADD COLUMN IF NOT EXISTS email_address TEXT;
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_users_email_address
  ON users (email_address);

Use CONCURRENTLY for index creation on Postgres — standard CREATE INDEX locks writes.

Liquibase — XML/YAML/SQL changesets with rollback blocks. Rollbacks are seductive and often untested; we use forward-only migrations with expand-contract instead of rollback in prod.

Lock migration execution to one runner. Flyway's flyway_schema_history table prevents double-apply; still use a pipeline mutex so two branches don't race.

Patterns that work in production

Nullable first, enforce later.

ALTER TABLE orders ADD COLUMN discount_cents INTEGER;
-- Later release, after backfill:
ALTER TABLE orders ALTER COLUMN discount_cents SET NOT NULL;
ALTER TABLE orders ALTER COLUMN discount_cents SET DEFAULT 0;

Feature flags for read path switches. App reads email_address if flag on, else email. Flip flag after backfill completes. Remove flag in contract release.

Online index creation. Postgres CREATE INDEX CONCURRENTLY, MySQL InnoDB online DDL (still watch lock waits on large tables).

Avoid table rewrites. Adding column with default in Postgres 11+ is metadata-only for nullable without default. Adding DEFAULT + NOT NULL in one statement rewrites the table on older versions.

When zero-downtime isn't worth it

Maintenance windows are fine for internal tools with scheduled downtime SLAs, databases under 100 MB where locks last seconds, or one-shot data fixes with communicated outage.

For customer-facing APIs with rolling deploys, the expand-contract tax pays for itself the first time you rollback an app without rolling back schema.

Tooling integration with ORMs

Prisma, Django, and Rails migrations generate SQL that may not be expand-contract friendly out of the box. Review generated SQL in PRs — frameworks happily emit DROP COLUMN in the same release as code changes.

For Prisma, split migrations manually when needed: create empty migration, edit SQL to expand-only, deploy app, second migration for contract phase. Django SeparateDatabaseAndState helps when model rename should not emit destructive SQL yet.

Document migration class (expand, contract, data) in PR title. Release managers skip contract migrations until deploy metrics confirm zero traffic on old app version.

Common production mistakes

Teams get database migrations zero downtime wrong in predictable ways:

• Skipping failure-mode rehearsal — run a game day or fault injection exercise before peak traffic, not after the first outage.
• Missing correlation context — every error path should carry request, trace, or tenant identifiers so incidents are debuggable.
• Optimizing for demo, not steady state — load tests, cache warm-up, and cold-start paths matter more than local dev latency.
• Undocumented trade-offs — if you chose speed over strict correctness (or vice versa), write that down for the next engineer.

Production implementations of database migrations zero downtime fail when staging mirrors production topology poorly, rollback is untested, and on-call runbooks describe the happy path only.

Debugging and triage workflow

When database migrations zero downtime misbehaves in production, work top-down instead of guessing:

1. Confirm scope — one tenant, region, or deployment stage? Narrow blast radius before deep diving.
2. Check recent changes — deploys, flag flips, config pushes, and schema migrations in the last 24 hours.
3. Compare golden signals — latency, error rate, saturation, and traffic for the affected surface vs. baseline.
4. Reproduce minimally — smallest input or scenario that triggers the failure; capture traces/logs with correlation IDs.
5. Fix forward or rollback — if rollback is faster than root-cause during incident, rollback first, postmortem second.
6. Add a guard — alert, integration test, or circuit breaker so the same class of failure is caught earlier next time.

Document the timeline during triage. Future you (and on-call) will need timestamps, not just conclusions.

Resources

• Flyway documentation
• Liquibase best practices
• PostgreSQL CREATE INDEX CONCURRENTLY
• Stripe's online migrations blog post
• Braintree expand-contract approach