Database Access from Serverless

By Michael Samuel Naeem · July 18, 2025 · 6 min read

Engineering

CloudWatch showed FATAL: too many connections for role "app" seconds after the marketing email dropped. Five hundred Lambdas each opened a Postgres connection because the code copied a Flask SQLAlchemy snippet. Serverless compute scales horizontally by default; relational databases scale connections vertically with hard limits. The fix is not "lower Lambda concurrency"—it is architecture that matches ephemeral workers to durable connection budgets.

RDS Proxy pattern

Lambda → RDS Proxy → Aurora/RDS
         (pools ~1000 clients → ~50 DB conns)

import psycopg2

def handler(event, context):
    conn = psycopg2.connect(
        host=os.environ["PROXY_ENDPOINT"],
        user=os.environ["DB_USER"],
        password=get_secret(),
        sslmode="require",
    )
    try:
        with conn.cursor() as cur:
            cur.execute("SELECT 1")
    finally:
        conn.close()  # returns to proxy pool quickly

Enable IAM authentication to avoid static passwords in env vars. Set Proxy max connections percent below database limit.

with production-like data volume before declaring done. Capture metrics baseline the week before change and compare for seven days after—subtle regressions hide in aggregates until a large tenant hits the path. Update the on-call runbook with the failure signature and rollback command so responders need not rediscover steps during an incident.

Reuse connections across invocations

Lambda may reuse execution environment:

conn = None

def handler(event, context):
    global conn
    if conn is None or conn.closed:
        conn = psycopg2.connect(...)
    ...

Reuse helps warm invocations but is insufficient alone at high concurrency—still need Proxy.

HTTP data APIs

Aurora Serverless v2 with RDS Data API or PostgREST layer exposes HTTP SQL without persistent connections from Lambda. Trade latency and feature limits for connection simplicity.

DynamoDB and serverless-native stores

table.put_item(Item={"pk": f"USER#{user_id}", "sk": "PROFILE", ...})

No connection pool; on-demand billing matches spiky traffic. Design access patterns upfront—GSIs for alternate queries.

Read replicas and timeouts

Set statement_timeout and connect_timeout low so stuck queries release proxy slots. Route read-only analytics to replica endpoint via separate Lambda or connection string.

Secrets and rotation

Fetch credentials from Secrets Manager with caching in global scope; rotate via dual-user pattern without redeploying all functions. IAM DB auth tokens expire—refresh before 15-minute boundary.

Opening connection per query inside loop
Running migrations from Lambda on every cold start
max_connections = 1000 on tiny instance instead of pooling

IAM DB auth tokens expire—refresh before 15-minute boundary in long-running handlers though Lambda lifetime usually shorter.

Statement_timeout and connect_timeout low so stuck queries release proxy slots. Read replica routing for analytics Lambdas separate from OLTP connection strings.

Anti-pattern: migrations on every cold start—run migrations from CI deploy job, not invocation path.