Async Rust with Tokio

By Michael Samuel Naeem · April 25, 2025 · 5 min read

Your first Axum handler awaited three HTTP calls sequentially and p99 latency hit 900ms. Someone added std::thread::sleep inside an async fn during a quick debug and production froze—because Tokio's executor assumes every .await yields quickly. Async Rust is not "threads but easier"; it is cooperative scheduling with explicit yield points. Tokio is the de facto runtime for network services, but only if you respect what runs on worker threads versus what belongs in spawn_blocking.

Runtime setup

#[tokio::main(flavor = "multi_thread", worker_threads = 4)]
async fn main() -> anyhow::Result<()> {
    let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await?;
    axum::serve(listener, app()).await?;
    Ok(())
}

worker_threads defaults to CPU count; tune down in sidecar containers sharing cores. Enable console subscriber feature during development to visualize task poll times.

Validate this in staging with production-like data volume before declaring done. Capture metrics baseline the week before change and compare for seven days after—subtle regressions hide in aggregates until a large tenant hits the path. Update the on-call runbook with the failure signature and rollback command so responders need not rediscover steps during an incident.

Spawning and joining tasks

let handle = tokio::spawn(async move {
    fetch_profile(user_id).await
});

let profile = handle.await??;

Dropping a JoinHandle without awaiting detaches the task—it keeps running. For request-scoped work, await all handles before returning the response or use tokio::task::JoinSet for dynamic fan-out with cancellation on client disconnect.

Validate this in staging with production-like data volume before declaring done. Capture metrics baseline the week before change and compare for seven days after—subtle regressions hide in aggregates until a large tenant hits the path. Update the on-call runbook with the failure signature and rollback command so responders need not rediscover steps during an incident.

Concurrent I/O with join! and try_join!

let (user, orders) = tokio::try_join!(
    user_client.get(user_id),
    order_client.list_for_user(user_id),
)?;

try_join! short-circuits on first error; join! waits for all even if one fails. Pick based on whether partial results are useful.

Validate this in staging with production-like data volume before declaring done. Capture metrics baseline the week before change and compare for seven days after—subtle regressions hide in aggregates until a large tenant hits the path. Update the on-call runbook with the failure signature and rollback command so responders need not rediscover steps during an incident.

select! for cancellation and timeouts

tokio::select! {
    result = operation() => { /* ... */ }
    _ = tokio::time::sleep(Duration::from_secs(5)) => {
        return Err(Error::Timeout);
    }
    _ = cancel_token.cancelled() => {
        return Err(Error::Cancelled);
    }
}

Use CancellationToken from tokio-util to propagate shutdown through nested tasks when SIGTERM arrives.

Validate this in staging with production-like data volume before declaring done. Capture metrics baseline the week before change and compare for seven days after—subtle regressions hide in aggregates until a large tenant hits the path. Update the on-call runbook with the failure signature and rollback command so responders need not rediscover steps during an incident.

Channels for backpressure

let (tx, mut rx) = tokio::sync::mpsc::channel(1024);

tokio::spawn(async move {
    while let Some(job) = rx.recv().await {
        process(job).await;
    }
});

Bounded channels apply backpressure: send().await blocks producers when consumers lag. For multiple consumers, broadcast or watch channels fit config updates and fan-out notifications.

Validate this in staging with production-like data volume before declaring done. Capture metrics baseline the week before change and compare for seven days after—subtle regressions hide in aggregates until a large tenant hits the path. Update the on-call runbook with the failure signature and rollback command so responders need not rediscover steps during an incident.

Isolating blocking code

let data = tokio::task::spawn_blocking(move || {
    expensive_cpu_or_blocking_io(input)
})
.await??;

If blocking work dominates, a dedicated pool (blocking_threads in runtime builder) prevents starving async I/O. Long term, replace with async drivers.

Validate this in staging with production-like data volume before declaring done. Capture metrics baseline the week before change and compare for seven days after—subtle regressions hide in aggregates until a large tenant hits the path. Update the on-call runbook with the failure signature and rollback command so responders need not rediscover steps during an incident.

Use #[tokio::test] with start_paused for time-dependent logic. tokio::test macros spin a runtime per test—fine for unit tests; integration tests may share one runtime via OnceLock.

Enable tokio-console during development to visualize task poll times. Tasks that block worker threads show up immediately as long polls—fix before production traffic multiplies the damage.

Structure concurrency to match connection lifetime. Unbounded spawn per request creates task churn; sometimes one read loop with state machine beats thousands of short-lived tasks. Semaphores cap in-flight work when downstream cannot absorb unlimited parallelism.

Graceful shutdown on SIGTERM should propagate CancellationToken through nested tasks. Kubernetes sends SIGTERM before SIGKILL; drain in-flight requests before exit so clients receive proper errors instead of connection resets mid-response.

Validate this in staging with production-like data volume before declaring done. Capture metrics baseline the week before change and compare for seven days after—subtle regressions hide in aggregates until a large tenant hits the path. Update the on-call runbook with the failure signature and rollback command so responders need not rediscover steps during an incident.

Document the decision, owner, and rollback path in your team wiki the same week you ship. Future you will not remember which environment variable toggled the behavior unless it is written next to the runbook entry and linked from the alert. That habit costs ten minutes per change and saves hours when pagination or auth misbehaves under a single large tenant.

Resources

• Tokio tutorial
• Tokio spawn documentation
• Async book
• Axum web framework
• Tokio tracing and console