> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mezmo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Request Lifecycle

> Request flow, timeout configuration, cancellation, and graceful shutdown behavior.

## Overview

AURA manages per-request state (cancellation tokens, subscriptions) across streaming SSE connections. This document covers the lifecycle, timeout configuration, and known limitations.

***

## Request Flow

```
Client POST /v1/chat/completions
         │
         ▼
┌─────────────────────────────────────┐
│ Shutdown middleware check           │
│   (503 if shutdown_token cancelled) │
└─────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│ Generate request_id (UUID)          │
└─────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│ Spawn producer task                 │
│   - Register cancellation token     │
│   - Subscribe to progress events    │
│   - Subscribe to tool events        │
└─────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│ Stream chat with TimeoutHook        │
│   - Tool calls set thread context   │
│   - Tool results clear context      │
└─────────────────────────────────────┘
         │
         ├──────────────────┬──────────────────┐
         ▼                  ▼                  ▼
┌────────────────┐  ┌──────────────────┐  ┌──────────────────┐
│ Normal         │  │ Timeout/         │  │ Shutdown         │
│ completion     │  │ Disconnect       │  │ (grace expired)  │
└────────────────┘  └──────────────────┘  └──────────────────┘
         │                  │                  │
         │                  ▼                  ▼
         │          ┌──────────────────┐  ┌──────────────────┐
         │          │ Cancel token     │  │ Cancel hook +    │
         │          │ Send MCP cancel  │  │  registry        │
         │          │ Evict from pool  │  │ Send [DONE]      │
         │          └──────────────────┘  │ Then MCP cleanup │
         │                  │             │ (no pool evict)  │
         │                  │             └──────────────────┘
         └──────────┬───────┴──────────────────┘
                    ▼
┌─────────────────────────────────────┐
│ Cleanup (RAII guard)                │
│   - Unregister cancellation         │
│   - Unsubscribe progress            │
│   - Unsubscribe tool events         │
└─────────────────────────────────────┘
```

***

## Timeout Configuration

### Production Defaults

| Setting               | Default | Env Variable               | Purpose                                           |
| --------------------- | ------- | -------------------------- | ------------------------------------------------- |
| First chunk timeout   | 90 sec  | `FIRST_CHUNK_TIMEOUT_SECS` | Max wait for first provider chunk                 |
| Stream timeout        | 15 min  | `STREAMING_TIMEOUT_SECS`   | Max request duration                              |
| Shutdown grace period | 30 sec  | `SHUTDOWN_TIMEOUT_SECS`    | Time for in-flight requests to finish on shutdown |
| Heartbeat             | 15 sec  | —                          | Disconnect detection                              |

### Rationale

* **First chunk timeout (90 sec)**: Catches provider connection failures early. If the LLM hasn't sent any data within this window, the request is aborted rather than hanging for the full stream timeout. The window is sized for reasoning models, which can legitimately take over a minute before the first chunk.
* **Stream timeout (15 min)**: Supports long-running MCP tools. Set to 0 to disable (not recommended).
* **Heartbeat (15 sec)**: Standard SSE keepalive. Detects disconnect during silent tool execution.

***

## Tool Event Correlation

Rig spawns tool execution in separate tokio tasks. The hook context (where the LLM decides to call a tool) and the execution context (where MCP actually runs) are decoupled, requiring a mechanism to correlate `tool_call_id` across these boundaries — AURA does this with a per-request FIFO queue. This relies on Rig's streaming mode executing tools sequentially within a request; see [Rig Fork Changes](https://github.com/mezmo/aura/blob/main/docs/rig-fork-changes.md) for the validation methodology if you're tracking Rig upstream compatibility.

***

## Cleanup Mechanism

Request resources (cancellation registration, progress/tool-event subscriptions) are cleaned up via an RAII guard that ensures cleanup runs even on panic. If the async runtime is already gone (process exit), cleanup is skipped — resources are reclaimed with the process.

***

## MCP Cancellation

On client disconnect or timeout:

1. The cancellation token signals all waiting code
2. Aura sends `notifications/cancelled` to MCP servers
3. The agent is evicted from its connection pool to prevent stale connections

MCP servers receive the cancellation notification and can abort in-progress operations.

***

## Graceful Shutdown

The server uses a two-phase shutdown with separate cancellation tokens:

| Token                   | Cancelled                                  | Purpose                                  |
| ----------------------- | ------------------------------------------ | ---------------------------------------- |
| `shutdown_token`        | Immediately on signal                      | Middleware rejects new requests with 503 |
| `stream_shutdown_token` | After `SHUTDOWN_TIMEOUT_SECS` grace period | Terminates remaining in-flight streams   |

### Shutdown Sequence

1. **SIGTERM/SIGINT** received
2. **Phase 1 (immediate)**: `shutdown_token` cancelled — middleware returns 503 for all new requests
3. **Grace period**: In-flight streams continue running for up to `SHUTDOWN_TIMEOUT_SECS` (default 30s). Streams that complete naturally during this window are unaffected.
4. **Phase 2 (drain)**: `stream_shutdown_token` cancelled — remaining streams:
   * Cancel hook + request registry (stops in-flight MCP tool execution)
   * Send `[DONE]` to client (before MCP cleanup, so client gets clean termination)
   * Run MCP cleanup (send `notifications/cancelled`, close connections)
   * No pool eviction (pool is dying with the server)
5. **Server stops**: Workers have 10s to complete Phase 2 cleanup

### Shutdown vs Disconnect/Timeout

| Behavior        | Disconnect/Timeout                                 | Shutdown                               |
| --------------- | -------------------------------------------------- | -------------------------------------- |
| Pool eviction   | Yes                                                | No (pool is dying)                     |
| `[DONE]` timing | After MCP cleanup (Timeout) / skipped (Disconnect) | Before MCP cleanup                     |
| Grace period    | None — immediate cancel                            | Configurable (`SHUTDOWN_TIMEOUT_SECS`) |