Rate limits

Tokentide enforces its own limits per key and per account. They exist to keep the service fair, to protect the shared upstream capacity, and to bound the damage of a leaked key. All limits answer 429 rate_limited with a Retry-After header (seconds).

Tokentide spreads upstream traffic across a pool of Venice keys, so the limits you see come from Tokentide’s own per-key limits and your trust tier below, never from a single upstream key. Venice’s headers about its own limits are never passed through.

Limits by trust tier#

Tiers. Accounts funded by card start as new (until they are at least 7 days old and have paid $100 by card). Tiers rise with payment history and verification; going above verified requires two-factor authentication. Business customers can ask for custom limits. GET /v1/account shows your tier and effective limits:

{
  "object": "account",
  "tier": "verified",
  "limits": {
    "requests_per_minute_per_key": 600,
    "concurrency": 32,
    "image_requests_per_minute": 6,
    "daily_cap_usd": null,
    "weekly_cap_usd": null,
    "monthly_cap_usd": "200",
    "account_daily_cap_usd": null,
    "account_weekly_cap_usd": "500",
    "account_monthly_cap_usd": null
  },
  "lowtide": { "allowance_usd": "4.2", "used_usd": "0.35", "cap_per_hour_usd": "2", "open": true }
}

Image models are scarce upstream (Venice allows about 20 image requests per minute account-wide), which is why the image limits are low.

Headers#

Requests authenticated with an API key carry our limit for that key (never Venice’s):

Congestion shedding#

Venice’s rate limits apply per model to the whole upstream account. When a model family approaches its limit, Tokentide first raises the price (the congestion surge, see Lanes and Tide Pricing), and at 97% utilization sheds the lowest-priority traffic with 429 rate_limited (“This model is at capacity right now”): Low Tide first, then new accounts. This happens before the upstream limit is reached, so customers see a clean 429 with Retry-After instead of upstream failures.

The 4xx circuit breaker#

The upstream account has a shared error budget: too many failed requests lock a model for everyone for 30 seconds. To keep one buggy client from causing that, Tokentide validates requests before forwarding them (model exists and is allowed, capabilities, token caps, input sizes), and if one customer still causes more than 10 upstream 4xx within 30 seconds, that customer receives 429 rate_limited for 30 seconds. Do not retry 400-class errors unchanged.

Body, duration and stream limits#

Non-streaming JSON responses (chat completions and responses without "stream": true, embeddings, decisions, JSON transcriptions) are relayed in one piece, up to 64 MB. A larger one answers 502 upstream_unavailable after Venice has already run and billed it, so it is still charged (see Errors inside a stream). Use "stream": true for large chat or responses outputs, and a smaller batch or "encoding_format": "base64" for embeddings.

There is also a coarse outer layer at the edge firewall (per IP and TLS fingerprint) that absorbs floods before they reach the API. Its 429 responses may not use the JSON error envelope; honor Retry-After when present.

Retrying well#

async function withRetry(send: () => Promise<Response>, maxAttempts = 5): Promise<Response> {
  let delayMs = 500;
  for (let attempt = 1; ; attempt++) {
    const res = await send();
    const retryable = res.status === 429 || res.status === 503 || res.status === 500 || res.status === 502;
    if (!retryable || attempt >= maxAttempts) return res;
    const retryAfter = Number(res.headers.get('retry-after'));
    const waitMs = Number.isFinite(retryAfter) && retryAfter > 0 ? retryAfter * 1000 : delayMs;
    await new Promise((r) => setTimeout(r, waitMs + Math.random() * 250));
    delayMs = Math.min(delayMs * 2, 30_000);
  }
}

• Add an Idempotency-Key to requests you retry after a network error, so a request that did run is not run twice.
• For batch work, let the Tide Runner schedule requests around Low Tide openings and limits.