Tokentide first, with a fallback provider

Tokentide sells a limited supply of discounted inference every day: the daily capacity we hold, shared by all customers, at up to 63% below list price. When the day’s supply is used up, it is sold out until the daily reset at 00:00 UTC, and your own balance, spend caps and rate limits can stop a request too (what can stop a request). That is why we recommend this setup for production: send every request to Tokentide first, and keep the provider you use today (Anthropic, OpenAI, Venice, Google or any other) as the fallback. You pay Tokentide’s price for every request we serve and your provider’s usual price only for the rest, and your users never notice the switch.

How it works#

1. Every request goes to Tokentide first

   Your code calls Tokentide with the OpenAI SDK, as in the Quickstart.

2. Tokentide refuses before anything runs

   When it can’t serve a request (today’s shared supply is sold out, your balance or a spend cap is used up, you went over a rate limit), Tokentide answers before anything is streamed or charged, with a Retry-After header that says when to try again: the 00:00 UTC reset for a sold-out day, the reset of a spend cap, a few seconds for a rate limit.

3. Your code sends the same request to your fallback

   It also remembers Retry-After: until that time has passed, requests go straight to your fallback, without a round trip to Tokentide.

4. When Retry-After has passed, requests flow back to Tokentide

   After a sold-out day that is the 00:00 UTC reset, when the supply renews. The first request after Retry-After goes to Tokentide again; when there is no Retry-After (an empty balance, a blocked key), the examples try again after 30 seconds.

This is what a sold-out answer looks like:

HTTP/2 503
content-type: application/json
retry-after: 14400
x-request-id: req_01JDV6Q7Z8R2M3N4P5Q6R7S8T9
 
{
  "error": {
    "message": "Today's discounted capacity is sold out. Sales reopen at the daily reset (00:00 UTC).",
    "type": "tokentide_error",
    "code": "capacity_unavailable",
    "param": null,
    "request_id": "req_01JDV6Q7Z8R2M3N4P5Q6R7S8T9"
  }
}

What can stop Tokentide serving a request#

The day selling out is the best-known reason, not the only one. Every refusal carries a code that says why and, when waiting helps, a Retry-After that says when to try again. The fallback handles all of them.

The shared supply#

• Today’s supply is sold out. All customers share one discounted supply each day, and it grows as we add capacity. When it is used up: 503 capacity_unavailable until the 00:00 UTC reset (Retry-After counts down to it). GET /v1/market shows sold_out and reopens_at ahead of time.
• A model is busy, paused or failing. A busy model turns new requests away, Low Tide first, then new accounts: 429 rate_limited for a few seconds. A paused model, or one that keeps failing, answers 503 capacity_unavailable or 502 upstream_unavailable for seconds to minutes. X-Tokentide-Fallback-Models can move such a request to another model on Tokentide.
• Image capacity is shared too: about 20 image requests per minute for all customers together, so image endpoints can answer 429 rate_limited sooner than text.
• Around the daily reset, image endpoints and quotes wait for the new day (503 epoch_rollover, usually under a minute), video and music jobs have a daily cutoff, and a request that may run past midnight pays tomorrow’s opening price, or waits a few seconds when too many such requests are running (503 capacity_unavailable, Retry-After: 10).

Your balance and spend caps#

• Your balance runs out: 402 insufficient_balance, with no Retry-After: it clears when you top up. Before that, chat answers are shortened to what your balance covers (X-Tokentide-Output-Capped). Turn on auto top-up and a low-balance alert. If an automatic top-up fails (an expired card, for example), the balance runs out as above.
• A spend cap you set is reached (daily, weekly or monthly, on a key or on the account): 402 spend_limit_reached, with a Retry-After until the cap resets.
• A card payment is disputed: the disputed credit is frozen until the dispute is resolved, which can leave too little to spend.

Your limits#

• Rate limits per key and per account: requests per minute, concurrent requests, the spending rate per minute and the cost of the requests running at once. They depend on your trust tier, and the ones counted in money are shares of the day’s capacity, so they grow with it: 429 rate_limited, with a Retry-After of seconds. The X-RateLimit-* headers and GET /v1/account show them.
• Your price limits (X-Tokentide-Max-Multiplier, X-Tokentide-Max-Cost-USD, or a key’s maximum multiplier): 409 price_above_limit, with a Retry-After until the scheduled price falls below your limit, or 409 cost_above_limit. A limit below the morning price keeps your requests out until the tide falls.
• Low Tide is closed, or your Low Tide allowance is used up: 409 lowtide_closed or 409 lowtide_cap_reached. Send X-Tokentide-Lowtide-Fallback: standard and the request runs on Standard instead.
• Repeated invalid requests: more than 10 4xx errors on one model within 30 seconds trigger a 30-second block (429 rate_limited).

Your key and account#

• The key was revoked or has expired (an expiry date you set, or the end of a rotation’s overlap): 401 invalid_api_key. The dashboard warns you before a key expires.
• The request came from an address outside the key’s IP allowlist: 403 ip_not_allowed.
• The account is suspended or closed: 403 account_restricted.
• The key’s privacy mode doesn’t allow the model: 403 model_not_allowed. That is a choice you made, so don’t route around it (see the table below).
• A model leaves the catalog: models due for retirement show their date and replacement in the model list; switch before the date.

Incidents#

• At the model provider: 502 upstream_unavailable after our retries, 504 upstream_timeout, or 429 rate_limited when the provider is overloaded.
• At Tokentide: 500 internal_error or 503 service_unavailable; maintenance can pause sales (503 capacity_unavailable).
• status.tokentide.app posts incidents and maintenance, with the model provider shown separately.

When to fall back#

Fall back when the request was fine but Tokentide can’t serve it right now. Don’t fall back when the request itself is wrong: the fallback would hide the bug, or fail the same way.

Every code is described in Errors. Refused requests are never charged.

Pick the model pairs#

Decide which model your fallback uses for each Tokentide model. Two patterns work well:

• The same model on both routes. Claude, GPT and Gemini models are in the Tokentide catalog as Anonymized models, so Anthropic, OpenAI and Google can serve the very same model at their own price; your Tokentide key then needs privacy mode all (privacy modes). If you already use Venice, its API takes the same model ids as Tokentide, Private models included.
• Private first. Use a Private model on Tokentide (for example zai-org-glm-5-2 or deepseek-v4-1-flash), and a vendor model of the same class as the fallback. Answers differ a little between the two, and what the fallback serves follows the vendor’s terms (see Costs and privacy).

Model ids differ, except at Venice: Tokentide uses its catalog’s ids, and each vendor uses its own.

Catalogs change, so check both lists before you ship (GET /v1/models with your key, and your vendor’s model list), and test the fallback so a wrong id shows up in staging rather than on a sold-out day.

Anthropic as the fallback#

Tokentide through the OpenAI SDK, Claude through Anthropic’s own SDK (pip install openai anthropic, or npm install openai @anthropic-ai/sdk). Set TOKENTIDE_API_KEY and ANTHROPIC_API_KEY.

import logging
import os
import time
 
import anthropic
import openai
from openai import OpenAI
 
tokentide = OpenAI(
    base_url="https://api.tokentide.app/v1",
    api_key=os.environ["TOKENTIDE_API_KEY"],  # privacy mode "all" reaches Claude models
    max_retries=0,  # hand over to the fallback at once instead of retrying
)
claude = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY
 
MODEL = "claude-opus-5-5"  # the same id on Tokentide and at Anthropic
FALL_BACK_ON = {402, 429, 500, 502, 503, 504}  # sold out, no credit, caps, limits, incidents
BLOCKED = {"invalid_api_key", "account_restricted", "ip_not_allowed"}  # fall back, and alert someone
skip_until = 0.0  # after a refusal, go straight to Anthropic until Retry-After has passed
 
 
def ask(system: str, prompt: str) -> str:
    global skip_until
    if time.time() >= skip_until:
        try:
            reply = tokentide.chat.completions.create(
                model=MODEL,
                messages=[
                    {"role": "system", "content": system},
                    {"role": "user", "content": prompt},
                ],
            )
            return reply.choices[0].message.content or ""
        except openai.APIStatusError as e:
            blocked = e.code in BLOCKED  # the key or the account: a person has to fix it
            if e.status_code not in FALL_BACK_ON and not blocked:
                raise  # a problem with the request itself: fix it rather than hide it
            if blocked:
                logging.error("Tokentide refused the key or the account: %s", e.code)
            skip_until = time.time() + int(e.response.headers.get("retry-after", "30"))
        except openai.APIConnectionError:
            skip_until = time.time() + 30
 
    message = claude.messages.create(
        model=MODEL,
        max_tokens=16000,
        system=system,
        messages=[{"role": "user", "content": prompt}],
    )
    if message.stop_reason == "refusal":
        raise RuntimeError("Claude declined this request")
    return "".join(block.text for block in message.content if block.type == "text")
 
 
print(ask("Answer in one sentence.", "Why do tides happen?"))

• The two APIs shape a request differently: Anthropic takes the system prompt in system and needs max_tokens. To pass a whole conversation, keep the user and assistant turns in messages and move the system prompt to system.
• Claude can decline a request (stop_reason: "refusal"). Handle it as the example does, or see Anthropic’s guide to refusals and fallbacks.

OpenAI as the fallback#

One SDK for both: two clients, two model ids, the same request (pip install openai, or npm install openai). Set TOKENTIDE_API_KEY and OPENAI_API_KEY.

import logging
import os
import time
 
import openai
from openai import OpenAI
 
tokentide = OpenAI(
    base_url="https://api.tokentide.app/v1",
    api_key=os.environ["TOKENTIDE_API_KEY"],  # privacy mode "all" reaches GPT models
    max_retries=0,  # hand over to the fallback at once instead of retrying
)
fallback = OpenAI()  # reads OPENAI_API_KEY
 
MODELS = {"tokentide": "openai-gpt-55", "fallback": "gpt-5.5"}  # one model, each provider's id
FALL_BACK_ON = {402, 429, 500, 502, 503, 504}  # sold out, no credit, caps, limits, incidents
BLOCKED = {"invalid_api_key", "account_restricted", "ip_not_allowed"}  # fall back, and alert someone
skip_until = 0.0  # after a refusal, go straight to OpenAI until Retry-After has passed
 
 
def chat(messages: list[dict], **params):
    global skip_until
    if time.time() >= skip_until:
        try:
            return tokentide.chat.completions.create(
                model=MODELS["tokentide"], messages=messages, **params
            )
        except openai.APIStatusError as e:
            blocked = e.code in BLOCKED  # the key or the account: a person has to fix it
            if e.status_code not in FALL_BACK_ON and not blocked:
                raise  # a problem with the request itself: fix it rather than hide it
            if blocked:
                logging.error("Tokentide refused the key or the account: %s", e.code)
            skip_until = time.time() + int(e.response.headers.get("retry-after", "30"))
        except openai.APIConnectionError:
            skip_until = time.time() + 30
    return fallback.chat.completions.create(model=MODELS["fallback"], messages=messages, **params)
 
 
reply = chat([{"role": "user", "content": "Why do tides happen? One sentence."}])
print(reply.choices[0].message.content)

Tools, structured output and every other request field go to whichever provider serves the request, unchanged.

Venice as the fallback#

If you already call Venice’s API with your own account, it can be your fallback. Tokentide uses the same model ids, so there is nothing to map, and Private models stay Private on both routes. Requests it serves are billed by Venice, at its prices. One SDK for both (pip install openai, or npm install openai). Set TOKENTIDE_API_KEY and VENICE_API_KEY (a key from your Venice account).

import logging
import os
import time
 
import openai
from openai import OpenAI
 
tokentide = OpenAI(
    base_url="https://api.tokentide.app/v1",
    api_key=os.environ["TOKENTIDE_API_KEY"],
    max_retries=0,  # hand over to the fallback at once instead of retrying
)
fallback = OpenAI(  # Venice's own API, with the key of your Venice account
    base_url="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
)
 
# Tokentide uses Venice's model ids, so both routes take the same id, Private models included.
MODELS = {"tokentide": "zai-org-glm-5-2", "fallback": "zai-org-glm-5-2"}
FALL_BACK_ON = {402, 429, 500, 502, 503, 504}  # sold out, no credit, caps, limits, incidents
BLOCKED = {"invalid_api_key", "account_restricted", "ip_not_allowed"}  # fall back, and alert someone
skip_until = 0.0  # after a refusal, go straight to Venice until Retry-After has passed
 
 
def chat(messages: list[dict], **params):
    global skip_until
    if time.time() >= skip_until:
        try:
            return tokentide.chat.completions.create(
                model=MODELS["tokentide"], messages=messages, **params
            )
        except openai.APIStatusError as e:
            blocked = e.code in BLOCKED  # the key or the account: a person has to fix it
            if e.status_code not in FALL_BACK_ON and not blocked:
                raise  # a problem with the request itself: fix it rather than hide it
            if blocked:
                logging.error("Tokentide refused the key or the account: %s", e.code)
            skip_until = time.time() + int(e.response.headers.get("retry-after", "30"))
        except openai.APIConnectionError:
            skip_until = time.time() + 30
    return fallback.chat.completions.create(model=MODELS["fallback"], messages=messages, **params)
 
 
reply = chat([{"role": "user", "content": "Why do tides happen? One sentence."}])
print(reply.choices[0].message.content)

• venice_parameters, the E2EE headers and every other request field work the same way on both routes.
• Venice sees your IP address and your account for the requests it serves, under its terms and privacy policy (on Private models, Venice keeps no prompts or responses, according to Venice’s privacy policy).
• Venice runs the models on both routes, so an incident there reaches both: for that case, chain a second fallback (Anthropic, OpenAI or Gemini) after Venice.

Gemini as the fallback#

Google also serves the Gemini API on an OpenAI-compatible endpoint, so the OpenAI SDK reaches both providers. Set TOKENTIDE_API_KEY and GEMINI_API_KEY (a Gemini API key from Google AI Studio).

import logging
import os
import time
 
import openai
from openai import OpenAI
 
tokentide = OpenAI(
    base_url="https://api.tokentide.app/v1",
    api_key=os.environ["TOKENTIDE_API_KEY"],  # privacy mode "all" reaches Gemini models
    max_retries=0,  # hand over to the fallback at once instead of retrying
)
fallback = OpenAI(  # Google's OpenAI-compatible endpoint for the Gemini API
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
    api_key=os.environ["GEMINI_API_KEY"],
)
 
MODELS = {"tokentide": "gemini-3-8-flash", "fallback": "gemini-3.8-flash"}  # one model, each provider's id
FALL_BACK_ON = {402, 429, 500, 502, 503, 504}  # sold out, no credit, caps, limits, incidents
BLOCKED = {"invalid_api_key", "account_restricted", "ip_not_allowed"}  # fall back, and alert someone
skip_until = 0.0  # after a refusal, go straight to Gemini until Retry-After has passed
 
 
def chat(messages: list[dict], **params):
    global skip_until
    if time.time() >= skip_until:
        try:
            return tokentide.chat.completions.create(
                model=MODELS["tokentide"], messages=messages, **params
            )
        except openai.APIStatusError as e:
            blocked = e.code in BLOCKED  # the key or the account: a person has to fix it
            if e.status_code not in FALL_BACK_ON and not blocked:
                raise  # a problem with the request itself: fix it rather than hide it
            if blocked:
                logging.error("Tokentide refused the key or the account: %s", e.code)
            skip_until = time.time() + int(e.response.headers.get("retry-after", "30"))
        except openai.APIConnectionError:
            skip_until = time.time() + 30
    return fallback.chat.completions.create(model=MODELS["fallback"], messages=messages, **params)
 
 
reply = chat([{"role": "user", "content": "Why do tides happen? One sentence."}])
print(reply.choices[0].message.content)

Google calls its OpenAI compatibility beta. If you prefer Google’s own SDK, replace only the fallback call (see Google’s OpenAI compatibility guide and its text generation guide): the Tokentide part stays the same.

Streaming#

The same pattern works for streams. A sold-out answer is an HTTP error that arrives before the first token, so there is still time to pick the fallback, and a stream that has already started is not affected when the day sells out. These functions reuse the clients and MODELS of the OpenAI, Venice and Gemini examples:

def stream(messages: list[dict], **params):
    global skip_until
    if time.time() >= skip_until:
        try:
            # A sold-out answer is raised here, as an HTTP error, before the first token.
            return tokentide.chat.completions.create(
                model=MODELS["tokentide"], messages=messages, stream=True, **params
            )
        except openai.APIStatusError as e:
            blocked = e.code in BLOCKED
            if e.status_code not in FALL_BACK_ON and not blocked:
                raise
            if blocked:
                logging.error("Tokentide refused the key or the account: %s", e.code)
            skip_until = time.time() + int(e.response.headers.get("retry-after", "30"))
        except openai.APIConnectionError:
            skip_until = time.time() + 30
    return fallback.chat.completions.create(
        model=MODELS["fallback"], messages=messages, stream=True, **params
    )
 
 
for chunk in stream([{"role": "user", "content": "Write a haiku about the sea."}]):
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="")

With Anthropic as the fallback, stream with claude.messages.stream(...) and read its text deltas: the place where you decide is the same.

Check ahead of time#

GET /v1/market (public, no key) says whether today’s supply is sold out and when sales reopen. Check it before a large batch, or show it in your own dashboard:

curl -s https://api.tokentide.app/v1/market | jq '{sold_out, reopens_at}'

While sold_out is true, reopens_at is the next 00:00 UTC; otherwise it is null. The market API describes every field.

Test the fallback#

A fallback you have never run fails on the day you need it. In staging:

1. Force the fallback: set skip_until (skipUntil) far in the future, or point the Tokentide client at an address where nothing listens, such as http://127.0.0.1:9/v1.
2. Check that every request comes back from your fallback, with the model you expect.
3. Record which provider served each request, and X-Tokentide-Charge-USD for the ones Tokentide served, so you can see your savings and how often you fall back.

Each process learns that the day is sold out from its own first refused request, so there is no need to share skip_until between servers.

Costs and privacy#

• Each provider bills what it serves. Requests Tokentide refuses are never charged, and your balance stays yours: it doesn’t expire at midnight. Requests your fallback serves are billed by that provider, at its prices.
• Spend caps work with the fallback. A Tokentide spend cap answers 402 spend_limit_reached, which sends requests to your fallback: you can cap what you spend on Tokentide without stopping your app.
• Privacy follows the provider that serves the request. Tokentide never stores prompts or responses. What your fallback serves is covered by your agreement with that provider. If you use Private models for privacy reasons, keep those requests off the fallback: queue them until the reset (Retry-After says when), or fall back only to a provider whose terms meet your requirements. With Venice as the fallback, they stay on the same Private model.

Fallbacks inside Tokentide#

Two headers handle the cases that don’t need another provider:

• X-Tokentide-Lowtide-Fallback: standard runs a Low Tide request on Standard when Low Tide is closed or your allowance is used up (Low Tide).
• X-Tokentide-Fallback-Models lists up to 3 backup models on Tokentide for when the requested model is offline or overloaded (request headers).

Neither helps on a sold-out day, when the whole supply is used up: that is what your fallback provider is for.

Gateways and routers#

If your requests go through an LLM gateway or router, add Tokentide as an OpenAI-compatible provider (base URL https://api.tokentide.app/v1), put it first, and set your current provider as its fallback. Check that the gateway falls back on 429, 5xx and timeouts, and that it waits for Retry-After before it tries Tokentide again.