Everything you need to know

No. The instrumentation takes under 1 millisecond per call. Events are queued in memory and flushed in a background daemon thread every 5 seconds — your application's critical path is never blocked.

OpenAI (chat completions, Responses API, embeddings, audio transcription/TTS, image generation — sync and async, streaming and non-streaming), Anthropic (messages API, sync and async, streaming and non-streaming, prompt cache tracking), Google AI (generate_content and generate_content_stream, sync and async), DeepSeek (chat, FIM, and beta chat — sync and async), and OpenRouter (registered client instances, sync and async). Custom providers can be added using BasePatch.

Call meter.set_limit('tenant-id', limit_usd=50.00) to set a monthly USD cap. When the limit is reached, the next intercepted LLM call raises a LimitExceededException. Catch it and show an upgrade prompt — the SDK never silently drops the call. A per_day frequency is also supported.

For providers not yet patched, or any custom logic, call meter.track_manually(provider='cohere', model='command-r-plus', input_tokens=512, output_tokens=128, tenant_id='acme-corp'). Any extra keyword arguments are passed through as event fields. Custom model pricing can be set so that manually-tracked calls are correctly costed against tenant spend limits.

Use meter.for_tenant() as a context manager in your route handlers or middleware. It uses Python's ContextVar system, so it's fully async-safe — concurrent requests with different tenant IDs are always isolated correctly. You can also call meter.set_tenant(request.tenant_id) imperatively in middleware.

Events are held in an in-memory queue (default max 1000). If the queue fills up, the oldest events are dropped. Configure on_flush_error to route failures to Sentry or your alerting system.

Use the on_event hook to inspect captured events in your test suite. The context manager form (with Meter(config) as meter:) guarantees patches are removed and the queue is flushed on exit — clean test isolation out of the box.

A gzip-compressed JSON POST with body {events: [...]}. Each event carries: provider, model, endpoint, tenant_id, input_tokens, output_tokens, total_tokens, cached_tokens, duration_ms, timestamp, and a tags dict. Auth via Bearer token. Configure the endpoint via url in MeterConfig.

Yes. Define a pricing entry for any model name, specifying input and output cost per million tokens (and optionally cached, cache_creation, image, audio, and character rates). TokenLimit uses these rates when computing spend for both auto-patched and manually-tracked events.