LLM observability.
Trace every model call. Attribute every token and every dollar. Alert on drift. Compare vendors on the same workload. Across OpenAI, Anthropic, Bedrock, Azure OpenAI, Gemini — on top of the OpenTelemetry GenAI standard.
What LLM observability means.
LLM observability is the practice of capturing every call your application makes to a large language model — the prompt, the completion, the token count, the latency, the cost, the model name, the vendor — so you can debug failures, attribute spend, and detect regressions when something changes.
The data lives in OpenTelemetry GenAI Semantic Conventions. That means one canonical shape across OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, and Gemini. Swap providers without losing trace continuity or rebuilding dashboards.
LLM observability is necessary but not sufficient for production AI workloads. Once your application is an agent — making decisions, calling tools, retrying on failure — you need agent observability on top, which is what Trefur ships.
Why LLM observability matters.
Five concrete outcomes when you turn it on.
Stop debugging from log greps
Every prompt, every completion, every tool call captured at the moment it happens. Search by trace ID, customer, or error class — open the trace, see what the model saw.
See cost the day it drifts
Tokens and dollars attributed per model call. Roll up by agent, build, deployment, vendor. Catch a 40% bill spike before it hits finance.
Compare vendors on real workload
Run the same prompt across OpenAI, Anthropic, and Bedrock. See tokens, latency, success rate side-by-side on the data your users actually generate.
Catch regressions before customers do
Envelope alerts on retry rate, error rate, latency, and cost-per-call — routed to oncall with the offending trace already attached.
Adopt without re-platforming
OpenTelemetry-native. Drop Trefur next to Datadog or Honeycomb as a second exporter. Your existing pipeline stays in place.
How Trefur does LLM observability.
Six capabilities. Drop-in for the runtime you already ship.
Model-call tracing
Every LLM call captured with prompt, completion, tokens, latency, cost, vendor.
Read moreToken + dollar attribution
Per-call cost rollups by agent, build, deployment, model vendor. Drift alerts on envelope.
Read moreVendor-agnostic shape
One canonical span shape across OpenAI, Anthropic, Bedrock, Azure OpenAI, Gemini.
Read moreDrift alerts
Retry-rate spikes, cost drift, latency walks, error envelopes — routed with the trace.
Read moreSelf-hostable collector
Run the collector in your VPC. Redact prompts before egress. Fan out to existing OTel backends.
Read moreReplay any call
Reconstruct the exact prompt the model saw — full system message, full history, full tool args.
Read moreGet started in under five minutes.
Drop the SDK in. Auto-instrumented OpenAI, Anthropic, Bedrock, Azure OpenAI, and Gemini calls show up in the trace immediately.
import { TrefurObserve, patchOpenAI, patchAnthropic } from "@trefur/observe";
import OpenAI from "openai";
import Anthropic from "@anthropic-ai/sdk";
TrefurObserve.init({ apiKey: "trf_obs_..." });
const obs = TrefurObserve.getInstance();
patchOpenAI(OpenAI, obs);
patchAnthropic(Anthropic, obs);
// Both clients are auto-instrumented.
const openai = new OpenAI();
const anthropic = new Anthropic();
// Every call appears in Trefur with prompt, completion, tokens, cost.Trace your LLM calls today.
Free tier. No card required. First trace in under five minutes.