LLM observability.
Trace every model call. Attribute every token and every dollar. Alert on drift. Compare vendors on the same workload. Across OpenAI, Anthropic, Bedrock, Azure OpenAI, Gemini — on top of the OpenTelemetry GenAI standard.
What LLM observability means.
LLM observability is the practice of capturing every call your application makes to a large language model — the prompt, the completion, the token count, the latency, the cost, the model name, the vendor — so you can debug failures, attribute spend, and detect regressions when something changes.
The data lives in OpenTelemetry GenAI Semantic Conventions. That means one canonical shape across OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, and Gemini. Swap providers without losing trace continuity or rebuilding dashboards.
LLM observability is necessary but not sufficient for production AI workloads. Once your application is an agent — making decisions, calling tools, retrying on failure — you need agent observability on top, which is what Trefur ships.
Why LLM observability matters.
Five concrete outcomes when you turn it on.
Debug from the trace
Every prompt, every completion, every tool call captured at the moment it happens. Search by trace ID, customer, or error class — open the trace, see what the model saw.
See cost the day it drifts
Tokens and dollars attributed per model call. Roll up by agent, build, deployment, vendor. Catch a 40% bill spike before it hits finance.
Compare vendors on real workload
Run the same prompt across OpenAI, Anthropic, and Bedrock. See tokens, latency, success rate side-by-side on the data your users actually generate.
Catch regressions before customers do
Envelope alerts on retry rate, error rate, latency, and cost-per-call — routed to oncall with the offending trace already attached.
Adopt alongside your current stack
OpenTelemetry-native. Drop Trefur next to Datadog or Honeycomb as a second exporter. Your existing pipeline stays in place.
How Trefur does LLM observability.
Six capabilities. Drop-in for the runtime you already ship.
Model-call tracing
Every LLM call captured with prompt, completion, tokens, latency, cost, vendor.
Read moreToken + dollar attribution
Per-call cost rollups by agent, build, deployment, model vendor. Drift alerts on envelope.
Read moreVendor-agnostic shape
One canonical span shape across OpenAI, Anthropic, Bedrock, Azure OpenAI, Gemini.
Read moreDrift alerts
Retry-rate spikes, cost drift, latency walks, error envelopes — routed with the trace.
Read moreReplay any call
Reconstruct the exact prompt the model saw — full system message, full history, full tool args.
Read moreGet started in under five minutes.
Point your OpenTelemetry exporter at Trefur with a telemetry key from the app. Model calls from OpenAI, Anthropic, Bedrock, Azure OpenAI, and Gemini show up in the trace.
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.trefur.com/v1/traces export OTEL_EXPORTER_OTLP_TRACES_HEADERS="Authorization=Bearer%20trf_obs_..." # Any OpenTelemetry-instrumented agent now sends its traces to Trefur.