LlamaIndex

LlamaIndex builds RAG + agent applications with retrieval as a first-class primitive. Trefur's patch_llamaindex adapter hooks the LlamaIndex global callback manager so every agent step + LLM call + tool invocation + node retrieval emits telemetry.

Install

pip install trefur-observe llama-index llama-index-llms-openai

Setup

import os
from llama_index.core.agent import ReActAgent
from llama_index.core.tools import QueryEngineTool, FunctionTool
from llama_index.llms.openai import OpenAI
from trefur_observe import TrefurObserve, patch_llamaindex

# Initialise observe once at startup.
TrefurObserve.init(
    api_key=os.environ["TREFUR_API_KEY"],
    agent={"name": "docs-rag", "framework": "llamaindex"},
)
patch_llamaindex(TrefurObserve.get_instance())

def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

llm = OpenAI(model="gpt-4o")
agent = ReActAgent.from_tools(
    [FunctionTool.from_defaults(fn=multiply)],
    llm=llm,
    verbose=False,
)
response = agent.chat("What is 42 multiplied by 17?")

What gets traced

  • Agent reasoning loops — each ReAct iteration shows the thought, action, action-input, observation chain.
  • Tool calls — every FunctionTool and QueryEngineTool invocation.
  • Node retrieval — vector + keyword + hybrid retrievers emit step type retrieval with the top-k node ids + similarity scores.
  • LLM completions — model, tokens, prompt + response (subject to your redaction level).
  • Re-ranking — Cohere, Jina, ColBERT re-rankers tagged on the retrieval step.

Common use case — RAG over docs

The standard pattern wraps a vector index in a QueryEngineTool and gives it to a ReAct agent. Trefur renders the full chain: retrieval → reasoning → answer synthesis, with retrieval latency + node ids visible on each step. Build drift alerts when retrieval-step latency rises above a baseline (often a sign of vector-store regressions).

Common pitfalls

  • Global callback manager. LlamaIndex's callbacks are installed on the global Settings object. If you construct your own Settings instance after patch_llamaindex, you replace the callbacks. Re-callpatch_llamaindex after any Settings rebuild.
  • Streaming chat. agent.stream_chat works, but emissions happen on stream-close. Premature client disconnect drops the trailing steps.
  • Async query engines. Use agent.achat and remember to await it before exit; otherwise call flush() explicitly.

See the Python SDK reference for the full adapter surface.