Best LLM Observability and Evaluation Tools in 2026

For most teams, start with Langfuse. It has an MIT-licensed core, free self-hosting, OTLP ingest, datasets, LLM-as-judge and a CI action, and a hosted Core plan at $29 per month. Choose LangSmith if you build on LangChain or LangGraph, Braintrust if evals drive your releases, Phoenix if you want open source with no feature gates, and Datadog or Splunk if one of them already runs your monitoring. Promptfoo is the pick for prompt regression tests and red teaming in CI.

The market also moved in 2026. Langfuse joined ClickHouse, Helicone changed hands and went into maintenance mode, Cisco completed its Galileo acquisition, and OpenAI agreed to buy Promptfoo. That makes portable, OpenTelemetry-based instrumentation worth more than it was a year ago.

Video: an official Langfuse demo of agent tracing, evals, experiments and prompt management, a useful picture of what the whole category does (published by the Langfuse YouTube channel).

Quick Picks

Master Comparison Table

Every cell comes from the vendor's own pricing or docs page, fetched on November 11, 2026. Units differ between vendors (traces, units, spans, GB, scores, requests), so compare your own traffic against each row rather than the headline price. Licenses were read from each GitHub repository or the vendor page.

What Changed in 2026

Four of the eleven tools have a new owner or a changed status this year, and the rest of the comparison reads differently once you know it.

• Langfuse announced on January 16, 2026 that it joined ClickHouse. The post says there are no planned licensing changes, self-hosting stays a first-class path, and Langfuse Cloud keeps running as is. It does not address pricing.
• Helicone was acquired by Mintlify, which said Helicone will keep running in maintenance mode with security updates, bug fixes and new models, and that it would help customers migrate. Helicone's own pricing page now carries a banner saying it is part of Experiential Labs, without further detail.
• Galileo is now Splunk Agent Observability after Cisco completed the acquisition. Its old pricing URL redirects to Splunk's pricing page.
• Promptfoo said on March 9, 2026 that it agreed to be acquired by OpenAI, subject to customary closing conditions, and that it will remain open source. I could not confirm a closing date in the sources I fetched.
• Weights & Biases pricing for Weave now lives on a CoreWeave Forge page, and the old wandb.ai pricing URL redirects there.

The practical implication, which is our inference and not a vendor statement, is that you should not couple application code to one vendor SDK when an OpenTelemetry exporter will do. If you also route model traffic through a proxy, see our best LLM API gateways roundup, since several gateways log requests and overlap with parts of this category.

LangSmith

LangSmith is LangChain's platform for tracing, evaluation and agent deployment. It is the natural choice when your agent is built with LangChain or LangGraph, because tracing is a switch rather than a project.

Pricing (as of Nov 11, 2026): Developer is $0 per seat with one seat and 5k base traces per month. Plus is $39 per seat per month with unlimited seats and 10k base traces. Enterprise is custom. Traces beyond the included amount are pay as you go, but the pricing page does not publish a per-1k trace price, so budget from the vendor calculator or a quote. Base traces keep 14 days of data and extended traces keep 180 days. Self-hosted and hybrid (SaaS control plane, self-hosted data plane) deployments are listed under Enterprise only, and the managed cloud runs in the US or EU.

Strengths. Evaluation coverage is broad: datasets built from curated cases, production traces or synthetic data, offline evals for regression testing, online evals with filters and sampling rates, LLM-as-judge (including reference-free online judges), human review, code rules and pairwise comparison. Studio gives you an interactive environment to debug agents. OpenTelemetry works through an /otel endpoint with an x-api-key header, and the langsmith[otel] extra can send traces only to your own OTel endpoint. For private-cloud deployments we have a separate walkthrough of the LangSmith BYOC on AWS architecture.

Weaknesses. Per-seat pricing grows with the team, the trace price is not public on the page, and self-hosting needs an Enterprise contract. Ingestion through OTel is asynchronous, and the docs warn that if a span's parent never arrives, the child is eventually dropped without an error. Best for: LangChain and LangGraph teams that want tracing, evals and deployment under one roof.

Video: LangChain's own overview of LangSmith for agent observability, evaluation and deployment.

Langfuse

Langfuse is the open-source option that most teams reach for first. It covers tracing, prompt management, datasets, experiments and evaluators, and it runs the same way in the cloud or on your own infrastructure. Its repository describes an MIT license, and the LICENSE file notes that portions of the software are licensed differently, so read it if you plan to redistribute.

Pricing (as of Nov 11, 2026): Hobby is free with 50k units per month, 30 days of data access and two users. Core is $29 per month with 100k units, 90 days and unlimited users. Pro is $199 per month with 100k units and three years of data access. Enterprise is $2,499 per month. Pro also has an optional Teams add-on at $300 per month. Overage is $8 per 100k units, graduating to $7 (1M to 10M), $6.50 (10M to 50M) and $6 (50M and up). Data retention management is limited to Pro and Enterprise. Self-hosting is free; the pricing page points to a separate self-host page for details.

Strengths. The evaluation docs list LLM-as-a-judge on live traces, reusable datasets, experiments that compare prompt, model or code changes, annotation queues, code evaluators, and score analytics that measure agreement between human and AI labels. The langfuse/experiment-action GitHub Action runs experiments on pull requests and raises a RegressionError when a score falls below your threshold. OpenTelemetry ingest sits at /api/public/otel, and the docs call the older ingestion endpoint deprecated in favor of OTLP. Regions include EU, US, Japan and a HIPAA cloud.

# Send OpenTelemetry traces from one app to Langfuse and a second backend
# via an OpenTelemetry Collector (pattern documented by LangSmith and Langfuse).
export OTEL_EXPORTER_OTLP_ENDPOINT="https://us.cloud.langfuse.com/api/public/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic ${AUTH_STRING},x-langfuse-ingestion-version=4"

# AUTH_STRING = base64 of "pk-lf-...:sk-lf-..."
echo -n "pk-lf-...:sk-lf-..." | base64

Weaknesses. Langfuse accepts OTLP over HTTP only, with no gRPC. Without the x-langfuse-ingestion-version: 4 header, direct OTel data can take up to ten minutes to appear, per the docs. Unit-based pricing means a chatty agent that emits many observations per request can burn units quickly. The ClickHouse acquisition is stable on paper, but pricing was not addressed in the announcement. Best for: teams that want an open, self-hostable default with a complete tracing and eval loop.

Braintrust

Braintrust positions itself as an observability platform for agents with evals at the center: trace production, score it, and catch regressions before users do. It is a hosted product, and the pricing page lists on-prem or hosted deployment only under Enterprise.

Pricing (as of Nov 11, 2026): Starter is $0 with 1 GB of processed data, 10k scores, 14 days of retention and unlimited users, projects, datasets and experiments. Pro is $249 per month with 5 GB, 50k scores and 30 days of retention. Overage is $4 per GB on Starter and $3 per GB on Pro for data, and $2.50 and $1.50 per 1,000 scores. Pro adds custom charts and environments and unlimited human review scores; Starter allows one per project. Pro retention beyond the included 30 days is $0.50 per GB per month. Qualifying startups get Pro free for 6 to 12 months. Enterprise adds custom RBAC and premium support.

Strengths. Scorers can be built from autoevals, LLM-as-judge or custom code, and online scoring evaluates production traces asynchronously with no latency impact. Datasets hold inputs with optional expected outputs, and the docs describe running experiments from code or the UI and in CI/CD. OpenTelemetry arrives over OTLP/HTTP at api.braintrust.dev/otel with a Bearer key and an x-bt-parent header, and there are EU and BYOC data-plane URLs. Weaknesses. Scores are a metered unit, so heavy online scoring can dominate the bill, and the 14 and 30 day retention on the base tiers is short next to Langfuse Pro's three years. It is not listed as open source. Best for: teams that gate releases on eval scores and want a polished hosted workflow.

Arize Phoenix and Arize AX

Phoenix is the open-source tracing and evaluation tool from Arize and its community. The docs say it is built on OpenTelemetry and uses OpenInference for instrumentation, with auto-instrumentation for frameworks, model providers and languages including Python, TypeScript and Java. Arize AX is the managed platform built on the same open standards.

Pricing (as of Nov 11, 2026): Phoenix is free to self-host with no feature limits, license fees or usage limits, according to its self-hosting docs. The repository LICENSE is the Elastic License 2.0, which is source-available rather than OSI open source, so check it against your use case. AX Free is $0 with 25k spans, 1 GB ingestion and 15 days of retention. AX Pro is $50 per month with 50k spans, 10 GB and 30 days. AX Enterprise is custom and the only plan listed for self-hosting. Users and evals are unlimited on all three AX plans. The pricing page does not describe overage.

Strengths. Evaluators can be LLM-based, code-based or human annotations, and Phoenix can import evaluators from Ragas, DeepEval and Cleanlab. It also offers prompt management, a playground, span replay, datasets and experiments. Deployment options include Docker, Kubernetes, Helm and one-click templates for Railway, Render, Cloud Run and Azure. Weaknesses. AX Pro is small at 50k spans, and the ELv2 license rules out offering Phoenix as a managed service to others. Best for: teams that want a no-gate open-source tool, especially for RAG and retrieval evaluation. If you debug retrieval quality, our vector database comparison covers the other half of that problem.

Weights & Biases Weave

Weave is W&B's toolkit for tracing and evaluating LLM applications. The Python SDK is Apache-2.0 on GitHub. Pricing and docs have moved to CoreWeave properties: wandb.ai/site/pricing redirects to the CoreWeave Forge pricing page, and the Weave docs redirect to docs.coreweave.com.

Pricing (as of Nov 11, 2026): Free is $0 with 1 GB of Weave data ingestion, 5 GB of storage and up to five model seats. Pro starts at $60 per month with 1.5 GB of ingestion, 100 GB of storage and up to ten seats, at $0.10 per MB beyond the included ingestion. Enterprise is custom with $0.20 per MB overage, a dedicated or self-managed deployment and a bring-your-own-bucket option. The page says Pro is intended for early-stage teams with fewer than 50 employees. Academic research is free. Retention for traces is not stated on the page.

Strengths. OpenTelemetry works through a protobuf endpoint at trace.wandb.ai/otel/v1/traces with a wandb-api-key header, and the Python SDK ships local small-model scorers (toxicity, bias, hallucination, context relevance, coherence, fluency, a composite trust score and PII detection via Presidio). If your team already tracks model training in W&B, one workspace for experiments and LLM traces is a real convenience. Weaknesses. Ingestion is billed per MB, which is harder to forecast than per-trace pricing, local scorers are Python only, and I could not confirm dataset and CI details from the pages I fetched. Best for: ML teams already on W&B.

Helicone

Helicone began as a proxy that logs LLM requests: you point your OpenAI-compatible client at its AI Gateway, and requests appear in a dashboard with model fallbacks and unified billing. The repository is Apache-2.0 on GitHub.

Pricing (as of Nov 11, 2026): Hobby is free with 10,000 requests, 7 days of retention and one seat. Pro is $79 per month with unlimited seats and one month of retention. Team is $799 per month with five organizations, three months of retention, SOC-2 and HIPAA compliance. Enterprise is custom and lists on-prem deployment and SAML SSO. Every tier includes 1 GB of storage.

The caveat is ownership. Mintlify announced it acquired Helicone and said the product would continue in maintenance mode, with security updates, bug fixes and new models, and help migrating customers to another platform. Helicone's pricing page now says it is part of Experiential Labs, with details to come. I found no public statement about long-term roadmap or self-hosting commitments. The quick-start docs describe the proxy integration and do not mention OpenTelemetry. Best for: existing users who want request logging today; for new projects, we would pick an actively developed alternative. If gateway features are what you want, see the gateways roundup.

Datadog LLM Observability

Datadog's offering follows each request through prompts, retrieval steps, tool calls and agent decisions, using the same tracer as the rest of its APM stack. Auto-instrumentation covers OpenAI, Anthropic, Gemini, Vertex AI, Bedrock, LangChain, CrewAI, Pydantic, LiteLLM and Strands Agents, with SDKs for Python, Node.js and Java.

Pricing (as of Nov 11, 2026, from the product page): Free is $0 for up to 40K LLM spans with 15-day retention. Pro is $160 per month for up to 100K LLM spans, with usage beyond that billed on demand. Only LLM spans are billed, defined as a single call to an LLM provider; tool, workflow, agent, embedding and retrieval spans are free. Retention add-ons extend traces to 30, 60 or 90 days. Evals have no separate fee, but any LLM calls an eval makes count as LLM spans. I could not retrieve the detailed rate card on the pricing page, so confirm with Datadog before budgeting.

Strengths. Built-in and custom evaluators, offline and online evals, human review, detection of hallucinations, prompt injection and PII, datasets from production traces and experiments that compare prompts and models. Agent runs correlate with APM services, infrastructure and RUM sessions, with monitors for latency, cost and quality. Custom frameworks and other languages can be instrumented through OpenTelemetry or the HTTP API. Weaknesses. It is SaaS only, it is the least useful if you are not already a Datadog customer, and retention is short by default. Best for: teams that want LLM telemetry beside existing infrastructure alerts.

Galileo (Splunk Agent Observability)

Galileo built evaluation and guardrails for agents. Splunk's acquisitions page states that Cisco has completed its acquisition of Galileo and that Galileo is now Splunk Agent Observability, which helps teams evaluate and improve agent behavior, track usage and cost, and apply real-time guardrails to all traffic. Press coverage dates the announcement to April 9, 2026 and describes Luna-2 as Galileo's family of small language models for always-on evaluation. The galileo.ai/pricing URL now redirects to Splunk's pricing page.

Pricing (as of Nov 11, 2026): the Splunk page lists Agent Observability starting at $100 per month per 1.2 million spans, billed annually, with 500 million Luna tokens and five Luna fine-tuning runs per year included, extended retention available, and availability in Observability Cloud and self-hosted deployments. The page does not mention Galileo by name and does not say what happens to the old free tier or existing Galileo contracts. Strengths: purpose-built small models for evaluation and guardrails, and the Splunk and Cisco backing for enterprise rollouts. Weaknesses:annual billing, an unclear migration story for former Galileo users, and no OpenTelemetry documentation in the pages I fetched. Best for: enterprises standardizing on Splunk Observability Cloud.

Opik (Comet)

Opik is Comet's open-source LLM evaluation and tracing platform. The repository is Apache-2.0 and very active, and the pricing page offers unlimited spans when you self-host.

Pricing (as of Nov 11, 2026): Open Source is $0 self-hosted with unlimited spans and retention. Free Cloud is $0 with 25k spans per month, 60 days of retention and up to ten team members. Pro Cloud is $19 per month with 100k spans, up to 50 team members, $5 per 100k additional spans and extended retention at $29 per 100k spans (60 to 400 days). Enterprise is custom and adds SSO, SLAs and compliance certifications.

Strengths. The cheapest paid cloud entry in this roundup. The evaluation docs describe Test Suites with natural-language assertions checked by an LLM judge, more than 30 pre-built metrics such as hallucination, relevance and coherence, side-by-side experiment tracking and annotation queues. OpenTelemetry works over OTLP/HTTP at /api/v1/private/otel. Weaknesses. The gRPC exporter causes errors, traces without a root span do not appear in the Traces tab, and I did not confirm pytest or online evaluation rules from the page I fetched. Best for: small teams that want an open, cheap option with a hosted tier.

Laminar

Laminar is an open-source observability platform aimed at agents. Its docs list tracing for the AI SDK, Claude Agent SDK, LangChain and Browser Use, and the repository is Apache-2.0.

Pricing (as of Nov 11, 2026): Free is $0 with 1 GB of data, 7 days of retention, one project and one seat, and no overage. Starter is $30 per month with 3 GB, $2 per GB overage and 30 days. Pro is $150 per month with 10 GB, $1.50 per GB overage and 6 months of retention. Starter and above get unlimited projects and seats. Enterprise adds on-premise deployment. Billing is by data volume, which suits agents with long traces but is different from per-span pricing.

Strengths. Laminar's backend accepts OTLP over gRPC, the recommended path, at api.lmnr.ai:8443. Its evaluation docs describe evaluate() locally or in CI, agent evaluations that run your agent in an isolated copy of its environment and score outcome and trajectory, datasets built from production traces, and a Debugger that records a run, replays it from cache up to the point under test, and verifies a fix with an eval. Weaknesses. A younger project with a smaller community than Langfuse or Phoenix, and the free tier keeps only 7 days. Best for: teams debugging long-running or browser-driven agents. Frameworks to instrument are covered in our agent frameworks roundup.

Promptfoo

Promptfoo is different from the others: it is a CLI and library for running evals and red teaming, and it runs completely locally. The docs say evals run on your machine and talk directly to the model provider. The repository is MIT-licensed.

Pricing (as of Nov 11, 2026): Community is free forever and includes all evaluation features, all providers, self-hosting or local runs, vulnerability scanning and red teaming capped at 10k probes per month. Enterprise is custom and adds team sharing, continuous monitoring, SSO, API access and managed cloud. On-Premise is custom and adds deployment on your own infrastructure and a dedicated runner. On March 9, 2026, Promptfoo said it agreed to be acquired by OpenAI, that the deal was subject to customary closing conditions, and that Promptfoo will remain open source. We did not confirm a closing date.

# promptfooconfig.yaml (pattern from the Promptfoo getting-started docs)
description: Automatic response evaluation using LLM rubric scoring
prompts:
  - file://prompts.txt
providers:
  - openai:gpt-6-sol
defaultTest:
  assert:
    - type: llm-rubric
      value: Do not mention that you are an AI or chat assistant
tests:
  - vars:
      name: Bob
      question: Can you help me find a specific product on your website?

Run it with npx promptfoo@latest eval and review results with npx promptfoo@latest view; results can also be written as JSON, YAML, HTML or a spreadsheet. For CI, the docs show a GitHub Actions workflow that runs the eval and fails the job when the failure count is above zero:

name: LLM Eval
on:
  pull_request:
    paths: ['prompts/**', 'promptfooconfig.yaml']
jobs:
  evaluate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '24' }
      - run: npx promptfoo@latest eval -c promptfooconfig.yaml -o results.json
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
      - run: |
          FAILURES=$(jq '.results.stats.failures' results.json)
          if [ "$FAILURES" -gt 0 ]; then exit 1; fi

Weaknesses. It is not a tracing backend, so you still need one of the tools above for production visibility. Best for: prompt regression tests and security testing of agents and RAG apps in CI. It pairs well with a coding agent that edits prompts; see our best AI coding agents guide.

OpenTelemetry Support Compared

OpenTelemetry (OTel) is the practical way to keep instrumentation portable. Most tools in this list now accept OTLP, but the transport, auth and attribute conventions differ, and those details decide whether a trace shows up cleanly. Langfuse and Opik take HTTP only. Laminar recommends gRPC. LangSmith maps langsmith.* and gen_ai.* attributes to its run fields, and Langfuse gives its own langfuse.* attributes precedence over generic GenAI conventions.

One pattern worth copying from the LangSmith and Langfuse docs is fanout through an OpenTelemetry Collector, so a single app sends the same spans to two backends during a migration. Keep the root span when you filter spans in a collector; both Langfuse and Opik warn that traces without a root span break or disappear.

Eval Features Compared

“Evals” covers four jobs: scoring live traces, running datasets before a release, collecting human labels, and gating CI. The table records what each vendor's docs page said; “not covered” means the page I fetched did not mention it, not that the feature does not exist.

If your agent also keeps long-term memory, evaluate recall quality separately; the agent memory tools roundup covers the options. For an example of a cloud vendor shipping evaluation as a platform feature, see our Gemini Enterprise agent evaluations guide.

Benchmarks and Independent Data

There is no neutral public benchmark that ranks these platforms. Tracing overhead, ingestion latency and eval accuracy depend on your SDK, sampling, payload size and judge model, and vendors do not publish comparable numbers. We did not run our own benchmark and do not publish any performance score. What we can verify is documented behavior, such as Langfuse's note that direct OTel data can lag up to ten minutes without the ingestion-version header, and LangSmith's note that ingestion is asynchronous. GitHub activity gave us a rough maintenance signal on November 12, 2026: Langfuse about 37.3k stars, Opik about 23.4k, Promptfoo about 27.1k, Phoenix about 12.3k, Helicone about 6.3k, Laminar about 3.5k and Weave about 1.1k. Stars measure attention, not quality, so treat them as context only.

How to Choose

A sensible rollout is three steps. First, instrument with OTel or the tool's SDK on one service and confirm that a full trace, including tool calls, appears. Second, turn 20 to 50 real production failures into a dataset and write two or three evaluators, at least one deterministic. Third, add a CI gate and an online sampled judge, and check judge scores against human labels before trusting them. If you also self-host a gateway for routing, the OmniRoute setup guide and the Vercel AI Gateway guide show where to attach logging. Hosting the agent itself is covered in best AI agent hosting platforms.

How We Evaluated

We fetched each vendor's pricing page, docs and repository on November 11, 2026, and used first-party sources for every price, limit and license. Where a page did not state something, we said so instead of filling the gap. Acquisition facts come from the companies' own posts where we could fetch them and from press coverage otherwise, and are labeled accordingly. There is no paid placement, affiliate link or sponsorship in this ranking. Prices change often, so confirm on the vendor page before you commit.

FAQ

Sources

All fetched on November 11, 2026 unless noted.

• LangSmith pricing
• LangSmith OpenTelemetry docs
• LangSmith evaluation docs
• LangSmith observability page
• Langfuse pricing
• Langfuse OpenTelemetry docs
• Langfuse evaluation overview
• Langfuse joins ClickHouse (Jan 16, 2026)
• Langfuse repository and LICENSE
• Braintrust pricing
• Braintrust OpenTelemetry docs
• Braintrust evaluate docs
• Arize pricing
• Phoenix docs
• Phoenix self-hosting docs
• Phoenix repository and LICENSE (Elastic License 2.0)
• W&B Weave pricing (CoreWeave Forge)
• Weave OpenTelemetry docs
• Weave local scorers docs
• Weave repository
• Helicone pricing
• Helicone quick start
• Mintlify acquires Helicone
• Helicone repository
• Datadog LLM Observability product page
• Splunk pricing (Agent Observability)
• Splunk: Cisco completes Galileo acquisition
• Opik pricing (Comet)
• Opik OpenTelemetry docs
• Opik evaluation docs
• Opik repository
• Laminar pricing
• Laminar docs overview
• Laminar OpenTelemetry docs
• Laminar repository
• Promptfoo pricing
• Promptfoo getting started
• Promptfoo CI/CD docs
• Promptfoo joining OpenAI (Mar 9, 2026)
• Promptfoo repository
• Langfuse walkthrough video (Langfuse YouTube)
• LangSmith overview video (LangChain YouTube)