Langfuse vs LangSmith: the real split, and what all three still miss

You are picking the stack that will watch your agent in production, and the shortlist has settled to three names: Langfuse, LangSmith, and Braintrust. Every comparison you have read agrees on the surface. Three capable trace viewers, three eval runners, three dataset stores. The line that forks the decision is a single line in a license file, and it is the one your finance and security teams will ask about in month three: can we run this ourselves, and who holds the trace data?

Choose Langfuse when you want an open-source, self-hostable default: its core is MIT-licensed and stayed that way after ClickHouse acquired it in January 2026.¹ Choose LangSmith when your agent already lives in LangChain or LangGraph and you want a managed platform wired natively into both. Choose Braintrust when evaluation is your center of gravity and you will pay for the tightest dataset-scorer-experiment loop. All three record scores and run LLM judges. None of them ships the test that tells you whether a pass rate moved for real (a confidence interval on the rate, a paired significance test on the delta), so gate whichever you pick with a statistical eval. Skip that last step and you will ship a model swap because the number went from 82% to 85%, with no idea whether that is signal or the same run resampled.

Who owns the trace data: Langfuse is MIT, LangSmith and Braintrust self-host only on enterprise plans

Start with the fact that survives the next release of any of these tools. Langfuse ships an MIT-licensed core, and you can self-host every core capability (tracing, evaluations, prompt management, experiments, annotation, and the playground) at production scale for free. The only exception is a set of enterprise features in the repository’s ee folder, which carry a commercial license: SCIM, audit logs, data-retention policies, project-level RBAC, server-side data masking, and a handful of other governance modules.² When ClickHouse acquired Langfuse on January 16, 2026, the license did not move: the announcement commits that Langfuse “remains 100% open-source under its existing MIT license for core features which allows for self-hosting at production scale”, and Langfuse already ran on ClickHouse under the hood, which is why the acquisition changed the org chart and not the terms.¹ Read that commitment for what it is. It is the acquirer’s own statement about its own product, made in its own announcement, and no third party holds it to that. The part you can check yourself is the MIT grant already sitting on the released code, and an MIT grant on a version already published cannot be withdrawn from the copies it was published under. A future relicense would reach the next release, not the one you are running.

LangSmith sits on the other side of that line. It is a proprietary, closed-source SaaS built by the LangChain team, offered as managed cloud, hybrid, or self-hosted, and LangChain’s own documentation describes self-hosting as “an add-on to the Enterprise plan designed for our largest, most security-conscious customers”, unlocked by a license key their sales team issues.³ There is a second condition worth reading before you file self-hosting under full custody: LangChain’s deployment guide states that “LangSmith requires egress to https://beacon.langchain.com for license verification and usage reporting” and that this “is required for LangSmith to function properly”, with the prerequisite qualified as applying if you are not running in offline mode.³ Langfuse’s self-hosting page makes the plainer promise for the same job, that it “can be deployed within a VPC or on-premises in high-security environments” and that “Internet access is optional.”⁴

Braintrust, the AI-evals platform, is proprietary too, with a twist worth pricing in. Its client SDKs are open source (the JS/TS SDK ships under Apache-2.0) while the platform they talk to is closed, it runs on its own storage engine, Brainstore, and self-hosting is a hybrid deployment where you run the data plane holding “experiment logs, traces, datasets, and prompts” in your own cloud while Braintrust keeps the control plane, UI, and authentication.⁵⁶ On-premises deployment sits in the Enterprise tier.⁷

If a security review or a data-residency rule is going to land on this decision, it lands here, before anyone opens a single trace.

Langfuse, LangSmith, and Braintrust converge on tracing and evals more than their landing pages admit

Feature-by-feature, the gap is narrower than the marketing implies. All three ingest OpenTelemetry spans, so you are not locked into a proprietary tracer to get data in.⁸⁹¹⁰ All three run LLM-as-judge evaluators, store datasets, and support human review or annotation. Langfuse and LangSmith both carry prompt management with versioning; Braintrust centers its whole product on the eval loop instead. On what an eval actually scores versus what a trace records, all three draw the same line: the span proves the run finished, the eval scores whether the answer was right.

Because the wire format is OpenTelemetry, the same exporter points at any of the three by changing the endpoint and the headers. The header count is the detail worth checking against current docs rather than against a comparison post: as of 2026-08-02, LangSmith is the only one of the three whose documented OTLP setup takes a single header. Langfuse and Braintrust each want a second one, and it is not an auth header in either case.

# One OpenTelemetry exporter; repoint it per platform. Endpoints checked 2026-08-02;
# each vendor also publishes regional hosts on the same path, though which regions differ.
export OTEL_EXPORTER_OTLP_ENDPOINT="https://cloud.langfuse.com/api/public/otel"    # Langfuse
# export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.smith.langchain.com/otel"        # LangSmith
# export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.braintrust.dev/otel"             # Braintrust

# Langfuse:   Authorization=Basic <base64 of public:secret>, x-langfuse-ingestion-version=4
# LangSmith:  x-api-key=<langsmith api key>
# Braintrust: Authorization=Bearer <api key>, x-bt-parent=project_id:<project id>
export OTEL_EXPORTER_OTLP_HEADERS="<platform headers, per the lines above>"
# Only the exporter config moves; your application code stays put.

Coupling and SDK reach

Where they genuinely diverge is coupling and SDK reach. LangSmith is authored by the LangChain team, so it has the deepest native integration with LangChain and LangGraph: a LangChain or LangGraph app is traced with almost no instrumentation code, where the other two want an OpenTelemetry exporter wired up. It stays framework-agnostic through OpenTelemetry and ships SDKs for Python, TypeScript, Go, and Java.¹¹

The other two carry the SDK spread instead of the framework tie. Langfuse is OpenTelemetry-first and framework-neutral, with Python and JS/TS SDKs plus drop-in integrations for the OpenAI SDK, LiteLLM, LlamaIndex, and Haystack, so it treats LangChain as one integration among many.² Braintrust has the broadest SDK spread of the three (Python, TypeScript, Go, Ruby, Java, C#, and Kotlin) and the most opinionated core, built on its own storage engine, though its API opens that store back up: logs, experiments, and datasets are queryable with standard SQL through a /btql endpoint, with responses in JSON.⁶

The practical split is coupling: if you are deep in LangChain or LangGraph, LangSmith’s native advantage is worth the proprietary trade; if you want the option to leave, Langfuse’s OpenTelemetry-first, MIT stance keeps your data portable.

Braintrust vs Langfuse: the eval-first platform against the open-source default

Braintrust is a proprietary, eval-first platform; Langfuse is the MIT-licensed, self-hostable observability default. The two optimize for different jobs. Braintrust’s strongest surface is the dataset-scorer-experiment loop: if your daily work is defining scorers, running experiments over labeled sets, and comparing model versions on those scores, Braintrust’s loop is the most tightly integrated of the three. You pay for it two ways. The core is closed source, so you run on Braintrust’s proprietary engine rather than a stack you own, and portability rests on its API rather than on storage you control, though that API concedes more than the closed-core framing implies: logs, experiments, and datasets are queryable with SQL through a /btql endpoint.⁵⁶ And the pricing runs through a free Starter tier, a paid Pro tier, and custom Enterprise, metered on data processed, number of scores, retention, and model credits.⁷

Langfuse answers a different question. Its gravity is observability plus prompt management, delivered under an MIT license you can run yourself, which makes it the default when data ownership and freedom from lock-in outrank a marginally slicker eval UI; its evals are capable. Langfuse Cloud meters on usage units (traces, observations, and scores), with a free Hobby tier and free self-hosting; the catch on self-hosting is operational, since you now run and scale a ClickHouse deployment yourself.⁷

Where TruLens fits for RAG evaluation

If RAG evaluation is the specific job, a fourth open-source option belongs on the shortlist: TruLens. It sits in the same code-first, framework-neutral family as the harnesses teams are moving to as OpenAI Evals winds down, and it centers on feedback functions, the programmatic scorers you attach to a RAG chain. Its signature is the RAG triad: context relevance (did retrieval pull the right passages), groundedness (is the answer supported by them), and answer relevance (does it address the question). Developed by TruEra and now maintained by Snowflake in the open, it is MIT-licensed and composable, so you can run it alongside whichever tracer you pick.¹²

Its scope is the honest limit. TruLens is a focused evaluation library, lighter on production tracing, dashboards, and team workflows than Langfuse, LangSmith, or Braintrust. The common pattern is to pair it: one of the three platforms for tracing and storage, TruLens for RAG-triad scoring on top.

Langfuse, LangSmith, or Braintrust: which should you pick?

Three questions settle it: do you need to own and self-host your trace data, does your stack already run on LangChain or LangGraph, and is a tight scoring loop your daily work? If the answer to the first is no because your estate already runs a generalist APM, the trade against that incumbent prices out differently. If you are here because the tool you already had went into maintenance mode, moving off Helicone covers the gateway half of that migration before the platform half. And if the agent under test speaks rather than types, the shortlist is a different one: voice-agent eval platforms compared covers that market. The table maps each answer to a platform and the catch that rides with it.

That table routes the platform choice. It does not close the reliability question, because all three rows share the same blind spot.

What all three store, and what none of them computes

Each of these platforms will happily store a pass rate and draw it on a chart. When your eval suite reads 82% on Monday’s run and 85% on Tuesday’s, the dashboard shows a green up-arrow. What none of the three documents is the test that clears that three-point move against the noise: their eval docs describe score storage, side-by-side run diffs, and per-case regression flags, and none computes a confidence interval on a pass rate or a paired significance test on the delta for you.¹³ That is a statistics question their trace stores are not built to answer.

Answering it takes two things a logging tool does not ship:

1. A confidence interval on each pass rate, because a rate measured over 40 runs and one measured over 400 carry very different uncertainty, and the width of that interval is what the eval confidence interval captures. Our pass-rate confidence interval calculator computes the Wilson, Clopper-Pearson, and Wald versions side by side from a k-of-n count.
2. A paired test on the delta, because the two runs scored the same items, so the right question is whether the difference between them is statistically significant rather than whether two independent bars look different. For pass/fail scores on the same items that test is McNemar’s, and our McNemar calculator for paired eval runs takes the paired cells and returns the p-value with an interval on the difference.

A green eval number is only as honest as the test behind it, the same trap that lets a benchmark look healthy while measuring the wrong thing.

The layer above the dashboard is where reliability actually lives. A single pass rate samples reliability once; the lane’s name for measuring it properly is reliability@k, the suite-level aggregate of per-task pass^k that asks whether an agent clears the bar on all k attempts. Getting there means measuring agent reliability past a single pass rate, and trusting the judge that produced those scores in the first place, which is its own discipline once you read what LLM evals are and are not. Langfuse, LangSmith, and Braintrust are all good at recording the inputs to that analysis. They do not run the analysis.