How to Migrate Off LangChain Without a Rewrite

// In plain English

LangChain is a popular toolkit for building LLM apps. It bundles chains, prompt templates, tool wrappers, memory, retrieval, and dozens of integrations behind one set of abstractions. That bundle is great when you start — but as an app matures, teams often want to drop it: to cut a heavy dependency, to debug what's actually being sent to the model, or to control prompts and retries directly.

Migrating off LangChain means replacing those abstractions with code you own — usually the provider's own SDK (the anthropic or openai client) plus a few small helpers. The trap people fear is the big-bang rewrite: rip everything out and rebuild from scratch in one painful pull request. This article is about the opposite: a staged, low-risk path where the app keeps working the whole way.

Think of it like renovating a house while still living in it. You don't demolish all four walls at once. You redo one room, check that the plumbing and power still work, then move to the next. Each room is shippable on its own, and if something breaks you only have one room to look at — not the entire house.

// Why it matters

Most teams don't migrate for fashion. They migrate because a specific pain has become expensive. The usual reasons:

• Opaque prompts. A framework assembles the final prompt for you, so when an answer goes wrong it's hard to see the exact string the model received. Owning the prompt makes debugging concrete.
• Dependency weight and churn. A large framework pulls in many transitive packages and ships frequent breaking changes. Each upgrade can cost a day of fixing imports that moved or arguments that were renamed.
• Leaky abstractions. The moment you need behavior the abstraction didn't anticipate — a custom retry, a streaming detail, a provider-specific parameter — you end up fighting the framework instead of using it.
• Cost and latency control. Hidden extra calls (for routing, parsing, or memory) add tokens and milliseconds you can't easily see or tune.
• Hiring and onboarding. Raw SDK code reads like the provider's own docs. A new engineer can understand a direct API call faster than a deep chain of framework objects.

But the honest counterweight: sometimes you should stay. If LangChain is working, your team knows it, and you lean on its integrations or LangSmith tracing, migrating is pure cost with little upside. Migration earns its keep only when a real, recurring pain is bigger than the work to remove it. Decide why first — then the staged plan below keeps the how cheap.

// How a staged migration works

The core idea is the strangler pattern, borrowed from legacy-system migrations: wrap the old code behind your own interface, then quietly replace what's behind that interface one piece at a time. Callers never see the swap. You're never more than one small change away from a working app.

A migration moves through five stages. The first two add no risk at all — they're just measurement and insulation — and they're where most of the safety comes from.

1. Inventory — what do you actually use?

You almost certainly use a small fraction of LangChain. List every import and group it into four buckets: LLM calls, prompt templates, tools / function calling, and retrieval / vector store. Anything else (memory, output parsers, callbacks) is usually a thin layer you can reimplement in a few lines. The inventory tells you the real size of the job — it's normally far smaller than it feels.

2. Isolate — wrap the framework behind your own interface

Before changing any behavior, hide LangChain behind a thin interface that you define. Your app stops calling LangChain directly and calls your LLM and Retriever types instead. At this point the implementation still uses LangChain underneath — nothing functionally changed — but now there's exactly one place to swap each capability.

3–5. Replace, verify, drop

Now replace the implementation behind your interface one layer at a time — typically LLM calls first (the simplest), then retrieval, then tools. After each swap you run a golden test set (recorded inputs and known-good outputs) to confirm behavior didn't drift, ship it, and move on. When the last caller of LangChain is gone, you delete the import and remove the package. The dependency drops out only at the very end, when nothing references it.

// A worked example: replacing the LLM layer

Here's the isolate-then-replace pattern in concrete code. Step one defines an interface and a thin LangChain-backed implementation — no behavior change yet, just insulation:

from typing import Protocol
from langchain_anthropic import ChatAnthropic

class LLM(Protocol):
    def complete(self, system: str, user: str) -> str: ...

# Old implementation: still LangChain under the hood.
class LangChainLLM:
    def __init__(self):
        self._model = ChatAnthropic(model="claude-sonnet-4-6")

    def complete(self, system: str, user: str) -> str:
        msg = self._model.invoke(
            [("system", system), ("human", user)]
        )
        return msg.content

# Your app depends ONLY on the LLM interface now.
llm: LLM = LangChainLLM()

Step two writes a second implementation of the same interface using the native Anthropic SDK directly. The rest of your app doesn't change a line — it still calls llm.complete(...).

from anthropic import Anthropic

class NativeLLM:
    def __init__(self):
        self._client = Anthropic()  # reads ANTHROPIC_API_KEY

    def complete(self, system: str, user: str) -> str:
        msg = self._client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=1024,
            system=system,                       # system is its own param
            messages=[{"role": "user", "content": user}],
        )
        return msg.content[0].text

# The ONLY line that changes to switch implementations:
llm: LLM = NativeLLM()

That single-line swap is the whole point of stage 2. Because every caller went through the LLM interface, replacing LangChain with the native SDK touches one line and nothing else. Run your golden tests, ship, and the LLM layer is migrated. Then repeat the exact same move for retrieval and tools.

// What replaces each LangChain piece

Each LangChain capability has a plain-code or small-library equivalent. Seeing the mapping up front removes most of the fear — almost nothing here is hard.

The two genuinely substantive layers are tools and the agent loop. The rest is mostly deleting indirection. If your app is agent-shaped — a model deciding which tools to call in a loop — you may prefer a leaner purpose-built option over hand-rolling, such as the Claude Agent SDK, rather than going all the way to raw API calls. Migration doesn't have to mean zero framework; it can mean a better-fitting one.

// Common pitfalls (and how to avoid them)

Migrations go wrong in predictable ways. Almost every failure comes from skipping the measurement and insulation stages and jumping straight to ripping code out.

• The big-bang rewrite. Replacing everything in one branch means a huge, untestable diff and weeks with no shippable state. Always migrate one layer behind the interface instead.
• No golden tests. Without recorded input/output pairs you can't tell whether a swap changed behavior. Capture real cases before you touch anything — that test set is your safety net.
• Forgetting the hidden prompt. The framework injected wording into your prompts (formatting rules, tool instructions). Drop it naively and quality can dip. Log the exact final prompt LangChain sends, then reproduce it in your native version.
• Re-implementing the whole framework. It's tempting to rebuild a generic chain engine. Don't — you only need the narrow slice your app uses. Rebuilding LangChain badly is worse than keeping LangChain.
• Migrating with no reason. If you can't name the concrete pain, you're spending real effort for a cleaner import list. That's rarely worth it.

// Going deeper

Once the mechanical swap is done, a few subtler topics decide whether the migration actually pays off.

Don't lose observability. A common reason migrations regret-spiral is that the team gives up the tracing they had from LangSmith. Before you delete it, decide what replaces it — OpenTelemetry spans around each model call, or another LLM-observability tool. Going dark on traces right when you change your call layer is the worst possible time.

Partial migration is a valid end state. You don't have to reach zero LangChain. Many teams keep its loaders and integrations for one-off ingestion scripts while running their hot request path on native SDKs. Migrate the latency- and cost-sensitive code; leave the convenience code alone if it isn't hurting.

Migrating off ≠ migrating to nothing. Two endpoints are common. One is no framework at all — a deliberate choice with its own tradeoffs, covered in building an agent without a framework. The other is a different framework that fits better; how to choose an agent framework and the broader framework comparison help you avoid swapping one ill-fitting tool for another.

Beware LangChain vs LangGraph confusion. If your real problem is a tangled multi-step control flow, the answer might be staying in the ecosystem and moving to its graph-based sibling rather than leaving entirely — see LangChain vs LangGraph. Make sure you're solving the dependency problem, not mislabeling an architecture problem as one.

The durable lesson: a framework is a bet that its abstractions match your needs. When they stop matching, the cheapest exit is never a rewrite — it's an interface you control, swapped out one tested layer at a time, with the app shippable at every step.