How Real Products Use RAG and LangChain in Production (2026)

Retrieval-augmented generation (RAG) and LangChain are no longer weekend experiments. Fintechs, social networks, security vendors and logistics companies now run them in front of real users and real data. The gap between a demo that answers ten test questions and a system that serves thousands of people every day is large, though. This guide looks at publicly documented production systems, the architecture they share, how to choose a vector database, where the money goes, and the pitfalls that sink most first attempts. It finishes with how we build these systems at DayDecoder using Laravel, Node.js and Python.

If you are new to the idea itself, start with our plain-English primer, RAG explained for business owners, then come back here for the engineering view.

Quick definitions: RAG, LangChain and LangGraph

RAG is a pattern, not a product. Before the language model answers, your application retrieves relevant pieces of your own data (documents, tickets, database rows, schemas) and puts them into the prompt. The model then answers from that context instead of relying only on what it memorised during training. This is how you get answers about your refund policy, your product catalogue or your data warehouse.

LangChain is an open-source framework (Python and JavaScript/TypeScript) that provides building blocks for this: document loaders, text splitters, embedding and vector store integrations, retrievers, and model wrappers for OpenAI, Anthropic, Google, open-weight models and others.

LangGraph is LangChain's library for building stateful, multi-step agents as graphs: nodes do work (retrieve, call a tool, ask the model, check permissions) and edges decide what happens next. LangSmith is the companion platform for tracing, testing and evaluating these systems. In production, the three are often used together.

Real products using RAG and LangChain in production

The examples below are drawn from the companies' own engineering blogs and LangChain's published case studies. We only include what has been stated publicly.

Klarna: customer support assistant

Klarna's AI Assistant handles customer tasks such as payments, refunds and payment escalations. According to LangChain's published case study, it is built on LangGraph and refined with LangSmith, serves a platform of around 85 million active users, cut average customer query resolution time by 80% and automated roughly 70% of repetitive support tasks. Two engineering choices stand out: a controllable agent architecture that routes each request to a specialised path, and prompts tailored dynamically to the scenario so the model only receives the context it needs, which reduces both token cost and latency.

Lesson: do not build one giant prompt that knows everything. Route first, then retrieve narrowly.

LinkedIn: SQL Bot for internal analytics

LinkedIn's engineering blog describes SQL Bot, an assistant inside its internal DARWIN data science platform that turns natural-language questions into SQL. It finds the right tables, writes the query, fixes errors and respects the user's permissions. LinkedIn describes it as a multi-agent system built on LangChain and LangGraph, used by hundreds of employees. Two details are especially useful for anyone building something similar:

• Retrieval over metadata, not just documents. The hard part was finding authoritative tables in a huge warehouse, so retrieval works over table descriptions, schemas and usage signals.
• Integration beats a standalone chat app. Embedding the bot directly in the tool people already use increased adoption 5–10x compared with the earlier standalone prototype, according to LinkedIn.

Elastic: security AI Assistant

Elastic launched an AI Assistant for its security product early in the generative-AI wave. LangChain has reported that it started on LangChain and later migrated to LangGraph as features grew, and that Elastic uses LangGraph to orchestrate agents for threat-detection scenarios, reducing labour-intensive SecOps work. Here, retrieval pulls in alerts, detection rules and knowledge-base content so analysts get answers grounded in their own environment.

Others with published case studies

LangChain's public case-study list also includes Replit (coding agent), Uber (developer productivity and code migration), GitLab, Vodafone (code generation and internal search), C.H. Robinson (logistics automation), and financial firms such as BlackRock and J.P. Morgan presenting domain-specific copilots. The pattern repeats: retrieval over company-specific knowledge, plus an orchestrated workflow around the model.

The production RAG architecture these systems share

Strip away company specifics and most production systems look like this:

1. Ingestion pipeline: connectors pull data from documents, help centres, databases, tickets or schemas. Content is cleaned, split into chunks (typically a few hundred tokens with overlap), enriched with metadata such as source, date, tenant and access level, and embedded.
2. Storage: embeddings plus metadata in a vector store, often alongside a keyword index for hybrid search.
3. Query routing: a lightweight classifier or graph node decides what kind of question this is and which data and tools are relevant.
4. Retrieval and re-ranking: hybrid search (vector + keyword) returns candidates, a re-ranker orders them, and permission filters remove anything the user is not allowed to see.
5. Generation with citations: the model answers from the retrieved context and cites sources.
6. Guardrails and fallbacks: if confidence is low or the request involves money or personal data, the system asks a clarifying question or hands off to a human.
7. Observability and evaluation: every step is traced, and a fixed test set of real questions is re-run whenever prompts, models or chunking change.

Choosing a vector database

There is no single best choice. What matters is your existing stack, data volume and operations budget.

• PostgreSQL with pgvector: our default for most SaaS products. Your vectors live next to your relational data, so tenant filters, joins and backups stay simple. It works on AWS RDS/Aurora, Google Cloud SQL and AlloyDB, and Supabase.
• Managed vector services (for example Pinecone): useful when you want minimal operations work and expect large collections. Watch costs and data-residency requirements.
• Open-source engines (Qdrant, Weaviate, Milvus): strong filtering and performance, self-hosted or managed. Good when you need advanced hybrid search or very large scale.
• Elasticsearch or OpenSearch: sensible if you already run them for search, since both support vector and keyword search together.
• Redis: can serve vector search and is excellent as a semantic cache in front of the model.

Start with pgvector unless you have a specific reason not to. Migrating later is far easier than running an extra database you did not need.

Where the cost actually goes

Model prices change often, so treat any per-token figure you read as temporary. The cost structure, however, is stable:

• Generation tokens are usually the largest running cost. Long prompts stuffed with ten retrieved chunks get expensive fast. Better retrieval with fewer, more relevant chunks saves money and improves answers.
• Embeddings are cheap per document, but re-embedding everything each time you change the model or chunking strategy adds up. Version your embeddings.
• Infrastructure: vector storage, queues, workers and observability tools.
• Engineering and evaluation time: often the biggest cost overall. Building a good test set and iterating on retrieval quality takes real effort.

Practical savers: route simple questions to smaller, cheaper models; cache frequent answers; trim context; and batch ingestion jobs. For budgeting a full project, see our guide to AI chatbot development cost in 2026.

Common pitfalls (and how to avoid them)

1. Bad chunking

Splitting a policy document in the middle of a table or a numbered list destroys meaning. Chunk by structure (headings, sections, rows) and keep the title and section path as metadata.

2. Ignoring permissions

If retrieval does not filter by tenant and role, your assistant will eventually leak data across customers. Apply access filters inside the retrieval query, not after generation. LinkedIn's emphasis on "appropriate permissions" is a reminder that this is a first-class requirement.

3. Stale data

An assistant quoting last year's prices damages trust quickly. Use event-driven re-indexing (for example a queue job whenever a record changes) instead of nightly full rebuilds.

4. No evaluation set

Without 50–200 real questions with expected answers, every prompt tweak is guesswork. Build the set before launch and run it in CI.

5. Over-engineering agents too early

Start with a simple retrieve-then-answer chain. Add LangGraph branches, tools and multi-agent routing only when your evaluation results show a clear need.

6. Prompt injection from retrieved content

Documents and web pages can contain instructions aimed at your model. Treat retrieved text as data, restrict which tools the model can call, and require confirmation for actions that move money or change records.

How DayDecoder builds RAG with Laravel, Node.js and Python

Most of our clients already have a Laravel or Node.js product. We rarely rewrite it. Instead, we add AI as a service alongside it:

• Laravel or Node.js stays the system of record: authentication, tenants, billing, permissions and UI.
• A Python service (FastAPI with LangChain/LangGraph) handles ingestion, retrieval and orchestration, where the AI ecosystem is richest.
• PostgreSQL + pgvector stores embeddings with tenant and access metadata; Redis handles queues and semantic caching.
• Queues (Laravel Horizon or BullMQ) trigger re-indexing whenever content changes.
• Tracing and evaluation run from day one, on AWS or Google Cloud.

Here is a simplified Python retrieval service using LangChain with pgvector and a tenant filter:

from fastapi import FastAPI
from pydantic import BaseModel
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_postgres import PGVector
from langchain_core.prompts import ChatPromptTemplate

app = FastAPI()
store = PGVector(
    embeddings=OpenAIEmbeddings(model="text-embedding-3-small"),
    collection_name="kb",
    connection="postgresql+psycopg://app:secret@db:5432/app",
)
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer only from the context. If unsure, say so. Cite sources.\n\n{context}"),
    ("human", "{question}"),
])

class Ask(BaseModel):
    tenant_id: int
    question: str

@app.post("/ask")
def ask(req: Ask):
    docs = store.similarity_search(
        req.question, k=4, filter={"tenant_id": req.tenant_id}
    )
    context = "\n\n".join(f"[{d.metadata['source']}] {d.page_content}" for d in docs)
    answer = (prompt | llm).invoke({"context": context, "question": req.question})
    return {"answer": answer.content, "sources": [d.metadata["source"] for d in docs]}

And the Laravel side calls it with the tenant taken from the authenticated user, never from the request body:

$response = Http::timeout(30)
    ->withToken(config('services.ai.token'))
    ->post(config('services.ai.url').'/ask', [
        'tenant_id' => $request->user()->tenant_id,
        'question'  => $request->validated('question'),
    ]);

return response()->json($response->json());

For a deeper look at fitting AI into an existing product, read adding AI features to a Laravel or Node.js SaaS product, or see our AI integration and automation services.

Summary

• Companies such as Klarna, LinkedIn and Elastic have publicly described running LangChain and LangGraph-based systems in production.
• The winning pattern is routing, narrow and permission-aware retrieval, cited answers, and continuous evaluation.
• PostgreSQL with pgvector is a sensible default vector store for most SaaS products.
• Most cost sits in generation tokens and engineering time, so retrieval quality is also your best cost lever.
• Keep your Laravel or Node.js app as the system of record and add a focused Python AI service beside it.

Ready to put RAG into your product?

DayDecoder designs and builds production RAG systems, AI assistants and LangGraph agents for SaaS companies and businesses in the US, UK, Europe and Australia. We work with your existing Laravel, Node.js or Python stack on AWS or Google Cloud, with security and evaluation built in from the start. Talk to our team about your project and get a clear plan and estimate.

FAQ

Is LangChain used in real production products?

Yes. Klarna, LinkedIn and Elastic have publicly described production systems built on LangChain and LangGraph, and LangChain publishes case studies from companies including Replit, Uber, Vodafone and C.H. Robinson.

Do I need LangChain to build a RAG system?

No. RAG is a pattern you can build with plain SDK calls. LangChain and LangGraph speed up development with ready-made integrations, orchestration and tracing, which helps most when workflows become multi-step.

Which vector database should I use for RAG?

For most SaaS products, PostgreSQL with pgvector is a strong default because it keeps vectors next to your relational data and permissions. Managed services like Pinecone or engines like Qdrant and Weaviate suit very large or specialised workloads.

How do you stop a RAG assistant from leaking data between customers?

Store tenant and access metadata with every chunk and apply those filters inside the retrieval query, using the identity of the authenticated user rather than anything sent by the client.

Can RAG be added to an existing Laravel or Node.js application?

Yes. We usually keep Laravel or Node.js as the system of record and add a separate Python AI service for ingestion, retrieval and orchestration, connected through an authenticated internal API and queues.

How long does it take to build a production RAG system?

A focused first version over one or two data sources typically takes a few weeks, including an evaluation set. Larger multi-source or agentic systems take longer depending on integrations and security requirements.