RAG Explained for Business Owners: When to Use It and How

Retrieval-augmented generation, RAG, is the pattern behind most “ask our docs” and “talk to our product data” features that are not theatre. The model still writes the sentence. Your systems still own the facts. A retriever sits in the middle and fetches a few relevant passages so the model is not guessing from memory, and so you are not pasting the entire company wiki into every request.

If you are a founder or CTO in the US, UK, Europe or Australia, you do not need the embedding-math. You need to know when RAG is the right spend, what a vector database is for, and what a competent agency should deliver in what order. That is this guide. DayDecoder is a founder-led team in Ahmedabad; we implement RAG on Laravel and Node.js products as part of AI integration and automation, not as a science fair.

The problem RAG is solving

A large language model is trained on a wide slice of public text. It is fluent. It is not looking at your pricing PDF from Tuesday, your tenant’s tickets, or the clause you amended last month. If you ask it anyway, it will often answer. That answer may be wrong in a confident voice.

You could dump everything into the prompt. Context windows are larger than they were, and that trick still fails in three boring ways: it is expensive, it is slow, and it mixes documents the current user must not see with documents they should. RAG is the alternative: search first, generate second, show your working.

Think of a careful new hire. You would not ask them to memorise the company. You would point them at the right folder, ask for a draft, and check the citations. RAG is that, automated, with a much faster reader and a much weaker sense of embarrassment.

When to use RAG, and when not to

A useful test: if the correct answer can change when a document changes, and a citation would help a human trust it, you are in RAG territory. If the user just handed you the only document that matters, you are not.

How the pieces fit (without the mysticism)

1. Sources. Help centre articles, PDFs, tickets, CRM notes, database text fields you have a right to use.
2. Chunking. Split those sources into passages small enough to retrieve and large enough to make sense. Headings beat blind character windows when you have them.
3. Embeddings. A model turns each chunk into a list of numbers that represent meaning. “Reset password” and “cannot log in” land near each other. Invoice numbers often do not; that is why keyword search still matters.
4. Vector store. Those numbers live somewhere you can search quickly, tagged with tenant, source id, and ACL metadata.
5. Retriever. At question time, embed the question, filter by permission, return the nearest chunks, optionally rerank.
6. Generator. OpenAI, Claude, or another LLM writes the answer using those chunks and your rules.
7. UI. Show sources. Allow “I don’t know”. Send a transcript to a human when confidence is low or the user asks.

Your application database remains the system of record. The vector store is an index, like a search engine for meaning. If you treat it as a second source of truth, you will eventually serve an answer from a document you unpublished in the CMS but forgot to delete in the index. That bug is so common it should be on the acceptance tests.

Vector databases, in business language

You do not buy a vector database because it is fashionable. You buy a place to put embeddings next to metadata you can filter. The options you will hear on calls:

• Postgres with pgvector. Attractive when you already run Postgres, the corpus is not enormous, and you want one backup story. Many Laravel and Node.js products should start here.
• A dedicated vector database (Pinecone, Qdrant, Weaviate, and similar). Attractive when ingest volume, tenant count, or operational isolation says the search workload should not share a box with checkout.
• Search products with vector features (OpenSearch, Elasticsearch, and similar). Attractive when you already operate them and need hybrid keyword-plus-vector from day one.

The agency should explain backup, deletion, and region, not only “cosine similarity”. If they cannot say how a GDPR or CCPA deletion request removes a customer’s chunks, they are not production-ready. We keep the choice operational in how to integrate AI into a Laravel or Node.js app.

Metadata that is not optional

• Tenant id (and user id if documents are private to a person).
• Source type and source primary key, so you can re-ingest or delete.
• Permissions or roles, if not every member of the tenant may see every doc.
• Version or updated-at, so stale chunks lose.
• Language, if you serve more than one.

Filter on those fields in the query. Retrieving first and dropping rows afterwards is how logs and timing side-channels leak. Treat that as a security bug, the same way you would treat a missing WHERE tenant_id on a SQL statement.

A real implementation roadmap

This is a sequence we would run, and that we would want to see in someone else’s proposal. Times vary with corpus quality; they are not a promise. The shape is the point.

Phase 0: decide the job

One user, one question type, one corpus. “Our employees can ask how to do X, answered only from the handbook.” Write failure: what the UI says when nothing relevant is retrieved. Name the reviewer on your side. If this page takes more than an afternoon, the project is still a strategy conversation, not an engineering one.

Phase 1: make the corpus boring

Export or connect one source. Remove drafts, duplicates, and pages that contradict each other. RAG amplifies a messy wiki; it does not tidy it. Split PDFs that are actually scans; OCR is its own line item. Put the files in a bucket or database you control, not in a personal drive.

Phase 2: ingest and retrieve, no chatbot yet

Build the pipeline: chunk, embed, store, delete on unpublished. Then a debug UI: type a question, see the chunks, see scores. You should be able to tell whether retrieval is any good before you spend a token on a pretty answer. If “password reset” does not retrieve the password article, stop and fix chunking or hybrid search.

Phase 3: generate with citations

Add the model. Force it to cite source ids. If it cannot cite, it must refuse. Log prompt version, model, chunks, and output. Put a feature flag around it. This is the first time a non-engineer should touch the feature, on staging, with a script of ten questions.

Phase 4: evaluation as a product artefact

Freeze 30–100 questions with acceptable answers or acceptable refusals. Include traps: questions outside the corpus, questions that would cross tenants, questions that need a number from a table. Re-run the set when you change embeddings, chunk size, or the generator. This is how you notice a “better” model that is worse on your policy questions.

Phase 5: production hardening

Tenant tests, rate limits, token caps, PII rules, retention, a kill switch, handoff to a human. Only then a public or all-staff launch. Then, and only then, a second source type or a read-only tool (look up ticket status). Agents that write data are a different budget; see AI chatbot development cost in 2026.

What goes wrong (and how you notice)

• Wrong neighbour’s document. Missing tenant filter. Test this the way you test IDOR bugs.
• Stale answers. Ingest on create but not on update or unpublish. Monitor source updated-at versus chunk updated-at.
• Keyword blindness. Semantic search misses SKUs and error codes. Add hybrid search.
• Over-stuffed prompts. Top-k is too high. Latency and cost rise; quality often falls. Measure both.
• Prompt injection via a document. A PDF says “ignore previous instructions”. Treat retrieved text as untrusted data.
• The confident shrug. Retrieval failed but the model answered anyway. Require citations or refuse.

None of this needs a fabricated case study. You will see it on staging if you look. If an agency only demos happy questions, ask them to try a cross-tenant question and a question the corpus cannot answer, on the call.

What to ask an agency before you hire

• Where do chunks live, in which region, and who can query them?
• How does a deletion request remove vectors and logs?
• How are Laravel policies or your Node.js authorizers applied at retrieve time?
• What does the evaluation set look like, and who maintains it after launch?
• What is in the first production slice versus the “platform” slide?

DayDecoder has 10+ years shipping software for 103 customers. We still start RAG projects from one corpus and a citation, not from a multi-agent diagram. If you want that sequence on your product, send the brief to contact or email contact@daydecoder.com.

FAQ

Is RAG the same as “training our own model”?

No. Fine-tuning changes how a model writes. RAG changes what it is allowed to read at answer time. Most product assistants need RAG (and good content). Very few need a custom-trained model as the first step, and those that do still need retrieval for facts that change.

Will RAG stop hallucinations?

It reduces them when retrieval is relevant and the prompt requires citations or refusal. It does not eliminate them. Design the UI and the review path as if a wrong sentence can still appear. That is honest engineering, not pessimism.

Do we have to move off MySQL?

Not for the application data. You need a place to store embeddings. That can be Postgres with pgvector (sometimes by adding Postgres alongside MySQL), or a dedicated vector store. The app database stays the source of truth for orders, users, and permissions.

How long does a first RAG slice take?

When the corpus is one well-structured source and the job is narrow, teams often reach a staging assistant in a small number of sprints, then spend more time on evaluation and permissions than on the first demo. Messy PDFs, many tenants, and unclear owners stretch that. We would rather estimate after seeing the source than invent a calendar here.

Can we start with a vendor “chat with PDF” tool?

Yes, to learn the job with non-sensitive files. Be careful the moment the files are customer data or tenant-specific. Check isolation, retention, and whether you can export the work. Many teams prototype on a vendor and productionise RAG inside their own Laravel or Node.js app once the question is proven.