How to Integrate AI (LLMs) into an Existing Laravel or Node.js App

If you already have a Laravel or Node.js product in production, “add AI” is not a feature request. It is an architecture change: a slow, sometimes unreliable model sitting next to a fast, transactional app, with new costs, new failure modes, and new ways to leak data. Founders and CTOs in the US, UK, Europe and Australia usually feel this first as a demo that looks magical, then as a production incident that looks obvious in hindsight.

This guide is the sequence we use at DayDecoder, a founder-led software company in Ahmedabad, when we add large language models such as OpenAI or Claude to an existing app. It is written for people who will hire an agency, not for people collecting framework trivia. The short version of our earlier technical note on AI in SaaS products is still true. This is the longer version: architecture, retrieval-augmented generation (RAG), cost control, and security, in the order you should actually decide them.

Start with a job, not a model

Pick one workflow with a measurable outcome: draft a support reply from the ticket and the last five comments; extract fields from an uploaded invoice; summarise a contract for an internal reviewer; answer a question only from the help centre. Write the input, the output, who reviews mistakes, and what happens when the model is wrong. If you cannot write those four lines, you are not ready to choose OpenAI, Claude, or anyone else.

Resist the urge to “put ChatGPT in the product”. A chat box with no retrieval, no audit log, and no handoff to a human is a liability. It also trains your team to judge the work by how fluent the prose is, which is the wrong test.

DayDecoder has spent 10+ years shipping web and mobile products for 103 customers. The pattern that survives contact with production is boring: a small, reversible slice, behind a flag, with logs you can read on Monday morning.

A one-page brief you can send an agency

• The user and the job (one sentence each).
• Where the source data lives today (MySQL, Postgres, S3, a helpdesk, Google Drive).
• Whether the answer may leave your cloud, and which region you need.
• The current stack: Laravel version or Node.js runtime, queue system, auth, tenancy model.
• Who owns the review desk on your side, even if that person is you.

If a vendor starts with a model comparison deck instead of that brief, they are selling a demo. A 30-minute call should be about your app. Write to contact@daydecoder.com if you want that call with us.

Architecture that does not fight Laravel or Node.js

Treat the LLM as an external API with tail latency, rate limits, and partial failures. Your existing HTTP workers are sized for database work. They are the wrong place to wait ten seconds for a completion.

The four runtime patterns

The default should be the background job. Chat is the exception, not the architecture. Agents are a later exception, after you can already retrieve the right paragraph and validate a JSON blob.

Keep a thin provider layer

Do not sprinkle HTTP calls to OpenAI or Anthropic through controllers. One module should own: API keys, timeouts, retries, token accounting, and the mapping from “our task” to “their request”. In Laravel that is typically an action or service plus a queued job. In Node.js it is a small client module plus a worker. Prompts belong in versioned templates (Blade views are a perfectly reasonable prompt store in Laravel), not in string concatenation spread across the codebase.

That layer is also where you pin model names. “Use the latest” is how a quiet provider change rewrites your invoices and your output format on the same day.

Store outcomes, not vibes

Every call should write a row you can query: tenant, user, feature flag, model, prompt version, token counts, latency, success or error class, and a pointer to the source records. You do not need a science project. You need to answer “what did we send, what did we get, and who saw it” when a customer forwards a bad answer to your CEO.

Keep the canonical business data in MySQL or Postgres. The model output is a suggestion until your code, or a human, accepts it. That one rule prevents a surprising number of “the AI overwrote the order” stories.

RAG, when you actually need it

Retrieval-augmented generation means: find the handful of passages that are relevant to this question, for this tenant, and put only those in the prompt. It is how you stop the model inventing policy, and how you stop sending the entire knowledge base on every call. If the task is “rewrite this paragraph in a calmer tone”, you do not need RAG. If the task is “answer from our docs / tickets / contracts”, you do. We walk through the business case at length in RAG explained for business owners.

The ingestion pipeline

1. On create or update of a source record (article, PDF, ticket, message), enqueue a job.
2. Extract text. For PDFs and scans this is its own problem; do not pretend a PDF is a string.
3. Chunk with overlap. Prefer structure (headings, pages, ticket comments) over a raw character window when you have it.
4. Embed the chunk with a model you pin, and store the vector with tenant id, source id, and ACL metadata.
5. Delete or refresh vectors when the source is deleted, unpublished, or permissions change. Stale vectors are a data leak with extra steps.

Postgres with pgvector is a sensible default when you already run Postgres and the corpus is not huge. Dedicated stores (Pinecone, Qdrant, Weaviate, and similar) are reasonable when you have many tenants, heavy write volume, or you do not want vector operations competing with checkout traffic. The product decision is operational, not aesthetic.

The retrieval pipeline

1. Embed the question with the same embedding model you used at ingest.
2. Filter by tenant and permission before similarity search, not after. “Search everything then drop other people’s rows” is how you leak a neighbour’s contract in a log line.
3. Take a small top-k. Five to fifteen chunks is a typical starting range; stuffing fifty is how you buy latency and hallucinations.
4. Optionally rerank. A cheap reranker often beats a larger generator with noisy context.
5. Build the prompt: system rules, the retrieved passages with citations, the user question, and the output schema.
6. Generate. Validate. Cite the source ids in the UI so a human can click through.

If the retriever returns nothing useful, say so. “I do not have that in the connected documents” is a better product than a confident paragraph. Hybrid search (keyword plus vectors) is worth adding once you see names, invoice numbers, and SKUs failing semantic search. That is a normal second iteration, not a reason to delay a first slice.

Cost control that finance will actually believe

Model prices move. Your architecture should assume they will. The bill that surprises people is rarely the first demo. It is the weekly job that embeds every ticket, the chat UI that sends the last 40 messages plus three PDFs, and the agent that retries a tool loop until the budget is gone.

Give finance two numbers, both labelled as typical and indicative rather than a quote: an engineering range for the first production slice, and a monthly model-usage envelope at a stated volume. Engineering for a focused integration into an existing Laravel or Node.js app is often a five-figure piece of work; complex RAG plus tools plus a review desk can run higher. Monthly model spend might be modest at hundreds of questions a day and painful at tens of thousands if you retrieve too much. We would rather under-promise on a call than invent a benchmark here. For chatbot-shaped work, see AI chatbot development cost in 2026.

Instrument tokens per feature, not as a single “AI” line. Product managers cannot cut what they cannot see.

Security is not a system prompt

A paragraph that says “never reveal secrets” will not stop a user from asking, and it will not stop your own code from putting an API key in the prompt “just for this test”. Design as if the model output will be shown to a hostile user, because sometimes it will.

A checklist we would not skip

• Tenancy: every embed, retrieve, log, and cache key includes the tenant. Test a cross-tenant question as if it were a login bug.
• Secrets: provider keys in the environment or a secret manager. Never in the repository, never in prompts, never in frontend bundles.
• PII: decide what may leave your region. Redact or tokenise when the task does not need the raw value (a support draft rarely needs a full card number).
• Auth for tools: the model does not get a god-mode API. Each tool runs as the current user, through the same policies as the UI.
• Allowlists: URLs, SQL, and shell are not toys. If you would not give a junior contractor root, do not give it to a completion.
• Output validation: JSON schema, Laravel validation rules, or Zod. Discard or quarantine anything that does not match.
• Prompt injection: retrieved documents and user tickets are untrusted. Treat them as data, wrap them clearly, and never let them change system instructions or tool policy.
• Human review: anything that sends email, moves money, changes access, or talks to a customer as “the company” needs a review path you can point to in a contract.
• Retention: how long you keep prompts and completions, and how a deletion request propagates to the vector store.

If you sell into the EU or the UK, write the data-flow diagram before the first sprint: where text goes, which subprocessors see it, and what the user agreement allows. US buyers with healthcare, education, or finance rules will ask the same questions in different acronyms. An agency that cannot draw that diagram is not ready to touch production data, wherever they sit.

A practical implementation sequence

1. Week zero. NDA if you need one, repo access, staging, and the one-page brief. Confirm region and data rules.
2. Spike (days, not months). One job, fake or sampled data, a provider client, logging, and a UI that shows uncertainty. No vector database yet unless the job clearly needs retrieval.
3. Production slice. Queue or stream, feature flag, token caps, tests for the validator, and a rollback that leaves the old product working.
4. RAG if required. Ingest one source type, filter by tenant, cite sources, and measure “answered from docs” versus “refused”.
5. Evaluation. A small set of frozen questions with acceptable answers. Re-run it when you change prompts, models, or chunking. This is not optional once customers see the feature.
6. Only then tools. Read-only tools first. Write actions later, with the same authorization as a human clicking the button.

That sequence is how you avoid a six-month “platform” that never ships a button. It is also how an offshore team in India can work like a small in-house team: overlap hours, demos you can click, and code you own in your git org. We describe that operating rhythm in working with an offshore software development company.

What “done” looks like for a first release

A first release is done when a real user can complete the job, a reviewer can see the source of an answer, you can turn the feature off without a deploy drama, and finance can see tokens for that feature alone. It is not done when a staging demo used the founder’s OpenAI key and a CSV on someone’s laptop.

Expect the prompt to change after launch. That is normal. Budget a short cycle of evaluation and prompt work rather than treating the model as a library you call once. The application around it (queues, tenancy, validation, UI) should be the stable part.

FAQ

Can we just call OpenAI from a Laravel controller?

You can, and it will work in a demo. In production it ties up PHP workers for the duration of the model call, which is often several seconds and sometimes much longer. Use a queue for work the user can wait for asynchronously, and a streaming endpoint for chat. Keep a timeout even then.

Do we need a vector database on day one?

Only if the model must answer from your private corpus. Classification, extraction, and rewriting often need none. When you do need retrieval, start with one source type and strict tenant filters. Postgres with pgvector is enough for many products; split the store out when operations demand it.

OpenAI or Claude?

Choose from a frozen evaluation set for your job, plus data-residency and procurement constraints, not from a social-media bake-off. Keep a provider layer so you can switch. Many teams use a smaller model for routing and a stronger one for the hard cases. We will say which we would try first on a call, after we have seen your brief.

How do we keep the monthly bill from exploding?

Cap retrieved context, cache embeddings, route easy tasks to smaller models, and put a per-tenant token ceiling in your own code. Log tokens per feature. The typical failure is unbounded chat history and nightly re-embedding, not the list price on a marketing page.

Will you work in our GitHub and our cloud?

That is the default we recommend. You should have the repo, staging, and an NDA if the data is sensitive. Custom code is yours once it is paid for. Email contact@daydecoder.com if you want that in writing before we see the codebase.