How to Reduce LLM API Costs in a SaaS Product (2026 Guide)

Summary

Once an AI feature leaves the prototype stage, the LLM API bill often becomes one of the fastest-growing lines in a SaaS budget. The good news is that most of that spend is avoidable. Teams usually pay for the same system prompt on every request, regenerate answers they have already produced, send simple tasks to their most expensive model and let responses run longer than anyone needs.

This guide is for CTOs, product owners and developers running AI features in production. It walks through the levers that actually move the bill, in the order we recommend applying them: measuring cost per feature, provider prompt caching, application caching, model routing, output control, batch processing, and smarter retrieval for RAG. Each step includes practical notes for Laravel and Node.js codebases.

Why LLM costs grow faster than your user base

LLM providers charge per token, separately for input and output, and output tokens are usually priced higher than input tokens. That pricing model creates a few patterns that catch teams out:

• Prompts grow quietly. Every new instruction, tool definition, few-shot example and retrieved document is added to every request. A prompt that started at a few hundred tokens can reach many thousands within a few releases.
• Conversation history compounds. Chat features resend the whole conversation on every turn, so long sessions cost far more per message than short ones.
• Agents multiply calls. An AI agent that plans, calls tools and reflects may make several model calls to answer a single user request.
• Retries and failures are billed too. A response that fails validation and is retried costs twice.

None of this is visible unless you measure it, which is why measurement comes first.

Step 1: Measure cost per feature, not per month

A single monthly invoice tells you nothing about where to act. Every provider returns token usage with each response, so log it alongside the feature, tenant and model that generated it. A week of this data usually shows that one or two features account for most of the spend.

In Laravel, a small table and a helper are enough to start:

// database/migrations/xxxx_create_llm_usage_table.php
Schema::create('llm_usage', function (Blueprint $table) {
    $table->id();
    $table->string('feature');          // e.g. "support_reply", "invoice_extract"
    $table->foreignId('tenant_id')->nullable();
    $table->string('model');
    $table->unsignedInteger('input_tokens');
    $table->unsignedInteger('cached_input_tokens')->default(0);
    $table->unsignedInteger('output_tokens');
    $table->boolean('succeeded')->default(true);
    $table->timestamps();
});

// app/Support/LlmUsage.php
class LlmUsage
{
    public static function record(string $feature, string $model, array $usage, ?int $tenantId = null, bool $ok = true): void
    {
        DB::table('llm_usage')->insert([
            'feature' => $feature,
            'tenant_id' => $tenantId,
            'model' => $model,
            'input_tokens' => $usage['input_tokens'] ?? 0,
            'cached_input_tokens' => $usage['cached_tokens'] ?? 0,
            'output_tokens' => $usage['output_tokens'] ?? 0,
            'succeeded' => $ok,
            'created_at' => now(),
            'updated_at' => now(),
        ]);
    }
}

The field names for usage differ between providers, so map them in one place. Track three numbers per feature: cost per successful outcome (not per call), cache-hit rate and escalation rate if you route between models. Cost per successful outcome matters because a cheap call that fails and has to be retried is not cheap.

Step 2: Structure prompts for provider prompt caching

OpenAI, Anthropic and Google all offer prompt caching: when the beginning of a request matches a recent request, the repeated prefix is billed at a discounted rate and is usually processed faster. Some providers apply it automatically above a minimum prompt length, others need you to mark cache breakpoints, and some charge extra to write the cache. Rates, minimum lengths and expiry times change, so check the current documentation for your provider and model.

The engineering rule is the same everywhere: the cache matches from the start of the prompt, so the first changed token ends the reusable part. Order every prompt from the content that changes least to the content that changes most:

1. System instructions
2. Tool or function definitions, always in the same order
3. Few-shot examples
4. Long reference material shared across users
5. Conversation history
6. The new user message, plus timestamps, request IDs and user names

Common mistakes that silently break caching include putting the current date or the user's name in the system prompt, building tool schemas from a map whose key order is not guaranteed, and adding tenant-specific text near the top of an otherwise shared prompt. In Node.js, a deterministic serializer avoids the ordering problem:

// Stable JSON for tool schemas so the prompt prefix is byte-identical
function stableStringify(value) {
  if (Array.isArray(value)) return `[${value.map(stableStringify).join(',')}]`;
  if (value && typeof value === 'object') {
    return `{${Object.keys(value).sort()
      .map((k) => `${JSON.stringify(k)}:${stableStringify(value[k])}`)
      .join(',')}}`;
  }
  return JSON.stringify(value);
}

Two honest caveats. Prompt caching only discounts input tokens, so it does nothing for long responses. And it only helps when the same prefix is reused before the cache expires, so a low-traffic feature with long gaps between calls may see little benefit.

Step 3: Cache answers in your own application

Prompt caching makes repeated input cheaper. Application caching removes the model call entirely, which saves output tokens too. There are two kinds.

Exact-match caching

If the same input always deserves the same output, hash the normalised input and cache the result. This suits tasks like categorising a product, summarising a static document, translating fixed UI text or extracting fields from an uploaded file that will not change.

$key = 'llm:product_category:v3:' . hash('sha256', mb_strtolower(trim($description)));

$category = Cache::remember($key, now()->addDays(30), function () use ($description) {
    return app(ProductClassifier::class)->classify($description);
});

Put a version number in the key (the v3 above) and bump it whenever you change the prompt or model, so stale answers are not served after an upgrade.

Semantic caching

A semantic cache embeds the incoming question, looks for a previously answered question that is close enough, and returns that answer. It can work well for support assistants and FAQ-style features where many users ask the same thing in different words. It is also the layer most likely to cause real damage, so treat it carefully:

• Namespace per tenant. If an answer depends on a customer's data, it must never be served to another customer.
• Only cache answers that do not depend on the user. Anything that depends on permissions, account state or today's date is not cacheable.
• Start with a strict similarity threshold and loosen it only after reviewing real matches.
• Expire entries when the underlying knowledge changes, for example when you re-index documents.

If you already run PostgreSQL, the pgvector extension is enough to store embeddings for a semantic cache without adding a new database.

Step 4: Route each task to the cheapest model that can do it

Most production traffic does not need your most capable model. Classification, extraction, short rewrites, tagging and intent detection usually run well on a small, fast model that costs a fraction of a flagship model. Keep the large model for multi-step reasoning, long-context synthesis and tasks where quality directly affects revenue.

Start with explicit rules rather than a learned router. The calling feature already knows what kind of task it is, so route on that, and escalate only when the small model's output fails a check:

const ROUTES = {
  classify: { model: process.env.LLM_SMALL_MODEL },
  extract:  { model: process.env.LLM_SMALL_MODEL, escalateTo: process.env.LLM_LARGE_MODEL },
  draft:    { model: process.env.LLM_MEDIUM_MODEL },
  reason:   { model: process.env.LLM_LARGE_MODEL },
};

async function runTask(task, input, { validate, callModel, recordUsage }) {
  const route = ROUTES[task];
  const first = await callModel(route.model, input);
  recordUsage(task, route.model, first.usage);

  if (!route.escalateTo || validate(first.output)) return first.output;

  // Pay for the large model only on the hard cases
  const second = await callModel(route.escalateTo, input);
  recordUsage(task, route.escalateTo, second.usage, { escalated: true });
  return second.output;
}

Validation can be a JSON schema check, a required-field check, a confidence score the model returns, or a business rule. Keep an evaluation set of real examples for every routed task and re-run it whenever you change a route, a prompt or a model version. A cheaper route that quietly lowers quality is not a saving. Watch the escalation rate too: if it climbs, the prompt or the cheap model needs attention.

Step 5: Control output length and format

Because output tokens usually cost more than input tokens, verbose answers are expensive. A few simple controls help:

• Set a maximum output token limit per feature, based on what the feature actually needs.
• When the output is read by code rather than a person, ask for structured JSON using the provider's structured output or JSON schema mode. It is shorter than prose and removes the need for retries caused by parsing errors.
• Tell the model the length you want ("answer in two sentences", "return only the fields below").
• For reasoning models, use the lowest reasoning effort setting that passes your evaluation set. Reasoning tokens are typically billed as output even when you do not see them.

Step 6: Move non-urgent work to batch processing

Not every AI job needs an instant answer. Nightly report summaries, bulk product description generation, back-filling embeddings and document classification can all wait minutes or hours. OpenAI, Anthropic and Google offer batch APIs that process requests asynchronously at a discounted rate compared with real-time calls; check each provider's current pricing and turnaround window.

In Laravel this maps naturally onto queued jobs: collect requests in a table, submit them as one batch from a scheduled command, then poll for completion in a follow-up job and write results back. In Node.js the same pattern works with BullMQ or any job queue you already use. Keep real-time calls for the parts of the product a user is actively waiting on.

Step 7: Send less context in RAG features

Retrieval-augmented generation (RAG) features are often the biggest consumers of input tokens, because they paste retrieved documents into every prompt. Better retrieval means fewer, more relevant tokens:

• Retrieve fewer, better chunks. Add a re-ranking step and send only the top results instead of a fixed large number.
• Chunk sensibly. Very large chunks carry a lot of irrelevant text into the prompt; very small ones lose meaning. Test chunk sizes against your evaluation set.
• Filter before searching. Use metadata such as tenant, product or document type to narrow the search, which improves both relevance and cost.
• Summarise long conversation history instead of resending every message, and keep the summary in a stable position so it can still benefit from prompt caching.

We cover the architecture side in more detail in our guide to RAG and LangChain in production.

Step 8: Add budget guardrails

Even a well-optimised system needs limits, because a bug, a looping agent or an abusive user can burn through a month's budget in hours.

• Set per-tenant and per-user quotas, and enforce them before the model call, not after.
• Cap the number of steps an agent can take per request.
• Rate-limit AI endpoints separately from the rest of your API. In Laravel, RateLimiter::for() with a dedicated limiter works well.
• Alert when daily spend for a feature exceeds its normal range, using the usage table from Step 1.
• Set spending limits in each provider's dashboard as a final safety net.

Quotas also help with pricing: once you know the cost per successful outcome for each feature, you can decide which AI features belong in which plan tier, or whether to sell them as usage-based add-ons.

A practical rollout order

Applying everything at once makes it impossible to see what worked. This order tends to give the biggest savings for the least risk:

1. Log tokens per feature for a week to get a baseline.
2. Reorder prompts for prefix caching and enable it where needed.
3. Cap output length and switch machine-read output to structured JSON.
4. Add exact-match caching for deterministic, non-personal tasks.
5. Introduce model routing with an evaluation set for each route.
6. Move non-urgent jobs to batch APIs.
7. Tune RAG retrieval, then consider semantic caching last.

Re-measure after each step. The goal is a lower cost per successful outcome with quality that is the same or better, not just a smaller invoice.

FAQ

What is the quickest way to reduce LLM API costs?

For most products it is restructuring prompts so the stable parts come first and benefit from provider prompt caching, combined with output length limits. Both are low risk and do not change which model answers.

Does prompt caching change the model's answers?

No. Provider prompt caching only reuses the processing of an identical prompt prefix. The model still generates a fresh response, so quality is unaffected.

Is it safe to use a semantic cache in a multi-tenant SaaS product?

It can be, if every cache entry is namespaced by tenant and you only cache answers that do not depend on user-specific data or permissions. Without those rules it can leak one customer's information to another.

Should we self-host an open-source model to save money?

Sometimes. Self-hosting can make sense at high, steady volume or when data must stay in your own infrastructure, but you take on GPU costs, scaling and maintenance. Most teams should optimise their API usage first and compare total cost afterwards.

How do we know a cheaper model is good enough?

Build an evaluation set of real inputs with expected outputs for each task, run both models against it and compare pass rates. Re-run it whenever a prompt, route or model version changes.

Can these techniques be added to an existing Laravel or Node.js app?

Yes. Usage logging, prompt restructuring, caching, routing and quotas all sit in the layer that calls the model, so they can usually be added without rewriting the rest of the application.