Adding AI to a Laravel App: What Actually Works in Production

Over the last 18 months I've added AI features to four different Laravel projects. A customer support bot, an article summarizer, a smart search feature, and a local-running assistant using Ollama. Here's what I actually learned — not the demo version, the production version.

Start With a Service Class, Not a Controller

The worst AI integration I've seen was a 300-line controller method making OpenAI calls directly, with prompts hardcoded inline. When the API changed, nothing was findable. When costs spiked, there was no single place to add caching. Do yourself a favor and wrap it immediately:

// app/Services/AiService.php
class AiService
{
    public function __construct(
        private readonly OpenAI\Client $client
    ) {}

    public function complete(string $systemPrompt, string $userMessage): string
    {
        $response = $this->client->chat()->create([
            'model'    => 'gpt-4o-mini',
            'messages' => [
                ['role' => 'system', 'content' => $systemPrompt],
                ['role' => 'user',   'content' => $userMessage],
            ],
            'max_tokens' => 1024,
        ]);

        return $response->choices[0]->message->content ?? '';
    }
}

Bind it in a service provider and inject it wherever you need it. Now you have one place to add caching, logging, fallbacks, and cost tracking.

Streaming for Chat-Like UX

If you're building anything interactive — a chat interface, a writing assistant — you need streaming. Waiting 8 seconds for a full response feels broken even if it technically works:

public function streamResponse(string $message): StreamedResponse
{
    return response()->stream(function () use ($message) {
        $stream = $this->client->chat()->createStreamed([
            'model'    => 'gpt-4o-mini',
            'messages' => [['role' => 'user', 'content' => $message]],
        ]);

        foreach ($stream as $chunk) {
            $content = $chunk->choices[0]->delta->content;

            if ($content !== null) {
                echo 'data: ' . json_encode(['text' => $content]) . "\n\n";
                ob_flush();
                flush();
            }
        }

        echo "data: [DONE]\n\n";

    }, 200, ['Content-Type' => 'text/event-stream', 'Cache-Control' => 'no-cache']);
}

Cache Aggressively or Your Bill Will Surprise You

OpenAI charges per token. Identical prompts should hit your cache, not the API. I set a cache duration based on how dynamic the content is — a support FAQ answer can be cached for hours; a personalized recommendation can't.

public function summarize(string $content): string
{
    $key = 'ai_summary_' . md5($content);

    return Cache::remember($key, now()->addHours(24), function () use ($content) {
        return $this->complete(
            'Summarize the following in 2 sentences.',
            $content
        );
    });
}

Always Queue Long AI Tasks

An OpenAI API call can take 5–15 seconds. Never do that in a synchronous HTTP request unless you absolutely need the response immediately (like a streaming chat). For background tasks — generating summaries, embeddings, reports — use a job:

class GenerateSummaryJob implements ShouldQueue
{
    use Queueable;

    public int $timeout = 90;
    public array $backoff = [10, 30, 60];

    public function handle(AiService $ai): void
    {
        $summary = $ai->summarize($this->article->content);
        $this->article->update(['ai_summary' => $summary]);
    }
}

Running Local LLMs with Ollama

For one project where the client had strict data privacy requirements, I ran Llama 3.2 locally via Ollama. The response quality is obviously behind GPT-4o, but for basic summarization and classification tasks it worked fine — and zero data leaves the server:

class OllamaService
{
    public function generate(string $prompt, string $model = 'llama3.2'): string
    {
        $response = Http::timeout(60)
            ->post('http://localhost:11434/api/generate', [
                'model'  => $model,
                'prompt' => $prompt,
                'stream' => false,
            ])
            ->throw()
            ->json();

        return $response['response'] ?? '';
    }
}

Ollama is impressively easy to set up on Linux — one command to install, one command to pull a model. The performance on a machine without a GPU is slow but acceptable for background jobs.

Rate Limit Per User

Don't let one user burn through your OpenAI budget. Set per-user rate limits:

RateLimiter::for('ai', function (Request $request) {
    return Limit::perDay(100)->by($request->user()->id);
});

What Didn't Work

Prompt chaining for complex workflows — it worked in testing and fell apart in production when the model returned a slightly different format than expected. Now I use strict JSON output format and validate the schema before using the response. That alone saved me a lot of debugging.