Google’s AI for Developers (2026)

How to Deploy Gemma 3 Locally

1. Assess your hardware (CPU, RAM, GPU VRAM) and select the right Gemma 3 variant (1B, 4B, 12B, or 27B).
2. Install Ollama on your machine and pull the target Gemma 3 model with ollama pull gemma3:4b.
3. Configure a custom Modelfile with your system prompt, temperature, and context window parameters.
4. Verify the local Ollama REST API is responding with a curl test request.
5. Build a Node.js Express backend with an SSE streaming /api/chat endpoint using the ollama npm package.
6. Create a React frontend that reads the SSE stream and renders tokens in real time.
7. Optimize performance by tuning quantization level, GPU layer offloading, and context window size.

Running a local LLM like Gemma 3 has become a realistic option for individual developers who need privacy, lower latency, zero per-token costs, and offline capability. This tutorial walks through the full pipeline: selecting the right Gemma 3 variant based on hardware constraints, deploying it locally using Ollama, building a Node.js backend that streams inference results, and wiring up a React frontend that renders tokens as they arrive.

Table of Contents

Why Local LLM Deployment Matters in 2026

Running a local LLM like Gemma 3 has become a realistic option for individual developers who need privacy, lower latency, zero per-token costs, and offline capability. Depending on cloud API endpoints introduces data residency concerns, unpredictable billing, and a hard requirement on network connectivity. Local deployment sidesteps all three, and the 2025-2026 generation of open-weight models has made the performance trade-off far less painful than it was even 18 months ago.

Google’s Gemma 3 family fits this shift well. Released with open weights under Google’s Gemma Terms of Use (review commercial use terms at ai.google.dev/gemma/terms before deploying), it spans four parameter counts (1B, 4B, 12B, and 27B), meaning developers can match a variant to their actual hardware rather than being locked into a single model size. The architecture brings sliding window attention, multimodal support, and expanded context lengths compared to its predecessor, Gemma 2.

This tutorial walks through the full pipeline: selecting the right Gemma 3 variant based on hardware constraints, deploying it locally using Ollama, building a Node.js backend that streams inference results, and wiring up a React frontend that renders tokens as they arrive. Every code example is copy-paste-safe and targets a JavaScript/Node.js stack.

Gemma 3 Model Variants: 1B, 4B, 12B, and 27B Parameter Trade-offs

Architecture Overview and What Changed in Gemma 3

Gemma 3 introduces sliding window attention, which limits the attention span of most layers to a fixed local window while reserving a subset of layers for full global attention. This hybrid approach keeps memory usage manageable at longer context lengths. The architecture also adds native multimodal support for image and text inputs across the 4B, 12B, and 27B variants (the 1B variant is text-only). Context lengths have jumped from Gemma 2’s 8K token limit to 128K tokens on the larger variants (the 1B supports 32K).

Source note: The specifications above are drawn from the official Gemma 3 model cards and technical report published by Google. Verify current details and any updates at ai.google.dev/gemma and the Gemma 3 model cards on Hugging Face. Specifications current as of early 2026.

Compared to Gemma 2, the Gemma 3 lineup fills out the lower end of the parameter spectrum (the 1B variant is new) while improving instruction-following quality and multilingual performance across the board.

Choosing the Right Variant for Your Use Case

1B: Edge and Embedded Use Cases

The smallest variant runs comfortably on CPU-only machines, including Raspberry Pi-class devices with 4GB of RAM (tested on Raspberry Pi 5; expect slower inference on earlier models). It handles text only and fits classification tasks, simple summarization, and tightly constrained environments where model size and power consumption matter more than output sophistication.

4B: The Developer Sweet Spot

For most developers, the 4B variant is the practical starting point. It requires roughly 8GB of RAM (at Q4_K_M quantization) and can run on CPU alone, though a consumer GPU pushes inference from the ~8-15 tok/s CPU range into the ~30-50 tok/s GPU range shown in the comparison table below. This is the right choice for code assistance, chatbots, and retrieval-augmented generation (RAG) pipelines where response quality needs to be adequate for single-turn code completions and short-form Q&A but not on par with the 27B or cloud-hosted models.

12B: Production-Quality Local Inference

At 12B parameters, output quality reaches a level suitable for complex reasoning, long-form content generation, and agentic workflows. Minimum hardware is 16GB of RAM (at Q4_K_M quantization) with a dedicated GPU carrying 8GB or more of VRAM recommended. CPU-only inference is technically possible but slow enough to be impractical for interactive use.

27B: Maximum Capability

Google’s largest open-weight Gemma 3 variant delivers near-cloud-quality output. It requires 32GB of RAM (at Q4_K_M quantization) and a GPU with 16GB or more of VRAM. Use this when hardware is not a constraint and the goal is to minimize the quality gap between local and API-based inference.

Comparison Table

¹ The 1B variant supports 32K context; the 4B, 12B, and 27B variants support 128K.

Token-per-second ranges reflect typical quantized (Q4_K_M) inference on consumer hardware. Actual numbers vary with prompt length, quantization level, and specific CPU/GPU model. RAM requirements assume Q4_K_M quantization; higher-precision quantizations require more.

Prerequisites and Environment Setup

Hardware Requirements Checklist

Before pulling any model, verify the target machine has sufficient resources. The comparison table above maps each variant to minimum RAM and recommended GPU. Storage requirements range from roughly 1GB for the 1B Q4 quantization to over 16GB for the 27B at Q8 quantization. A quick self-assessment: if the machine has 8GB of RAM and no dedicated GPU, start with the 4B variant. If it has 16GB of RAM and any NVIDIA or Apple Silicon GPU, the 12B variant is within reach.

Software Dependencies

The stack for this tutorial requires Node.js 20+ (LTS release) and Ollama 0.3+, which handles downloading and serving models locally. On the frontend, React 18+ scaffolded with Vite. You will also need three npm packages: ollama@0.5.x (official JavaScript client), express@4.x, and cors@2.x.

curl -fsSL https://ollama.com/install.sh | sh






ollama pull gemma3:4b


ollama list


ollama run gemma3:4b "Hello, confirm you are running locally."

The ollama list command should display gemma3:4b with its size and quantization level. The ollama run command starts an interactive session; receiving a coherent response confirms the model is loaded and functional.

Note: Verify exact available tags for Gemma 3 at ollama.com/library/gemma3 before pulling. Tag formats are case-sensitive and may change across Ollama versions.

Deploying Gemma 3 Locally with Ollama

Pulling and Running the Model

Ollama distributes models in GGUF format with multiple quantization levels. Quantization reduces model precision from full 16-bit floating point to lower bit-widths, trading a small amount of output quality for reduced memory usage and faster inference. The most common levels:

Q4_K_M (4-bit) gives the best balance of speed and quality for most local use cases. Q5_K_M (5-bit) trades slightly more memory for slightly better quality. Q8_0 (8-bit) stays closest to full precision without running the unquantized model.

ollama pull gemma3:4b-q4_K_M
ollama pull gemma3:4b-q8_0


ollama run gemma3:4b-q4_K_M --verbose


ollama run gemma3:4b /set parameter temperature 0.7
ollama run gemma3:4b /set parameter num_ctx 8192


ollama run gemma3:4b /set parameter num_gpu 20

The --verbose flag prints tokens-per-second and timing information, which is essential for benchmarking. The num_gpu parameter controls how many model layers are offloaded to the GPU; setting it lower than the total layer count enables partial offloading on machines with limited VRAM. To find the total layer count for a variant, run ollama show gemma3:4b --verbose.

Configuring a Custom Modelfile

A Modelfile lets developers lock in a system prompt, default generation parameters, and stop tokens so they do not need to pass them on every request.

FROM gemma3:4b-q4_K_M

PARAMETER temperature 0.7
PARAMETER num_ctx 8192
PARAMETER stop ""
PARAMETER top_p 0.9

SYSTEM """You are a senior software engineer assistant. You provide concise, 
accurate code examples and technical explanations. You always specify the 
language and framework version when giving code. If you are unsure about 
something, you say so explicitly."""

Save this as Modelfile and create the custom model:

ollama create gemma3-dev -f Modelfile
ollama run gemma3-dev "Explain the event loop in Node.js 20."

Verifying Deployment with the REST API

Before connecting Node.js, verify the Ollama HTTP API is responding correctly:

curl http://localhost:11434/api/generate -d '{
  "model": "gemma3-dev",
  "prompt": "What is server-sent events in one sentence?",
  "stream": false
}'

Expected response structure:

{
  "model": "gemma3-dev",
  "created_at": "",
  "response": "Server-Sent Events (SSE) is a standard that enables a server to push real-time updates to a client over a single HTTP connection.",
  "done": true,
  "total_duration": 1250000000,
  "eval_count": 28,
  "eval_duration": 980000000
}

The eval_count and eval_duration fields allow computing tokens per second. If this request returns a valid response, the model is ready for integration.

Local deployment sidesteps all three, and the 2025-2026 generation of open-weight models has made the performance trade-off far less painful than it was even 18 months ago.

Building the Node.js Backend API

Project Structure

The backend is a minimal Express server with a single inference route, middleware for validation and rate limiting, and the ollama npm package as the client. Initialize the project:

mkdir gemma3-backend && cd gemma3-backend
npm init -y
npm pkg set type=module
npm install express@4.18.2 ollama@0.5.13 cors@2.8.5

The npm pkg set type=module command adds "type": "module" to package.json, which is required because the server code uses ES module import syntax. Without this, Node.js will throw SyntaxError: Cannot use import statement in a module.

Creating the Inference Endpoint

The core endpoint accepts a messages array and streams the response back to the client using Server-Sent Events.

import express from 'express';
import { Ollama } from 'ollama';
import cors from 'cors';
import { validateChatRequest, rateLimit } from './middleware.js';

const app = express();


app.set('trust proxy', 1);

const ALLOWED_ORIGIN = process.env.ALLOWED_ORIGIN || 'http://localhost:5173';
const PORT = process.env.PORT || 3001;
const MODEL = process.env.GEMMA_MODEL || 'gemma3-dev';

const ollama = new Ollama({ host: 'http://localhost:11434' });

app.use(cors({ origin: ALLOWED_ORIGIN }));
app.use(express.json());
app.use(rateLimit(10));

app.post('/api/chat', validateChatRequest, async (req, res) => {
  const { messages } = req.body;

  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');

  try {
    const response = await ollama.chat({
      model: MODEL,
      messages,
      stream: true,
    });

    for await (const chunk of response) {
      const data = JSON.stringify({
        token: chunk.message.content,
        done: chunk.done,
      });
      res.write(`data: ${data}

`);

      if (chunk.done) {
        
        const trueTokens = chunk.eval_count ?? 0;
        const trueElapsed = (chunk.eval_duration ?? 0) / 1e9;
        const tokensPerSecond = trueElapsed > 0
          ? (trueTokens / trueElapsed).toFixed(1)
          : '0';

        const meta = JSON.stringify({
          done: true,
          totalTokens: trueTokens,
          tokensPerSecond,
          model: MODEL,
          elapsed: trueElapsed.toFixed(2),
        });
        res.write(`data: ${meta}

`);
      }
    }
  } catch (err) {
    
    console.error('[/api/chat] Inference error:', err);
    const errorMsg = err?.status === 404
      ? `Model not found: ${MODEL}. Run: ollama pull ${MODEL}`
      : 'Inference failed. Check server logs.';

    if (!res.headersSent) {
      res.status(500).json({ error: errorMsg });
    } else {
      res.write(`data: ${JSON.stringify({ error: errorMsg })}

`);
    }
  } finally {
    res.end();
  }
});

app.listen(PORT, () => console.log(`Backend running on http://localhost:${PORT}`));

The streaming loop iterates over each chunk from the Ollama client, writing each token as an SSE event. When the model signals completion (chunk.done === true), the final chunk includes eval_count (total output tokens) and eval_duration (inference time in nanoseconds). These are used to compute accurate tokens-per-second statistics in the metadata event, rather than relying on a manual chunk counter.

Adding Request Validation and Rate Limiting

Even on a local machine, simultaneous requests can cause GPU memory contention and degrade inference speed. Basic validation and rate limiting prevent runaway usage.

const MAX_MESSAGES = 100;
const MAX_TOTAL_CHARS = 32_000;
const MAX_MESSAGE_CHARS = 4_000;

export function validateChatRequest(req, res, next) {
  const { messages } = req.body;

  if (!messages || !Array.isArray(messages) || messages.length === 0) {
    return res.status(400).json({ error: 'messages must be a non-empty array.' });
  }

  if (messages.length > MAX_MESSAGES) {
    return res.status(400).json({ error: `Too many messages (max ${MAX_MESSAGES}).` });
  }

  let totalChars = 0;
  for (const msg of messages) {
    if (!['user', 'assistant', 'system'].includes(msg.role)) {
      return res.status(400).json({ error: `Invalid role: ${msg.role}` });
    }
    if (typeof msg.content !== 'string' || msg.content.length > MAX_MESSAGE_CHARS) {
      return res.status(400).json({
        error: `Message content must be a string under ${MAX_MESSAGE_CHARS} chars.`,
      });
    }
    totalChars += msg.content.length;
    if (totalChars > MAX_TOTAL_CHARS) {
      return res.status(400).json({
        error: `Total message content exceeds ${MAX_TOTAL_CHARS} chars.`,
      });
    }
  }
  next();
}

const requestLog = new Map();

let prunerHandle;

function startPruner() {
  prunerHandle = setInterval(() => {
    const now = Date.now();
    for (const [ip, times] of requestLog) {
      const fresh = times.filter(t => now - t < 60_000);
      if (fresh.length === 0) requestLog.delete(ip);
      else requestLog.set(ip, fresh);
    }
  }, 60_000);
  
  prunerHandle.unref?.();
  return prunerHandle;
}

export function rateLimit(maxPerMinute = 10) {
  return (req, res, next) => {
    const ip = req.ip || 'unknown';
    const now = Date.now();
    const entries = (requestLog.get(ip) || []).filter(t => now - t < 60_000);
    if (entries.length >= maxPerMinute) {
      return res.status(429).json({ error: 'Rate limit exceeded.' });
    }
    entries.push(now);
    requestLog.set(ip, entries);
    next();
  };
}


startPruner();

The middleware is wired into server.js via the import and app.use / route-level calls shown above. The rateLimit middleware is applied globally, and validateChatRequest is applied specifically to the /api/chat route.

Building the React Frontend

Chat Interface Component Architecture

The frontend is scaffolded with Vite and React 18. The component tree is straightforward: App renders ChatWindow, which contains MessageList (a scrollable list of messages) and InputBar (text input with a send button). State management uses useState for the message history and useRef to hold a reference to the accumulating assistant response during streaming.

npm create vite@latest gemma3-frontend -- --template react
cd gemma3-frontend && npm install

Implementing Streaming Chat with SSE

The ChatWindow component handles the fetch-to-stream pipeline. It sends messages to the backend, reads the SSE stream using the browser’s ReadableStream API, and appends tokens to the current assistant message as they arrive.

Note on SSE parsing: The code below accumulates a buffer across reads to handle the case where a TCP chunk splits an SSE event across two reader.read() calls. Without this buffering, JSON.parse can fail intermittently on slow or congested networks.

import { useState, useRef, useEffect } from 'react';
import ModelInfoBar from './ModelInfoBar';

const API_URL = import.meta.env.VITE_API_URL || 'http://localhost:3001';

export default function ChatWindow() {
  const [messages, setMessages] = useState([]);
  const [input, setInput] = useState('');
  const [loading, setLoading] = useState(false);
  const [modelInfo, setModelInfo] = useState(null);
  const assistantMsg = useRef('');
  const abortRef = useRef(null);

  
  useEffect(() => {
    return () => { abortRef.current?.abort(); };
  }, []);

  async function sendMessage() {
    if (!input.trim() || loading) return;

    const userMessage = { role: 'user', content: input, id: crypto.randomUUID() };
    const updatedMessages = [...messages, userMessage];
    const assistantPlaceholder = { role: 'assistant', content: '', id: crypto.randomUUID() };
    setMessages([...updatedMessages, assistantPlaceholder]);
    setInput('');
    setLoading(true);
    assistantMsg.current = '';

    const controller = new AbortController();
    abortRef.current = controller;

    try {
      const res = await fetch(`${API_URL}/api/chat`, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          messages: updatedMessages.map(({ role, content }) => ({ role, content })),
        }),
        signal: controller.signal,
      });

      
      if (!res.ok) {
        const errBody = await res.json().catch(() => ({ error: `HTTP ${res.status}` }));
        throw new Error(errBody.error || `HTTP ${res.status}`);
      }

      const reader = res.body.getReader();
      const decoder = new TextDecoder();
      let buffer = '';
      let errorEncountered = false;

      while (true) {
        const { done, value } = await reader.read();

        if (done) {
          
          buffer += decoder.decode();
          break;
        }

        buffer += decoder.decode(value, { stream: true });
        const parts = buffer.split('

');
        
        buffer = parts.pop() ?? '';

        let tokenUpdated = false;

        for (const part of parts) {
          if (errorEncountered) break;
          const lines = part.split('
').filter(l => l.startsWith('data: '));
          for (const line of lines) {
            let parsed;
            try {
              parsed = JSON.parse(line.slice(6));
            } catch {
              continue; 
            }

            if (parsed.error) {
              assistantMsg.current = `Error: ${parsed.error}`;
              tokenUpdated = true;
              errorEncountered = true;
              await reader.cancel();
              break;
            } else if (parsed.tokensPerSecond !== undefined) {
              setModelInfo({
                model: parsed.model,
                totalTokens: parsed.totalTokens,
                tokensPerSecond: parsed.tokensPerSecond,
                elapsed: parsed.elapsed,
              });
            } else if (typeof parsed.token === 'string') {
              assistantMsg.current += parsed.token;
              tokenUpdated = true;
            }
          }
        }

        if (tokenUpdated) {
          const snapshot = assistantMsg.current;
          setMessages(prev => {
            const updated = [...prev];
            updated[updated.length - 1] = {
              ...updated[updated.length - 1],
              content: snapshot,
            };
            return updated;
          });
        }

        if (errorEncountered) break;
      }
    } catch (err) {
      if (err.name === 'AbortError') return; 
      const msg = err.message || 'Connection failed. Is the backend running?';
      assistantMsg.current = msg;
      setMessages(prev => {
        const updated = [...prev];
        updated[updated.length - 1] = {
          ...updated[updated.length - 1],
          content: msg,
        };
        return updated;
      });
    } finally {
      setLoading(false);
      abortRef.current = null;
    }
  }

  return (
    <div style={{ maxWidth: 640, margin: '2rem auto', fontFamily: 'sans-serif' }}>
      <div style={{ height: 400, overflowY: 'auto', border: '1px solid #ccc', padding: 12 }}>
        {messages.map((msg) => (
          <div key={msg.id} style={{ marginBottom: 8 }}>
            <strong>{msg.role}:strong> {msg.content}
          div>
        ))}
      div>
      {modelInfo && <ModelInfoBar info={modelInfo} />}
      <div style={{ display: 'flex', gap: 8, marginTop: 8 }}>
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          onKeyDown={e => e.key === 'Enter' && sendMessage()}
          style={{ flex: 1, padding: 8 }}
          placeholder="Ask Gemma 3 something..."
          disabled={loading}
        />
        <button onClick={sendMessage} disabled={loading}>
          {loading ? '...' : 'Send'}
        button>
      div>
    div>
  );
}

Displaying Model Metadata

A small component renders the inference statistics from the final SSE event. This is valuable for developer tooling, letting the user verify which model variant is active and how fast inference is running.

export default function ModelInfoBar({ info }) {
  if (!info) return null;

  return (
    <div style={{
      display: 'flex',
      gap: 16,
      padding: '6px 12px',
      background: '#f4f4f4',
      fontSize: 13,
      borderRadius: 4,
      marginTop: 4,
    }}>
      <span><strong>Model:strong> {info.model}span>
      <span><strong>Tokens:strong> {info.totalTokens}span>
      <span><strong>Speed:strong> {info.tokensPerSecond} tok/sspan>
      <span><strong>Time:strong> {info.elapsed}sspan>
    div>
  );
}

Performance Optimization and Benchmarks

Quantization Impact on Quality and Speed

Quantization level has a direct, measurable effect on both inference speed and output quality. Q4_K_M can deliver higher tokens per second compared to Q8_0 on the same hardware (speed gains vary by hardware; benchmark on your own machine with ollama run gemma3:4b-q4_K_M --verbose and ollama run gemma3:4b-q8_0 --verbose to measure the difference). You will not notice quality loss in chatbot or code-assistance tasks, but tasks requiring nuanced reasoning or precise factual recall expose the gap. Run the same prompt at Q4 and Q8 and compare the outputs side by side to judge whether the difference matters for your use case. For the 4B variant, Q4_K_M is almost always the correct default. For the 12B and 27B variants, Q4_K_M makes the difference between fitting in VRAM or not on consumer GPUs.

Q8_0 is worth the memory cost only when output fidelity matters more than speed, such as content generation or evaluation tasks. The 1B variant is small enough that Q8_0 adds trivial overhead, so there is little reason to quantize it further.

GPU Offloading and Layer Splitting

On machines where the model does not fully fit in VRAM, the num_gpu parameter controls how many transformer layers are offloaded to the GPU. The remaining layers run on CPU. To find the total layer count for a specific variant, run ollama show gemma3:4b --verbose. Setting num_gpu to roughly half the total layer count, for example, splits the work and avoids out-of-memory crashes while still accelerating inference compared to pure CPU execution. Tuning this parameter is iterative: increase num_gpu until VRAM usage approaches the limit, then back off by one or two layers.

Context Window and Memory Management

Longer conversations consume more memory proportionally. The num_ctx parameter sets the maximum context length and directly affects memory allocation. For most interactive use cases, 4096 or 8192 tokens is sufficient. Setting num_ctx to the model’s full 128K maximum on a machine with limited RAM will cause out-of-memory errors. Start with num_ctx 4096 and increase incrementally, monitoring memory usage with ollama ps. In the Node.js layer, a practical approach is to truncate the messages array to the most recent N messages (or N tokens) before sending it to Ollama, preserving the system prompt while discarding older conversation history.

Implementation Checklist

1. Assess hardware (CPU, RAM, GPU VRAM) and select a variant from the comparison table.
2. Install Ollama and pull the target Gemma 3 model.
3. Create and test a custom Modelfile with system prompt and parameters.
4. Verify the local Ollama API with a curl request.
5. Scaffold the Node.js Express server, set "type": "module", and install dependencies (express, ollama, cors).
6. Implement the /api/chat endpoint with SSE streaming.
7. Add request validation and rate limiting middleware.
8. Build the React chat interface with streaming token rendering.
9. Test end-to-end streaming from input to displayed response.
10. Benchmark quantization levels and tune num_gpu layer offloading.
11. (Optional) Containerize the full stack with Docker for team deployment.

Common Pitfalls and Troubleshooting

Model Fails to Load or OOM Errors

The most common cause is num_ctx being set higher than available memory supports. Reduce it to 4096 and retry. If that still fails, switch to a smaller quantization (Q4_K_M instead of Q8_0) or drop to a smaller variant. Run ollama ps to verify no other models are consuming memory simultaneously.

Slow Inference on CPU-Only Machines

CPU inference for the 12B and 27B variants produces single-digit tokens per second on most consumer hardware. Expect this – it is not a configuration error. If interactive latency is unacceptable, drop to the 4B or 1B variant. The num_thread parameter controls how many CPU threads Ollama uses; setting it to the number of physical cores (not logical/hyperthreaded cores) generally gives the best throughput, though results can vary by CPU architecture – benchmark to confirm.

CORS and Networking Issues Between React and Node.js

The React dev server (Vite, default port 5173) and the Express backend (port 3001) are different origins. The cors middleware on the Express server must explicitly allow the frontend origin. The code examples above include this configuration. If requests fail with CORS errors in the browser console, verify the origin value in the cors() call matches the exact URL of the Vite dev server. The backend reads this value from the ALLOWED_ORIGIN environment variable (defaulting to http://localhost:5173), so set this variable if running on a non-default port or in production.

ESM / Module Errors on Server Startup

If node server.js throws SyntaxError: Cannot use import statement in a module, your package.json is missing "type": "module". Run npm pkg set type=module and retry.

What to Build Next

At this point, the full pipeline is operational: Gemma 3 running locally via Ollama, a Node.js backend streaming inference results over SSE, and a React frontend rendering tokens in real time. The comparison table and implementation checklist above serve as ongoing references as hardware or requirements change.

The highest-value extension is retrieval-augmented generation with a local vector store like ChromaDB or LanceDB, which lets Gemma 3 answer questions grounded in your own documents without fine-tuning. Beyond that, consider fine-tuning a variant with LoRA for domain-specific tasks or containerizing the entire stack with Docker for reproducible team-wide deployment.