Production RAG Knowledge Base

Introduction

30 Second Summary

Imagine asking your AI assistant a question about a 50-page safety document and getting the exact answer in seconds, with the source paragraph cited. That is what Retrieval Augmented Generation makes possible.

In this project, you will build a production RAG infrastructure layer that lets your team upload any document to a knowledge base and query it with natural language. You will replace crash-prone defaults with PostgreSQL, Apache Tika, and local Ollama embeddings.

What You'll Build

Upload a real company document to your AI platform, ask a specific technical question about its contents, and watch the AI retrieve the exact relevant passage with the source cited.

By the end of this project, you'll have:

• A knowledge base that accepts PDFs, Word documents, and scanned files, extracts their text automatically, and indexes the content for instant retrieval.
• A local, private AI that answers natural language questions about uploaded documents with accurate, cited responses. No data leaves your network.
• A tuned hybrid search system that combines keyword precision with semantic understanding, delivering correct answers even from dense technical documents full of model numbers and compliance codes.
• Secret Mission: Build an automated backup and restore script that protects your entire knowledge base from data loss.

Are there any prerequisites?

This is Phase 2 of the CLIVE project. You need Phase 1 complete: Ubuntu 24.04 LTS Desktop with Docker, Docker Compose, Ollama (with llama3.2:2b), and Open WebUI all running.

If you have not completed Phase 1, go back and finish it first. This project builds directly on that infrastructure.

Before We Start

Before diving into containers and configuration, let's lock in what you're building in this phase and why it matters for CVC's day-to-day operations.

Deploy PGVector and Tika Containers

Open WebUI's default RAG setup uses an embedded ChromaDB database and a built-in pypdf extractor. ChromaDB crashes under multi-process access because it relies on a local SQLite backend. The default pypdf extractor leaks memory under sustained load, causing your Open WebUI process to grow in memory continuously.

In this step, you will deploy production-grade replacements: PostgreSQL with the PGVector extension for vector storage, and Apache Tika for document extraction. These run as dedicated containers alongside Ollama and Open WebUI in a single Docker Compose stack.

In this step, get ready to:

• Create the phase2 directory structure and a full docker-compose.yml defining all four services.
• Configure environment variables for database credentials and Open WebUI's RAG backend.
• Bring up the entire stack and verify all containers are healthy.

Create the phase2 directory structure and docker-compose.yml

Phase 2 gets its own directory with a fresh Docker Compose file that defines the complete stack. This compose file replaces Phase 1's setup by adding the PGVector and Tika containers.

• In your terminal, navigate to your home directory and create the phase2 project structure by running these commands:

cd ~
mkdir -p phase2/configs/postgres phase2/scripts
cd phase2

• Confirm the directories were created by running:

ls -R

You should see configs/postgres and scripts listed in the output.

Now you will build the docker-compose.yml file piece by piece. This file defines four services: Ollama, Open WebUI, PostgreSQL with PGVector, and Apache Tika.

• Create docker-compose.yml and add the Ollama service definition by pasting this first section:

services:
  ollama:
    image: ollama/ollama
    container_name: clive-ollama
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

What does this section do?

• services: begins the service definitions for the compose stack.
• The ollama service mirrors your Phase 1 setup. It exposes port 11434 and reserves the GPU for model inference.
• ollama_data is a named volume that persists your downloaded models across container restarts.

• Add the Open WebUI service directly below the Ollama service in the same file:

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: clive-webui
    restart: unless-stopped
    ports:
      - "3000:8080"
    volumes:
      - openwebui_data:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - VECTOR_DB=pgvector
      - PGVECTOR_DB_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}
      - CONTENT_EXTRACTION_ENGINE=tika
      - TIKA_SERVER_URL=http://tika:9998
      - RAG_EMBEDDING_ENGINE=ollama
      - RAG_EMBEDDING_MODEL=nomic-embed-text
      - ENABLE_RAG_HYBRID_SEARCH=true
    depends_on:
      postgres:
        condition: service_healthy
      tika:
        condition: service_started
      ollama:
        condition: service_started

What do these environment variables configure?

• VECTOR_DB=pgvector tells Open WebUI to use PGVector instead of the default ChromaDB.
• CONTENT_EXTRACTION_ENGINE=tika routes document parsing to the Tika container instead of the memory-leaking pypdf.
• RAG_EMBEDDING_ENGINE=ollama and RAG_EMBEDDING_MODEL=nomic-embed-text configure local embeddings so no data leaves your network.
• ENABLE_RAG_HYBRID_SEARCH=true activates BM25 keyword search alongside vector similarity.
• depends_on ensures PostgreSQL is healthy before Open WebUI starts, preventing connection errors on boot.

• Add the PostgreSQL + PGVector service below Open WebUI:

  postgres:
    image: pgvector/pgvector:0.8.6-pg16
    container_name: clive-postgres
    restart: unless-stopped
    environment:
      - POSTGRES_DB=${POSTGRES_DB}
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
    volumes:
      - pgvector_data:/var/lib/postgresql/data
      - ./configs/postgres/init-pgvector.sql:/docker-entrypoint-initdb.d/init-pgvector.sql
    ports:
      - "127.0.0.1:5432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s

What does the postgres service configure?

• pgvector/pgvector:0.8.6-pg16 is PostgreSQL 16 with the PGVector extension pre-installed.
• The port binding 127.0.0.1:5432:5432 restricts access to localhost only, preventing external network connections to your database.
• The init-pgvector.sql mount runs automatically on first startup to enable the vector extension.
• The healthcheck uses pg_isready so Open WebUI waits until PostgreSQL is actually accepting connections before starting.

• Add the Tika service and the volumes declaration to complete the file:

  tika:
    image: apache/tika:3.3.1.0-full
    container_name: clive-tika
    restart: unless-stopped
    ports:
      - "127.0.0.1:9998:9998"

volumes:
  ollama_data:
  openwebui_data:
  pgvector_data:

What does this section do?

• apache/tika:3.3.1.0-full includes Tesseract OCR, so Tika can extract text from scanned PDFs and images as well as standard documents.
• Like PostgreSQL, Tika is bound to 127.0.0.1 only for security.
• The volumes: block at the bottom declares the three named volumes that persist data across container restarts.

• Save docker-compose.yml.

✔️ Awesome, I've got everything!

Great. Make sure you have saved the file before moving on.

ⓧ I'd like to double check the full code

Your docker-compose.yml should contain these four sections in order. Scroll up and confirm each is present:

• services: header with the ollama service (GPU reservation, port 11434, named volume).
• open-webui service (port 3000, all RAG environment variables, depends_on with health conditions).
• postgres service (pgvector image, localhost-only port 5432, healthcheck, init SQL mount).
• tika service (localhost-only port 9998) followed by the volumes: declaration with ollama_data, openwebui_data, and pgvector_data.

Create the .env file with database credentials

The compose file references variables like ${POSTGRES_USER} and ${POSTGRES_PASSWORD}. Docker Compose reads these from a .env file in the same directory.

• Create the .env file in your phase2 directory with the following content:

# PostgreSQL + PGVector
POSTGRES_DB=clive
POSTGRES_USER=clive
POSTGRES_PASSWORD=change-this-to-a-secure-password

# Backup settings
BACKUP_DIR=./backups
BACKUP_RETENTION_DAYS=7

What do these variables control?

• POSTGRES_DB, POSTGRES_USER, and POSTGRES_PASSWORD set the database name, username, and password used by both the PostgreSQL container and Open WebUI's connection string.
• BACKUP_DIR and BACKUP_RETENTION_DAYS are used by the backup script you will create in the Secret Mission.

Change the password for production

The placeholder password is fine for local development. For any networked deployment, replace it with a strong, unique password.

• Save .env.
• Validate that Docker Compose can read both files together by running:

docker compose config --services

You should see all four service names printed: ollama, open-webui, postgres, and tika.

Seeing a YAML parse error?

• YAML is whitespace-sensitive. Each indentation level must be exactly 2 spaces, not tabs.
• Check that each service is indented under services: and that the volumes: block is at the root level (no indentation).
• Make sure the .env file is in the same directory as docker-compose.yml.

Help me debug my docker-compose.yml YAML syntax error

Create the init script, bring up the stack, and verify

PostgreSQL needs the PGVector extension enabled before Open WebUI can store vectors. A small SQL script mounted into the container's /docker-entrypoint-initdb.d/ directory runs automatically on first boot.

• Create the initialization script by pasting the following into configs/postgres/init-pgvector.sql:

-- Enable the pgvector extension for vector similarity search
CREATE EXTENSION IF NOT EXISTS vector;

What does this script do?

CREATE EXTENSION IF NOT EXISTS vector loads the PGVector library into your database. This gives PostgreSQL the ability to store and query high-dimensional vectors, which is how RAG embeddings are persisted.

• Save the file.

Before starting the Phase 2 stack, you need to stop your Phase 1 containers to free up the ports they share (11434 and 3000).

• Stop the Phase 1 containers by running:

docker stop $(docker ps -q) 2>/dev/null || true

• Now bring up the full Phase 2 stack by running:

docker compose up -d

What happens when you run this?

Docker Compose pulls any images you do not already have (pgvector/pgvector:0.8.6-pg16 and apache/tika:3.3.1.0-full), creates the named volumes, and starts all four containers. PostgreSQL runs the init-pgvector.sql script on first boot. Open WebUI waits for the PostgreSQL healthcheck to pass before starting.

The first run may take 1-2 minutes while images download.

Before you check, what do you expect to see when you run docker compose ps? Think about how many containers should be listed and what their status should be.

• Verify all containers are running by executing:

docker compose ps

You should see four containers: clive-ollama, clive-webui, clive-postgres, and clive-tika all showing "healthy" or "running" status.

Containers not all showing healthy?

• If clive-postgres shows "starting", wait 30 seconds and run docker compose ps again. The healthcheck has a 30-second start period.
• If clive-webui is restarting, check its logs with docker compose logs open-webui. A common cause is a typo in the PGVECTOR_DB_URL connection string.
• If a port conflict is reported, confirm you stopped all Phase 1 containers. Run docker ps to check for any lingering containers using ports 3000, 11434, 5432, or 9998.

Help me troubleshoot my CLIVE Phase 2 containers not starting correctly

Now create a verification script that checks each service in detail. This script confirms the PGVector extension is loaded, Tika responds to HTTP requests, and Open WebUI is accessible.

• Create scripts/verify.sh and add the header and first two checks:

#!/bin/bash
# CLIVE Phase 2 - Service Health Verification
# Run this after docker compose up to confirm all services are operational.

set -e

echo "=== CLIVE Phase 2 - Service Verification ==="
echo ""

# Check Docker containers are running
echo "[1/5] Checking container status..."
docker compose ps --format "table {{.Name}}\t{{.Status}}" | grep -E "(clive-)" || {
    echo "ERROR: Containers not found. Run 'docker compose up -d' first."
    exit 1
}
echo ""

# Check PostgreSQL with PGVector
echo "[2/5] Checking PostgreSQL + PGVector..."
docker exec clive-postgres psql -U clive -d clive -c "SELECT extname, extversion FROM pg_extension WHERE extname = 'vector';" 2>/dev/null || {
    echo "ERROR: PGVector extension not available."
    exit 1
}
echo ""

What do these checks verify?

• Check 1 uses docker compose ps to confirm all containers with "clive-" prefix are listed.
• Check 2 runs a psql query inside the PostgreSQL container to confirm the vector extension is actually loaded and accessible.

• Add the Tika and Ollama checks below the existing content:

# Check Apache Tika
echo "[3/5] Checking Apache Tika..."
TIKA_STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:9998/tika)
if [ "$TIKA_STATUS" = "200" ]; then
    echo "Tika is responding (HTTP 200)"
else
    echo "ERROR: Tika not responding (HTTP $TIKA_STATUS)"
    exit 1
fi
echo ""

# Check Ollama
echo "[4/5] Checking Ollama..."
OLLAMA_STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:11434/api/tags)
if [ "$OLLAMA_STATUS" = "200" ]; then
    echo "Ollama is responding (HTTP 200)"
    # Check for nomic-embed-text
    if curl -s http://localhost:11434/api/tags | grep -q "nomic-embed-text"; then
        echo "nomic-embed-text model is available"
    else
        echo "WARNING: nomic-embed-text not pulled yet. Run: ollama pull nomic-embed-text"
    fi
else
    echo "ERROR: Ollama not responding (HTTP $OLLAMA_STATUS)"
    exit 1
fi
echo ""

• Add the final Open WebUI check and summary output:

# Check Open WebUI
echo "[5/5] Checking Open WebUI..."
WEBUI_STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:3000)
if [ "$WEBUI_STATUS" = "200" ] || [ "$WEBUI_STATUS" = "302" ]; then
    echo "Open WebUI is responding (HTTP $WEBUI_STATUS)"
else
    echo "ERROR: Open WebUI not responding (HTTP $WEBUI_STATUS)"
    exit 1
fi
echo ""

echo "=== All services verified successfully ==="
echo ""
echo "Access Open WebUI at: http://localhost:3000"
echo "Qdrant Dashboard at: N/A (using PGVector)"
echo "Tika endpoint at:    http://localhost:9998"

• Save verify.sh and make it executable by running:

chmod +x scripts/verify.sh

• Run the verification script:

./scripts/verify.sh

You should see all five checks pass, ending with "All services verified successfully" and the access URLs printed. The nomic-embed-text warning is expected at this point. You will pull that model in the next step.

Verify script failing on PGVector check?

• If the PGVector check fails, the init script may not have run. This only executes on a fresh database volume. Run docker compose down -v to remove volumes, then docker compose up -d to recreate everything from scratch.
• If Tika fails, it may still be starting up (the -full image is large). Wait 30 seconds and retry.

Help me fix my CLIVE Phase 2 verify.sh script failures

Your production RAG infrastructure is now running. Next up, you will pull the embedding model and prove the full pipeline works end-to-end by uploading a document and querying it.

Pull the Embedding Model and Test RAG

Your infrastructure is running, but it has never processed a real document. The containers are wired together on paper. You need to prove the full pipeline actually works end-to-end: pull the embedding model, upload a document, and confirm that the AI can answer questions from it.

In this step you will pull the nomic-embed-text embedding model into Ollama, create a Knowledge Base in Open WebUI, upload a test PDF, and verify the AI retrieves accurate answers from it.

In this step, get ready to:

• Pull the nomic-embed-text embedding model and confirm it is available.
• Create a Knowledge Base collection and upload a short PDF.
• Query the knowledge base in chat and verify the AI cites the uploaded document.

Pull the nomic-embed-text embedding model

Open WebUI is configured to use nomic-embed-text for generating vector embeddings. The model needs to be pulled into Ollama before the RAG pipeline can embed any uploaded documents.

• Pull the embedding model by running this command:

docker exec clive-ollama ollama pull nomic-embed-text

What does this command do?

This runs ollama pull inside the clive-ollama container to download the nomic-embed-text model (274MB). This model converts text into vector embeddings that PGVector stores for similarity search.

Unlike the LLM (llama3.2:2b), this model does not generate text. It produces a numerical representation of meaning, which is what powers retrieval.

• Confirm the model downloaded successfully by listing the available models:

docker exec clive-ollama ollama list

You should see both llama3.2:2b and nomic-embed-text listed in the output.

Don't see nomic-embed-text in the list?

• Check your internet connection. The pull command downloads 274MB from the Ollama registry.
• Ensure the clive-ollama container is running with docker compose ps.
• Try running the pull command again. Interrupted downloads resume where they left off.

Help me troubleshoot why nomic-embed-text did not download into my Ollama container.

Create a Knowledge Base and upload a test document

With the embedding model in place, you can now create a Knowledge Base collection in Open WebUI and upload a document. Open WebUI will use Apache Tika to extract the text, nomic-embed-text to generate embeddings, and PGVector to store them.

• Open your browser and navigate to http://localhost:3000.
• Click Workspace in the sidebar.
• Select the Knowledge tab.
• Click the Create button in the Workspace header.
• Enter Test Documents as the name and add a short description like Initial RAG pipeline test.
• Click Create to save the new Knowledge Base.

You now have an empty Knowledge Base. Next, upload a short PDF to it.

• Inside the Test Documents Knowledge Base, click the upload area or drag and drop a short 1-2 page PDF from your computer.

Pick a document with clear facts

Choose any short PDF you have available. A company document, a product specification, or even a one-page memo works well. The key is that it contains at least one specific, verifiable fact you can ask about later (a date, a name, a number).

• Wait for the upload to finish. Open WebUI displays a processing indicator while Tika extracts text and the embedding model generates vectors.
• Once processing completes, the file appears in the Knowledge Base list with its name and size.

Your document is now embedded and stored in PGVector. The entire ingestion pipeline just ran: Tika extracted text, nomic-embed-text generated vectors, and PostgreSQL stored them.

Document stuck on processing?

• Check that the clive-tika container is running: docker compose ps.
• Verify Tika is responding: curl http://localhost:9998/tika should return HTTP 200.
• Check Open WebUI logs for errors: docker logs clive-webui --tail 30.

Help me debug why my document upload is stuck processing in Open WebUI with PGVector and Tika.

Test RAG retrieval in chat

The moment of truth. You will start a chat, attach the Knowledge Base, and ask a question that can only be answered from the uploaded document.

• Click New Chat in the Open WebUI sidebar.
• In the message input, type # to open the attachment selector.
• Select Test Documents from the list. A document icon appears above the message input confirming the Knowledge Base is attached.

What does the # shortcut do?

Typing # in the chat input opens a suggestion box listing your Knowledge Bases and files. Selecting one attaches it to the current conversation so the AI can search and retrieve its contents when answering.

Before you send your question, think about what you expect. The AI should answer using information from your uploaded document, not from its general training data.

• Type a specific factual question about the document contents. For example, if you uploaded a product spec, ask about a specific model number or rating mentioned in the document.
• Press Enter to send the message.

Do you expect the AI to answer from the document, or from its own knowledge? It should pull the answer directly from your uploaded PDF.

The AI responds with an answer that references information from your document. You should see the source document cited in the response, confirming the retrieval pipeline is working.

What just happened end-to-end?

Your question triggered the full RAG pipeline:

• Open WebUI embedded your question using nomic-embed-text.
• PGVector performed a vector similarity search against the stored document embeddings.
• The most relevant chunks were injected into the prompt for llama3.2:2b.
• The LLM generated an answer grounded in the retrieved text.

AI not using the document content?

• Make sure you attached the Knowledge Base with # before sending your message. The document icon must be visible above the input.
• Verify the document finished processing (no spinner next to it in the Knowledge Base view).
• Try asking a very specific question with a keyword that appears verbatim in the document.

Help me figure out why my AI is not retrieving content from my Knowledge Base in Open WebUI.

Your production RAG pipeline is confirmed working. Documents go in, accurate answers come out. Next up, you will test it against a real multi-page technical document to find where the default settings fall short.

Upload a Real Technical Document and Hit the Retrieval Limit

Your short test document proved the pipeline works end to end. Upload, embed, store, retrieve, answer.

CVC's real documents are a different challenge. Multi-page RAMS documents, dense equipment specifications, and lengthy contracts stress the retrieval system in ways a short PDF never will.

In this step, get ready to:

• Upload a real multi-page CVC document to the knowledge base.
• Ask a specific technical question targeting a fact buried deep in the document.
• Observe how the system handles dense technical content with default settings.

Upload a real multi-page document to the knowledge base

A 1-2 page test PDF barely exercises the chunking system. To see how the retrieval pipeline performs under realistic conditions, you need a document that is at least 5 pages of dense, technical content.

• Choose a real CVC document from your files. A RAMS document, equipment specification, contract, or tender response that is 5 or more pages long.
• Note down one specific fact buried inside the document. This should be something precise: a product code, an IP rating, a compliance clause number, or a cable specification. You will use this fact to test retrieval quality in the next substep.
• Return to Open WebUI in your browser at http://localhost:3000.
• Click Workspace in the left sidebar.
• Click Knowledge to see your collections.
• Open the Test Documents collection you created earlier.
• Click the upload button and select your multi-page CVC document.
• Wait for the document to finish processing. Apache Tika extracts the text and nomic-embed-text generates the embeddings. This may take a minute for larger documents.

Once processing completes, you should see both documents listed in the Test Documents collection.

Document stuck processing?

If the upload seems to hang, check that the Tika container is still running. Open a terminal and run docker compose ps from your phase2 folder. The clive-tika container should show as running.

help me troubleshoot why my document upload is stuck in Open WebUI

Ask a specific technical question about the document

Now for the real test. You are going to ask the AI a question that requires finding one specific fact buried inside your multi-page document.

• Click New Chat in the top left to start a fresh conversation.
• Click the + icon in the message input area, then select your Test Documents knowledge base to attach it to this chat.

Before you send your question, think about what you expect. You noted a specific fact from the document earlier. Do you think the AI will find it accurately, or will it struggle?

• Type a question that targets the specific fact you noted. For example, if your document mentions a particular cable type on page 7, ask "What cable type is specified for the main distribution?" If it contains a clause about working at height, ask "What does clause 4.3.2 say about working at height requirements?"
• Press Enter to send your question.

Pick the hardest question

The more specific your question, the better this test works. Ask about a product code, an exact IP rating, a numbered clause, or a specific compliance threshold. Vague questions like "summarise this document" will not reveal the retrieval limitation.

Observe the retrieval failure on dense content

Read the AI's response carefully. Compare it against the fact you know is in the document.

• Check whether the AI found the exact fact you were looking for.
• Look at the cited source passages (if shown). Are they the relevant section of the document, or are they from unrelated paragraphs?

You will likely see one of these outcomes: the AI gives a vague answer without the specific detail, returns content from the wrong section of the document, or says it cannot find the information.

Why does this happen?

Two problems are working against you here. First, the default chunking strategy splits your document into small pieces (around 1000 characters with 100 character overlap). When a critical fact sits near a chunk boundary, the context gets severed.

Second, pure vector search relies on semantic similarity. It understands meaning but struggles with exact keyword matches. When you ask about "clause 4.3.2" or "IP68 rating", the vector search looks for semantically similar content rather than the exact term. This is why technical documents with precise codes and numbers need a different approach.

This is the expected result. You have just hit the wall that every enterprise RAG deployment faces with technical documents. The next step fixes both problems.

You have proven that the default settings fail on real CVC documents. In the next step, you will enable hybrid search and tune the chunking parameters to fix this retrieval gap.

Enable Hybrid Search and Tune Chunking

In Step 3, you saw the default RAG settings fail on a real CVC document. The AI either missed the answer entirely or returned irrelevant context when you asked a specific technical question.

Pure vector search struggles with exact technical terms like model numbers, IP ratings, and clause references. Semantic similarity alone does not capture keyword precision. The fix combines two changes: hybrid search adds BM25 keyword matching alongside vector similarity, and larger chunks with more overlap preserve the context that dense technical documents need.

In this step, get ready to:

• Enable hybrid search (BM25 + vector + CrossEncoder reranking) in Open WebUI Admin Settings.
• Tune chunk size and overlap for dense technical documents and reindex the knowledge base.
• Verify retrieval accuracy on the same question that previously failed.

Enable hybrid search in the admin panel

BM25 is a keyword-matching algorithm that excels at finding exact terms. When combined with vector similarity search and CrossEncoder reranking, it creates a retrieval pipeline that handles both conceptual questions and precise technical lookups.

• In Open WebUI at http://localhost:3000, click your profile avatar in the bottom-left corner of the sidebar.
• Click Settings.
• Select Admin, then Tools, then Documents.
• Find the Hybrid Search toggle and confirm it is enabled (switched on).

What does hybrid search do?

With hybrid search enabled, every retrieval query runs two searches in parallel. BM25 finds chunks containing the exact keywords you used (perfect for model numbers and clause references). Vector search finds chunks that are semantically similar (perfect for paraphrased questions).

A CrossEncoder reranker then scores all results together and returns the most relevant passages. This combination handles both precise technical lookups and broad conceptual questions.

Adjust chunk size and reindex the knowledge base

The default chunk size (1000 characters with 100 character overlap) splits dense technical paragraphs at arbitrary boundaries. A product specification that spans two chunks loses critical context at the split point.

• On the same Documents settings page, find the Chunk Size field.
• Change the value to 1500.
• Find the Chunk Overlap field.
• Change the value to 200.
• Click Save to apply the new settings.

Why 1500 characters with 200 overlap?

Larger chunks (1500 characters) keep more surrounding context intact per retrieval hit. A technical specification paragraph that would have been cut in half at 1000 characters now stays whole.

The 200-character overlap means adjacent chunks share content at their boundaries. If a key sentence falls near a split point, it appears in both chunks, so retrieval can still find it.

Existing documents were chunked with the old settings. You need to reindex them so the new chunk size takes effect.

• On the same Documents settings page, click the Reindex button.
• Wait for the reindex to complete. This re-embeds all documents in the knowledge base using the new chunk parameters.

The reindex duration depends on how many documents you have uploaded. For a couple of documents, it should finish within a minute or two.

Reindex seems stuck or takes too long?

The reindex runs embeddings through Ollama, so it depends on your GPU or CPU speed. Check that the Ollama container is still healthy by running docker compose ps in your terminal. If clive-ollama shows as unhealthy, restart the stack with docker compose restart and try the reindex again.

Having trouble?

Re-ask your technical question and verify accurate retrieval

The infrastructure is now tuned. Hybrid search is active, chunks are larger with more overlap, and the knowledge base has been reindexed. Time to test whether these changes fix the retrieval problem you observed in Step 3.

• Start a new chat in Open WebUI with the Test Documents knowledge base attached.

Before you ask your question, think back to Step 3. Do you expect the AI to find the correct answer this time? What changed that might help it locate the specific detail it missed before?

• Ask the exact same specific technical question you asked in Step 3 (the one that returned an incomplete or incorrect answer).
• Review the response. The AI should now return the correct, specific answer with the relevant source passage cited.

You should see the AI accurately answer your question and cite the document as its source. BM25 keyword matching found the exact term, and the larger chunks kept enough surrounding context for the AI to formulate a complete answer.

What if the answer improved but is still not perfect?

RAG quality depends on multiple factors. If the answer improved but is still not fully accurate, try asking the question in a slightly different way. You can also experiment with increasing the Top K value (how many chunks are retrieved) in the same Admin Documents settings page.

The key indicator of success is that the response now cites the correct document and references information from the relevant section.

Still getting irrelevant or missing answers?

Confirm that hybrid search is enabled in the Admin Documents settings. If you toggled it but did not click Save, the setting was not applied.

Confirm that you clicked Reindex after changing the chunk size. Old documents retain their original chunking until reindexed.

Need more help?

Your production RAG pipeline is fully tuned and verified. CVC staff can now upload any company document and get accurate, cited answers from the AI. The combination of PGVector, Apache Tika, local Ollama embeddings, and hybrid search creates an enterprise-grade knowledge base that handles dense technical content reliably.

Secret mission

Automated PGVector Backup with Verification

Your knowledge base now holds real company documents. What happens if the PostgreSQL volume gets corrupted or someone accidentally runs docker compose down -v? Build an automated backup script that dumps, verifies, and rotates your PGVector database so the knowledge base is always recoverable.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally using Docker containers, so there are no ongoing financial costs.

Resources you used:

• PostgreSQL container with PGVector (persistent volume: pgvector_data). Stores all document embeddings and metadata on disk.
• Apache Tika container (stateless, no persistent data stored).
• Ollama container with nomic-embed-text model (persistent volume: ollama_data, 274MB model on disk).
• Open WebUI container with application data (persistent volume: openwebui_data).
• Local phase2/ project directory (docker-compose.yml, .env, scripts, configs, backups).

Keep everything running

No action needed. Choose this if you're still actively building on the CLIVE platform or plan to continue with Phase 3.

• Your containers will continue running in the background. Resource usage is minimal when idle (approximately 200MB RAM for PostgreSQL, 100MB for Tika).
• All uploaded documents, embeddings, and knowledge base collections remain available.
• The backup script in phase2/scripts/backup.sh is ready to run whenever you need a fresh backup.

Pause - I'll come back to this later

Shut down running containers to free up memory without losing any data. Your volumes and local files remain intact.

• Navigate to your phase2/ directory in the terminal.
• Stop all containers by running:

docker compose stop

• This halts all four containers (postgres, tika, ollama, open-webui) but keeps volumes and configuration intact.
• When you're ready to resume, navigate back to phase2/ and run:

docker compose start

• All services come back online with your knowledge base, embeddings, and settings preserved.

Delete - I don't want to use this again

Remove all containers, volumes, and local files permanently. This deletes all uploaded documents, embeddings, and backups.

This is permanent

Running these commands permanently deletes all uploaded documents, embeddings, knowledge base collections, and backup files. Make sure you have copied anything you need before proceeding.

• Navigate to your phase2/ directory in the terminal.
• Remove all containers and their persistent volumes by running:

docker compose down -v

• This removes all four containers and deletes the pgvector_data, ollama_data, and openwebui_data volumes permanently.
• Delete the local project directory by running:

cd ~
rm -rf phase2/

• This removes the docker-compose.yml, .env, init-pgvector.sql, verify.sh, backup.sh, and all backup files from your system.
• Confirm everything is gone by running:

docker ps -a | grep clive

• You should see no output. All CLIVE Phase 2 resources have been removed.

Nice Work!

Nice Work!

Nice work. You have built a production-grade RAG infrastructure for the CLIVE platform that CVC staff can use to upload any company document and get accurate, cited answers from the AI.

You've learned how to:

• Deploy PostgreSQL with PGVector and Apache Tika as production replacements for Open WebUI's crash-prone defaults, giving your knowledge base a stable foundation that handles thousands of documents.
• Configure fully local Ollama embeddings with nomic-embed-text, keeping all CVC data on your network while powering semantic search across uploaded documents.
• Diagnose retrieval failures on dense technical documents and resolve them by enabling hybrid search (BM25 + vector) with tuned chunk parameters for precise, keyword-aware retrieval.
• (Secret Mission) Built an automated PGVector backup and restore script with integrity verification and rotation, protecting the entire knowledge base from data loss.

Ready to quiz yourself?