Chat with Any Codebase Using RAG

Build a local RAG pipeline to query any GitHub repo with cited answers.

Introduction

30 Second Summary

Ever opened a massive codebase and wished you could just ask "how does authentication work here?" instead of digging through dozens of files? Large language models can answer general coding questions, but they hallucinate when asked about code they have never seen.

In this project, you will build a command-line tool that ingests any GitHub repository into a local vector database and lets you ask natural-language questions about the code. Every answer comes grounded in the actual source files, with citations pointing to exact file paths and line numbers.

What You'll Build

You'll paste a GitHub URL into your terminal, watch the tool clone and index the entire codebase, then have a conversation where the LLM answers using real code from the repo and tells you exactly which files its answer came from.

By the end of this project, you'll have:

  • A working RAG pipeline that clones any GitHub repo, splits code into chunks, converts them into vector embeddings, and stores them in a searchable database.
  • A chat interface where you ask questions about unfamiliar code and get answers citing exact file paths and line numbers from the repository.
  • Firsthand proof that LLMs hallucinate without context, by comparing a blind query to a retrieval-powered one using the same model.
  • Secret Mission: File-type filtering so you can narrow searches to only Python files, only JavaScript files, or any specific extension.

Are there any prerequisites?

You should be comfortable with Python fundamentals like functions, file I/O, and running scripts from the command line. You also need Ollama already installed on your machine. If you do not have it yet, grab it from ollama.com before starting.

Before We Start

Before we dive in, take a moment to lock in what you're building and which codebase you want to explore. Having a specific repository and question in mind will make every step feel purposeful.

Set Up Your Tools

Your RAG pipeline needs two things to work: a vector database to store code as searchable embeddings, and AI models to generate those embeddings and produce answers.

In this step, you'll install the Python packages and pull the AI models that power the entire system.

In this step, get ready to:
  • Install the chromadb and ollama Python packages.
  • Pull the llama3.2 and nomic-embed-text models.
  • Create your project directory and verify the setup.
Install Python packages

ChromaDB is the vector database that stores your code as searchable embeddings. The ollama Python package lets your scripts communicate with the Ollama models running on your machine.

  • Install both packages by running this command in your terminal:
pip install chromadb ollama

You'll see pip downloading and installing both packages along with their dependencies. This may take a minute.

Seeing a permissions error or command not found?

If pip is not recognized, try pip3 instead. On some systems, Python 3 uses pip3 as the default command.

If you see a permissions error on macOS or Linux, add --user to the end of the command to install into your user directory.

Help me troubleshoot this pip install error.

Pull the AI models

Your pipeline uses two separate models for different jobs. One generates human-readable answers. The other converts text into numerical vectors for similarity search.

  • Pull the language model by running:
ollama pull llama3.2

This downloads the llama3.2 model (2GB). You'll see a progress bar as it downloads.

  • Once that finishes, pull the embedding model by running:
ollama pull nomic-embed-text

This downloads nomic-embed-text (274MB). It finishes much faster than the first model.

Why two separate models?

The llama3.2 model is a language model that reads code context and generates human-readable answers. It handles the "generation" part of Retrieval-Augmented Generation.

The nomic-embed-text model is an embedding model that converts text into numerical vectors. These vectors let you find code that is semantically similar to a question, even if the exact words differ.

Seeing a connection error?

Make sure Ollama is running in the background. It serves models on http://localhost:11434 by default. If you closed it, reopen the Ollama application.

Help me check if Ollama is running.

Create your project directory and verify

With both models downloaded, you need a project folder to hold the scripts you'll write in the next steps.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir codebase-chat
cd codebase-chat
  • Confirm the folder exists by running:
ls

You should see an empty directory. This is where all your project files will live.

Before you run the next command, what do you think you'll see in the output? How many models should appear?

  • Verify both models are ready by running:
ollama list

You should see both llama3.2 and nomic-embed-text listed in the NAME column, along with their sizes and download dates.

Only see one model or neither?

If a model is missing, re-run the pull command for that specific model. The download may have been interrupted.

If ollama list itself fails, confirm Ollama is running by checking for its icon in your system tray (Windows) or menu bar (macOS).

Help me figure out what went wrong with the model download.

Your tools are locked and loaded. Next up, you'll see exactly why these models need retrieval to answer questions about code they have never seen.

Ask the LLM Without Code Context

Your tools are installed and both AI models are ready to go. Before building the retrieval system, you need to see the exact problem it solves.

In this step, you'll ask the LLM about your chosen codebase without giving it any actual code to reference. The response reveals a fundamental limitation: language models cannot describe code they have never seen.

In this step, get ready to:
  • Write a script that queries llama3.2 about a GitHub repository.
  • Run the script and observe how the LLM responds without any code context.
Write a direct LLM query

You'll create a short script that sends your question about a codebase straight to llama3.2 with nothing but the repo URL. The model has no way to look at the actual code.

  • Create a file called ask_llm.py inside your codebase-chat folder by running:
touch ask_llm.py
  • Confirm the file was created by running:
ls

You should see ask_llm.py listed in the output.

  • Open ask_llm.py in VS Code and paste the following code:
import ollama

# Collect the repo URL and question from the user
repo_url = input("Enter the GitHub repo URL: ")
question = input("Ask a question about this codebase: ")

# Send the question to llama3.2 with only the URL as context
response = ollama.chat(
    model='llama3.2',
    messages=[
        {
            'role': 'user',
            'content': f'I have a codebase from {repo_url}. {question}'
        }
    ],
)

# Display the LLM's answer
print("\nLLM Response (without seeing the code):")
print(response.message.content)

What does this code do?

  • input() prompts you to type a GitHub URL and a question when the script runs.
  • ollama.chat() sends your question to the local llama3.2 model. The only context the model receives is the URL string and your question.
  • The model never clones or reads the repository. It can only guess based on patterns from its training data.
  • Save ask_llm.py.

✔️ Awesome, I've got everything!

Great. Make sure you've saved the file before moving on.

ⓧ I'd like to double check the full code

import ollama

repo_url = input("Enter the GitHub repo URL: ")
question = input("Ask a question about this codebase: ")

response = ollama.chat(
    model='llama3.2',
    messages=[
        {
            'role': 'user',
            'content': f'I have a codebase from {repo_url}. {question}'
        }
    ],
)

print("\nLLM Response (without seeing the code):")
print(response.message.content)
Run the script and examine the response

The script will prompt you for two inputs: a GitHub URL and a question about that codebase. Use the same repository URL you chose in Step 0 and ask a specific question about its internals.

Before you run it, think about this: the model has never cloned or opened that repository. Do you think it can accurately describe the file structure or name real functions from the code?

  • Run your script by entering this command:
python ask_llm.py
  • When prompted, paste your GitHub repository URL and press Enter.
  • Type a specific question about the codebase (for example: "how does authentication work?" or "what is the project structure?") and press Enter.

Read the response carefully. The LLM either invents file names that do not exist, describes architecture it made up, or gives a generic textbook answer that could apply to any project.

Why does the LLM get it wrong?

This is the hallucination problem. The model was trained on public code, so it recognises patterns and can guess plausible-sounding answers. But it cannot see code it was not trained on.

When you ask about a specific repository, the model fills gaps with fabricated details rather than admitting it does not know. This is exactly the problem that RAG (Retrieval-Augmented Generation) solves by feeding real code into the prompt.

Seeing a connection error?

  • Make sure Ollama is running. Open a separate terminal and run ollama serve if it is not already active.
  • Confirm llama3.2 is pulled by running ollama list and checking it appears in the output.
  • Help me troubleshoot my Ollama connection

You have now seen the problem firsthand. Next, you'll clone the repository, break its code into chunks, and store those chunks in a vector database so the LLM can actually reference real code when answering.

Ingest a Codebase into ChromaDB

In Step 2, you saw the problem firsthand. The LLM either hallucinated file names that do not exist or gave a generic answer that could apply to any codebase.

The fix is retrieval. In this step, you will clone your chosen repository, break every code file into searchable chunks, convert each chunk into a vector embedding, and store everything in a local ChromaDB database. Once the code lives in the vector database, you can query it by meaning rather than exact keyword matches.

In this step, get ready to:
  • Write an ingestion script that clones, chunks, embeds, and stores a codebase.
  • Run the script against your chosen GitHub repository.
  • Verify that ChromaDB has your code stored and ready to query.
Write the ingestion script

The ingestion pipeline has four stages: clone the repo, find code files, split them into overlapping chunks, and embed each chunk into the vector database. You will build each stage as a function in a single file called ingest.py.

  • Create a new file called ingest.py inside your codebase-chat folder.
  • Add the imports, configuration constants, and the first two helper functions by pasting this code:
import os
import shutil
import subprocess
import sys
import ollama
import chromadb

# File types to include when scanning the repo
EXTENSIONS = {'.py', '.js', '.ts', '.md', '.txt', '.html', '.css', '.java', '.go', '.rs', '.jsx', '.tsx'}
CHUNK_SIZE = 60  # Lines per chunk
CHUNK_OVERLAP = 10  # Overlapping lines between chunks

def clone_repo(url, dest="./repo"):
    # Remove old clone if it exists, then shallow-clone the repo
    if os.path.exists(dest):
        shutil.rmtree(dest, ignore_errors=True)
    subprocess.run(["git", "clone", "--depth", "1", url, dest], check=True)
    return dest

def get_code_files(repo_path):
    # Walk the file tree, skipping non-essential directories
    code_files = []
    skip_dirs = {'.git', 'node_modules', '__pycache__', 'venv', '.venv', 'dist', 'build'}
    for root, dirs, filenames in os.walk(repo_path):
        dirs[:] = [d for d in dirs if d not in skip_dirs and not d.startswith('.')]
        for filename in filenames:
            ext = os.path.splitext(filename)[1]
            if ext in EXTENSIONS:
                code_files.append(os.path.join(root, filename))
    return code_files

What does this code do?

  • EXTENSIONS defines which file types count as "code" worth indexing. Anything not in this set is ignored.
  • CHUNK_SIZE and CHUNK_OVERLAP control how files are split. Each chunk is roughly 60 lines, and adjacent chunks share 10 lines so that a function spanning a boundary is not cut in half.
  • clone_repo does a shallow clone (--depth 1) to grab only the latest snapshot without full history.
  • get_code_files walks the directory tree, skipping noise like node_modules and __pycache__, and returns only files with code-like extensions.
  • Add the chunking function and the embedding helper below get_code_files in ingest.py:
def chunk_file(filepath, repo_path):
    # Read the file and split into overlapping chunks
    try:
        with open(filepath, 'r', encoding='utf-8', errors='ignore') as f:
            lines = f.readlines()
    except Exception:
        return []

    relative_path = os.path.relpath(filepath, repo_path)
    chunks = []

    for i in range(0, len(lines), CHUNK_SIZE - CHUNK_OVERLAP):
        chunk_lines = lines[i:i + CHUNK_SIZE]
        if not chunk_lines:
            break
        content = ''.join(chunk_lines)
        if content.strip():
            chunks.append({
                'content': content,
                'filepath': relative_path,
                'start_line': i + 1,
                'end_line': i + len(chunk_lines),
            })

    return chunks

def embed_texts(texts):
    # Convert text chunks into vector embeddings using nomic-embed-text
    response = ollama.embed(model='nomic-embed-text', input=texts)
    return response.embeddings

What does this code do?

chunk_file reads each file line by line and produces overlapping slices. Each chunk records its file path and line numbers so you can trace answers back to exact source locations later.

embed_texts sends a batch of text strings to nomic-embed-text via Ollama and returns one vector per string. These vectors are what ChromaDB uses to find code that is semantically similar to a question.

  • Save ingest.py and confirm the module loads without syntax errors by running:
python -c "import ingest; print('Module loaded OK')"

You should see Module loaded OK printed with no errors.

Seeing a syntax error?

  • Check that the chunks.append({ block inside chunk_file uses consistent indentation. Mixing tabs and spaces causes Python to reject the file.
  • Make sure embed_texts is at the same indentation level as the other functions (not indented inside chunk_file).

Help me debug a syntax error in my ingest.py file

  • Now add the main function below embed_texts in ingest.py. This orchestrates the full pipeline:
def main():
    if len(sys.argv) < 2:
        print("Usage: python ingest.py <github_url>")
        sys.exit(1)

    url = sys.argv[1]
    repo_path = "./repo"

    # Clone the repository
    print(f"Cloning {url}...")
    clone_repo(url, repo_path)

    # Find and chunk all code files
    print("Finding code files...")
    files = get_code_files(repo_path)
    print(f"Found {len(files)} code files.")

    print("Chunking files...")
    all_chunks = []
    for filepath in files:
        chunks = chunk_file(filepath, repo_path)
        all_chunks.extend(chunks)
    print(f"Created {len(all_chunks)} chunks.")

    if not all_chunks:
        print("No code chunks found. Check the repository URL.")
        sys.exit(1)

if __name__ == "__main__":
    main()

What does main() do so far?

The function reads the GitHub URL from the command line, calls your helper functions in sequence, and prints progress at each stage. If no code files are found (wrong URL, empty repo), it exits early with a helpful message.

The if __name__ == "__main__" guard means the script only runs main() when executed directly, not when imported.

  • Save ingest.py and do a quick test run with your GitHub URL:
python ingest.py [[REPO_URL="https://github.com/username/repo"]]

You should see the script clone the repo, count files, and count chunks. It then exits because the storage step is not written yet.

Seeing 'fatal: repository not found'?

  • Double-check that your URL is a valid, public GitHub repository. Private repos require authentication that this script does not handle.
  • Make sure you include the full URL starting with https://github.com/.

Help me fix my git clone error when running ingest.py

  • Finally, add the ChromaDB storage code. In ingest.py, find the line if __name__ == "__main__": at the bottom and paste the following code directly above it, inside the main() function (keep the indentation level matching the if not all_chunks block above it):
    print("Embedding and storing in ChromaDB...")
    client = chromadb.PersistentClient(path="./chroma_db")

    try:
        client.delete_collection("codebase")
    except Exception:
        pass

    collection = client.create_collection(name="codebase")

    batch_size = 50
    for i in range(0, len(all_chunks), batch_size):
        batch = all_chunks[i:i + batch_size]
        texts = [chunk['content'] for chunk in batch]
        embeddings = embed_texts(texts)
        collection.add(
            ids=[f"chunk_{i + j}" for j in range(len(batch))],
            embeddings=embeddings,
            documents=texts,
            metadatas=[
                {
                    'filepath': chunk['filepath'],
                    'start_line': chunk['start_line'],
                    'end_line': chunk['end_line'],
                }
                for chunk in batch
            ],
        )
        print(f"  Stored {min(i + batch_size, len(all_chunks))}/{len(all_chunks)} chunks...")

    print(f"\nDone! Stored {collection.count()} chunks in ChromaDB.")

What does the storage code do?

PersistentClient creates a ChromaDB database at ./chroma_db that survives between script runs. Data is saved to disk automatically.

The script deletes any existing codebase collection first so re-running ingestion starts fresh. Then it processes chunks in batches of 50, embedding each batch with nomic-embed-text before storing the vectors, documents, and metadata (file path, line numbers) together.

  • Save ingest.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved ingest.py and that the storage code sits inside main() between the early exit check and if __name__ == "__main__":.

ⓧ I'd like to double check the full code

Since ingest.py is a long file, confirm you have these sections in order from top to bottom:

  • Imports (os, shutil, subprocess, sys, ollama, chromadb).
  • Constants: EXTENSIONS, CHUNK_SIZE = 60, CHUNK_OVERLAP = 10.
  • clone_repo function using git clone --depth 1.
  • get_code_files function walking the tree with os.walk.
  • chunk_file function splitting files into overlapping chunks.
  • embed_texts function calling ollama.embed.
  • main() function: arg check, clone, find files, chunk, early exit, then ChromaDB storage ending with the Done print.
  • if __name__ == "__main__": main() at the very bottom.

If anything is out of order or missing, scroll up and copy the relevant chunk into place.

Run ingestion on your chosen repo

With the full script assembled, it is time to run the complete ingestion pipeline.

  • Before you run this, predict: how many chunks do you think your repository will produce? A small repo might produce 20-50. A medium one could produce several hundred.
  • Run the full ingestion by executing:
python ingest.py [[REPO_URL="https://github.com/username/repo"]]

Watch the output as it progresses through each stage. You should see something like:

Cloning https://github.com/username/repo...
Finding code files...
Found 14 code files.
Chunking files...
Created 87 chunks.
Embedding and storing in ChromaDB...
  Stored 50/87 chunks...
  Stored 87/87 chunks...

Done! Stored 87 chunks in ChromaDB.

Why does embedding take a moment?

Each batch of 50 chunks is sent to nomic-embed-text running locally on your machine. The model converts each chunk into a 768-dimensional vector. Depending on your hardware, each batch takes a few seconds.

Script hangs or crashes during embedding?

  • Make sure Ollama is still running. Open a separate terminal and run ollama list to confirm it responds.
  • If you chose a very large repository (thousands of files), the process may take several minutes. Try a smaller repo first to validate the pipeline works.
  • On Windows, if you see a permission error during shutil.rmtree, close any editors or terminals that have the ./repo folder open, then re-run.

Help me fix my ingest.py script that hangs or crashes during the embedding step

Verify the stored data

The final print confirmed a count, but you can double-check independently that ChromaDB persisted your data by querying the collection directly.

  • Confirm the data is stored by running this quick check:
python -c "import chromadb; client = chromadb.PersistentClient(path='./chroma_db'); coll = client.get_collection('codebase'); print(f'Chunks in database: {coll.count()}')"

You should see a non-zero count matching the number from the ingestion output.

Seeing 0 chunks or a collection error?

  • Make sure you are running this command from the codebase-chat directory. The ./chroma_db path is relative to where you run the command.
  • If you see an error about the collection not existing, re-run python ingest.py with your URL. The collection is only created during ingestion.

Help me fix my ChromaDB verification that shows 0 chunks or a missing collection

Your codebase is now searchable. Every chunk of code is stored as a vector that can be compared against any question you ask. Next up, you will build the chat interface that retrieves relevant chunks and feeds them to the LLM so it answers from real code instead of guessing.

Build RAG-Powered Code Chat

Your codebase is now indexed. Every code file has been chunked, embedded, and stored as searchable vectors in ChromaDB.

Now you can close the loop. When you ask a question, the system will embed your question, find the most similar code chunks, and inject that real code into the LLM prompt. The LLM answers from actual source code instead of guessing.

In this step, get ready to:
  • Write a retrieval-augmented chat script that queries ChromaDB and prompts llama3.2 with real code context.
  • Run the same question from Step 2 and compare the grounded answer to the hallucinated one.
  • Explore different questions to see how semantic search surfaces different code chunks each time.
Write the RAG chat script

This script has three parts: a retrieval function that finds relevant code, a prompt builder that formats retrieved code for the LLM, and a chat loop that ties everything together.

  • Create a new file called chat.py inside your codebase-chat folder.
  • Add the imports and the retrieval function by pasting this code:
import ollama
import chromadb


# Embed the question and find the most similar code chunks
def get_relevant_chunks(question, collection, n_results=5):
    query_embedding = ollama.embed(model='nomic-embed-text', input=question).embeddings[0]
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
    )
    return results

What does this code do?

The get_relevant_chunks function takes the user's question and converts it into a vector using nomic-embed-text. It then asks ChromaDB to find the 5 code chunks whose vectors are most similar to the question vector.

This is the "retrieval" in Retrieval-Augmented Generation. Instead of asking the LLM to recall information from training, you fetch the actual relevant code and hand it over directly.

  • Save chat.py.
  • Verify the retrieval function loads without errors by running this command:
python -c "from chat import get_relevant_chunks; print('Retrieval function loaded')"

You should see Retrieval function loaded printed in your terminal.

Seeing an ImportError?

Make sure you saved chat.py in the codebase-chat folder (the same directory you're running the command from). Check for typos in the import lines at the top of the file.

Help me debug my chat.py import error

  • Next, add the prompt builder function below get_relevant_chunks in chat.py:
# Format retrieved code chunks into a structured prompt for the LLM
def build_prompt(question, results):
    context_parts = []
    for doc, metadata in zip(results['documents'][0], results['metadatas'][0]):
        source = f"{metadata['filepath']} (lines {metadata['start_line']}-{metadata['end_line']})"
        context_parts.append(f"--- Source: {source} ---\n{doc}")

    context = "\n\n".join(context_parts)

    prompt = f"""You are a code assistant. Answer the question based ONLY on the following code context.
If the context doesn't contain enough information to answer, say so.
Always cite which file(s) your answer comes from.

Context:
{context}

Question: {question}"""

    return prompt

What does this code do?

The build_prompt function assembles the retrieved code chunks into a single prompt string. Each chunk is labelled with its source file and line numbers.

The prompt instructs the LLM to answer ONLY from the provided context and to cite sources. This constraint prevents the model from falling back to hallucination when the context is insufficient.

  • Now add the main function and entry point below build_prompt in chat.py:
def main():
    # Connect to the persistent ChromaDB and load the codebase collection
    client = chromadb.PersistentClient(path="./chroma_db")
    try:
        collection = client.get_collection("codebase")
    except Exception:
        print("No codebase found. Run ingest.py first.")
        return
    print(f"Loaded collection with {collection.count()} chunks.")
    print("Chat with your codebase! Type 'quit' to exit.\n")
    # Chat loop: take questions, retrieve relevant chunks, generate answers
    while True:
        question = input("You: ").strip()
        if question.lower() in ('quit', 'exit'):
            break
        if not question:
            continue
        results = get_relevant_chunks(question, collection)
        prompt = build_prompt(question, results)
        response = ollama.chat(
            model='llama3.2',
            messages=[{'role': 'user', 'content': prompt}],
        )
        print(f"\nAssistant: {response.message.content}\n")
        print("Sources:")
        for metadata in results['metadatas'][0]:
            print(f"  - {metadata['filepath']} (lines {metadata['start_line']}-{metadata['end_line']})")
        print()


if __name__ == "__main__":
    main()

What does this code do?

The main function connects to the existing ChromaDB collection you created in Step 3. If the collection does not exist, it exits early with a helpful message.

The chat loop takes each question, retrieves the 5 most relevant chunks, builds the augmented prompt, sends it to llama3.2, and prints both the answer and the source files the answer came from.

  • Save chat.py.

Seeing a syntax error?

The most common issue is the triple-quoted f-string in build_prompt. Make sure the closing """ is on the same line as Question: {question} with no extra whitespace after it.

Help me fix a syntax error in my chat.py

✔️ Awesome, I've got everything!

Great. Double check you've saved chat.py and you're ready to test.

ⓧ I'd like to double check the full code

Scroll up and confirm your chat.py contains all three sections in this order:

  • Imports (ollama and chromadb) plus the get_relevant_chunks function.
  • The build_prompt function with the triple-quoted f-string prompt.
  • The main function with the chat loop, followed by the if __name__ == "__main__" guard at the bottom.
Test with the same question from Step 2

This is the moment of truth. You are about to ask the exact same question that produced a hallucinated or generic answer in Step 2.

  • Before you run the script, think about what you expect. Do you think the answer will reference actual file names from the repository this time? Why or why not?
  • Run the chat script by running this command:
python chat.py
  • When you see the You: prompt, type the exact same question you asked in Step 2.

The answer now references real file names and line numbers from the repository. Compare this to the output from ask_llm.py in Step 2. The earlier answer was generic or fabricated. This answer is grounded in actual code the LLM can see.

Why is the answer so different now?

In Step 2, the LLM had zero code to reference. It could only guess based on its training data. Now, your RAG pipeline retrieves the 5 most relevant code chunks from the actual repository and injects them directly into the prompt.

The LLM reads real source code and cites the files its answer comes from. This is the core value of retrieval-augmented generation: grounding AI responses in real data instead of relying on memorization.

  • Type quit to exit the chat loop.

Getting generic answers without file references?

Make sure you ran ingest.py successfully in Step 3 before running chat.py. The script needs the chroma_db folder to exist with your indexed code chunks.

If you see "No codebase found", re-run ingest.py with your GitHub URL from Step 3.

Help me debug why chat.py isn't citing real files

Explore with different questions

Each question you ask triggers a different semantic search. The retrieval finds code chunks that are conceptually similar to your question, even if they do not share the exact same words.

  • Run chat.py again and try several different questions. For example:
  • "Where are the routes defined?"
  • "How does error handling work?"
  • "What dependencies does this project use?"
  • Notice how the Sources section changes with each question. Different questions pull different code chunks from the vector database.
  • Type quit when you are done exploring.

Why do different questions return different sources?

When you ask about "routes", the embedding model converts that into a vector that is mathematically close to code chunks containing route definitions, URL handlers, and path logic. When you ask about "error handling", the vector lands near try/except blocks and error classes.

This is semantic search in action. It finds meaning, not just keyword matches.

Your RAG pipeline is fully working. You now have a tool that ingests any GitHub repository and lets you have a grounded conversation about its code. Next up, you can extend it with file-type filtering in the Secret Mission.

Secret mission

Add File-Type Filtering

Your RAG tool searches all code files at once. What if you only want to look at Python files, or just the README? Add a file-type filter so you can narrow searches to specific extensions.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing costs.

Resources you used:

  • Project folder codebase-chat/ on your Desktop (contains all scripts, the cloned repo, and the ChromaDB database).
  • Ollama models: llama3.2 (2GB) and nomic-embed-text (274MB).
  • Python packages: chromadb and ollama.

Keep everything running

No action needed. Choose this if you plan to ingest more repositories or continue experimenting with your RAG pipeline.

  • Your codebase-chat/ folder stays on your Desktop with all scripts and the ChromaDB database intact.
  • The llama3.2 and nomic-embed-text models remain available in Ollama for future use.
  • The chromadb and ollama Python packages remain installed.

Pause - I'll come back to this later

Free up disk space from the cloned repository while keeping your scripts and indexed database for later.

  • Delete the cloned repository to free space. Your ChromaDB database already has the indexed chunks, so you do not need the raw clone.
  • On macOS/Linux, run:
rm -rf ~/Desktop/codebase-chat/repo
  • On Windows, run:
rmdir /s /q %USERPROFILE%\Desktop\codebase-chat\repo
  • Your scripts (ask_llm.py, ingest.py, chat.py) and the chroma_db/ folder remain untouched so you can pick up where you left off.
  • Ollama models and Python packages stay installed for when you return.

Delete - I don't want to use this again

Remove all project resources and start fresh. This deletes everything you built in this project.

  • Delete the entire project folder. On macOS/Linux, run:
rm -rf ~/Desktop/codebase-chat
  • On Windows, run:
rmdir /s /q %USERPROFILE%\Desktop\codebase-chat
  • Remove the Ollama models to free up 2.3GB of disk space:
ollama rm llama3.2
ollama rm nomic-embed-text
  • Uninstall the Python packages:
pip uninstall -y chromadb ollama

Nice Work!

Nice Work!

You just built a fully local RAG-powered code assistant from scratch. Point it at any GitHub repository, ask questions in plain English, and get answers grounded in the actual source code with file-and-line citations.

You've learned how to:

  • Build a complete RAG pipeline from scratch: cloning a repo, chunking code into searchable pieces, embedding with nomic-embed-text, storing in a vector database, retrieving relevant context, and generating grounded answers.
  • Prove that LLMs hallucinate without context and that retrieval-augmented generation fixes this by injecting real code into the prompt.
  • Use Ollama for local inference and ChromaDB as a persistent local vector database for semantic search, all at zero cost.
  • Secret Mission: Added metadata-based filtering to narrow searches by file type using ChromaDB's where clause.

Ready to quiz yourself?