Build a Local AI Engineering Workspace

Set up a typed Python project that talks to a local LLM via Ollama.

Introduction

30 Second Summary

Every AI engineer's first few projects start the same way: one giant script with hardcoded values, print statements for debugging, and no version control. It works on your machine, but you'd never show it to a hiring manager.

In this project, you will build a production-style Python workspace that communicates with a local LLM running on your own hardware. You'll add typed configuration, structured logging, and a reusable client class that mirrors how real AI engineering teams structure their code.

What You'll Build

Run a single command and watch your local qwen2.5:7b model answer a sustainability question directly in your terminal, with structured log lines showing connection status and response timing.

By the end of this project, you'll have:

  • A Git-tracked project you can push to GitHub, with a professional folder structure that signals production engineering to anyone reviewing your code.
  • Typed settings powered by Pydantic that load from a .env file. Change the model name or temperature without editing a single line of Python.
  • A reusable Ollama client that logs every request, handles connection failures gracefully, and returns structured responses you can pass to any downstream component.
  • Secret Mission: Add an interactive chat loop that maintains conversation history across turns.

Are there any prerequisites?

This project assumes you already have Python 3.12.7 installed, VS Code set up as your editor, and Ollama running with the qwen2.5-coder:7b model already pulled.

You'll install Git and the project's Python packages in Step 1.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters for your AI Engineering portfolio. This is your moment to commit to the goal before any hands-on work begins.

Install Git and Project Dependencies

Without Git and the right packages, you cannot track changes or build typed, production-quality code.

This step gets your development environment operational so every subsequent step produces real, trackable output. You will install Git, configure it with your identity, and add three Python packages to your virtual environment.

In this step, get ready to:
  • Install Git for Windows and configure your identity.
  • Install project dependencies into the existing rag_1 virtual environment.
  • Verify the full toolchain is operational.
Install and configure Git

Git is the version control system used by virtually every software team. It tracks every change you make so you can undo mistakes, review history, and collaborate later.

  • Open a terminal by pressing the Windows key, typing PowerShell, and pressing Enter.
  • Check whether Git is already installed by running this command:
git --version

✔️ I see a version number

Git is already installed. Skip ahead to the configuration step below.

ⓧ Command not found

Git is not installed yet. Install it using Windows Package Manager by running this command:

winget install --id Git.Git -e --source winget

What happens during installation?

The Git for Windows installer runs with its default settings. It will add git to your system PATH so you can use it from any terminal. The process takes about a minute.

If winget prompts you to agree to source agreements, type Y and press Enter.

  • After installation completes, close your PowerShell window completely.
  • Open a fresh PowerShell window (press the Windows key, type PowerShell, press Enter).
  • Verify the installation by running:
git --version

You should see a version number printed, confirming Git is ready.

Still seeing 'command not found'?

Make sure you opened a brand-new PowerShell window after the install. The old window does not pick up PATH changes.

If it still fails, try restarting your computer, then open a new PowerShell and run git --version again.

help me troubleshoot why Git is not found after installing with winget on Windows

Now configure Git with your name and email. These values appear in every commit you make.

  • Set your name by running this command (replace the placeholder with your actual name):
git config --global user.name [[YOURNAME="your full name"]]
  • Set your email by running this command (replace the placeholder with your actual email):
git config --global user.email [[YOUREMAIL="your email address"]]
  • Verify your configuration by running:
git config --global --list

You should see your user.name and user.email listed in the output.

Why does Git need my name and email?

Every Git commit records who made the change. This is metadata attached to the commit history, not an account or login. You can use any name and email you want for a local project.

Install project dependencies into your virtual environment

Your rag_1 virtual environment already exists on your machine, but it is not yet active. You need to activate it first so that packages install into the right place.

  • Activate your virtual environment by running this command:
rag_1\Scripts\Activate.ps1

Your terminal prompt now shows (rag_1) at the beginning, confirming the environment is active.

Seeing a script execution error?

Windows may block scripts by default. Run this command first, then try activating again:

Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned

help me fix a PowerShell execution policy error when activating a Python virtual environment

  • Install the three project packages by running:
pip install httpx==0.28.1 pydantic-settings==2.14.2 ruff==0.11.12

What are these packages?

httpx is a modern HTTP client. You will use it to send requests to Ollama's API. pydantic-settings lets you define typed configuration that loads from a .env file. ruff is a fast Python linter and formatter that catches errors before you run your code.

  • Confirm the packages installed correctly by running:
pip list

You should see httpx, pydantic-settings, and ruff in the list with their version numbers.

Package installation failed?

Make sure your prompt starts with (rag_1). If it does not, your virtual environment is not active and pip may target the wrong Python installation.

help me troubleshoot pip install failures in my Python virtual environment

Verify the full toolchain

Before moving on, confirm that every tool in your stack is operational. This single check covers Python, Git, Ollama, and your installed packages.

  • Before you run the next command, predict what you expect to see. Will all four tools report successfully?
  • Run this combined verification:
python --version && git --version && ollama --version && pip list | findstr "httpx pydantic-settings ruff"

You should see output similar to this:

  • Python 3.12.7 (or your installed patch version).
  • A Git version number.
  • An Ollama version number.
  • Three lines showing httpx, pydantic-settings, and ruff with their pinned versions.

One of the tools not showing up?

If ollama --version fails, make sure Ollama is running. Open it from your Start menu if needed.

If python --version shows the wrong version, you may need to use python3 instead, or check that your virtual environment is still active.

help me figure out why one of my tools is not showing in the verification output

Your development environment is fully operational. Next up, you will scaffold a professional project structure and make your first Git commit.

Scaffold the Project and Make Your First Commit

Your tools are installed and ready. Now you need a place for your code to live.

A professional project structure tells hiring managers you understand software engineering, not just AI. Without it, code becomes unmaintainable the moment you add a second module. In this step, you'll create a production-style folder layout, track it with Git, and configure environment variables the right way.

In this step, get ready to:
  • Create a professional folder structure with properly organized modules.
  • Initialize a Git repository and make your first commit.
  • Set up environment-based configuration with a .env file.
Create the project folder structure

Every real AI engineering repository follows a predictable layout. Reviewers and collaborators know where to find things because the structure follows conventions.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir esg-rag
cd esg-rag
  • Create the directory structure and placeholder files by running these commands:
mkdir app/config, app/services, data/raw, data/processed, tests
New-Item -ItemType File -Path app/__init__.py, app/config/__init__.py, app/services/__init__.py
New-Item -ItemType File -Path tests/__init__.py, tests/test_placeholder.py
New-Item -ItemType File -Path data/raw/.gitkeep, data/processed/.gitkeep

What does this create?

  • app/ holds your application code, split into config/ for settings and services/ for business logic.
  • __init__.py files mark each folder as a Python package so you can import between them.
  • data/ stores raw documents and processed outputs (the RAG pipeline's fuel).
  • tests/ holds your test files. The placeholder test_placeholder.py gives the folder structure from day one so tests are never an afterthought.
  • .gitkeep files are empty placeholders that force Git to track otherwise-empty directories.
  • Verify your folder structure by running:
tree /f

You should see the full tree with app/, data/, and tests/ directories, each containing their placeholder files.

Not seeing the folders?

Make sure you ran cd esg-rag before running tree /f. You need to be inside the project folder.

If the mkdir command gave an error about paths, try running each directory creation separately: mkdir app, then mkdir app/config, and so on.

help me create the project folder structure on Windows

  • Open your project in VS Code. Press the Windows key, type VS Code, and press Enter to launch it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the esg-rag folder, and click Select Folder.

You should see your folder structure in the Explorer sidebar on the left.

  • Open the integrated terminal in VS Code by pressing Ctrl+` (backtick key, usually above Tab).
Create the .gitignore file

A .gitignore file tells Git which files to never track. You want to exclude compiled Python files, your virtual environment, and your secret .env file.

  • In VS Code's Explorer sidebar, right-click in the empty space below your folders and select New File.
  • Name it .gitignore and press Enter.
  • Paste the following content into the file:
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
dist/
*.egg-info/
*.egg

# Virtual environment
venv/
.venv/
rag_1/

# Environment variables
.env

# IDE
.vscode/
.idea/

# OS
Thumbs.db
.DS_Store

# Data
data/raw/*
data/processed/*
!data/raw/.gitkeep
!data/processed/.gitkeep

What does this file do?

  • __pycache__/ and *.py[cod] exclude compiled Python bytecode that gets regenerated automatically.
  • rag_1/ excludes your virtual environment folder from version control.
  • .env keeps your secret configuration out of Git (critical for API keys later).
  • The data/ rules exclude data files but keep the .gitkeep placeholders so the directory structure is preserved.
  • Save the file with Ctrl+S.
Create the README and pyproject.toml

A README.md is the first thing anyone sees when they visit your repository. A pyproject.toml is the modern standard for Python project configuration. It declares your dependencies, metadata, and tool settings in one place.

  • Create a new file in the Explorer sidebar called README.md.
  • Paste the following content at the top of the file:
# ESG RAG System

A production-inspired Retrieval-Augmented Generation system for sustainability and ESG reports. Built as an AI Engineering portfolio project demonstrating clean architecture, typed configuration, and local LLM integration.

## Setup

1. Install Python 3.12+
2. Create and activate a virtual environment
3. Install in editable mode: `pip install -e .`
4. Copy `.env.example` to `.env` and configure
5. Ensure Ollama is running with `qwen2.5:7b` model
  • Below the Setup section, paste the rest of the README:
## Usage

```bash
# Single question
python main.py

# Interactive chat
python main.py --chat
```

## Project Structure

```
app/
  config/       - Configuration and logging setup
  services/     - Service layer (LLM client, future: retrieval, ingestion)
data/
  raw/          - Source documents (PDFs)
  processed/    - Processed chunks and embeddings
tests/          - Test files
```

## Tech Stack

- Python 3.12
- Ollama (local LLM inference)
- Qwen2.5 7B (generation model)
- httpx (HTTP client)
- Pydantic (configuration and data validation)
- Ruff (linting and formatting)
  • Save README.md with Ctrl+S.
  • Create another new file in the Explorer sidebar called pyproject.toml.
  • Paste the following content:
[project]
name = "esg-rag"
version = "0.1.0"
description = "RAG system for ESG and sustainability reports"
requires-python = ">=3.12"
dependencies = [
    "httpx==0.28.1",
    "pydantic-settings==2.14.2",
]

[project.optional-dependencies]
dev = [
    "ruff==0.11.12",
    "pytest>=8.0",
]

[tool.ruff]
line-length = 88
target-version = "py312"

[tool.ruff.lint]
select = ["E", "F", "I"]

[tool.pytest.ini_options]
testpaths = ["tests"]

Why pyproject.toml instead of requirements.txt?

pyproject.toml is the official Python packaging standard (PEP 621). It replaces the old requirements.txt + setup.py combination with a single file that declares dependencies, metadata, and tool configuration.

The [project.optional-dependencies] section separates dev tools (Ruff, pytest) from production dependencies. The [tool.ruff] and [tool.pytest] sections configure those tools without needing extra config files.

  • Save pyproject.toml with Ctrl+S.

Your Explorer sidebar should now show .gitignore, README.md, and pyproject.toml at the root level alongside your app/, data/, and tests/ folders.

✔️ Awesome, I've got everything!

Great. Make sure all three files are saved before moving on.

ⓧ I'd like to double check the full code

Scroll up and confirm these are all present and saved:

  • .gitignore with Python, venv, .env, IDE, OS, and data exclusions.
  • README.md with Setup, Usage, Project Structure, and Tech Stack sections.
  • pyproject.toml with project metadata, pinned dependencies, and Ruff/pytest configuration.
  • Install the project in editable mode (including dev tools) by running this command in VS Code's terminal:
pip install -e ".[dev]"

What does editable mode do?

The -e flag installs your project as an editable package. Python registers the app/ folder so imports like from app.config.settings import get_settings work from anywhere in the project without PATH hacks.

The .[dev] part also installs the optional dev dependencies (Ruff and pytest) alongside the production ones.

  • Verify the install worked by running:
pip list | findstr "esg-rag"

You should see esg-rag listed with version 0.1.0 and an editable project location pointing to your Desktop folder.

Seeing an error during pip install?

  • Make sure your virtual environment rag_1 is active. You should see (rag_1) in your terminal prompt.
  • Confirm you are inside the esg-rag folder (the one containing pyproject.toml). Run ls pyproject.toml to check.

help me fix a pip install -e error with my pyproject.toml

Initialize Git and make your first commit

You have a clean project structure. Time to put it under version control so every change from here is tracked.

  • Initialize a Git repository in your project folder by running this command in VS Code's terminal:
git init

You should see a message confirming the repository was initialized.

  • Stage all your files and make the first commit by running these commands:
git add .
git commit -m "feat: scaffold project structure"

What just happened?

  • git add . stages every file in the folder (except those listed in .gitignore).
  • git commit -m saves a snapshot with a message. The feat: prefix is a convention called Conventional Commits that makes your history scannable.

Before you verify, what do you think git log will show?

  • Verify your commit exists by running:
git log --oneline

You should see one line with a short hash followed by feat: scaffold project structure.

Seeing an error about identity?

If Git says it needs your name or email, you may have skipped the configuration in Step 1. Run git config --global user.name "Your Name" and git config --global user.email "you@example.com", then try the commit again.

help me fix my Git identity configuration

Create environment configuration

Configuration values like your model name and API URL should live in a .env file, not scattered through your code. You'll create two versions: .env (your actual config, never committed) and .env.example (a template committed to Git so collaborators know what to set up).

  • Create a new file in the Explorer sidebar called .env.example.
  • Paste the following configuration values:
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=qwen2.5:7b
OLLAMA_TEMPERATURE=0.7
OLLAMA_MAX_TOKENS=1024
LOG_LEVEL=INFO

What do these settings control?

  • OLLAMA_BASE_URL points to your local Ollama server (default port 11434).
  • OLLAMA_MODEL specifies qwen2.5:7b, the general-purpose model for RAG generation.
  • OLLAMA_TEMPERATURE controls response randomness (0.7 is a balanced default).
  • OLLAMA_MAX_TOKENS limits response length to 1024 tokens.
  • LOG_LEVEL sets how verbose your application logs are.
  • Save .env.example with Ctrl+S.
  • Create another new file called .env (no 'example' suffix).
  • Paste the exact same content:
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=qwen2.5:7b
OLLAMA_TEMPERATURE=0.7
OLLAMA_MAX_TOKENS=1024
LOG_LEVEL=INFO
  • Save .env with Ctrl+S.

Now verify that your .gitignore is doing its job. The .env file should be invisible to Git.

  • Check what Git sees by running:
git status

You should see .env.example listed as an untracked file. The .env file should NOT appear at all. That means your .gitignore is working correctly.

  • Commit the environment template by running:
git add .env.example
git commit -m "chore: add environment configuration template"

Seeing .env in git status?

If .env appears in git status, your .gitignore might be missing the .env line. Open .gitignore and confirm it contains a line that says exactly .env under the # Environment variables section.

help me fix my .gitignore so .env is excluded

Do you think git log --oneline will now show one commit or two?

  • Confirm your full commit history by running:
git log --oneline

You should see two commits: the scaffold commit and the environment template commit.

  • Verify the folder structure one final time by running:
tree /f app

You should see app/ with __init__.py at each level: app/__init__.py, app/config/__init__.py, and app/services/__init__.py.

Your project has a professional skeleton with version control running. Next up, you'll write your first script that actually talks to your local LLM.

Talk to Your Local LLM

Your project structure is committed and version-controlled. Now it's time to prove the underlying infrastructure actually works.

Before building any abstraction layer on top of your LLM, you need direct evidence that your hardware, Ollama, and the model can communicate. A quick script that hits Ollama's HTTP endpoint and prints a response gives you that confidence. This is your first visible win.

In this step, get ready to:
  • Pull the qwen2.5:7b general-purpose model in Ollama.
  • Write a minimal test script that talks to Ollama's API.
  • Run it and see the model respond to a question in your terminal.
Pull the qwen2.5:7b model

You already have qwen2.5-coder:7b installed for your IDE coding assistant. The RAG system needs a different model optimized for natural language Q&A rather than code generation.

  • Download the general-purpose model by running this command:
ollama pull qwen2.5:7b

This downloads approximately 4.7 GB of model weights. It may take a few minutes depending on your internet connection.

  • Confirm the model is available by running:
ollama ls

You should see qwen2.5:7b listed alongside qwen2.5-coder:7b in the output.

Why two separate models?

Qwen2.5 7B (general) has stronger instruction-following for natural language Q&A. The Coder variant is optimized for code generation. You keep the Coder model for your IDE assistant and use the general model for RAG generation.

🙋‍♀️ Download seems stuck or fails?

Make sure Ollama is running in the background. If the pull stalls, press Ctrl+C and retry the command. Ollama resumes from where it left off.

help me troubleshoot why ollama pull is failing

Write a quick test script

Ollama exposes an OpenAI-compatible REST API on your machine. You can send it an HTTP POST request and get back a model response, just like calling OpenAI's API but entirely local and free.

  • Create a file called quick_test.py in your project root (the same folder that contains README.md) by running:
type nul > quick_test.py
  • Open quick_test.py in VS Code and paste the following code:
"""Quick test to verify Ollama is working. Delete after Step 3."""
import httpx

response = httpx.post(
    "http://localhost:11434/v1/chat/completions",
    json={
        "model": "qwen2.5:7b",
        "messages": [
            {"role": "user", "content": "What is ESG in one sentence?"}
        ],
        "stream": False,
    },
    timeout=60.0,
)

data = response.json()
print(data["choices"][0]["message"]["content"])

What does this code do?

  • import httpx loads the HTTP client library you installed in Step 1.
  • httpx.post(...) sends a POST request to Ollama's OpenAI-compatible chat endpoint at http://localhost:11434/v1/chat/completions.
  • The JSON payload specifies which model to use, the user's message, and that you want the full response at once (not streamed).
  • timeout=60.0 gives the model up to 60 seconds to respond. Local models can be slow on first load.
  • The last two lines parse the JSON response and print only the assistant's reply.
  • Save quick_test.py.
Run the script and see the model respond

Before you run this, what do you think will happen? Will it print a response immediately, or take a few seconds?

  • Run your test script:
python quick_test.py

After a few seconds, you should see qwen2.5:7b print a concise definition of ESG directly in your terminal. The first run may take longer as Ollama loads the model weights into your GPU's VRAM.

What should I see?

The model will print something like: "ESG stands for Environmental, Social, and Governance, a framework used to assess a company's sustainability and ethical impact." Your exact wording will differ because LLM outputs are non-deterministic.

🙋‍♀️ Seeing a connection error?

If you see a connection refused error, Ollama may not be running. Open a separate terminal and run ollama serve to start it, then try again.

If you see a timeout, the model may still be loading into VRAM. Wait 30 seconds and run the command again.

help me fix a connection error when running quick_test.py against Ollama

✔️ Awesome, I've got everything!

Great. Double check you've saved quick_test.py and confirmed the model responded.

ⓧ I'd like to double check the full code

Here is the complete quick_test.py file:

"""Quick test to verify Ollama is working. Delete after Step 3."""
import httpx

response = httpx.post(
    "http://localhost:11434/v1/chat/completions",
    json={
        "model": "qwen2.5:7b",
        "messages": [
            {"role": "user", "content": "What is ESG in one sentence?"}
        ],
        "stream": False,
    },
    timeout=60.0,
)

data = response.json()
print(data["choices"][0]["message"]["content"])

Your local AI stack is operational. You just proved that Python, Ollama, and the qwen2.5:7b model can communicate end to end. Next up, you'll replace those hardcoded values with a proper typed configuration system that loads from your .env file.

Build Typed Configuration with Logging

Your quick test proved the LLM connection works. But look at quick_test.py again: the model name, URL, and temperature are all hardcoded directly in the script.

Production code separates configuration from logic. In this step, you'll use Pydantic Settings to load typed values from your .env file, and Python's logging module to replace print statements with proper timestamps and severity levels.

In this step, get ready to:
  • Create a typed settings module that loads configuration from your .env file.
  • Configure structured logging with timestamps and severity levels.
  • Run Ruff linting and commit your changes.
Create the typed settings module

Pydantic's BaseSettings class reads environment variables and validates their types automatically. You define each setting as a typed field, and Pydantic handles parsing strings into the correct Python types.

  • In VS Code, open the file app/config/settings.py (it exists from your scaffold but is currently empty).
  • Add the following code to define your settings class:
from functools import lru_cache

from pydantic_settings import BaseSettings


class Settings(BaseSettings):
    """Application settings loaded from environment variables."""

    ollama_base_url: str = "http://localhost:11434"
    ollama_model: str = "qwen2.5:7b"
    ollama_temperature: float = 0.7
    ollama_max_tokens: int = 1024
    ollama_timeout: float = 60.0
    log_level: str = "INFO"

    model_config = {"env_file": ".env", "env_file_encoding": "utf-8"}


@lru_cache
def get_settings() -> Settings:
    """Return cached settings instance."""
    return Settings()

What does this code do?

The Settings class inherits from BaseSettings, which tells Pydantic to look for matching environment variables (case-insensitive). Each field has a type annotation and a default value as a fallback.

The model_config dictionary tells Pydantic to also load from your .env file. The @lru_cache decorator on get_settings() ensures the settings are loaded once and reused everywhere, instead of re-reading the file on every call.

  • Save app/config/settings.py.

Before you run the next command, what do you think it will print? It imports your new settings module and asks for the model name.

  • Verify the settings load correctly by running this command in your terminal:
python -c "from app.config.settings import get_settings; s = get_settings(); print(s.ollama_model)"

You should see qwen2.5:7b printed. That value comes from your .env file, not from the default in the code.

Seeing a ModuleNotFoundError?

  • Make sure your rag_1 virtual environment is active. You should see (rag_1) in your terminal prompt.
  • Confirm you're running the command from your project root folder (the one containing the app/ directory).
  • Check that pydantic-settings is installed by running pip list and looking for it in the output.

Still stuck? Help me debug a ModuleNotFoundError when importing my Pydantic settings module.

✔️ Awesome, I've got everything!

Great. Double check you've saved app/config/settings.py and move on to the next file.

ⓧ I'd like to double check the full code

from functools import lru_cache

from pydantic_settings import BaseSettings


class Settings(BaseSettings):
    """Application settings loaded from environment variables."""

    ollama_base_url: str = "http://localhost:11434"
    ollama_model: str = "qwen2.5:7b"
    ollama_temperature: float = 0.7
    ollama_max_tokens: int = 1024
    ollama_timeout: float = 60.0
    log_level: str = "INFO"

    model_config = {"env_file": ".env", "env_file_encoding": "utf-8"}


@lru_cache
def get_settings() -> Settings:
    """Return cached settings instance."""
    return Settings()
Create the logging configuration

Python's built-in logging module gives you severity levels (INFO, WARNING, ERROR) and timestamps. This makes debugging production issues possible because you can see exactly when something happened and how severe it was.

  • Open app/config/logging.py in VS Code (currently empty) and add the following code:
import logging
import sys

from app.config.settings import get_settings


def setup_logging() -> None:
    """Configure logging for the application."""
    settings = get_settings()

    logging.basicConfig(
        level=settings.log_level,
        format="%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
        datefmt="%H:%M:%S",
        handlers=[logging.StreamHandler(sys.stdout)],
    )

    # Reduce noise from third-party libraries
    logging.getLogger("httpx").setLevel(logging.WARNING)
    logging.getLogger("httpcore").setLevel(logging.WARNING)

What does this code do?

The setup_logging() function configures Python's logging system with a format that shows the time, severity level, module name, and message for every log entry.

It pulls the log_level from your settings (which loads from .env). The last two lines suppress noisy debug output from httpx and httpcore so your logs stay readable.

  • Save app/config/logging.py.
  • Verify the logging module works by running:
python -c "from app.config.logging import setup_logging; setup_logging(); import logging; logging.info('Logging is working')"

You should see output like 12:34:56 | INFO | root | Logging is working with your current time. That formatted output is your logging module in action.

Seeing an ImportError?

  • Make sure you saved both app/config/settings.py and app/config/logging.py. The logging module imports from settings.
  • Check that the app/config/__init__.py file exists (it should from your scaffold step).

Need more help? Help me debug an ImportError in my logging configuration module.

✔️ Awesome, I've got everything!

Great. Double check you've saved app/config/logging.py and move on to the final substep.

ⓧ I'd like to double check the full code

import logging
import sys

from app.config.settings import get_settings


def setup_logging() -> None:
    """Configure logging for the application."""
    settings = get_settings()

    logging.basicConfig(
        level=settings.log_level,
        format="%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
        datefmt="%H:%M:%S",
        handlers=[logging.StreamHandler(sys.stdout)],
    )

    # Reduce noise from third-party libraries
    logging.getLogger("httpx").setLevel(logging.WARNING)
    logging.getLogger("httpcore").setLevel(logging.WARNING)
Run Ruff and commit your changes

Ruff is a linter and formatter that catches style issues and potential bugs in your Python code. Running it before each commit keeps your codebase clean.

  • Run the linter to check for issues:
ruff check .

If Ruff finds no issues, you'll see no output. That means your code is clean.

  • Run the formatter to enforce consistent style:
ruff format .

Ruff prints the names of any files it reformatted. If your files already match its style, you'll see 2 files left unchanged.

  • Stage all changes and commit by running these commands:
git add .
git commit -m "feat: add typed configuration and logging"
  • Verify the commit landed by running:
git log --oneline

You should see two commits now: your scaffold commit from the previous step and the new feat: add typed configuration and logging commit at the top.

Ruff reported errors?

  • If ruff check . shows errors, read the line numbers in the output and fix the specific issues in your code.
  • Common issues include unused imports or missing blank lines between top-level definitions.
  • After fixing, run ruff check . again to confirm it passes cleanly.

Need help? Help me fix Ruff linting errors in my Python configuration files.

Your configuration layer is solid. Next up, you'll build the Ollama client service that uses these settings to send real prompts to your local LLM.

Build the Ollama Client Service

Your configuration and logging modules are wired up and tested. Now you need something that actually uses them.

The quick test script from Step 3 proved your hardware and model work, but it is a single-use throwaway. Every future component in this project (ingestion, retrieval, generation) will need to call the LLM. A proper client class with error handling, timeout configuration, logging, and typed responses means you write the HTTP logic once and reuse it everywhere.

In this step, get ready to:
  • Create an OllamaClient class with typed responses, logging, and error handling.
  • Wire up a main.py entry point that sends a question and displays the formatted response.
  • Verify error handling works gracefully when Ollama is unreachable.
Create the LLM client module

This module lives in app/services/llm_client.py and contains two things: a Pydantic response model that gives you typed fields instead of raw dictionaries, and a client class that wraps all the HTTP logic.

  • Create the file app/services/llm_client.py and add the imports, response model, and class initialization:
import logging
import time

import httpx
from pydantic import BaseModel

from app.config.settings import get_settings

logger = logging.getLogger(__name__)


class LLMResponse(BaseModel):
    """Structured response from the LLM."""

    content: str
    model: str
    response_time: float


class OllamaClient:
    """Client for communicating with the local Ollama instance."""

    def __init__(self) -> None:
        settings = get_settings()
        self.base_url = settings.ollama_base_url
        self.model = settings.ollama_model
        self.temperature = settings.ollama_temperature
        self.max_tokens = settings.ollama_max_tokens
        self.timeout = settings.ollama_timeout
        logger.info("OllamaClient initialized (model=%s, base_url=%s)", self.model, self.base_url)

What does this code do?

LLMResponse is a Pydantic model that structures the raw API response into typed fields: the text content, which model answered, and how long it took.

OllamaClient.__init__ reads all configuration from your .env file via get_settings() and stores the values as instance attributes. The logger.info call confirms initialization so you can trace what happened when debugging.

  • Save app/services/llm_client.py.
  • Verify the file is importable by running:
python -c "from app.services.llm_client import LLMResponse; print(LLMResponse.model_fields.keys())"

You should see dict_keys(['content', 'model', 'response_time']) printed.

Seeing an ImportError?

  • Make sure your virtual environment rag_1 is activated. You need pydantic-settings installed for the get_settings() import to resolve.
  • Check that app/services/__init__.py exists (it should be empty but present from Step 2).

Why is my llm_client import failing?

  • Now add the chat() method to the OllamaClient class. This method builds the request payload and sends it to Ollama. Copy this directly below the logger.info line in __init__:
    def chat(self, messages: list[dict[str, str]]) -> LLMResponse:
        """Send a chat request to Ollama and return a structured response."""
        url = f"{self.base_url}/v1/chat/completions"
        payload = {
            "model": self.model,
            "messages": messages,
            "temperature": self.temperature,
            "max_tokens": self.max_tokens,
            "stream": False,
        }

        logger.debug("Sending request to %s with %d messages", url, len(messages))
        start_time = time.time()

        try:
            with httpx.Client(timeout=self.timeout) as client:
                response = client.post(url, json=payload)
                response.raise_for_status()
        except httpx.ConnectError:
            logger.error("Connection failed to %s", self.base_url)
            raise ConnectionError(
                f"Cannot connect to Ollama at {self.base_url}. "
                "Is Ollama running? Start it with: ollama serve"
            )

What does this section do?

  • The URL targets Ollama's OpenAI-compatible endpoint at /v1/chat/completions.
  • httpx.Client opens a connection with your configured timeout, sends the JSON payload, and calls raise_for_status() to catch non-2xx responses.
  • The first except catches httpx.ConnectError (Ollama not running) and raises a clear ConnectionError with an actionable message.
  • Continue the chat() method by adding the remaining error handlers, response parsing, and the generate() convenience method directly below the ConnectError handler:
        except httpx.TimeoutException:
            logger.error("Request timed out after %.1fs", self.timeout)
            raise TimeoutError(
                f"Ollama did not respond within {self.timeout} seconds. "
                "The model may still be loading. Try again in a moment."
            )
        except httpx.HTTPStatusError as e:
            logger.error("HTTP error %d: %s", e.response.status_code, e.response.text)
            raise RuntimeError(
                f"Ollama returned an error: {e.response.status_code} - "
                f"{e.response.text}"
            )

        elapsed = time.time() - start_time
        data = response.json()
        content = data["choices"][0]["message"]["content"]

        logger.info("Response received in %.1fs (%d chars)", elapsed, len(content))

        return LLMResponse(
            content=content,
            model=self.model,
            response_time=elapsed,
        )

    def generate(self, prompt: str, system_prompt: str = "") -> LLMResponse:
        """Send a single prompt and return the response."""
        messages = []
        if system_prompt:
            messages.append({"role": "system", "content": system_prompt})
        messages.append({"role": "user", "content": prompt})
        return self.chat(messages)

What does this section do?

  • httpx.TimeoutException catches cases where the model takes too long (loading a model for the first time, for example).
  • httpx.HTTPStatusError catches non-2xx responses (wrong model name, bad request format).
  • After a successful response, the code extracts the text content, logs the response time, and returns a typed LLMResponse object.
  • generate() is a convenience wrapper. It builds the messages list from a single prompt and optional system prompt, then delegates to chat().
  • Save app/services/llm_client.py.
  • Verify the complete module imports cleanly by running:
python -c "from app.services.llm_client import OllamaClient; print('OllamaClient ready')"

You should see OllamaClient ready printed with no errors.

Seeing a SyntaxError or IndentationError?

  • Make sure the except blocks align at the same indentation level as the try keyword (8 spaces inside the method).
  • Check that generate() is indented at the same level as chat() (4 spaces inside the class).
  • Confirm there are no stray lines between the except httpx.ConnectError block and the except httpx.TimeoutException block.

Help me fix a SyntaxError in my llm_client.py

✔️ Awesome, I've got everything!

Great. Double check you have saved app/services/llm_client.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete app/services/llm_client.py at this point. Scroll through and confirm each section matches yours:

  • Imports and logger (lines 1-9).
  • LLMResponse model with three fields.
  • OllamaClient.__init__ reading from settings.
  • chat() method with payload, try/except, and LLMResponse return.
  • generate() helper at the bottom.
Wire up the entry point

The client module is ready. You need an entry point that ties logging, the client, and a prompt together into something you can actually run.

  • Create main.py in your project root (the same folder as quick_test.py) and add the imports and single_shot() function:
import argparse
import sys

from app.config.logging import setup_logging
from app.services.llm_client import OllamaClient


def single_shot() -> None:
    """Send a single question and print the response."""
    client = OllamaClient()

    try:
        response = client.generate(
            prompt="What are the three main pillars of ESG reporting?",
            system_prompt="You are a sustainability expert. Give concise, informative answers.",
        )
    except (ConnectionError, TimeoutError, RuntimeError) as e:
        print(f"\nError: {e}")
        sys.exit(1)

    print(f"\nResponse from {response.model} ({response.response_time:.1f}s):")
    print("-" * 60)
    print(response.content)
    print("-" * 60)

What does single_shot() do?

It creates an OllamaClient, sends a sustainability question via generate(), and prints the response with the model name and timing. If anything goes wrong (connection, timeout, HTTP error), it catches the exception and exits with a clear message instead of a traceback.

  • Below single_shot(), add the interactive_chat() function. This maintains a conversation history so the model remembers previous turns. Add this code:
def interactive_chat() -> None:
    """Run an interactive chat session with conversation history."""
    client = OllamaClient()
    print(f"Chat with {client.model} (Ctrl+C to exit)")
    print("=" * 60)

    messages: list[dict[str, str]] = [
        {
            "role": "system",
            "content": "You are a sustainability expert. Give concise, informative answers.",
        }
    ]
    turn_count = 0
  • Continue interactive_chat() by adding the main loop directly below turn_count = 0:
    try:
        while True:
            user_input = input("\nYou: ").strip()
            if not user_input:
                continue

            messages.append({"role": "user", "content": user_input})

            try:
                response = client.chat(messages)
            except (ConnectionError, TimeoutError, RuntimeError) as e:
                print(f"\nError: {e}")
                messages.pop()
                continue

            messages.append({"role": "assistant", "content": response.content})
            turn_count += 1

            print(f"\nAssistant ({response.response_time:.1f}s):")
            print(response.content)

    except KeyboardInterrupt:
        print(f"\n\nChat ended. {turn_count} turns completed.")

How does conversation history work?

The messages list accumulates every user and assistant message. Each time you call client.chat(messages), Ollama receives the full history so it can reference earlier context.

If a request fails, messages.pop() removes the unanswered user message so the history stays clean. Pressing Ctrl+C triggers KeyboardInterrupt and prints a summary of how many turns occurred.

  • Finally, add the main() entry point and the if __name__ guard below interactive_chat():
def main() -> None:
    parser = argparse.ArgumentParser(description="ESG RAG System")
    parser.add_argument("--chat", action="store_true", help="Start interactive chat mode")
    args = parser.parse_args()

    setup_logging()

    if args.chat:
        interactive_chat()
    else:
        single_shot()


if __name__ == "__main__":
    main()

What does argparse do here?

argparse reads command-line flags. Running python main.py calls single_shot() for a one-off question. Adding --chat switches to the interactive loop. The setup_logging() call activates the structured logging you configured in Step 4.

  • Save main.py.
  • Before you run it, predict: what do you expect to see in the output given that logging is now configured?
  • Run the entry point:
python main.py

You should see log lines showing OllamaClient initialized and Response received with a time, followed by a formatted answer about the three pillars of ESG reporting.

Seeing a ConnectionError?

  • Make sure Ollama is running. Open a separate terminal and run ollama serve if needed.
  • Confirm qwen2.5:7b is pulled by running ollama list and checking it appears.
  • Verify your .env file has OLLAMA_BASE_URL=http://localhost:11434 with no trailing slash.

Help me debug why my main.py cannot connect to Ollama

✔️ Awesome, I've got everything!

Great. Double check you have saved main.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete main.py at this point. Scroll through and confirm each section matches yours:

  • Imports: argparse, sys, setup_logging, OllamaClient.
  • single_shot() function with error handling and formatted output.
  • interactive_chat() function with messages list and KeyboardInterrupt handling.
  • main() with argparse --chat flag and setup_logging() call.
  • if __name__ == "__main__": main() guard at the bottom.
Lint, verify error handling, and commit

Your client works when Ollama is healthy. Production code also needs to handle the unhappy path. You will intentionally break the connection to prove your error handling works.

  • Run Ruff to check code quality:
ruff check .

You should see All checks passed! with no warnings.

Ruff reporting issues?

  • If you see unused import warnings, check that every import in llm_client.py is actually used in the code below it.
  • If you see line length errors, check for lines exceeding 88 characters and wrap them.

Help me fix Ruff linting errors in my project

  • Now test the error path. Stop Ollama by closing its terminal window or running:
taskkill /IM ollama.exe /F
  • With Ollama stopped, predict what will happen when you run the script again. Will it crash with a traceback, or handle the error gracefully?
  • Run it again:
python main.py

Instead of a raw traceback, you should see a log line with Connection failed followed by a clean error message telling you Ollama is not running. This is the graceful error handling you built.

  • Restart Ollama so your system is back to normal:
ollama serve

Keep Ollama running in its own terminal

Leave ollama serve running in a dedicated terminal tab. Switch back to your project terminal for the remaining commands.

  • Stage and commit your new files:
git add .
git commit -m "feat: add Ollama client service with error handling"
  • Verify the commit landed by running:
git log --oneline

You should see three commits now: your scaffold, your configuration, and the new client service commit at the top.

Your AI stack is fully operational. You have typed configuration, structured logging, and a reusable client with proper error handling. This is the foundation every RAG component will plug into.

Secret mission

Add an Interactive Chat Loop

Your single-shot client proves the LLM works, but real AI applications need multi-turn conversations. Can you test the interactive chat mode, prove that the model remembers previous turns, and lock it in with a clean commit?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Project folder on your Desktop (all Python files, config, and Git history).
  • Python packages installed in your rag_1 virtual environment (httpx, pydantic-settings, ruff).
  • The qwen2.5:7b model downloaded in Ollama.
  • Git for Windows (installed in Step 1).

Keep everything running

No action needed. Choose this if you're still actively building on this project or plan to continue with the next project in the series.

  • Your project folder stays on your Desktop.
  • Ollama keeps the qwen2.5:7b model loaded and ready for the next project.
  • Git stays installed for future version control.
  • Your rag_1 virtual environment retains all installed packages.

Pause - I'll come back to this later

Shut down running processes to free up memory. Your files and installations stay intact for when you return.

  • Close VS Code.
  • Close the Ollama application (right-click the Ollama icon in your system tray and select Quit). This frees the VRAM used by loaded models.
  • Deactivate your virtual environment by running:
deactivate

When you come back, reactivate your environment and start Ollama again.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below.

  • Delete the project folder from your Desktop by running:
cd ~\Desktop
rmdir /s /q esg-rag-system
  • Remove the qwen2.5:7b model from Ollama by running:
ollama rm qwen2.5:7b
  • Uninstall the Python packages from your rag_1 virtual environment by running:
pip uninstall httpx pydantic-settings ruff -y
  • Uninstall Git for Windows. Open Control Panel, then navigate to Programs and Features. Find Git in the list, right-click it, and select Uninstall.

Nice Work!

Nice Work!

You just built a production-style AI engineering workspace from scratch. You have a clean, Git-tracked Python project that talks to your local qwen2.5:7b model through your own typed code with structured logging.

You've learned how to:

  • Build a professional project structure with Git version control, meaningful commit messages, and a folder layout that mirrors real AI engineering repositories.
  • Create typed configuration with Pydantic Settings that loads from environment variables, and replace print statements with structured logging using Python's logging module.
  • Build a reusable Ollama client service with httpx that handles connection errors, timeouts, and HTTP failures gracefully instead of crashing.
  • Secret Mission: Added an interactive chat loop with conversation history management that maintains context across multiple turns.

Ready to quiz yourself?