Build a Local AI Chat Client in C++

Build a C++ chatbot that talks to a local LLM over HTTP using REST APIs.

Introduction

30 Second Summary

Every time you use an AI chatbot, your message travels over a network to a server that processes it and sends a reply back. That round trip relies on HTTP requests, JSON data, and a pattern called a REST API.

In this project, you will build your own AI chatbot in C++ that runs entirely on your Mac. Your program sends HTTP requests to a local AI model, parses JSON responses, and maintains a full conversation with memory.

What You'll Build

Picture typing a question into your terminal, watching your C++ program fire off an HTTP request to a local AI, and getting an intelligent response printed back in seconds.

By the end of this project, you'll have:

  • A working terminal chatbot where you type messages and get intelligent AI responses in real-time from your command line.
  • A multi-turn conversation memory that keeps the AI aware of everything you've discussed, so follow-up questions work naturally.
  • Clean object-oriented code with two classes that separate HTTP networking logic from conversation session management.
  • Secret Mission: the ability to save and load entire conversations as JSON files so you can pick up where you left off.

Are there any prerequisites?

You need basic familiarity with any programming language (variables, loops, functions). Some exposure to C++ syntax is helpful but not required. This project teaches networking and OOP concepts from the ground up.

Before We Start

Before We Start...

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Your C++ and AI Environment

Your AI chatbot needs three things before it can send its first message: a C++ compiler to build your code, a local AI server to generate responses, and two library headers that handle HTTP requests and JSON data.

In this step, you will install those tools and download the files that make it all possible. Once everything is in place, you will test the AI model directly in your terminal to confirm it responds.

In this step, get ready to:
  • Install the clang++ compiler and verify it supports C++17.
  • Install Ollama and download the gemma4 AI model.
  • Set up your project folder with two header-only libraries.
Install Xcode Command Line Tools and verify clang++

C++ code does not run directly like a script. You need a compiler to translate your source files into an executable program. On macOS, Apple provides clang++ as part of the Xcode Command Line Tools.

  • Open Terminal by pressing Cmd+Space to open Spotlight, typing Terminal, and pressing Enter.
  • Install the Xcode Command Line Tools by running this command:
xcode-select --install

What will happen when you run this?

If the tools are not installed, a system dialog appears asking you to confirm the installation. Click Install and wait for the download to finish (this can take a few minutes).

If you see a message saying tools are already installed, that means you are all set and can move on.

  • After the installation finishes, verify clang++ is available by running:
clang++ --version

✔️ I see a version number

You should see output that includes Apple clang version and a target like arm64-apple-darwin. This confirms your compiler is ready.

ⓧ Command not found

If Terminal cannot find clang++, the installation may not have completed.

  • Run xcode-select --install again and watch for the system dialog.
  • Click Install when the dialog appears and wait for it to complete fully.
  • Once it finishes, try clang++ --version again.
Install Ollama and pull the gemma4 model

Ollama is a lightweight server that runs AI models locally on your Mac. Your C++ program will send HTTP requests to Ollama the same way a web app talks to a cloud API, but everything stays on your machine.

  • Download the Ollama installer by opening this link in your browser:
  • https://ollama.com/download/Ollama.dmg
  • Open the downloaded Ollama.dmg file.
  • Drag the Ollama icon into the Applications folder.
  • Open Ollama from your Applications folder (double-click it).

What happens when you first open Ollama?

Ollama will ask for permission to add its command-line tool to your PATH. Allow this so you can use the ollama command in Terminal. You will see a small llama icon appear in your menu bar, confirming the server is running.

  • Back in Terminal, download the gemma4 model by running this command:
ollama pull gemma4

What does this command do?

This downloads the gemma4 model (about 5GB) to your local machine. You will see a progress bar as the model layers download. This may take a few minutes depending on your internet speed.

Seeing an error about connecting?

If you see a connection error, Ollama's background server may not be running. Check that the llama icon is visible in your menu bar. If not, open Ollama from Applications again, then retry the pull command.

help me fix the Ollama connection error

Before you run the next command, what do you think will happen if you type a question to the AI?

  • Test the model interactively by running:
ollama run gemma4
  • Type any question (for example, "What is HTTP?") and press Enter.
  • You should see the AI respond with a helpful answer directly in your terminal.
  • Type /bye and press Enter to exit the chat.

Why test this way?

Running ollama run gemma4 confirms three things at once: the model downloaded correctly, the server is running, and the model can generate responses. Your C++ program will talk to this exact same server in the next step.

Set up the project folder with header libraries

Your chatbot needs two external libraries: cpp-httplib for making HTTP requests, and nlohmann/json for parsing JSON responses. Both are header-only, which means you download a single file for each and include it in your code.

  • Create your project folder and move into it by running these commands:
mkdir ~/ai-chat-cpp && cd ~/ai-chat-cpp
  • Download the cpp-httplib header (v0.47.0) by running:
curl -LO https://github.com/yhirose/cpp-httplib/raw/refs/tags/v0.47.0/httplib.h
  • Download the nlohmann/json header (v3.12.0) by running:
curl -LO https://github.com/nlohmann/json/releases/download/v3.12.0/json.hpp
  • Confirm both files downloaded by running:
ls

You should see httplib.h and json.hpp listed in the output.

Don't see both files?

If a file is missing, the curl command may have failed silently. Check your internet connection and re-run the curl -LO command for the missing file. Make sure you are inside the ~/ai-chat-cpp folder by running cd ~/ai-chat-cpp first.

help me troubleshoot the missing header file download

  • Open the project folder in VS Code by running:
code .

Getting 'command not found' for code?

The code command needs to be added to your PATH from inside VS Code.

  • Open VS Code manually (press Cmd+Space, type Visual Studio Code, press Enter).
  • Press Cmd+Shift+P to open the Command Palette.
  • Type Shell Command and select Shell Command: Install 'code' command in PATH.
  • Restart your terminal, then try code . again from inside ~/ai-chat-cpp.

VS Code opens with your ai-chat-cpp folder in the Explorer sidebar. You should see httplib.h and json.hpp listed in the file tree.

Your environment is ready. You have a compiler, a running AI server, and two libraries waiting to be used. Next up, you will write your first C++ program and send a real message to the AI.

Send Your First Message to the AI

You have Ollama running and a project folder ready with two header libraries. Now it is time to make your C++ code actually talk to the AI.

In this step, you will send an HTTP POST request to Ollama's REST API, see the raw response, then build an interactive chat loop. By the end, you will type messages and get AI replies in real-time from your terminal.

In this step, get ready to:
  • Send an HTTP POST request to Ollama and confirm you get a response.
  • Parse the JSON response and build an interactive chat loop.
  • Discover a limitation in the AI's memory.
Write a basic main.cpp that sends a POST request

The simplest possible test is to send one hardcoded message to Ollama and print whatever comes back. If you see a JSON response, you know the connection works.

  • Create a new file called main.cpp in VS Code by clicking File in the menu bar, then New File.
  • Save it immediately as main.cpp inside your ai-chat-cpp folder.
  • Copy the following code into main.cpp:
#include "httplib.h"
#include "json.hpp"
#include <iostream>
#include <string>

using json = nlohmann::json;

int main() {
    // Connect to Ollama's local server
    std::string model = "gemma4";
    httplib::Client cli("localhost", 11434);
    cli.set_read_timeout(120, 0);

    // Build a JSON request body with a test message
    json request_body = {
        {"model", model},
        {"messages", {{"role", "user"}, {"content", "Why is the sky blue?"}}},
        {"stream", false}
    };

    // Send POST request to the chat endpoint
    auto res = cli.Post("/api/chat", request_body.dump(), "application/json");
    if (!res) {
        std::cout << "[Error] Could not connect to Ollama." << std::endl;
        return 1;
    }

    // Print the raw response body
    std::cout << res->body << std::endl;
    return 0;
}

What does this code do?

  • httplib::Client cli("localhost", 11434) creates an HTTP client pointing at Ollama's local server on port 11434.
  • The request_body is a JSON object with the model name, a single user message, and stream: false so Ollama returns one complete response instead of streaming chunks.
  • cli.Post("/api/chat", ...) sends the request to Ollama's chat endpoint and stores the response.
  • The timeout is set to 120 seconds because AI models can take a moment to generate their first response.
  • Save main.cpp.
  • Open your terminal in VS Code by pressing Cmd+` (macOS) or Ctrl+` (Windows).
  • Compile and run the program by running this command:
clang++ -std=c++17 -o chat main.cpp && ./chat

After a few seconds, you should see a wall of raw JSON printed to your terminal. Look for a "content" field buried in the output containing the AI's answer about the sky.

Seeing "Could not connect to Ollama"?

  • Make sure Ollama is running. Look for the Ollama icon in your menu bar. If it is not there, open Ollama from Applications.
  • Confirm the model is downloaded by running ollama list in a separate terminal tab. You should see gemma4 listed.
  • help me debug why my C++ program cannot connect to Ollama on localhost:11434
Parse the JSON response and add a chat loop

The raw JSON proved the connection works. Now you will parse out just the AI's reply and wrap everything in a loop so you can have a back-and-forth conversation.

  • Select all the code in main.cpp and replace it with the following. Start with the top half:
#include "httplib.h"
#include "json.hpp"
#include <iostream>
#include <string>

using json = nlohmann::json;

int main() {
    std::string model = "gemma4";
    httplib::Client cli("localhost", 11434);
    cli.set_read_timeout(120, 0);

    // Print welcome banner
    std::cout << "Chat with " << model << " (type 'quit' to exit)" << std::endl;
    std::cout << "-------------------------------------------" << std::endl;

    std::string input;
    while (true) {
        // Prompt the user and read their message
        std::cout << "\nYou: ";
        std::getline(std::cin, input);

        // Exit on quit or exit command
        if (input == "quit" || input == "exit") {
            std::cout << "\nGoodbye!" << std::endl;
            break;
        }

        // Skip empty input
        if (input.empty()) {
            continue;
        }

What is happening here?

  • The while (true) loop keeps the program running until the user types quit or exit.
  • std::getline(std::cin, input) reads a full line of text from the keyboard, including spaces.
  • The empty-input check prevents sending blank messages to the AI.
  • Directly below the code you just pasted, add this second half to complete the file:
        // Build the request with the user's message
        json request_body = {
            {"model", model},
            {"messages", {{"role", "user"}, {"content", input}}},
            {"stream", false}
        };

        // Send to Ollama and handle connection errors
        auto res = cli.Post("/api/chat", request_body.dump(), "application/json");

        if (!res) {
            std::cout << "[Error] Could not connect to Ollama." << std::endl;
            continue;
        }

        // Parse the JSON and print only the AI's reply
        json response = json::parse(res->body);
        std::string reply = response["message"]["content"].get<std::string>();
        std::cout << "\nAI: " << reply << std::endl;
    }

    return 0;
}

What does the parsing do?

  • json::parse(res->body) converts the raw response string into a structured JSON object you can navigate.
  • response["message"]["content"] drills into the response to extract only the AI's text reply.
  • .get<std::string>() converts the JSON value to a C++ string so you can print it cleanly.
  • Unlike the previous version that dumped raw JSON, you now see a clean readable response.
  • Save main.cpp.
  • Compile and run the updated program:
clang++ -std=c++17 -o chat main.cpp && ./chat

You should see the welcome banner, then a You: prompt waiting for your input. Type a question and press Enter.

Before you try a second message, predict: if you ask "What did I just ask you?", do you think the AI will remember your first question?

  • Type any question (for example, "What is the speed of light?") and press Enter. Wait for the AI's reply.
  • Now type: What did I just ask you?

The AI has no idea what you asked before. It responds as if this is the very first message it has ever received.

Why does the AI have amnesia?

Look at the request body. Every time you send a message, you only include that single message in the messages field. The AI never sees your previous messages because HTTP is stateless.

Each request is independent. For the AI to "remember", your program would need to send the entire conversation history with every request. That is exactly what you will build in the next step.

Not seeing any AI response?

  • If the program hangs for more than 2 minutes, Ollama may be loading the model for the first time. Let it finish. Subsequent messages will be faster.
  • If you see a compile error about json.hpp or httplib.h, make sure both header files are in the same folder as main.cpp.
  • If you see a json::parse error at runtime, Ollama might be returning an error message instead of a valid response. Run ollama list to confirm the gemma4 model is available.
  • help me fix my C++ Ollama chat program that is not returning AI responses

✔️ Awesome, I've got everything!

Great. Double check you have saved main.cpp and that the program compiles and runs without errors.

ⓧ I'd like to double check the full code

Your main.cpp should contain these sections in order. Scroll up and confirm each one is present:

  • The four #include lines and the using json alias.
  • Inside main(): the model string, httplib::Client creation, and set_read_timeout call.
  • Two std::cout lines printing the welcome banner.
  • The while (true) loop with std::getline, quit check, and empty-input check.
  • The request_body JSON construction, cli.Post call, null check, and the three lines that parse the response and print the reply.
  • Closing brace for the while loop, return 0;, and closing brace for main().

Your chat program works, but each message exists in isolation. Next up, you will give the AI a memory so it can hold a real conversation.

Give the AI a Memory

In the last step, you proved that your C++ code can talk to Ollama and get responses. But you also discovered a gap: when you asked a follow-up question, the AI had no idea what you were talking about.

HTTP is a stateless protocol. Every request your program sends is completely independent. The AI only knows what you include in that single request. To give it memory, your program needs to send the full conversation history with every message.

In this step, get ready to:
  • Store every user message in a persistent array.
  • Store every AI reply in the same array.
  • Send the full conversation history with each request so the AI remembers context.
Store conversation history in a vector

Right now your code builds a fresh request body each time through the loop, containing only the latest message. The fix is to declare a persistent json array before the loop that accumulates every message the user sends.

  • In main.cpp, find this line (just before the std::cout lines):
    cli.set_read_timeout(120, 0);
  • Add a blank line after it, then add this line:
    json messages = json::array();

What does this line do?

This creates an empty JSON array that lives outside the loop. Because it is declared before while (true), it persists across iterations. Every message you send will be pushed onto this array so the full history travels with each request.

Now you need to change how the loop body builds the request. Instead of constructing a single message inline, you will push the user's input onto the messages array and send the entire array.

  • Inside the while loop, find this block (it comes right after the if (input.empty()) check):
        json request_body = {
            {"model", model},
            {"messages", {{"role", "user"}, {"content", input}}},
            {"stream", false}
        };
  • Replace that entire block with the following code:
        // Add the user's message to conversation history
        messages.push_back({{"role", "user"}, {"content", input}});

        json request_body = {
            {"model", model},
            {"messages", messages},
            {"stream", false}
        };

What changed here?

Two things changed. First, messages.push_back(...) appends the user's input as a new entry in the conversation array before the request is built.

Second, the "messages" field in the request body now points to the full messages array instead of a single inline message. Every request now carries the complete history of what the user has said.

  • Save main.cpp.
  • Compile and run to confirm the restructured code still works, by running this command:
clang++ -std=c++17 -o chat main.cpp && ./chat
  • Type a message like "Hello" and confirm you get a response from the AI.
  • Type quit to exit.

Seeing a compile error?

Make sure you removed the old json request_body block completely before pasting the new one. Having both will cause a redefinition error.

Check that json messages = json::array(); appears before the while loop, not inside it. If it is inside the loop, the array resets every iteration.

help me fix a compile error in my conversation history code

Store assistant replies in history too

Your messages array now tracks what the user says, but the AI's own replies are missing from history. Without them, the AI sees a sequence of user messages with no responses between them. To build true conversational memory, you need to store the assistant's replies too.

  • In main.cpp, find this line inside the loop:
        std::string reply = response["message"]["content"].get<std::string>();
  • Add this line directly below it (before the std::cout line):
        // Store the AI's reply in conversation history
        messages.push_back({{"role", "assistant"}, {"content", reply}});

Why store both sides of the conversation?

Ollama's chat API expects alternating user/assistant messages. By pushing the assistant reply after every exchange, the messages array contains the full back-and-forth. Each new request shows the AI everything that has been said, by both sides.

  • Save main.cpp.

Time to test whether the AI truly remembers. Before you compile, think about this: if you tell the AI your name in the first message, then ask "What is my name?" in the second message, what do you expect to happen now compared to last step?

  • Compile and run by running this command:
clang++ -std=c++17 -o chat main.cpp && ./chat
  • Type: My name is Alex (or your own name) and press Enter.
  • Wait for the AI's response, then type: What is my name? and press Enter.

The AI correctly recalls your name. That is your conversation history at work. Every request now carries the full dialogue, so the AI always has context.

  • Type quit to exit.

AI still not remembering?

Make sure messages.push_back for the user input appears before json request_body, not after it. The message must be in the array before you build the request.

Check that json messages = json::array(); is outside the while loop. If it is inside, the array resets to empty on every iteration.

help me debug why my AI still doesn't remember previous messages

✔️ Awesome, I've got everything!

Great. Double check you have saved main.cpp.

ⓧ I'd like to double check the full code

Scroll through your main.cpp and confirm each of these is present, in order:

  • json messages = json::array(); appears after cli.set_read_timeout(120, 0); and before the std::cout lines.
  • messages.push_back({{"role", "user"}, {"content", input}}); appears inside the loop, before json request_body.
  • The request_body uses {"messages", messages} (the array variable, not an inline object).
  • messages.push_back({{"role", "assistant"}, {"content", reply}}); appears after std::string reply = ... and before std::cout << "\nAI: ".

Your chatbot now has real conversational memory. Every message travels with the full history, so the AI can reference anything from earlier in the conversation. Next up, you will refactor this working code into clean object-oriented classes that separate networking logic from session management.

Refactor into Clean OOP Classes

Your chatbot works, but all the logic lives inside main(). Real C++ projects separate concerns into classes. This makes code reusable, testable, and easier to extend.

In this step, you'll extract the HTTP logic into an OllamaClient class and the conversation management into a ChatSession class. The program will behave identically to before, but now it supports a system prompt that shapes the AI's personality.

In this step, get ready to:
  • Create an OllamaClient class that handles HTTP communication with Ollama.
  • Create a ChatSession class that manages conversation history and system prompts.
  • Wire both classes together in a clean main() function.
Create the OllamaClient class

The OllamaClient class will own the HTTP connection and expose a single chat() method. Everything about how to talk to Ollama lives in this one place.

  • Select all the code in main.cpp and delete it. You are rebuilding from scratch.
  • Start fresh by pasting these includes at the top of the empty file:
#include "httplib.h"
#include "json.hpp"
#include <iostream>
#include <string>
#include <vector>

using json = nlohmann::json;

What changed in the includes?

You added #include <vector>. The classes you are about to write use std::vector internally through the json library's array type. The rest is identical to before.

  • Below the using statement, add the OllamaClient class. Start with the class declaration, constructor, and the chat() method:
class OllamaClient {
public:
    OllamaClient(const std::string& host, int port, const std::string& model)
        : client_(host, port), model_(model) {
        client_.set_read_timeout(120, 0);
    }

    std::string chat(const json& messages) {
        json request_body = {
            {"model", model_},
            {"messages", messages},
            {"stream", false}
        };

        auto res = client_.Post("/api/chat", request_body.dump(), "application/json");

        if (!res) {
            return "[Error] Could not connect to Ollama. Is it running?";
        }

        if (res->status != 200) {
            return "[Error] Ollama returned status " + std::to_string(res->status);
        }

        json response = json::parse(res->body);
        return response["message"]["content"].get<std::string>();
    }
  • Immediately below the closing } of the chat() method, close the class with its private members:

private:
    httplib::Client client_;
    std::string model_;
};

What does this class do?

The constructor creates an HTTP client pointed at the given host and port, and stores the model name. It also sets a 120-second read timeout so long AI responses do not get cut off.

The chat() method builds the JSON request, sends a POST to /api/chat, and returns the AI's reply as a string. It handles two failure modes: the server not running (null response) and a non-200 status code.

The private members (client_ and model_) are hidden from outside code. Only the class itself can touch them. This is encapsulation.

  • Save main.cpp.
  • Verify the class compiles by running this command (the -c flag compiles without linking, so the missing main() is not a problem):
clang++ -std=c++17 -c main.cpp

If nothing prints and you see no errors, your OllamaClient class is valid C++.

Seeing a syntax error?

Check that the closing }; of the class comes after the private members. A missing semicolon after the closing brace is the most common C++ class error.

Also confirm the chat() method's closing brace } appears before the private: label.

help me fix a compile error in my OllamaClient class

Create the ChatSession class

The ChatSession class manages the conversation history array and knows how to prepend a system prompt. It uses OllamaClient to send messages but does not know anything about HTTP itself.

  • Below the closing }; of OllamaClient, add the ChatSession class:
class ChatSession {
public:
    ChatSession(OllamaClient& client, const std::string& system_prompt = "")
        : client_(client) {
        if (!system_prompt.empty()) {
            messages_.push_back({{"role", "system"}, {"content", system_prompt}});
        }
    }

    std::string send(const std::string& user_message) {
        messages_.push_back({{"role", "user"}, {"content", user_message}});

        std::string response = client_.chat(messages_);

        messages_.push_back({{"role", "assistant"}, {"content", response}});

        return response;
    }

    const json& get_messages() const {
        return messages_;
    }

    void set_messages(const json& messages) {
        messages_ = messages;
    }

private:
    OllamaClient& client_;
    json messages_ = json::array();
};

What does this class do?

The constructor takes a reference to an OllamaClient (so it can send messages) and an optional system prompt. If a system prompt is provided, it gets pushed as the first message with role system. This shapes how the AI responds to everything that follows.

The send() method pushes the user message, calls client_.chat() with the full history, then pushes the assistant reply. Every call sends the entire conversation, giving the AI full context.

The get_messages() and set_messages() methods let external code read or replace the conversation history. You will use these in the Secret Mission.

  • Save main.cpp.
  • Compile again to verify both classes are valid:
clang++ -std=c++17 -c main.cpp

No output means no errors. Both classes compile cleanly.

Getting an undeclared identifier error?

ChatSession references OllamaClient, so it must appear below OllamaClient in the file. C++ reads top to bottom. If you placed ChatSession above OllamaClient, move it below.

help me fix an undeclared identifier error in my ChatSession class

Wire everything together in main()

With both classes ready, main() becomes a thin shell. It creates the objects, runs the loop, and prints results.

  • Below the ChatSession class, add this main() function:
int main() {
    std::string model = "gemma4";
    std::string system_prompt = "You are a helpful assistant. Keep responses concise and clear.";

    OllamaClient client("localhost", 11434, model);
    ChatSession session(client, system_prompt);

    std::cout << "Chat with " << model << " (type 'quit' to exit)" << std::endl;
    std::cout << "-------------------------------------------" << std::endl;

    std::string input;
    while (true) {
        std::cout << "\nYou: ";
        std::getline(std::cin, input);

        if (input == "quit" || input == "exit") {
            std::cout << "\nGoodbye!" << std::endl;
            break;
        }

        if (input.empty()) {
            continue;
        }

        std::string response = session.send(input);
        std::cout << "\nAI: " << response << std::endl;
    }

    return 0;
}

What changed in main()?

The HTTP client setup, request building, and response parsing are all gone. In their place, OllamaClient client("localhost", 11434, model) handles the connection and session.send(input) handles the entire conversation flow.

The system prompt on line 3 tells the AI how to behave. Every message the AI sees will be preceded by this instruction.

  • Save main.cpp.
  • Before you compile, predict: will the AI respond differently than before, or identically?
  • Compile and run the refactored program:
clang++ -std=c++17 -o chat main.cpp && ./chat

The AI now responds more concisely thanks to the system prompt. Try asking it a question and notice how it keeps answers brief and clear.

  • Type quit to exit, then try a different personality. Open main.cpp and change the system prompt string to:
std::string system_prompt = "You are a pirate. Respond in pirate speak.";
  • Save main.cpp, recompile, and run again:
clang++ -std=c++17 -o chat main.cpp && ./chat

Ask the pirate AI anything. The system prompt completely changes the AI's personality without touching any other code. That is the power of separating concerns into classes.

  • Once you have confirmed the pirate personality works, change the system prompt back to the original value:
std::string system_prompt = "You are a helpful assistant. Keep responses concise and clear.";
  • Save main.cpp.

AI not responding in pirate speak?

Make sure you recompiled after changing the system prompt. C++ is a compiled language, so saving the file alone does not update the running program.

Also check that the system prompt string is assigned to the system_prompt variable, not the model variable.

help me debug why my system prompt is not changing the AI personality

✔️ Awesome, I've got everything!

Great. Double check you have saved main.cpp with the original system prompt restored.

ⓧ I'd like to double check the full code

Your main.cpp should contain these sections in order. Scroll up and confirm each one is present:

  • Includes block (5 includes + using json statement).
  • OllamaClient class (constructor + chat() method + private members).
  • ChatSession class (constructor + send(), get_messages(), set_messages() methods + private members).
  • main() function with model, system_prompt, OllamaClient, ChatSession, and the while loop.

Your code is now professionally structured. The OllamaClient class handles all HTTP concerns, and the ChatSession class manages conversation state. Next up, you will push these classes further in the Secret Mission by adding the ability to save and load entire conversations as JSON files.

Secret mission

Save and Load Conversations

Your chatbot works great in a single session, but the moment you quit, the entire conversation vanishes. In this secret mission, you will add the ability to save conversations as JSON files on disk and load them back later. You will combine file I/O, JSON serialization, and the OOP patterns you have already built into a persistent chat history system.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • ~/ai-chat-cpp/ folder with source code and header files (local storage only).
  • Ollama app installed in Applications (background service for local AI inference).
  • Ollama models stored in ~/.ollama/ (the gemma4 model is approximately 5GB on disk).

Keep everything running

No action needed. Choose this if you plan to keep chatting with your AI client or want to build on this project.

  • Ollama uses no CPU or memory when idle. It only activates when you send a request.
  • The gemma4 model stays on disk in ~/.ollama/ and takes up approximately 5GB of storage.
  • Your project folder ~/ai-chat-cpp/ remains available for future edits.

Pause - I'll come back to this later

Stop the Ollama background service to free up memory. Your models and project files stay on disk for next time.

  • Click the Ollama icon in your menu bar (top right of your screen).
  • Click Quit Ollama.
  • Your downloaded models and project files remain on disk. Next time you want to chat, just open Ollama from Applications again.

Delete - I don't want to use this again

Remove all project resources and reclaim disk space. This cannot be undone.

  • Remove the gemma4 model by running this command in Terminal:
ollama rm gemma4
  • Uninstall Ollama completely by running these commands:
sudo rm -rf /Applications/Ollama.app
sudo rm /usr/local/bin/ollama
rm -rf ~/.ollama
  • Delete your project folder by running:
rm -rf ~/ai-chat-cpp

What about Xcode Command Line Tools?

Xcode Command Line Tools are a general-purpose developer toolkit used by many macOS applications and development workflows. You likely want to keep them installed even if you remove this project.

Nice Work!

Nice Work!

You just built a fully functional AI chatbot in C++ that communicates with a local language model over HTTP. From a blank file to a multi-turn conversation engine with clean OOP architecture, all running on your own machine.

Here's what you learned:

  • Built a REST API client in C++ that sends HTTP POST requests with JSON bodies to a local AI server and parses structured responses.
  • Practiced object-oriented programming by separating concerns into two classes: OllamaClient for HTTP communication and ChatSession for conversation state management.
  • Managed stateful conversations over a stateless protocol by accumulating the full message history and sending it with every request.
  • Secret Mission: Added persistent conversation storage with file I/O and JSON serialization, letting you save, load, and browse past conversations.

Ready to quiz yourself?