Talk to Your AI Agent from Your Phone

Connect your AI agent to your iPhone using Tailscale and Hermex.

Introduction

30 Second Summary

Your AI agent is powerful, but only when you are sitting at your computer. Imagine being able to pull out your phone and ask it to build something for you, no matter where you are.

In this project, you will give your existing Hermes Agent a web-based chat interface, connect your computer and iPhone through a private network, and install a native mobile app so you can talk to your agent from anywhere.

What You'll Build

A mobile-accessible AI agent setup that connects your local Hermes Agent to your iPhone through a secure mesh VPN and native iOS app.

By the end of this project, you'll have:

  • Hermes WebUI running on your computer, serving a browser-based chat interface for your agent.
  • Tailscale mesh VPN connecting your computer and iPhone so they can reach each other from anywhere.
  • Hermex native iOS app on your phone, connected to your agent, with a working app built entirely from your phone.
  • Secret Mission: Set up a scheduled cron job so your agent works autonomously on a recurring task while you sleep.

Are there any prerequisites?

Yes. You need to have completed the "Set Up Hermes Agent for Free" project first. That project installs Hermes Agent on your computer and configures an OpenRouter API key with a free model.

You also need an iPhone for the Tailscale and Hermex mobile steps.

Install Hermes WebUI

Your Hermes Agent is installed and working, but right now you can only talk to it through the terminal. To access it from your phone later, you need a web-based interface that runs in a browser.

Hermes WebUI is the official browser-based chat interface for Hermes Agent. It auto-detects your existing agent installation, shares the same sessions and config, and gives you a full chat experience at a local URL. In this step, you will install it and confirm everything works.

In this step, get ready to:
  • Clone the Hermes WebUI repository and run the bootstrap script.
  • Launch the WebUI and complete the onboarding wizard.
  • Send a test message to confirm your agent responds through the web interface.
Clone the repository and run the bootstrap script

The WebUI lives in its own repository on GitHub. The bootstrap script handles everything: it finds your existing Hermes Agent, installs Python dependencies, starts the web server, and opens the onboarding wizard in your browser.

  • Navigate to the directory of your choice and clone the Hermes WebUI repository by running these commands:
cd ~
git clone https://github.com/nesquena/hermes-webui.git hermes-webui
cd hermes-webui
python3 bootstrap.py

What does this do?

  • cd ~ moves you to your home directory so the WebUI installs at ~/hermes-webui.
  • git clone downloads the full WebUI repository into a folder called hermes-webui.
  • python3 bootstrap.py detects your existing Hermes Agent, installs WebUI dependencies, starts the web server, waits for a health check, and opens your browser automatically.

💡 What should I expect to see?

The bootstrap script takes 30-60 seconds. You will see terminal output as it detects your agent, installs dependencies, and starts the server. Once it finishes, your default browser opens automatically to http://localhost:8787 with the onboarding wizard loaded.

  • Confirm the hermes-webui folder exists by running:
ls ~/hermes-webui
  • You should see files like bootstrap.py, server.py, and start.sh in the output.

Bootstrap failed or browser didn't open?

  • If you see a Python error, make sure you have Python 3 installed. Run python3 --version to check.
  • If the browser did not open, manually visit http://localhost:8787 in your browser.
  • If you see an error about Hermes Agent not being found, confirm your prerequisite project is complete and ~/.hermes exists.

Help me troubleshoot the Hermes WebUI bootstrap script failing on my machine.

Complete the onboarding wizard

The bootstrap script opens the onboarding wizard automatically in your browser. Since you already have Hermes Agent configured with OpenRouter from the prerequisite project, the wizard detects your existing setup and confirms it.

  • In your browser at http://localhost:8787, you should see the onboarding wizard's first screen showing a system check.
  • The wizard checks that Hermes Agent is importable, your provider (OpenRouter) is configured, and your config paths are set.
  • If the wizard detects your existing config.yaml, click **Confirm** to acknowledge your existing configuration.
  • Select a workspace when prompted. The default is fine for this project.
  • Skip the password step for now (you will add one in a later step for remote access).
  • Click **Finish** to complete the wizard.

What is the onboarding wizard doing?

The wizard is a 5-step process: system check, provider configuration, workspace selection, password setup, and finish. Since your Hermes Agent already has OpenRouter configured, most steps are pre-filled or skippable.

The wizard writes its completion state to a settings.json file so it only runs once. Future launches go straight to the chat interface.

  • After completing the wizard, you should see the main chat interface with a text input at the bottom and your session list on the left.

Wizard shows provider incomplete?

  • If the wizard says your provider setup is incomplete, select **OpenRouter** from the provider list and enter your existing API key when prompted.
  • Click **Test connection** to verify the connection works before continuing.

Help me get past the onboarding wizard when it says my provider is incomplete.

Send a test message to your agent

With the wizard complete, you are now in the chat interface. Time to confirm your agent actually responds through the web UI.

  • Click the text input field at the bottom of the chat interface.
  • Type Hello! What can you help me with? and press **Enter** to send.
  • Wait a few seconds for your agent to respond. You should see a reply appear in the chat window.

Your agent responds in the browser, confirming the WebUI is connected to your Hermes Agent and routing messages through your OpenRouter provider.

No response from the agent?

  • Check that the terminal where you ran python3 bootstrap.py is still running. If it was closed, restart with ./start.sh from the ~/hermes-webui directory.
  • If you see an error in the chat, your OpenRouter API key may have expired. Confirm it is still valid from your OpenRouter dashboard.
  • Check the terminal output for error messages. Network or authentication issues appear there.

Help me debug why my Hermes Agent is not responding through the WebUI.

Your agent now has a proper web interface. Next up, you will install Tailscale so your phone can reach this WebUI from anywhere.

Set Up Tailscale on Your Computer

Right now, your agent's web interface only works on the computer running it. To reach it from your phone, both devices need to be on the same private network.

Tailscale is a mesh VPN that creates a secure private network between your devices. In this step, you'll install Tailscale on your computer and get it connected.

In this step, get ready to:
  • Install Tailscale on your computer.
  • Create a free Tailscale account and sign in.
  • Confirm your computer's Tailscale IP address.
Download and install Tailscale

Tailscale runs as a lightweight app on your computer. It connects your devices through a private network without any port forwarding or firewall changes.

  • Check if Tailscale is already installed on your computer by running this command in your terminal:
tailscale version

✔️ I see a version number

Tailscale is already installed on your computer. Skip ahead to the next substep to sign in.

ⓧ Command not found

Tailscale is not installed yet. Follow the instructions for your operating system below.

macOS

  • Open the Tailscale download page in your browser:
  • https://tailscale.com/download
  • Click the macOS download option. This downloads the standalone .pkg installer from Tailscale's package server.
  • Open the downloaded .pkg file and follow the installer prompts to complete the installation.
  • After installation, Tailscale appears as a new icon in your menu bar at the top of your screen.
Why the standalone installer?

Tailscale recommends the standalone variant over the Mac App Store version. It offers more features and is not subject to App Store sandboxing limitations.

Windows

  • Open the Tailscale download page in your browser:
  • https://tailscale.com/download
  • Click the Windows download option. The .exe installer works on both 32-bit and 64-bit Windows automatically.
  • Run the downloaded installer and follow the prompts.
  • After installation, a Tailscale icon appears in your system tray (bottom-right corner of your screen). If you don't see it, click the up arrow to find it in the overflow area.

Linux

  • Install Tailscale by running this command:
curl -fsSL https://tailscale.com/install.sh | sh
  • After the script finishes, start the Tailscale service by running:
sudo tailscale up

This opens a URL in your terminal that you'll use to sign in during the next substep.

Installation not working?

Make sure you're downloading Tailscale from the official site (tailscale.com/download). If the macOS installer asks for permission to install a System Extension, click Allow in System Settings > Privacy & Security.

help me troubleshoot my Tailscale installation

Create your Tailscale account and sign in

Tailscale uses your existing identity provider (Google, Microsoft, GitHub, etc.) to create your account. There is no separate password to manage.

  • Click the Tailscale icon in your menu bar (macOS) or system tray (Windows).
  • Click Log in. This opens a browser window.
  • Choose an identity provider (Google, Microsoft, GitHub, or Apple) and sign in. This creates your free Tailscale account automatically.
  • After signing in, the browser confirms your device is connected. Close this tab and return to the Tailscale app.

What is the Tailscale Personal plan?

Tailscale's Personal plan is free forever. It supports up to 6 users with unlimited devices. You will not be charged for anything in this project.

  • Confirm your computer is connected by clicking the Tailscale icon again. You should see a green "Connected" status.

Where is the Tailscale admin console?

You can also verify your device at https://login.tailscale.com/admin/machines. Your computer should appear in the machines list with a green dot.

🙋‍♀️ Not seeing Connected status?

Make sure you completed the sign-in flow in your browser. If the Tailscale icon shows a gray or disconnected state, click Log in again and re-authenticate.

help me connect my computer to Tailscale

Find your computer's Tailscale IP address

Every device on your Tailscale network gets a unique IP address in the 100.x.x.x range. This is the address your phone will use to reach your computer.

  • Find your Tailscale IP by running this command in your terminal:
tailscale ip -4

You should see a single IP address starting with 100. (for example, 100.64.1.23). Write this down. You'll need it in the next step.

Find it in the app too

On macOS, click the Tailscale icon in your menu bar to see your IP address listed under your device name. On Windows, hover over the Tailscale icon in the system tray.

🙋‍♀️ Command returns an error?

If you see an error about Tailscale not running, make sure the Tailscale app is open and showing Connected status. On Linux, run sudo tailscale up first to start the connection.

help me find my Tailscale IP address

Your computer is now on a private Tailscale network with its own IP address. Next up, you'll connect your iPhone to the same network and access the WebUI remotely.

Connect Your Phone and Access the WebUI

Your computer is now on a private Tailscale network with its own IP address. The missing piece is getting your iPhone onto that same network so it can reach the WebUI.

In this step, you'll install Tailscale on your iPhone, restart the WebUI so it accepts connections from other devices, and confirm you can chat with your agent from your phone's browser.

In this step, get ready to:
  • Install Tailscale on your iPhone and join the same private network.
  • Restart the WebUI with remote access and password protection enabled.
  • Access the WebUI from Safari on your phone and chat with your agent.
Install Tailscale on your iPhone

Your computer already has a Tailscale IP. Now your iPhone needs one too, so both devices can see each other on the same private network.

  • On your iPhone, open the App Store and search for Tailscale.
  • Tap Get to install the app.
  • Open the Tailscale app and tap Get Started.
  • When prompted, tap Allow to install the VPN configuration. This is how Tailscale routes traffic between your devices.
  • Tap Allow for push notifications when prompted. These alert you when keys need reauthentication.
  • Tap Log in and sign in with the same identity provider (Google, Apple, GitHub, etc.) you used on your computer.

Why the same account?

Tailscale groups all your devices into a private network called a tailnet. Both devices must be signed in to the same account to join the same tailnet and see each other.

  • After signing in, confirm that the Tailscale app shows your iPhone as Connected.
  • Switch back to the Tailscale app on your computer. You should see both your computer and your iPhone listed as connected devices.

Only see one device?

Make sure you signed in with the exact same account on both devices. Different providers (e.g. Google on one, Apple on the other) create separate tailnets.

If your iPhone shows a connection error, toggle the VPN off and on in the Tailscale app, or check that your iPhone has an active internet connection.

Why can't my phone see my computer on Tailscale?

Restart the WebUI with remote access enabled

Right now the Hermes WebUI only listens on localhost. That means only your computer can reach it. To let your phone connect, you need to restart it so it listens on all network interfaces and requires a password.

  • Switch back to the terminal where the WebUI is running.
  • Stop the current WebUI process by pressing Ctrl+C. If you already stopped it earlier, skip this.
  • Restart the WebUI with remote access and password protection by running this command:
HERMES_WEBUI_HOST=0.0.0.0 HERMES_WEBUI_PASSWORD=your-secret ./start.sh
  • Replace your-secret with any password you choose. You will type this on your phone to access the WebUI.

What does this command do?

HERMES_WEBUI_HOST=0.0.0.0 tells the WebUI to accept connections from any device on your network, not just your computer.

HERMES_WEBUI_PASSWORD=your-secret sets a password so only you can access the WebUI remotely. Anyone who tries to connect must enter this password first.

  • Confirm the WebUI starts successfully. You should see output indicating it is listening on 0.0.0.0:8787.

WebUI won't start?

Make sure you are running the command from inside the ~/hermes-webui directory. If you're somewhere else, run cd ~/hermes-webui first.

If you see an error about port 8787 already being in use, the previous process may not have stopped. Press Ctrl+C again or close and reopen your terminal.

Help me troubleshoot my Hermes WebUI not starting

Access the WebUI from your phone

Your WebUI is now listening for connections from any device on your Tailscale network. Time to open it on your iPhone.

  • You will need your computer's Tailscale IP from the previous step. If you need to find it again, run this in a new terminal tab:
tailscale ip -4
  • On your iPhone, open Safari.
  • In the address bar, type this URL (replacing the IP with your own):
http://[[TAILSCALE_IP="your computer's Tailscale IP (e.g. 100.64.0.1)"]]:8787
  • When the password prompt appears, enter the password you set in the previous substep.
  • The Hermes WebUI should load in your phone's browser with your agent ready to chat.
  • Send a test message like "Hello from my phone" and confirm your agent responds.

Why does this work from anywhere?

Tailscale creates an encrypted tunnel between your devices regardless of what Wi-Fi network they are on. Your phone could be on cellular data across town and still reach your computer's WebUI through the Tailscale IP.

🙋‍♀️ Page won't load on your phone?

Check that Tailscale is connected on your iPhone. Open the Tailscale app and confirm it shows Connected.

Make sure you typed http:// (not https://) at the beginning of the URL. The WebUI does not use SSL by default.

Verify the Tailscale IP is correct. On your computer, run tailscale ip -4 and compare it to what you typed in Safari.

Help me connect my phone to the Hermes WebUI through Tailscale

Your agent is now reachable from your phone's browser anywhere in the world. Next up, you'll install a native iOS app that gives you a much more polished mobile experience.

Install Hermex and Build an App from Your Phone

Your phone can now reach the WebUI through Tailscale, but a browser tab is not the best experience for chatting with an agent on mobile.

In this step, you will install Hermex, a native iOS app purpose-built for Hermes Agent. It gives you session sync, model switching, and a share extension. Then you will prove the full setup works by having your agent build a complete app triggered entirely from your phone.

In this step, get ready to:
  • Install Hermex on your iPhone and connect it to your agent server.
  • Send a build prompt to have your agent create a habit tracker app.
  • Verify the finished app works on your computer.
Install Hermex and connect to your server

Hermex is listed as "Hermes Agent Mobile" on the App Store. It connects directly to your Hermes WebUI server and syncs all your sessions.

  • On your iPhone, open the App Store.
  • Search for Hermes Agent Mobile and install the app (free, by Uzair Ansar).
  • Open Hermex once it finishes installing.
  • When prompted for your server URL, enter:
http://[[TAILSCALE_IP="your Tailscale IP address"]]:8787
  • Enter the password you set when you started the WebUI with remote access in Step 3.
  • Tap Connect (or the equivalent confirmation button).
  • Confirm you can see your existing sessions from the WebUI listed in the Hermex interface.

What is Hermex syncing?

Hermex connects to the same server your browser uses. Any sessions you started in the WebUI earlier show up here automatically, and any sessions you start from your phone appear in the WebUI too.

🙋‍♀️ Not seeing your sessions?

Make sure Tailscale is still connected on both your computer and iPhone. Check the Tailscale app on your iPhone and verify it shows a green "Connected" status.

Double-check that your server URL uses http:// (not https://) and includes port 8787 at the end.

help me connect Hermex to my Hermes WebUI server

Send a build prompt from your phone

Now for the real test. You will ask your agent to build a complete web app, with the entire conversation happening from your phone.

  • In Hermex, tap the button to start a new session.
  • Copy and paste the following prompt into the message field:
Build me a habit tracker app. Single file index.html, all CSS and JS inline, no external dependencies. Track daily habits with checkboxes, show a streak counter for each habit, and save progress to localStorage. Save it to ~/hermes-app/habit-tracker.html
  • Send the message.
  • If Hermex shows any tool-call approval prompts (asking permission to create files or run commands), approve them.
  • Wait for the agent to finish building. You will see it create the directory, write the HTML file, and confirm completion in the chat.

What is happening on your computer?

Even though you are typing on your phone, the agent runs on your computer. It creates the ~/hermes-app/ directory and writes habit-tracker.html there. Your phone is just a remote control.

💡 Keep your computer awake

Your agent needs your computer to be awake and the WebUI process running. If your computer goes to sleep mid-build, the task will stall until it wakes up.

Verify the app on your computer

The agent has finished building. Time to check the result on your computer.

  • On your computer, confirm the file exists by running this command in your terminal:
ls ~/hermes-app/habit-tracker.html

You should see the file path printed back, confirming it exists.

  • Open the habit tracker in your browser by running:

macOS

open ~/hermes-app/habit-tracker.html

Windows

In your WSL2 terminal, open the folder containing your app:

explorer.exe .

Run this from the ~/hermes-app directory (cd ~/hermes-app first if needed). Windows File Explorer opens showing your files. Double-click habit-tracker.html to open it in your default browser.

  • In the browser, try adding a new habit (e.g. "Drink water").
  • Check the habit off for today using the checkbox.
  • Confirm the streak counter increments to 1.
  • Refresh the page and verify your habits and streaks are still there (saved to localStorage).

File not found?

If ls returns "No such file or directory", your agent may not have finished building. Switch back to Hermex on your phone and check if the session shows a completed response.

If the agent produced an error, start a new session and resend the prompt. Make sure your OpenRouter API key is still active and has available credits.

help me figure out why my agent did not create the habit tracker file

You just built a working web app without ever touching your computer's keyboard. Your phone told the agent what to build, and the agent did all the work locally on your machine.

Secret mission

Set Up a Scheduled Agent Task

Your agent can do more than wait for you to type. What if it could run tasks on its own schedule, working while you sleep, and have the results waiting when you wake up? In this secret mission, you will configure a cron job so your agent autonomously maintains a summary of everything it has built for you.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally with free services, so there are no ongoing costs.

Resources you used:

  • Hermes WebUI installation at ~/hermes-webui (free, local files only).
  • Tailscale Personal plan on your computer and iPhone (free, up to 6 users, unlimited devices).
  • Hermex iOS app (free, no in-app purchases).
  • Agent-created files at ~/hermes-app/ (free, local files only).

Keep everything running

No action needed. Choose this if you are still actively building with your agent from your phone.

  • Leave the WebUI running with HERMES_WEBUI_HOST=0.0.0.0 and your password set. Your phone can reach your agent whenever Tailscale is connected and the WebUI process is running.
  • For persistent access without keeping a terminal window open, run ctl.sh start from inside the ~/hermes-webui directory to launch the WebUI as a background daemon.
  • Tailscale stays connected in the background on both devices. No further action needed.

Pause - I'll come back to this later

Shut down the WebUI process to free up memory. Tailscale stays connected but there is nothing to reach until you restart.

  • Press Ctrl+C in the terminal running the WebUI to stop it.
  • Restart any time by running this command from inside the ~/hermes-webui directory:
HERMES_WEBUI_HOST=0.0.0.0 HERMES_WEBUI_PASSWORD=your-secret ./start.sh
  • Replace your-secret with the password you chose earlier.

Delete - I don't want to use this again

Remove all project resources and start fresh. Your Hermes Agent installation from the prerequisite project is unaffected.

  • Stop the WebUI if it is still running by pressing Ctrl+C in its terminal window.
  • Remove the Hermes WebUI and the agent-created app files by running:
rm -rf ~/hermes-webui
rm -rf ~/hermes-app
  • Uninstall Tailscale from your computer:
  • On macOS, delete Tailscale.app from your Applications folder and empty the Trash.
  • On Windows, go to Settings, then Apps. Find Tailscale and press Uninstall.
  • On Linux, run:
sudo apt remove tailscale
  • Uninstall Tailscale from your iPhone by long-pressing the Tailscale app icon, tapping Remove App, then tapping Delete App.
  • Delete the Hermex app from your iPhone using the same method.

That's a Wrap!

That's a Wrap!

Nice work. You just turned your local AI agent into a mobile-accessible powerhouse that you can reach from anywhere in the world.

You've learned how to:

  • Installed Hermes WebUI as a browser-based chat interface for your existing Hermes Agent, giving it a proper web UI at http://localhost:8787.
  • Set up Tailscale to create a private mesh network between your computer and iPhone, then connected Hermex for a native mobile experience with session sync and model switching.
  • Built a working web app triggered entirely from your phone, proving your agent is accessible from anywhere without needing a terminal or the same Wi-Fi network.
  • (Secret Mission) Configured a scheduled cron job so your agent works autonomously on recurring tasks while you sleep.

Ready to quiz yourself?