Build a Voice AI Customer Support Agent

Build a voice AI agent that holds real-time conversations using ElevenLabs.

Introduction

30 Second Summary

Every time you call a customer support line and hear a friendly voice guiding you through options, there is a chance you are talking to an AI. The technology behind these voice agents used to require large engineering teams, but today you can build one yourself in under an hour.

In this project, you will build a voice AI customer support agent for a fictional coffee shop called "Bean & Brew" using ElevenLabs. Your agent will hold real-time voice conversations, answer questions from a knowledge base, and run as an embeddable widget anyone can access in their browser.

What You'll Build

A fully functional Conversational AI agent that acts as a customer support representative for a coffee shop, powered by ElevenLabs ElevenAgents and capable of holding natural voice conversations in real time.

By the end of this project, you'll have:

  • A configured voice AI agent with a custom persona, system prompt, and natural-sounding voice.
  • A knowledge base that grounds the agent's responses in accurate business information instead of hallucinated answers.
  • An embeddable web widget that lets anyone talk to your agent in a browser, plus evaluation criteria that automatically score each conversation for quality.
  • Secret Mission: Add multi-step conversation workflows that route customers to different paths based on their intent.

Are there any prerequisites?

None. This project uses a completely no-code approach through the ElevenLabs web dashboard. You do not need any programming experience, and the free tier (15 minutes of calls included) covers everything you will build.

Before We Start

Before We Start...

Set Up Your ElevenLabs Account

Before you can build your voice AI agent, you need a platform to host it. ElevenLabs is the leading voice AI platform, offering thousands of natural-sounding voices and a no-code dashboard for building conversational agents.

In this step, you will create your free ElevenLabs account and navigate to the ElevenAgents dashboard where you will build your agent in the next steps.

In this step, get ready to:
  • Create your free ElevenLabs account.
  • Navigate to the ElevenAgents dashboard.
  • Verify your free tier allowance.
Create your free ElevenLabs account

ElevenLabs offers a free plan that includes 15 minutes of voice calls. That is more than enough to build and test your agent during this project.

  • Open your web browser and go to elevenlabs.io.
  • Click Sign up in the top right corner of the page.
  • Create your account using your email address, or sign in with Google or GitHub.
  • If you signed up with email, check your inbox for a verification email from ElevenLabs and click the verification link.

What sign-in method should I choose?

Any option works for this project. Google or GitHub sign-in is faster because it skips the email verification step. If you use email, you will need to confirm your address before you can proceed.

After signing up, you will land on the ElevenLabs dashboard.

Navigate to ElevenAgents

ElevenAgents is the dedicated section of ElevenLabs for building conversational AI agents. This is where you will create, configure, and test your voice agent.

  • In the left sidebar of the dashboard, find and click Agents.
  • You should see the ElevenAgents dashboard with the option to create a new agent.

Don't see Agents in the sidebar?

Make sure you are signed in and your email is verified. If the sidebar looks different, try refreshing the page. The ElevenLabs dashboard occasionally updates its layout, but the Agents section is always accessible from the left sidebar navigation.

help me find the Agents section in my ElevenLabs dashboard

Verify your free tier allowance

Before building your agent, confirm that your free plan is active and covers everything you need for this project.

  • Navigate to your subscription or billing page to confirm the Free plan is active.
  • Confirm your plan includes the following features:
  • 15 minutes of calls.
  • 4 concurrent calls.
  • Workflow Builder.
  • Knowledge Base.
  • Multilingual support.
  • Widget embedding.

Will this cost me anything?

The Free plan costs $0 per month and does not require a credit card. You will use roughly 5 to 8 minutes of call time during testing, which is well within the 15-minute allowance.

Voice calls are charged by connection duration, with a 95% discount for periods of silence longer than 10 seconds. This means your testing time goes even further.

Your account is ready. Next up, you will create your first voice agent and give it a personality, a name, and a voice.

Create and Configure Your Voice Agent

Your account is ready and the ElevenAgents dashboard is open. Now it's time to bring your coffee shop assistant to life.

In this step, you'll create a brand new voice agent and shape its personality. You'll define how it greets customers, what it knows about, and how it sounds when it speaks.

In this step, get ready to:
  • Create a new agent from a blank template.
  • Write a first message and system prompt that define your agent's personality.
  • Select a voice that matches your friendly barista persona.
Create a new agent

Every voice agent starts as a blank slate. You'll give yours a name and a starting template that you'll customize from scratch.

  • In the ElevenAgents dashboard, click Create Blank Agent.
  • Enter the name Bean & Brew Support in the agent name field.
  • Select the Blank template option.
  • Confirm the creation. You should land on the agent configuration page with empty settings ready to fill in.

What is ElevenAgents?

ElevenAgents is ElevenLabs' platform for building conversational AI agents. It handles the complex pipeline of listening to speech, understanding it, generating a response, and speaking it back. All without writing code.

Configure the agent's first message and system prompt

The first message is what your agent says the moment a customer connects. The system prompt is the hidden set of instructions that defines your agent's entire personality and behavior.

  • Navigate to the Agent tab in your agent's settings.
  • Find the First Message field and replace any placeholder text with the following:
Hey there! Welcome to Bean and Brew coffee. I'm Alex, your virtual barista. What can I help you with today?

Why does the first message matter?

The first message sets the tone for the entire conversation. It tells the customer who they're talking to and invites them to speak. A warm, specific greeting builds trust immediately.

  • Scroll down to the System Prompt field. This is where you define how your agent behaves behind the scenes.
  • Paste the following system prompt into the field:
You are Alex, a friendly and knowledgeable virtual barista at Bean & Brew coffee shop.

Your role:
- Answer questions about our menu, store hours, and locations
- Help customers choose drinks based on their preferences
- Provide information about our ordering process

Guidelines:
- Keep your answers concise and conversational
- Maintain a warm, welcoming tone at all times
- If you are unsure about something, ask a clarifying question rather than guessing
- Stay in character as a barista who loves coffee and enjoys helping people find their perfect drink

What does the system prompt do?

The system prompt is your agent's rulebook. It never gets spoken aloud, but it controls everything the agent says. Think of it as the training manual you'd give a new employee on their first day.

The "Guidelines" section is especially important. Telling the agent to ask clarifying questions prevents it from making up answers when it doesn't have the information.

Keep prompts specific

Vague prompts like "be helpful" produce generic responses. Specific instructions like "stay in character as a barista" give the agent a clear identity to maintain throughout the conversation.

Select a voice for your agent

Your agent needs a voice that matches its personality. ElevenLabs offers over 5,000 voices across 31 languages, so you have plenty of options.

  • Click on the Voice tab in your agent's settings.
  • Browse the voice library for a voice that sounds friendly and approachable.
  • Click the preview or play button next to a voice to hear a sample.
  • Try 2-3 different voices before making your choice. Look for one that sounds natural and conversational, not robotic or overly formal.
  • Select your preferred voice by clicking on it.

What is text-to-speech?

Text-to-speech is the technology that converts written text into spoken audio. ElevenLabs uses AI models that produce natural-sounding voices with realistic intonation and emotion, making your agent sound like a real person rather than a robotic reader.

Balance quality and latency

Higher quality voices may increase response time slightly. For a customer support agent, a natural-sounding voice with quick response times is more important than a studio-quality voice with noticeable pauses.

Don't see the Voice tab?

Make sure you're inside your agent's configuration page, not the main ElevenAgents dashboard. Click on your Bean & Brew Support agent name first, then look for the tab navigation at the top of the settings area.

help me find the Voice tab in my ElevenAgents agent settings

Confirm that your agent now has all three pieces configured: the name Bean & Brew Support, the first message and system prompt in the Agent tab, and a selected voice in the Voice tab.

Your agent has a personality, a greeting, and a voice. Next up, you'll give it actual knowledge about Bean & Brew so it can answer real customer questions accurately.

Build the Knowledge Base

Your agent has a personality and a voice, but right now it has no actual information to draw from. If someone asks about the menu or store hours, the agent would have to guess or make things up.

A knowledge base solves this by giving your agent a library of facts it can reference during conversations. In this step, you will upload Bean & Brew's business details so your agent can answer customer questions accurately.

In this step, get ready to:
  • Prepare and upload Bean & Brew business information to the knowledge base.
  • Add FAQ content covering common customer questions.
  • Test the agent to verify it references the knowledge base accurately.
Prepare your knowledge base content

Before you upload anything, you need the actual information your agent will reference. You will create two text documents: one with core business details and one with frequently asked questions.

  • Open a text editor on your computer (Notepad, TextEdit, or any plain text app).
  • Create a new file and paste in the following Bean & Brew business details:
Bean & Brew Coffee Shop - Business Information

== MENU ==

Hot Drinks:
- House Blend Coffee: $3.50 (small), $4.50 (medium), $5.50 (large)
- Cappuccino: $4.75 (small), $5.75 (medium), $6.75 (large)
- Vanilla Latte: $5.25 (small), $6.25 (medium), $7.25 (large)
- Chai Tea Latte: $4.50 (small), $5.50 (medium), $6.50 (large)
- Hot Chocolate: $4.00 (small), $5.00 (medium), $6.00 (large)

Cold Drinks:
- Iced Americano: $4.00 (small), $5.00 (medium), $6.00 (large)
- Cold Brew: $4.50 (small), $5.50 (medium), $6.50 (large)
- Iced Matcha Latte: $5.50 (small), $6.50 (medium), $7.50 (large)
- Strawberry Smoothie: $6.00 (one size)

Food:
- Butter Croissant: $3.75
- Avocado Toast: $8.50
- Blueberry Muffin: $4.25
- Granola Bowl: $7.00

== STORE HOURS ==

Monday to Friday: 6:00 AM - 8:00 PM
Saturday: 7:00 AM - 9:00 PM
Sunday: 7:00 AM - 6:00 PM

== LOCATIONS ==

1. Downtown: 42 Maple Street, Downtown District
   - Parking: Street parking and public garage on Elm Avenue
   - Seating: 30 indoor seats, 12 outdoor patio seats

2. Riverside: 118 River Road, Riverside Plaza
   - Parking: Free lot behind the building
   - Seating: 24 indoor seats, 8 outdoor seats

== ORDERING PROCESS ==

1. Walk in and order at the counter, or call ahead for pickup.
2. Phone orders: Call (555) 123-BREW for Downtown, (555) 456-BREW for Riverside.
3. Average wait time: 3-5 minutes for drinks, 5-8 minutes for food items.
4. We accept cash, all major credit cards, and mobile payments.

== POPULAR RECOMMENDATIONS ==

- Most popular drink: Vanilla Latte
- Staff favorite: Iced Matcha Latte
- Best seller food item: Avocado Toast
- Customer favorite combo: Cold Brew + Butter Croissant

What is this document for?

This text document gives your agent concrete facts to reference. Without it, the agent would either refuse to answer or hallucinate incorrect information about menu items and prices.

The structure (clear headings, consistent formatting) helps the platform index the content so the agent can quickly find relevant answers during a live conversation.

  • Save this file on your computer. You will upload it in the next substep.
  • Create a second text file with the following FAQ content:
Bean & Brew Coffee Shop - Frequently Asked Questions

== ALLERGENS ==

Q: Do you have allergen information for your menu items?
A: Yes. Our croissants and muffins contain gluten and dairy. The granola bowl contains nuts (almonds and walnuts). All drinks can be made dairy-free on request. Please inform your barista of any allergies when ordering.

Q: Are any of your food items gluten-free?
A: The Granola Bowl is gluten-free. We also offer gluten-free muffins on weekends (Saturday and Sunday only).

== DAIRY ALTERNATIVES ==

Q: What milk alternatives do you offer?
A: We offer oat milk, almond milk, soy milk, and coconut milk. Oat milk and almond milk are available at no extra charge. Soy milk and coconut milk add $0.50.

Q: Which dairy alternative do you recommend for lattes?
A: Our baristas recommend oat milk for the creamiest texture in lattes and cappuccinos.

== LOYALTY PROGRAM ==

Q: Do you have a loyalty program?
A: Yes! Our Bean Points program rewards you for every purchase. Earn 1 point per dollar spent. Redeem 50 points for a free drink of any size, or 75 points for a free food item.

Q: How do I sign up for Bean Points?
A: Ask any barista to sign you up at the counter. You just need to provide your name and phone number. You can also sign up on our website at beanandbrew.com/rewards.

== DELIVERY ==

Q: Do you offer delivery?
A: We partner with local delivery apps for the Downtown location only. Delivery is available during store hours with a $3.00 delivery fee. Riverside location is pickup-only at this time.

Q: What is the delivery radius?
A: We deliver within a 3-mile radius of our Downtown location at 42 Maple Street.
  • Save this second file on your computer.
Upload to the knowledge base

Now that you have your content ready, you will add both documents to your agent's knowledge base in the ElevenLabs dashboard.

  • Return to your Bean & Brew Support agent settings in the ElevenLabs dashboard.
  • In the Agent tab, scroll down to the Knowledge Base section.
  • Click Add document.
  • Choose the option to upload a new document.
  • Upload your bean-and-brew-details.txt file.
  • Wait a moment for the platform to index the document. You should see it appear in the knowledge base list.

What does indexing mean?

When you upload a document, ElevenLabs breaks it into smaller chunks and creates a searchable index. During a live conversation, the agent searches this index to find the most relevant chunk that answers the user's question.

  • Click Add document again to add a second entry.
  • Upload your bean-and-brew-faq.txt file.
  • Confirm that both documents now appear in the Knowledge Base section of your agent settings.

Keep documents focused

Separating business details from FAQs makes it easier to update one without touching the other. If your hours change next month, you only need to update the first document.

Don't see your documents listed?

Make sure your files are in a supported format like TXT, PDF, or DOCX. Plain text files work best for this project.

If the upload seems stuck, try refreshing the page and uploading again. Help me troubleshoot my knowledge base upload.

Test the knowledge base integration

The real test is whether your agent actually uses the knowledge base content when answering questions. You will run a test conversation and ask specific questions that require knowledge base lookups.

  • Click the Test AI Agent button in the dashboard.
  • Ask your agent: "What's your most popular drink?"
  • Verify the agent responds with "Vanilla Latte" (from the popular recommendations section of your document).
  • Ask: "Are you open on Sundays?"
  • Verify the agent responds with the Sunday hours: 7:00 AM to 6:00 PM.
  • Ask: "Do you have oat milk?"
  • Verify the agent confirms oat milk is available at no extra charge (from the FAQ document).

What if the agent gets something wrong?

If your agent makes up information instead of referencing the knowledge base, check that both documents are listed and indexed in the Knowledge Base section. Sometimes it takes a few seconds for newly uploaded content to become available.

You can also refine your content by making headings clearer or adding more explicit answers to common questions.

Your agent now has real information to draw from during conversations. Next up, you will set up evaluation criteria to automatically score how well the agent performs and embed it as a widget anyone can use in their browser.

Set Up Evaluation and Embed Your Agent

Your Bean & Brew agent can answer questions accurately, but how do you know if it is actually helping customers? Without measurement, you are flying blind.

In this step, you will add evaluation criteria that automatically score every conversation. You will also set up data collection to extract insights, then embed your agent as a widget anyone can use in their browser.

In this step, get ready to:
  • Configure evaluation criteria that automatically assess conversation quality.
  • Set up data collection to extract customer intent from every conversation.
  • Embed your agent as a web widget and test a full voice conversation.
Configure evaluation criteria

Evaluation criteria tell ElevenAgents what "success" looks like for a conversation. After every call, the platform passes the transcript to an LLM that checks whether your defined goals were met.

  • In your Bean & Brew Support agent settings, click the Analysis tab.
  • Click Add criteria to create your first evaluation criterion.
  • In the Identifier field, enter resolved_customer_query.
  • In the Description field, enter a prompt that defines success: the agent answered the customer's question using knowledge base information or correctly directed them to an appropriate channel.
  • Click Add criteria again to add a second criterion.
  • Set the Identifier to maintained_persona.
  • Set the Description to a prompt that checks whether the agent stayed in character as a friendly barista throughout the conversation.

What do evaluation criteria do?

After every conversation ends, the platform reviews the full transcript against each criterion you defined. It returns one of three results: success, failure, or unknown. Each result includes a rationale explaining why the verdict was chosen.

This means you can track whether your agent is resolving queries and staying on-brand without manually listening to every call.

Configure data collection

Data collection lets you extract structured information from each conversation automatically. This is useful for understanding patterns across many calls.

  • Stay in the Analysis tab and scroll down to the Data Collection section.
  • Click Add item.
  • Set Data type to string.
  • Set Identifier to customer_intent.
  • Set Description to: Extract the primary intent of the customer from the conversation (e.g., menu inquiry, hours question, complaint, general chat).

What does the description field do?

The description is passed directly to the LLM as an instruction for how to extract data from the transcript. The more specific you make it, the better the extraction will be.

After each conversation, the extracted value appears in your call history alongside the evaluation results.

  • Confirm both your evaluation criteria (resolved_customer_query and maintained_persona) and your data collection item (customer_intent) are visible in the Analysis tab.

Don't see the Data Collection section?

Make sure you are in the Analysis tab of your agent's settings, not the main dashboard. The Data Collection section sits below the evaluation criteria on the same page.

help me find the Data Collection section in my ElevenLabs agent settings

Embed your agent as a web widget

The fastest way to let anyone talk to your agent is by embedding it as a widget in an HTML file. No server or framework is needed.

  • In your agent settings, click the Widget tab.
  • Look for the embed code snippet. It contains your agent's unique ID.
  • Open any text editor on your computer (TextEdit on macOS, Notepad on Windows).
  • Paste the following HTML into a new file, replacing the agent ID with your own:
<!DOCTYPE html>
<html>
<head>
  <title>Bean and Brew Support</title>
</head>
<body>
  <h1>Bean and Brew Customer Support</h1>
  <p>Click the widget in the corner to start a voice conversation.</p>

  <!-- ElevenLabs voice agent widget -->
  <elevenlabs-convai agent-id="[[AGENT_ID="your agent ID from the Widget tab"]]"></elevenlabs-convai>
  <script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async type="text/javascript"></script>
</body>
</html>

What does this code do?

The <elevenlabs-convai> element is a custom HTML component that renders the voice widget. The agent-id attribute connects it to your specific agent on the ElevenLabs platform.

The <script> tag loads the widget library from unpkg.com/@elevenlabs/convai-widget-embed. This handles all the voice streaming, UI rendering, and connection to ElevenLabs' servers.

  • Save the file as index.html on your Desktop.
  • Double-click index.html to open it in your browser.
  • You should see a page with the heading "Bean and Brew Customer Support" and a widget icon in the bottom-right corner.
  • Click the widget icon and allow microphone access when prompted by your browser.
  • Have a short voice conversation with your agent. Try asking about menu items or store hours.

Allow microphone access

Your browser will ask for permission to use your microphone the first time you click the widget. Click Allow to enable voice input. Without it, the widget cannot hear you.

  • After your conversation ends, return to the ElevenLabs dashboard.
  • Click the Call History tab to view your test conversation.
  • Verify that evaluation results appear (success, failure, or unknown) for each criterion.
  • Check that the customer_intent data field shows an extracted value from your conversation.

Widget not appearing?

Make sure you replaced the agent-id value in your HTML with your actual agent ID from the Widget tab. The ID looks something like agent_7101k5zvyjhmfg983brhmhkd98n6.

Also confirm you saved the file with the .html extension (not .txt). On macOS, uncheck "If no extension is provided, use .txt" in the save dialog.

help me troubleshoot why my ElevenLabs widget is not showing up in my HTML file

Your voice agent is live, measurable, and shareable. In the Secret Mission, you will take it further by adding conversation workflows that route customers to different paths based on their intent.

Secret mission

Add Conversation Workflows

Your agent handles single-turn questions well. But real customers have different needs. Some want to place an order. Others have a complaint. Can you build a workflow that detects intent and routes each customer down the right conversation path?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on the ElevenLabs free tier, so there are no ongoing costs while your agent exists. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Bean & Brew Support agent (includes knowledge base, workflow, evaluation criteria, and data collection configuration).
  • Local index.html file with the embedded widget.
  • ElevenLabs account (Free plan).

Keep everything running

No action needed. Choose this if you are still actively building or want to keep testing your voice agent.

  • Your agent stays live and accessible via the widget embed.
  • Your 15 minutes of free call time remains available until used.
  • Your ElevenLabs account and all configurations persist.

Pause - I'll come back to this later

Free up your local files while keeping your agent configuration in the cloud. You can return and test again anytime.

  • Delete the local index.html file from your computer. You can recreate it later by copying the widget embed snippet from your agent settings.
  • Your agent, knowledge base, workflow, and evaluation criteria all remain saved in ElevenLabs.

Delete - I don't want to use this again

Remove all project resources and start fresh. This cannot be undone.

  • Delete the local index.html file from your computer by moving it to the Trash (macOS) or Recycle Bin (Windows).
  • In the ElevenLabs dashboard, open your Bean & Brew Support agent and delete it. Deleting the agent also removes its associated knowledge base entries, workflow, evaluation criteria, and data collection configuration.
  • To delete your ElevenLabs account entirely: click your profile icon in the top-right corner, select Settings, scroll down to the red Delete Account button, enter your email address to confirm, and click Delete Account. This is permanent and you cannot reuse the same email address.

Nice Work!

Nice Work!

You just built a fully functional voice AI customer support agent from scratch using ElevenLabs ElevenAgents. Your Bean & Brew barista can hold real-time voice conversations, answer questions from a knowledge base, and run entirely in a browser widget.

You've learned how to:

  • Built a voice AI agent with a custom persona, system prompt, and natural-sounding voice selected from the ElevenLabs voice library.
  • Created a knowledge base that grounds the agent's responses in accurate business information instead of hallucinated answers.
  • Embedded the agent as a web widget and configured evaluation criteria to automatically score conversation quality after every call.
  • Secret Mission: Added multi-step conversation workflows with conditional routing based on customer intent, directing callers to ordering, complaint handling, or general Q&A paths.

Ready to quiz yourself?