Polarion Defect Tracker Agent

Build an AI agent in Copilot Studio that queries Polarion defects in plain English.

Introduction

30 Second Summary

Test engineers spend a surprising amount of time navigating the same Polarion screens over and over to check defect statuses. What if you could just ask a question in plain English and get the answer instantly?

In this project, you will build a Defect Tracker AI agent in Microsoft Copilot Studio that connects to your company's Polarion ALM system via its MCP server. You'll configure the agent to answer natural-language questions about open defects filtered by project, sprint, PI, and status.

What You'll Build

A production-ready AI agent in Copilot Studio that queries your company's Polarion system through the Model Context Protocol, translating plain-English questions into structured defect data.

By the end of this project, you'll have:

A custom connector in Power Apps linked to your company's Polarion MCP OpenAPI descriptor with Bearer token authentication.
A Copilot Studio agent with generative orchestration enabled that understands natural-language defect queries.
Tested workflows for querying defects by project, sprint, PI, and status using precise agent instructions.
Secret Mission: Add requirement traceability so your agent can show which requirements are linked to defects.

Are there any prerequisites?

You need a Microsoft Copilot Studio license (through your work account), access to your company's Polarion MCP server endpoint, and a Polarion Personal Access Token (PAT). All work is done in browser-based portals, so no additional software installation is required.

Create Your Agent in Copilot Studio

Before you can connect your agent to Polarion, you need to create the agent itself in Microsoft Copilot Studio.

In this step, you will create a blank custom agent, confirm that generative orchestration is enabled (required for MCP tools), and write initial instructions that define how your agent responds to defect queries.

In this step, get ready to:

Create a new custom agent named "Polarion Defect Tracker" in Copilot Studio.
Verify generative AI orchestration is enabled.
Write initial agent instructions that define the agent's role and behaviour.

Navigate to Copilot Studio and create your agent

Copilot Studio is a browser-based platform where you build, test, and publish AI agents. You will start by signing in and creating a blank agent with a specific name and description.

In your browser, navigate to copilotstudio.microsoft.com.
Sign in with your work account (the same account that has your Copilot Studio license).
On the Agents page, select Create blank agent.

Why create a blank agent?

When you describe what you want in natural language, Copilot Studio auto-generates a name, description, and instructions. Creating a blank agent gives you full control over each of these fields from the start. This matters when you need precise instructions for MCP tool routing.

On the agent's Overview page, find the Details section and select Edit.
Set the Name to Polarion Defect Tracker.
Set the Description to Queries open defects from Polarion ALM filtered by project, sprint, PI, and status.
Select Save.

You should now see the Overview page for your new Polarion Defect Tracker agent with your name and description displayed in the Details section.

Don't see the Create blank agent option?

If you do not see the option, try selecting the Agents page from the left sidebar. Look for Create blank agent there. If you still cannot find it, confirm that your work account has the Copilot Studio license assigned.

Help me troubleshoot why I cannot create a blank agent in Copilot Studio

Verify generative orchestration is enabled

Generative orchestration is what allows your agent to reason about which MCP tool to call based on the user's natural-language question. Without it, MCP tools will not work.

From your agent's page, select Settings (gear icon in the top navigation).
In the Generative AI section, find Orchestration.
Confirm that "Use generative AI orchestration for your agent's responses?" is set to Yes.

What is generative orchestration?

Generative orchestration uses GPT-4.1 as the default model to decide which tools or topics to invoke based on the user's message. It is enabled by default for all newly created agents, so you should see it already set to Yes.

This feature is currently supported in English only. If your Polarion data uses English field names, you are all set.

If the toggle is already set to Yes, no changes are needed. If it is set to No, toggle it to Yes and select Save at the top of the page.

Toggle is greyed out?

If you cannot change the orchestration setting, your admin may have restricted it at the environment level. Contact your Power Platform administrator to enable generative orchestration for your environment.

Help me figure out why generative orchestration is disabled in my Copilot Studio environment

Add initial agent instructions

Agent instructions tell the AI how to behave, what tools to use, and how to format responses. The agent uses these instructions plus the tool descriptions to decide what to do with each user message.

Navigate back to your agent's Overview page.
Find the Instructions section and select Edit.
Paste the following instructions into the Instructions field:

You are a defect tracking assistant for test engineers.

Your role:
- Help users query open defects from the Polarion ALM system.
- Always ask for the project ID if the user has not provided one.
- Use the Polarion MCP tools to query work items. Never make up defect data.

Formatting:
- Present results as a numbered list.
- Each item should show: Defect ID, Title, Status, and Assignee.
- If no results are found, let the user know and suggest checking the project ID or filters.

What do these instructions do?

The role section establishes the agent's identity and scope. It will only answer Polarion defect queries.
"Always ask for the project ID" prevents the agent from guessing which project to query.
The formatting section ensures consistent, scannable output every time the agent responds.

Select Save at the top of the page.
Confirm the Instructions section on the Overview page now displays your text.

Instructions not saving?

Agent instructions have a character limit. The instructions above are well within this limit. If you see an error, check for any special characters that may have been introduced during copy-paste.

Help me troubleshoot why my agent instructions are not saving in Copilot Studio

✔️ Awesome, I've got everything!

Great. Double check that your agent's Overview page shows the name Polarion Defect Tracker, the description, and your instructions text.

ⓧ I'd like to double check the full configuration

Here is what your agent configuration should look like at this point:

Name: Polarion Defect Tracker.
Description: Queries open defects from Polarion ALM filtered by project, sprint, PI, and status.
Generative AI Orchestration: Yes.

Instructions field content:

You are a defect tracking assistant for test engineers.

Your role:
- Help users query open defects from the Polarion ALM system.
- Always ask for the project ID if the user has not provided one.
- Use the Polarion MCP tools to query work items. Never make up defect data.

Formatting:
- Present results as a numbered list.
- Each item should show: Defect ID, Title, Status, and Assignee.
- If no results are found, let the user know and suggest checking the project ID or filters.

Your agent shell is ready. Next up, you will create the custom connector that gives your agent secure access to your company's Polarion MCP server.

Create the Custom MCP Connector

Your agent is set up with instructions and generative orchestration ready to go. Now it needs a way to actually reach your company's Polarion system.

That connection happens through a custom connector. A custom connector is a bridge that tells Copilot Studio where your MCP server lives, what operations it supports, and how to authenticate. You'll build one by importing your company's OpenAPI descriptor into Power Apps.

In this step, get ready to:

Import your company's MCP OpenAPI descriptor as a custom connector in Power Apps.
Configure API Key authentication with your Polarion Personal Access Token.
Test the connector to confirm successful connectivity to the MCP server.

Navigate to Power Apps and import the OpenAPI file

Your company provides an OpenAPI descriptor at its /mcp/openapi.yaml endpoint. This file describes every operation the MCP server supports, so Power Apps can auto-generate the connector's actions for you.

Download the OpenAPI descriptor from your company's MCP server at the /mcp/openapi.yaml endpoint. Save the file to a location you can find easily (like your Desktop).
Switch back to your Copilot Studio agent.
Select the Tools page in the left navigation.
Click Add a tool.
Select New tool.
Select Custom connector. This redirects you to the Power Apps portal.
In Power Apps, select New custom connector, then select Import an OpenAPI file.
Enter Polarion MCP Server as the connector name.
Browse to the OpenAPI file you downloaded and select it.
Click Continue.

What is an OpenAPI descriptor?

An OpenAPI descriptor is a structured file that documents every endpoint, parameter, and response an API supports. When you import it into Power Apps, the wizard auto-populates the connector's host, base URL, and available operations so you don't have to configure them manually.

OpenAPI version requirements

Custom connectors require the OpenAPI definition to be in OpenAPI 2.0 (Swagger) format and under 1 MB. If your company's file is in OpenAPI 3.0 format, the creation process automatically translates it to a 2.0 specification.

Configure API Key authentication on the Security tab

The connector wizard opens to the General page after import. You need to verify the server details look correct, then set up authentication so requests include your Polarion Personal Access Token.

On the General page, review the imported Host and Base URL fields. Confirm they match your company's MCP server endpoint.
Select Security at the bottom to navigate to the Security tab.
In the Authentication type dropdown, select API Key.
In the Parameter label field, enter Authorization.
In the Parameter name field, enter Authorization.
Set the Parameter location to Header.

Why use API Key authentication for Bearer tokens?

Power Platform's API Key auth type lets you send a value in a specific HTTP header with every request. By setting the parameter name to Authorization and the location to Header, every request your connector makes will include an Authorization header.

When you create the connection, you'll prefix your PAT with Bearer so the full header value becomes Bearer your-token. This matches the authentication format Polarion expects.

Don't see the Security tab?

Make sure you are in the custom connector wizard (the multi-step editor with General, Security, Definition, and Test tabs across the top). If you only see a single-page summary, click Edit on the connector to re-enter the wizard.

Help me find the Security tab in the custom connector wizard.

Create the connector and test connectivity

With authentication configured, you can now create the connector and verify it can reach your MCP server.

Select Create connector at the top of the wizard.
Wait a moment for the connector to be created.
Navigate to the Test tab.
Click New connection.
In the API key field, enter Bearer followed by your Polarion Personal Access Token (for example, Bearer your-pat-here).
Click Create connection.
Select one of the listed operations from the dropdown.
Click Test operation.
Confirm the response returns a successful status (a 200-level response code).

Wait before testing

It can take a few minutes after creating the connector before it is ready to connect to the API. If your first test fails, wait a minute and try again.

Test returning an error?

Check that you included Bearer (with a space after it) before your PAT when creating the connection. The full value should look like Bearer abc123....

Also verify that the Host on the General page matches your company's MCP server URL exactly. A trailing slash or missing path segment can cause connection failures.

Help me troubleshoot my custom connector test failure.

Your custom connector is live and talking to Polarion. Next up, you'll add this connector to your agent as an MCP tool so it can call Polarion operations during conversations.

Add MCP Tools to Your Agent

Your custom connector is live and tested in Power Apps. The next step is to wire it into your agent so the AI can actually call those MCP tools at runtime.

Right now your agent has instructions but no tools. It's like giving someone a job description without access to any systems. In this step, you'll connect the Polarion MCP Server to your agent and make sure the AI knows when and how to use each tool.

In this step, get ready to:

Add the Polarion MCP Server connector as an MCP tool in your agent.
Review the available tools exposed by the MCP server.
Update tool descriptions so the AI routes queries accurately.

Connect the MCP connector to your agent

Your connector lives in Power Apps, but your agent needs to reference it through Copilot Studio's MCP tool category. This bridges the two platforms.

Switch back to Copilot Studio in your browser.
Navigate to your Polarion Defect Tracker agent's Tools page.
Select Add a tool.
Select Model Context Protocol from the tool categories.
Find your Polarion MCP Server connector in the list and select it.
When prompted to authorize the connection, confirm it.

Why Model Context Protocol instead of just a connector?

Adding your connector through the MCP category tells Copilot Studio to treat it as an AI-callable tool set. The agent's generative orchestration layer can then decide which tool to invoke based on the user's natural-language question.

A plain connector without the MCP registration would require manual topic-based routing instead of AI-driven selection.

Don't see your connector in the list?

Make sure you're signed into Copilot Studio with the same work account you used to create the connector in Power Apps. Connectors created in Power Apps are automatically available in Copilot Studio under the same tenant.

If it still doesn't appear, return to the Power Apps portal and confirm the connector was saved successfully (look for a green checkmark or success message on the connector page).

My Polarion MCP Server connector isn't showing up in Copilot Studio's MCP tool list. How do I troubleshoot this?

Review the available MCP tools

Once you add the connector, Copilot Studio pulls in all the tools that the MCP server exposes. You need to verify the right tools are present before the agent can use them.

After adding the connector, the MCP server settings page opens showing all available tools with their names and descriptions.
Locate the tools that map to polarion.listProjects and polarion.getDocumentWorkItems (or your company's equivalent work item query tools).
Confirm the Allow all toggle is enabled. This keeps all tools turned on for testing.

What does the Allow all toggle do?

When Allow all is on, every tool from the MCP server is available to your agent. Turning it off lets you enable tools individually. Any new tools added to the server later will default to off when Allow all is disabled.

For initial testing, keep it enabled so your agent has access to everything. You can restrict tools later once you know which ones your defect queries actually need.

Identify the right tools for defect queries

Your primary tools for this project are polarion.listProjects (to resolve project scope) and polarion.getDocumentWorkItems (to query defects with filters). Your company's MCP server may use slightly different names. Look for tools related to listing projects and retrieving work items.

Update tool descriptions for accurate routing

The agent uses each tool's name and description to decide which tool to call. Vague descriptions lead to wrong tool selections or the agent ignoring a tool entirely.

For each tool listed on the settings page, review the auto-populated name and description.
If any descriptions are vague or generic, select the tool to edit it.
Update the work items query tool description to:

Searches Polarion for work items (defects, requirements, tasks) filtered by project, type, status, sprint, and PI. Use this tool when the user asks about defects, bugs, or work items.

Review the other tools and update their descriptions to be similarly specific about what they do and when to use them.
Save your changes.

Why do tool descriptions matter so much?

The agent's generative orchestration reads tool descriptions to decide which tool fits the user's request. A description like "Gets data" gives the AI almost nothing to work with.

A specific description like the one above tells the AI exactly what data the tool returns, what filters it supports, and when to call it. This is the single biggest factor in whether your agent picks the right tool.

Can't edit tool descriptions?

If the description fields appear read-only, make sure the Allow all toggle is turned off first. Disabling Allow all unlocks per-tool configuration including description edits. Re-enable Allow all after saving your description updates.

I can't edit the tool descriptions on my MCP server settings page in Copilot Studio. How do I enable editing?

Your agent now has the tools it needs to reach Polarion. Next up, you'll write detailed instructions that tell the agent exactly how to use these tools for defect queries.

Configure Agent Instructions for Defect Queries

Your MCP tools are connected and ready. Now the agent needs clear instructions so it knows when to call those tools and how to format the results.

Agent instructions are what guide the orchestrator's decision-making. Without detailed instructions, the agent might call the wrong tool, forget to ask for a project ID, or return raw data instead of a clear summary. In this step, you will write precise instructions, add guardrails, and enable a setting that lets your agent ask follow-up questions naturally.

In this step, get ready to:

Write detailed agent instructions for defect queries.
Add guardrails that prevent the agent from modifying data or going off-topic.
Enable ungrounded responses so the agent can ask clarifying questions.

Update agent instructions with defect query behavior

The orchestrator uses your instructions to decide which tool to call, what parameters to pass, and how to present results. The more specific your instructions are, the more reliably your agent handles queries.

Return to the Overview page of your Polarion Defect Tracker agent.
In the Instructions section, select Edit.
Replace the existing instructions with the following text:

# Role
You are a defect tracking assistant for test engineers. You help users query open defects from Polarion ALM.

# Tool Usage
1. When a user asks about defects, bugs, or work items, always use the Polarion work items query tool.
2. Default the work item type filter to "defect" unless the user specifies a different type (e.g., requirement, task).
3. If the user does not provide a project ID, ask them for it before querying.
4. When the user filters by sprint or PI, pass those values as query parameters to the tool.

# Response Format
Format results as a numbered list showing:
- Defect ID
- Title
- Status
- Severity
- Assignee

If more than 10 results are returned, summarize the total count and show the first 10.

What do these instructions do?

Each numbered rule tells the orchestrator exactly how to behave. The # Role section sets the agent's identity. The # Tool Usage section names when and how to call the MCP tool. The # Response Format section ensures results come back in a consistent, readable shape.

Copilot Studio supports Markdown in the instruction field. Using headings and lists helps the AI parse your intent more reliably.

Reference tools with the slash shortcut

While editing instructions, you can type / at any point to reference a specific tool by name. This creates a direct link between your instruction and the connected tool, which helps the orchestrator make more accurate decisions.

Confirm your instructions look correct, then move on to adding guardrails in the same text field.

Add guardrails to prevent unintended actions

Guardrails tell the agent what it should not do. Without them, the agent might attempt to create or modify work items, or respond to unrelated questions using its general knowledge.

In the same Instructions text field, add the following section below your existing instructions:

# Guardrails
1. Only query Polarion data. Never attempt to create, update, or delete work items.
2. If a query returns no results, suggest the user verify the project ID, sprint name, or status value they provided.
3. If the user asks about something unrelated to Polarion defects or work items, politely let them know you can only help with Polarion defect queries and suggest they rephrase their question.

Why add guardrails?

Guardrails prevent the agent from taking actions your MCP server tools could theoretically support but that you do not want triggered by natural-language queries. They also improve the user experience by providing helpful feedback when a query returns empty results instead of a generic failure message.

Select Save to apply your updated instructions.

Your full instructions should now contain a Role section, Tool Usage section, Response Format section, and Guardrails section.

Instructions not saving?

The instructions field has a maximum of 8,000 characters. If your text is too long, shorten descriptions while keeping the numbered rules intact. Also confirm you are on the Overview page and not inside a topic editor.

Help me troubleshoot why my agent instructions won't save in Copilot Studio.

Enable ungrounded responses for conversational flow

Your instructions tell the agent to ask for a project ID if the user does not provide one. For this to work, the agent needs permission to respond without calling a tool first. By default, Copilot Studio blocks any response that does not use a knowledge source or tool in that turn.

Go to the Settings page for your agent.
Select the Generative AI section.
Scroll down to the Knowledge area.
Turn on the Allow ungrounded responses toggle.
Select Save at the top of the page.

What does Allow ungrounded responses do?

When this setting is on, the agent can generate a response using the model's general knowledge, even when it does not call any tool or knowledge source in that turn. This means the agent can ask follow-up questions like "Which project ID should I search?" without being blocked.

When this setting is off, the orchestrator considers clarifying questions ungrounded and suppresses them. The agent would default to a generic fallback message instead of gathering the information it needs.

Return to the Overview page to confirm your instructions are still in place.

Agent still not asking follow-up questions?

Double-check that generative orchestration is still enabled. Go to Settings > Generative AI > Orchestration and confirm the toggle is set to Yes. Both generative orchestration and Allow ungrounded responses must be enabled for the agent to ask clarifying questions.

Help me figure out why my Copilot Studio agent is not asking follow-up questions when it should.

Your agent now has precise instructions, clear guardrails, and the ability to ask clarifying questions. Next up, you will test it with real Polarion queries to see how well it performs.

Test Your Defect Tracker Agent

Your agent has instructions, guardrails, and MCP tools all wired up. The final step is validating that everything works together.

You need to confirm the agent actually calls the right MCP tool, passes the correct parameters, and returns formatted results. Without testing, you won't know if a vague tool description or missing instruction is silently breaking the query flow.

In this step, get ready to:

Run a simple defect query and verify MCP tool invocation.
Test complex queries with sprint, PI, and assignee filters.
Refine agent instructions if the agent misroutes queries.

Run your first test query

The Test panel lets you interact with your agent in real time and inspect which tools it invokes behind the scenes.

In Copilot Studio, click the Test button at the top of the page to open the Test panel.
Select the three-dot menu in the Test panel.
Turn on Show activity map when testing.

What is the activity map?

The activity map shows a visual diagram of your agent's execution plan in real time. It reveals which MCP server and specific tool your agent invoked, along with the input arguments and outputs returned.

Type the following query into the Test panel chat box:

List all open defects in project [[PROJECT_ID="your Polarion project ID"]]

Press Enter to send the message.

Your agent should respond with a numbered list of defects showing Defect ID, Title, Status, Severity, and Assignee.

Click on the nodes in the activity map to confirm the agent invoked the Polarion MCP Server and used the work items query tool.
Review the input arguments to verify the agent passed defect as the work item type and your project ID as the project filter.

Agent not calling the MCP tool?

If the agent responds with generic text instead of querying Polarion, check that generative orchestration is still set to Yes in your agent's Settings page under Generative AI. MCP tools require generative orchestration to be active.

Also verify your MCP tool connection is still authorized on the Tools page. A broken connection prevents the agent from invoking any tools.

Help me figure out why my agent isn't calling the Polarion MCP tool when I ask about defects.

Test complex filter queries

A single basic query passing is a good start, but your daily workflow involves filtering by sprint, PI, and assignee. You need to confirm the agent translates these natural-language filters into the correct query parameters.

Type this query to test sprint filtering:

Show me defects under analysis in Sprint 3.2

Verify the agent passes the sprint value as a query parameter in the activity map.
Type this query to test PI filtering:

What implemented defects are in PI 25.2?

Verify the agent includes the PI value and implemented status in the tool parameters.
Type this query to test assignee filtering:

Are there any open defects assigned to [[YOURNAME="your name"]]?

Verify the agent includes the assignee name in the query parameters.

For each query, check both the response content and the activity map. The agent should correctly translate your natural language into the right combination of type, status, sprint, PI, and assignee filters.

What if a query returns no results?

Empty results don't necessarily mean your agent is broken. Your guardrail instructions tell the agent to suggest verifying the project ID, sprint name, or status value when no results come back.

Confirm the agent gives this helpful suggestion rather than silently returning nothing. That behavior proves your guardrails are working.

Refine instructions if the agent misroutes queries

If any of your test queries failed to call the right tool or passed incorrect parameters, the fix is almost always in your agent instructions or tool descriptions. The agent uses tool names and descriptions to decide which tool to invoke.

Return to your agent's Overview page to edit the instructions.
If the agent picked the wrong tool, add an explicit tool reference to your instructions. For example, add a line like: "When the user mentions defects, use the Polarion Work Items Query tool."
If the agent missed a filter parameter, add example queries to the instructions that demonstrate the expected parameter mapping.
If tool descriptions are vague, switch to the Tools page and update the tool description to be more specific about what parameters it accepts.

Tool descriptions drive tool selection

The agent uses each tool's name and description to decide what to call. A description like "Queries work items" is too vague. A description like "Searches Polarion for work items filtered by project, type, status, sprint, and PI" gives the agent the signal it needs to match the right query to the right tool.

After making any changes, return to the Test panel.
Click the Start new test session icon at the top of the Test panel to reset the conversation.
Re-run the query that previously failed to confirm your fix works.

Agent still passing wrong parameters?

Double-check that your instructions explicitly mention defaulting to type defect unless the user specifies otherwise. Without this default, the agent may query all work item types.

If the agent ignores sprint or PI filters, add a specific instruction line such as: "When the user mentions a sprint or PI, always pass it as a query parameter to the work items tool."

Help me fix my agent's parameter passing for sprint and PI filters.

Secret mission

Add Requirement Traceability

Your agent can find defects, but can it trace them back to the requirements they came from? Extend your Defect Tracker with multi-step reasoning so it queries defects first, then finds linked requirement documents automatically.

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely within your existing Copilot Studio and Power Apps licenses, so there are no ongoing costs.

Resources you used:

Copilot Studio agent: Polarion Defect Tracker.
Power Apps custom connector: Polarion MCP Server.
Test connection created during connector setup.

Keep everything running

No action needed. Choose this if you plan to continue using the Polarion Defect Tracker agent in your daily workflow.

Your agent remains available in Copilot Studio for testing and future publishing.
Your custom connector stays active in Power Apps and can be reused by other agents or flows.
No charges are incurred beyond your existing corporate licenses.

Pause - I'll come back to this later

Disable the agent so it stops responding to users, but keep the configuration intact for when you return.

In Copilot Studio, go to your agent's settings and turn off any published channels. The agent configuration and instructions remain saved.
Leave the custom connector in Power Apps. It consumes no resources when not actively called.
When you come back, simply re-enable the channels and test the agent again.

Delete - I don't want to use this again

Remove all project resources permanently. This cannot be undone.

Delete the Copilot Studio agent: go to the Agents page in Copilot Studio, select your Polarion Defect Tracker agent, click the three dots (...) next to Settings on the top menu bar, and select Delete. Enter the agent name to confirm, then select Delete agent.
Delete the custom connector: in Power Apps, select Discover from the home page menu. On the Discover page, find the Data tile and select Custom connectors. Find your Polarion MCP Server connector, select the ellipses (...) button, and select Delete. Confirm by selecting OK.

What about the test connection?

Deleting the custom connector automatically removes all connections associated with it. Your test connection is cleaned up when you delete the connector.

Nice Work!

You just built a production-ready AI agent that saves you from repetitive Polarion navigation every single day. No more clicking through the ALM interface just to check on open defects.

You've learned how to:

Built a custom connector in Power Apps using your company's MCP OpenAPI descriptor with Bearer token authentication.
Created a Copilot Studio agent with generative orchestration that translates natural-language defect queries into live Polarion queries via MCP tools.
Configured precise agent instructions and guardrails that guide the AI to use the correct MCP tools with the right parameters for project, sprint, PI, and status filters.
Added requirement traceability so you can instantly see which requirements are linked to your defects.

Ready to quiz yourself?