GitHub to Azure DevOps Pipeline

Create a CI/CD pipeline in Azure with staging environments and automatic rollback.

Introduction

This project has been archived

We're no longer updating this project - we recommend checking out our active projects instead!

If you've already completed this project, don't worry! Your documentation is still available for you to check out and share.

If you have any questions, leave a question in our Discord and we'll get back to you ASAP!

⚡️ 30 second Summary

How often do you deploy changes to prod? Amazon deploys code 136,000 times per day - that's 1.6 deployments every second. Netflix runs over 4,000 automated pipeline steps for a single change.

They achieve this with CI/CD pipelines - automated workflows that test code, deploy to staging, and then deploy to production. If something breaks, they roll back in seconds. New team members can ship code on day one because the pipeline handles the risky parts.

In this final part of the series, you'll build a CI/CD pipeline with Azure DevOps - connecting your GitHub repository to a multi-stage pipeline that deploys your app safely.

What you'll build:

In this project you will create an Azure DevOps pipeline that pulls from your GitHub repo and deploys changes to a staging or production environment.

By the end of this project, you'll have:

  • 🐠 An Azure DevOps project connected to your GitHub repository.
  • 🏰 Staging and production Function Apps with separate environment configs.
  • 💈 A multi-stage pipeline that runs tests, deploys to staging, and deploys to production.
  • 💎 Secret Mission: Automatic rollback if production health checks fail.

Want a complete demo of how to do this project, from start to finish? Check out our 🎬 walkthrough with Kahu

Prerequisite

This project builds on the Azure x AI series. You'll need the preivous projects or working Azure Functions project in GitHub to deploy through the pipeline.

Not sure if this project is right for you? Check if it matches your goals

If you're up for a bit of a challenge, quiz yourself on the key concepts up ahead in this project.

This project is part of a series:

  1. Part 1: Build Your First Azure Function
  2. Part 2: Build AI Chat Moderator with Gemini
  3. Part 3: Orchestrating Parallel Event Streams with Azure
  4. Part 4: You are here!

Connect Azure DevOps to Your Code

Alright! Let's get this show on the road.

First thing we need to do is set up Azure DevOps and connect it to your streaming backend app (the thing you've been working on in previous projects of this series).

This is where the magic starts - you'll create an Azure DevOps project, link your GitHub repository, and set up separate staging and production environments.

In this step, get ready to:

  • Create an Azure DevOps project
  • Push your code to GitHub
  • Create staging and production Function Apps
  • Configure environment variables for each stage

Check Your Prerequisites

This project builds on the previous projects in the Azure Functions series. Let's make sure you have what you need.

✅ I completed Parts 1, 2 & 3

Great! You should have:

  • A working Azure Functions project in a GitHub repository
  • An Azure subscription with Function Apps deployed
  • Cosmos DB with messages, violations, analytics, and stats containers
  • AI moderation with Gemini integrated
  • Durable Functions with fan-out/fan-in orchestration

Continue to set up Azure DevOps below.

🔄 I completed Parts 1 & 2

You need to complete Part 3 (Event-Driven Workflows) before continuing.

Create the Alerts Queue

  • Open the Azure Portal and navigate to your Storage Account
  • Go to Storage browser → Queues → + Add queue
  • Name it: alerts
  • Click Ok

Verify Your Function App Connection

  • Go to your Function App → Settings → Environment variables
  • Look for AzureWebJobsStorage in the list
  • If missing, add it:
    • In Storage Account → Security + networking → Access keys → Copy connection string
    • In Function App → Environment variables → + Add → Name: AzureWebJobsStorage, Value: connection string
    • Click Apply twice

Create the Durable Functions App

Open Cursor chat and send this prompt:

Create an Azure Durable Functions app in function_app.py for processing stream events (followers, subscriptions, donations).

Requirements:
1. Create a queue trigger function that listens to the "alerts" queue (connection: AzureWebJobsStorage)
2. When a message arrives, start a Durable Functions orchestration called "event_orchestrator"
3. The orchestrator should use the fan-out pattern to run three activities in parallel:
   - update_stats: receives event data, returns status "updated"
   - send_alert: receives event data, builds alert message like "New {type} from {username}!", returns status "sent"
   - log_analytics: receives event data, returns status "logged"
4. The orchestrator should wait for all activities to complete (fan-in), then return a summary with status "completed"
5. Add an HTTP POST endpoint at /api/event for testing that starts the orchestration directly
6. Use azure.functions and azure.durable_functions libraries

Update requirements.txt

Ensure it includes:

azure-functions
azure-functions-durable
azure-cosmos
requests

Create Cosmos DB Containers

In Azure Portal → your Cosmos DB account → Data Explorer:

  1. Create analytics container:
    • Click New Container
    • Database id: Create new → StreamData
    • Container id: analytics
    • Partition key: /stream_id
  2. Create stats container:
    • Click New Container
    • Database id: Use existing → StreamData
    • Container id: stats
    • Partition key: /stream_id

Add Cosmos DB Connection String

  • In Cosmos DB → Settings → Keys → Copy PRIMARY CONNECTION STRING
  • In Function App → Environment variables → + Add
  • Name: CosmosDBConnection, Value: paste connection string
  • Click Apply twice

Update Activity Functions

In Cursor chat:

Update my Azure Functions app to use Cosmos DB for the activity functions.

Requirements:
1. Get the Cosmos DB connection string from environment variable CosmosDBConnection
2. Connect to database "StreamData" with containers "analytics" and "stats"
3. Update the log_analytics activity to:
   - Create a document with id (UUID), stream_id, type, username, amount, and timestamp
   - Save it to the analytics container
   - Return status "logged" with the document_id
4. Update the update_stats activity to:
   - Try to read the existing stats document for the stream_id
   - If it doesn't exist, create one with follower_count, sub_count, and total_donations all set to 0
   - Increment the appropriate counter based on event type (follower, subscription, or donation)
   - Use upsert_item to save the updated stats
   - Return status "updated"
5. Use the azure.cosmos SDK (CosmosClient)

Add Stats Endpoint

Add a GET endpoint at /api/stats/{stream_id} to retrieve stream statistics.

Requirements:
1. Extract stream_id from the route parameters
2. Try to read the stats document from the stats container
3. If found, return JSON with stream_id, follower_count, sub_count, and total_donations
4. If not found, return JSON with all counts set to 0
5. Return appropriate HTTP responses with application/json content type

Deploy and Test

func azure functionapp publish streaming-backend

Deploy and test before continuing.

🔄 I only completed Part 1

You need to complete Parts 2 & 3 before continuing.

Part 2: Add AI Moderation

Get Your Google AI API Key

  • Go to https://aistudio.google.com/ and sign in with your Google account.
  • Click Get API key in the sidebar.
  • Click Create API key → Create project → Name it → Create key.
  • Copy your API key.

Add the API Key to Azure Function App

  • Go to the Azure Portal → your Function App.
  • Select Configuration under Settings.
  • Click New application setting:
    • Name: GEMINI_API_KEY
    • Value: Your Gemini API key
  • Click OK → Save → Continue.

Create the Violations Container

  • Go to Azure Portal → Cosmos DB → your account → Data Explorer.
  • Click New Container:
    • Database id: Use existing (e.g., streamingdb)
    • Container id: violations
    • Partition key: /streamId
  • Click OK.

Create the Moderation Module

Open your streaming backend project in Cursor.

  • Create moderation.py and send this prompt:
Create a Python module called moderation.py that checks messages for toxic content using the Gemini API.

Requirements:
1. Get the API key from environment variable GEMINI_API_KEY
2. Use the Gemini API endpoint: https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent
3. Create a check_message(message_content: str) function that:
   - Sends the message to Gemini with a prompt asking it to classify as clean/toxic/spam/harassment
   - Returns a dict with: is_allowed (bool), category (str), confidence (float 0-1), reason (str)
   - Uses "fail-open" pattern: if the API fails, return is_allowed=True
   - Handles markdown code blocks in Gemini's response (strip them before parsing JSON)
4. Use the requests library for HTTP calls

Update the POST /message Endpoint

  • Open function_app.py and send this prompt:
Update my Azure Function POST /message endpoint to add content moderation.

Requirements:
1. Import check_message from the moderation module I just created
2. Before saving a message, call check_message(message_content)
3. If is_allowed is False:
   - Save the message to a "violations" container in Cosmos DB (include category, confidence, reason, timestamp)
   - Return HTTP 403 with the category and reason in the response
4. If is_allowed is True:
   - Save to the messages container as before
   - Return HTTP 200 success
5. Keep my existing Cosmos DB connection code

Update requirements.txt

Make sure requests is in your requirements.txt:

azure-functions
azure-cosmos
requests

Deploy and test before continuing to Part 3.

Part 3: Event-Driven Workflows

Create the Alerts Queue

  • Open the Azure Portal and navigate to your Storage Account
  • Go to Storage browser → Queues → + Add queue
  • Name it: alerts
  • Click Ok

Verify Your Function App Connection

  • Go to your Function App → Settings → Environment variables
  • Look for AzureWebJobsStorage in the list
  • If missing, add it from Storage Account → Security + networking → Access keys

Create the Durable Functions App

Open Cursor chat and send this prompt:

Create an Azure Durable Functions app in function_app.py for processing stream events (followers, subscriptions, donations).

Requirements:
1. Create a queue trigger function that listens to the "alerts" queue (connection: AzureWebJobsStorage)
2. When a message arrives, start a Durable Functions orchestration called "event_orchestrator"
3. The orchestrator should use the fan-out pattern to run three activities in parallel:
   - update_stats: receives event data, returns status "updated"
   - send_alert: receives event data, builds alert message like "New {type} from {username}!", returns status "sent"
   - log_analytics: receives event data, returns status "logged"
4. The orchestrator should wait for all activities to complete (fan-in), then return a summary with status "completed"
5. Add an HTTP POST endpoint at /api/event for testing that starts the orchestration directly
6. Use azure.functions and azure.durable_functions libraries

Update requirements.txt

Ensure it includes:

azure-functions
azure-functions-durable
azure-cosmos
requests

Create Cosmos DB Containers

In Azure Portal → your Cosmos DB account → Data Explorer:

  1. Create analytics container:
    • Click New Container
    • Database id: Create new → StreamData
    • Container id: analytics
    • Partition key: /stream_id
  2. Create stats container:
    • Click New Container
    • Database id: Use existing → StreamData
    • Container id: stats
    • Partition key: /stream_id

Add Cosmos DB Connection String

  • In Cosmos DB → Settings → Keys → Copy PRIMARY CONNECTION STRING
  • In Function App → Environment variables → + Add
  • Name: CosmosDBConnection, Value: paste connection string
  • Click Apply twice

Update Activity Functions

In Cursor chat:

Update my Azure Functions app to use Cosmos DB for the activity functions.

Requirements:
1. Get the Cosmos DB connection string from environment variable CosmosDBConnection
2. Connect to database "StreamData" with containers "analytics" and "stats"
3. Update the log_analytics activity to:
   - Create a document with id (UUID), stream_id, type, username, amount, and timestamp
   - Save it to the analytics container
   - Return status "logged" with the document_id
4. Update the update_stats activity to:
   - Try to read the existing stats document for the stream_id
   - If it doesn't exist, create one with follower_count, sub_count, and total_donations all set to 0
   - Increment the appropriate counter based on event type (follower, subscription, or donation)
   - Use upsert_item to save the updated stats
   - Return status "updated"
5. Use the azure.cosmos SDK (CosmosClient)

Add Stats Endpoint

Add a GET endpoint at /api/stats/{stream_id} to retrieve stream statistics.

Requirements:
1. Extract stream_id from the route parameters
2. Try to read the stats document from the stats container
3. If found, return JSON with stream_id, follower_count, sub_count, and total_donations
4. If not found, return JSON with all counts set to 0
5. Return appropriate HTTP responses with application/json content type

Deploy and Test

func azure functionapp publish streaming-backend

Deploy and test before continuing.

🚀 I need to start from scratch

You need to complete Parts 1, 2 & 3 before continuing.

Part 1: Set Up Your Azure Environment

Create an Azure Account

  • Go to azure.microsoft.com/free and sign up (or log in if you have one).
  • Azure gives you $200 in free credits for 30 days.

Create a Resource Group

  • Go to the Azure Portal.
  • Click Create a resource → Search Resource group → Create.
  • Fill in:
    • Subscription: Select yours.
    • Resource group: streaming-backend-rg.
    • Region: Select a region close to you.
  • Click Review + create → Create.

Install Azure Functions Core Tools

Check if installed:

func --version

If not installed:

macOS:

brew tap azure/functions
brew install azure-functions-core-tools@4

Windows:

npm install -g azure-functions-core-tools@4 --unsafe-perm true

Install the Azure CLI

Check if installed:

az --version

If not installed:

macOS: brew install azure-cli

Windows: Download from aka.ms/installazurecliwindows

Log in to Azure

az login

Create the Project

  • Create a folder called streaming-backend and open it in your editor.
  • Open the terminal and initialize the project:
func init --python

Create the Message Endpoint

  • Create a new HTTP trigger function:
func new --name message --template "HTTP trigger" --authlevel "anonymous"
  • Open function_app.py and use your AI assistant to generate code with this prompt:
Create an Azure Function app with three HTTP endpoints:

1. POST /message - receives JSON with "content" and optional "username", stores in an in-memory list, returns the created message with a UUID and timestamp
2. GET /messages - returns all stored messages as JSON
3. GET /health - returns "Healthy" status

Use the azure.functions SDK with @app.route decorators. Include proper error handling.

Create a Storage Account

  • In Azure Portal: Create a resource → Storage account → Create.
  • Resource group: streaming-backend-rg.
  • Storage account name: streamingbackendstorage (globally unique, lowercase).
  • Performance: Standard.
  • Redundancy: LRS.
  • Click Review + create → Create.

Create the Function App

  • Search Function App → Create.
  • Hosting option: Consumption.
  • Resource group: streaming-backend-rg.
  • Function App name: streaming-backend (globally unique).
  • Runtime: Python 3.11.
  • Region: Same as your resource group.
  • Storage: Select the storage account you created.
  • Review + create → Create.

Deploy Your Function

func azure functionapp publish streaming-backend

Add Cosmos DB (Secret Mission from Part 1)

  • In Azure Portal: Create a resource → Azure Cosmos DB → Create → Azure Cosmos DB for NoSQL.
  • Resource group: streaming-backend-rg.
  • Account name: streaming-cosmos (globally unique).
  • Capacity mode: Serverless.
  • Create database streamingdb with container messages (partition key: /id).
  • Copy the PRIMARY CONNECTION STRING from Settings → Keys.
  • Add it to your Function App's Configuration as COSMOS_CONNECTION_STRING.

Part 2: Add AI Moderation

Get Your Google AI API Key

  • Go to https://aistudio.google.com/ and sign in with your Google account.
  • Click Get API key in the sidebar.
  • Click Create API key → Create project → Name it → Create key.
  • Copy your API key.

Add the API Key to Azure Function App

  • Go to the Azure Portal → your Function App.
  • Select Configuration under Settings.
  • Click New application setting:
    • Name: GEMINI_API_KEY
    • Value: Your Gemini API key
  • Click OK → Save → Continue.

Create the Violations Container

  • Go to Azure Portal → Cosmos DB → your account → Data Explorer.
  • Click New Container:
    • Database id: Use existing (e.g., streamingdb)
    • Container id: violations
    • Partition key: /streamId
  • Click OK.

Create the Moderation Module

  • Create moderation.py and send this prompt:
Create a Python module called moderation.py that checks messages for toxic content using the Gemini API.

Requirements:
1. Get the API key from environment variable GEMINI_API_KEY
2. Use the Gemini API endpoint: https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent
3. Create a check_message(message_content: str) function that:
   - Sends the message to Gemini with a prompt asking it to classify as clean/toxic/spam/harassment
   - Returns a dict with: is_allowed (bool), category (str), confidence (float 0-1), reason (str)
   - Uses "fail-open" pattern: if the API fails, return is_allowed=True
   - Handles markdown code blocks in Gemini's response (strip them before parsing JSON)
4. Use the requests library for HTTP calls

Update the POST /message Endpoint

  • Open function_app.py and send this prompt:
Update my Azure Function POST /message endpoint to add content moderation.

Requirements:
1. Import check_message from the moderation module I just created
2. Before saving a message, call check_message(message_content)
3. If is_allowed is False:
   - Save the message to a "violations" container in Cosmos DB (include category, confidence, reason, timestamp)
   - Return HTTP 403 with the category and reason in the response
4. If is_allowed is True:
   - Save to the messages container as before
   - Return HTTP 200 success
5. Keep my existing Cosmos DB connection code

Update requirements.txt

azure-functions
azure-cosmos
requests

Part 3: Event-Driven Workflows

Create the Alerts Queue

  • Open the Azure Portal and navigate to your Storage Account
  • Go to Storage browser → Queues → + Add queue
  • Name it: alerts
  • Click Ok

Verify Your Function App Connection

  • Go to your Function App → Settings → Environment variables
  • Look for AzureWebJobsStorage in the list
  • If missing, add it from Storage Account → Security + networking → Access keys

Create the Durable Functions App

Open Cursor chat and send this prompt:

Create an Azure Durable Functions app in function_app.py for processing stream events (followers, subscriptions, donations).

Requirements:
1. Create a queue trigger function that listens to the "alerts" queue (connection: AzureWebJobsStorage)
2. When a message arrives, start a Durable Functions orchestration called "event_orchestrator"
3. The orchestrator should use the fan-out pattern to run three activities in parallel:
   - update_stats: receives event data, returns status "updated"
   - send_alert: receives event data, builds alert message like "New {type} from {username}!", returns status "sent"
   - log_analytics: receives event data, returns status "logged"
4. The orchestrator should wait for all activities to complete (fan-in), then return a summary with status "completed"
5. Add an HTTP POST endpoint at /api/event for testing that starts the orchestration directly
6. Use azure.functions and azure.durable_functions libraries

Update requirements.txt

azure-functions
azure-functions-durable
azure-cosmos
requests

Create Cosmos DB Containers

In Azure Portal → your Cosmos DB account → Data Explorer:

  1. Create analytics container:
    • Click New Container
    • Database id: Create new → StreamData
    • Container id: analytics
    • Partition key: /stream_id
  2. Create stats container:
    • Click New Container
    • Database id: Use existing → StreamData
    • Container id: stats
    • Partition key: /stream_id

Add Cosmos DB Connection String

  • In Cosmos DB → Settings → Keys → Copy PRIMARY CONNECTION STRING
  • In Function App → Environment variables → + Add
  • Name: CosmosDBConnection, Value: paste connection string
  • Click Apply twice

Update Activity Functions

In Cursor chat:

Update my Azure Functions app to use Cosmos DB for the activity functions.

Requirements:
1. Get the Cosmos DB connection string from environment variable CosmosDBConnection
2. Connect to database "StreamData" with containers "analytics" and "stats"
3. Update the log_analytics activity to:
   - Create a document with id (UUID), stream_id, type, username, amount, and timestamp
   - Save it to the analytics container
   - Return status "logged" with the document_id
4. Update the update_stats activity to:
   - Try to read the existing stats document for the stream_id
   - If it doesn't exist, create one with follower_count, sub_count, and total_donations all set to 0
   - Increment the appropriate counter based on event type (follower, subscription, or donation)
   - Use upsert_item to save the updated stats
   - Return status "updated"
5. Use the azure.cosmos SDK (CosmosClient)

Add Stats Endpoint

Add a GET endpoint at /api/stats/{stream_id} to retrieve stream statistics.

Requirements:
1. Extract stream_id from the route parameters
2. Try to read the stats document from the stats container
3. If found, return JSON with stream_id, follower_count, sub_count, and total_donations
4. If not found, return JSON with all counts set to 0
5. Return appropriate HTTP responses with application/json content type

Deploy and Test

func azure functionapp publish streaming-backend

Deploy and test before continuing.

Create an Azure DevOps Organization

First, let's set up Azure DevOps if you don't already have it.

  • Go to the Azure Portal and search for Azure DevOps organizations.
  • Click on Azure DevOps organizations to open it.
  • Click View my organizations.
  • Fill in your details (name, email, country) and click Continue.
  • Click Create new organization.
  • Click Continue to proceed.
  • Name your organization (e.g., yourname-streaming).
  • Select your region and Azure subscription.
  • Complete the captcha and click Continue.

Wait for your organization to be created.

Create a Project

Now create a project for your streaming backend:

  • Name it streaming-backend-cicd.
  • Set visibility to Private.
  • Click Create project.

You'll see your new project dashboard with options for Boards, Repos, Pipelines, etc.

What is Azure DevOps?

Azure DevOps is Microsoft's platform for managing software projects. It provides tools for source control, CI/CD pipelines, project tracking, and more. In this project, we're using Azure DevOps Pipelines to automate deploying your code whenever you push changes to GitHub.

Set Up a Self-Hosted Agent

Azure DevOps pipelines need an "agent" - a machine that runs your pipeline steps. Instead of using Microsoft's hosted agents (which require approval for new organizations), you'll set up a self-hosted agent on your own computer. This works immediately with no waiting.

  • In your Azure DevOps project, click Project Settings (bottom left).
  • Under Pipelines, click Agent pools.
  • Click Add pool.
  • Select Self-hosted as the pool type.
  • Name it Default (or any name you prefer).
  • Check Grant access permission to all pipelines.
  • Click Create.
  • Click on your new pool (Default).
  • Click New agent.
  • Select your operating system (Windows or macOS).
  • Click Download to download the agent.

🪟 Windows

  • Once the download completes, you'll see the zip file in your Downloads folder.
  • Right-click the zip file and select Extract All.
  • After extraction, you'll see the agent folder with all the files.
  • Right-click the extracted folder and select Open in Terminal.
  • Run the configuration command:
.\config.cmd

When prompted:

  • Server URL: Enter your Azure DevOps URL (e.g., https://dev.azure.com/yourorg)
  • Authentication type: Press Enter for PAT (Personal Access Token)
  • Personal access token: You'll need to create one - see below

Create a Personal Access Token (PAT):

  • In Azure DevOps, click your profile icon (top right).
  • Click Personal access tokens.
  • Click + New Token.
  • Name: agent-setup
  • Expiration: 30 days (or longer)
  • Scopes: Select Full access.
  • Click Create and copy the token immediately.
  • Paste the token in PowerShell when prompted.

Continue with the remaining prompts:

  • Agent pool: Press Enter for Default
  • Agent name: Press Enter to use your computer name
  • Work folder: Press Enter for default
  • Run agent as service?: Enter N
  • Configure autologon?: Enter N

After configuration completes, start the agent:

.\run.cmd

You should see "Listening for Jobs" - your agent is ready!

Keep this terminal open

The agent runs in this terminal window. Keep it open while you run your pipeline. You can close it after testing, but the pipeline won't run without it.

🍎 macOS

  • Open Terminal and run:
mkdir ~/agent && cd ~/agent
tar zxvf ~/Downloads/vsts-agent-osx-*.tar.gz
./config.sh

When prompted: - Server URL: Enter your Azure DevOps URL (e.g., https://dev.azure.com/yourorg) - Authentication type: Press Enter for PAT (Personal Access Token) - Personal access token: You'll need to create one - see below

Create a Personal Access Token (PAT):

  • In Azure DevOps, click your profile icon (top right) → Personal access tokens.
  • Click + New Token.
  • Name: agent-setup
  • Expiration: 30 days (or longer)
  • Scopes: Select Full access.
  • Click Create and copy the token immediately.

Continue with the remaining prompts:

  • Agent pool: Press Enter for Default
  • Agent name: Press Enter to use your computer name
  • Work folder: Press Enter for default
  • Run agent as service?: Enter N
  • Configure autologon?: Enter N

After configuration completes, start the agent:

./run.sh

You should see "Listening for Jobs" - your agent is ready!

Keep this terminal open

The agent runs in this terminal window. Keep it open while you run your pipeline. You can close it after testing, but the pipeline won't run without it.

Verify your agent is online:

  • Go back to Azure DevOps → Project Settings → Agent pools → Default.
  • Click the Agents tab.
  • You should see your agent listed as Online.

What is a self-hosted agent?

An agent is simply a machine that runs your pipeline steps. Microsoft offers "hosted agents" (their servers), but new organizations need approval to use them. A self-hosted agent runs on YOUR computer instead - it connects to Azure DevOps, picks up pipeline jobs, and runs them locally. This gives you full control and works immediately without any approval process.

Push Your Code to GitHub

Azure DevOps needs your code in a GitHub repository to build the CI/CD pipeline. Let's create a repo and push your streaming backend code.

I need to create a GitHub repository

  • Name your repository: streaming-backend
  • Set visibility to Private (recommended) or Public.
  • Do NOT initialize with README, .gitignore, or license (you already have code).
  • Click Create repository.
  • Copy your repository URL:

[[GITHUB_REPO_URL="https://github.com/YOUR-USERNAME/streaming-backend.git"]]

Now open your streaming backend project in Cursor:

  • Open the Cursor app.
  • Click File → Open Folder (or use the welcome screen).
  • Navigate to and select your streaming-backend folder.
  • Open the terminal and push your existing code:
git init
git add .
git commit -m "Initial commit: streaming backend with AI moderation and event workflows"
git branch -M main
git remote add origin [[GITHUB_REPO_URL="https://github.com/YOUR-USERNAME/streaming-backend.git"]]
  • Now run: git push -set-upstream origin main

You should see your code pushed successfully. You can verify by checking your GitHub repo, you should see all your files.

My code is already on GitHub

Make sure your streaming backend code (from Parts 1-3) is pushed to your GitHub repository.

Verify by going to your repo on GitHub and checking that you see: - function_app.py - requirements.txt - moderation.py

If you need to push recent changes:

git add .
git commit -m "Update streaming backend"
git push origin main

What is a service connection?

A service connection is a secure link between Azure DevOps and an external service (like GitHub or Azure). It stores credentials so your pipeline can authenticate without exposing sensitive tokens. Once connected, Azure DevOps can access your code and deploy to your cloud resources.

Create a Staging Function App

You already have a Function App (streaming-backend) from the previous projects - that will be your production environment. Now you need a second Function App for staging (testing).

Why staging and production environments?

Staging is your "dress rehearsal" environment. It mirrors production but isn't used by real users. You deploy to staging first, run tests, and verify everything works before touching production. This safety net catches bugs before they impact your users. If staging breaks, no one notices. If production breaks, everyone notices.

  • Search for Function App and click Create.
  • Select Flex Consumption as the hosting option and click Select.

For the staging environment:

  • Subscription: Select your subscription.
  • Resource Group: Select your existing streaming-backend-rg.
  • Function App name: streaming-backend-staging (must be globally unique).
  • Region: Same region as your existing Function App.
  • Runtime stack: Python.
  • Version: 3.11 or higher.
  • Click Review + create.
  • Review your settings and click Create.

✅ Deployment succeeded

Wait for deployment to complete (usually 1-2 minutes). You'll see a notification when it's ready.

❌ Quota or subscription error

If you see a quota or subscription error:

  • Check that your Azure subscription is active (not expired or disabled).
  • You may need to upgrade from a free trial if you've hit resource limits.
  • Try selecting a different region that might have more availability.
  • Contact Azure support if the issue persists.

Your existing streaming-backend Function App from Parts 1-3 will serve as your production environment. You now have two Function Apps:

  • streaming-backend-staging - for testing new deployments
  • streaming-backend - your production environment (already configured)

Configure Environment Variables

Your staging Function App needs the same environment variables as production. Let's get the Cosmos DB connection string and add it to your staging app.

Get the Cosmos DB Connection String

  • In your Cosmos DB account, go to Settings → Keys.
  • Copy the PRIMARY CONNECTION STRING (not the URI - the connection string starts with AccountEndpoint=).

Important

Copy the PRIMARY CONNECTION STRING, not the URI. The connection string looks like AccountEndpoint=https://...;AccountKey=... and is much longer than the URI.

Add Environment Variables to Staging

  • Go to your Function App → Settings → Environment variables.
  • Click + Add.
  • CosmosDBConnection: Paste the connection string you copied

Get Your Gemini API Key

  • In the left sidebar, click Get API key.
  • Click on the API key
  • Click the Copy icon next to your key to copy it.

Add the Gemini API Key to Staging

  • Go back to your staging Function App's Environment variables.
  • Click + Add.
  • Add the following setting:
    • Name: GEMINI_API_KEY
    • Value: Paste your Gemini API key
  • Click Apply.
  • Back on the Environment variables page, click Apply to save your changes.
  • Click Confirm to restart the app.

Why copy the same settings?

For this project, we're using the same Cosmos DB for both staging and production to keep things simple. In a real production environment, you'd typically have separate databases for each environment to avoid test data mixing with real user data.

Portal checkpoint: Function Apps → See both streaming-backend-staging and streaming-backend with matching configs

Nice! All your environment varaibles are setup for both Function App's

Build Your Multi-Stage Pipeline

Now for the main event - building a multi-stage CI/CD pipeline! This is exactly how Netflix orchestrates 4,000+ automated steps per deployment. You'll create a pipeline that runs tests, deploys to staging, then deploys to production - all automatically triggered by a simple git push.

In this step, get ready to:

  • Create the azure-pipelines.yml file
  • Configure multi-stage deployment

Create the Pipeline YAML File

The pipeline is defined in a YAML file that lives in your repository. You'll build it using 4 Cursor prompts, one for each stage. This helps you understand what each part of the pipeline does instead of copy-pasting a giant YAML file.

Open your streaming backend project in Cursor, create a new file called azure-pipelines.yml in the root directory, and open the Cursor chat panel (press Ctrl+L or Cmd+L).

Prompt 1: Build and Test Stage

  • Send this prompt to Cursor:
Create an Azure DevOps pipeline YAML file with a Build and Test stage.
The pipeline should trigger when code is pushed to the main branch.
Use a self-hosted agent pool named 'Default' instead of a Microsoft-hosted agent.
Add variables for Python version 3.11, a staging app called streaming-backend-staging, and a production app called streaming-backend.
The Build stage should set up Python, install dependencies from requirements.txt, install pytest, run the tests in the tests/ folder, and publish the test results even if tests fail.
  • Click Accept to apply the changes.

What's happening in this prompt?

This creates the pipeline foundation and the Build stage: when to trigger, which agent pool to use (your self-hosted agent), variables to reuse, and steps to install dependencies, run tests, and publish results.

Prompt 2: Deploy to Staging Stage

  • Send this prompt to Cursor:

🪟 Windows

Add a Deploy to Staging stage that only runs after the Build stage succeeds.
Use a deployment job with the staging environment.
It should checkout the code, set up Python, install dependencies, then deploy using AzureCLI@2 task.
In the AzureCLI task, use scriptType 'ps' (Windows PowerShell), install azure-functions-core-tools globally via npm, then run func azure functionapp publish with the stagingApp variable and --python flag.
Use 'your-azure-subscription' as a placeholder for the subscription name.

🍎 macOS

Add a Deploy to Staging stage that only runs after the Build stage succeeds.
Use a deployment job with the staging environment.
It should checkout the code, set up Python, install dependencies, then deploy using AzureCLI@2 task.
In the AzureCLI task, use scriptType 'bash', install azure-functions-core-tools globally via npm, then run func azure functionapp publish with the stagingApp variable and --python flag.
Use 'your-azure-subscription' as a placeholder for the subscription name.
  • Click Accept to apply the changes.

What's happening in this prompt?

This creates the Deploy to Staging stage, which only runs if Build passes. We use AzureCLI@2 with func azure functionapp publish instead of AzureFunctionApp@2 because it works better with Flex Consumption Function Apps. The deployment job type tracks deployments.

Prompt 3: Integration Tests Stage

  • Send this prompt to Cursor:
Add an Integration Tests stage that runs after the staging deployment succeeds.
It should wait 30 seconds for the deployment to stabilize, then curl the staging health endpoint to verify the deployment works.
Use the stagingAppName variable to build the URL.
  • Click Accept to apply the changes.

What's happening in this prompt?

This creates the Integration Tests stage, which verifies your staging deployment actually works by hitting the health endpoint before allowing production deployment.

Prompt 4: Deploy to Production Stage

  • Send this prompt to Cursor:

🪟 Windows

Add a Deploy to Production stage that runs after the integration tests pass.
Use a deployment job with the production environment.
It should checkout the code, set up Python, install dependencies, then deploy using AzureCLI@2 task.
In the AzureCLI task, use scriptType 'ps' (Windows PowerShell), install azure-functions-core-tools globally via npm, then run func azure functionapp publish with the productionApp variable and --python flag.
Use 'your-azure-subscription' as the subscription name.

🍎 macOS

Add a Deploy to Production stage that runs after the integration tests pass.
Use a deployment job with the production environment.
It should checkout the code, set up Python, install dependencies, then deploy using AzureCLI@2 task.
In the AzureCLI task, use scriptType 'bash', install azure-functions-core-tools globally via npm, then run func azure functionapp publish with the productionApp variable and --python flag.
Use 'your-azure-subscription' as the subscription name.
  • Click Accept to apply the changes.

What's happening in this prompt?

This creates the final stage that deploys to production after integration tests pass.

💡 What does this pipeline do?

This YAML file defines a multi-stage pipeline with four stages:

  1. Build - Installs Python, installs dependencies, runs tests, and publishes test results.
  2. DeployStaging - Deploys your code to the staging Function App after Build succeeds.
  3. IntegrationTests - Runs health checks against staging to verify the deployment works.
  4. DeployProduction - Deploys to production after integration tests pass. The trigger: - main means this pipeline runs automatically whenever you push to the main branch. The dependsOn and condition settings ensure stages run in order and only when previous stages succeed.

Push the Pipeline to GitHub

Before configuring Azure DevOps, push your new pipeline file to GitHub:

git add -A
git commit -m "Add Azure DevOps pipeline configuration"
git push origin main

Configure Azure Service Connection

The pipeline needs permission to deploy to your Azure Function Apps.

  • Go back to the Azure Portal and search for Azure DevOps organizations (the same way you did earlier in Step 1).
  • Click My Azure DevOps Organizations → Select your organization → Open your streaming-backend-cicd project.
  • Click Project Settings (bottom left corner).
  • Under Pipelines, click Service connections.
  • Click Create service connection.
  • Select Azure Resource Manager → Next.
  • Choose Service principal (automatic).
  • Select your subscription.
  • Leave Resource group empty (to access all resources).
  • Name it your-azure-subscription (must match the name in your YAML).
  • Click Save.

What is a service principal?

A service principal is like a robot identity for your pipeline. Instead of using your personal credentials (which would be a security risk), Azure creates a dedicated identity with limited permissions. This principal can only do what you allow it to do - in this case, deploy to your Function Apps.

Set Up Environments

The pipeline references staging and production environments. Let's create them.

  • In Azure DevOps, go to Pipelines → Environments.
  • Click New environment.
  • Name: staging, Resource: None.
  • Click Create.

Now create the production environment:

  • Click New environment.
  • Name: production, Resource: None.
  • Click Create.

Update the Pipeline YAML

Update the azure-pipelines.yml to use your actual Azure subscription connection name:

  • Replace your-azure-subscription with the name of your Azure service connection.
  • Replace streaming-backend-staging and streaming-backend with your actual Function App names if different.
  • Commit and push your changes:
git add azure-pipelines.yml
git commit -m "Add CI/CD pipeline configuration"
git push origin main

✅ Push succeeded

Your pipeline configuration is now in your repository. Continue to Step 3 to create and run the pipeline in Azure DevOps.

⚠️ Authentication error

If you get an authentication error when pushing:

  • Make sure you're authenticated with GitHub. Run: gh auth login
  • Or set up SSH keys: ssh-keygen -t ed25519 and add the public key to GitHub.
  • Try pushing again after authenticating.

❌ Push was rejected

If your push was rejected (e.g., "non-fast-forward"):

  • Someone else may have pushed changes. Pull first: git pull origin main
  • Resolve any merge conflicts if they appear.
  • Push again: git push origin main

Ship Code Like the Tech Giants

Time to see your pipeline in action! This is the moment of truth - you'll trigger a deployment, watch it flow through each stage, and verify everything is live. By the end, you'll understand why companies like Amazon can confidently deploy 136,000 times per day.

In this step, get ready to:

  • Create the pipeline in Azure DevOps
  • Watch the build and staging deployment
  • Verify production is live

Create the Pipeline in Azure DevOps

  • In Azure DevOps, go to Pipelines → Pipelines.
  • Click Create Pipeline.
  • Select GitHub as your code location.
  • Authorize Azure DevOps to access your GitHub if prompted.
  • Select your streaming-backend repository.
  • On the "Configure your pipeline" screen, select Existing Azure Pipelines YAML file.
  • Leave the branch as main and path as /azure-pipelines.yml.
  • Click Continue.
  • Review the YAML and click Run to start the pipeline.

How does Azure DevOps find your pipeline?

Azure DevOps automatically scans your repository for files named azure-pipelines.yml. When it finds one, it reads the configuration and knows exactly what stages, jobs, and steps to run. You can also have multiple pipeline files with different names if you need separate pipelines for different purposes.

Watch the Pipeline Run

The pipeline will execute in stages:

  1. Build and Test - Installs dependencies and runs your pytest tests.
  2. Deploy to Staging - Deploys your code to the staging Function App.
  3. Integration Tests Runs health checks against the staging environment.
  4. Deploy to Production - Deploys to production after integration tests pass.

Watch each stage complete:

  • Click on the pipeline run to see details.
  • Expand each stage to see individual step logs.

The first time you run the pipeline, it will need permission to access your agent pool. Click View to see the permission request.

  • Click Permit to allow the pipeline to use your self-hosted agent.
  • Click Permit again in the confirmation dialog.

Now the pipeline will start running. You can see the progress in your PowerShell terminal where the agent is running.

When the staging deployment completes, you'll need to approve access to the staging environment.

Your terminal will show the deployment progress.

✅ Pipeline is running

Watch as each stage completes. The pipeline will progress through:

  1. Build and Test (green checkmark when done)
  2. Deploy to Staging (green checkmark when done)
  3. Integration Tests (green checkmark when done)
  4. Deploy to Production (green checkmark when done)

Continue to verify production is live below.

❌ Build stage failed

If the Build stage fails, check these common issues:

  • Missing requirements.txt - Make sure your project has a requirements.txt file.
  • Invalid Python syntax - Check for syntax errors in your code.
  • Missing dependencies - Add any missing packages to requirements.txt.

Click on the failed step to see the detailed error logs, then fix the issue and push again.

Still stuck? Ask in the NextWork community or use this prompt to debug.

❌ Deploy stage failed

If the Deploy to Staging stage fails:

  • Service connection error - Verify your Azure service connection name matches exactly in the YAML.
  • Function App not found - Check that your Function App names are correct.
  • Permission denied - Ensure the service connection has Contributor access to your subscription.

Check the error logs, fix the issue in your YAML or Azure configuration, and push again.

Still stuck? Ask in the NextWork community or use this prompt to debug.

⚠️ Pipeline not starting

If the pipeline isn't starting after pushing:

  • Go to Pipelines → Pipelines and check if your pipeline exists.
  • If not, you may need to create it manually (continue to "Create the Pipeline" section).
  • Check that the trigger is set to main and you pushed to the main branch.
  • Verify the azure-pipelines.yml file is in the root of your repository.

Still stuck? Ask in the NextWork community or use this prompt to debug.

Wait for Pipeline Completion

The pipeline will continue through all stages and deploy to production. Once complete, all four stages will show green checkmarks.

Your terminal will show all jobs completed successfully.

Verify Production is Live

After the production deployment completes:

  • Go to the Azure Portal.
  • Search for Function App and click on it.
  • Click on your streaming-backend Function App (the production one, not staging).
  • In the Essentials section, find the Default domain and copy it.
  • Test the health endpoint in your browser or terminal:

Enter your production domain here

🪟 Windows

Invoke-WebRequest -Uri "https://[[PROD_DOMAIN="Enter your production domain here"]]/api/health" -UseBasicParsing

🍎 macOS

curl https://[[PROD_DOMAIN="streaming-backend.azurewebsites.net"]]/api/health

✅ Got a successful response

You should see a response like status: Healthy with a 200 status code. Your production API is live and working!

⚠️ Got a 404 Not Found

If you get a 404 error:

  • Make sure you have a /api/health endpoint in your Function App code.
  • Check that the function is deployed by going to your Function App in the Azure Portal → Functions.
  • Verify the function URL matches what you're calling.

Still stuck? Ask in the NextWork community or use this prompt to debug.

❌ Got a 500 error

If you get a 500 Internal Server Error:

  • Check the Function App logs in the Azure Portal → Log stream.
  • Make sure your environment variables are configured correctly.
  • Verify your code doesn't have runtime errors.

Still stuck? Ask in the NextWork community or use this prompt to debug.

❌ Request timed out

If your request times out or doesn't connect:

  • The Function App might be cold starting (first request after inactivity). Wait 30 seconds and try again.
  • Check that your Function App is running in the Azure Portal.
  • Verify the URL is correct (especially the Function App name).

Still stuck? Ask in the NextWork community or use this prompt to debug.

Why is this workflow powerful?

You just experienced the full CI/CD loop. Every time you push code, your pipeline automatically builds, tests, deploys to staging, runs integration tests, and deploys to production. This automation means you can ship features faster with confidence. No more manual FTP uploads, no more "it works on my machine" bugs, no more forgetting to run tests.

Secret mission

Your pipeline runs tests, but what about code quality? Right now, you could push code with inconsistent formatting, unused imports, or style violations - and the pipeline would happily deploy it.

Google's engineering teams enforce strict code standards - every line must pass automated linters before it can be merged. This isn't about being picky - it's about maintaining code that hundreds of engineers can work on without stepping on each other's toes.

In this secret mission, you'll add the same capability. Your pipeline will automatically check code quality and fail fast if standards aren't met - before wasting time on tests and deployment.

Add Code Quality Gates

Clean Up Your Resources

Clean Up Your Resources

Now that we've built and tested our CI/CD pipeline, it's time to decide what to do with the resources we created.

Note

Heads up! Your Azure DevOps project and pipeline configuration is portfolio-worthy work! Consider keeping it to showcase your CI/CD skills to employers. You can always delete it later if you change your mind.

Resources to manage:

  • Azure DevOps project and pipeline
  • Staging and production Function Apps
  • Self-hosted agent (VM or local machine)
  • Local project files

🟢 Keep everything running

If you want to continue experimenting or building on this project:

  • Your Function Apps will remain available.
  • The CI/CD pipeline will continue running on each push to main.
  • You'll incur costs for any Function App executions and storage.

No action needed - just continue building!

🟡 Stop but keep resources

If you want to pause costs but keep your setup for later:

Stop the Function Apps (keeps configuration):

  • Go to the Azure Portal.
  • Navigate to each Function App (streaming-backend-staging and streaming-backend).
  • Click Stop at the top of the overview page.
  • This stops billing for compute but keeps your configuration.

Disable the Pipeline:

  • Go to Azure DevOps → Pipelines.
  • Click on your pipeline → ⋮ menu → Settings.
  • Toggle Disabled to stop automatic runs.

You can restart everything later by clicking Start on the Function Apps and re-enabling the pipeline.

🔴 Delete everything

If you're completely done with this project:

Delete the Resource Group (deletes all Azure resources):

  • Go to the Azure Portal.
  • Navigate to Resource groups → streaming-backend-rg.
  • Click Delete resource group.
  • Type the resource group name to confirm.
  • Click Delete.

This will delete both Function Apps and all associated resources.

Delete Azure DevOps Project (optional):

  • Go to dev.azure.com.
  • Open your project.
  • Go to Project Settings (bottom left).
  • Scroll to the bottom and click Delete.
  • Type the project name to confirm.

Should I delete my Azure DevOps project?

Your Azure DevOps project showcases your CI/CD skills and is great for your portfolio! Many employers look at candidates who can set up automated deployment pipelines.

We recommend keeping your project unless you're certain you won't need it.

That's a wrap!

That's a wrap!

Nice work! 🚀 You've just built a production-grade CI/CD pipeline for your streaming backend using Azure DevOps.

You've learned how to:

  • 🔧 Set up Azure DevOps and connect it to GitHub.
  • 🏃 Configure a self-hosted agent to run your pipelines.
  • 🌍 Create staging and production environments with separate configurations.
  • 📝 Write multi-stage YAML pipelines with dependencies.
  • 🧪 Run automated tests and integration checks.
  • 🔄 Experience the full CI/CD loop from code push to production.
  • 💎 Add code quality gates with linting and formatting checks.

Congratulations! 🎉

You now have the skills to build automated deployment pipelines like professional DevOps engineers.

Ready to quiz yourself? 💪

Congratulations!

You've completed all four projects in the Azure Functions × AI Series! 🎊

Explore other NextWork projects to continue building your cloud skills:

p.s. Does it say "Still tasks to complete!" at the bottom of the screen?

This means you still have screenshots left to upload, or questions left to answer!

  1. Press Ctrl+F (Windows) or Command+F (Mac) on your keyboard.
  2. Search for the text Return to later.
  3. Jump straight to your incomplete tasks!
  4. 🙋‍♀️ Still stuck? Ask the community!