Orchestrating Parallel Event Streams with Azure

Introduction

This project has been archived

We're no longer updating this project - we recommend checking out our active projects instead!

If you've already completed this project, don't worry! Your documentation is still available for you to check out and share.

If you have any questions, leave a question in our Discord and we'll get back to you ASAP!

⚡️ 30 second Summary

Part of handling apps with large volume is working out how to run multiple events in parallel. You might need to update stats, send alerts, and log analytics, all at the same time. Doing all these events in parallel helps to prevent your app from slowing down and crashing.

Toyota uses Azure Durable Functions to coordinate parallel processing across engineering teams, specifically for their O-Beya AI system.

You'll build this exact same pattern - creating orchestrations with Durable Functions that fan out to multiple activities in parallel, aggregate stats in real-time, and retry failed operations to keep your app stable and running.

What you'll build

In this project, you'll stream alerts into an Azure Storage Queue and use Durable Function orchestration to update stats, send alerts, and log analytics, all at the same time. At the end of it all, we'll use Cosmos DB to make the stats available to any developers who are interested.

By the end of this project, you'll have:

• 🦆 A working event queue that buffers stream events during traffic spikes.
• 🪭 Durable Function orchestrations with fan-out patterns for parallel processing.
• 🌯 Real-time stats aggregation with Cosmos DB.
• 💎 Secret Mission: Retry policies and dead-letter handling for failed notifications.

Want a complete demo of how to do this project, from start to finish? Check out our 🎬 walkthrough with Kahu

Not sure if this project is right for you? Check if it matches your goals

If you're up for a bit of a challenge, quiz yourself on the key concepts up ahead in this project.

This project is part of a series:

1. Part 1: Build Your First Azure Function
2. Part 2: Build AI Chat Moderator with Gemini
3. Part 3: You are here!
4. Part 4: GitHub to Azure DevOps Pipeline

Build Your Event Buffer

Every enterprise system that handles unpredictable traffic uses some form of message queue. When Toyota's engineering systems need to coordinate work across multiple teams, or when fintech platforms process thousands of payroll transactions, queues prevent the system from getting overwhelmed.

You'll create an Azure Storage Queue that buffers incoming stream events - followers, subscriptions, and donations. This is the first layer of resilience in your event-driven architecture.

What is a message queue and why do we need one?

A message queue is like a waiting room between event producers and event consumers. When a streamer goes viral and gets thousands of new followers in seconds, the queue holds all those events temporarily. Your backend can then process them at a steady pace instead of getting overwhelmed.

This pattern is called queue-based load leveling - it decouples the rate at which events arrive from the rate at which they're processed. Without a queue, a sudden traffic spike could crash your server or cause events to be lost.

Prerequisites from Parts 1 & 2

✅ I completed Parts 1 & 2

You should have:

• A logged in Azure account.
• An Azure Function App deployed and running.
• A Cosmos DB account with messages and violations containers.
• Working POST /message and GET /messages endpoints.
• AI moderation with Gemini integrated.

Continue to the next section!

🔄 I only completed Part 1

You need to complete Part 2 (AI Content Moderation) before continuing.

Get Your Google AI API Key

• Go to https://aistudio.google.com/ and sign in with your Google account.
• Click Get API key in the sidebar.
• Click Create API key → Create project → Name it → Create key.

• Copy your API key.

Add the API Key to Azure Function App

• Go to the Azure Portal → your Function App.
• Select Configuration under Settings.
• Click New application setting:
  • Name: GEMINI_API_KEY
  • Value: Your Gemini API key
• Click OK → Save → Continue.

Create the Violations Container

• Go to Azure Portal → Cosmos DB → your account → Data Explorer.
• Click New Container:
  • Database id: Use existing (e.g., streamingdb)
  • Container id: violations
  • Partition key: /streamId
• Click OK.

Create the Moderation Module

• Create moderation.py and send this prompt in Cursor:

Create a Python module called moderation.py that checks messages for toxic content using the Gemini API.

Requirements:
1. Get the API key from environment variable GEMINI_API_KEY
2. Use the Gemini API endpoint: https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent
3. Create a check_message(message_content: str) function that:
   - Sends the message to Gemini with a prompt asking it to classify as clean/toxic/spam/harassment
   - Returns a dict with: is_allowed (bool), category (str), confidence (float 0-1), reason (str)
   - Uses "fail-open" pattern: if the API fails, return is_allowed=True
   - Handles markdown code blocks in Gemini's response (strip them before parsing JSON)
4. Use the requests library for HTTP calls

Update the POST /message Endpoint

• Open function_app.py and send this prompt:

Update my Azure Function POST /message endpoint to add content moderation.

Requirements:
1. Import check_message from the moderation module I just created
2. Before saving a message, call check_message(message_content)
3. If is_allowed is False:
   - Save the message to a "violations" container in Cosmos DB (include category, confidence, reason, timestamp)
   - Return HTTP 403 with the category and reason in the response
4. If is_allowed is True:
   - Save to the messages container as before
   - Return HTTP 200 success
5. Keep my existing Cosmos DB connection code

Update requirements.txt

Make sure requests is in your requirements.txt:

azure-functions
azure-cosmos
requests

Deploy and test before continuing.

🚀 I need to start from scratch

You need to complete Parts 1 & 2 before continuing.

Part 1: Set Up Azure Environment

• Go to azure.microsoft.com/free and sign up.

• In Azure Portal: Create a resource → Search Resource group → Create.
  • Resource group: streaming-backend-rg
  • Region: Select a region close to you.

• Install Azure Functions Core Tools:

macOS:

brew tap azure/functions
brew install azure-functions-core-tools@4

Windows:

npm install -g azure-functions-core-tools@4 --unsafe-perm true

• Install Azure CLI and log in:

az login

Create Your Function App

• Create a folder called streaming-backend and open it in Cursor.
• Initialize the project:

func init --python

• Create the message endpoint:

func new --name message --template "HTTP trigger" --authlevel "anonymous"

• Open function_app.py and send this prompt in Cursor:

Create an Azure Function app with three HTTP endpoints:

1. POST /message - receives JSON with "content" and optional "username", stores in an in-memory list, returns the created message with a UUID and timestamp
2. GET /messages - returns all stored messages as JSON
3. GET /health - returns "Healthy" status

Use the azure.functions SDK with @app.route decorators. Include proper error handling.

Deploy to Azure

• In Azure Portal: Create a resource → Storage account → Create.
  • Resource group: streaming-backend-rg
  • Storage account name: streamingbackendstorage (globally unique)
  • Performance: Standard
  • Redundancy: LRS

• Search Function App → Create.
  • Hosting option: Consumption
  • Resource group: streaming-backend-rg
  • Function App name: streaming-backend (globally unique)
  • Runtime: Python 3.11
  • Storage: Select the storage account you created

• Deploy:

func azure functionapp publish streaming-backend

Add Cosmos DB (Secret Mission from Part 1)

• In Azure Portal: Create a resource → Azure Cosmos DB → Create → Azure Cosmos DB for NoSQL.
  • Resource group: streaming-backend-rg
  • Account name: streaming-cosmos (globally unique)
  • Capacity mode: Serverless
• Create database streamingdb with container messages (partition key: /id).
• Copy the PRIMARY CONNECTION STRING from Settings → Keys.
• Add it to your Function App's Configuration as CosmosDBConnection.

Part 2: Add AI Moderation

• Go to https://aistudio.google.com/ → Get API key → Create API key.

• Add GEMINI_API_KEY to your Function App's Configuration.
• Create violations container in Cosmos DB (partition key: /streamId).

• Create moderation.py with Cursor prompt for Gemini integration.
• Update POST /message endpoint to check messages before saving.
• Add requests to requirements.txt.

Deploy and test before continuing.

In this step, you're going to:

• Set up a Storage Queue for alerts in your existing Storage Account
• Verify your Function App can connect to the queue

Navigate to Your Storage Account

You already have a Storage Account from Part 1 - let's use that for your queue.

• Open the Azure Portal and sign in with your Azure account.
• Search for Storage accounts in the top search bar.

• Select your existing Storage Account (e.g., streamingbackendstorage).

Why reuse the existing Storage Account?

Your Function App already has a Storage Account that was created in Part 1. Azure Storage Accounts can hold multiple services - blobs, queues, tables, and files. Adding a queue to your existing account keeps everything organized and avoids unnecessary resource sprawl.

Create the Alerts Queue

Now let's create the queue that will receive stream events.

• In the left menu, select Storage browser.

• Click Queues in the storage browser.

• Click + Add queue.

• Enter the name: alerts
• Click Ok.

Your queue is now ready to receive messages.

Why name it alerts?

The queue name must match what your function code expects. We're calling it alerts because it will hold stream alert events (followers, subscriptions, donations). In the next step, you'll write code that listens specifically to this queue name.

Verify Your Function App Connection

Your Function App was configured with AzureWebJobsStorage when you created it in Part 1 - this connection string already points to your Storage Account. Let's verify it's set up correctly.

• Search for Function App in the Azure Portal.

• Select your Function App.

• Go to Settings → Environment variables.

• Look for AzureWebJobsStorage in the list.

✅ I see AzureWebJobsStorage

Nice! Your Function App is already connected to the Storage Account. Since the queue is in the same Storage Account, your function will automatically be able to read from and write to it.

❌ I don't see it

If AzureWebJobsStorage is missing, you need to add it:

• In your Storage Account, go to Security + networking → Access keys.
• Click Show next to Connection string for key1.
• Copy the connection string.
• Go back to your Function App → Settings → Environment variables.
• Click + Add.
• Name: AzureWebJobsStorage
• Value: Paste the connection string.
• Click Apply, then Apply again to save.

Still stuck?

• Get help with Azure Storage connection or ask the NextWork community!

Orchestrate Parallel Processing

This is where the real power of Durable Functions comes in. Toyota chose Durable Functions for their O-Beya AI system specifically because of its ability to fan out work across multiple agents in parallel, then aggregate the results.

You'll build a Durable Function orchestration that fans out to three activities simultaneously - updating stats, sending alerts, and logging analytics. The fan-out pattern is how you process work in parallel without losing track of any pieces.

What are Durable Functions?

Regular Azure Functions are stateless - they run, complete, and forget everything. But what if you need to coordinate multiple steps, run tasks in parallel, or track progress over time?

That's where Durable Functions come in. They add state management on top of regular functions, letting you write complex workflows that can pause, wait for external events, and resume where they left off. Behind the scenes, Durable Functions automatically save their state to Azure Storage, so they're resilient to failures and restarts.

In this step, you're going to:

• Create a queue-triggered function to start orchestrations
• Build the orchestrator function with fan-out pattern
• Implement activity functions for stats, alerts, and logging

Understand the Architecture

Before we code, let's understand what we're building:

When an event arrives in the queue, it triggers a starter function that kicks off an orchestration. The orchestrator then fans out to run multiple activities in parallel: updating stats, sending an alert, and logging analytics.

What is the fan-out/fan-in pattern?

Fan-out/fan-in is a pattern for parallel processing.

• Fan-out means starting multiple tasks at the same time (like dealing cards to multiple players).
• Fan-in means waiting for all those tasks to finish and combining the results. In this project, when a stream event arrives, we "fan out" to three activities simultaneously: updating stats, sending an alert, and logging analytics. This is much faster than running them one after another.

Set Up Your Development Environment

Before writing code, make sure you have the required tools installed.

Run this command in your terminal to check:

func --version

✅ I see a version number (4.x)

You have Azure Functions Core Tools installed. Make sure you also have:

• Azure Functions Core Tools (v4)
• Cursor (recommended code editor for this project)

Also check that Python 3.11 is installed:

python --version

✅ I see Python 3.11

You're all set! Continue to the next section.

❌ I need to install Python

🍎 macOS

brew install python@3.11

Or download from python.org.

🪟 Windows

• Download the installer from python.org.
• Run the installer.

Important

Check "Add Python to PATH" during installation.

• Restart your terminal after installation.
• Verify installation:

python --version

🔧 Command not found

Let's install the Azure Functions Core Tools for local development.

🍎 macOS

• Open Terminal
• Install using Homebrew:

brew tap azure/functions
brew install azure-functions-core-tools@4

• Verify installation:

func --version

You should see version 4.x.

🪟 Windows

• Download the installer from Azure Functions Core Tools
• Run the installer and follow the prompts
• Restart your terminal after installation
• Verify installation:

func --version

You should see version 4.x.

Also make sure you have Python 3.11 installed:

python --version

If not, install Python 3.11:

🍎 macOS

brew install python@3.11

Or download from python.org.

🪟 Windows

Download and run the installer from python.org.

Important

Check "Add Python to PATH" during installation.

Open Your Project in Cursor

Time to write some code! Let's open your streaming backend project in Cursor.

• Open Cursor on your computer.

• Click File → Open Folder.
• Navigate to your streaming-backend project folder and select it.

You should see your project files in the sidebar, including function_app.py and requirements.txt from Part 1.

Don't have Cursor?

Download it from cursor.com. Cursor is an AI-powered code editor that can generate code from prompts. You can also use VS Code with GitHub Copilot if you prefer.

Create the Durable Functions App

• Open the Cursor chat panel (press Ctrl+L or Cmd+L).
• Send this prompt:

Create an Azure Durable Functions app in function_app.py for processing stream events (followers, subscriptions, donations).

Requirements:
1. Use a single df.DFApp() instance for all functions — do NOT create a separate func.FunctionApp()                     
2. Create a queue trigger on the "alerts" queue (connection: AzureWebJobsStorage) that starts a "event_orchestrator"
orchestration
3. The orchestrator uses fan-out/fan-in to run three activities in parallel:
  - update_stats: returns status "updated"
  - send_alert: builds "New {type} from {username}!", returns status "sent"
  - log_analytics: returns status "logged"
4. Add an HTTP POST endpoint at /api/event for testing
5. req.get_json() is synchronous — do NOT await it
6. Do NOT use df.DurableClient as a type hint (it doesn't exist)

What's happening in this prompt?

This prompt asks Cursor to create a complete Durable Functions workflow:

• A queue trigger that starts orchestrations when events arrive
• An orchestrator that coordinates parallel processing using fan-out/fan-in
• Three activity functions that do the actual work independently
• An HTTP endpoint for manual testing

The fan-out pattern runs all activities simultaneously, then waits for all to complete before returning.

• (If prompted) Click Accept to apply the changes.

What do the decorators mean?

• @app.queue_trigger: Tells Azure Functions to run this code whenever a message arrives in the specified queue
• @app.durable_client_input: Injects a Durable Functions client that can start and manage orchestrations
• @app.orchestration_trigger: Marks a function as an orchestrator that coordinates the workflow
• @app.activity_trigger: Marks a function as an activity that does actual work

Update requirements.txt

• Open requirements.txt.

• Make sure it includes these four packages:

azure-functions
azure-functions-durable
azure-cosmos
requests

Important

Make sure the package name is spelled correctly as azure-functions-durable (not azure-durable-functions). The wrong spelling will cause deployment to fail with "No matching distribution found".

Deploy and Test

Deploy your function and test the orchestration.

• Open the terminal in Cursor by pressing Ctrl+` (or Cmd+` on macOS).
• Make sure you're in your streaming-backend project folder.
• Check if you're logged in to Azure:

az account show

✅ I see my subscription info

You're already logged in. You should see output like this:

🔐 Please run 'az login'

If you see "Please run 'az login'":

Log in using the Azure CLI:

az login

• Select your Azure account in the browser window that opens.

• If prompted, select your subscription:

• Verify you're logged in:

az account show

• Deploy using the Azure Functions Core Tools:

func azure functionapp publish streaming-backend

• Wait for deployment to complete - you should see your functions listed in the output.

Now test the orchestration.

Copy your Function App URL:

[[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]

Send a test event:

🍎 macOS / Linux

curl -X POST [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/event \
  -H "Content-Type: application/json" \
  -d '{"type": "follower", "stream_id": "stream123", "username": "testuser"}'

🪟 Windows (PowerShell)

Invoke-WebRequest -Uri [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/event `
  -Method POST `
  -Headers @{"Content-Type"="application/json"} `
  -Body '{"type": "follower", "stream_id": "stream123", "username": "testuser"}'

Check your results:

✅ I got an instance_id response

You should see a response like:

The response includes a statusQueryGetUri - this is a URL you can use to check the orchestration status. Copy that URL and paste it in your browser (or run it with Invoke-WebRequest). You should see "runtimeStatus": "Completed" which means the orchestration finished successfully.

Why use statusQueryGetUri instead of the Azure Portal?

The Azure Portal's Monitor view can have delays showing orchestration results. The statusQueryGetUri queries the orchestration status directly and always shows real-time results.

❌ I'm having issues

Got a 404 error?

• Make sure deployment completed successfully
• Verify the URL matches your Function App name exactly
• Check that the function app is running in the Azure Portal
• Try refreshing the function list in Azure Portal

Got a 500 error?

• Go to your Function App → Monitor → Logs to see error details
• Make sure the AzureWebJobsStorage connection string is configured correctly
• Verify all dependencies are in requirements.txt
• Check that the Durable Functions extension is installed

Still stuck?

• Get help with Azure Functions or ask the NextWork community!

Aggregate Real-Time Analytics

Netflix processes billions of viewing events daily to power their recommendation engine. Spotify aggregates listening data in real-time to update personalized playlists. Your orchestration is processing events in parallel - now you need somewhere to store the results and aggregate them into meaningful metrics.

Why do we need a database for analytics?

Right now, your activity functions process events but don't persist the results anywhere. To show aggregate statistics (total followers, total donations), you need to store data across events.

Cosmos DB is a good choice because it's serverless (like your functions), scales automatically, and integrates well with Azure Functions. You'll store each event for historical analysis AND maintain running totals for fast queries.

In this step, you're going to:

• Set up Cosmos DB for storing analytics data
• Implement the stats aggregation endpoint
• Test with simulated events

Set Up Cosmos DB Containers

You already have a Cosmos DB account from Part 1. Let's add new containers for analytics data.

• In the Azure Portal, search for Azure Cosmos DB.

• Select your existing Cosmos DB account (e.g., streaming-backend-cosmos).

Create the Containers

• Navigate to your Cosmos DB account.
• Go to Data Explorer.

• Click New Container.

• Database id: Create new → StreamData
• Container id: analytics
• Partition key: /stream_id

• Click OK.

Create another container for aggregated stats:

• Click New Container.
• Database id: Use existing → StreamData
• Container id: stats
• Partition key: /stream_id

• Click OK.

Why two containers?

We use separate containers for different access patterns:

• analytics - Stores raw events (every follower, sub, donation). Good for detailed reporting and debugging.
• stats - Stores aggregated counts per stream. Fast lookups for real-time dashboards.

The /stream_id partition key groups data by stream, so queries for a specific stream are fast. This is a common NoSQL pattern - optimize for your most frequent query.

Add the Cosmos DB Connection String

Your Function App needs the connection string to access Cosmos DB. Let's add it now.

• In your Cosmos DB account, go to Settings → Keys.

• Copy the PRIMARY CONNECTION STRING (not the URI - the connection string starts with AccountEndpoint=).

Important

Copy the PRIMARY CONNECTION STRING, not the URI. The connection string looks like AccountEndpoint=https://...;AccountKey=... and is much longer than the URI.

• Go to your Function App → Settings → Environment variables.

• Click + Add.

• Name: CosmosDBConnection
• Value: Paste the connection string you copied.
• Click Apply, then Apply again to save.

Why a separate connection string?

You already have COSMOS_CONNECTION_STRING from Part 1 for the streamingdb database. This new CosmosDBConnection variable connects to the same Cosmos DB account but we use it for the StreamData database. Using separate variables makes it clear which database each part of your code connects to.

Update the Activity Functions

Now let's update your activity functions to write to Cosmos DB.

• In Cursor chat, send this prompt:

Update my Azure Functions app to use Cosmos DB for the activity functions.

Requirements:
1. Get the Cosmos DB connection string from environment variable CosmosDBConnection
2. Connect to database "StreamData" with containers "analytics" and "stats"
3. Update the log_analytics activity to:
   - Create a document with id (UUID), stream_id, type, username, amount, and timestamp
   - Save it to the analytics container
   - Return status "logged" with the document_id
4. Update the update_stats activity to:
   - Try to read the existing stats document for the stream_id
   - If it doesn't exist, create one with follower_count, sub_count, and total_donations all set to 0
   - Increment the appropriate counter based on event type (follower, subscription, or donation)
   - Use upsert_item to save the updated stats
   - Return status "updated"
5. Use the azure.cosmos SDK (CosmosClient)

What's happening in this prompt?

This prompt asks Cursor to connect your activity functions to Cosmos DB:

• The log_analytics activity saves every event for historical analysis
• The update_stats activity maintains running totals using upsert (update if exists, insert if not)
• Both activities use the same partition key (/stream_id) for efficient queries

💡 What does upsert_item do?

upsert is a combination of "update" and "insert." If a document with that ID already exists, it updates it. If it doesn't exist, it creates a new one. This is perfect for our stats document - on the first event for a stream, it creates the stats doc. On subsequent events, it updates the existing counts.

• (If prompted) Click Accept to apply the changes.

Add the Stats Endpoint

Now add an HTTP endpoint to retrieve stats for a stream.

• In Cursor chat, send this prompt:

Add a GET endpoint at /api/stats/{stream_id} to retrieve stream statistics.

Requirements:
1. Extract stream_id from the route parameters
2. Try to read the stats document from the stats container
3. If found, return JSON with stream_id, follower_count, sub_count, and total_donations
4. If not found, return JSON with all counts set to 0
5. Return appropriate HTTP responses with application/json content type

What's happening in this prompt?

This prompt adds a read endpoint so you can query the aggregated stats. It handles the case where no stats exist yet (returns zeros instead of an error).

Cursor will add the new endpoint to your function_app.py.

• (If prompted) Click Accept to apply the changes.

Deploy Your Updated Code

You've added new functionality - now deploy it to Azure:

func azure functionapp publish streaming-backend

Wait for deployment to complete before testing.

Update requirements.txt

Cursor may have already added the Cosmos DB library. Let's check!

• Open requirements.txt in your editor and look for azure-cosmos.

✅ I see azure-cosmos

If you see azure-cosmos in your requirements.txt, you're good to go!

❌ I don't see it

Add azure-cosmos to your requirements.txt:

azure-functions
azure-functions-durable
azure-cosmos

Test with Simulated Events

Deploy your updated functions and test the full flow.

Send several test events:

🍎 macOS / Linux

Send a follower event:

curl -X POST [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/event \
  -H "Content-Type: application/json" \
  -d '{"type": "follower", "stream_id": "stream123", "username": "user1"}'

Send a subscription event:

curl -X POST [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/event \
  -H "Content-Type: application/json" \
  -d '{"type": "subscription", "stream_id": "stream123", "username": "user2"}'

Send a donation event:

curl -X POST [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/event \
  -H "Content-Type: application/json" \
  -d '{"type": "donation", "stream_id": "stream123", "username": "user3", "amount": 10.00}'

Check the stats:

curl [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/stats/stream123

🪟 Windows (PowerShell)

Send a follower event:

Invoke-WebRequest -Uri [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/event `
  -Method POST `
  -Headers @{"Content-Type"="application/json"} `
  -Body '{"type": "follower", "stream_id": "stream123", "username": "user1"}'

Send a subscription event:

Invoke-WebRequest -Uri [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/event `
  -Method POST `
  -Headers @{"Content-Type"="application/json"} `
  -Body '{"type": "subscription", "stream_id": "stream123", "username": "user2"}'

Send a donation event:

Invoke-WebRequest -Uri [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/event `
  -Method POST `
  -Headers @{"Content-Type"="application/json"} `
  -Body '{"type": "donation", "stream_id": "stream123", "username": "user3", "amount": 10.00}'

Check the stats:

Invoke-WebRequest -Uri [[FUNCTION_URL="https://streaming-backend.azurewebsites.net"]]/api/stats/stream123

Check your results:

✅ I see follower/sub/donation counts

You should see something like:

Why are the counts zero?

The orchestration runs asynchronously - when you call the stats endpoint immediately after sending events, the activities may not have finished writing to Cosmos DB yet. Wait a few seconds and try again, or check the Azure Portal → Function App → Monitor to see if the orchestrations completed.

Your stats aggregation is working. Verify in the Azure Portal: Go to Cosmos DB → Data Explorer → Browse the analytics and stats containers to see your data.

❌ I'm having issues

I see all zeros?

This means the events were sent but may not have been processed yet.

• Go to your Function App → Monitor to see if orchestrations completed
• Check that the CosmosDBConnection environment variable is set correctly
• Verify the activity functions ran without errors in the logs
• Wait a few seconds and try the curl command again

I got an error?

• Connection refused: Make sure your Function App is running
• 500 error: Check the Function App logs for Cosmos DB connection issues
• 404 error: Make sure you deployed the latest code with the stats endpoint

Go to your Function App → Monitor → Logs to see detailed error messages.

Still stuck?

• Get help with stats aggregation or ask the NextWork community!

Secret mission

Your orchestration works when everything goes right. But what happens when an external notification service goes down? When a database connection times out? Production systems fail - what matters is how they recover.

Stripe processes billions of dollars in payments by using retry policies with exponential backoff. When a payment processor is temporarily unavailable, they don't crash - they wait and retry. Events that permanently fail go to dead-letter queues for investigation.

In this secret mission, you'll add the same production-grade error handling to your orchestration.

Handle Failures Like the Pros

Clean Up Your Resources

Clean Up Your Resources

Nice work completing this project! Let's clean up the resources to avoid ongoing charges.

🟢 I want to keep my resources

If you plan to continue using these resources for other projects or learning:

• No action needed - your resources will remain running
• Be aware of ongoing costs:
  • Function App (Consumption plan): Pay-per-execution (very low if not actively used)
  • Storage Account: Small cost for storage used
  • Cosmos DB (Serverless): Pay-per-request (minimal if not actively querying)

To monitor costs:

• Go to Azure Portal → Cost Management + Billing
• Check Cost analysis for your subscription
• Set up budget alerts to notify you of unexpected charges

🟡 I want to stop services temporarily

To pause your services without deleting them:

Function App:

• Navigate to your Function App in Azure Portal
• Click Stop in the top menu
• Your functions won't run (no charges for execution)
• Click Start when you want to resume

Note: Storage Account and Cosmos DB don't have a "stop" option - they're always available. You'll still incur small storage costs, but no execution costs.

🔴 I want to delete everything

Option 1: Delete the Resource Group (Recommended)

If you created a dedicated resource group for this project, delete everything at once:

• In the Azure Portal, go to Resource groups
• Select your resource group (e.g., rg-stream-alerts)
• Click Delete resource group
• Type the resource group name to confirm
• Click Delete

This removes all resources in the group, including the Storage Account, Cosmos DB, and Function App.

Option 2: Delete Individual Resources

If you want to keep some resources:

• Navigate to each resource in the Azure Portal
• Click Delete in the top menu
• Confirm the deletion

Resources to delete:

• Azure Storage Account
• Azure Cosmos DB account
• Azure Function App

That's a wrap!

That's a wrap!

You did it! You've built an event-driven streaming backend with async processing using Azure Durable Functions.

What you learned:

• 📨 How message queues buffer events and prevent system overload during traffic spikes
• ⚡ Azure Durable Functions and the fan-out/fan-in pattern for parallel processing
• 🗄️ Cosmos DB for NoSQL data storage with partition keys and upsert operations
• 📊 Data aggregation patterns: storing raw events vs. maintaining running totals
• 🔄 Event-driven architecture: decoupling producers from consumers for scalable systems

Ready to quiz yourself? 💪

What's next?

In Part 4 of this series (coming soon), get ready to add real-time notifications with Azure SignalR!

Make sure to join the NextWork Community to be the first to hear when it's out 🤝

p.s. Does it say "Still tasks to complete!" at the bottom of the screen?

This means you still have screenshots left to upload, or questions left to answer!

1. Press Ctrl+F (Windows) or Command+F (Mac) on your keyboard.
2. Search for the text Return to later.
3. Jump straight to your incomplete tasks!
4. 🙋‍♀️ Still stuck? Ask the community!