Event-Driven Pipeline with Azure Functions

Build a serverless pipeline that catalogs file uploads to Blob Storage in Cosmos DB.

Introduction

30 Second Summary

Every day, businesses upload thousands of files to cloud storage. Without automation, someone has to manually catalog and track each one.

In this project, you will build an event-driven document processing pipeline on Azure. When a file lands in storage, a serverless function automatically extracts its metadata and writes it to a database you can query through an API.

What You'll Build

You'll upload a file, watch your serverless function fire in the terminal, and then query the full catalog of processed documents through a live HTTP endpoint you deployed to Azure.

By the end of this project, you'll have:

  • Upload any file to Azure Blob Storage and watch an Azure Function fire automatically, extracting metadata like file name, size, and timestamp.
  • Call an HTTP endpoint you built and get back a JSON array of every processed document, pulled straight from Azure Cosmos DB.
  • A live deployment on Azure with a public URL, where your entire pipeline runs in the cloud without any servers to manage.
  • An optional Secret Mission to push your skills further.

Do I need an Azure account?

Yes. This project uses Azure's free tier, which covers all the resources you'll create. You can sign up at azure.microsoft.com/free if you don't have one yet.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. Taking a moment to think about the goal now helps you stay oriented as the project comes together.

Set Up Your Azure Environment

To build an event-driven pipeline, you need a local development environment that can talk to Azure. Three tools make this possible: the Azure CLI for managing cloud resources, Azure Functions Core Tools for developing and testing functions locally, and Node.js as the runtime for your function code.

In this step, you will install all three tools, log in to Azure, and create the resource group that holds every cloud resource for this project.

In this step, get ready to:
  • Install and verify Node.js, Azure CLI, and Azure Functions Core Tools.
  • Log in to Azure and create a resource group for the project.
  • Confirm all tools are working with version checks.
Install Node.js

Node.js is the runtime that executes your Azure Functions code. This project requires v22 LTS or higher.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Check whether Node.js is already installed by running:
node --version

✔️ I see version 22 or higher

Your Node.js version is ready. Skip ahead to the next substep.

ⓧ I see an older version

You have Node.js installed, but your version is below v22. You need to upgrade to avoid compatibility issues with the Azure Functions v4 programming model.

macOS

  • Install the v22 formula by running:
brew install node@22
  • Link it as your default Node.js by running:
brew link --overwrite node@22
  • Verify the upgrade by running node --version. You should see a version starting with v22.

Windows

  • Install Node.js v22 by running:
winget install --id OpenJS.NodeJS.22 -e
  • Close and reopen your terminal after installation.
  • Verify the upgrade by running node --version. You should see a version starting with v22.

ⓧ Command not found

Node.js is not installed yet. Let's install v22 LTS now.

macOS

  • Install Node.js v22 using Homebrew by running:
brew install node@22
  • Link it as your default node command by running:
brew link --overwrite node@22
  • Verify the install by running node --version. You should see a version starting with v22.

Windows

  • Install Node.js v22 by running:
winget install --id OpenJS.NodeJS.22 -e
  • Close and reopen your terminal after installation.
  • Verify the install by running node --version. You should see a version starting with v22.
Install Azure CLI and Azure Functions Core Tools

The Azure CLI lets you create and manage cloud resources from your terminal. Azure Functions Core Tools lets you run and debug functions on your machine before deploying them.

macOS

  • Install the Azure CLI by running:
brew update && brew install azure-cli
  • Confirm it installed by running:
az version

You should see a JSON object with a version number. The exact number does not matter as long as the command succeeds.

  • Now install Azure Functions Core Tools v4 by running these two commands:
brew tap azure/functions
brew install azure-functions-core-tools@4
  • Confirm it installed by running:
func --version

You should see a version number starting with 4.

Windows

  • Install the Azure CLI by running:
winget install --exact --id Microsoft.AzureCLI
  • Close and reopen your terminal after installation.
  • Confirm it installed by running:
az version

You should see a JSON object with a version number. The exact number does not matter as long as the command succeeds.

  • Now install Azure Functions Core Tools by running:
winget install Microsoft.Azure.FunctionsCoreTools
  • Close and reopen your terminal after installation.
  • Confirm it installed by running:
func --version

You should see a version number starting with 4.

Seeing 'command not found' after installing?

Close your terminal completely and open a fresh one. Newly installed tools are only available in new terminal sessions.

On macOS, make sure Homebrew itself is installed. Visit brew.sh for install instructions.

Help me troubleshoot my Azure CLI or Functions Core Tools installation.

Log in to Azure and create a resource group

A resource group is a container that holds all the Azure resources for a project. Creating one now means every service you provision later lives in one place that you can clean up with a single command.

  • Log in to your Azure account by running:
az login

What happens when you run az login?

Your default browser opens an Azure sign-in page. Sign in with the Microsoft account tied to your Azure subscription. Once you authenticate, the terminal prints your subscription details.

If you have multiple subscriptions, look for the one where isDefault is true. That is the subscription the CLI will use for all subsequent commands.

  • Create the resource group for this project by running:
az group create --name doc-pipeline-rg --location eastus

What is a resource group?

Think of a resource group as a folder for cloud resources. Every Azure service you create in this project (storage, database, functions) will live inside doc-pipeline-rg. When the project is done, deleting the resource group removes everything inside it at once.

The --location eastus flag sets the region where your resources are physically hosted. You can change this to a region closer to you, but eastus has broad service availability and works well for learning projects.

You should see a JSON response where provisioningState is Succeeded. That confirms your resource group is live.

Getting an authorization or subscription error?

Make sure you completed the az login step and that your Azure account has an active subscription. Free-tier accounts work for this project.

If you see a message about no subscriptions, run az account list to check which subscriptions are available. Set the correct one with az account set --subscription followed by your subscription ID.

Help me fix my Azure login or subscription error.

Before moving on, let's confirm every tool is ready in one go.

  • What do you expect to see when you run all three version commands together? Take a moment to predict, then run:
az version && func --version && node --version

All three commands print version numbers without errors. You should see the Azure CLI version as a JSON object, a 4.x number for Core Tools, and a v22.x number for Node.js.

Your local environment is locked in and your Azure resource group is standing by. Next up, you will scaffold your first Azure Function and see it respond to requests on your own machine.

Create Your First Function and See It Run Locally

Your Azure toolkit is ready. You have the CLI, Functions Core Tools, and Node.js installed, and a resource group called doc-pipeline-rg waiting in the cloud.

Before wiring up cloud services, you need to confirm the local development loop works. In this step, you'll scaffold an Azure Functions project, add an HTTP-triggered function, and run it to see instant output.

In this step, get ready to:
  • Scaffold a new Azure Functions JavaScript project using the v4 programming model.
  • Add an HTTP-triggered function to the project.
  • Run the function locally and test it in your browser.
Scaffold the Azure Functions project

An Azure Functions project is a folder containing configuration files and your function code. The func init command scaffolds this project structure for you.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Scaffold a new Azure Functions project by running this command:
func init doc-processing-pipeline --worker-runtime javascript --model V4

What does this command do?

The --worker-runtime javascript flag tells Core Tools to create a Node.js project. The --model V4 flag selects the v4 programming model, which uses a code-centric approach where you register functions directly in JavaScript instead of separate config files.

  • Move into the new project folder by running:
cd doc-processing-pipeline
  • Confirm the scaffolded files exist by running:
ls

You should see host.json, local.settings.json, package.json, and a few other files like .gitignore in the output.

What are these files?

  • host.json holds app-level configuration for the Functions runtime, including extension bundle versions.
  • local.settings.json stores connection strings and environment variables for local development. This file never gets published to Azure.
  • package.json tracks your npm dependencies and defines the entry point for your function code.
Update local settings and add an HTTP trigger

The scaffolded local.settings.json needs a small update before you add your first function. You'll configure the local storage emulator and add a placeholder for a database connection you'll wire up in a later step.

  • Open the doc-processing-pipeline project in your editor. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code (or Cursor) and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the doc-processing-pipeline folder on your Desktop, then click Open.
  • Open local.settings.json in your editor and replace its entire contents with the following:
{
    "IsEncrypted": false,
    "Values": {
        "AzureWebJobsStorage": "UseDevelopmentStorage=true",
        "FUNCTIONS_WORKER_RUNTIME": "node",
        "CosmosDBConnectionString": "your-cosmos-db-connection-string-here"
    }
}

What do these settings do?

  • AzureWebJobsStorage set to UseDevelopmentStorage=true tells the runtime to use the local Azurite storage emulator instead of a live Azure Storage account.
  • FUNCTIONS_WORKER_RUNTIME set to node tells the runtime this is a Node.js project.
  • CosmosDBConnectionString is a placeholder. You'll replace it with a real connection string when you set up Cosmos DB in Step 4.
  • Save local.settings.json.

Now you'll add an HTTP-triggered function to the project. This is the function you'll call from your browser to confirm the local dev loop works.

  • Return to your terminal (still inside the doc-processing-pipeline folder) and add an HTTP trigger function by running this command:
func new --name HttpExample --template "HTTP trigger" --authlevel "anonymous"

You'll see output confirming the function was created under src/functions/httpExample.js.

What does this command do?

  • --name HttpExample sets the function's name to HttpExample.
  • --template "HTTP trigger" tells Core Tools to use the HTTP trigger template, which creates a function that responds to web requests.
  • --authlevel "anonymous" means anyone can call the endpoint without an API key. This makes local testing simple.
  • Open src/functions/httpExample.js in your editor. Here's the code that was generated:
const { app } = require('@azure/functions');

app.http('HttpExample', {
    methods: ['GET', 'POST'],
    authLevel: 'anonymous',
    handler: async (request, context) => {
        context.log(`Http function processed request for url "${request.url}"`);

        const name = request.query.get('name') || (await request.text()) || 'world';

        return { body: `Hello, ${name}!` };
    },
});

What does this function do?

This is the v4 programming model in action. Instead of a separate config file, the function registers itself directly in code using app.http().

  • It accepts both GET and POST requests at the /api/HttpExample endpoint.
  • It reads a name value from the query string or request body. If neither is provided, it defaults to world.
  • It returns a greeting like Hello, Azure! in the response body.

Don't see httpExample.js?

Make sure you ran func new from inside the doc-processing-pipeline folder, not from your Desktop. The command creates the file relative to the current directory.

Help me find my httpExample.js file.

Run the function locally and test it

Everything is in place. Time to start the local Functions runtime and see your function respond to a real HTTP request.

  • Install the project dependencies by running:
npm install

You should see output confirming that packages were installed successfully.

  • Start the local Azure Functions runtime by running:
func start

What does func start do?

This command starts a local Functions host that mimics the Azure runtime on your machine. It loads all registered functions and prints their HTTP endpoints to the terminal.

The command takes over the terminal while it runs. You'll need to keep it running to test the function.

In the terminal output, look for a line that shows the HTTP endpoint URL. It looks something like this:

HttpExample: [GET,POST] http://localhost:7071/api/HttpExample

Before you test the endpoint, think about what you expect to see. If you open http://localhost:7071/api/HttpExample?name=Azure in a browser, what do you think the response will be?

  • Open your web browser and paste this URL into the address bar:
http://localhost:7071/api/HttpExample?name=Azure

You should see Hello, Azure! displayed in your browser. That's your function responding to a live HTTP request, running entirely on your machine.

Seeing an error or no response?

  • Check the terminal where func start is running. If it shows an error about a missing storage emulator, make sure AzureWebJobsStorage in local.settings.json is set to UseDevelopmentStorage=true.
  • If port 7071 is in use, func start will fail. Close any other process using that port and try again.
  • Make sure you ran npm install before func start. Missing dependencies cause the runtime to fail on startup.

Help me troubleshoot why my function isn't responding.

✔️ Awesome, I've got everything!

Great. Double check that you've saved all your files, then move on.

ⓧ I'd like to double check the full code

Here are the key files you should have in your project. Compare them with what's in your editor.

{
    "IsEncrypted": false,
    "Values": {
        "AzureWebJobsStorage": "UseDevelopmentStorage=true",
        "FUNCTIONS_WORKER_RUNTIME": "node",
        "CosmosDBConnectionString": "your-cosmos-db-connection-string-here"
    }
}
const { app } = require('@azure/functions');

app.http('HttpExample', {
    methods: ['GET', 'POST'],
    authLevel: 'anonymous',
    handler: async (request, context) => {
        context.log(`Http function processed request for url "${request.url}"`);

        const name = request.query.get('name') || (await request.text()) || 'world';

        return { body: `Hello, ${name}!` };
    },
});
{
    "version": "2.0",
    "logging": {
        "applicationInsights": {
            "samplingSettings": {
                "isEnabled": true,
                "excludedTypes": "Request"
            }
        }
    },
    "extensionBundle": {
        "id": "Microsoft.Azure.Functions.ExtensionBundle",
        "version": "[4.*, 5.0.0)"
    }
}

Your first Azure Function is running locally and responding to requests. Next up, you'll add a Blob Storage trigger so your pipeline can react automatically when files are uploaded.

Wire Up Blob Storage and Watch It Fail

In the previous step, you got your first Azure Function running locally and saw it respond to HTTP requests. That confirmed your development loop works.

Now the real pipeline begins. You want your function to react automatically when a file is uploaded to Azure Blob Storage. In this step, you will add a Blob-triggered function that fires whenever a file lands in a storage container. But there is a catch. The function can log metadata about the file, but it has nowhere to send that data. It logs and forgets.

In this step, get ready to:
  • Add a Blob-triggered function that monitors a storage container for file uploads.
  • Set up Azurite for local storage emulation so you can test blob triggers without a live Azure account.
  • Upload a test file and see the function fire, then discover the gap when the data disappears.
Create the Blob-triggered function

A Blob Storage trigger tells Azure Functions to run your code every time a new file appears in a specific container. You configure the container name and path in the function registration.

  • Create a new file called processBlobUpload.js inside the src/functions folder.
  • Add the following code to processBlobUpload.js:
const { app } = require('@azure/functions');

app.storageBlob('processBlobUpload', {
    path: 'documents/{name}',
    connection: 'AzureWebJobsStorage',
    handler: async (blob, context) => {
        // Extract file metadata from the trigger
        const fileName = context.triggerMetadata.name;
        const blobSize = blob.length;
        const timestamp = new Date().toISOString();

        context.log(`Processing blob: "${fileName}", Size: ${blobSize} bytes`);
    },
});

What does this code do?

app.storageBlob() registers a function that fires whenever a new blob appears in the documents container. The {name} in the path is a binding expression that captures the file name.

Inside the handler, context.triggerMetadata.name gives you the uploaded file's name. blob.length gives you the file size in bytes. The function logs both values to the console.

  • Save src/functions/processBlobUpload.js.

File not saving?

Make sure you created the file inside the src/functions folder, not in the project root. The Azure Functions runtime only discovers functions in the path defined by package.json's main field.

help me figure out why my function file isn't being detected

Install and start Azurite for local storage emulation

Blob triggers need a storage service to monitor. Instead of connecting to a live Azure Storage account, you can use Azurite, a local emulator that simulates Azure Blob, Queue, and Table storage on your machine.

Why use Azurite instead of a live storage account?

Azurite lets you develop and test storage triggers entirely offline. Your local.settings.json already has AzureWebJobsStorage set to UseDevelopmentStorage=true, which tells the Functions runtime to route all storage operations through Azurite.

  • Install Azurite globally by running this command:
npm install -g azurite
  • Confirm Azurite installed by running:
azurite --version

You should see a version number in the output.

Command not found?

If azurite is not recognized, your global npm bin directory may not be on your system PATH. Try closing and reopening your terminal, then run the command again.

help me fix Azurite not being found after npm install

Azurite needs its own terminal window because it runs as a background service. You will need two terminals open: one for Azurite and one for func start.

  • Open a second terminal window.
  • Start Azurite in the new terminal by running:
azurite

You should see output showing the Blob, Queue, and Table services starting on their default ports.

  • Leave this terminal running and switch back to your original terminal.
Upload a test file and watch the function fire

With Azurite running, you can start the Functions host and upload a file to trigger the Blob function. First, you need to create the documents container in Azurite and then upload a file into it.

  • In your original terminal (not the Azurite terminal), create a small test file by running:
echo "Hello from my test document" > test-doc.txt
  • Start the Functions host by running:
func start

What should I see?

The terminal should show the Functions host starting up. You will see both HttpExample and processBlobUpload listed as registered functions.

  • Open a third terminal window (Azurite stays in the second one, func start stays in the original one).
  • Navigate to your project folder in this new terminal.
  • Create the documents container in Azurite by running:
az storage container create --name documents --connection-string "UseDevelopmentStorage=true"
  • Upload the test file to the documents container by running:
az storage blob upload --container-name documents --name test-doc.txt --file test-doc.txt --connection-string "UseDevelopmentStorage=true"

Before you check the terminal running func start, what do you think you will see there? Will the Blob function have fired?

  • Switch to the terminal running func start.

You should see a log line like Processing blob: "test-doc.txt", Size: 28 bytes. The Blob trigger detected your upload and fired the function automatically.

Don't see the log line?

Blob triggers in the Consumption plan use a polling mechanism, so there can be a short delay (up to a few seconds) before the function fires. Wait 10-15 seconds and check the terminal again.

If the function still does not fire, make sure Azurite is still running in its terminal. Also check that you created the container named documents (matching the path in your function code).

help me figure out why my Blob trigger is not firing

The function logged the file name and size. But now ask yourself: where did that metadata go? Nowhere. It was logged to the console and then forgotten. If you restart func start, every trace of that metadata is gone.

Why is the data lost?

The function has no output binding. It reads the blob, extracts metadata, logs it, and exits. There is no database, no file, no queue receiving the output. Every invocation produces useful data that immediately vanishes.

This is exactly the gap the next step fills. You will add a Cosmos DB output binding so every upload automatically writes a persistent document.

✔️ Awesome, I've got everything!

Great. Double check you have saved processBlobUpload.js and that your function fired on upload.

ⓧ I'd like to double check the full code

Here is the complete code for src/functions/processBlobUpload.js:

const { app } = require('@azure/functions');

app.storageBlob('processBlobUpload', {
    path: 'documents/{name}',
    connection: 'AzureWebJobsStorage',
    handler: async (blob, context) => {
        const fileName = context.triggerMetadata.name;
        const blobSize = blob.length;
        const timestamp = new Date().toISOString();

        context.log(`Processing blob: "${fileName}", Size: ${blobSize} bytes`);
    },
});

Your pipeline can now detect file uploads and extract metadata. In the next step, you will give that metadata a permanent home by connecting Cosmos DB as an output binding.

Connect Cosmos DB and Complete the Pipeline

Your Blob-triggered function fires every time a file lands in the documents container. It logs the file name and size, but that data vanishes when the invocation ends.

In this step, you will create an Azure Cosmos DB account and wire it to your function as a persistent output. Every upload will automatically write a structured metadata document to the database.

In this step, get ready to:
  • Create a Cosmos DB account, database, and container via the Azure CLI.
  • Add a Cosmos DB output binding to the Blob-triggered function.
  • Upload a test file and verify the metadata document appears in Cosmos DB.
Create the Cosmos DB account, database, and container

Your function needs a database to write metadata into. You will create a Cosmos DB account using the free tier, then add a database and a container inside it.

  • Create a new Cosmos DB account with the free tier enabled by running this command:
az cosmosdb create \
    --name doc-pipeline-cosmos \
    --resource-group doc-pipeline-rg \
    --enable-free-tier true \
    --default-consistency-level Session \
    --locations regionName=eastus

What does this command do?

This creates a Cosmos DB account named doc-pipeline-cosmos in your existing resource group. The --enable-free-tier true flag activates the lifetime free tier, which provides 1,000 RU/s and 25 GB of storage at no cost.

The account name must be globally unique and lowercase. If doc-pipeline-cosmos is taken, append a few random digits (e.g., doc-pipeline-cosmos-42).

This command takes a few minutes

Cosmos DB account provisioning typically takes 3 to 5 minutes. Wait for the command to return before continuing.

  • Once the account is ready, create a SQL database inside it by running:
az cosmosdb sql database create \
    --account-name doc-pipeline-cosmos \
    --name file-metadata-db \
    --resource-group doc-pipeline-rg
  • Create a container inside the database by running:
az cosmosdb sql container create \
    --account-name doc-pipeline-cosmos \
    --database-name file-metadata-db \
    --name metadata \
    --partition-key-path /id \
    --resource-group doc-pipeline-rg

What is a partition key?

The --partition-key-path /id tells Cosmos DB how to distribute documents across physical partitions. Using /id means each document's unique id field determines which partition it lands in.

  • Confirm all three resources exist by running:
az cosmosdb sql container show \
    --account-name doc-pipeline-cosmos \
    --database-name file-metadata-db \
    --name metadata \
    --resource-group doc-pipeline-rg \
    --query "resource.id" \
    --output tsv

You should see metadata printed in the terminal. That confirms the account, database, and container all exist.

Seeing a "not found" error?

Double check that the account name matches exactly what you used in the az cosmosdb create command. If you appended random digits, use that same name in every subsequent command.

help me debug my Cosmos DB CLI commands

Get the connection string and update local settings

Your function needs a connection string to authenticate with Cosmos DB. You will retrieve it from Azure and paste it into your local settings file.

  • Retrieve the Cosmos DB connection strings by running:
az cosmosdb keys list \
    --name doc-pipeline-cosmos \
    --resource-group doc-pipeline-rg \
    --type connection-strings

The command outputs a JSON object with several connection strings. Copy the value of the Primary SQL Connection String. It starts with AccountEndpoint=https://....

  • Open local.settings.json in your editor.
  • Find the line that says "CosmosDBConnectionString": "your-cosmos-db-connection-string-here".
  • Replace your-cosmos-db-connection-string-here with the connection string you just copied.
  • Save local.settings.json.

Why not use a variable for the connection string?

Connection strings are sensitive credentials. You paste this value directly into local.settings.json, which is gitignored by default in Azure Functions projects. Never commit connection strings to version control.

Add the Cosmos DB output binding to your function

With the connection string in place, you can wire your function to write documents into Cosmos DB. The @azure/functions v4 SDK provides an output.cosmosDB() helper that handles the connection for you.

  • Open src/functions/processBlobUpload.js in your editor.
  • Replace the entire file contents with the following code:
const { app, output } = require('@azure/functions');

// Define the Cosmos DB output binding
const cosmosOutput = output.cosmosDB({
    databaseName: 'file-metadata-db',
    containerName: 'metadata',
    createIfNotExists: true,
    connection: 'CosmosDBConnectionString',
});

What does this code do?

The output.cosmosDB() call configures a binding that sends documents to your metadata container inside the file-metadata-db database.

The connection property points to the CosmosDBConnectionString key in local.settings.json. The createIfNotExists: true flag is a development convenience that auto-creates the database and container if they are missing.

  • Below the binding definition in the same file, add the function registration with the Cosmos DB output wired in:
app.storageBlob('processBlobUpload', {
    path: 'documents/{name}',
    connection: 'AzureWebJobsStorage',
    extraOutputs: [cosmosOutput],
    handler: async (blob, context) => {
        const fileName = context.triggerMetadata.name;
        const blobSize = blob.length;
        const timestamp = new Date().toISOString();

        context.log(`Processing blob: "${fileName}", Size: ${blobSize} bytes`);

        // Build the metadata document
        const metadataDoc = {
            id: timestamp + Math.random().toString(36).substring(2, 10),
            fileName: fileName,
            fileSize: blobSize,
            processedAt: timestamp,
            status: 'processed',
        };

        // Write the document to Cosmos DB
        context.extraOutputs.set(cosmosOutput, metadataDoc);

        context.log(`Metadata written to Cosmos DB for: ${fileName}`);
    },
});

What changed from the previous version?

The import now includes output alongside app. The function registration gains an extraOutputs: [cosmosOutput] array that tells Azure Functions about the Cosmos DB binding.

Inside the handler, metadataDoc is a JavaScript object with a unique id, the file name, size, timestamp, and a status field. The call to context.extraOutputs.set(cosmosOutput, metadataDoc) sends it to Cosmos DB automatically. No SDK client setup needed.

  • Save src/functions/processBlobUpload.js.

✔️ Awesome, I've got everything!

Great. Double check you have saved the file before moving on.

ⓧ I'd like to double check the full code

Here is the complete src/functions/processBlobUpload.js file. Make sure your code matches this exactly:

const { app, output } = require('@azure/functions');

const cosmosOutput = output.cosmosDB({
    databaseName: 'file-metadata-db',
    containerName: 'metadata',
    createIfNotExists: true,
    connection: 'CosmosDBConnectionString',
});

app.storageBlob('processBlobUpload', {
    path: 'documents/{name}',
    connection: 'AzureWebJobsStorage',
    extraOutputs: [cosmosOutput],
    handler: async (blob, context) => {
        const fileName = context.triggerMetadata.name;
        const blobSize = blob.length;
        const timestamp = new Date().toISOString();

        context.log(`Processing blob: "${fileName}", Size: ${blobSize} bytes`);

        const metadataDoc = {
            id: timestamp + Math.random().toString(36).substring(2, 10),
            fileName: fileName,
            fileSize: blobSize,
            processedAt: timestamp,
            status: 'processed',
        };

        context.extraOutputs.set(cosmosOutput, metadataDoc);

        context.log(`Metadata written to Cosmos DB for: ${fileName}`);
    },
});
Upload a test file and verify the document in Cosmos DB

The pipeline is wired. Time to prove it works end to end by uploading a file and checking that its metadata appears in the database.

  • Make sure Azurite is running. If you stopped it earlier, start it again.
  • Start the function host by running:
func start
  • Open a second terminal window (keep func start running in the first).
  • Create a small test file by running:
echo "test document content" > test-doc.txt
  • Upload the test file to the documents container in Azurite. You can do this through the VS Code Azure Storage extension: right-click the documents container under Local Emulator and select Upload Files..., then choose test-doc.txt.

Before you look at the terminal, what do you expect to see? The function should fire and log two messages: the blob name/size and a confirmation that metadata was written to Cosmos DB.

Switch back to the terminal running func start. You should see log lines that include the file name, the byte size, and Metadata written to Cosmos DB for: test-doc.txt.

Function didn't fire?

The Blob Storage trigger in the Consumption plan uses polling, so there can be a delay of up to 60 seconds. Wait a moment and check the terminal again.

If nothing appears after a minute, confirm Azurite is running and that the documents container exists. Also check that AzureWebJobsStorage is set to UseDevelopmentStorage=true in local.settings.json.

help me debug my blob trigger

Now verify the document landed in Cosmos DB.

  • Open the Azure Portal at portal.azure.com.
  • Navigate to your Cosmos DB account (doc-pipeline-cosmos).
  • In the left menu, select Data Explorer.
  • Expand file-metadata-db, then metadata, then select Items.

You should see a document with fields for fileName, fileSize, processedAt, and status. The fileName should match the file you uploaded.

Don't see your document in Data Explorer?

Check the func start terminal for error messages. A common issue is an incorrect CosmosDBConnectionString in local.settings.json. Make sure the entire connection string is pasted as a single line with no extra spaces.

help me debug my Cosmos DB output binding

Your pipeline is complete. Every file uploaded to Blob Storage now automatically catalogs its metadata in Cosmos DB. Next up, you will add an HTTP API to read those documents back out.

Add an HTTP API to Read Processed Documents

Your pipeline is fully wired. Files uploaded to Blob Storage get cataloged in Cosmos DB automatically, and you can confirm each record in Data Explorer.

The problem is that Data Explorer is a manual tool for developers. There is no way for another application, script, or teammate to query your processed documents programmatically. In this step, you will add an HTTP-triggered function with a Cosmos DB input binding that reads all metadata records and returns them as JSON. Then you will deploy the entire Function App to Azure so the endpoint is live.

In this step, get ready to:
  • Add an HTTP-triggered function that reads documents from Cosmos DB.
  • Return the documents as a JSON array in the HTTP response.
  • Deploy the Function App to Azure and test the live endpoint.
Create the getDocuments function file

The Cosmos DB input binding lets an Azure Function pull data from a Cosmos DB container without writing any SDK boilerplate. You declare a SQL query and the binding hands you the results as a JavaScript array.

  • Create a new file called getDocuments.js inside the src/functions folder.
  • Add the imports and Cosmos DB input binding configuration by pasting this code:
const { app, input } = require('@azure/functions');

// Define the Cosmos DB input binding
// Reads all documents from the metadata container, newest first
const cosmosInput = input.cosmosDB({
    databaseName: 'file-metadata-db',
    containerName: 'metadata',
    sqlQuery: 'SELECT * FROM c ORDER BY c.processedAt DESC',
    connection: 'CosmosDBConnectionString',
});

What does this code do?

The input.cosmosDB() function creates an input binding that automatically queries Cosmos DB when the function runs. The sqlQuery property tells it exactly which documents to fetch and in what order.

The connection property points to the CosmosDBConnectionString setting you already have in local.settings.json. The binding reuses that same connection to read data.

  • Now add the HTTP handler that returns the documents. Paste this code below the binding configuration in the same file:
app.http('getDocuments', {
    methods: ['GET'],
    authLevel: 'anonymous',
    extraInputs: [cosmosInput],
    handler: async (request, context) => {
        // Retrieve all documents from the Cosmos DB input binding
        const documents = context.extraInputs.get(cosmosInput);

        return {
            jsonBody: documents,
        };
    },
});

What does this handler do?

The extraInputs: [cosmosInput] line wires the Cosmos DB binding into this HTTP function. When a GET request arrives, the runtime executes the SQL query automatically and loads the results into context.extraInputs.

The jsonBody property tells Azure Functions to serialize the documents array as JSON and set the correct Content-Type header automatically.

  • Save src/functions/getDocuments.js.

✔️ Awesome, I've got everything!

Great. Double check you have saved the file before moving on.

ⓧ I'd like to double check the full code

const { app, input } = require('@azure/functions');

const cosmosInput = input.cosmosDB({
    databaseName: 'file-metadata-db',
    containerName: 'metadata',
    sqlQuery: 'SELECT * FROM c ORDER BY c.processedAt DESC',
    connection: 'CosmosDBConnectionString',
});

app.http('getDocuments', {
    methods: ['GET'],
    authLevel: 'anonymous',
    extraInputs: [cosmosInput],
    handler: async (request, context) => {
        const documents = context.extraInputs.get(cosmosInput);

        return {
            jsonBody: documents,
        };
    },
});
Test the API locally

Before deploying anything to Azure, confirm the new endpoint works on your machine. This catches binding misconfigurations early.

  • Start the local Functions host by running:
func start

In the terminal output, you should now see three functions listed instead of two. Look for a line that includes getDocuments alongside HttpExample and processBlobUpload.

  • Open a new terminal tab or window.
  • Query the new endpoint by running:
curl http://localhost:7071/api/getDocuments

Before you check the response, think about what you expect to see. You uploaded test files in the previous step and those triggered metadata writes to Cosmos DB. What format should the response be in?

You should see a JSON array containing the metadata documents you created earlier. Each document includes fileName, fileSize, processedAt, and status fields.

Getting an empty array or an error?

  • If you see [] (an empty array), no documents exist in Cosmos DB yet. Go back to the previous step and upload a test file to the documents container, then try again.
  • If you see a connection error, confirm that CosmosDBConnectionString in local.settings.json is set to your actual Cosmos DB connection string, not a placeholder.
  • If getDocuments does not appear in the function list, make sure the file is saved at src/functions/getDocuments.js (not in the project root).

Still stuck? Help me debug my getDocuments function that returns data from Cosmos DB.

Deploy the Function App to Azure

The local version works. Now you need to create an Azure Function App resource and push your code to it so the endpoint is accessible from anywhere.

  • Stop the local Functions host by pressing Ctrl+C in the terminal where func start is running.
  • Create a storage account for the Function App by running:
az storage account create --name [[STORAGE_ACCOUNT_NAME="docpipelinestor (append a unique suffix)"]] --location eastus --resource-group doc-pipeline-rg --sku Standard_LRS

Why a new storage account?

Every Azure Function App requires its own storage account to manage triggers, logging, and internal state. This is separate from the Blob Storage container you used for file uploads. The name must be globally unique and contain only lowercase letters and numbers.

  • Create the Function App on the Consumption plan by running:
az functionapp create --name [[FUNCTION_APP_NAME="docpipelinefunc (append a unique suffix)"]] --storage-account [[STORAGE_ACCOUNT_NAME="docpipelinestor (same name from above)"]] --consumption-plan-location eastus --resource-group doc-pipeline-rg --runtime node --runtime-version 22 --functions-version 4

What does this command do?

This creates a Function App on the Consumption plan, which means you only pay when your functions execute. The --runtime node and --runtime-version 22 flags match the Node.js version you have been using locally. The --functions-version 4 flag sets the Azure Functions runtime to v4.

Your deployed functions need to connect to Cosmos DB, just like they do locally. You need to push the CosmosDBConnectionString setting to your Azure Function App.

  • Set the Cosmos DB connection string on your Function App by running:
az functionapp config appsettings set --name [[FUNCTION_APP_NAME="docpipelinefunc (same name from above)"]] --resource-group doc-pipeline-rg --settings CosmosDBConnectionString="your-cosmos-db-connection-string-here"

Find your connection string

Use the same Cosmos DB connection string value from local.settings.json. You can copy it directly from that file.

  • Deploy your project to Azure by running:
func azure functionapp publish [[FUNCTION_APP_NAME="docpipelinefunc (same name from above)"]]

The publish command uploads your local project files and installs dependencies on the Azure side. This may take a minute or two.

Once it finishes, the terminal displays the live endpoint URLs for each function. Look for a line containing getDocuments with an https:// URL.

Publish failed or no URLs shown?

  • Make sure you are logged in to Azure. Run az login if your session has expired.
  • Confirm the Function App name matches exactly what you used in the az functionapp create command above.
  • If you see a version mismatch error, verify your local node --version matches what you specified with --runtime-version.

Still stuck? Help me troubleshoot my Azure Functions deployment with func azure functionapp publish.

Before you test the live endpoint, what do you expect the response to contain? Think about what documents are currently stored in Cosmos DB from your earlier uploads.

  • Test the live endpoint by running (replace the URL with the one from your publish output):
curl https://[[FUNCTION_APP_NAME="docpipelinefunc (same name from above)"]].azurewebsites.net/api/getDocuments

You should see the same JSON array of metadata documents you saw locally. Each document includes the file name, size, timestamp, and status of every file you uploaded to Blob Storage.

Getting an empty array or a 500 error?

  • If you see [], the Blob-triggered function may not have fired in the cloud yet. Upload a new test file to the documents container in Azure and wait a moment, then retry.
  • A 500 error usually means the CosmosDBConnectionString app setting is missing or incorrect. Verify it with:
az functionapp config appsettings list --name [[FUNCTION_APP_NAME="docpipelinefunc (same name from above)"]] --resource-group doc-pipeline-rg

Still stuck? Help me debug my deployed Azure Function that reads from Cosmos DB.

Your event-driven pipeline is live. Files uploaded to Blob Storage trigger metadata extraction, Cosmos DB stores the records, and your HTTP API serves them on demand. Next up, push your skills further with an optional Secret Mission.

Secret mission

Add Search-by-Filename Filtering

Your API returns every document in the database, every time. What if your Cosmos DB grows to hundreds of files and you only need one? Add a search-by-filename query parameter so you can filter results right from the URL.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses Azure free tier resources that cost $0 as long as you stay within the free tier limits, but cleaning up when you are done is good practice.

Cost warning

The Cosmos DB free tier provides 1,000 RU/s and 25 GB of storage for the lifetime of your account at no charge. The Azure Functions Consumption plan includes 1 million free executions per month. Storage costs for this project are negligible.

If you exceed these limits or create additional paid resources in the same resource group, charges will apply. Delete the resource group when you are finished to avoid any risk.

Resources you used:

  • Azure Resource Group (doc-pipeline-rg) containing all cloud resources below.
  • Azure Storage Account with a documents blob container for file uploads.
  • Azure Cosmos DB Account (free tier) with the file-metadata-db database and metadata container.
  • Azure Function App (Consumption plan) with three deployed functions.
  • Local project folder (doc-processing-pipeline) on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively building or want to continue experimenting with your pipeline.

  • Your Azure resources stay live inside the doc-pipeline-rg resource group. Within the free tier limits (1,000 RU/s, 25 GB for Cosmos DB and 1M executions/month for Functions), costs remain $0.
  • Your local doc-processing-pipeline folder remains on your Desktop for future edits.
  • Azure CLI and Azure Functions Core Tools remain installed on your machine for other projects.

Pause - I'll come back to this later

Serverless resources like Azure Functions and Cosmos DB only incur charges when actively used. There is no separate "pause" action for these services.

  • Your Function App on the Consumption plan costs nothing when it is not processing requests. No action needed.
  • Your Cosmos DB account uses the free tier, so stored data costs nothing as long as you stay under 25 GB.
  • Your Storage Account only charges for stored blobs. The files you uploaded for testing are negligible in size.
  • When you come back, open a terminal, navigate to your doc-processing-pipeline folder, and run func start to resume local development.

Delete - I don't want to use this again

Remove all project resources and start fresh. Deleting the Azure resource group removes every cloud resource inside it in one command.

  • Delete all Azure resources by running this command in your terminal:
az group delete --name doc-pipeline-rg --yes --no-wait

What does this command do?

This deletes the entire doc-pipeline-rg resource group, which removes the Storage Account, Cosmos DB account, and Function App all at once. The --no-wait flag lets the command return immediately while Azure processes the deletion in the background.

  • Delete your local project folder. On macOS or Linux, run:
rm -rf ~/Desktop/doc-processing-pipeline
  • On Windows, run:
rmdir /s /q %USERPROFILE%\Desktop\doc-processing-pipeline
  • Optionally, uninstall Azure Functions Core Tools by running:
brew uninstall azure-functions-core-tools@4
  • Optionally, uninstall Azure CLI by running:
brew uninstall azure-cli

Keep the CLI tools if you plan to do more Azure projects

Azure CLI and Azure Functions Core Tools are useful for many Azure projects. Only uninstall them if you are sure you will not need them again.

Nice Work!

Nice Work!

You just built and deployed a fully event-driven document processing pipeline on Azure. Files go in, metadata comes out, and you can query it all through a live HTTP endpoint you built from scratch.

You've learned how to:

  • Build an event-driven serverless pipeline where uploading a file to Azure Blob Storage automatically triggers an Azure Function that extracts file metadata and persists it to Cosmos DB using an output binding.
  • Wire together triggers and bindings across Azure services. A Blob trigger reacts to uploads, a Cosmos DB output binding writes documents, and a Cosmos DB input binding reads them back through an HTTP endpoint.
  • Deploy a complete serverless application to Azure and test it live with curl, proving the full read/write loop works end to end in the cloud.
  • Secret Mission: Added search-by-filename filtering to your HTTP API using query string parameters, returning filtered results or a 404 message when no documents match.

Ready to quiz yourself?