Cloud Resource Tag Validator: Part 3

Extend a Python CLI to validate live AWS resource tags and audit results.

Introduction

30 Second Summary

A checker can look dependable when every example is neatly prepared. Live data can hide beyond the first page inside a shape your code has never seen.

In this project, you will connect the Part 2 tag validator to live AWS inventory while preserving its original JSON mode. The finished CLI turns paginated cloud records into policy results with reviewable audit evidence.

What You'll Build

Picture running the validator over SSH to watch two live AWS Systems Manager Parameter Store parameters arrive on separate pages before the terminal reports one PASS result plus one FAIL result.

By the end of this project, you'll have:

  • A live-source command that uses temporary AWS credentials while preserving the simulated JSON source as its default.
  • A paginated AWS Resource Groups Tagging API inventory that displays two page messages before normalizing ResourceARN plus Tags into the validator's existing resource contract.
  • A dedicated JSON Lines audit log plus an aggregated JSON summary that preserve the live policy results for later review.
  • Secret Mission: Export the normalized live AWS inventory as a clean JSON portfolio artifact.

Are there any prerequisites?

You need a completed Cloud Resource Tag Validator: Part 2 workspace on your Windows computer, plus an AWS account with an IAM user. That IAM user needs SignInLocalDevelopmentAccess, ssm:PutParameter, ssm:AddTagsToResource, ssm:DeleteParameter, plus tag:GetResources.

Before We Start

Before any AWS setup, this checkpoint commits you to replacing only the simulated inventory-loading boundary from Part 2. The existing config_loader.py, validator.py, and reporter.py responsibilities remain reusable because they operate on the portable resource_id, resource_type, and tags contract.

Connect the CLI to AWS Securely

The Part 2 validator now needs permission to call AWS from its Ubuntu workspace. Without a supported command-line session, the live inventory boundary cannot reach your account.

You'll authenticate the AWS CLI with temporary credentials. You'll isolate Boto3 inside a Python virtual environment so the Part 2 workspace remains controlled.

In this step, get ready to:
  • Reconnect to the Part 2 workspace over SSH.
  • Set up an authenticated AWS CLI session.
  • Activate a Python environment containing the pinned Boto3 package.
Reconnect to the Part 2 workspace

Your existing workspace contains the portable validator from Part 2. Reconnecting to that same directory preserves the modules you already proved work.

  • In Windows PowerShell, reconnect to the Ubuntu SSH session you used in Part 2.
  • Return to ~/cloud-tag-validator in that SSH session using the navigation command from Part 2.
  • Confirm that your shell prompt shows ~/cloud-tag-validator before continuing.
  • Check the Python version by running this command:
python3 --version

What does this command check?

This asks the Python executable in your Ubuntu session to print its installed version. The project requires Python 3.12 or later for the pinned SDK.

✔️ I see version 3.12 or higher

Your Python runtime meets the project requirement. Keep this SSH session open for the remaining setup.

ⓧ I see an older version

The installed Python runtime is below the project requirement. Pause this project until the Part 2 environment uses Python 3.12 or later.

  • Return to the Python setup instructions in Part 2.
  • Use the official Python Unix guidance if you need installation details.
  • Repeat the version check after upgrading by running this command:
python3 --version

Why check again?

The second check confirms that your shell now resolves to the required Python runtime. Continue only when the reported version is 3.12 or later.

ⓧ Command not found

The Ubuntu environment cannot find Python. Part 2 must provide Python before this AWS integration can continue.

  • Return to the Python installation section in Part 2.
  • Use the official Python Unix guidance if you need installation details.
  • Repeat the version check after installing Python by running this command:
python3 --version

Why repeat the check?

The version output proves that Ubuntu can find the new Python installation. Continue when it reports Python 3.12 or later.

Install the AWS CLI and sign in

The AWS CLI creates the login-backed default profile that Boto3 uses later. Version 2.32.0 or later supports this console-login flow.

  • Check the installed AWS CLI version by running this command:
aws --version

What does this command check?

This prints the AWS CLI version available to your SSH session. The result determines whether you can continue or need the current installer.

✔️ I see version 2.32.0 or higher

Your AWS CLI supports console login. Continue to the authentication command below.

ⓧ I see an older version

The installed AWS CLI is too old for this login flow. The current installer upgrades the system-wide command.

  • Install the current AWS CLI for all users by running this command:
  • Enter your Ubuntu password if sudo requests it.
curl -fsSL https://awscli.amazonaws.com/v2/install.sh | sudo bash -s -- --system

What does this installer do?

This downloads the official AWS CLI installation script. The system option makes the command available to every user on the Ubuntu host.

  • Verify the upgraded AWS CLI by running this command:
aws --version

What confirms the upgrade?

The version output should now meet the minimum required for console login. This confirms that your shell is using the upgraded installation.

Installer did not complete?

Check that Ubuntu can reach the internet. Confirm that your account can use sudo for a system-wide installation.

If the installer still fails, help me diagnose the AWS CLI installation.

ⓧ Command not found

The AWS CLI is not installed in the Ubuntu environment. The official installer adds the current release system-wide.

  • Install the current AWS CLI for all users by running this command:
  • Enter your Ubuntu password if sudo requests it.
curl -fsSL https://awscli.amazonaws.com/v2/install.sh | sudo bash -s -- --system

What does this installer do?

This downloads the official AWS CLI installation script. The system option makes the command available to every user on the Ubuntu host.

  • Verify the new AWS CLI installation by running this command:
aws --version

What confirms the installation?

A version response proves that Ubuntu can now find the AWS CLI. The reported version should meet the console-login requirement.

Installer did not complete?

Check that Ubuntu can reach the internet. Confirm that your account can use sudo for a system-wide installation.

If the installer still fails, help me diagnose the AWS CLI installation.

The next command displays a URL that you complete in your Windows browser. The resulting credentials are temporary, so no long-term access keys enter the project.

  • Start the remote console-login flow by running this command:
aws login --remote

What does remote login do?

Remote login prepares authentication on the SSH host while moving the browser interaction to Windows. The AWS CLI stores the resulting login session in its default profile.

  • Copy the URL displayed in the Ubuntu terminal.
  • Paste the URL into the address bar of your Windows browser.
  • Complete the AWS sign-in prompts with your IAM user.
  • Return to the Ubuntu SSH session after the browser confirms authorization.

That sign-in gives your SSH session temporary AWS credentials without placing long-term keys in the workspace.

Before you run the identity check, picture the IAM identity you expect AWS to return.

  • Verify the default AWS CLI profile by running this command:
aws sts get-caller-identity

What does the identity check prove?

The identity lookup shows which IAM user or role signs the request. A successful response proves that the Ubuntu session can authenticate to AWS.

You should see identity details for the IAM principal you used during browser sign-in. Your default profile is now ready for SDK requests.

Login denied or identity unavailable?

If AWS denies console login, ask your administrator to grant the AWS managed SignInLocalDevelopmentAccess policy before trying again.

If the identity check still fails, help me troubleshoot my AWS console-login session.

The next step uses AWS Systems Manager Parameter Store to create fixtures. The AWS Resource Groups Tagging API discovers those fixtures.

  • Ask your AWS administrator to confirm that this IAM identity can use ssm:PutParameter, ssm:AddTagsToResource, ssm:DeleteParameter, and tag:GetResources in your selected commercial AWS Region.
  • Pause here if any required permission is missing.

Why confirm these permissions now?

The SSM permissions cover fixture creation, tagging, and deletion. The tagging permission allows the live inventory request to discover matching resources.

Create the virtual environment

A project-specific environment keeps the AWS SDK separate from Ubuntu's system packages. Activation also ensures that later Python commands use the pinned dependency.

  • Create the .venv environment inside ~/cloud-tag-validator by running this command:
python3 -m venv .venv

What does this command create?

This creates an isolated Python environment in the .venv directory. Packages installed there stay scoped to the validator workspace.

  • Activate the new environment by running this command:
source .venv/bin/activate

What does activation change?

Activation updates the current shell so its Python commands use the environment in .venv. The environment name at the start of your prompt confirms the change.

You should see the virtual environment name at the start of your Ubuntu shell prompt.

  • Install the pinned SDK with AWS Common Runtime support by running this command:
python -m pip install "boto3[crt]==1.43.93"

What does this installation provide?

This pins boto3[crt]==1.43.93 inside the active environment. The declared Common Runtime extra supports the AWS CLI console-login credentials used by Boto3.

The terminal should show package download and installation progress before returning to the virtual environment prompt.

Boto3 installation failed?

Confirm that the virtual environment name appears in your prompt. Check that the Ubuntu session can reach the Python package registry.

If installation still fails, help me troubleshoot the pinned Boto3 installation.

Before you test the import, consider whether the active environment should now find Boto3.

  • Verify the Boto3 import by running this command:
python -c "import boto3; print('Boto3 import ready')"

What does the import check prove?

This starts Python inside the active environment. It imports Boto3 before printing a short confirmation, so the message appears only after the import succeeds.

You should see Boto3 import ready. That's the foundation complete: the Ubuntu workspace can authenticate to AWS and load the pinned SDK.

Import did not succeed?

Confirm that the virtual environment is still active. Recheck the installation output for a dependency or network failure.

If Python still cannot import the package, help me diagnose the Boto3 import.

Your authenticated workspace is ready. Next, you'll create the tagged AWS fixtures that give the live inventory a deterministic test case.

Create and Discover Tagged Resources

Your secure AWS connection is ready. The validator now needs a predictable pair of live resources where one passes the production policy while the other fails it.

Two Standard parameters in AWS Systems Manager Parameter Store give you that deterministic pair. Their tags mirror the production policy from Part 2.

Boto3 creates the fixtures. The Resource Groups Tagging API returns their tag records through a paginator.

Why use the Resource Groups Tagging API?

An SSM-specific listing would couple discovery to Parameter Store. It would also require a separate tag lookup.

The GetResources operation provides one tag-shaped response across supported resource types. That boundary can support other AWS resources later.

In this step, get ready to:
  • Build a reusable fixture helper.
  • Create two tagged Standard parameters.
  • Print raw tag mappings across paginator pages.
Build the fixture helper

A fixture helper gives every live test the same starting resources. It also provides a cleanup route for the end of the project.

  • Create fixtures.py inside ~/cloud-tag-validator with the terminal editor you used in Part 2.
  • Paste the imports plus fixture definitions at the top of fixtures.py:
import argparse
import sys

import boto3
from botocore.exceptions import BotoCoreError, ClientError


PARAMETERS = [
    {
        "name": "/nextwork/tag-validator/compliant",
        "tags": {
            "Project": "nextwork-tag-validator",
            "Environment": "prod",
            "Owner": "platform-team",
            "CostCenter": "CC-100",
        },
    },
    {
        "name": "/nextwork/tag-validator/noncompliant",
        "tags": {
            "Project": "nextwork-tag-validator",
            "Environment": "production",
        },
    },
]

What does this setup define?

  • The exception imports let the helper handle SDK failures without exposing a Python traceback.
  • The first entry contains every tag required by the production policy.
  • The second entry omits Owner plus CostCenter. Its Environment value is outside the allowed list.
  • Save fixtures.py.
  • Review the PARAMETERS list in your editor.

You should see two parameter definitions. The compliant entry has four tags while the non-compliant entry has two.

  • Add create_fixtures() below the PARAMETERS list by pasting this code:
def create_fixtures():
    client = boto3.client("ssm")
    for parameter in PARAMETERS:
        client.put_parameter(
            Name=parameter["name"],
            Description="NextWork cloud tag validator fixture",
            Value="demo",
            Type="String",
            Tier="Standard",
            Tags=[
                {"Key": key, "Value": value}
                for key, value in parameter["tags"].items()
            ],
        )
        print(f"Created {parameter['name']}")

How does fixture creation work?

  • The SSM client sends each parameter definition to Parameter Store.
  • The tag comprehension converts each tag dictionary into the key-value objects expected by AWS.
  • The confirmation line identifies each parameter after AWS creates it.
  • Save fixtures.py.
  • Review the create_fixtures() function in your editor.

The creation route now requests Standard parameters. Each request includes its complete tag set.

  • Add delete_fixtures() below create_fixtures() by pasting this code:
def delete_fixtures():
    client = boto3.client("ssm")
    for parameter in PARAMETERS:
        try:
            client.delete_parameter(Name=parameter["name"])
            print(f"Deleted {parameter['name']}")
        except client.exceptions.ParameterNotFound:
            print(f"Already absent: {parameter['name']}")

How does cleanup stay reusable?

  • The function uses the same PARAMETERS list as the creation route.
  • A missing parameter produces an Already absent message. That makes repeated cleanup safe.
  • Save fixtures.py.
  • Review the exception path inside delete_fixtures().

The helper can now remove either fixture without failing when the parameter is already absent.

  • Add the command routing below delete_fixtures() by pasting this code:
def main():
    parser = argparse.ArgumentParser(description="Manage AWS validator fixtures")
    parser.add_argument("action", choices=("create", "delete"))
    args = parser.parse_args()

    try:
        if args.action == "create":
            create_fixtures()
        else:
            delete_fixtures()
    except (BotoCoreError, ClientError) as error:
        print(f"AWS fixture operation failed: {error}", file=sys.stderr)
        return 2

    return 0


if __name__ == "__main__":
    raise SystemExit(main())

How does the helper choose an action?

  • The positional action argument accepts create or delete.
  • The error handler catches SDK failures. It returns status 2 so automation can detect an unsuccessful AWS operation.
  • The final guard runs main() when you execute the file directly.
  • Save fixtures.py.
  • Compare your completed helper with the reference below.

✔️ Awesome, I've got everything!

Your fixture helper is ready. Make sure fixtures.py is saved before creating the parameters.

ⓧ I'd like to double check the full code

import argparse
import sys

import boto3
from botocore.exceptions import BotoCoreError, ClientError


PARAMETERS = [
    {
        "name": "/nextwork/tag-validator/compliant",
        "tags": {
            "Project": "nextwork-tag-validator",
            "Environment": "prod",
            "Owner": "platform-team",
            "CostCenter": "CC-100",
        },
    },
    {
        "name": "/nextwork/tag-validator/noncompliant",
        "tags": {
            "Project": "nextwork-tag-validator",
            "Environment": "production",
        },
    },
]


def create_fixtures():
    client = boto3.client("ssm")
    for parameter in PARAMETERS:
        client.put_parameter(
            Name=parameter["name"],
            Description="NextWork cloud tag validator fixture",
            Value="demo",
            Type="String",
            Tier="Standard",
            Tags=[
                {"Key": key, "Value": value}
                for key, value in parameter["tags"].items()
            ],
        )
        print(f"Created {parameter['name']}")


def delete_fixtures():
    client = boto3.client("ssm")
    for parameter in PARAMETERS:
        try:
            client.delete_parameter(Name=parameter["name"])
            print(f"Deleted {parameter['name']}")
        except client.exceptions.ParameterNotFound:
            print(f"Already absent: {parameter['name']}")


def main():
    parser = argparse.ArgumentParser(description="Manage AWS validator fixtures")
    parser.add_argument("action", choices=("create", "delete"))
    args = parser.parse_args()

    try:
        if args.action == "create":
            create_fixtures()
        else:
            delete_fixtures()
    except (BotoCoreError, ClientError) as error:
        print(f"AWS fixture operation failed: {error}", file=sys.stderr)
        return 2

    return 0


if __name__ == "__main__":
    raise SystemExit(main())

How to compare this file

Check the order of the four sections. Confirm that every parameter name plus tag value matches this reference exactly.

Create the tagged parameters

Creating cloud resources can feel risky. These two Standard parameters have no additional storage charge under the documented Standard tier.

  • Create both tagged parameters by running this command:
python fixtures.py create

What does this command do?

The create action sends both entries from PARAMETERS to AWS. Each successful request prints the parameter name.

You should see Created /nextwork/tag-validator/compliant followed by Created /nextwork/tag-validator/noncompliant.

That is the fixture pair ready. Your live inventory now has one policy pass plus one deliberate policy failure.

Seeing an access error?

An AccessDenied response means the current IAM identity cannot complete the fixture operation.

  • Stop before continuing.
  • Ask your administrator to grant ssm:PutParameter for the project parameter path.
  • Ask your administrator to grant ssm:AddTagsToResource for the project parameter path.
  • Rerun the creation command above after the permissions are updated.

Help me diagnose my Parameter Store permission error.

Inspect raw paginator pages

The validator eventually needs a portable resource schema. First you need to see the raw AWS response that the adapter must transform.

A page size of 1 makes pagination visible by placing one mapping on each page. This is a teaching setting for the two-resource demo.

  • Create aws_inventory.py inside ~/cloud-tag-validator with the terminal editor you used in Part 2.
  • Paste the imports plus inventory setup at the top of aws_inventory.py:
import json

import boto3


def load_aws_resources(
    resource_type="ssm:parameter",
    tag_key="Project",
    tag_value="nextwork-tag-validator",
    page_size=1,
):
    client = boto3.client("resourcegroupstaggingapi")
    paginator = client.get_paginator("get_resources")
    resources = []

What does the inventory setup define?

  • The defaults select SSM parameters tagged with Project=nextwork-tag-validator.
  • The API client creates a paginator for get_resources.
  • The resources list collects mappings from every page.
  • Save aws_inventory.py.
  • Review the four default filter values in load_aws_resources().

The inventory boundary now has reusable defaults for the project tag plus the SSM parameter resource type.

  • Add the pagination loop below resources = [] by pasting this code:
    for page_number, page in enumerate(
        paginator.paginate(
            TagFilters=[{"Key": tag_key, "Values": [tag_value]}],
            ResourceTypeFilters=[resource_type],
            PaginationConfig={"PageSize": page_size},
        ),
        start=1,
    ):
        mappings = page.get("ResourceTagMappingList", [])
        print(f"AWS page {page_number}: {len(mappings)} resource(s)")
        print(json.dumps(mappings, indent=2))
        resources.extend(mappings)

    return resources

How does the paginator expose AWS data?

  • The tag filter selects resources with the project tag value.
  • The resource type filter limits results to SSM parameters.
  • The pagination setting requests one mapping per page.
  • Each page prints its raw ResourceTagMappingList before adding those mappings to the combined list.
  • Save aws_inventory.py.
  • Compare your completed inventory helper with the reference below.

✔️ Awesome, I've got everything!

Your raw inventory helper is ready. Make sure aws_inventory.py is saved before inspecting the paginator output.

ⓧ I'd like to double check the full code

import json

import boto3


def load_aws_resources(
    resource_type="ssm:parameter",
    tag_key="Project",
    tag_value="nextwork-tag-validator",
    page_size=1,
):
    client = boto3.client("resourcegroupstaggingapi")
    paginator = client.get_paginator("get_resources")
    resources = []

    for page_number, page in enumerate(
        paginator.paginate(
            TagFilters=[{"Key": tag_key, "Values": [tag_value]}],
            ResourceTypeFilters=[resource_type],
            PaginationConfig={"PageSize": page_size},
        ),
        start=1,
    ):
        mappings = page.get("ResourceTagMappingList", [])
        print(f"AWS page {page_number}: {len(mappings)} resource(s)")
        print(json.dumps(mappings, indent=2))
        resources.extend(mappings)

    return resources

How to compare this file

Confirm that the filters sit inside paginator.paginate(). Check that the raw mappings are printed before the function returns the combined list.

Before you run this, how many paginator pages do you expect from two matching resources when each page can contain one mapping?

  • Inspect the raw AWS inventory by running this command:
python aws_inventory.py

What should the inspection reveal?

The page counter exposes each paginator iteration. The indented output preserves the AWS field names so you can inspect the external schema directly.

You should see two page messages with one mapping on each page. Each raw JSON mapping contains ResourceARN plus a Tags list.

You have now proved that AWS returns both fixtures across separate pages. The response shape is ready for the validator boundary in the next step.

Is the first response empty?

AWS does not publish a propagation-time guarantee for newly created tags. An immediate query can return no matching mappings.

  • Confirm that the fixture creation command reported both parameter names.
  • Rerun the inventory inspection command above.
  • Check that your AWS CLI profile uses the same Region where the fixtures were created.

Help me troubleshoot an empty tagged-resource inventory.

Your live fixtures are discoverable through a paginated AWS API. Next, you will pass those raw mappings into the Part 2 validator to expose the schema mismatch.

Expose the AWS Schema Mismatch

You have already proved that the AWS Resource Groups Tagging API can discover both tagged parameters across separate pages. Those records now need to enter the Part 2 validation flow.

The live AWS response uses a different schema from your validator. This step routes the raw records into the unchanged validator so you can see that boundary fail for yourself.

In this step, get ready to:
  • Add source controls to the existing validation command.
  • Route JSON files or live AWS records through one loading boundary.
  • Run the live validation to expose the schema mismatch.
Add the AWS source options

The existing command always loads data/resources.json. A source option lets the same command choose between that JSON file and the live inventory.

  • Return to main.py in the editor you used for Part 2.
  • Find validate_parser.add_argument("--fail-on-issues", action="store_true") inside build_parser().
  • Add these validation arguments directly below that line:
    validate_parser.add_argument(
        "--source",
        choices=("json", "aws"),
        default="json",
    )
    validate_parser.add_argument("--resource-type", default="ssm:parameter")
    validate_parser.add_argument("--tag-key", default="Project")
    validate_parser.add_argument(
        "--tag-value",
        default="nextwork-tag-validator",
    )
    validate_parser.add_argument("--page-size", type=int, default=1)

What do these arguments control?

  • The --source option chooses the inventory provider. Its json default preserves the Part 2 workflow.
  • The resource type and tag options describe which AWS resources the live source requests.
  • The page size keeps each parameter on a separate paginator page for this demonstration.
  • Save main.py.
  • Confirm that the original JSON source still runs by using this command:
python main.py validate

What should you see?

You should see the familiar Part 2 validation totals followed by its PASS and FAIL results. This confirms that omitting --source still selects the existing JSON workflow.

Does the JSON validation stop early?

Check that every new argument remains inside build_parser(). Match the indentation shown in the code block.

If Python points to an argument line, help me check the new parser arguments in main.py.

Route both inventory sources

A resource-loading boundary gives run_validation() one place to request resources. The validator receives whatever list that boundary returns.

  • Switch to aws_inventory.py in your editor.
  • Replace its import section with this import group and error class:
import boto3
from botocore.exceptions import BotoCoreError, ClientError


class AwsInventoryError(RuntimeError):
    pass

Why define an inventory error?

The inventory boundary gives AWS request failures a project-specific error type. The raw resource mappings remain unchanged in this step.

  • Replace the existing load_aws_resources() function with this version:
def load_aws_resources(
    resource_type="ssm:parameter",
    tag_key="Project",
    tag_value="nextwork-tag-validator",
    page_size=1,
):
    try:
        client = boto3.client("resourcegroupstaggingapi")
        paginator = client.get_paginator("get_resources")
        resources = []

        for page_number, page in enumerate(
            paginator.paginate(
                TagFilters=[{"Key": tag_key, "Values": [tag_value]}],
                ResourceTypeFilters=[resource_type],
                PaginationConfig={"PageSize": page_size},
            ),
            start=1,
        ):
            mappings = page.get("ResourceTagMappingList", [])
            print(f"AWS page {page_number}: {len(mappings)} resource(s)")
            resources.extend(mappings)

        return resources
    except (BotoCoreError, ClientError) as error:
        raise AwsInventoryError(f"AWS inventory request failed: {error}") from error

What does the loader return?

The paginator still adds each raw ResourceTagMappingList item to resources. Each item therefore keeps its AWS-provided ResourceARN and Tags fields.

The exception block only gives request failures a clear boundary. It does not transform the resource schema.

  • Save aws_inventory.py.
  • Compare the complete file using the tabs below.

✔️ Awesome, I've got everything!

Great. Your AWS loader still returns raw mappings from each paginator page.

ⓧ I'd like to double check the full code

import boto3
from botocore.exceptions import BotoCoreError, ClientError


class AwsInventoryError(RuntimeError):
    pass


def load_aws_resources(
    resource_type="ssm:parameter",
    tag_key="Project",
    tag_value="nextwork-tag-validator",
    page_size=1,
):
    try:
        client = boto3.client("resourcegroupstaggingapi")
        paginator = client.get_paginator("get_resources")
        resources = []

        for page_number, page in enumerate(
            paginator.paginate(
                TagFilters=[{"Key": tag_key, "Values": [tag_value]}],
                ResourceTypeFilters=[resource_type],
                PaginationConfig={"PageSize": page_size},
            ),
            start=1,
        ):
            mappings = page.get("ResourceTagMappingList", [])
            print(f"AWS page {page_number}: {len(mappings)} resource(s)")
            resources.extend(mappings)

        return resources
    except (BotoCoreError, ClientError) as error:
        raise AwsInventoryError(f"AWS inventory request failed: {error}") from error
  • Return to main.py.
  • Add this import beside the existing project imports:
from aws_inventory import AwsInventoryError, load_aws_resources

What does this import provide?

The import makes the raw AWS loader available to main.py. It also makes the inventory-specific error available for the CLI boundary.

  • Find run_validation() in main.py.
  • Add this loading function directly above run_validation():
def load_resources(args):
    if args.source == "aws":
        return load_aws_resources(
            resource_type=args.resource_type,
            tag_key=args.tag_key,
            tag_value=args.tag_value,
            page_size=args.page_size,
        )
    return load_json(args.resources)

How does source routing work?

  • An aws source sends the filter arguments to load_aws_resources().
  • Every other accepted source follows the existing load_json() path.
  • Both paths return a list for the next validation stage.
  • Replace the existing run_validation() function with this version:
def run_validation(args):
    resources = load_resources(args)
    policy = load_json(args.policy)
    environment = get_active_environment()
    results = validate_resources(resources, policy, environment)
    print_results(results, environment)
    append_audit_event(args.log, results, environment)

    if args.fail_on_issues and any(
        not result["compliant"] for result in results
    ):
        return 1
    return 0

What changed in validation?

The first line now delegates inventory selection to load_resources(). Policy loading and validation keep their Part 2 responsibilities.

  • Save main.py.
  • Confirm that the new boundary preserves the default JSON workflow by running:
python main.py validate

What should the boundary preserve?

You should see the same Part 2 validation result again. This proves that load_resources() still selects data/resources.json when no source is supplied.

Does the default source fail now?

Check that return load_json(args.resources) sits outside the if block. Confirm that run_validation() calls load_resources(args).

If the source boundary still fails, help me debug the load_resources routing in main.py.

Reveal the raw schema mismatch

The live route now reaches the unchanged policy validator. The key check is whether an AWS mapping already contains the field names that validate_resources() reads.

Before you run this, do you think the raw AWS records will pass through the validator or stop at the schema boundary?

  • Select the production policy and validate the live AWS source by running:
export VALIDATOR_ENV=prod
python main.py validate --source aws --log output/aws-validator.log

The failure is the evidence

You should first see two AWS page messages. Python then stops with KeyError: 'resource_id'.

The raw mapping provides ResourceARN plus a list-shaped Tags value. The unchanged validator expects resource_id, resource_type, and a dictionary-shaped tags value.

This traceback proves that retrieval works. It also pinpoints the external-schema boundary that needs adaptation.

  • Read the first raw mapping printed above the traceback.
  • Compare its ResourceARN field with the validator's resource_id lookup.
  • Compare its Tags list with the validator's dictionary-shaped tags contract.

That deliberate failure is a useful result. Your CLI reaches live AWS inventory while the traceback identifies the exact boundary the next step must normalize.

Normalize and Validate AWS Inventory

The previous step proved that your live AWS request reaches the unchanged validator. The raw mappings still use fields the validator cannot read.

This step adds a dedicated data adapter at the cloud boundary. It converts every paginator page into the existing resource contract.

Targeted exception handling keeps retrieval failures from leaking into policy logic.

In this step, get ready to:
  • Normalize each paginated tag record into the existing resource contract.
  • Turn inventory failures plus empty results into clear adapter errors.
  • Validate both live fixtures through the unchanged policy modules.
Normalize each AWS mapping

The AWS Resource Groups Tagging API returns each resource through ResourceARN plus a Tags list. Your validator expects resource_id plus a dictionary-shaped tags field.

  • In aws_inventory.py, find the line containing resources.extend(mappings).
  • Replace that line with this normalization loop:
            for mapping in mappings:
                tags = {
                    tag["Key"]: tag["Value"]
                    for tag in mapping.get("Tags", [])
                }
                resources.append(
                    {
                        "resource_id": mapping["ResourceARN"],
                        "resource_type": resource_type,
                        "tags": tags,
                    }
                )

What Does This Code Do?

  • The tags comprehension converts every Key plus Value object into one dictionary entry.
  • The resources.append() call emits the exact fields expected by validate_resources().
  • The selected resource_type becomes part of every normalized record.
  • Save aws_inventory.py.
  • Check the normalized path by running this command:
python main.py validate --source aws

Why Does This Check Work?

The command routes inventory loading through load_aws_resources(). The existing validator receives the normalized records.

The default log path keeps this check separate from the dedicated live audit log used at the end of the step.

You should see two page messages. The resource results should include one PASS plus one FAIL.

Still Seeing a Schema Failure?

Check that the new for mapping in mappings: loop is indented inside the paginator loop.

Check that resources.extend(mappings) is gone.

Ask for help with the normalization loop.

Harden inventory errors

A reusable adapter also owns failures from its external dependency. Empty inventories need a clear message before they reach validation.

Client failures also need cloud-specific context. The adapter can provide that context without changing policy code.

  • In aws_inventory.py, find the return resources line near the existing exception handler.
  • Replace the function tail with this guarded return path:
        if not resources:
            raise AwsInventoryError(
                f"No AWS resources matched {tag_key}={tag_value} "
                f"for {resource_type}. Rerun after confirming the tags and Region."
            )

        return resources
    except (BotoCoreError, ClientError) as error:
        raise AwsInventoryError(
            f"AWS inventory request failed: {error}. Run aws login again "
            "or check the Region and IAM permissions."
        ) from error

How Does the Error Boundary Work?

  • The empty-result guard raises AwsInventoryError when the filters match no resources.
  • The exception handler converts BotoCoreError plus ClientError into the adapter's error type.
  • The from error clause preserves the original cause for debugging.
  • Save aws_inventory.py.
  • Confirm the successful inventory path still works by running this command:
python main.py validate --source aws

What Does This Check Prove?

The successful path still returns normalized resources after the new guard. The validator can continue using the adapter without learning about cloud errors.

You should still see two paginator page messages followed by one passing resource plus one failing resource.

Seeing an Indentation Error?

Keep the empty-result guard inside the try block. Keep the except block aligned with try.

Ask for help checking the function structure.

✔️ Awesome, I've got everything!

Your adapter now owns the complete AWS inventory boundary.

ⓧ I'd like to double check the full code

import boto3
from botocore.exceptions import BotoCoreError, ClientError


class AwsInventoryError(RuntimeError):
    pass


def load_aws_resources(
    resource_type="ssm:parameter",
    tag_key="Project",
    tag_value="nextwork-tag-validator",
    page_size=1,
):
    try:
        client = boto3.client("resourcegroupstaggingapi")
        paginator = client.get_paginator("get_resources")
        resources = []

        for page_number, page in enumerate(
            paginator.paginate(
                TagFilters=[{"Key": tag_key, "Values": [tag_value]}],
                ResourceTypeFilters=[resource_type],
                PaginationConfig={"PageSize": page_size},
            ),
            start=1,
        ):
            mappings = page.get("ResourceTagMappingList", [])
            print(f"AWS page {page_number}: {len(mappings)} resource(s)")

            for mapping in mappings:
                tags = {
                    tag["Key"]: tag["Value"]
                    for tag in mapping.get("Tags", [])
                }
                resources.append(
                    {
                        "resource_id": mapping["ResourceARN"],
                        "resource_type": resource_type,
                        "tags": tags,
                    }
                )

        if not resources:
            raise AwsInventoryError(
                f"No AWS resources matched {tag_key}={tag_value} "
                f"for {resource_type}. Rerun after confirming the tags and Region."
            )

        return resources
    except (BotoCoreError, ClientError) as error:
        raise AwsInventoryError(
            f"AWS inventory request failed: {error}. Run aws login again "
            "or check the Region and IAM permissions."
        ) from error
Catch adapter failures and verify

The adapter now reports one project-specific error type. The CLI needs one handler that turns this error into a clean terminal message.

  • At the top of main.py, add AwsInventoryError to the existing import from aws_inventory.
  • Save main.py.
  • Confirm the imported adapter error is available by running this command:
python main.py validate --source aws

What Does This Import Check Prove?

Python loads AwsInventoryError before dispatching the validation command. A normal validation result confirms the new import resolves correctly.

You should see the live validation complete without an import failure.

  • In main.py, find the existing main() function.
  • Replace the entire function with this guarded command dispatcher:
def main():
    args = build_parser().parse_args()
    try:
        if args.command == "validate":
            return run_validation(args)
        return run_analysis(args)
    except AwsInventoryError as error:
        print(error, file=sys.stderr)
        return 2

What Does This Handler Do?

  • The try block covers command dispatch plus AWS inventory loading.
  • The handler writes the adapter message to standard error.
  • Returning 2 gives the shell a predictable operational-error status without a traceback.
  • Save main.py.

Before you run the live validation, predict which fixture the production policy will reject.

  • Run the final live validation with its dedicated audit log by using this command:
python main.py validate --source aws --log output/aws-validator.log

What Does This Command Prove?

  • The aws source routes retrieval through the normalized adapter.
  • The existing policy validator evaluates both live parameters.
  • The reporter appends one JSON Lines audit event to output/aws-validator.log.

You should see two paginator page messages.

You should see Resources checked: 2.

You should see Compliant: 1.

You should see Non-compliant: 1.

One parameter ARN should show PASS.

The other parameter ARN should show FAIL.

The failing ARN should list INVALID_VALUE for Environment.

It should also list MISSING_TAG for Owner.

It should list MISSING_TAG for CostCenter.

You have closed the hard part. Live AWS records now pass through the unchanged validator.

Live Validation Did Not Complete?

If the adapter reports no matching resources, confirm that both fixtures remain in the same AWS Region.

If authentication has expired, repeat the remote login flow from Step 1. If you see a traceback, check that the new handler wraps command dispatch inside main().

Ask for help diagnosing the live validation.

✔️ Awesome, I've got everything!

Your CLI now validates live AWS inventory through a guarded adapter.

ⓧ I'd like to double check the full code

import argparse
import sys
from pathlib import Path

from aws_inventory import AwsInventoryError, load_aws_resources
from config_loader import get_active_environment, load_json
from reporter import analyze_log, append_audit_event, print_results, write_report
from validator import validate_resources


def build_parser():
    parser = argparse.ArgumentParser(
        description="Validate simulated or live AWS resource tags"
    )
    subparsers = parser.add_subparsers(dest="command", required=True)

    validate_parser = subparsers.add_parser("validate")
    validate_parser.add_argument("--resources", type=Path, default=Path("data/resources.json"))
    validate_parser.add_argument("--policy", type=Path, default=Path("config/policy.json"))
    validate_parser.add_argument("--log", type=Path, default=Path("output/validator.log"))
    validate_parser.add_argument("--fail-on-issues", action="store_true")
    validate_parser.add_argument("--source", choices=("json", "aws"), default="json")
    validate_parser.add_argument("--resource-type", default="ssm:parameter")
    validate_parser.add_argument("--tag-key", default="Project")
    validate_parser.add_argument("--tag-value", default="nextwork-tag-validator")
    validate_parser.add_argument("--page-size", type=int, default=1)

    analyze_parser = subparsers.add_parser("analyze")
    analyze_parser.add_argument("--log", type=Path, default=Path("output/validator.log"))
    analyze_parser.add_argument("--report", type=Path, default=Path("output/summary.json"))
    return parser


def load_resources(args):
    if args.source == "aws":
        return load_aws_resources(resource_type=args.resource_type, tag_key=args.tag_key, tag_value=args.tag_value, page_size=args.page_size)
    return load_json(args.resources)


def run_validation(args):
    resources = load_resources(args)
    policy = load_json(args.policy)
    environment = get_active_environment()
    results = validate_resources(resources, policy, environment)
    print_results(results, environment)
    append_audit_event(args.log, results, environment)
    if args.fail_on_issues and any(not result["compliant"] for result in results):
        return 1
    return 0


def run_analysis(args):
    summary = analyze_log(args.log)
    write_report(args.report, summary)
    print(f"Events analyzed: {summary['events_analyzed']}")
    print(f"Resources checked: {summary['resources_checked']}")
    print(f"Skipped lines: {summary['skipped_lines']}")
    for code, count in sorted(summary["issue_counts"].items()):
        print(f"{code}: {count}")
    return 0


def main():
    args = build_parser().parse_args()
    try:
        if args.command == "validate":
            return run_validation(args)
        return run_analysis(args)
    except AwsInventoryError as error:
        print(error, file=sys.stderr)
        return 2


if __name__ == "__main__":
    raise SystemExit(main())

Your live inventory path is reliable now. Next, you will turn its audit event into a reusable summary.

Produce Audit Evidence

Your live AWS inventory now moves through the existing validator. The production run proves both policy outcomes.

Terminal output only proves what happened during one session. This step turns the existing JSON Lines audit event into a formatted summary that remains useful after the terminal scrolls away.

In this step, get ready to:
  • Preserve the single successful production audit event.
  • Analyze the audit log into a machine-readable summary.
  • Capture the paginator output plus the summary counters in one screenshot.
Preserve the single audit event

Every successful validation appends another event to the audit log. Running the validation again would increase the report counters.

  • Return to the SSH terminal from the successful production validation.
  • Leave output/aws-validator.log unchanged by skipping another validation run.
Analyze the live audit log

The analyzer from Part 2 already understands the audit format. It can aggregate the live event without any changes to the reporting code.

Before you run the analyzer, predict whether the single audit event produces one analyzed event or more.

  • Create output/aws-summary.json from the existing audit log by running this command:
python main.py analyze --log output/aws-validator.log --report output/aws-summary.json

What does this command do?

  • The analyze route sends the existing audit records through analyze_log().
  • The --log option selects the dedicated live AWS audit log.
  • The --report option sends the aggregated counters to output/aws-summary.json.
  • Confirm the terminal prints Events analyzed: 1.
  • Confirm the terminal prints Resources checked: 2.
  • Confirm the terminal prints Skipped lines: 0.
  • Confirm the terminal prints INVALID_VALUE: 1.
  • Confirm the terminal prints MISSING_TAG: 2.

Summary counters look different?

If the analyzer cannot find the log, check that the --log path is output/aws-validator.log.

If Events analyzed is greater than 1, extra validation runs added records. Keep only the single successful production event before running the analyzer again.

Help me reconcile unexpected AWS audit summary counters.

Format and capture the evidence

A formatted report makes each counter easy to review. It also confirms that the generated file contains valid JSON.

Before you format the report, predict which counters will show that one live run checked both fixtures.

  • Format output/aws-summary.json by running this command:
python -m json.tool output/aws-summary.json

What does this command do?

Python's json.tool reads the report as JSON. It prints an indented version when the syntax is valid.

You will see events_analyzed as 1. You will also see resources_checked as 2.

You will see skipped_lines as 0.

The issue_counts object shows INVALID_VALUE as 1. It shows MISSING_TAG as 2.

Report not formatting?

If the report is missing, confirm that the analyzer wrote output/aws-summary.json.

If the formatter reports invalid JSON, run the analyzer again to replace the summary with output from write_report().

Help me fix a JSON summary that Python cannot format.

  • Frame one SSH terminal view with the two paginator page messages, the PASS parameter ARN, the FAIL parameter ARN, the audit-log path, and the summary counters.
  • Take a screenshot of that SSH terminal view.

Strong finish. Your screenshot now connects the live inventory run to its durable summary.

The aws_inventory.py module owns AWS retrieval plus schema normalization. The config_loader.py module keeps selecting the active environment plus policy data.

The validator.py module keeps evaluating normalized resources. The reporter.py module keeps recording audit events plus producing summaries.

Secret mission

Export the Normalized AWS Inventory

Your audit summary proves that the live validation worked. Add an optional snapshot that exports the normalized AWS inventory as two portable JSON objects for direct review.

Clean Up Your Resources

Clean Up Your Resources

The two Standard parameters use standard throughput at no additional charge. Decide whether to keep your resources, pause your environment, or delete the project.

Resources you used:

  • Two Standard AWS Systems Manager Parameter Store fixtures at /nextwork/tag-validator/compliant and /nextwork/tag-validator/noncompliant.
  • The ~/cloud-tag-validator workspace with its .venv, fixtures.py, aws_inventory.py, output/aws-validator.log, output/aws-summary.json, and output/aws-inventory.json artifacts.
  • The active SSH connection and Windows Subsystem for Linux 2 session.

Keep everything running

No action needed. Choose this if you want to demonstrate the live validator again or extend its AWS inventory features.

  • Leave both Standard parameters available for future validation runs.
  • Keep the ~/cloud-tag-validator workspace with its audit log, summary, and normalized inventory snapshot.
  • Reconnect over SSH when you want to continue working in the activated environment.

Your temporary AWS CLI console-login credentials expire automatically within the configured IAM session duration. The maximum duration is 12 hours.

Pause - I'll come back to this later

Remove the AWS fixtures to clear the live inventory. Keep the local code and evidence so you can resume later.

  • Remove both Parameter Store fixtures from the activated .venv by running:
python fixtures.py delete

What does this command do?

The delete route calls Parameter Store once for each project fixture. It also handles fixtures that are already absent.

  • Check the terminal output for Deleted /nextwork/tag-validator/compliant.
  • Check the terminal output for Deleted /nextwork/tag-validator/noncompliant.

Could Not Delete the Fixtures?

AWS may deny the operation if your IAM user lacks the required deletion permission.

  • Ask your administrator to grant ssm:DeleteParameter for the project parameter path.

Help me diagnose why fixtures.py could not delete my Parameter Store fixtures.

  • Close the SSH session by running:
exit

What does this command do?

This closes the Ubuntu SSH session. Your terminal returns to Windows PowerShell on your computer.

  • Stop Windows Subsystem for Linux 2 from Windows PowerShell by running:
wsl --shutdown

What does this command do?

This stops the running WSL environment while preserving ~/cloud-tag-validator and its evidence files.

  • Confirm that the Windows PowerShell prompt returns without an error.

Your temporary AWS CLI credentials expire automatically within the configured IAM session duration.

Delete - I don't want to use this again

Remove all project resources if you no longer need Parts 1 through 3. Deleting ~/cloud-tag-validator permanently removes the code and audit evidence inside it.

  • Remove both Parameter Store fixtures from the activated .venv by running:
python fixtures.py delete

What does this command do?

The helper deletes both parameters from the selected AWS Region. An Already absent: message also confirms that a fixture no longer exists.

  • Check the terminal output for a deletion or already-absent message for each parameter.

Could Not Remove the Parameters?

A denied deletion usually means the IAM user cannot remove parameters from the project path.

  • Ask your administrator to grant ssm:DeleteParameter before removing the local workspace.

Help me troubleshoot why the fixture deletion failed before I remove my local workspace.

  • Leave the Python virtual environment by running:
deactivate

What does this command do?

This removes .venv from the active shell session. The environment files remain inside the workspace until you delete the folder.

  • Check that (.venv) no longer appears at the start of your terminal prompt.
  • Delete the complete Ubuntu workspace by running these commands:
cd ~
rm -rf ~/cloud-tag-validator

What do these commands do?

The first command moves your shell to the Ubuntu home directory. The second permanently removes ~/cloud-tag-validator with every project file inside it.

  • Confirm that the workspace is gone by listing your Ubuntu home directory:
ls ~

What should you see?

You should no longer see cloud-tag-validator in the directory listing. This confirms that the virtual environment, code, log, summary, and snapshot were removed.

Still See the Workspace?

Check that the deletion command targeted ~/cloud-tag-validator exactly. A different path leaves the project folder untouched.

Help me confirm why the cloud-tag-validator folder still exists after the removal command.

  • Close the SSH session by running:
exit

What does this command do?

This ends the connection to Ubuntu after the cloud resources and local workspace are removed.

  • Stop Windows Subsystem for Linux 2 from Windows PowerShell by running:
wsl --shutdown

What does this command do?

This stops the remaining WSL processes. Your temporary AWS CLI credentials expire automatically within the configured IAM session duration.

  • Confirm that the Windows PowerShell prompt returns without an error.

Nice Work!

Nice Work!

You did it! Your Part 2 validator now audits live AWS resources through the same policy engine. It also preserves the original JSON input mode.

You now have durable evidence that your CLI can discover paginated cloud inventory, translate an external schema, evaluate tag compliance, and handle operational failures.

You learned how to:

  • Connect a live AWS inventory source to your existing CLI using temporary console-login credentials. You also created deterministic Parameter Store fixtures without deploying compute infrastructure.
  • Use the Resource Groups Tagging API paginator to retrieve multiple pages. You normalized each AWS record into the validator's existing schema.
  • Produce JSON Lines audit evidence from a live validation run. You turned that evidence into an aggregated JSON summary with compliance counters.
  • Secret Mission: Added an optional normalized inventory snapshot that exports live resources to output/aws-inventory.json without changing the default validation behavior.

Ready to quiz yourself?