Architect an Enterprise SRE Platform

Introduction

30 Second Summary

A resilience plan can look complete until one outage removes the services needed to recover it. Limited hardware can hide the same weakness behind a successful demo.

In this project, you will create a version-controlled production reference architecture for Nova Trek's reliability platform. You will translate that architecture into a compact rotating lab that labels each behavior as Real, Real on shared hardware, or Simulated.

What You'll Build

Your finished repository gives a design reviewer one path from an approved change through failure detection, recovery, and controlled failback.

By the end of this project, you'll have:

• A two-site resilience map you can walk through to show how independent fault domains stop failures from spreading. Its recovery paths cover SQL Server Always On availability groups, independent backups, Zabbix monitoring, identity, and deployment.
• A scenario rotation matrix that names the exact roles required for each future Hyper-V test. Every result is labelled Real, Real on shared hardware, or Simulated.
• A review-ready Git repository that lets another engineer inspect Architecture Decision Records, capacity gates, security boundaries, failure tests, and rollback evidence. Its roadmap sequences the design into focused implementation projects. Its trust model traces approved work from Azure Pipelines through PowerShell JEA or Ansible.
• Secret Mission: Tabletop a complete primary-site outage to expose hidden control-plane dependencies, recovery-order conflicts, and evidence gaps.

Are there any prerequisites?

You need Git and Visual Studio Code installed on your Windows host plus basic familiarity with Windows administration and Git. You also need access to an Azure DevOps organization, while every lab workload remains undeployed in this design-only project.

Before We Start

This is your moment to commit to what Nova Trek must prove and why a production-first design with a compact rotating lab is the right constraint before any hands-on work begins.

Set Up the Architecture Repository

Nova Trek's architecture must become a reviewable source of truth before any diagrams are added. A local Git repository records the first artifact as a traceable baseline.

Visual Studio Code keeps the design files beside the terminal that versions them. You'll create the documentation structure before committing a clean baseline.

In this step, get ready to:

• Open the nova-trek-sre-lab workspace.
• Create the README plus the documentation structure.
• Commit a clean repository baseline.

Open the workspace and confirm Git

A VS Code workspace makes one folder the context for its Explorer. The integrated terminal uses the same folder.

• Press the Windows key to open app search.
• Type Visual Studio Code.
• Press Enter to open Visual Studio Code.
• Select File in the top menu.
• Select Open Folder....
• Select your Desktop in the folder dialog.
• Select the folder creation control in the dialog.
• Enter nova-trek-sre-lab as the folder name.

Your Workspace Location

Creating the folder on your Desktop gives the repository a predictable location. Every artifact in this project stays inside nova-trek-sre-lab.

• Open the new nova-trek-sre-lab folder.
• Confirm that you trust the folder if Visual Studio Code displays a Workspace Trust prompt.
• Select Terminal in the top menu.
• Select New Terminal.
• Confirm that the installed Git executable is available by running this command:

git --version

What Does This Command Prove?

The command asks Git to report its installed version. A version number confirms that the integrated terminal can reach the existing Git installation.

You'll see a line containing a Git version number. The terminal is ready to create the repository.

Git Missing From the Terminal?

Close the integrated terminal after confirming that Git remains installed on Windows. Create a new terminal so Visual Studio Code reloads the current system path.

If Git remains unavailable, help me diagnose the Git path in Visual Studio Code.

Create the design structure

The README.md file establishes the contract that governs every later design. Its Markdown sections also define how the lab describes implementation fidelity.

• Use the PowerShell terminal from earlier to create the documentation tree plus README.md by running this code:

New-Item -ItemType "Directory" -Path "docs"
New-Item -ItemType "Directory" -Path "docs\architecture", "docs\decisions", "docs\failures", "docs\roadmap"

$readme = @'
# Nova Trek SRE Architecture

## Design Contract

Nova Trek must prove that its production database, backup, monitoring, identity, and automation planes can remain available or recover through documented fault domains, while a compact rotating lab honestly labels shared-host and simulated behavior.

## Fidelity Labels

- `Real`: The actual product behavior is exercised.
- `Real on shared hardware`: The actual software mechanism runs, but all nodes share the laptop's physical fault domain.
- `Simulated`: The event or dependency is represented through a tabletop, controlled shutdown, network rule, or mocked evidence instead of independent infrastructure.

## Source Control Rule

Secrets and generated artifacts never enter source control.
'@
Set-Content -Path "README.md" -Value $readme

What Does This Setup Create?

• The docs/architecture folder is reserved for production plus lab architecture documents.
• The docs/decisions folder is reserved for architecture decision records.
• The docs/failures folder is reserved for failure scenarios plus tabletop exercises.
• The docs/roadmap folder is reserved for the incremental implementation roadmap.

The $readme value holds the complete document before Set-Content writes it to README.md.

• Select Explorer in the left sidebar.
• Expand the docs folder.

You'll see README.md beside docs. Inside docs, you'll see the empty architecture, decisions, failures, and roadmap folders.

Structure Missing From Explorer?

Confirm that the terminal prompt points to the nova-trek-sre-lab folder. Commands run from another location create the structure there.

If the folder tree differs, help me correct my Nova Trek documentation structure.

✔️ Awesome, I've got everything!

Your README.md now records the design contract plus the three fidelity labels.

ⓧ I'd like to double check the full code

Compare your README.md with this complete reference.

# Nova Trek SRE Architecture

## Design Contract

Nova Trek must prove that its production database, backup, monitoring, identity, and automation planes can remain available or recover through documented fault domains, while a compact rotating lab honestly labels shared-host and simulated behavior.

## Fidelity Labels

- `Real`: The actual product behavior is exercised.
- `Real on shared hardware`: The actual software mechanism runs, but all nodes share the laptop's physical fault domain.
- `Simulated`: The event or dependency is represented through a tabletop, controlled shutdown, network rule, or mocked evidence instead of independent infrastructure.

## Source Control Rule

Secrets and generated artifacts never enter source control.

What Should Match?

The design contract should match the project constraint. Each fidelity label should have its own definition.

Commit a clean baseline

A baseline commit creates the first reviewable point in the repository history. Later architecture changes can be compared with this known state.

• Initialize the local repository by running this command:

git init

What Does This Command Do?

The command adds Git's repository metadata to nova-trek-sre-lab. Git can now record revisions of files inside this folder.

The terminal confirms that an empty Git repository was initialized inside nova-trek-sre-lab.

Repository Initialized Elsewhere?

Check the folder shown in the terminal prompt. It should end with nova-trek-sre-lab.

If the location is unclear, help me confirm where Git initialized my repository.

• Stage the repository files by running this command:

git add .

What Does Staging Do?

Staging selects the current repository files for the next commit. At this point, the selected artifact is README.md.

• Inspect the staged baseline by running this command:

git status

What Does This Check Prove?

Git separates staged files from pending changes. You should see README.md listed for the first commit.

The empty documentation folders remain on your computer. Git begins tracking each folder when a later step adds its first document.

You'll see README.md staged for the initial commit. This confirms that the design contract belongs to the baseline.

• Create the initial architecture commit by running this command:

git commit -m "Initialize Nova Trek SRE architecture"

What Does This Commit Record?

The command saves the staged README under the message Initialize Nova Trek SRE architecture. That message identifies the repository's starting point.

That's the repository foundation in place. The terminal confirms that the first commit includes README.md.

Commit Blocked by Author Identity?

Git may require an author identity before creating the first commit. Follow the configuration guidance printed in the terminal.

If you need help, help me configure the identity required for my first Git commit.

Before the final check, do you expect Git to find any pending files?

• Verify the committed repository state by running this command:

git status

What Does the Final Check Prove?

The command compares the files on disk with the latest commit. A clean result proves that no modified or untracked files remain.

Git reports a clean working tree with no modified or untracked files. Your Nova Trek architecture now has a reproducible starting point.

Working Tree Still Has Changes?

Read the file name reported by Git. Save that file before staging it for another commit.

If the status remains unclear, help me reach a clean Nova Trek working tree.

Your versioned architecture workspace is ready. Next, you'll document Nova Trek's production fault domains before applying the laptop constraints.

Design the Production Reference Architecture

Your repository now gives Nova Trek a reviewable source of truth. The next risk is drawing a compact laptop lab first and mistaking shared hardware for independent fault domains.

A truthful enterprise design exposes the complete high-availability and disaster-recovery model before any compression. You will map that production topology before testing whether the current host can preserve its boundaries.

In this step, get ready to:

• Map the production sites and their independent fault domains.
• Define the SQL Server availability and backup design.
• Document the operations trust boundaries and test the host capacity.

Map the production fault domains

A production reference begins with failure boundaries that can be reviewed before products are configured. A text diagram keeps those boundaries visible inside a version-controlled Markdown document.

• In Visual Studio Code, select the existing docs/architecture folder in the Explorer view.
• Select New File... in the Explorer view.
• Enter production-reference.md as the file name.
• Press Enter to create the file.
• Add the design contract and production topology to production-reference.md by pasting this block:

# Nova Trek Production Reference Architecture

> **Design contract:** Nova Trek must preserve service availability, recoverability, observability, and controlled automation across independent production fault domains.
> **Scope:** This document defines production intent only. It deploys no infrastructure.

## Production topology

<!-- This diagram exposes site and zone boundaries before implementation choices. -->
```text
CLIENTS
  |
External client-access load-balancing layer
  |
PRIMARY SITE                                  DISASTER-RECOVERY SITE
Management zone: Active Directory            Management zone: Active Directory
Application zone: redundant app nodes        Application zone: redundant app nodes
Database-replication zone: DB-01, DB-02       Database-replication zone: DB-03
Backup zone: independent SMB service         Backup zone: independent SMB service
Monitoring zone: server HA, frontends, proxies Monitoring zone: server HA, frontends, proxies
Client-access zone: redundant entry points    Client-access zone: redundant entry points
Compute: independent physical hosts          Compute: independent physical hosts
Storage: independent storage systems         Storage: independent storage systems
Network: independent switches and paths      Network: independent switches and paths

THIRD LOCATION
Independent WSFC quorum or witness service
```

What does this topology establish?

• The primary and disaster-recovery sites each contain management, application, database-replication, backup, monitoring, and client-access zones.
• The compute, storage, and network rows make physical independence an architecture requirement.
• The third location keeps the WSFC quorum or witness service outside both workload sites.

• Save production-reference.md.
• Open the side-by-side Markdown preview by pressing Ctrl+K V.

You should see two workload sites with matching zones. You should also see the third quorum location outside both sites.

Is the diagram swallowing the document?

An unclosed text fence can place the rest of the document inside the diagram. Confirm that the final line of the diagram contains three backticks.

Help me fix the topology preview.

The topology names the locations. The next table makes each service path traceable to the boundary that protects it.

• Add the component and data-flow map below the topology by pasting this block at the end of production-reference.md:

## Component, boundary, and flow map

<!-- Each row makes a service path and its failure boundary reviewable. -->
| Plane | Primary site | DR site | Independent boundary and flow |
| --- | --- | --- | --- |
| Management | Site-local Active Directory and administration services | Site-local Active Directory and recovery administration | Identity services use separate compute, storage, and network paths. Authentication stays available when one site fails. |
| Application | Redundant application nodes | Redundant recovery application nodes | Client traffic enters through the client-access layer. Application traffic reaches the database service. |
| Database replication | `DB-01` and `DB-02` | `DB-03` | Database replicas use separate hosts and storage. Replication uses its own network zone. |
| Backup | Independently available SMB backup service | Independently available SMB backup service and offsite-copy path | Database backups leave database storage. A separate copy leaves the site. |
| Monitoring | Redundant servers, frontends, and site proxy group | Redundant servers, frontends, and site proxy group | Agent data reaches the local proxy group. Proxies reach the protected monitoring service. |
| Client access | Redundant site entry points | Redundant recovery entry points | An external load-balancing layer routes clients to an available site. |
| Control plane | Isolated deployment and automation services | Isolated recovery automation services | Approved changes cross a controlled boundary before reaching managed endpoints. |
| Quorum | WSFC nodes reach the witness | WSFC nodes reach the witness | The witness lives in a third location outside both workload sites. |

How does the map expose data flow?

• Each row shows the production component represented in both workload sites.
• The final column traces traffic or data through an independent boundary.
• The control-plane row keeps deployment access separate from the workloads it manages.

• Save production-reference.md.
• Check the updated side-by-side preview.

You should see a four-column table with one row for every architecture plane. Each row should name a boundary plus the flow that crosses it.

Does the table render as plain text?

A missing pipe can stop a Markdown table from rendering. Compare the number of pipe characters across the header row and its separator row.

Help me repair the component table.

The table identifies broad service boundaries. A short rule set prevents later implementation choices from quietly merging those boundaries.

• Add the explicit boundary rules below the component table by pasting this block:

## Explicit infrastructure boundaries

<!-- These rules stop shared dependencies from becoming hidden single points of failure. -->
- Physical hosts, storage systems, and network switches are independent within each production tier.
- Identity, database, monitoring database, and deployment control planes have separate recovery paths.
- A node is not a failover target when it shares the failed component's physical boundary.

Why make the boundaries explicit?

A diagram can show two nodes while hiding their shared switch or storage system. These rules force every failover claim to include the physical dependency beneath it.

• Save production-reference.md.
• Check the boundary rules in the preview.

You should see three rules covering physical infrastructure, service control planes, and valid failover targets.

Are any boundaries still hidden?

Compare the rules with the component table. Every plane should have an independent compute, storage, network, identity, database, or control-plane recovery path.

Help me identify a hidden shared dependency.

Define SQL availability and recovery

The database design needs separate answers for local availability, site recovery, and backup. SQL Server Always On availability groups use Windows Server Failover Clustering for replica health and failover.

An independently available SMB service protects native backups outside database storage. A Failover Cluster Instance remains visible as the instance-level alternative.

• Add the SQL Server replica roles below the boundary rules by pasting this block:

## SQL Server availability and recovery

<!-- Replica roles separate local availability from disaster recovery. -->
| Replica | Site | Commit mode | Failover posture |
| --- | --- | --- | --- |
| `DB-01` | Primary | Synchronous commit | Automatic-failover eligible with `DB-02` while synchronized |
| `DB-02` | Primary | Synchronous commit | Automatic-failover eligible with `DB-01` while synchronized |
| `DB-03` | Disaster recovery | Asynchronous commit | Explicit recovery decision with possible data-loss acknowledgement |

- `DB-01` and `DB-02` reside on different nodes of the same WSFC.
- The two local replicas use independent hosts, storage, and replication paths.
- Automatic failover requires synchronous-commit mode on the current primary and one secondary.
- The secondary is synchronized before automatic failover is eligible.
- `DB-03` is promoted only by the named recovery authority after quorum and data-loss review.
- Availability-group replicas use independent storage and protect selected databases.

How do the replica roles differ?

• The synchronous pair provides local automatic-failover eligibility when the secondary is synchronized.
• The asynchronous replica preserves a site-recovery path across distance.
• The explicit recovery decision for DB-03 makes potential data loss a governed business choice.

• Save production-reference.md.
• Check the SQL Server section in the preview.

You should see the local synchronous pair separated from the asynchronous disaster-recovery replica. Only the synchronized local pair should be eligible for automatic failover.

Do all replicas look automatic?

Check the failover-posture column for DB-03. It must require an explicit recovery decision with possible data-loss acknowledgement.

Help me distinguish local failover from DR promotion.

Replica availability keeps a database service running through selected failures. Backups provide a separate recovery layer for corruption, deletion, and recovery to an earlier point.

• Add the independent backup design below the SQL replica section by pasting this block:

## Independent backup and restore

<!-- Backup remains a separate recovery layer from database replication. -->
- SQL Server writes native backups to an independently available SMB backup service.
- SQL Server service identities receive only the share and file permissions needed for backup and restore.
- Backup storage remains separate from database data and log storage.
- Every backup set has a separate offsite copy outside the primary-site fault domain.
- Remote backups are checked with `RESTORE VERIFYONLY` after transfer.
- Availability-group replication does not replace tested backup and restore.

What protects the backup path?

• The SMB service stays independent from the database data and log storage.
• The offsite copy keeps a recovery set outside the primary-site fault domain.
• The verification step checks the remote backup after network transfer.

• Save production-reference.md.
• Check the backup section in the preview.

You should see independent storage, workload-specific permissions, an offsite copy, and backup verification. The final line should keep replication and backup as separate recovery layers.

Does backup still share database storage?

Check that the SMB service has its own availability path. Confirm that the offsite copy leaves the primary-site fault domain.

Help me separate backup from replication.

Availability groups protect selected databases with independent replica storage. FCI protects an entire SQL Server instance through a shared-storage failover unit.

• Add the FCI comparison below the backup section by pasting this block:

## FCI alternative

<!-- FCI stays visible as a deliberate architecture choice with different trade-offs. -->
- A Failover Cluster Instance uses shared storage.
- Its failover unit includes the server, instance, and databases.
- FCI is the documented instance-level alternative.
- Availability groups remain the default database-level pattern because their replicas do not require shared storage.
- An availability replica hosted by an FCI uses manual availability-group failover.

Why document FCI as an alternative?

FCI protects the complete instance and depends on shared storage. The availability-group design keeps database replicas on independent storage and exposes database-level synchronization choices.

• Save production-reference.md.
• Check the FCI comparison in the preview.

You should see FCI described as an instance-level shared-storage alternative. The availability group should remain the default database-level pattern.

Are the two SQL patterns blurred together?

Look for the failover unit in each description. FCI protects the server, instance, and databases together while an availability group protects selected databases.

Help me clarify the SQL architecture choice.

Design the operations stack and test capacity

The operations plane combines Active Directory, managed service identities, and Zabbix without giving any one dependency control over every recovery path.

Deployment changes move from Git through Azure Pipelines before constrained execution through a Linux Ansible tier or PowerShell JEA endpoints.

• Add the identity plane below the FCI comparison by pasting this block:

## Operations architecture

### Identity plane

<!-- Service identities separate workload permissions from human administration. -->
- Active Directory remains available through site-independent domain services.
- Managed service identities or gMSAs are assigned per supported workload.
- Human administrators use separate named identities.
- Deployment agents receive no standing production-administrator role.

What boundary does identity create?

Site-independent domain services preserve authentication during a site failure. Workload-specific identities limit the access held by each service or deployment path.

• Save production-reference.md.
• Check the identity plane in the preview.

You should see separate identities for human administration, supported workloads, and deployment agents. No deployment agent should hold standing production administration.

Does one identity still control everything?

Check each identity category against a single workload purpose. Split any identity that crosses human administration, pipeline execution, and service operation.

Help me reduce identity privilege.

Zabbix server redundancy still depends on one shared database service. The monitoring database therefore needs an availability path separate from the server nodes and frontends.

• Add the monitoring plane below the identity plane by pasting this block:

### Monitoring plane

<!-- Zabbix components fail independently instead of sharing one recovery path. -->
- Redundant Zabbix server nodes run as active and standby nodes against the same separately protected database service.
- Redundant Zabbix frontends sit behind an external load-balancing layer.
- Frontends use consistent configuration and discover the active server through the database.
- Each site has its own Zabbix proxy group.
- Zabbix Agent 2 runs on Windows nodes and sends observations to the local proxy group.
- Each proxy uses a database separate from the Zabbix server database.
- Alerts and audit evidence leave the monitored workload fault domain.

How does monitoring survive component failure?

• The server nodes share a separately protected database service while only one server remains active.
• The redundant frontends stay outside the server-node failure boundary.
• Per-site proxy groups keep collection close to monitored Windows nodes.

• Save production-reference.md.
• Check the monitoring plane in the preview.

You should see separate frontend, server, database, proxy, and endpoint layers. Each proxy should use a database separate from the Zabbix server database.

Does Zabbix still have one hidden dependency?

Check whether the shared Zabbix database has its own protected service. Confirm that proxy databases remain separate from the server database.

Help me review the Zabbix failure domains.

The deployment control plane decides which approved change may reach production. Privileged execution stays behind JEA or the Ansible control tier so the pipeline agent remains constrained.

• Add the deployment and automation plane below the monitoring plane by pasting this block:

### Deployment and automation plane

<!-- Trust boundaries keep orchestration separate from privileged execution. -->
- Git changes flow through Azure Pipelines environments and checks.
- Resource administrators own approvals and checks outside pipeline YAML.
- Dedicated self-hosted agent pools are isolated by project and security zone.
- Agent runtime identities are nonprivileged and differ from pool-registration identities.
- Secrets remain in approved secret stores and never appear in logs or command lines.
- A Linux Ansible control tier manages approved convergence operations.
- PowerShell JEA endpoints expose only approved commands on Windows targets.
- Managed service identities or gMSAs provide workload-specific execution identities.
- The evidence trail records the commit, approval, agent, constrained action, result, and operator.
- Approved changes flow from Git to Azure Pipelines, then to isolated agents, then to Ansible or JEA endpoints.

How does an approved change reach a target?

• Azure Pipelines environments and checks control whether the deployment may start.
• Isolated agents orchestrate the approved work without holding permanent production administration.
• Ansible or JEA performs the constrained target-side action while the evidence trail records every boundary crossed.

• Save production-reference.md.
• Check the deployment flow in the preview.

You should see the flow from Git through approvals to an isolated agent. The final privileged action should pass through Ansible or a JEA endpoint.

Does the agent still hold permanent administration?

Check that the runtime identity differs from the pool-registration identity. Confirm that privileged target actions pass through a constrained execution layer.

Help me tighten the automation trust model.

Before you test this design against the laptop, do you think one host with 16 GB RAM or less and under 200 GB free SSD space can run every independent role concurrently while preserving these boundaries?

• Count the independent compute, storage, network, identity, database, monitoring, and control-plane roles in the preview.
• Compare that count with the current host limits from the project brief.

The complete design fails the single-host capacity test. One laptop collapses the physical host, storage, switch, power, rack, and site boundaries that production resilience requires.

That shortfall is intentional. Recording it prevents later lab work from presenting shared-host demonstrations as physical resilience.

• Record the failed capacity test at the end of production-reference.md by pasting this block:

## Capacity verdict: RED

> **RED:** The complete production topology must not be deployed concurrently on the current host.

- The host has `16 GB` RAM or less and under `200 GB` free SSD space.
- One laptop cannot provide independent physical host, storage, switch, power, rack, or site fault domains.
- Running every role together would collapse the boundaries this reference architecture requires.
- No high-availability or disaster-recovery claim is valid without its prerequisite boundary and observable failover evidence.
- The lab mapping must rotate scenarios and label each claim `Real`, `Real on shared hardware`, or `Simulated`.

Why is RED the correct verdict?

The production topology depends on independent physical failure domains. Concurrent deployment on one host would preserve some software mechanisms while removing the infrastructure boundaries needed to support the full claim.

• Save production-reference.md.
• Check the capacity verdict at the bottom of the preview.

You should see a red verdict that blocks concurrent deployment on the current host. It should direct the future lab toward rotating scenarios with explicit fidelity labels.

Does the verdict only mention memory?

Capacity includes more than RAM and disk. Confirm that the verdict also names the physical boundaries one laptop cannot provide.

Help me strengthen the capacity verdict.

✔️ Awesome, I've got everything!

Great. Double check that production-reference.md is saved and its side-by-side preview reaches the red capacity verdict.

ⓧ I'd like to double check the full code

Compare your complete production-reference.md file with this reference:

# Nova Trek Production Reference Architecture

> **Design contract:** Nova Trek must preserve service availability, recoverability, observability, and controlled automation across independent production fault domains.
> **Scope:** This document defines production intent only. It deploys no infrastructure.

## Production topology

<!-- This diagram exposes site and zone boundaries before implementation choices. -->
```text
CLIENTS
  |
External client-access load-balancing layer
  |
PRIMARY SITE                                  DISASTER-RECOVERY SITE
Management zone: Active Directory            Management zone: Active Directory
Application zone: redundant app nodes        Application zone: redundant app nodes
Database-replication zone: DB-01, DB-02       Database-replication zone: DB-03
Backup zone: independent SMB service         Backup zone: independent SMB service
Monitoring zone: server HA, frontends, proxies Monitoring zone: server HA, frontends, proxies
Client-access zone: redundant entry points    Client-access zone: redundant entry points
Compute: independent physical hosts          Compute: independent physical hosts
Storage: independent storage systems         Storage: independent storage systems
Network: independent switches and paths      Network: independent switches and paths

THIRD LOCATION
Independent WSFC quorum or witness service
```

## Component, boundary, and flow map

<!-- Each row makes a service path and its failure boundary reviewable. -->
| Plane | Primary site | DR site | Independent boundary and flow |
| --- | --- | --- | --- |
| Management | Site-local Active Directory and administration services | Site-local Active Directory and recovery administration | Identity services use separate compute, storage, and network paths. Authentication stays available when one site fails. |
| Application | Redundant application nodes | Redundant recovery application nodes | Client traffic enters through the client-access layer. Application traffic reaches the database service. |
| Database replication | `DB-01` and `DB-02` | `DB-03` | Database replicas use separate hosts and storage. Replication uses its own network zone. |
| Backup | Independently available SMB backup service | Independently available SMB backup service and offsite-copy path | Database backups leave database storage. A separate copy leaves the site. |
| Monitoring | Redundant servers, frontends, and site proxy group | Redundant servers, frontends, and site proxy group | Agent data reaches the local proxy group. Proxies reach the protected monitoring service. |
| Client access | Redundant site entry points | Redundant recovery entry points | An external load-balancing layer routes clients to an available site. |
| Control plane | Isolated deployment and automation services | Isolated recovery automation services | Approved changes cross a controlled boundary before reaching managed endpoints. |
| Quorum | WSFC nodes reach the witness | WSFC nodes reach the witness | The witness lives in a third location outside both workload sites. |

## Explicit infrastructure boundaries

<!-- These rules stop shared dependencies from becoming hidden single points of failure. -->
- Physical hosts, storage systems, and network switches are independent within each production tier.
- Identity, database, monitoring database, and deployment control planes have separate recovery paths.
- A node is not a failover target when it shares the failed component's physical boundary.

## SQL Server availability and recovery

<!-- Replica roles separate local availability from disaster recovery. -->
| Replica | Site | Commit mode | Failover posture |
| --- | --- | --- | --- |
| `DB-01` | Primary | Synchronous commit | Automatic-failover eligible with `DB-02` while synchronized |
| `DB-02` | Primary | Synchronous commit | Automatic-failover eligible with `DB-01` while synchronized |
| `DB-03` | Disaster recovery | Asynchronous commit | Explicit recovery decision with possible data-loss acknowledgement |

- `DB-01` and `DB-02` reside on different nodes of the same WSFC.
- The two local replicas use independent hosts, storage, and replication paths.
- Automatic failover requires synchronous-commit mode on the current primary and one secondary.
- The secondary is synchronized before automatic failover is eligible.
- `DB-03` is promoted only by the named recovery authority after quorum and data-loss review.
- Availability-group replicas use independent storage and protect selected databases.

## Independent backup and restore

<!-- Backup remains a separate recovery layer from database replication. -->
- SQL Server writes native backups to an independently available SMB backup service.
- SQL Server service identities receive only the share and file permissions needed for backup and restore.
- Backup storage remains separate from database data and log storage.
- Every backup set has a separate offsite copy outside the primary-site fault domain.
- Remote backups are checked with `RESTORE VERIFYONLY` after transfer.
- Availability-group replication does not replace tested backup and restore.

## FCI alternative

<!-- FCI stays visible as a deliberate architecture choice with different trade-offs. -->
- A Failover Cluster Instance uses shared storage.
- Its failover unit includes the server, instance, and databases.
- FCI is the documented instance-level alternative.
- Availability groups remain the default database-level pattern because their replicas do not require shared storage.
- An availability replica hosted by an FCI uses manual availability-group failover.

## Operations architecture

### Identity plane

<!-- Service identities separate workload permissions from human administration. -->
- Active Directory remains available through site-independent domain services.
- Managed service identities or gMSAs are assigned per supported workload.
- Human administrators use separate named identities.
- Deployment agents receive no standing production-administrator role.

### Monitoring plane

<!-- Zabbix components fail independently instead of sharing one recovery path. -->
- Redundant Zabbix server nodes run as active and standby nodes against the same separately protected database service.
- Redundant Zabbix frontends sit behind an external load-balancing layer.
- Frontends use consistent configuration and discover the active server through the database.
- Each site has its own Zabbix proxy group.
- Zabbix Agent 2 runs on Windows nodes and sends observations to the local proxy group.
- Each proxy uses a database separate from the Zabbix server database.
- Alerts and audit evidence leave the monitored workload fault domain.

### Deployment and automation plane

<!-- Trust boundaries keep orchestration separate from privileged execution. -->
- Git changes flow through Azure Pipelines environments and checks.
- Resource administrators own approvals and checks outside pipeline YAML.
- Dedicated self-hosted agent pools are isolated by project and security zone.
- Agent runtime identities are nonprivileged and differ from pool-registration identities.
- Secrets remain in approved secret stores and never appear in logs or command lines.
- A Linux Ansible control tier manages approved convergence operations.
- PowerShell JEA endpoints expose only approved commands on Windows targets.
- Managed service identities or gMSAs provide workload-specific execution identities.
- The evidence trail records the commit, approval, agent, constrained action, result, and operator.
- Approved changes flow from Git to Azure Pipelines, then to isolated agents, then to Ansible or JEA endpoints.

## Capacity verdict: RED

> **RED:** The complete production topology must not be deployed concurrently on the current host.

- The host has `16 GB` RAM or less and under `200 GB` free SSD space.
- One laptop cannot provide independent physical host, storage, switch, power, rack, or site fault domains.
- Running every role together would collapse the boundaries this reference architecture requires.
- No high-availability or disaster-recovery claim is valid without its prerequisite boundary and observable failover evidence.
- The lab mapping must rotate scenarios and label each claim `Real`, `Real on shared hardware`, or `Simulated`.

Before the final review, choose one node in the preview. Do you expect its documented failover target to remain available after that node is removed?

• Review production-reference.md from the production topology through the capacity verdict.
• Trace each client, replication, backup, monitoring, identity, and deployment flow to its next component.
• Remove one node mentally from each redundant tier.
• Confirm that the remaining failover target does not share the removed node's documented physical boundary.

You should see the primary site, disaster-recovery site, and third witness location in the preview. Every production plane should have a visible flow plus an independent failover or recovery path.

That is the production reference locked in. Nova Trek now has a reviewable blueprint that separates local availability, disaster recovery, backup, monitoring, identity, and controlled automation.

Your complete architecture now exposes more independent infrastructure than the current host can provide. Next, you will preserve its mental model by mapping each mechanism into a rotating lab scenario.

Map Production to a Rotating Lab

Your production reference now exposes independent fault domains across the primary site, disaster-recovery site, and witness location. The complete design exceeds the current host's capacity.

This step translates that design into rotating Hyper-V scenarios. Each scenario preserves the production mental model while keeping infrastructure undeployed.

In this step, get ready to:

• Classify every mechanism with an evidence-bound fidelity label.
• Define the roles, shutdown order, preservation needs, rollback point, and evidence for six rotating scenarios.
• Add measured capacity gates that control when each scenario can start.

Classify each mechanism by fidelity

A fidelity label sets the boundary of an architecture claim. This distinction is easy to blur because software failover can succeed while every node still depends on one laptop.

• In the Visual Studio Code Explorer sidebar, create lab-mapping.md inside docs/architecture.
• Build the fidelity model by pasting this content into docs/architecture/lab-mapping.md:

# Nova Trek Rotating Lab Mapping

This document maps the production reference architecture to rotating, scenario-specific lab configurations for the constrained Windows host. No scenario proves independence from the host.

## Fidelity labels

- **Real**: The actual component behavior is exercised when the scenario is deployed.
- **Real on shared hardware**: The actual software mechanism is exercised, but every node shares one physical failure domain.
- **Simulated**: The event or dependency is represented through a tabletop, controlled shutdown, network rule, or mocked evidence.

## Fidelity matrix

| Capability | Fidelity | Claim boundary |
| --- | --- | --- |
| Native SQL backup and restore | Real | Claim only the backup, restore, and verification behavior that is directly observed. |
| SMB permissions | Real | Claim only the access behavior tested with the intended identity. |
| Zabbix Agent 2 collection | Real | Claim only the metric collection observed from the deployed endpoint. |
| Azure Pipelines execution | Real | Claim only the approved pipeline run and captured audit evidence. |
| PowerShell remoting | Real | Claim only the remote operation allowed by the deployed endpoint. |
| Ansible convergence | Real | Claim only the state change and repeatable result observed on the managed node. |
| Guest WSFC failover | Real on shared hardware | Claim the cluster mechanism while naming the shared host boundary. |
| SQL availability-group failover | Real on shared hardware | Claim replica synchronization and failover while naming the shared host boundary. |
| Zabbix native server HA | Real on shared hardware | Claim active and standby behavior while naming the shared database and host boundary. |
| Proxy-group failover | Real on shared hardware | Claim host redistribution while naming the shared host boundary. |
| Rack loss | Simulated | Record the assumed loss and expected recovery sequence. |
| Independent storage failure | Simulated | Record the assumed storage loss and expected recovery sequence. |
| Redundant network paths | Simulated | Record the path loss and expected traffic behavior. |
| Multi-host Hyper-V continuity | Simulated | Record the host loss and evidence needed from a second physical host. |
| Full site loss | Simulated | Record the site outage and dependency-aware recovery sequence. |

How does the fidelity matrix work?

• The fidelity label states how closely the lab exercise matches the production mechanism.
• The claim boundary states what the resulting evidence can support.
• The shared-hardware label preserves successful software failover without claiming physical resilience.
• The simulated label keeps missing infrastructure visible as an evidence gap.

• Save docs/architecture/lab-mapping.md.
• Use the preview control in the upper-right corner of the editor to open the Markdown preview.

You should see three fidelity definitions followed by a table containing fifteen classified capabilities.

Table not rendering correctly?

• Confirm that every table row begins and ends with a vertical bar.
• Check that the separator row sits directly beneath the table headings.
• Remove any blank line inserted between the separator row and the first capability.

Ask for help with the table structure:

Define rotating scenarios

A rotating scenario keeps only the roles required for one test active at the same time. The scenario matrix connects each high availability claim to a shutdown order, rollback point, and observable result.

• Switch back to the editor for docs/architecture/lab-mapping.md.
• Add the six rotating scenarios beneath the fidelity matrix by pasting this content:

## Rotating scenarios

Run one scenario at a time. Preserve configuration outside the running machines so each scenario can be rebuilt without keeping the full topology online.

| Scenario | Minimum concurrent machines or roles | Shutdown order | Preserved configuration | Production component represented | Rollback point | Required observable evidence |
| --- | --- | --- | --- | --- | --- | --- |
| Backup scenario | Active Directory node, `DB-01`, SMB backup service, and offsite-copy target | Stop `DB-01` first. Stop the offsite-copy target next. Stop the SMB backup service next. Stop Active Directory last. | SQL backup definitions, SMB permissions, restore runbook, and offsite-copy record | Independent backup and restore plane | Last verified database backup plus the recorded SMB permission baseline | Successful native backup, separate copy, verified restore, and identity-based access result |
| Monitoring scenario: Zabbix proxy failover | Active Directory node, protected Zabbix database role, active and standby Zabbix server roles, two proxies, one frontend, and one monitored Windows endpoint | Stop the monitored endpoint first. Stop the proxies next. Stop the frontend next. Stop the standby server next. Stop the active server next. Stop the database next. Stop Active Directory last. | Zabbix database backup, server configuration, proxy configuration, frontend configuration, and endpoint registration | Monitoring database, server HA, frontend, proxy group, and Agent 2 collection | Last healthy monitoring baseline plus the versioned server and proxy configuration | Agent 2 data, active-server transition, proxy-group redistribution, and frontend visibility |
| SQL HA scenario: automatic failover | Active Directory node, `DB-01`, `DB-02`, WSFC witness, and one client role | Stop the client first. Stop `DB-02` next. Stop `DB-01` next. Stop the witness next. Stop Active Directory last. | WSFC configuration, availability-group configuration, listener details, database backup, and failover runbook | Synchronous local SQL availability-group pair | Last synchronized replica state plus a verified database backup | Synchronized replicas, automatic failover, client reconnection, and recorded shared-host limitation |
| CI/CD scenario: Azure Pipelines deployment | Azure Pipelines environment, isolated self-hosted agent role, Active Directory node, JEA endpoint, and target Windows node | Stop the target Windows node first. Stop the agent role next. Stop Active Directory last. | Pipeline definition, environment checks, agent-pool boundary, JEA configuration, and audit record | Approved deployment path with constrained administration | Last approved pipeline commit plus the captured pre-deployment target state | Passed checks, isolated agent execution, constrained remote action, and audit evidence |
| Ansible scenario | Linux Ansible control role, Active Directory node, and target Windows node | Stop the target Windows node first. Stop the Linux control role next. Stop Active Directory last. | Inventory, automation content, remoting configuration, identity mapping, and pre-run state | Linux automation control tier for Windows convergence | Versioned automation content plus the captured pre-run target state | Successful convergence, repeatable second run, and recorded remote identity |
| Compound-failure scenario | Active Directory node, `DB-01`, `DB-02`, WSFC witness, SMB backup role, Zabbix database role, Zabbix server role, proxy role, deployment agent role, JEA endpoint, Linux Ansible control role, and one client role | Stop the client first. Stop automation roles next. Stop monitoring roles next. Stop database roles next. Stop backup roles next. Stop the witness next. Stop Active Directory last. | Every component-level baseline, dependency map, recovery authority record, and evidence-gap register | Cross-plane failure response | Last proven component-level baseline plus the documented recovery decision point | Dependency-aware recovery order, surviving control-plane functions, unsupported claims, and explicit evidence gaps |

What does each scenario row prove?

• Minimum roles define the smallest concurrent configuration that can exercise the scenario.
• Shutdown order protects dependencies while releasing host capacity predictably.
• Preserved configuration allows the scenario to return without keeping every machine active.
• The rollback point names the last known state that can reverse the exercise.
• Observable evidence controls which production claim the scenario can support.

• Save docs/architecture/lab-mapping.md.
• Return to the Markdown preview.

You should see six scenario rows. Each row should expose its concurrent roles, shutdown order, preservation needs, production mapping, rollback point, and evidence.

Scenario row too wide?

A wide table can require horizontal scrolling in preview. Keep each responsibility in its assigned column so the relationship remains traceable.

Ask for help checking the scenario columns:

Gate each scenario with evidence

Fixed virtual-machine sizes would hide the host's real operating margin. Measured gates decide whether a scenario can start safely on the day it runs.

• Switch back to the editor for docs/architecture/lab-mapping.md.
• Complete the mapping with capacity gates and the claim-promotion rule by pasting this content:

## Scenario admission gates

Every scenario must pass every gate before startup. Record the measurement, decision, and rollback point with the scenario evidence.

| Gate | Required record | Stop condition |
| --- | --- | --- |
| Host-free memory | Measure available host memory before starting any scenario role. | Stop when the measured allowance cannot support the planned concurrent roles. |
| Disk growth allowance | Measure free disk space and estimate scenario growth before startup. | Stop when the remaining allowance cannot absorb expected data, logs, and temporary files. |
| Checkpoint-merge headroom | Reserve measured disk headroom for any planned checkpoint merge. | Stop when a merge could exhaust the remaining disk space. |
| Startup validation | Start roles in dependency order and confirm each required service before the test. | Stop at the first failed dependency. |
| Rollback point | Name the preserved configuration, backup, or clean state used to reverse the scenario. | Stop when recovery from the planned change has no tested path. |

## Claim promotion rule

A mechanism moves from **Simulated** to **Real on shared hardware** or **Real** only when its prerequisites are present and its observable failover evidence is captured.
Evidence from one physical host cannot prove independent host, storage, rack, network-path, or site resilience.

Why use measured gates?

• Host-free memory reflects the capacity available when the scenario begins.
• Disk growth allowance accounts for data, logs, and temporary files created during the exercise.
• Checkpoint-merge headroom prevents a planned rollback aid from consuming the remaining disk space.
• Startup validation stops the scenario when a required dependency is already unhealthy.
• The promotion rule requires observable evidence before the fidelity claim can change.

• Save docs/architecture/lab-mapping.md.
• Return to the Markdown preview.

You should see five admission gates followed by a claim-promotion rule that preserves the single-host evidence boundary.

Missing a capacity gate?

• Confirm that the table contains host-free memory, disk growth allowance, checkpoint-merge headroom, startup validation, and rollback point.
• Check that each gate has both a required record and a stop condition.
• Keep all capacity values measured so the document does not invent virtual-machine sizes.

Ask for help reviewing the gates:

✔️ Awesome, I've got everything!

Great. Double check that you saved docs/architecture/lab-mapping.md with all three sections.

ⓧ I'd like to double check the full code

Compare your complete docs/architecture/lab-mapping.md file with this reference.

# Nova Trek Rotating Lab Mapping

This document maps the production reference architecture to rotating, scenario-specific lab configurations for the constrained Windows host. No scenario proves independence from the host.

## Fidelity labels

- **Real**: The actual component behavior is exercised when the scenario is deployed.
- **Real on shared hardware**: The actual software mechanism is exercised, but every node shares one physical failure domain.
- **Simulated**: The event or dependency is represented through a tabletop, controlled shutdown, network rule, or mocked evidence.

## Fidelity matrix

| Capability | Fidelity | Claim boundary |
| --- | --- | --- |
| Native SQL backup and restore | Real | Claim only the backup, restore, and verification behavior that is directly observed. |
| SMB permissions | Real | Claim only the access behavior tested with the intended identity. |
| Zabbix Agent 2 collection | Real | Claim only the metric collection observed from the deployed endpoint. |
| Azure Pipelines execution | Real | Claim only the approved pipeline run and captured audit evidence. |
| PowerShell remoting | Real | Claim only the remote operation allowed by the deployed endpoint. |
| Ansible convergence | Real | Claim only the state change and repeatable result observed on the managed node. |
| Guest WSFC failover | Real on shared hardware | Claim the cluster mechanism while naming the shared host boundary. |
| SQL availability-group failover | Real on shared hardware | Claim replica synchronization and failover while naming the shared host boundary. |
| Zabbix native server HA | Real on shared hardware | Claim active and standby behavior while naming the shared database and host boundary. |
| Proxy-group failover | Real on shared hardware | Claim host redistribution while naming the shared host boundary. |
| Rack loss | Simulated | Record the assumed loss and expected recovery sequence. |
| Independent storage failure | Simulated | Record the assumed storage loss and expected recovery sequence. |
| Redundant network paths | Simulated | Record the path loss and expected traffic behavior. |
| Multi-host Hyper-V continuity | Simulated | Record the host loss and evidence needed from a second physical host. |
| Full site loss | Simulated | Record the site outage and dependency-aware recovery sequence. |

## Rotating scenarios

Run one scenario at a time. Preserve configuration outside the running machines so each scenario can be rebuilt without keeping the full topology online.

| Scenario | Minimum concurrent machines or roles | Shutdown order | Preserved configuration | Production component represented | Rollback point | Required observable evidence |
| --- | --- | --- | --- | --- | --- | --- |
| Backup scenario | Active Directory node, `DB-01`, SMB backup service, and offsite-copy target | Stop `DB-01` first. Stop the offsite-copy target next. Stop the SMB backup service next. Stop Active Directory last. | SQL backup definitions, SMB permissions, restore runbook, and offsite-copy record | Independent backup and restore plane | Last verified database backup plus the recorded SMB permission baseline | Successful native backup, separate copy, verified restore, and identity-based access result |
| Monitoring scenario: Zabbix proxy failover | Active Directory node, protected Zabbix database role, active and standby Zabbix server roles, two proxies, one frontend, and one monitored Windows endpoint | Stop the monitored endpoint first. Stop the proxies next. Stop the frontend next. Stop the standby server next. Stop the active server next. Stop the database next. Stop Active Directory last. | Zabbix database backup, server configuration, proxy configuration, frontend configuration, and endpoint registration | Monitoring database, server HA, frontend, proxy group, and Agent 2 collection | Last healthy monitoring baseline plus the versioned server and proxy configuration | Agent 2 data, active-server transition, proxy-group redistribution, and frontend visibility |
| SQL HA scenario: automatic failover | Active Directory node, `DB-01`, `DB-02`, WSFC witness, and one client role | Stop the client first. Stop `DB-02` next. Stop `DB-01` next. Stop the witness next. Stop Active Directory last. | WSFC configuration, availability-group configuration, listener details, database backup, and failover runbook | Synchronous local SQL availability-group pair | Last synchronized replica state plus a verified database backup | Synchronized replicas, automatic failover, client reconnection, and recorded shared-host limitation |
| CI/CD scenario: Azure Pipelines deployment | Azure Pipelines environment, isolated self-hosted agent role, Active Directory node, JEA endpoint, and target Windows node | Stop the target Windows node first. Stop the agent role next. Stop Active Directory last. | Pipeline definition, environment checks, agent-pool boundary, JEA configuration, and audit record | Approved deployment path with constrained administration | Last approved pipeline commit plus the captured pre-deployment target state | Passed checks, isolated agent execution, constrained remote action, and audit evidence |
| Ansible scenario | Linux Ansible control role, Active Directory node, and target Windows node | Stop the target Windows node first. Stop the Linux control role next. Stop Active Directory last. | Inventory, automation content, remoting configuration, identity mapping, and pre-run state | Linux automation control tier for Windows convergence | Versioned automation content plus the captured pre-run target state | Successful convergence, repeatable second run, and recorded remote identity |
| Compound-failure scenario | Active Directory node, `DB-01`, `DB-02`, WSFC witness, SMB backup role, Zabbix database role, Zabbix server role, proxy role, deployment agent role, JEA endpoint, Linux Ansible control role, and one client role | Stop the client first. Stop automation roles next. Stop monitoring roles next. Stop database roles next. Stop backup roles next. Stop the witness next. Stop Active Directory last. | Every component-level baseline, dependency map, recovery authority record, and evidence-gap register | Cross-plane failure response | Last proven component-level baseline plus the documented recovery decision point | Dependency-aware recovery order, surviving control-plane functions, unsupported claims, and explicit evidence gaps |

## Scenario admission gates

Every scenario must pass every gate before startup. Record the measurement, decision, and rollback point with the scenario evidence.

| Gate | Required record | Stop condition |
| --- | --- | --- |
| Host-free memory | Measure available host memory before starting any scenario role. | Stop when the measured allowance cannot support the planned concurrent roles. |
| Disk growth allowance | Measure free disk space and estimate scenario growth before startup. | Stop when the remaining allowance cannot absorb expected data, logs, and temporary files. |
| Checkpoint-merge headroom | Reserve measured disk headroom for any planned checkpoint merge. | Stop when a merge could exhaust the remaining disk space. |
| Startup validation | Start roles in dependency order and confirm each required service before the test. | Stop at the first failed dependency. |
| Rollback point | Name the preserved configuration, backup, or clean state used to reverse the scenario. | Stop when recovery from the planned change has no tested path. |

## Claim promotion rule

A mechanism moves from **Simulated** to **Real on shared hardware** or **Real** only when its prerequisites are present and its observable failover evidence is captured.
Evidence from one physical host cannot prove independent host, storage, rack, network-path, or site resilience.

How to use the full-file reference

The reference shows the cumulative document after all three edits. Matching its structure keeps every fidelity claim connected to a scenario, gate, rollback point, and evidence requirement.

Before you inspect the finished mapping, which physical boundary do you expect to limit the strongest claim?

• Return to the preview for docs/architecture/lab-mapping.md.
• Choose the SQL Server automatic-failover row, the Zabbix proxy-failover row, or the Azure Pipelines deployment row.
• Trace the selected row across every column.

You should see the exact concurrent roles, shutdown order, preserved configuration, production mapping, rollback point, and required evidence for your selection. Its fidelity boundary should also state what the shared host prevents you from claiming.

That is a strong architecture checkpoint. Your compact lab now preserves enterprise mechanisms without overstating the resilience one host can prove.

Your rotating lab contract is ready. Next, you'll lock these boundaries into architecture decisions and a sequenced implementation roadmap.

Lock Decisions and the Implementation Roadmap

Your production reference now exposes the enterprise fault domains. The rotating lab mapping preserves those ideas on the constrained host.

Without explicit decisions or gates, later modules can drift toward convenient shortcuts. Those shortcuts can create overprivileged automation or false high availability claims.

Architecture Decision Records turn each choice into a reviewable contract. A gated roadmap ties every future project to measurable proof.

In this step, get ready to:

• Lock the architecture choices in decision records.
• Sequence sub-hour implementation projects behind evidence gates.
• Commit the design package as a traceable baseline.

Capture the architecture decisions

An ADR preserves the reason behind a choice alongside its consequences. The empty docs/decisions folder in your open Visual Studio Code workspace is reserved for these records.

Production and fidelity decisions

The production-first-design.md record makes the enterprise production reference the design authority. The compact lab must preserve its boundaries when the topology is compressed.

• The Real label means the actual product behavior is exercised.
• The Real on shared hardware label means the actual software mechanism runs inside one physical fault domain.
• The Simulated label means the event is represented through a tabletop or controlled test.
• The fidelity-labels.md record requires observable evidence before any claim receives a stronger label.

• Use the new-file control at the top of the Explorer sidebar to create docs/decisions/production-first-design.md.
• Write the production-first brief under Status, Context, Decision, Consequences, and Evidence Required sections.
• Set the status to Accepted.
• Save production-first-design.md using the editor's save control.
• Use the preview control in the editor's top-right corner to confirm that production remains the design authority.
• Use the new-file control in the Explorer sidebar to create docs/decisions/fidelity-labels.md.
• Write the three fidelity definitions from the brief above under the same five ADR sections.
• Save fidelity-labels.md using the editor's save control.

The Explorer sidebar now lists the first two decision records. Their rendered previews should preserve the production-first contract plus the exact three fidelity labels.

Database availability and recovery decisions

The sql-availability-pattern.md record selects SQL Server availability groups as the primary database-level pattern. DB-01 plus DB-02 remain synchronous replicas eligible for automatic failover.

DB-03 remains an asynchronous disaster-recovery replica that requires an explicit recovery decision. Windows Server Failover Clustering provides the cluster foundation.

A Failover Cluster Instance remains the documented instance-level shared-storage alternative. The availability-group design remains the database-level default.

The independent-backup-and-restore.md record requires an independent SMB backup service with a separate offsite copy. Restore validation provides the recovery evidence.

• Use the new-file control in the Explorer sidebar to create docs/decisions/sql-availability-pattern.md.
• Write the SQL availability brief under the five ADR sections used in the earlier records.
• Set the status to Accepted.
• Save sql-availability-pattern.md using the editor's save control.
• Use the preview control to confirm that DB-03 requires an explicit disaster-recovery decision.
• Use the new-file control in the Explorer sidebar to create docs/decisions/independent-backup-and-restore.md.
• Write the backup brief under the same five ADR sections.
• Save independent-backup-and-restore.md using the editor's save control.

The database records now separate availability from recoverability. Replication protects service continuity while independently validated backups protect recovery.

Operations and host-boundary decisions

The operations-trust-boundaries.md record requires Zabbix server nodes to use a separately protected database service. It also preserves redundant frontends plus site proxy groups.

The automation boundary uses dedicated Azure Pipelines agents with environment checks. Constrained administration passes through PowerShell JEA, managed identities, or the Linux Ansible control tier.

The same record keeps secrets outside source control. Deployment agents receive no standing production administration.

The single-host-limitations.md record limits the current host to rotating Hyper-V scenarios. Shared-host testing cannot support claims of independent physical resilience.

• Use the new-file control in the Explorer sidebar to create docs/decisions/operations-trust-boundaries.md.
• Write the operations trust brief under the five ADR sections used in the earlier records.
• Set the status to Accepted.
• Save operations-trust-boundaries.md using the editor's save control.
• Use the preview control to confirm that every privileged action crosses a constrained trust boundary.
• Use the new-file control in the Explorer sidebar to create docs/decisions/single-host-limitations.md.
• Write the single-host brief under the same five ADR sections.
• Save single-host-limitations.md using the editor's save control.

You now have six decisions that stop convenience from becoming architecture. Each record identifies the evidence required before a stronger claim can be made.

Build the gated program roadmap

A roadmap converts the decisions into a safe execution order. Each project proves one primary concept before a later project depends on it.

The implementation sequence

• Host instrumentation and Hyper-V establishes measured capacity gates plus the virtualization foundation.
• Identity and secure remoting establishes authentication boundaries plus constrained remote access.
• SMB backup and restore proves independent backup access plus recovery validation.
• PowerShell automation turns repeatable administration into auditable operations.
• Zabbix baseline monitoring establishes endpoint visibility before monitoring redundancy is introduced.
• SQL WSFC and availability groups proves quorum behavior plus database failover.
• Zabbix HA and proxy groups proves server coordination plus monitoring redistribution.
• Azure Pipelines introduces protected environments plus isolated deployment agents.
• Ansible convergence introduces the Linux control tier plus repeatable managed-node state.
• Security hardening tests the combined trust boundaries against least-privilege requirements.
• Compound failure game days test recovery when multiple dependencies disappear together.

The review gate for every project

• Prerequisites identify the state that must already exist.
• The real-versus-simulated claim selects one defined fidelity label.
• Failure injection identifies the condition the project deliberately triggers.
• Observable evidence defines what the learner must capture.
• The rollback plan restores a known-good state.
• The security boundary names the identities plus privileges involved.
• The artifact to commit preserves the implementation evidence.
• Promotion criteria define what larger infrastructure must prove.

• Return to the Explorer sidebar from earlier.
• Use the new-file control inside docs/roadmap to create docs/roadmap/program-roadmap.md.

The new roadmap file should now appear under docs/roadmap in the Explorer sidebar.

• Add a title that identifies the document as the Nova Trek implementation roadmap.
• State that each future project must stay under one hour.
• Save program-roadmap.md using the editor's save control.
• Use the preview control to confirm that the title plus time constraint render at the top.

The preview now establishes the delivery constraint before any project is sequenced.

• Add roadmap entries one through four from the implementation sequence above.
• Complete every review-gate field for those four entries.
• Save program-roadmap.md using the editor's save control.
• Refresh the rendered preview to confirm that host instrumentation appears first.

The first group should end with PowerShell automation. Each entry should identify one failure to inject plus one observable result.

• Add roadmap entries five through eight from the implementation sequence above.
• Complete every review-gate field for those four entries.
• Save program-roadmap.md using the editor's save control.
• Refresh the rendered preview to confirm that Azure Pipelines follows the monitoring plus database projects.

The middle group should progress from baseline monitoring to protected deployment orchestration. Its ordering keeps later control planes dependent on earlier evidence.

• Add roadmap entries nine through eleven from the implementation sequence above.
• Complete every review-gate field for those three entries.
• Save program-roadmap.md using the editor's save control.
• Refresh the rendered preview to confirm that compound failure game days finish the sequence.

The completed roadmap should contain eleven sub-hour projects. Every entry should end with promotion criteria for larger infrastructure.

Commit the design baseline

Git turns the completed package into a reviewable checkpoint. The latest commit should capture the production reference plus its lab translation plus every decision record plus the roadmap.

• Save every open document using Visual Studio Code's save control.
• Stage the completed design package by running this command in the integrated terminal from earlier:

git add .

What does this command do?

The git add . command stages the architecture documents plus decision records plus roadmap from the current repository. The next commit captures that staged design state.

Files missing from the staged changes?

Confirm that the integrated terminal still points to the nova-trek-sre-lab folder. A terminal opened from another folder stages a different working tree.

Ask for help checking the active repository path: Why are my Nova Trek files missing after I stage the repository?

• Return to the Source Control sidebar in Visual Studio Code.
• Enter Complete Nova Trek SRE architecture blueprint in the commit message field.
• Select the Commit control at the top of the Source Control sidebar.

That gives the entire design package one traceable checkpoint. Your architecture choices now travel with their implementation gates.

Before you check, which commit do you expect to occupy the top line of the history?

• Inspect the compact repository history by running this command:

git log --oneline

What should you see?

The top line should contain an abbreviated commit identifier followed by Complete Nova Trek SRE architecture blueprint. The earlier initialization commit should remain below it.

Latest commit missing from the history?

Return to the Source Control sidebar to check whether staged files remain. Complete the local commit before inspecting the history again.

Ask for help diagnosing the missing checkpoint: Why does my latest Nova Trek commit not appear in the Git history?

• Return to the rendered docs/roadmap/program-roadmap.md preview from earlier.
• Trace the sequence from Host instrumentation and Hyper-V through Compound failure game days.

Every project should show one primary concept plus its entry gate. You should also see evidence requirements plus promotion criteria for larger infrastructure.

Secret mission

Survive a Primary-Site Outage

Assume the primary site's compute, storage, and network disappear together. Trace every surviving data plane and control plane before writing the only safe recovery order. Then mark every unsupported resilience claim as an evidence gap.

Clean Up Your Resources

Clean Up Your Resources

Your only project resource is the local nova-trek-sre-lab Git repository. No infrastructure is running, so you can keep the repository, close it for later, or delete it without ongoing costs.

Resources you used:

• The local nova-trek-sre-lab repository containing the production reference, rotating-lab mapping, architecture decision records, implementation roadmap, primary-site outage tabletop, and Git history.

Keep everything running

No action is needed. Choose this if you plan to implement the roadmap or refine the architecture later.

• Keep the nova-trek-sre-lab repository in its current location.
• Return to docs/roadmap/program-roadmap.md when you are ready to begin the first implementation project.

Pause - I'll come back to this later

There is no running lab to stop. Choose this option when you want to free your screen while preserving every design artifact.

• Close Visual Studio Code with the nova-trek-sre-lab folder still saved on disk.
• Leave the nova-trek-sre-lab folder untouched until you resume the roadmap.

Delete - I don't want to use this again

Deleting the folder permanently removes every architecture document plus its Git history. No external infrastructure is affected because this project deployed none.

• Copy the nova-trek-sre-lab folder to another location first if you want an archive.
• Switch back to the Visual Studio Code integrated terminal from earlier.
• Remove the nova-trek-sre-lab repository from its parent folder by running these commands:

cmd
cd ..
rmdir /s /q nova-trek-sre-lab

What do these commands do?

• The first command switches the terminal session to Windows Command Prompt.
• The second command moves the terminal to the folder containing the repository.
• The final command permanently removes the repository plus all of its contents without another confirmation prompt.

• Check the repository parent folder in File Explorer.

You should no longer see the nova-trek-sre-lab folder.

That completes the cleanup. The Nova Trek design package is gone from your host.

Folder still visible?

Close any Visual Studio Code windows that reference the repository. Return to Command Prompt in the parent folder. Repeat the final deletion command.

Help me remove a folder that stays in use.

Nice Work!

Nice Work!

You did it! You turned Nova Trek's resilience requirements into a version-controlled SRE blueprint that another engineer can review before any infrastructure is deployed.

What you learned:

• Build a version-controlled architecture repository in Git. Its production reference exposes independent fault domains across database, backup, monitoring, identity, and deployment planes.
• Distinguish high availability, disaster recovery, backup, restart, retry, and simulation. Your SQL Server availability groups design separates local automatic failover from explicit asynchronous disaster recovery.
• Define least-privilege trust boundaries for Azure Pipelines, PowerShell JEA, managed identities, and Ansible. Your roadmap ties each future module to capacity gates, failure injection, rollback, evidence, and promotion criteria.
• Secret Mission: Tabletop-tested a complete primary-site outage across database recovery, SMB, Zabbix, identity, deployment, PowerShell, and Ansible. The recovery timeline now captures decision authority, split-brain prevention, controlled failback, and evidence gaps.

Ready to quiz yourself?