Train a Solo LiDAR PPO Racer

Introduction

30 Second Summary

A high score usually feels like proof that something is going well. However, a self-driving race car can collect points while reversing over the same stretch or crashing before it finishes a lap.

In this project, you will build a compact 2D F1TENTH-style racing research harness that trains a policy from 24 LiDAR-like readings with PPO. A behavioural gate keeps traffic locked until the policy proves it can complete solo laps.

What You'll Build

On screen, your racer follows a visible circuit while a behavioural report decides whether the policy has earned access to moving traffic.

By the end of this project, you'll have:

• A visible circuit preview that shows the racer tracing the track while the terminal confirms the exact 33-value observation contract.
• A reward exploit demonstration that produces a positive v1 round-trip total before showing a negative v2 total for the same forward-backward motion.
• A three-seed solo smoke report comparing trained checkpoints with untrained and pure-pursuit baselines. When the lap-completion gate passes, an animated top-down replay shows a traffic-trained car and scripted opponent moving around the circuit.
• Secret Mission: Evaluate frozen traffic-qualified policies on a held-out circuit geometry without retraining.

Do I need any experience before starting?

Basic Python familiarity is enough for this project. You need Windows, Git, and Visual Studio Code, but no reinforcement learning framework experience.

Before We Start

Before the hands-on work begins, commit to a reproducible smoke experiment whose failed runs remain useful evidence. Success comes from behavioural evidence of safe forward lap completion inside track limits with no wall contacts, caused collisions, or reverse driving because reward is only a training proxy.

Set Up the CPU-Only RL Workspace

Reinforcement learning failures are hard to diagnose when dependency versions drift between runs. A pinned virtual environment gives this smoke experiment a repeatable software base.

In this step, you will confirm a supported Python interpreter. You will create an isolated Windows PowerShell workspace with the exact packages used by the racing harness.

In this step, get ready to:

• Confirm that Python 3.12, 3.13, or 3.14 is available.
• Create the isolated lidar-ppo-smoke workspace.
• Install the pinned reinforcement-learning packages.

Confirm a supported Python interpreter

The package pins support Python 3.12 through 3.14. The launcher must show one of those versions before you create the workspace.

• Press the Windows key to open Windows search.
• Type Windows PowerShell into the search field.
• Press Enter to open Windows PowerShell.
• List the installed Python runtimes by running this command:

py --list

What does this command show?

The py --list command asks the Python launcher to display every interpreter it can use. A listed version from 3.12 through 3.14 provides a compatible base for the pinned packages.

✔️ I see a supported version

Your launcher can see a compatible interpreter. That gives the project a stable base for its isolated environment.

ⓧ I see only an unsupported version

Keep your existing interpreter installed. Add the supported Python release alongside it so your other projects remain unchanged.

• Visit the official Python for Windows downloads page.
• Download the current 64-bit Python 3.13.17 installer.
• Run the installer with Python launcher support included.
• Rerun the launcher-listing command above.

Continue when the list contains Python 3.13.

ⓧ Command not found

The Python launcher is unavailable. Installing the supported Windows release adds the interpreter required by this project.

• Visit the official Python for Windows downloads page.
• Download the current 64-bit Python 3.13.17 installer.
• Run the installer with Python launcher support included.
• Close Windows PowerShell after the installation finishes.
• Open a fresh Windows PowerShell window through Windows search.
• Rerun the launcher-listing command above.

Continue when the list contains Python 3.13.

Prepare the isolated workspace

The .venv folder keeps this experiment's packages separate from every other Python project. Activating it makes later commands use the isolated interpreter automatically.

• Replace the minor version in the command below with 3.12 or 3.14 only if your supported interpreter uses that version.
• Create lidar-ppo-smoke with a Python 3.13 environment by running this PowerShell command:

mkdir lidar-ppo-smoke; cd lidar-ppo-smoke; py -3.13 -m venv .venv; .\.venv\Scripts\Activate.ps1

What does this command do?

• The mkdir lidar-ppo-smoke section creates the local project folder.
• The cd lidar-ppo-smoke section moves PowerShell into that folder.
• The py -3.13 -m venv .venv section creates the isolated environment.
• The .\.venv\Scripts\Activate.ps1 section activates that environment for the current PowerShell session.

• Check that the PowerShell prompt now names the .venv environment.

Your commands now use the isolated interpreter inside .venv.

Did the environment fail to activate?

• Confirm that you used a supported minor version in the environment command.
• Check that the prompt already ends inside the lidar-ppo-smoke folder.
• If PowerShell reports that script execution is blocked, resolve the local policy restriction before retrying the activation path.

Help me activate the project virtual environment in Windows PowerShell.

Visual Studio Code treats an opened folder as a workspace. Finding a new folder in the Windows picker can be fiddly because it sits inside your user folder.

• Press the Windows key to open Windows search.
• Type Visual Studio Code into the search field.
• Press Enter to open Visual Studio Code.
• Select File from the top menu.
• Select Open Folder....
• Choose lidar-ppo-smoke from your Windows user folder.
• Select Select Folder.
• Select Yes, I trust the authors if the Workspace Trust dialog appears.

You will see lidar-ppo-smoke at the top of the Explorer view. The .venv folder confirms that VS Code opened the correct workspace.

Can't find the workspace?

• Confirm that the folder name is exactly lidar-ppo-smoke.
• Return to the Windows user folder in the folder picker.
• Keep Windows PowerShell open so you can confirm that its prompt ends inside lidar-ppo-smoke.

Help me locate my lidar-ppo-smoke workspace in Visual Studio Code.

Pin the RL dependencies

The dependency manifest locks NumPy, Gymnasium, PyTorch, Stable-Baselines3, and pytest to the versions used by the complete racing harness. This prevents package drift from masquerading as a simulator or training bug.

• Select the New File icon in the Explorer view.
• Name the file requirements.txt.
• Fill requirements.txt by pasting the following package pins:

numpy==2.5.4
gymnasium==1.4.0
torch==2.14.1
stable-baselines3==2.9.2
pytest==9.2.0

What does this file control?

• NumPy provides the numeric arrays used for observations and actions.
• Gymnasium provides the custom environment contract.
• PyTorch provides the tensor runtime used by the policy.
• Stable-Baselines3 provides the PPO implementation used by the smoke experiment.
• pytest runs the environment contract tests.

• Press Ctrl+S to save requirements.txt.
• Check that the Explorer view lists requirements.txt inside lidar-ppo-smoke.

You will see the saved dependency file beside the .venv folder.

Don't see all five pins?

• Confirm that the filename is exactly requirements.txt.
• Compare each package name with the reference below.
• Remove any blank package lines containing stray spaces.

Help me compare my requirements.txt file with the five pinned dependencies.

✔️ Awesome, I've got everything!

Your requirements.txt manifest is ready for installation.

ⓧ I'd like to double check the full code

numpy==2.5.4
gymnasium==1.4.0
torch==2.14.1
stable-baselines3==2.9.2
pytest==9.2.0

Why these exact lines?

Each line fixes one dependency to the project version. Matching this file exactly makes later test results reproducible.

The installation may take several minutes while pip downloads PyTorch. A quiet pause during the download does not mean the process has stopped.

• Switch back to Windows PowerShell from earlier.
• Install the pinned packages by running this command:

python -m pip install -r requirements.txt

What does this command install?

The active Python interpreter runs pip against requirements.txt. Every package is installed inside .venv at its pinned version.

When the PowerShell prompt returns without an installation failure, all five dependencies are available to the isolated environment.

Did a package fail to install?

• Confirm that the PowerShell prompt still names the .venv environment.
• Confirm that PowerShell is inside lidar-ppo-smoke.
• Check that your Python interpreter is within the supported version range.
• Retry the installation after restoring any interrupted network connection.

Help me diagnose my pinned package installation failure.

The pinned reinforcement-learning stack is now installed inside .venv.

Why keep PPO on the CPU?

This project uses a compact multilayer PPO policy. Future model configurations explicitly use device="cpu" so the smoke runs follow the same execution path on computers with or without a GPU.

Before you check, which five version numbers do you expect the active environment to report?

• Verify every import by running the final setup check:

python -c "import torch, gymnasium, stable_baselines3, numpy, pytest; print(torch.__version__, gymnasium.__version__, stable_baselines3.__version__, numpy.__version__, pytest.__version__)"

What does this check prove?

• The import section confirms that the active interpreter can load every required package.
• The print section exposes the installed versions in a fixed order.

You should see 2.14.1 1.4.0 2.9.2 2.5.4 9.2.0 on one line. That's the setup locked down: your active environment imports the complete pinned stack successfully.

Do the versions differ?

• Confirm that .venv is still active in the current PowerShell window.
• Compare requirements.txt with the full-file reference above.
• Rerun the installation command after correcting any mismatched pin.

Help me fix a package version mismatch in my active virtual environment.

Your pinned CPU workspace is ready. Next, you will define the racing contract before trusting any policy training.

Build and Test the Racing Contract

Your pinned CPU workspace is ready. Now the racing simulator needs a contract that every future policy can rely on.

A PPO policy cannot be trusted until the geometry, hazards, observations, actions, lap counting, and episode boundaries are visible and tested. This step creates that evidence before any training begins.

In this step, get ready to:

• Document the observation, action, reward, and evaluation contracts.
• Build the racing environment with deterministic tests.
• Generate a visible preview of the circuit and trajectory.

Define the racing contract

A reward contract records the behaviour you want before reward values start shaping the policy. It also keeps behavioural evaluation separate from the proxy used during training.

• In the VS Code Explorer sidebar, create a file named reward_contract.md inside the lidar-ppo-smoke folder.
• Fill reward_contract.md by pasting this contract:

# Reward Contract

## Intended behaviour

Complete fast laps while staying inside track limits, avoiding walls, avoiding caused contacts, and making forward progress. Reward is a training proxy, not the evaluation target.

## Observation and action contract

The observation is a 33-value float vector: 24 normalized LiDAR-like ranges, speed, steering angle, heading error, lateral offset, three curvature look-ahead values, opponent gap, and opponent relative speed.

The action is `[steering_command, acceleration_command]`, with both values clipped to `[-1, 1]`.

## Reward v1: deliberately flawed smoke probe

- Pays for absolute centreline progress.
- Pays a lap bonus.
- Applies weak track-limit and collision penalties.

Known failure: moving forward and backward over the same section can collect positive proxy reward without completing a lap.

## Reward v2: training contract

- Pays signed centreline progress.
- Pays a lap-completion bonus.
- Penalizes reverse progress, wall proximity, leaving the track, collisions, and contacts caused while closing faster than the opponent.

## Behavioural evaluation

Do not use episode reward as the result. Report completion, lap time, wall contacts, track-limit violations, caused contacts, overtakes, and reverse steps. This project is a smoke experiment with fixed small budgets, not a performance claim.

What Does This Contract Protect?

• The observation section fixes the policy input at 33 values.
• The action section fixes steering and acceleration to two bounded values.
• The reward sections preserve the deliberately flawed first proxy beside its corrected replacement.
• The evaluation section makes driving behaviour the evidence for success.

• Save reward_contract.md.
• Confirm the file shows sections for intended behaviour, both reward profiles, and behavioural evaluation.

Contract Formatting Look Wrong?

Check that each heading begins with the same number of # characters shown above. Markdown preview problems do not change the contract text stored in the file.

Ask for help if the headings or lists are not rendering correctly: Help me check the Markdown structure in my reward_contract.md file.

Build the environment and tests

The environment implements the Gymnasium interaction contract. Each reset returns an observation plus information. Each step separates task termination from time-limit truncation.

What makes this F1TENTH-style?

The simulator uses a periodic spline-based Frenet frame around an asymmetric, counter-clockwise circuit. A uniform 2.0-unit ribbon carries the car through the main straight, chicane, carousel, hairpins, mountain, and outer loops.

Each of the 24 LiDAR-like beams intersects the complete inner and outer wall geometry. The resulting ranges change as the car approaches angled walls, blind apexes, and nearby folds in the circuit.

The pit lane and crossover lines are visual reference markings in the preview. The drivable racing surface remains one closed loop with no branches or dead ends.

Use this circuit plan as the visual reference for the asymmetric sectors, central mountain, parallel pit markings, and flowing outer loops.

• Create racing_env.py from the VS Code Explorer sidebar.

You should see racing_env.py beside requirements.txt and reward_contract.md.

• Paste the complete environment from the double-check tab into racing_env.py.
• Save racing_env.py.

✔️ Awesome, I've got everything!

Your saved environment now defines both tracks, the observation contract, traffic behaviour, hazards, rewards, and episode boundaries.

ⓧ I'd like to double check the full code

from __future__ import annotations

from dataclasses import dataclass
import math

import gymnasium as gym
import numpy as np
from gymnasium import spaces


def wrap_angle(angle: float) -> float:
    # Keep heading errors inside one signed revolution.
    return (angle + math.pi) % (2.0 * math.pi) - math.pi


@dataclass(frozen=True)
class TrackSpec:
    name: str
    control_points: tuple[tuple[float, float], ...]
    main_straight: tuple[tuple[float, float], tuple[float, float]]
    half_width: float = 1.0
    samples_per_segment: int = 18

    def __post_init__(self):
        # Convert the design points into a smooth, closed Catmull-Rom centreline.
        controls = np.asarray(self.control_points, dtype=np.float64)
        if controls.shape[0] < 8:
            raise ValueError("a circuit needs at least eight control points")

        samples = []
        tangents = []
        curvatures = []
        for index in range(len(controls)):
            p0 = controls[(index - 1) % len(controls)]
            p1 = controls[index]
            p2 = controls[(index + 1) % len(controls)]
            p3 = controls[(index + 2) % len(controls)]
            for t in np.linspace(0.0, 1.0, self.samples_per_segment, endpoint=False):
                t2 = t * t
                t3 = t2 * t
                position = 0.5 * (
                    2.0 * p1
                    + (-p0 + p2) * t
                    + (2.0 * p0 - 5.0 * p1 + 4.0 * p2 - p3) * t2
                    + (-p0 + 3.0 * p1 - 3.0 * p2 + p3) * t3
                )
                first = 0.5 * (
                    -p0
                    + p2
                    + 2.0 * (2.0 * p0 - 5.0 * p1 + 4.0 * p2 - p3) * t
                    + 3.0 * (-p0 + 3.0 * p1 - 3.0 * p2 + p3) * t2
                )
                second = 0.5 * (
                    2.0 * (2.0 * p0 - 5.0 * p1 + 4.0 * p2 - p3)
                    + 6.0 * (-p0 + 3.0 * p1 - 3.0 * p2 + p3) * t
                )
                speed = max(float(np.linalg.norm(first)), 1e-9)
                samples.append(position)
                tangents.append(first / speed)
                curvatures.append(
                    float((first[0] * second[1] - first[1] * second[0]) / speed**3)
                )

        samples_array = np.asarray(samples, dtype=np.float64)
        tangents_array = np.asarray(tangents, dtype=np.float64)
        curvature_array = np.asarray(curvatures, dtype=np.float64)
        segment_vectors = np.roll(samples_array, -1, axis=0) - samples_array
        segment_lengths = np.linalg.norm(segment_vectors, axis=1)
        cumulative = np.concatenate(([0.0], np.cumsum(segment_lengths)))

        object.__setattr__(self, "_samples", samples_array)
        object.__setattr__(self, "_tangents", tangents_array)
        object.__setattr__(self, "_curvatures", curvature_array)
        object.__setattr__(self, "_segment_vectors", segment_vectors)
        object.__setattr__(self, "_segment_lengths", segment_lengths)
        object.__setattr__(self, "_cumulative", cumulative)
        object.__setattr__(self, "_lap_length", float(cumulative[-1]))

        # Cache both walls for fast global LiDAR ray intersections.
        outer = self.sample(self.half_width, len(samples_array))
        inner = self.sample(-self.half_width, len(samples_array))
        wall_starts = np.concatenate((outer, inner), axis=0)
        wall_ends = np.concatenate(
            (np.roll(outer, -1, axis=0), np.roll(inner, -1, axis=0)), axis=0
        )
        wall_midpoints = 0.5 * (wall_starts + wall_ends)
        wall_half_lengths = 0.5 * np.linalg.norm(wall_ends - wall_starts, axis=1)
        all_edges = np.concatenate((outer, inner), axis=0)
        bounds = (
            float(all_edges[:, 0].min()),
            float(all_edges[:, 0].max()),
            float(all_edges[:, 1].min()),
            float(all_edges[:, 1].max()),
        )
        object.__setattr__(self, "_wall_starts", wall_starts)
        object.__setattr__(self, "_wall_ends", wall_ends)
        object.__setattr__(self, "_wall_midpoints", wall_midpoints)
        object.__setattr__(self, "_wall_half_lengths", wall_half_lengths)
        object.__setattr__(self, "_bounds", bounds)

    @property
    def lap_length(self) -> float:
        return self._lap_length

    @property
    def width(self) -> float:
        return 2.0 * self.half_width

    @property
    def main_straight_length(self) -> float:
        start = np.asarray(self.main_straight[0], dtype=np.float64)
        finish = np.asarray(self.main_straight[1], dtype=np.float64)
        return float(np.linalg.norm(finish - start))

    @property
    def bounds(self) -> tuple[float, float, float, float]:
        return self._bounds

    def geometry(self, progress: float, offset: float = 0.0):
        # Interpolate position, tangent, and signed curvature by arc length.
        wrapped = progress % self.lap_length
        index = int(np.searchsorted(self._cumulative, wrapped, side="right") - 1)
        index = min(index, len(self._samples) - 1)
        length = max(float(self._segment_lengths[index]), 1e-9)
        fraction = (wrapped - self._cumulative[index]) / length
        next_index = (index + 1) % len(self._samples)

        center = self._samples[index] + fraction * self._segment_vectors[index]
        tangent = (
            (1.0 - fraction) * self._tangents[index]
            + fraction * self._tangents[next_index]
        )
        tangent /= max(float(np.linalg.norm(tangent)), 1e-9)
        normal = np.array([-tangent[1], tangent[0]], dtype=np.float64)
        position = center + normal * offset
        heading = math.atan2(tangent[1], tangent[0])
        curvature = float(
            (1.0 - fraction) * self._curvatures[index]
            + fraction * self._curvatures[next_index]
        )
        return position, heading, curvature

    def sample(self, offset: float = 0.0, count: int = 360) -> np.ndarray:
        # Return equally spaced arc-length samples for tests and rendering.
        progress_values = np.linspace(0.0, self.lap_length, count, endpoint=False)
        return np.asarray(
            [self.geometry(progress, offset)[0] for progress in progress_values],
            dtype=np.float64,
        )

    def ray_distance(
        self, origin: np.ndarray, direction: np.ndarray, max_range: float
    ) -> float:
        # Intersect one LiDAR ray with nearby inner and outer wall segments.
        midpoint_distance = np.linalg.norm(self._wall_midpoints - origin, axis=1)
        nearby = midpoint_distance - self._wall_half_lengths <= max_range
        starts = self._wall_starts[nearby]
        segments = self._wall_ends[nearby] - starts
        relative = starts - origin
        denominator = direction[0] * segments[:, 1] - direction[1] * segments[:, 0]
        valid_denominator = np.abs(denominator) > 1e-9

        safe_denominator = np.where(valid_denominator, denominator, 1.0)
        ray_t = (
            relative[:, 0] * segments[:, 1] - relative[:, 1] * segments[:, 0]
        ) / safe_denominator
        segment_u = (
            relative[:, 0] * direction[1] - relative[:, 1] * direction[0]
        ) / safe_denominator
        valid = (
            valid_denominator
            & (ray_t > 1e-4)
            & (ray_t <= max_range)
            & (segment_u >= 0.0)
            & (segment_u <= 1.0)
        )
        if not np.any(valid):
            return max_range
        return float(np.min(ray_t[valid]))


# The bottom control points define a 15-unit start/finish straight.
TRAIN_POINTS = (
    (-7.5, -6.0),
    (0.0, -6.0),
    (7.5, -6.0),
    (10.5, -5.5),
    (13.0, -3.0),
    (13.5, 1.0),
    (13.0, 4.5),
    (11.0, 6.5),
    (8.0, 7.0),
    (6.5, 6.0),
    (6.0, 4.3),
    (7.0, 3.2),
    (9.0, 3.0),
    (10.2, 1.5),
    (10.0, -0.5),
    (8.8, -2.2),
    (7.2, -2.7),
    (5.8, -1.5),
    (2.0, 2.2),
    (0.0, 5.5),
    (-2.0, 2.0),
    (-3.2, -2.8),
    (-5.0, -3.8),
    (-7.2, -3.5),
    (-8.6, -2.2),
    (-8.7, -0.8),
    (-7.5, 0.2),
    (-5.7, 0.3),
    (-4.5, 1.2),
    (-4.6, 2.4),
    (-6.0, 3.1),
    (-8.0, 3.0),
    (-9.2, 3.8),
    (-9.0, 5.5),
    (-10.5, 7.0),
    (-13.0, 6.5),
    (-14.5, 4.5),
    (-15.0, 1.5),
    (-14.8, -2.5),
    (-13.0, -5.0),
    (-10.0, -6.0),
)

# The held-out circuit keeps the same sectors with shifted radii and apex positions.
HELDOUT_POINTS = tuple(
    (
        0.97 * x + 0.18 * math.sin(index * 0.73),
        0.94 * y + 0.15 * math.cos(index * 0.61),
    )
    for index, (x, y) in enumerate(TRAIN_POINTS)
)

TRACKS = {
    "train": TrackSpec(
        "train",
        control_points=TRAIN_POINTS,
        main_straight=((-7.5, -6.0), (7.5, -6.0)),
        half_width=1.0,
    ),
    "heldout": TrackSpec(
        "heldout",
        control_points=HELDOUT_POINTS,
        main_straight=((-7.5, -5.49), (7.5, -5.49)),
        half_width=1.0,
    ),
}


class RacingEnv(gym.Env):
    metadata = {"render_modes": []}

    BEAMS = 24
    MAX_RANGE = 8.0
    MAX_SPEED = 4.0
    MIN_SPEED = -0.8
    MAX_STEER = 0.45
    WHEELBASE = 0.33
    ACCELERATION = 2.5
    DT = 0.1

    def __init__(
        self,
        *,
        traffic: bool = False,
        opponent_style: str = "steady",
        reward_profile: str = "v2",
        track: str = "train",
        max_steps: int = 800,
    ):
        super().__init__()
        if reward_profile not in {"v1", "v2"}:
            raise ValueError("reward_profile must be 'v1' or 'v2'")
        if track not in TRACKS:
            raise ValueError(f"unknown track: {track}")
        if opponent_style not in {"steady", "weaver"}:
            raise ValueError("opponent_style must be 'steady' or 'weaver'")

        # Keep the learning contract independent from the selected circuit.
        self.traffic = traffic
        self.opponent_style = opponent_style
        self.reward_profile = reward_profile
        self.track = TRACKS[track]
        self.max_steps = max_steps
        self.action_space = spaces.Box(-1.0, 1.0, shape=(2,), dtype=np.float32)
        self.observation_space = spaces.Box(-1.0, 1.0, shape=(33,), dtype=np.float32)
        self.beam_angles = np.linspace(
            -0.75 * math.pi, 0.75 * math.pi, self.BEAMS, dtype=np.float64
        )

    def reset(self, *, seed: int | None = None, options=None):
        super().reset(seed=seed)
        # Reset race state while retaining deterministic seeded offsets.
        self.s_total = 0.0
        self.lateral_offset = float(self.np_random.uniform(-0.05, 0.05))
        self.heading_error = float(self.np_random.uniform(-0.03, 0.03))
        self.speed = 1.0
        self.steering = 0.0
        self.step_count = 0
        self.wall_contacts = 0
        self.track_limit_violations = 0
        self.caused_contacts = 0
        self.overtakes = 0
        self.reverse_steps = 0
        self.previous_wall = False
        self.previous_out = False
        self.previous_race_difference = -0.25 * self.track.lap_length
        self.opponent_total = 0.25 * self.track.lap_length
        self.opponent_speed = 1.7
        self.last_lap_time = None
        return self._observation(), self._info(completion=False)

    def ego_position(self) -> np.ndarray:
        return self.track.geometry(self.s_total, self.lateral_offset)[0]

    def _opponent_state(self):
        # Scripted traffic either holds its line or weaves across the ribbon.
        if self.opponent_style == "steady":
            speed = 1.7
            offset = 0.0
        else:
            speed = 1.55 + 0.30 * math.sin(0.08 * self.step_count)
            offset = 0.28 * math.sin(0.18 * self.opponent_total)
        position = self.track.geometry(self.opponent_total, offset)[0]
        return position, speed, offset

    def _lidar(self) -> np.ndarray:
        # Cast each beam against the complete circuit so angled walls affect range.
        ego_position, track_heading, _ = self.track.geometry(
            self.s_total, self.lateral_offset
        )
        car_yaw = track_heading + self.heading_error
        ranges = np.full(self.BEAMS, self.MAX_RANGE, dtype=np.float64)
        for index, beam in enumerate(self.beam_angles):
            direction = np.array(
                [math.cos(car_yaw + beam), math.sin(car_yaw + beam)],
                dtype=np.float64,
            )
            ranges[index] = self.track.ray_distance(
                ego_position, direction, self.MAX_RANGE
            )

        # Treat the opponent as a circle that can occlude a LiDAR beam.
        if self.traffic:
            opponent_position, _, _ = self._opponent_state()
            relative = opponent_position - ego_position
            for index, beam in enumerate(self.beam_angles):
                direction = np.array(
                    [math.cos(car_yaw + beam), math.sin(car_yaw + beam)]
                )
                projection = float(np.dot(relative, direction))
                perpendicular_sq = float(np.dot(relative, relative) - projection**2)
                radius = 0.24
                if projection > 0.0 and perpendicular_sq <= radius**2:
                    distance = projection - math.sqrt(
                        max(0.0, radius**2 - perpendicular_sq)
                    )
                    if distance > 0.0:
                        ranges[index] = min(ranges[index], distance)

        return np.clip(ranges, 0.0, self.MAX_RANGE).astype(np.float32)

    def _observation(self) -> np.ndarray:
        # Combine LiDAR, vehicle state, future curvature, and traffic state.
        curvature_values = []
        for lookahead in (0.0, 4.0, 8.0):
            curvature = self.track.geometry(self.s_total + lookahead)[2]
            curvature_values.append(float(np.clip(curvature / 0.35, -1.0, 1.0)))

        if self.traffic:
            gap = (self.opponent_total - self.s_total) % self.track.lap_length
            _, opponent_speed, _ = self._opponent_state()
            relative_speed = (opponent_speed - self.speed) / self.MAX_SPEED
        else:
            gap = self.track.lap_length
            relative_speed = 0.0

        observation = np.concatenate(
            [
                self._lidar() / self.MAX_RANGE,
                np.array(
                    [
                        self.speed / self.MAX_SPEED,
                        self.steering / self.MAX_STEER,
                        self.heading_error / math.pi,
                        self.lateral_offset / self.track.half_width,
                    ],
                    dtype=np.float32,
                ),
                np.asarray(curvature_values, dtype=np.float32),
                np.array(
                    [gap / self.track.lap_length, relative_speed], dtype=np.float32
                ),
            ]
        )
        return np.clip(observation, -1.0, 1.0).astype(np.float32)

    def _hazards(self):
        # Track limits remain tied to the uniform two-unit racing ribbon.
        wall = abs(self.lateral_offset) >= self.track.half_width - 0.12
        out = abs(self.lateral_offset) > self.track.half_width
        collision = False
        caused = False
        if self.traffic:
            opponent_position, opponent_speed, _ = self._opponent_state()
            collision = float(np.linalg.norm(self.ego_position() - opponent_position)) < 0.42
            caused = collision and self.speed > opponent_speed + 0.2
        return wall, out, collision, caused

    def _reward(
        self,
        delta_progress: float,
        lap_event: bool,
        wall: bool,
        out: bool,
        collision: bool,
        caused: bool,
    ):
        # Preserve the flawed v1 proxy beside the corrected v2 contract.
        if self.reward_profile == "v1":
            terms = {
                "progress": 4.0 * abs(delta_progress),
                "lap": 25.0 * float(lap_event),
                "wall": 0.0,
                "out": -2.0 * float(out),
                "collision": -2.0 * float(collision),
                "caused": 0.0,
                "reverse": 0.0,
            }
        else:
            terms = {
                "progress": 4.0 * delta_progress,
                "lap": 25.0 * float(lap_event),
                "wall": -1.5 * float(wall),
                "out": -20.0 * float(out),
                "collision": -25.0 * float(collision),
                "caused": -10.0 * float(caused),
                "reverse": -2.0 * float(delta_progress < 0.0),
            }
        return float(sum(terms.values())), terms

    def step(self, action):
        # Apply bounded steering and acceleration in the local Frenet frame.
        action = np.clip(np.asarray(action, dtype=np.float32), -1.0, 1.0)
        target_steering = float(action[0]) * self.MAX_STEER
        self.steering = 0.7 * self.steering + 0.3 * target_steering
        self.speed = float(
            np.clip(
                self.speed + float(action[1]) * self.ACCELERATION * self.DT,
                self.MIN_SPEED,
                self.MAX_SPEED,
            )
        )

        _, _, curvature = self.track.geometry(self.s_total)
        denominator = max(0.5, 1.0 - curvature * self.lateral_offset)
        longitudinal_rate = self.speed * math.cos(self.heading_error) / denominator
        lateral_rate = self.speed * math.sin(self.heading_error)
        yaw_rate = self.speed * math.tan(self.steering) / self.WHEELBASE

        previous_progress = self.s_total
        previous_laps = math.floor(max(0.0, previous_progress) / self.track.lap_length)
        self.s_total += longitudinal_rate * self.DT
        self.lateral_offset += lateral_rate * self.DT
        self.heading_error = wrap_angle(
            self.heading_error
            + (yaw_rate - curvature * longitudinal_rate) * self.DT
        )
        delta_progress = self.s_total - previous_progress
        current_laps = math.floor(max(0.0, self.s_total) / self.track.lap_length)
        lap_event = current_laps > previous_laps

        if self.traffic:
            _, self.opponent_speed, _ = self._opponent_state()
            self.opponent_total += self.opponent_speed * self.DT

        # Record behavioural metrics independently from episode reward.
        self.step_count += 1
        wall, out, collision, caused = self._hazards()
        if wall and not self.previous_wall:
            self.wall_contacts += 1
        if out and not self.previous_out:
            self.track_limit_violations += 1
        if caused:
            self.caused_contacts += 1
        if delta_progress < -1e-4:
            self.reverse_steps += 1

        if self.traffic:
            difference = self.s_total - self.opponent_total
            if self.previous_race_difference <= 0.0 < difference:
                self.overtakes += 1
            self.previous_race_difference = difference

        reward, reward_terms = self._reward(
            delta_progress, lap_event, wall, out, collision, caused
        )
        terminated = bool(lap_event or out or collision)
        truncated = bool(self.step_count >= self.max_steps and not terminated)
        completion = bool(lap_event)
        if completion:
            self.last_lap_time = self.step_count * self.DT

        self.previous_wall = wall
        self.previous_out = out
        info = self._info(completion=completion)
        info["reward_terms"] = reward_terms
        return self._observation(), reward, terminated, truncated, info

    def _info(self, *, completion: bool):
        # Expose evaluation evidence without using the reward total as a result.
        return {
            "completion": completion,
            "lap_time": self.last_lap_time,
            "wall_contacts": self.wall_contacts,
            "track_limit_violations": self.track_limit_violations,
            "caused_contacts": self.caused_contacts,
            "overtakes": self.overtakes,
            "reverse_steps": self.reverse_steps,
            "distance": self.s_total,
        }

How is the environment organised?

• The track specification samples a closed spline by arc length and returns positions, headings, signed curvature, and constant-width boundaries.
• The LiDAR model casts each beam against nearby segments from both circuit walls.
• The environment builds the 33-value observation from range readings, vehicle state, future curvature, and opponent state.
• The step logic updates motion before checking walls, track limits, collisions, laps, and time limits.
• The information dictionary exposes behavioural metrics without depending on episode reward.

• Create a tests folder from the VS Code Explorer sidebar.

You should see the new tests folder inside lidar-ppo-smoke.

• Create test_racing_env.py inside the tests folder.
• Paste the complete test suite from the double-check tab into tests/test_racing_env.py.

✔️ Awesome, I've got everything!

Your test suite now checks the learning contracts, deterministic resets, circuit geometry, the non-intersecting centreline, LiDAR ranges, hazards, reward behaviour, lap termination, and time-limit truncation.

ⓧ I'd like to double check the full code

import numpy as np
import pytest

from racing_env import RacingEnv, TRACKS


def segments_intersect(a, b, c, d):
    # Detect proper crossings while allowing neighbouring segments to share endpoints.
    first = b - a
    second = d - c
    denominator = first[0] * second[1] - first[1] * second[0]
    if abs(denominator) < 1e-9:
        return False
    relative = c - a
    first_fraction = (
        relative[0] * second[1] - relative[1] * second[0]
    ) / denominator
    second_fraction = (
        relative[0] * first[1] - relative[1] * first[0]
    ) / denominator
    return 1e-6 < first_fraction < 1.0 - 1e-6 and 1e-6 < second_fraction < 1.0 - 1e-6


def assert_simple_closed_polyline(points):
    # Compare every non-neighbouring segment to enforce one continuous loop.
    segment_count = len(points)
    for first_index in range(segment_count):
        first_next = (first_index + 1) % segment_count
        for second_index in range(first_index + 1, segment_count):
            second_next = (second_index + 1) % segment_count
            neighbours = {
                first_index,
                first_next,
            } & {
                second_index,
                second_next,
            }
            if neighbours:
                continue
            assert not segments_intersect(
                points[first_index],
                points[first_next],
                points[second_index],
                points[second_next],
            )


def test_observation_and_action_contract():
    env = RacingEnv()
    observation, _ = env.reset(seed=7)
    assert observation.shape == (33,)
    assert observation.dtype == np.float32
    assert env.observation_space.contains(observation)
    assert env.action_space.shape == (2,)


def test_reset_is_seeded():
    first = RacingEnv()
    second = RacingEnv()
    observation_a, _ = first.reset(seed=42)
    observation_b, _ = second.reset(seed=42)
    assert np.allclose(observation_a, observation_b)


def test_track_geometry_contract():
    track = TRACKS["train"]
    assert track.width == pytest.approx(2.0)
    assert track.main_straight_length == pytest.approx(15.0)
    assert np.allclose(track.geometry(0.0)[0], track.geometry(track.lap_length)[0])

    # Check uniform width and alternating turn directions around the complete lap.
    progress_values = np.linspace(0.0, track.lap_length, 64, endpoint=False)
    curvatures = []
    for progress in progress_values:
        left = track.geometry(progress, track.half_width)[0]
        right = track.geometry(progress, -track.half_width)[0]
        assert np.linalg.norm(left - right) == pytest.approx(2.0, abs=1e-6)
        curvatures.append(track.geometry(progress)[2])
    assert max(curvatures) > 0.03
    assert min(curvatures) < -0.03


def test_track_has_no_self_intersections():
    track = TRACKS["train"]
    assert_simple_closed_polyline(track.sample(count=180))


def test_lidar_reads_complex_wall_angles():
    env = RacingEnv()
    env.reset(seed=5)
    ranges = env._lidar()
    assert ranges.shape == (24,)
    assert np.all(ranges > 0.0)
    assert np.all(ranges <= env.MAX_RANGE)
    assert np.ptp(ranges) > 0.1


def test_track_limit_and_collision_logic():
    solo = RacingEnv()
    solo.reset(seed=1)
    solo.lateral_offset = solo.track.half_width + 0.01
    _, out, collision, _ = solo._hazards()
    assert out
    assert not collision

    traffic = RacingEnv(traffic=True)
    traffic.reset(seed=1)
    traffic.opponent_total = traffic.s_total
    _, _, collision, _ = traffic._hazards()
    assert collision


def test_reward_terms_remove_round_trip_exploit():
    v1 = RacingEnv(reward_profile="v1")
    v2 = RacingEnv(reward_profile="v2")
    v1_round_trip = sum(
        v1._reward(delta, False, False, False, False, False)[0]
        for delta in (0.1, -0.1)
    )
    v2_round_trip = sum(
        v2._reward(delta, False, False, False, False, False)[0]
        for delta in (0.1, -0.1)
    )
    assert v1_round_trip > 0.0
    assert v2_round_trip < 0.0


def test_lap_completion_terminates():
    env = RacingEnv()
    env.reset(seed=2)
    env.s_total = env.track.lap_length - 0.02
    env.speed = 1.5
    _, _, terminated, truncated, info = env.step(np.array([0.0, 0.0]))
    assert terminated
    assert not truncated
    assert info["completion"]
    assert info["lap_time"] is not None


def test_time_limit_truncates_without_termination():
    env = RacingEnv(max_steps=1)
    env.reset(seed=3)
    env.speed = 0.0
    _, _, terminated, truncated, info = env.step(np.array([0.0, 0.0]))
    assert not terminated
    assert truncated
    assert not info["completion"]

What does the test suite cover?

The nine tests turn the learning and circuit contracts into executable checks. A future edit fails quickly if it changes the input shape, action shape, seeded reset, uniform width, closed centreline, LiDAR readings, hazard logic, reward behaviour, or episode boundaries.

• Save tests/test_racing_env.py.

Before you run the suite, do you expect all nine checks to agree with the new circuit and learning contracts?

• Test the environment from the activated PowerShell terminal by running:

python -m pytest

What Does This Command Prove?

The active Python interpreter invokes pytest against the tests in this workspace. Each passing test confirms one part of the environment contract.

You should see all nine tests pass. That proves the observation, action, seeded reset, geometry, LiDAR, hazard, reward, termination, and truncation checks agree with the simulator.

Tests Not Passing?

• Confirm test_racing_env.py is inside the tests folder.
• Check that racing_env.py is beside requirements.txt.
• Compare the failing test with the matching method in RacingEnv.

Use this prompt to investigate the first failure: Help me diagnose my first failing racing environment test.

Add and run the visual preview

Passing tests establish the contract at specific checkpoints. The preview adds a visible trajectory so you can inspect the circuit boundaries and confirm that the environment drives.

• Create experiment.py from the VS Code Explorer sidebar.

You should see experiment.py beside racing_env.py.

• Paste the complete experiment runner from the double-check tab into experiment.py.
• Save experiment.py.

✔️ Awesome, I've got everything!

Your experiment runner now contains preview, exploit, solo training, traffic training, animated demo, and held-out evaluation commands.

ⓧ I'd like to double check the full code

from __future__ import annotations

import argparse
import csv
import json
import math
from pathlib import Path
from statistics import mean, pstdev

import numpy as np
import torch as th
from stable_baselines3 import PPO
from stable_baselines3.common.env_checker import check_env

from racing_env import RacingEnv, TRACKS

ARTIFACTS = Path("artifacts")
TRAIN_SEEDS = [11, 22, 33]
EVAL_SEEDS = [101, 202, 303]
SOLO_STEPS = 20_000
TRAFFIC_STEPS = 10_000


def make_model(env: RacingEnv, seed: int) -> PPO:
    # Keep the actor and critic small enough for reproducible CPU smoke runs.
    policy_kwargs = dict(
        activation_fn=th.nn.Tanh,
        net_arch=dict(pi=[64, 64], vf=[64, 64]),
    )
    return PPO(
        "MlpPolicy",
        env,
        learning_rate=3e-4,
        n_steps=1024,
        batch_size=64,
        n_epochs=10,
        gamma=0.99,
        gae_lambda=0.95,
        clip_range=0.2,
        policy_kwargs=policy_kwargs,
        seed=seed,
        device="cpu",
        verbose=0,
    )


def pure_pursuit_action(env: RacingEnv) -> np.ndarray:
    # Add curvature feed-forward so the baseline can enter tight corners early.
    _, _, curvature = env.track.geometry(env.s_total + 1.2)
    feed_forward = math.atan(env.WHEELBASE * curvature)
    steering = np.clip(
        (feed_forward - 1.4 * env.heading_error - 0.8 * env.lateral_offset)
        / env.MAX_STEER,
        -1.0,
        1.0,
    )
    target_speed = float(np.clip(2.8 / (1.0 + 5.0 * abs(curvature)), 1.0, 2.6))
    acceleration = np.clip((target_speed - env.speed) / 1.5, -1.0, 1.0)
    return np.array([steering, acceleration], dtype=np.float32)


def rollout(
    label: str,
    *,
    model: PPO | None,
    seed: int,
    traffic: bool,
    opponent_style: str,
    track: str,
):
    # Run one episode and keep behavioural evidence plus the driven trace.
    env = RacingEnv(
        traffic=traffic,
        opponent_style=opponent_style,
        reward_profile="v2",
        track=track,
    )
    observation, info = env.reset(seed=seed)
    trace = [env.ego_position().copy()]
    terminated = truncated = False
    while not (terminated or truncated):
        if label == "pure_pursuit":
            action = pure_pursuit_action(env)
        else:
            action, _ = model.predict(observation, deterministic=True)
        observation, _, terminated, truncated, info = env.step(action)
        trace.append(env.ego_position().copy())
    row = {
        "policy": label,
        "seed": seed,
        "completion": int(info["completion"]),
        "lap_time": "" if info["lap_time"] is None else info["lap_time"],
        "wall_contacts": info["wall_contacts"],
        "track_limit_violations": info["track_limit_violations"],
        "caused_contacts": info["caused_contacts"],
        "overtakes": info["overtakes"],
        "reverse_steps": info["reverse_steps"],
        "distance": info["distance"],
    }
    env.close()
    return row, trace


def evaluate(
    label: str,
    model: PPO | None,
    *,
    traffic: bool,
    opponent_style: str,
    track: str,
):
    # Repeat evaluation across fixed seeds without changing the training budget.
    rows = []
    first_trace = None
    for seed in EVAL_SEEDS:
        row, trace = rollout(
            label,
            model=model,
            seed=seed,
            traffic=traffic,
            opponent_style=opponent_style,
            track=track,
        )
        rows.append(row)
        if first_trace is None:
            first_trace = trace
    return rows, first_trace


def write_csv(path: Path, rows: list[dict]):
    # Store the same behavioural columns for every policy and circuit.
    path.parent.mkdir(parents=True, exist_ok=True)
    fields = [
        "policy",
        "seed",
        "completion",
        "lap_time",
        "wall_contacts",
        "track_limit_violations",
        "caused_contacts",
        "overtakes",
        "reverse_steps",
        "distance",
    ]
    with path.open("w", newline="", encoding="utf-8") as handle:
        writer = csv.DictWriter(handle, fieldnames=fields)
        writer.writeheader()
        writer.writerows(rows)


def print_summary(rows: list[dict]):
    # Print smoke statistics without presenting them as final performance claims.
    print("Smoke metrics, not final results")
    labels = sorted({row["policy"] for row in rows})
    for label in labels:
        selected = [row for row in rows if row["policy"] == label]
        completions = [float(row["completion"]) for row in selected]
        violations = [float(row["track_limit_violations"]) for row in selected]
        print(
            f"{label}: completion={mean(completions):.3f} +/- {pstdev(completions):.3f}, "
            f"track_limits={mean(violations):.3f} +/- {pstdev(violations):.3f}"
        )


def circuit_features(track):
    # Build reusable geometry for static and animated circuit previews.
    center = track.sample(0.0, 480)
    outer = track.sample(track.half_width, 480)
    inner = track.sample(-track.half_width, 480)

    start = np.asarray(track.main_straight[0], dtype=np.float64)
    finish = np.asarray(track.main_straight[1], dtype=np.float64)
    straight = finish - start
    direction = straight / np.linalg.norm(straight)
    outside = np.array([direction[1], -direction[0]], dtype=np.float64)
    entry = start + 0.16 * straight
    exit_point = start + 0.84 * straight
    pit = np.asarray(
        [
            entry,
            entry + 1.45 * track.half_width * outside,
            exit_point + 1.45 * track.half_width * outside,
            exit_point,
        ]
    )

    # Alternate curb colours only where signed curvature identifies an inner apex.
    curbs = []
    curb_count = 420
    for index in range(curb_count):
        first_progress = index * track.lap_length / curb_count
        second_progress = (index + 1) * track.lap_length / curb_count
        _, _, curvature = track.geometry(0.5 * (first_progress + second_progress))
        if abs(curvature) < 0.075:
            continue
        inner_offset = math.copysign(track.half_width, curvature)
        first = track.geometry(first_progress, inner_offset)[0]
        second = track.geometry(second_progress, inner_offset)[0]
        colour = "#d62828" if index % 2 == 0 else "#f7f7f2"
        curbs.append((first, second, colour))

    start_finish = np.asarray(
        [
            track.geometry(0.0, -track.half_width)[0],
            track.geometry(0.0, track.half_width)[0],
        ]
    )
    return center, outer, inner, pit, curbs, start_finish


def write_svg(path: Path, trace: list[np.ndarray], track_name: str):
    # Render arbitrary spline bounds instead of assuming an ellipse.
    track = TRACKS[track_name]
    width, height, padding = 1100, 680, 42
    minimum_x, maximum_x, minimum_y, maximum_y = track.bounds
    margin = 1.8
    minimum_x -= margin
    maximum_x += margin
    minimum_y -= margin
    maximum_y += margin
    scale = min(
        (width - 2 * padding) / (maximum_x - minimum_x),
        (height - 2 * padding) / (maximum_y - minimum_y),
    )

    def screen(point):
        return (
            padding + (point[0] - minimum_x) * scale,
            height - padding - (point[1] - minimum_y) * scale,
        )

    def polyline(points, closed=False):
        rendered = list(points)
        if closed:
            rendered.append(points[0])
        return " ".join(f"{x:.1f},{y:.1f}" for x, y in map(screen, rendered))

    center, outer, inner, pit, curbs, start_finish = circuit_features(track)
    asphalt_width = 2.0 * track.half_width * scale
    curb_svg = "\n".join(
        f'<line x1="{screen(first)[0]:.1f}" y1="{screen(first)[1]:.1f}" '
        f'x2="{screen(second)[0]:.1f}" y2="{screen(second)[1]:.1f}" '
        f'stroke="{colour}" stroke-width="7"/>'
        for first, second, colour in curbs
    )
    start_a, start_b = map(screen, start_finish)

    path.parent.mkdir(parents=True, exist_ok=True)
    svg = f"""<svg xmlns="http://www.w3.org/2000/svg" width="{width}" height="{height}" viewBox="0 0 {width} {height}">
<rect width="100%" height="100%" fill="#0b3d25"/>
<polyline points="{polyline(center, closed=True)}" fill="none" stroke="#3f464d" stroke-width="{asphalt_width:.1f}" stroke-linejoin="round" stroke-linecap="round"/>
<polyline points="{polyline(outer, closed=True)}" fill="none" stroke="#d8dee9" stroke-width="3"/>
<polyline points="{polyline(inner, closed=True)}" fill="none" stroke="#d8dee9" stroke-width="3"/>
{curb_svg}
<polyline points="{polyline(pit)}" fill="none" stroke="#f4d35e" stroke-width="3" stroke-dasharray="10 7"/>
<line x1="{start_a[0]:.1f}" y1="{start_a[1]:.1f}" x2="{start_b[0]:.1f}" y2="{start_b[1]:.1f}" stroke="white" stroke-width="6" stroke-dasharray="7 7"/>
<polyline points="{polyline(trace)}" fill="none" stroke="#ff5a5f" stroke-width="3"/>
<text x="25" y="35" fill="white" font-family="sans-serif" font-size="20">Complex circuit smoke trajectory: {track.name}</text>
</svg>"""
    path.write_text(svg, encoding="utf-8")


def demo_frame(env: RacingEnv) -> dict:
    # Capture positions and headings so the browser can draw both cars.
    ego_position, track_heading, _ = env.track.geometry(
        env.s_total, env.lateral_offset
    )
    opponent_position, _, opponent_offset = env._opponent_state()
    _, opponent_heading, _ = env.track.geometry(env.opponent_total, opponent_offset)
    return {
        "ego": [*ego_position.tolist(), track_heading + env.heading_error],
        "opponent": [*opponent_position.tolist(), opponent_heading],
    }


def collect_demo(model: PPO, seed: int):
    # Replay one deterministic evaluation with the trained car and traffic.
    env = RacingEnv(
        traffic=True,
        opponent_style="weaver",
        reward_profile="v2",
        track="train",
    )
    observation, info = env.reset(seed=seed)
    frames = [demo_frame(env)]
    terminated = truncated = False
    while not (terminated or truncated):
        action, _ = model.predict(observation, deterministic=True)
        observation, _, terminated, truncated, info = env.step(action)
        frames.append(demo_frame(env))
    env.close()
    return frames, info


def write_demo_html(
    path: Path,
    frames: list[dict],
    info: dict,
    checkpoint: str,
):
    # Package the measured rollout and complete circuit styling into one page.
    track = TRACKS["train"]
    center, outer, inner, pit, curbs, start_finish = circuit_features(track)
    minimum_x, maximum_x, minimum_y, maximum_y = track.bounds
    payload = json.dumps(
        {
            "center": center.tolist(),
            "outer": outer.tolist(),
            "inner": inner.tolist(),
            "pit": pit.tolist(),
            "curbs": [
                [first.tolist(), second.tolist(), colour]
                for first, second, colour in curbs
            ],
            "startFinish": start_finish.tolist(),
            "frames": frames,
            "checkpoint": checkpoint,
            "completion": bool(info["completion"]),
            "bounds": [minimum_x, maximum_x, minimum_y, maximum_y],
            "trackWidth": track.width,
            "intervalMs": 50,
        }
    )
    html = """<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Traffic-trained PPO replay</title>
<style>
body { margin: 0; background: #0b3d25; color: #f5f7fa; font-family: system-ui, sans-serif; text-align: center; }
canvas { width: min(96vw, 1100px); height: auto; border: 1px solid #313943; background: #0b3d25; }
.legend { display: flex; justify-content: center; gap: 24px; margin: 8px; }
.ego { color: #ff5a5f; } .traffic { color: #56b4e9; }
</style>
</head>
<body>
<h1>Traffic-trained PPO replay</h1>
<p id="status"></p>
<div class="legend"><span class="ego">Trained PPO car</span><span class="traffic">Scripted traffic</span></div>
<canvas width="1100" height="680"></canvas>
<script>
const data = __DATA__;
const canvas = document.querySelector("canvas");
const ctx = canvas.getContext("2d");
const padding = 42;
const [minX, maxX, minY, maxY] = data.bounds;
const scale = Math.min((canvas.width - 2 * padding) / (maxX - minX + 3.6), (canvas.height - 2 * padding) / (maxY - minY + 3.6));
const screen = p => [padding + (p[0] - minX + 1.8) * scale, canvas.height - padding - (p[1] - minY + 1.8) * scale];
function line(points, close = false) {
  ctx.beginPath();
  points.forEach((point, index) => { const [x, y] = screen(point); index ? ctx.lineTo(x, y) : ctx.moveTo(x, y); });
  if (close) ctx.closePath();
  ctx.stroke();
}
function car(state, color, label) {
  const [x, y] = screen(state);
  ctx.save(); ctx.translate(x, y); ctx.rotate(-state[2]);
  ctx.fillStyle = color; ctx.fillRect(-11, -6, 22, 12);
  ctx.fillStyle = "#101418"; ctx.fillRect(3, -4, 5, 8); ctx.restore();
  ctx.fillStyle = color; ctx.fillText(label, x + 12, y - 10);
}
let index = 0;
function draw() {
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  ctx.strokeStyle = "#3f464d"; ctx.lineWidth = data.trackWidth * scale; ctx.lineJoin = "round"; ctx.lineCap = "round"; line(data.center, true);
  ctx.strokeStyle = "#d8dee9"; ctx.lineWidth = 3; line(data.outer, true); line(data.inner, true);
  data.curbs.forEach(curb => { ctx.strokeStyle = curb[2]; ctx.lineWidth = 7; line([curb[0], curb[1]]); });
  ctx.setLineDash([10, 7]); ctx.strokeStyle = "#f4d35e"; ctx.lineWidth = 3; line(data.pit); ctx.setLineDash([]);
  ctx.setLineDash([7, 7]); ctx.strokeStyle = "white"; ctx.lineWidth = 6; line(data.startFinish); ctx.setLineDash([]);
  const frame = data.frames[index];
  car(frame.ego, "#ff5a5f", "PPO"); car(frame.opponent, "#56b4e9", "Traffic");
  document.querySelector("#status").textContent = `${data.checkpoint} | frame ${index + 1}/${data.frames.length} | lap complete: ${data.completion}`;
  index = (index + 1) % data.frames.length;
}
draw(); setInterval(draw, data.intervalMs);
</script>
</body>
</html>""".replace("__DATA__", payload)
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(html, encoding="utf-8")


def preview():
    # Validate the environment before writing the first visible circuit result.
    ARTIFACTS.mkdir(exist_ok=True)
    env = RacingEnv(traffic=False, reward_profile="v2")
    check_env(env)
    observation, _ = env.reset(seed=101)
    print(f"Observation shape: {observation.shape}")
    _, trace = evaluate(
        "pure_pursuit",
        None,
        traffic=False,
        opponent_style="steady",
        track="train",
    )
    write_svg(ARTIFACTS / "preview.svg", trace, "train")
    print("Wrote artifacts/preview.svg")


def exploit():
    # Run the deterministic round trip against both reward profiles.
    v1 = RacingEnv(reward_profile="v1")
    v2 = RacingEnv(reward_profile="v2")
    v1_total = 0.0
    v2_total = 0.0
    for delta in [0.1, -0.1] * 10:
        v1_total += v1._reward(delta, False, False, False, False, False)[0]
        v2_total += v2._reward(delta, False, False, False, False, False)[0]
    print(f"v1 round-trip proxy total: {v1_total:.3f}")
    print(f"v2 round-trip proxy total: {v2_total:.3f}")
    print("The v1 proxy pays for progress farming; v2 does not.")


def train_solo():
    # Train three independent solo policies and apply the lap-completion gate.
    ARTIFACTS.mkdir(exist_ok=True)
    all_rows = []
    passing_seeds = []
    saved_trace = None

    baseline_env = RacingEnv(traffic=False, reward_profile="v2")
    untrained = make_model(baseline_env, seed=999)
    rows, _ = evaluate(
        "untrained",
        untrained,
        traffic=False,
        opponent_style="steady",
        track="train",
    )
    all_rows.extend(rows)
    rows, _ = evaluate(
        "pure_pursuit",
        None,
        traffic=False,
        opponent_style="steady",
        track="train",
    )
    all_rows.extend(rows)

    for seed in TRAIN_SEEDS:
        env = RacingEnv(traffic=False, reward_profile="v2")
        model = make_model(env, seed)
        print(f"Training solo seed {seed} for {SOLO_STEPS} steps")
        model.learn(total_timesteps=SOLO_STEPS)
        model.save(str(ARTIFACTS / f"solo_seed_{seed}"))
        rows, trace = evaluate(
            f"ppo_seed_{seed}",
            model,
            traffic=False,
            opponent_style="steady",
            track="train",
        )
        all_rows.extend(rows)
        if sum(row["completion"] for row in rows) >= 2:
            passing_seeds.append(seed)
        if saved_trace is None:
            saved_trace = trace
        env.close()

    gate_passed = len(passing_seeds) >= 2
    summary = {
        "label": "smoke experiment",
        "training_seeds": TRAIN_SEEDS,
        "evaluation_seeds": EVAL_SEEDS,
        "solo_steps_per_seed": SOLO_STEPS,
        "passing_seeds": passing_seeds,
        "traffic_gate_passed": gate_passed,
    }
    write_csv(ARTIFACTS / "solo_smoke.csv", all_rows)
    (ARTIFACTS / "solo_summary.json").write_text(
        json.dumps(summary, indent=2), encoding="utf-8"
    )
    write_svg(ARTIFACTS / "solo_trajectory.svg", saved_trace, "train")
    print_summary(all_rows)
    print(f"Traffic gate passed: {gate_passed}; passing seeds: {passing_seeds}")


def load_solo_summary():
    # Refuse traffic training until the solo experiment has produced a gate result.
    path = ARTIFACTS / "solo_summary.json"
    if not path.exists():
        raise SystemExit("Run train-solo first. Traffic remains disabled.")
    return json.loads(path.read_text(encoding="utf-8"))


def train_traffic():
    # Continue only the solo checkpoints that passed the fixed completion gate.
    summary = load_solo_summary()
    if not summary["traffic_gate_passed"]:
        print("Traffic skipped: fewer than two solo policies passed the lap gate.")
        return

    all_rows = []
    saved_trace = None
    for seed in summary["passing_seeds"]:
        env = RacingEnv(
            traffic=True, opponent_style="steady", reward_profile="v2"
        )
        model = PPO.load(
            str(ARTIFACTS / f"solo_seed_{seed}"), env=env, device="cpu"
        )
        print(f"Continuing seed {seed} with traffic for {TRAFFIC_STEPS} steps")
        model.learn(total_timesteps=TRAFFIC_STEPS, reset_num_timesteps=False)
        model.save(str(ARTIFACTS / f"traffic_seed_{seed}"))
        rows, trace = evaluate(
            f"traffic_ppo_seed_{seed}",
            model,
            traffic=True,
            opponent_style="weaver",
            track="train",
        )
        all_rows.extend(rows)
        if saved_trace is None:
            saved_trace = trace
        env.close()

    baseline_env = RacingEnv(
        traffic=True, opponent_style="weaver", reward_profile="v2"
    )
    untrained = make_model(baseline_env, seed=999)
    rows, _ = evaluate(
        "untrained",
        untrained,
        traffic=True,
        opponent_style="weaver",
        track="train",
    )
    all_rows.extend(rows)
    rows, _ = evaluate(
        "pure_pursuit",
        None,
        traffic=True,
        opponent_style="weaver",
        track="train",
    )
    all_rows.extend(rows)

    write_csv(ARTIFACTS / "traffic_smoke.csv", all_rows)
    write_svg(ARTIFACTS / "traffic_trajectory.svg", saved_trace, "train")
    print_summary(all_rows)


def demo():
    # Use the first traffic-qualified checkpoint for a reproducible visual proof.
    summary = load_solo_summary()
    models = [
        seed
        for seed in summary["passing_seeds"]
        if (ARTIFACTS / f"traffic_seed_{seed}.zip").exists()
    ]
    if not models:
        print("No traffic-qualified checkpoint exists; animated demo skipped.")
        return
    seed = models[0]
    checkpoint = f"traffic_seed_{seed}"
    model = PPO.load(str(ARTIFACTS / checkpoint), device="cpu")
    frames, info = collect_demo(model, seed=EVAL_SEEDS[0])
    write_demo_html(
        ARTIFACTS / "traffic_demo.html",
        frames,
        info,
        checkpoint,
    )
    print("Wrote artifacts/traffic_demo.html")


def heldout():
    # Measure qualified traffic policies on the altered circuit geometry.
    summary = load_solo_summary()
    models = [
        seed
        for seed in summary["passing_seeds"]
        if (ARTIFACTS / f"traffic_seed_{seed}.zip").exists()
    ]
    if not models:
        print("No traffic-qualified checkpoint exists; held-out circuit skipped.")
        return

    rows = []
    saved_trace = None
    for seed in models:
        model = PPO.load(ARTIFACTS / f"traffic_seed_{seed}", device="cpu")
        model_rows, trace = evaluate(
            f"traffic_ppo_seed_{seed}",
            model,
            traffic=True,
            opponent_style="weaver",
            track="heldout",
        )
        rows.extend(model_rows)
        if saved_trace is None:
            saved_trace = trace
    write_csv(ARTIFACTS / "heldout_circuit.csv", rows)
    write_svg(ARTIFACTS / "heldout_trajectory.svg", saved_trace, "heldout")
    print_summary(rows)


def main():
    # Dispatch one reproducible experiment command at a time.
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "command",
        choices=[
            "preview",
            "exploit",
            "train-solo",
            "train-traffic",
            "demo",
            "heldout",
        ],
    )
    args = parser.parse_args()
    {
        "preview": preview,
        "exploit": exploit,
        "train-solo": train_solo,
        "train-traffic": train_traffic,
        "demo": demo,
        "heldout": heldout,
    }[args.command]()


if __name__ == "__main__":
    main()

What Does the Runner Add?

• The model builder fixes future policies to CPU execution with separate actor and critic networks.
• The rollout functions collect behavioural metrics without reusing episode reward.
• The SVG writer converts track boundaries and trajectory points into a visible report.
• The HTML replay writer records the trained car and scripted opponent as a self-contained browser animation.
• The command dispatcher keeps preview, exploit, solo, traffic, animated demo, and held-out runs reproducible.

Before you run the preview, what observation shape do you expect the environment contract to report?

• Check the environment and generate the preview from the activated PowerShell terminal by running:

python experiment.py preview

What Does the Preview Command Do?

The runner checks the custom environment against Stable-Baselines3 expectations. It then resets the environment and prints the observation shape.

A pure-pursuit controller drives three seeded evaluations. The first trace becomes the red trajectory in the SVG report.

You should see Observation shape: (33,) followed by Wrote artifacts/preview.svg in the terminal.

• Expand the artifacts folder in the VS Code Explorer sidebar.
• Open preview.svg from the artifacts folder.

You should see a dark asphalt circuit with a 15-unit bottom straight, asymmetric technical sectors, red-and-white inner curbs, yellow pit-lane markings, a start/finish line, and a red driven trajectory. That is the first visible result from the new racing geometry.

Preview Missing or Empty?

• Confirm the terminal printed the observation shape before checking the SVG.
• Check that preview.svg is inside the generated artifacts folder.
• Rerun the test suite if the preview command stops before writing the file.

Use this prompt if the command stops early: Help me debug why my racing preview did not create artifacts/preview.svg.

Your racing contract now has executable tests and a visible circuit trace. Next, you will run the deliberately flawed reward probe and watch its progress signal fail.

Catch the First Reward Exploit

Your preview proved that the racing environment can produce a visible circuit trajectory. The tests already protect its geometry plus its episode boundaries.

However, a proxy reward can score repeated motion as useful progress. A policy can exploit that signal without completing a lap.

This step runs a deterministic round trip against the flawed profile. The observed failure becomes part of your experiment record.

In this step, get ready to:

• Compare both reward profiles with the deterministic round-trip probe.
• Record the failed proxy assumption in experiment_notes.md.
• Audit the training contract plus its reward-independent evaluation metrics.

Expose the round-trip exploit

The probe feeds +0.1 progress followed by -0.1 progress into each reward profile. Every pair returns to zero net progress.

Do you expect a zero-net-progress round trip to earn a positive score? The probe tests your prediction next.

• Test the round-trip reward in the activated PowerShell terminal by running this command:

python experiment.py exploit

What Did the Probe Expose?

You should see v1 round-trip proxy total: 8.000. You should then see v2 round-trip proxy total: -20.000.

• The v1 profile turns every progress delta positive with abs(delta_progress).
• The v2 profile keeps the direction of delta_progress.
• Each backward v2 delta triggers a reverse penalty.

You caught the intended failure. The positive v1 total proves that reversing across the same track can farm proxy reward.

Probe Not Running?

• Confirm that the PowerShell prompt shows the activated .venv environment.
• Confirm that the current folder is lidar-ppo-smoke.
• Check that exploit is spelled exactly as shown.

Help me diagnose my reward exploit probe.

Record the first failure

A failed assumption is evidence about the experiment. Recording it prevents later training output from hiding the weakness in the original reward.

• Switch back to Visual Studio Code from earlier.
• Select the Explorer view in the Activity Bar.
• Select New File....

You should see a filename field inside the Explorer view.

• Enter experiment_notes.md in the filename field.
• Press Enter.

You should see an empty experiment_notes.md file beside experiment.py.

• Fill experiment_notes.md by pasting this content:

# First failure

`python experiment.py exploit` showed a positive v1 round-trip proxy total and a negative v2 round-trip total. The v1 absolute-progress reward can be farmed by reversing over already traversed track. Training remains on reward_profile="v2"; evaluation will not report episode reward as the result.

What Does This Note Preserve?

The note connects the observed output to the absolute-progress exploit. It also records the reward profile used by later training.

The last sentence keeps episode reward separate from behavioural evidence. That distinction protects the integrity of the smoke experiment.

• Press Ctrl+S to save experiment_notes.md.
• Confirm that First failure is the first heading.
• Confirm that the paragraph records the positive v1 result plus the negative v2 result.

Note Missing From the Explorer?

• Confirm that experiment_notes.md sits beside experiment.py.
• Check that the filename ends with .md.
• Press Ctrl+S if the editor tab still shows an unsaved-change indicator.

Help me check my experiment note.

✔️ Awesome, I've got everything!

Your first reward failure is saved as part of the experiment record.

ⓧ I'd like to double check the full code

# First failure

`python experiment.py exploit` showed a positive v1 round-trip proxy total and a negative v2 round-trip total. The v1 absolute-progress reward can be farmed by reversing over already traversed track. Training remains on reward_profile="v2"; evaluation will not report episode reward as the result.

Confirm the corrected training contract

The observed exploit gives signed progress a measured purpose. The v2 contract preserves movement direction plus separate hazard penalties.

The experiment already selects reward_profile="v2" for training plus evaluation. The flawed profile remains isolated inside exploit() as a reproducible failure probe.

• Select experiment.py in the Explorer view.
• Review the RacingEnv(...) calls inside rollout(), preview(), train_solo(), plus train_traffic().

You should find six uses of reward_profile="v2" in experiment.py. The only reward_profile="v1" environment belongs to exploit().

• Select racing_env.py in the Explorer view.
• Locate the RacingEnv._reward() method.
• Review the terms inside its v2 branch.

You should find signed progress plus separate reverse, wall, out, collision, plus caused terms.

• Switch back to experiment.py.
• Locate the write_csv() function.
• Inspect the fields list.

The CSV report tracks completion, lap time, wall contacts, track-limit violations, caused contacts, overtakes, plus reverse steps. Episode reward is absent from the report.

What Does the Audit Prove?

• Signed progress removes the positive payment for backward movement.
• The reverse term adds a direct penalty for backward progress.
• The wall term penalizes the wall-proximity threshold.
• The out term penalizes track-limit violations.
• The collision term penalizes vehicle contact.
• The caused term adds responsibility for closing into the opponent.
• The evaluation fields measure driving behaviour independently of training reward.

Which reward profile do you expect to remain positive after equal forward plus backward progress? The final probe tests your audit.

• Verify the reward correction in the activated PowerShell terminal by running this command:

python experiment.py exploit

What Should You See?

You should see v1 round-trip proxy total: 8.000 followed by v2 round-trip proxy total: -20.000.

The positive first total reproduces the progress-farming exploit. The negative second total confirms that signed progress plus reverse penalties remove this incentive.

🙋‍♀️ Results Changed Unexpectedly?

• Confirm that exploit() still evaluates [0.1, -0.1] * 10.
• Confirm that the probe creates one reward_profile="v1" environment plus one reward_profile="v2" environment.
• Check that RacingEnv._reward() still uses abs(delta_progress) only inside the v1 branch.

Help me compare my exploit results with the reward code.

The first reward failure is now reproduced, explained, plus preserved in your notes. Next, you will train three fixed-seed solo smoke runs with the corrected contract.

Run the Three-Seed Solo Smoke Experiment

Your v2 reward contract now blocks the round-trip progress exploit. The tested environment is ready for a controlled PPO run.

A single short training run can look convincing by chance. Stable-Baselines3 trains three fixed seeds within the same limited budget.

Each checkpoint is judged on three unseen evaluation seeds. An untrained network plus a pure-pursuit controller provide two useful baselines.

In this step, get ready to:

• Review the fixed training recipe for the solo smoke experiment.
• Train three solo policies with a limited CPU budget.
• Inspect the reward-independent behavioural reports.

Review the fixed experiment design

A smoke experiment checks whether the complete training pipeline runs and produces measurable behaviour. The fixed recipe keeps the comparison consistent across seeds.

• Switch back to experiment.py in Visual Studio Code.
• Locate TRAIN_SEEDS = [11, 22, 33].
• Locate EVAL_SEEDS = [101, 202, 303].
• Locate SOLO_STEPS = 20_000.
• Find the make_model function.
• Confirm that train_solo creates each environment with reward_profile="v2".

What makes these runs comparable?

• The "MlpPolicy" setting selects a multilayer policy for the vector observation.
• The activation_fn=th.nn.Tanh setting uses tanh activations throughout the hidden layers.
• The net_arch=dict(pi=[64, 64], vf=[64, 64]) setting gives the actor two 64-unit layers. It gives the critic the same structure.
• The learning_rate=3e-4 setting controls the update size. The n_steps=1024 setting controls the rollout length.
• The batch_size=64 setting divides rollout data into update batches. The n_epochs=10 setting controls repeated passes over that data.
• The gamma=0.99 setting discounts future outcomes. The gae_lambda=0.95 setting controls advantage estimation. The clip_range=0.2 setting limits each policy update.
• The seed=seed setting makes each run identifiable. The device="cpu" setting keeps execution on the CPU.

Train the three solo policies

This is the longest run in the project. CPU speed determines its duration.

Before you run it, do you expect three policies with the same settings to produce identical behavioural results?

• Keep the activated PowerShell terminal from earlier available for the full run.
• Start the three-seed solo experiment by running:

python experiment.py train-solo

What does this command do?

• The untrained model uses seed 999 to create a neural-network baseline.
• The pure-pursuit controller creates a hand-designed driving baseline.
• Each training seed receives 20_000 environment steps with traffic disabled.
• Each trained checkpoint is evaluated on seeds 101, 202, and 303.
• The experiment saves measured checkpoints plus behavioural reports inside artifacts.

You will see one training announcement for each of seeds 11, 22, and 33. The PowerShell prompt returns after every training and evaluation phase finishes.

That is the heavy run complete. You now have three CPU-trained checkpoints backed by unseen-seed evaluations.

Did the training stop early?

• Return to the activated .venv terminal from earlier if Python reports a missing dependency.
• Rerun the same training command if PowerShell was closed before the prompt returned.
• Check that your computer has free local disk space if a checkpoint cannot be saved.

Share the terminal output without any private file paths: Help me diagnose why my three-seed solo PPO run stopped before creating every artifact.

Inspect the behavioural evidence

Episode reward stays outside the evaluation result. The CSV report records measured driving behaviour for every policy and evaluation seed.

The JSON summary stores the measured gate status. The SVG captures a trajectory from the first evaluated trained policy.

Before you inspect the reports, do you expect the training seed with the best-looking trajectory to be the only evidence that matters?

• Expand artifacts in the Visual Studio Code file sidebar from earlier.
• Confirm that solo_seed_11.zip, solo_seed_22.zip, and solo_seed_33.zip exist.
• Open artifacts/solo_smoke.csv.
• Compare the trained policy rows with the untrained and pure-pursuit rows across all evaluation seeds.
• Open artifacts/solo_summary.json.
• Read the measured passing_seeds list.
• Read the measured traffic_gate_passed value.
• Select artifacts/solo_trajectory.svg in the file sidebar.

You should see 15 measured data rows in solo_smoke.csv. They cover five policy labels across three unseen evaluation seeds.

The summary contains the fixed seeds plus the 20_000-step budget. Its gate value can honestly be either true or false.

Missing a solo artifact?

• Check whether the PowerShell prompt returned after all three training announcements.
• Rerun the training command if one of the checkpoint files is absent.
• Refresh the Visual Studio Code file sidebar if the terminal completed but the files are not listed.

Get help checking which phase failed: Help me trace a missing solo smoke artifact from my training output.

The generated file set proves the full training and evaluation pipeline ran. The metric values show what each policy actually did.

How should you read the smoke report?

• The completion field records whether the policy finished a lap.
• The lap_time field records elapsed time only for a completed lap.
• The wall_contacts field records wall encounters. The track_limit_violations field records departures from the circuit.
• The reverse_steps field exposes backward motion. The distance field records total centreline progress.
• The caused_contacts and overtakes fields preserve the report schema for later traffic evaluation.
• A false traffic gate is valid smoke evidence. It shows that the limited solo budget did not meet the behavioural requirement.

Your three-seed solo evidence is now saved as checkpoints and reward-independent reports. Next, the measured gate decides whether moving traffic remains locked or becomes the next training stage.

Gate Traffic Behind Solo Lap Completion

Your solo smoke run produced three seeded PPO checkpoints. Its reward-independent report now decides whether any policy is ready for a moving opponent.

Traffic adds opponent interaction to a driving task the policy may not have mastered alone. This gate keeps those learning problems separate until solo lap completion supports the next stage.

In this step, get ready to:

• Inspect the measured solo gate status.
• Run the gate-controlled traffic experiment.
• Verify the behavioural evidence and play the animated traffic replay.

Read the solo gate result

The JSON summary turns solo lap completion into a curriculum gate. A policy qualifies after completing at least two of its three unseen evaluations.

Traffic starts only when at least two policies qualify. This represents a completion rate of at least 2/3 for each qualifying policy.

Why gate traffic?

Solo driving is one learning problem. Opponent interaction is another.

The gate ties curriculum progression to measured lap completion. A visually appealing trajectory cannot override that evidence.

• Switch back to Visual Studio Code from earlier.
• Expand the artifacts folder in the file sidebar.
• Select solo_summary.json.
• Locate the measured passing_seeds list.
• Locate the measured traffic_gate_passed value.

You should see the seeds that completed at least two evaluations. You should also see the Boolean decision that controls traffic training.

Run the gated traffic experiment

The train-traffic command treats solo_summary.json as the authority. This prevents manual seed selection from bypassing the measured gate.

If the gate is true, the CPU continuation may take several minutes while each qualifying checkpoint trains for TRAFFIC_STEPS=10_000 steps. A quiet stretch is expected while PPO updates the policy.

• Predict whether your measured gate allows traffic or blocks it.

• Start the gate-controlled traffic experiment by running this command:

python experiment.py train-traffic

What does this command do?

• The load_solo_summary() function reads the measured gate decision from artifacts/solo_summary.json.
• A false traffic_gate_passed value ends the command before traffic training.
• A true gate loads every solo checkpoint named by passing_seeds.
• Each qualifying policy continues against opponent_style="steady" with reward_profile="v2".
• Each continued checkpoint uses device="cpu".
• The evaluation switches to opponent_style="weaver" on seeds 101, 202, plus 303.
• Fresh untrained PPO results provide one baseline. Pure pursuit provides the other baseline.

When the command finishes, the terminal either reports that traffic was skipped or prints smoke metric summaries after traffic evaluation.

Did the traffic command stop early?

• Confirm the activated .venv environment is still visible in PowerShell.
• Confirm artifacts/solo_summary.json still exists inside lidar-ppo-smoke.
• Run the same traffic command again after resolving an interrupted process.

Help me diagnose why the traffic command stopped.

Verify the traffic outcome

The final evidence depends on the measured gate branch. A passed gate creates a CSV report from traffic evaluation.

A blocked gate creates no traffic checkpoint or evaluation artifact. That blocked state proves the curriculum rule remained active.

• Predict which traffic evidence should exist for your measured branch.

✔️ Traffic artifacts were created

That is a meaningful checkpoint. Your curriculum advanced because measured solo behaviour met the gate.

• Confirm a traffic checkpoint ZIP file exists for every seed listed in passing_seeds.
• Confirm each traffic checkpoint name begins with traffic_seed_.
• Select artifacts/traffic_smoke.csv in the file sidebar.
• Compare the traffic PPO rows with the fresh untrained baseline.
• Compare the traffic PPO rows with the pure-pursuit baseline.
• Select artifacts/traffic_trajectory.svg in the file sidebar.

You should see measured evaluation rows for seeds 101, 202, plus 303. The SVG should show the first qualifying traffic policy on the train circuit.

What evidence should you inspect?

• The completion column records whether the policy completed a lap.
• The lap_time column records the measured completion time.
• The wall_contacts column records contact with circuit boundaries.
• The track_limit_violations column records excursions beyond the track.
• The caused_contacts column records collisions attributed to the policy.
• The overtakes column records moves past the scripted opponent.
• The reverse_steps column records backward progress during evaluation.

ⓧ Traffic was skipped by the gate

The guardrail did its job. Your experiment stopped before adding a second learning problem to policies that had not demonstrated enough solo lap completion.

Your terminal should show Traffic skipped: fewer than two solo policies passed the lap gate..

• Return to the artifacts folder in the file sidebar.
• Confirm no new traffic checkpoint ZIP files exist.
• Confirm traffic_smoke.csv was not created.
• Confirm traffic_trajectory.svg was not created.
• Use solo_summary.json as the record of the blocked outcome.

Why stopping is valid evidence

The failed gate records that the current reward contract plus training budget did not produce enough solo completion. A later experiment can adjust the reward or budget before trying traffic again.

The reports measure behaviour across several runs. The animated replay adds a direct visual check by loading the first traffic-qualified checkpoint and running it beside the weaving opponent.

Before you generate the replay, do you expect your measured gate branch to produce an animation or preserve a skipped result?

• Generate the top-down traffic replay by running this command:

python experiment.py demo

What does the demo command do?

The command reads solo_summary.json and selects the first traffic checkpoint that actually exists.

It runs one deterministic evaluation with the trained PPO car and the scripted weaving opponent. The recorded positions become a self-contained animation in artifacts/traffic_demo.html.

A qualified checkpoint produces Wrote artifacts/traffic_demo.html in the terminal. A blocked gate produces No traffic-qualified checkpoint exists; animated demo skipped. instead.

✔️ The demo file was created

Your trained policy and traffic simulation are ready for a visible replay.

• Expand the artifacts folder in the Visual Studio Code Explorer sidebar.
• Right-click traffic_demo.html.
• Select Reveal in File Explorer.
• Double-click traffic_demo.html in File Explorer.

You should see a red trained PPO car and a blue scripted traffic car moving around the top-down circuit. The status line names the loaded checkpoint and reports whether this measured rollout completed a lap.

ⓧ The demo was skipped

The animation remains unavailable because no policy earned traffic training. That result preserves the curriculum gate instead of presenting an unqualified model as a success.

• Open artifacts/solo_summary.json in the Visual Studio Code Explorer sidebar.
• Confirm traffic_gate_passed remains false.

Replay missing or showing source code?

• Confirm the terminal printed Wrote artifacts/traffic_demo.html before looking for the file.
• Refresh the Visual Studio Code Explorer if the generated file is not visible.
• Open the file from File Explorer if Visual Studio Code displays the HTML source instead of the animation.

Help me diagnose the missing traffic replay.

Secret mission

Challenge the Policy on a Held-Out Circuit

Test whether any traffic-qualified policy transfers to a circuit geometry excluded from training. The checkpoints remain frozen so the result measures transfer without retraining.

Clean Up Your Resources

Clean Up Your Resources

Everything in this project runs locally inside your lidar-ppo-smoke workspace. There are no cloud resources or ongoing financial costs.

Choose whether to keep the workspace ready for more experiments, pause the environment, or remove its local files.

Resources you used:

• A local Python virtual environment in .venv containing the pinned reinforcement-learning packages.
• Project files including racing_env.py, experiment.py, requirements.txt, reward_contract.md, experiment_notes.md, and tests/test_racing_env.py.
• An artifacts folder containing PPO checkpoints, smoke reports, gate summaries, and trajectory graphics from the runs that qualified.

Keep everything running

No action needed. Choose this if you want to compare future reward changes or training budgets against this smoke experiment.

• Keep the lidar-ppo-smoke folder in its current location.
• Preserve .venv so the pinned package environment remains available.
• Preserve the artifacts folder so your solo, traffic, and held-out evidence stays available.
• Leave the checkpoint files unchanged so future evaluations use the same frozen policies.

Pause - I'll come back to this later

Pausing deactivates the virtual environment while preserving every source file, checkpoint, and report.

• Deactivate the environment in the PowerShell workspace from earlier by running:

deactivate

You're safely paused now. Your PowerShell prompt no longer marks the virtual environment as active.

• Switch back to the lidar-ppo-smoke folder in PowerShell when you want to resume.
• Reactivate the saved environment by running:

.\.venv\Scripts\Activate.ps1

The environment becomes active again with the same pinned packages. Your checkpoints and measured reports remain unchanged.

Delete - I don't want to use this again

Deleting the whole workspace permanently removes your source code, checkpoints, and measured reports. You can reclaim only the package space if you want to keep the experiment evidence.

• Choose one deletion depth below.

Delete only the environment

This option removes the installed packages while keeping your code, checkpoints, and reports.

• Deactivate the environment in PowerShell by running:

deactivate

The virtual environment is now inactive. Its files can be removed without affecting the rest of the workspace.

• Open File Explorer from the Windows taskbar.
• Locate the lidar-ppo-smoke folder.
• Open the lidar-ppo-smoke folder.
• Select the .venv folder.
• Use the delete option in its context menu.
• Confirm that .venv no longer appears beside your source files.

Your package space is reclaimed. The experiment code and all generated evidence remain available.

Delete the whole project

This option removes every local resource listed above. Save any reports you want to keep before continuing.

• Deactivate the environment in PowerShell by running:

deactivate

The environment is now inactive. The complete workspace can be removed as one folder.

• Close Visual Studio Code to release the workspace.
• Open File Explorer from the Windows taskbar.
• Locate the parent location containing lidar-ppo-smoke.
• Select the lidar-ppo-smoke folder.
• Use the delete option in its context menu.
• Confirm that lidar-ppo-smoke no longer appears in the parent location.

That's the full local cleanup complete. The virtual environment, source files, checkpoints, and generated reports are removed.

Nice Work!

Nice Work!

You did it! You built a CPU-only racing research harness with a visible 33-value LiDAR-based observation contract. When the traffic gate passed, you also produced an animated proof showing the trained car and scripted opponent on the top-down circuit.

You've learned how to:

• Build a tested F1TENTH-style environment with 24 range beams. Verify its geometry, hazards, lap counting, observation shape, continuous controls, termination behaviour, and truncation behaviour.
• Expose a deliberate proxy-reward exploit with a deterministic round-trip probe. Replace absolute progress with a signed-progress training contract. Keep behavioural evaluation separate from episode reward.
• Train three CPU-only PPO smoke checkpoints with fixed seeds. Use a solo lap-completion gate to control traffic training. Generate a browser replay that shows a qualified trained car and scripted opponent moving around the circuit.
• Secret Mission: Apply a held-out circuit gate to frozen traffic-qualified checkpoints. Keep the geometry report separate when qualifying checkpoints exist. Preserve the honest skipped result when none exist.

Ready to quiz yourself?