Falsify a Momentum Strategy

Introduction

30 Second Summary

A profitable-looking result can feel persuasive until the next stretch of data wipes it out. Reusing the same history to choose a strategy can hide that fragility.

In this project, you will build a terminal-based quant research lab in Python. The finished workflow uses NumPy plus pandas to carry a momentum hypothesis from reproducible simulation to an evidence-based verdict.

What You'll Build

When you run the finished program, you will watch a promising signal face unseen data before the terminal prints its evidence-based verdict.

By the end of this project, you'll have:

• A deterministic market simulation where every run reproduces the same regime change.
• A falsification workflow that puts a tempting full-history winner through chronological testing with transaction costs.
• A research memo that reports the unseen result. It records the p-value from a permutation test. The memo closes with limitations plus a verdict.
• Secret Mission: A red-team review of the memo that identifies three threats to validity before defining a falsification experiment with a promotion hurdle.

Are there any prerequisites?

This project assumes comfort with Python functions plus pandas or NumPy. Working knowledge of probability, sample statistics, and hypothesis testing is also required.

Before We Start

Before any hands-on work, commit to testing whether a lagged momentum signal survives a regime change. Defining the rejection evidence now keeps the final verdict grounded in results.

Build a Reproducible Research Workspace

You have defined the evidence that would falsify your momentum hypothesis. That commitment only matters if every later result can be reproduced from a clean starting point.

A virtual environment isolates this lab's Python packages from the rest of your computer. Pinned NumPy and pandas versions keep the dependencies consistent across repeated runs.

Visual Studio Code keeps your script beside its terminal output. Its integrated Windows PowerShell terminal gives every setup command the same workspace context.

In this step, get ready to:

• Verify that Python 3.12 or newer is available.
• Create an isolated environment with NumPy 2.5.4 plus pandas 3.1.0.
• Build a runnable quant_research.py workspace skeleton.

Verify Python and create the workspace

NumPy 2.5.4 requires Python 3.12 or newer. Checking the interpreter first prevents a confusing package installation failure later.

Before you check, which Python version do you expect your Windows launcher to report?

• Check the Python version from PowerShell by running:

py -3 --version

What does this command do?

The py launcher selects your installed Python 3 interpreter. The version output tells you whether that interpreter supports the pinned NumPy release.

✔️ I see Python 3.12 or higher

Your interpreter meets the project's minimum version. That removes the first reproducibility risk from your workspace.

ⓧ I see an older version

The installed interpreter cannot use NumPy 2.5.4. Upgrade Python before creating the environment.

• Follow the official Visual Studio Code Windows Python setup guide.
• Install Python 3.12 or newer.
• Close the current PowerShell window.
• Repeat the version check shown above in a new PowerShell window.

ⓧ Command not found

The Windows Python launcher is unavailable from your current terminal. Install Python before continuing.

• Follow the official Visual Studio Code Windows Python setup guide.
• Install Python 3.12 or newer.
• Close the current PowerShell window.
• Repeat the version check shown above in a new PowerShell window.

The quant-alpha-lab folder keeps the script beside its isolated environment. Creating it on your Desktop also makes the workspace easy to find.

• Press the Windows key to open Windows search.
• Type PowerShell and press Enter.
• Create the quant-alpha-lab folder on your Desktop by running these commands:

Set-Location -Path "$HOME\Desktop"
New-Item -ItemType Directory -Path "quant-alpha-lab"
Set-Location -Path "quant-alpha-lab"
Get-Location

What do these commands do?

• Set-Location moves PowerShell to your Desktop.
• New-Item creates the quant-alpha-lab directory.
• Get-Location prints the active path so you can confirm where the project lives.

The final path in PowerShell should end with Desktop\quant-alpha-lab.

• Press the Windows key to open Windows search.
• Type Visual Studio Code and press Enter.
• Select File from the top menu.
• Select Open Folder....
• Select the quant-alpha-lab folder from your Desktop.
• Click Select Folder.
• Review the folder if the Workspace Trust dialog appears.
• Select Yes, I trust the authors because you created this empty folder.

Good progress. The Explorer sidebar now shows quant-alpha-lab as your active workspace.

Isolate and install the dependencies

The environment named venv keeps this project's interpreter plus packages together. Activating it makes subsequent installation commands target that isolated environment.

• Press Ctrl+` to open the integrated terminal.
• Select PowerShell from the terminal profile list if another shell opens.
• Create and activate venv by running these commands:

python -m venv venv
.\venv\Scripts\activate

How does the environment work?

The first command creates a local Python environment inside venv. The activation command updates the current terminal so python plus pip use that environment.

Your PowerShell prompt should now begin with (venv). The venv folder should also appear in the Explorer sidebar.

Environment not activating?

Confirm that the terminal is PowerShell. Confirm that venv appears inside quant-alpha-lab before repeating the activation command.

If PowerShell blocks the activation script, help me diagnose the activation policy safely.

If the problem continues, share the full terminal output in the NextWork community.

• Install the exact NumPy and pandas versions into the active environment by running:

python -m pip install numpy==2.5.4 pandas==3.1.0

Why pin the package versions?

The == pins install exact releases. Repeated runs now use the same library behavior instead of silently adopting future package changes.

The installation output should complete with NumPy 2.5.4 plus pandas 3.1.0 listed among the installed packages.

Package installation failing?

Confirm that your prompt starts with (venv). Confirm that the earlier version check reported Python 3.12 or newer.

For environment-specific help, help me debug the pinned package installation.

If the problem continues, share the full installation output in the NextWork community.

Type and run the research skeleton

The first script establishes the imports plus fixed experiment settings used throughout the lab. The readiness message gives you immediate evidence that Python can load both installed packages.

The fixed SEED controls the pseudo-random sequence used by later simulation code. Reusing it makes the generated values reproducible across runs.

• Hover over quant-alpha-lab in the Explorer sidebar.
• Click the new file icon.
• Name the file quant_research.py.
• Type these imports plus the temporary readiness message into quant_research.py:

from pathlib import Path

import numpy as np
import pandas as pd


print("Quant research workspace ready.")

What does this code set up?

• Path provides the file-writing tool used by the later research memo.
• np gives the script a short name for NumPy's numerical tools.
• pd gives the script a short name for pandas data structures.
• print() creates a visible checkpoint after all three imports succeed.

• Save quant_research.py by pressing Ctrl+S.
• Test the imports in the active terminal by running:

python quant_research.py

What should I see?

You should see Quant research workspace ready. with no import error. This proves that the active environment can load both pinned packages.

Seeing an import error?

Check that the terminal prompt begins with (venv). Repeat the package installation command from the previous substep if the environment is active.

You can also ask for help with the failing import.

• Place your cursor above print("Quant research workspace ready.").
• Type the project constants in that position:

SEED = 20261030
N_OBSERVATIONS = 1200
TRAIN_SIZE = 800
LOOKBACKS = [1, 2, 5, 10, 20]
COST_PER_UNIT_TURNOVER = 0.0002
PERMUTATIONS = 500

What do these settings control?

• SEED fixes the future pseudo-random sequence.
• N_OBSERVATIONS sets the total simulated history length.
• TRAIN_SIZE reserves the first portion for parameter selection.
• LOOKBACKS defines the candidate momentum windows.
• COST_PER_UNIT_TURNOVER defines the later simulated trading penalty.
• PERMUTATIONS sets the later null-experiment count.

• Save quant_research.py by pressing Ctrl+S.

✔️ Awesome, I've got everything!

Your imports, constants, plus readiness check are saved in quant_research.py.

ⓧ I'd like to double check the full code

Compare your complete quant_research.py file with this checksum:

from pathlib import Path

import numpy as np
import pandas as pd


SEED = 20261030
N_OBSERVATIONS = 1200
TRAIN_SIZE = 800
LOOKBACKS = [1, 2, 5, 10, 20]
COST_PER_UNIT_TURNOVER = 0.0002
PERMUTATIONS = 500


print("Quant research workspace ready.")

How to compare the file

Check the import order first. Check every constant name plus value before comparing the final readiness message.

Before the final run, do you expect the constants to change the readiness output?

• Verify the complete workspace by running:

python quant_research.py

Workspace verified

You should see Quant research workspace ready. without an import error. The constants store settings without printing them, so the visible output remains unchanged.

Your isolated research workspace is ready to produce repeatable results. Next, you will use the fixed seed to simulate a market whose behavior changes over time.

Simulate a Changing Market

Your isolated workspace now runs the project with pinned NumPy and pandas packages. That gives you a stable base for a repeatable experiment.

Real market data hides the process that generated each return. In this step, you'll build a controlled simulation where the regime change is known.

In this step, get ready to:

• Predict how positive and negative persistence change next-period dependence.
• Type a seeded market simulation with a known regime change.
• Verify deterministic output across two runs.

Predict the dependence shift

Serial dependence describes how one return relates to an earlier return. The persistence coefficient multiplies the previous return before a fresh shock is added.

• Predict the sign of next-period dependence when persistence is 0.65.
• Predict the sign of next-period dependence when persistence is -0.65.
• Explain which regime should reward a momentum rule.
• Explain which regime should punish a momentum rule.

What do the signs imply?

With 0.65, the previous return pushes the next return in the same direction. This creates positive serial dependence.

With -0.65, the previous return pushes the next return in the opposite direction. This creates negative serial dependence.

Momentum can benefit from repeated direction in the training regime. The reversal pattern in the test regime challenges that signal.

Build the market simulation

A seeded random generator produces the shocks that move the simulated market. The return list carries the latest market state into the next observation.

• Select the existing quant_research.py file in the Explorer sidebar.
• Find the temporary print("Quant research workspace ready.") line.
• Place your cursor on the blank line above the temporary print line.
• Type the function below one logical block at a time:
• Pause at each blank line to name the state created by the preceding block.

def simulate_market() -> pd.Series:
    rng = np.random.default_rng(SEED)
    shocks = rng.normal(0.0, 0.006, N_OBSERVATIONS)
    market_returns = [0.0]

    for time_index in range(1, N_OBSERVATIONS):
        persistence = 0.65 if time_index < TRAIN_SIZE else -0.65
        next_return = persistence * market_returns[-1] + float(shocks[time_index])
        market_returns.append(next_return)

    return pd.Series(market_returns, name="market_return")

How does the simulation evolve?

• The rng generator uses SEED to reproduce the same pseudo-random sequence.
• The shocks array holds one Gaussian shock for each observation.
• The market_returns list is the state carried through the simulation.
• The time_index value determines which persistence regime applies.
• The next_return value combines persistence with the current shock.
• The returned pandas Series gives the observations a consistent name for later analysis.

• Save quant_research.py by pressing Ctrl+S.

Before you test the new function, do you think Python will reach the existing workspace message or stop before it?

• Switch back to the PowerShell terminal from earlier.
• Check that the file still runs by running:

python quant_research.py

You should still see Quant research workspace ready. without an import or syntax error.

That is the simulation engine in place: Python reaches the existing workspace message without stopping.

Script stopping before the message?

• Check that every line inside the for loop uses the same indentation.
• Check that return pd.Series(...) aligns with the for statement.
• Compare every parenthesis with the simulation function above.

Help me find the syntax or indentation problem in my simulate_market function.

Print and verify the market preview

The simulation function now creates the data. A small entry point makes its regime sizes and first five returns visible in the terminal.

• Return to quant_research.py.
• Delete the temporary print("Quant research workspace ready.") line.
• Type this entry point below simulate_market():

def main() -> None:
    market_returns = simulate_market()

    print("SIMULATED MARKET")
    print(f"Observations: {N_OBSERVATIONS}")
    print(f"Training observations: {TRAIN_SIZE}")
    print(f"First five returns: {[round(value, 6) for value in list(market_returns)[:5]]}")


if __name__ == "__main__":
    main()

What does the entry point show?

• The main() function runs the simulation once.
• The market_returns variable holds the complete simulated Series.
• The first three print calls expose the heading and regime sizes.
• The final print call rounds the first five returns to six decimal places.
• The __main__ guard calls main() when you execute this file directly.

• Save quant_research.py by pressing Ctrl+S.

✔️ Awesome, I've got everything!

Your simulation function and entry point are saved in quant_research.py.

ⓧ I'd like to double check the full code

Compare your complete quant_research.py file with this checksum:

from pathlib import Path

import numpy as np
import pandas as pd


SEED = 20261030
N_OBSERVATIONS = 1200
TRAIN_SIZE = 800
LOOKBACKS = [1, 2, 5, 10, 20]
COST_PER_UNIT_TURNOVER = 0.0002
PERMUTATIONS = 500


def simulate_market() -> pd.Series:
    rng = np.random.default_rng(SEED)
    shocks = rng.normal(0.0, 0.006, N_OBSERVATIONS)
    market_returns = [0.0]

    for time_index in range(1, N_OBSERVATIONS):
        persistence = 0.65 if time_index < TRAIN_SIZE else -0.65
        next_return = persistence * market_returns[-1] + float(shocks[time_index])
        market_returns.append(next_return)

    return pd.Series(market_returns, name="market_return")


def main() -> None:
    market_returns = simulate_market()

    print("SIMULATED MARKET")
    print(f"Observations: {N_OBSERVATIONS}")
    print(f"Training observations: {TRAIN_SIZE}")
    print(f"First five returns: {[round(value, 6) for value in list(market_returns)[:5]]}")


if __name__ == "__main__":
    main()

Before you run the finished simulation, what do you expect the fixed SEED to do to the five-value preview across repeated runs?

• Switch back to the PowerShell terminal from earlier.
• Run the simulated market by entering:

python quant_research.py

You should see SIMULATED MARKET, followed by Observations: 1200 and Training observations: 800.

The First five returns: line should contain five rounded values.

• Run the same simulation again to test reproducibility:

python quant_research.py

You should see the same five return values in both previews. That repeat is your reproducibility check.

Preview missing or changing?

• Check that np.random.default_rng(SEED) receives SEED exactly.
• Check that N_OBSERVATIONS remains 1200.
• Check that TRAIN_SIZE remains 800.

Help me debug why my simulated market preview is missing or changes between runs.

Your script now rebuilds the same changing market whenever you run it. Next, you'll test several momentum lookbacks against that controlled path.

Find a Misleading Alpha

Your Visual Studio Code workspace now runs a deterministic Python simulation with a known regime change. You can now test whether a momentum rule can exploit that return path.

A parameter search can produce an impressive backtest winner when every candidate is scored on the same history. This step lets that result earn your attention before you interrogate it.

In this step, get ready to:

• Predict how quickly each candidate lookback responds to recent returns.
• Build lagged positions with turnover-aware performance measurements.
• Search the full history for the strongest total return.

Predict which lookbacks react fastest

A shorter lookback uses fewer recent observations. It responds more quickly when the direction of returns changes.

The candidates are 1, 2, 5, 10, and 20. Testing several lookbacks gives the search more chances to find an apparently strong result on this exact path.

• Complete the prediction below:

Build positions and score returns

A momentum position follows the sign of a recent average return. The average must be lagged so the current return cannot choose the position that earns that same return.

• Switch back to the open quant_research.py tab.
• Find the end of simulate_market().
• Define the position rule below simulate_market() by typing this function:

def make_positions(market_returns: pd.Series, lookback: int) -> list[float]:
    lagged_average = (
        market_returns.rolling(lookback)
        .mean()
        .shift(1)
        .fillna(0.0)
    )

    positions = []
    for value in lagged_average:
        if value > 0.0:
            positions.append(1.0)
        elif value < 0.0:
            positions.append(-1.0)
        else:
            positions.append(0.0)
    return positions

What does this code do?

• The rolling average summarizes the most recent returns for each candidate lookback.
• .shift(1) moves each average forward by one observation. The current return can only use information from earlier observations.
• .fillna(0.0) assigns a neutral signal where a complete rolling window is unavailable.
• The loop converts positive averages to long positions. It converts negative averages to short positions.

• Add the return application function below make_positions() by typing this code:

def apply_positions(
    market_returns: list[float],
    positions: list[float],
    cost_rate: float,
    previous_position: float = 0.0,
) -> list[float]:
    net_returns = []

    for market_return, position in zip(market_returns, positions):
        turnover = abs(position - previous_position)
        trading_cost = cost_rate * turnover
        net_returns.append(position * market_return - trading_cost)
        previous_position = position

    return net_returns

How are trading costs applied?

• Turnover measures the distance between the current position and the previous position.
• A move from 1.0 to -1.0 creates two units of turnover.
• The cost rate multiplies that turnover before the cost is subtracted from the strategy return.
• Updating previous_position preserves the position needed for the next turnover calculation.

• Save quant_research.py by pressing Ctrl+S.
• Check that both functions load by running this command in the existing terminal:

python quant_research.py

What does this check prove?

The command makes Python load every function definition before it reaches main(). Reaching the simulation output confirms that both new functions have valid syntax.

You should see the familiar simulated-market preview without a traceback. Your position and cost functions are now part of the runnable script.

Does the script stop before the preview?

Check the indentation inside both loops. Each calculation inside a loop needs one additional indentation level.

Compare the closing parenthesis in the make_positions() rolling calculation with the code above.

Help me fix the new position or cost function.

A return path needs summary measurements before candidates can be compared. You will track compounded return, maximum drawdown, and the proportion of positive periods.

• Define the performance measurements below apply_positions() by typing this function:

def measure(net_returns: list[float]) -> dict[str, float]:
    equity = 1.0
    peak = 1.0
    max_drawdown = 0.0
    positive_periods = 0

    for net_return in net_returns:
        equity *= 1.0 + net_return
        peak = max(peak, equity)
        max_drawdown = min(max_drawdown, equity / peak - 1.0)
        if net_return > 0.0:
            positive_periods += 1

    observations = len(net_returns)
    hit_rate = positive_periods / observations if observations else 0.0
    return {
        "total_return": equity - 1.0,
        "max_drawdown": max_drawdown,
        "hit_rate": hit_rate,
    }

What do these measurements reveal?

• Equity compounds each net return from a starting value of 1.0.
• Peak stores the highest equity reached so far.
• Maximum drawdown records the largest percentage decline from a previous peak.
• Hit rate measures the share of observations with a positive net return.

• Save quant_research.py.
• Confirm the measurement function loads by running:

python quant_research.py

What does this run confirm?

You should reach the same deterministic five-return preview. That output confirms Python parsed the measurement function successfully.

Does the measurement function fail to load?

Check that every returned dictionary key has matching quotation marks. Confirm that the four setup values align at the same indentation level.

Help me debug the measurement function.

Search the full history

The search needs one consistent process for testing every value in LOOKBACKS. Each row will preserve the candidate lookback beside its performance measurements.

• Define the window evaluator below measure() by typing this function:

def evaluate_window(
    market_returns: pd.Series,
    start: int,
    stop: int,
    cost_rate: float,
) -> list[dict[str, float]]:
    market_values = list(market_returns)
    results = []

    for lookback in LOOKBACKS:
        positions = make_positions(market_returns, lookback)
        net_returns = apply_positions(market_values, positions, cost_rate)
        metrics = measure(net_returns[start:stop])
        metrics["lookback"] = float(lookback)
        results.append(metrics)

    return results

How does the search evaluate candidates?

• The function converts the return series to values that apply_positions() can process.
• The loop builds a complete position path for each value in LOOKBACKS.
• The start and stop boundaries choose which net returns are measured.
• Each result stores its lookback so the printed metrics remain tied to the tested parameter.

• Add the winner selection and table formatting below evaluate_window() by typing these functions:

def best_by_total_return(results: list[dict[str, float]]) -> dict[str, float]:
    return max(results, key=lambda row: row["total_return"])


def print_results(title: str, results: list[dict[str, float]]) -> None:
    print(f"\n{title}")
    print("lookback | total return | max drawdown | hit rate")
    print("-" * 57)
    for row in results:
        print(
            f"{int(row['lookback']):>8} | "
            f"{row['total_return']:>11.2%} | "
            f"{row['max_drawdown']:>12.2%} | "
            f"{row['hit_rate']:>7.2%}"
        )

How is the winner presented?

• best_by_total_return() selects the result with the largest total_return value.
• print_results() formats each candidate as one table row.
• The percentage formatting makes returns, drawdowns, and hit rates directly comparable.

• Save quant_research.py.
• Check that all search helpers load by running:

python quant_research.py

What does this helper check prove?

You should see the simulated-market preview complete normally. The result confirms that the evaluator, selector, and formatter all load before the search is connected to main().

Do the search helpers contain a syntax problem?

Check that the formatted strings inside print_results() retain their opening and closing quotation marks. Confirm that the closing parenthesis after the final formatted string aligns with print(.

Help me debug the search helper functions.

The helper functions are ready. The final addition deliberately searches all 1,200 observations with a cost rate of 0.0.

• Find the simulated-market preview inside main().
• Add the full-sample search beneath the return preview by typing this code with four spaces of indentation:

    full_sample_results = evaluate_window(
        market_returns,
        start=0,
        stop=N_OBSERVATIONS,
        cost_rate=0.0,
    )
    print_results("FULL-SAMPLE SEARCH (intentionally misleading)", full_sample_results)
    full_sample_winner = best_by_total_return(full_sample_results)
    print(f"Full-sample winner: {int(full_sample_winner['lookback'])}")

What does this search do?

• start=0 begins the evaluation at the first observation.
• stop=N_OBSERVATIONS extends the evaluation through the complete simulated history.
• cost_rate=0.0 removes trading costs from this first comparison.
• The final two lines print the candidate table before selecting its highest-return row.

• Save quant_research.py.

✔️ Awesome, I've got everything!

Good work. Your file now contains the signal, scoring, search, and table functions needed for the full-sample experiment.

ⓧ I'd like to double check the full code

Compare this checksum with your file after typing it yourself:

from pathlib import Path

import numpy as np
import pandas as pd


SEED = 20261030
N_OBSERVATIONS = 1200
TRAIN_SIZE = 800
LOOKBACKS = [1, 2, 5, 10, 20]
COST_PER_UNIT_TURNOVER = 0.0002
PERMUTATIONS = 500


def simulate_market() -> pd.Series:
    rng = np.random.default_rng(SEED)
    shocks = rng.normal(0.0, 0.006, N_OBSERVATIONS)
    market_returns = [0.0]

    for time_index in range(1, N_OBSERVATIONS):
        persistence = 0.65 if time_index < TRAIN_SIZE else -0.65
        next_return = persistence * market_returns[-1] + float(shocks[time_index])
        market_returns.append(next_return)

    return pd.Series(market_returns, name="market_return")


def make_positions(market_returns: pd.Series, lookback: int) -> list[float]:
    lagged_average = (
        market_returns.rolling(lookback)
        .mean()
        .shift(1)
        .fillna(0.0)
    )

    positions = []
    for value in lagged_average:
        if value > 0.0:
            positions.append(1.0)
        elif value < 0.0:
            positions.append(-1.0)
        else:
            positions.append(0.0)
    return positions


def apply_positions(
    market_returns: list[float],
    positions: list[float],
    cost_rate: float,
    previous_position: float = 0.0,
) -> list[float]:
    net_returns = []

    for market_return, position in zip(market_returns, positions):
        turnover = abs(position - previous_position)
        trading_cost = cost_rate * turnover
        net_returns.append(position * market_return - trading_cost)
        previous_position = position

    return net_returns


def measure(net_returns: list[float]) -> dict[str, float]:
    equity = 1.0
    peak = 1.0
    max_drawdown = 0.0
    positive_periods = 0

    for net_return in net_returns:
        equity *= 1.0 + net_return
        peak = max(peak, equity)
        max_drawdown = min(max_drawdown, equity / peak - 1.0)
        if net_return > 0.0:
            positive_periods += 1

    observations = len(net_returns)
    hit_rate = positive_periods / observations if observations else 0.0
    return {
        "total_return": equity - 1.0,
        "max_drawdown": max_drawdown,
        "hit_rate": hit_rate,
    }


def evaluate_window(
    market_returns: pd.Series,
    start: int,
    stop: int,
    cost_rate: float,
) -> list[dict[str, float]]:
    market_values = list(market_returns)
    results = []

    for lookback in LOOKBACKS:
        positions = make_positions(market_returns, lookback)
        net_returns = apply_positions(market_values, positions, cost_rate)
        metrics = measure(net_returns[start:stop])
        metrics["lookback"] = float(lookback)
        results.append(metrics)

    return results


def best_by_total_return(results: list[dict[str, float]]) -> dict[str, float]:
    return max(results, key=lambda row: row["total_return"])


def print_results(title: str, results: list[dict[str, float]]) -> None:
    print(f"\n{title}")
    print("lookback | total return | max drawdown | hit rate")
    print("-" * 57)
    for row in results:
        print(
            f"{int(row['lookback']):>8} | "
            f"{row['total_return']:>11.2%} | "
            f"{row['max_drawdown']:>12.2%} | "
            f"{row['hit_rate']:>7.2%}"
        )


def main() -> None:
    market_returns = simulate_market()

    print("SIMULATED MARKET")
    print(f"Observations: {N_OBSERVATIONS}")
    print(f"Training observations: {TRAIN_SIZE}")
    print(f"First five returns: {[round(value, 6) for value in list(market_returns)[:5]]}")

    full_sample_results = evaluate_window(
        market_returns,
        start=0,
        stop=N_OBSERVATIONS,
        cost_rate=0.0,
    )
    print_results("FULL-SAMPLE SEARCH (intentionally misleading)", full_sample_results)
    full_sample_winner = best_by_total_return(full_sample_results)
    print(f"Full-sample winner: {int(full_sample_winner['lookback'])}")


if __name__ == "__main__":
    main()

How should I use this checksum?

Compare the function order, indentation, and names with your typed file. The reference matches the exact project state for this step.

Before you run the completed search, which lookback do you think will finish with the highest full-sample return?

• Run the full-sample experiment with this command:

python quant_research.py

What should I see?

Under FULL-SAMPLE SEARCH (intentionally misleading), you should see five rows comparing total return, maximum drawdown, and hit rate.

The final line should print one Full-sample winner selected from those five candidates.

The table makes one candidate look like an evidence-backed winner. The same 1,200 observations selected that candidate and scored its performance.

That shared history creates look-ahead bias because future observations influenced the parameter choice. Label this result misleading before you treat it as evidence.

Do you only see the market preview?

Confirm that the full-sample block sits inside main() beneath the first-five-returns print statement. Every line in that block needs four leading spaces.

If the table begins printing before the script stops, compare the dictionary keys in measure() with the keys used by print_results().

Help me debug the missing full-sample table.

You have exposed how easily a polished table can create false confidence. Next, you will force the selected signal to face observations that played no part in choosing it.

Test the Signal on the Future

The full-sample search found a convincing winner. Its score reused the observations that chose its parameter.

That reuse creates look-ahead bias. You will repair it with a chronological train/test split.

The training regime chooses the lookback using transaction costs. The unseen negative-persistence regime decides whether the signal deserves promotion.

In this step, get ready to:

• Predict whether training-selected momentum survives the regime change.
• Select a lookback using past observations with turnover costs.
• Evaluate the selected lookback on unseen future observations.

Predict the regime test

A momentum position follows the recent direction of returns. Your simulated market changes from positive persistence to negative persistence at TRAIN_SIZE.

This regime change tests whether a rule selected in one process can generalize to a process with the opposite dependence.

• Record your expected unseen result in the checkpoint below.

Select the lookback using past data

Past-only selection keeps observations from TRAIN_SIZE onward untouched. Charging COST_PER_UNIT_TURNOVER also prevents frequent position changes from looking free.

• In quant_research.py, find the line that prints Full-sample winner.
• Place your cursor on the empty line below it.
• Type the past-only training search exactly as shown:

    training_results = evaluate_window(
        market_returns,
        start=0,
        stop=TRAIN_SIZE,
        cost_rate=COST_PER_UNIT_TURNOVER,
    )
    print_results("TRAINING SEARCH (past data only)", training_results)
    training_winner = best_by_total_return(training_results)

What does this code do?

• The start=0 value begins evaluation at the first simulated observation.
• The stop=TRAIN_SIZE value prevents future observations from contributing to candidate selection.
• The cost_rate argument charges every unit of turnover during the training search.
• The best_by_total_return() call selects one candidate from the training results.

• Save quant_research.py.

Before you run this check, which selection table do you expect to appear after the full-sample search?

• Run the updated script in the PowerShell terminal from earlier by entering:

python quant_research.py

What should I see?

You have crossed the key boundary: candidate selection now stops at TRAIN_SIZE.

You should see TRAINING SEARCH (past data only) after the full-sample table. Its rows compare the five lookbacks using only the training period.

Training table missing?

Check that the new block sits inside main() with four spaces of indentation. Code placed below the final main() call does not run as part of the pipeline.

Check that print_results() receives training_results. Passing the earlier full-sample results repeats the misleading table.

Ask for help with the exact section that is missing. Help me debug why my past-only training table does not print from quant_research.py.

Evaluate the unseen future

The chosen lookback now comes only from the training results. Its position path extends into the test slice without reopening parameter selection.

The previous_position value carries the boundary position into the test cost calculation. This charges turnover at the first unseen observation.

• In quant_research.py, find the line that defines training_winner.
• Place your cursor on the empty line below it.
• Type the chronological test block exactly as shown:

    chosen_lookback = int(training_winner["lookback"])
    chosen_positions = make_positions(market_returns, chosen_lookback)
    test_market = list(market_returns)[TRAIN_SIZE:]
    test_positions = chosen_positions[TRAIN_SIZE:]
    test_net_returns = apply_positions(
        test_market,
        test_positions,
        COST_PER_UNIT_TURNOVER,
        previous_position=chosen_positions[TRAIN_SIZE - 1],
    )
    test_result = measure(test_net_returns)

    print("\nCHRONOLOGICAL TEST (unseen future)")
    print(f"Chosen lookback: {chosen_lookback}")
    print(f"Total return: {test_result['total_return']:.2%}")
    print(f"Maximum drawdown: {test_result['max_drawdown']:.2%}")
    print(f"Hit rate: {test_result['hit_rate']:.2%}")

How does the chronological test work?

• The chosen_lookback value converts the training winner into the integer expected by make_positions().
• The TRAIN_SIZE: slices isolate the future market returns plus their matching positions.
• The position immediately before the test boundary becomes previous_position.
• The apply_positions() call subtracts turnover costs from the unseen strategy returns.
• The measure() call calculates the unseen total return plus drawdown plus hit rate.

• Save quant_research.py.

✔️ Awesome, I've got everything!

Your chronological selection code is saved. The training search now feeds one untouched future test.

ⓧ I'd like to double check the full code

The complete quant_research.py file is your checksum:

from pathlib import Path

import numpy as np
import pandas as pd


SEED = 20261030
N_OBSERVATIONS = 1200
TRAIN_SIZE = 800
LOOKBACKS = [1, 2, 5, 10, 20]
COST_PER_UNIT_TURNOVER = 0.0002
PERMUTATIONS = 500


def simulate_market() -> pd.Series:
    rng = np.random.default_rng(SEED)
    shocks = rng.normal(0.0, 0.006, N_OBSERVATIONS)
    market_returns = [0.0]

    for time_index in range(1, N_OBSERVATIONS):
        persistence = 0.65 if time_index < TRAIN_SIZE else -0.65
        next_return = persistence * market_returns[-1] + float(shocks[time_index])
        market_returns.append(next_return)

    return pd.Series(market_returns, name="market_return")


def make_positions(market_returns: pd.Series, lookback: int) -> list[float]:
    lagged_average = (
        market_returns.rolling(lookback)
        .mean()
        .shift(1)
        .fillna(0.0)
    )

    positions = []
    for value in lagged_average:
        if value > 0.0:
            positions.append(1.0)
        elif value < 0.0:
            positions.append(-1.0)
        else:
            positions.append(0.0)
    return positions


def apply_positions(
    market_returns: list[float],
    positions: list[float],
    cost_rate: float,
    previous_position: float = 0.0,
) -> list[float]:
    net_returns = []

    for market_return, position in zip(market_returns, positions):
        turnover = abs(position - previous_position)
        trading_cost = cost_rate * turnover
        net_returns.append(position * market_return - trading_cost)
        previous_position = position

    return net_returns


def measure(net_returns: list[float]) -> dict[str, float]:
    equity = 1.0
    peak = 1.0
    max_drawdown = 0.0
    positive_periods = 0

    for net_return in net_returns:
        equity *= 1.0 + net_return
        peak = max(peak, equity)
        max_drawdown = min(max_drawdown, equity / peak - 1.0)
        if net_return > 0.0:
            positive_periods += 1

    observations = len(net_returns)
    hit_rate = positive_periods / observations if observations else 0.0
    return {
        "total_return": equity - 1.0,
        "max_drawdown": max_drawdown,
        "hit_rate": hit_rate,
    }


def evaluate_window(
    market_returns: pd.Series,
    start: int,
    stop: int,
    cost_rate: float,
) -> list[dict[str, float]]:
    market_values = list(market_returns)
    results = []

    for lookback in LOOKBACKS:
        positions = make_positions(market_returns, lookback)
        net_returns = apply_positions(market_values, positions, cost_rate)
        metrics = measure(net_returns[start:stop])
        metrics["lookback"] = float(lookback)
        results.append(metrics)

    return results


def best_by_total_return(results: list[dict[str, float]]) -> dict[str, float]:
    return max(results, key=lambda row: row["total_return"])


def print_results(title: str, results: list[dict[str, float]]) -> None:
    print(f"\n{title}")
    print("lookback | total return | max drawdown | hit rate")
    print("-" * 57)
    for row in results:
        print(
            f"{int(row['lookback']):>8} | "
            f"{row['total_return']:>11.2%} | "
            f"{row['max_drawdown']:>12.2%} | "
            f"{row['hit_rate']:>7.2%}"
        )


def main() -> None:
    market_returns = simulate_market()

    print("SIMULATED MARKET")
    print(f"Observations: {N_OBSERVATIONS}")
    print(f"Training observations: {TRAIN_SIZE}")
    print(f"First five returns: {[round(value, 6) for value in list(market_returns)[:5]]}")

    full_sample_results = evaluate_window(
        market_returns,
        start=0,
        stop=N_OBSERVATIONS,
        cost_rate=0.0,
    )
    print_results("FULL-SAMPLE SEARCH (intentionally misleading)", full_sample_results)
    full_sample_winner = best_by_total_return(full_sample_results)
    print(f"Full-sample winner: {int(full_sample_winner['lookback'])}")

    training_results = evaluate_window(
        market_returns,
        start=0,
        stop=TRAIN_SIZE,
        cost_rate=COST_PER_UNIT_TURNOVER,
    )
    print_results("TRAINING SEARCH (past data only)", training_results)
    training_winner = best_by_total_return(training_results)

    chosen_lookback = int(training_winner["lookback"])
    chosen_positions = make_positions(market_returns, chosen_lookback)
    test_market = list(market_returns)[TRAIN_SIZE:]
    test_positions = chosen_positions[TRAIN_SIZE:]
    test_net_returns = apply_positions(
        test_market,
        test_positions,
        COST_PER_UNIT_TURNOVER,
        previous_position=chosen_positions[TRAIN_SIZE - 1],
    )
    test_result = measure(test_net_returns)

    print("\nCHRONOLOGICAL TEST (unseen future)")
    print(f"Chosen lookback: {chosen_lookback}")
    print(f"Total return: {test_result['total_return']:.2%}")
    print(f"Maximum drawdown: {test_result['max_drawdown']:.2%}")
    print(f"Hit rate: {test_result['hit_rate']:.2%}")


if __name__ == "__main__":
    main()

Before you run the final check, do you expect the training winner to remain convincing in the negative-persistence future?

• Run the complete chronological experiment in the PowerShell terminal from earlier by entering:

python quant_research.py

What should I see?

You should see TRAINING SEARCH (past data only) followed by CHRONOLOGICAL TEST (unseen future).

The second section reports the chosen lookback plus its total return plus its maximum drawdown plus its hit rate. These metrics come only from observations after TRAIN_SIZE.

A negative or materially weaker test return is evidence against promotion. The untouched future slice has exposed the regime change.

Chronological test not printing?

Check that the chronological block remains inside main(). Every line in the block starts with four spaces.

Check both TRAIN_SIZE: slices. A missing colon selects one value instead of the unseen range.

Check that previous_position uses TRAIN_SIZE - 1. Help me debug my chronological test in quant_research.py.

• Compare the Full-sample winner with the Chosen lookback shown by the chronological test.

That is the hard comparison complete: your training-selected signal has now faced a cost-aware unseen test. Next, you will test whether its result beats a chance-based null.

Test Against Chance and Write the Memo

Your past-only selection has forced the momentum rule to face an unseen regime. The chronological test now gives you evidence from data that never influenced the chosen lookback.

One backtest can still reflect a lucky path. In this step, a permutation test creates a chance-based comparison before your program records its verdict in a research memo.

In this step, get ready to:

• Compare the unseen test return with shuffled return paths.
• Calculate a one-sided p-value from the null outcomes.
• Generate and defend a research memo with an evidence-based verdict.

Predict what chance should produce

The null experiment shuffles the unseen returns while holding the chosen position path fixed. It also keeps the turnover costs fixed.

What does the null test ask?

Each shuffle breaks the original ordering of the test returns. This removes the strategy's opportunity to benefit from that specific sequence.

The one-sided p-value counts how often a shuffled path earns at least the observed return. A large value gives the observed result little support against this chance-based comparison.

Build the permutation test

A reproducible null experiment needs its own seeded generator. The function uses SEED + 1 so its shuffles remain deterministic without reusing the market simulation's random stream.

• In quant_research.py, locate def main() near the bottom of the file.
• Type the permutation-test function directly above def main() using this code:

def run_permutation_test(
    market_returns: pd.Series,
    positions: list[float],
    observed_total_return: float,
) -> float:
    rng = np.random.default_rng(SEED + 1)
    test_market = list(market_returns)[TRAIN_SIZE:]
    test_positions = positions[TRAIN_SIZE:]
    previous_position = positions[TRAIN_SIZE - 1]
    null_scores = []

    for _ in range(PERMUTATIONS):
        permuted_market = list(rng.permutation(test_market))
        permuted_net = apply_positions(
            permuted_market,
            test_positions,
            COST_PER_UNIT_TURNOVER,
            previous_position,
        )
        null_scores.append(measure(permuted_net)["total_return"])

    outcomes_at_least_as_large = sum(
        score >= observed_total_return for score in null_scores
    )
    return (1 + outcomes_at_least_as_large) / (PERMUTATIONS + 1)

What does this function do?

• The test slices market_returns and positions at TRAIN_SIZE. This keeps the null experiment inside the unseen test period.
• The loop shuffles the test returns 500 times. Every path uses the original test positions.
• The call to apply_positions() preserves the original starting position. It also charges COST_PER_UNIT_TURNOVER.
• The return statement applies a one-count correction to the numerator. It applies the same correction to the denominator.

• In main(), find the line that prints the unseen hit rate.
• Type the permutation-test call directly below that line:

    p_value = run_permutation_test(
        market_returns,
        chosen_positions,
        test_result["total_return"],
    )
    print("\nPERMUTATION TEST")
    print(f"One-sided p-value: {p_value:.4f}")

How is the observed result tested?

The function receives the observed unseen return as its comparison threshold. The two print statements make the resulting p-value visible in the terminal.

• Save quant_research.py by pressing Ctrl+S.

Before you run the updated program, do you expect the observed momentum result to beat most of the shuffled paths?

• Run the null experiment from the activated PowerShell terminal with this command:

python quant_research.py

What should I see?

Below the chronological test, you should see PERMUTATION TEST followed by One-sided p-value: 1.0000.

Every shuffled score reached at least the deeply negative observed return. This result provides no evidence for promoting the signal.

Permutation test not running?

• Check that run_permutation_test() appears above main().
• Check that PERMUTATIONS remains set to 500 near the top of the file.
• Check that the function receives chosen_positions from the past-only winner.

Help me debug my permutation test.

Write the research memo

A statistical result needs a scoped conclusion. The memo ties the hypothesis to the method before reporting the evidence.

Why write the verdict into the pipeline?

A generated memo records the assumptions used for the same run that produced the metrics. This keeps the conclusion connected to the evidence.

The verdict permits another falsification test only when the unseen return is positive and the p-value is at most 0.05. Every other result rejects promotion.

• Type write_report() directly below run_permutation_test() using the next three consecutive blocks.
• Finish all three blocks before saving or running the file.

def write_report(
    full_sample_winner: dict[str, float],
    training_winner: dict[str, float],
    test_result: dict[str, float],
    p_value: float,
) -> str:
    passed = test_result["total_return"] > 0.0 and p_value <= 0.05
    verdict = (
        "Promising enough for another falsification test, not deployment."
        if passed
        else "Do not promote: the signal failed the out-of-sample evidence hurdle."
    )

    report = f"""# Quant Research Memo

How does the verdict work?

The passed condition requires both a positive unseen return and a small p-value. The conditional expression converts that evidence hurdle into a concise verdict.

The final line opens a multiline formatted string. Keep typing the next block inside that string.

## Question

Does a lagged momentum rule generalize when the return process changes from positive to negative persistence?

## Hypothesis

A short-lookback momentum position should perform in the positive-persistence training regime but should weaken or reverse when the unseen regime becomes negatively persistent.

## Method

- Simulated {N_OBSERVATIONS} observations with seed `{SEED}` and a regime boundary at observation {TRAIN_SIZE}.
- Compared lookbacks {LOOKBACKS}.
- Selected the production candidate using training observations only.
- Charged `{COST_PER_UNIT_TURNOVER:.4f}` per unit of position turnover.
- Compared the unseen test return with {PERMUTATIONS} permuted-return paths while holding positions and costs fixed.

What does the method record?

The opening sections state the economic question and its falsifiable expectation. The method records the seed, regime boundary, candidate lookbacks, cost rate, and null experiment.

The formatted values make the memo reflect the program's actual configuration. Continue the same multiline string with the final block.

## Results

- Misleading full-sample winner: lookback {int(full_sample_winner['lookback'])}, total return {full_sample_winner['total_return']:.2%}.
- Training-selected winner: lookback {int(training_winner['lookback'])}, training return {training_winner['total_return']:.2%}.
- Unseen test return: {test_result['total_return']:.2%}.
- Unseen test maximum drawdown: {test_result['max_drawdown']:.2%}.
- Unseen test hit rate: {test_result['hit_rate']:.2%}.
- One-sided permutation p-value: {p_value:.4f}.

## Verdict

{verdict}

## Limitations

This is synthetic data from one seeded path, the cost model is deliberately simple, the parameter set is small, and the permutation test breaks serial ordering. The exercise evaluates research discipline, not the profitability of momentum in real markets.
"""

    Path("research_report.md").write_text(report, encoding="utf-8")
    return verdict

How is the memo completed?

The results separate the misleading full-sample winner from the training-selected winner. They report the unseen metrics and p-value independently.

The limitations section narrows the conclusion to this synthetic experiment. Path.write_text() saves the completed memo as research_report.md using utf-8 encoding.

• In main(), locate the line that prints the one-sided p-value.
• Type the report call directly below that line:

    verdict = write_report(
        full_sample_winner,
        training_winner,
        test_result,
        p_value,
    )
    print(f"Verdict: {verdict}")
    print("Report written to research_report.md")

What completes the pipeline?

The report writer receives the misleading winner, the past-only winner, the unseen result, and the p-value. It returns the same verdict that appears in the memo.

The final confirmation names research_report.md so you can inspect the generated artifact immediately.

• Save quant_research.py by pressing Ctrl+S.

✔️ Awesome, I've got everything!

Your permutation test and report writer are connected. Double check that quant_research.py is saved before the final run.

ⓧ I'd like to double check the full code

Compare your saved quant_research.py with this complete answer key:

from pathlib import Path

import numpy as np
import pandas as pd


SEED = 20261030
N_OBSERVATIONS = 1200
TRAIN_SIZE = 800
LOOKBACKS = [1, 2, 5, 10, 20]
COST_PER_UNIT_TURNOVER = 0.0002
PERMUTATIONS = 500


def simulate_market() -> pd.Series:
    rng = np.random.default_rng(SEED)
    shocks = rng.normal(0.0, 0.006, N_OBSERVATIONS)
    market_returns = [0.0]

    for time_index in range(1, N_OBSERVATIONS):
        persistence = 0.65 if time_index < TRAIN_SIZE else -0.65
        next_return = persistence * market_returns[-1] + float(shocks[time_index])
        market_returns.append(next_return)

    return pd.Series(market_returns, name="market_return")


def make_positions(market_returns: pd.Series, lookback: int) -> list[float]:
    lagged_average = (
        market_returns.rolling(lookback)
        .mean()
        .shift(1)
        .fillna(0.0)
    )

    positions = []
    for value in lagged_average:
        if value > 0.0:
            positions.append(1.0)
        elif value < 0.0:
            positions.append(-1.0)
        else:
            positions.append(0.0)
    return positions


def apply_positions(
    market_returns: list[float],
    positions: list[float],
    cost_rate: float,
    previous_position: float = 0.0,
) -> list[float]:
    net_returns = []

    for market_return, position in zip(market_returns, positions):
        turnover = abs(position - previous_position)
        trading_cost = cost_rate * turnover
        net_returns.append(position * market_return - trading_cost)
        previous_position = position

    return net_returns


def measure(net_returns: list[float]) -> dict[str, float]:
    equity = 1.0
    peak = 1.0
    max_drawdown = 0.0
    positive_periods = 0

    for net_return in net_returns:
        equity *= 1.0 + net_return
        peak = max(peak, equity)
        max_drawdown = min(max_drawdown, equity / peak - 1.0)
        if net_return > 0.0:
            positive_periods += 1

    observations = len(net_returns)
    hit_rate = positive_periods / observations if observations else 0.0
    return {
        "total_return": equity - 1.0,
        "max_drawdown": max_drawdown,
        "hit_rate": hit_rate,
    }


def evaluate_window(
    market_returns: pd.Series,
    start: int,
    stop: int,
    cost_rate: float,
) -> list[dict[str, float]]:
    market_values = list(market_returns)
    results = []

    for lookback in LOOKBACKS:
        positions = make_positions(market_returns, lookback)
        net_returns = apply_positions(market_values, positions, cost_rate)
        metrics = measure(net_returns[start:stop])
        metrics["lookback"] = float(lookback)
        results.append(metrics)

    return results


def best_by_total_return(results: list[dict[str, float]]) -> dict[str, float]:
    return max(results, key=lambda row: row["total_return"])


def print_results(title: str, results: list[dict[str, float]]) -> None:
    print(f"\n{title}")
    print("lookback | total return | max drawdown | hit rate")
    print("-" * 57)
    for row in results:
        print(
            f"{int(row['lookback']):>8} | "
            f"{row['total_return']:>11.2%} | "
            f"{row['max_drawdown']:>12.2%} | "
            f"{row['hit_rate']:>7.2%}"
        )


def run_permutation_test(
    market_returns: pd.Series,
    positions: list[float],
    observed_total_return: float,
) -> float:
    rng = np.random.default_rng(SEED + 1)
    test_market = list(market_returns)[TRAIN_SIZE:]
    test_positions = positions[TRAIN_SIZE:]
    previous_position = positions[TRAIN_SIZE - 1]
    null_scores = []

    for _ in range(PERMUTATIONS):
        permuted_market = list(rng.permutation(test_market))
        permuted_net = apply_positions(
            permuted_market,
            test_positions,
            COST_PER_UNIT_TURNOVER,
            previous_position,
        )
        null_scores.append(measure(permuted_net)["total_return"])

    outcomes_at_least_as_large = sum(
        score >= observed_total_return for score in null_scores
    )
    return (1 + outcomes_at_least_as_large) / (PERMUTATIONS + 1)


def write_report(
    full_sample_winner: dict[str, float],
    training_winner: dict[str, float],
    test_result: dict[str, float],
    p_value: float,
) -> str:
    passed = test_result["total_return"] > 0.0 and p_value <= 0.05
    verdict = (
        "Promising enough for another falsification test, not deployment."
        if passed
        else "Do not promote: the signal failed the out-of-sample evidence hurdle."
    )

    report = f"""# Quant Research Memo

## Question

Does a lagged momentum rule generalize when the return process changes from positive to negative persistence?

## Hypothesis

A short-lookback momentum position should perform in the positive-persistence training regime but should weaken or reverse when the unseen regime becomes negatively persistent.

## Method

- Simulated {N_OBSERVATIONS} observations with seed `{SEED}` and a regime boundary at observation {TRAIN_SIZE}.
- Compared lookbacks {LOOKBACKS}.
- Selected the production candidate using training observations only.
- Charged `{COST_PER_UNIT_TURNOVER:.4f}` per unit of position turnover.
- Compared the unseen test return with {PERMUTATIONS} permuted-return paths while holding positions and costs fixed.

## Results

- Misleading full-sample winner: lookback {int(full_sample_winner['lookback'])}, total return {full_sample_winner['total_return']:.2%}.
- Training-selected winner: lookback {int(training_winner['lookback'])}, training return {training_winner['total_return']:.2%}.
- Unseen test return: {test_result['total_return']:.2%}.
- Unseen test maximum drawdown: {test_result['max_drawdown']:.2%}.
- Unseen test hit rate: {test_result['hit_rate']:.2%}.
- One-sided permutation p-value: {p_value:.4f}.

## Verdict

{verdict}

## Limitations

This is synthetic data from one seeded path, the cost model is deliberately simple, the parameter set is small, and the permutation test breaks serial ordering. The exercise evaluates research discipline, not the profitability of momentum in real markets.
"""

    Path("research_report.md").write_text(report, encoding="utf-8")
    return verdict


def main() -> None:
    market_returns = simulate_market()

    print("SIMULATED MARKET")
    print(f"Observations: {N_OBSERVATIONS}")
    print(f"Training observations: {TRAIN_SIZE}")
    print(f"First five returns: {[round(value, 6) for value in list(market_returns)[:5]]}")

    full_sample_results = evaluate_window(
        market_returns,
        start=0,
        stop=N_OBSERVATIONS,
        cost_rate=0.0,
    )
    print_results("FULL-SAMPLE SEARCH (intentionally misleading)", full_sample_results)
    full_sample_winner = best_by_total_return(full_sample_results)
    print(f"Full-sample winner: {int(full_sample_winner['lookback'])}")

    training_results = evaluate_window(
        market_returns,
        start=0,
        stop=TRAIN_SIZE,
        cost_rate=COST_PER_UNIT_TURNOVER,
    )
    print_results("TRAINING SEARCH (past data only)", training_results)
    training_winner = best_by_total_return(training_results)

    chosen_lookback = int(training_winner["lookback"])
    chosen_positions = make_positions(market_returns, chosen_lookback)
    test_market = list(market_returns)[TRAIN_SIZE:]
    test_positions = chosen_positions[TRAIN_SIZE:]
    test_net_returns = apply_positions(
        test_market,
        test_positions,
        COST_PER_UNIT_TURNOVER,
        previous_position=chosen_positions[TRAIN_SIZE - 1],
    )
    test_result = measure(test_net_returns)

    print("\nCHRONOLOGICAL TEST (unseen future)")
    print(f"Chosen lookback: {chosen_lookback}")
    print(f"Total return: {test_result['total_return']:.2%}")
    print(f"Maximum drawdown: {test_result['max_drawdown']:.2%}")
    print(f"Hit rate: {test_result['hit_rate']:.2%}")

    p_value = run_permutation_test(
        market_returns,
        chosen_positions,
        test_result["total_return"],
    )
    print("\nPERMUTATION TEST")
    print(f"One-sided p-value: {p_value:.4f}")

    verdict = write_report(
        full_sample_winner,
        training_winner,
        test_result,
        p_value,
    )
    print(f"Verdict: {verdict}")
    print("Report written to research_report.md")


if __name__ == "__main__":
    main()

Run and defend the complete pipeline

The complete program now turns one deterministic experiment into a visible verdict and a saved memo. The final run checks both outputs together.

Before you run it, which verdict do you expect from a negative unseen return with a large p-value?

• Run the completed research pipeline with this command:

python quant_research.py

What should I see?

The terminal should print PERMUTATION TEST and a one-sided p-value. It should then print Verdict: Do not promote: the signal failed the out-of-sample evidence hurdle..

The final line should read Report written to research_report.md. That line confirms the memo was generated from the same run.

• Select research_report.md in the Visual Studio Code Explorer sidebar.
• Confirm the memo contains Question, Hypothesis, Method, Results, Verdict, and Limitations.

How should you defend the verdict?

Lead with the chronological test because it measures performance after selection. Use the p-value to show that the observed result failed to stand out against the shuffled paths.

Keep the conclusion scoped to this simulation and this validation design. The evidence rejects promotion under the stated hurdle.

• Deliver a 60-second spoken defense that connects the unseen return, the p-value, the verdict, and the limitations.

Memo missing or incomplete?

• Check that write_report() appears above main().
• Check that the call to write_report() appears after the p-value is calculated.
• Check that the multiline report string closes before Path("research_report.md").write_text(...).

Help me debug the generated memo.

You have completed the full hypothesis-to-verdict workflow. Your pipeline now rejects a fragile signal with reproducible evidence and records the reasoning in a research memo.

Secret mission

Red-Team Your Own Research

The memo reaches a verdict from one controlled study. Review it as if a deployment decision depended on your skepticism. Identify three threats before precommitting the next falsification experiment with its promotion hurdle.

Clean Up Your Resources

Clean Up Your Resources

Your local research lab creates no cloud or market-data costs. Choose whether to keep it ready, pause it for later, or delete it.

Resources you used:

• Local project folder quant-alpha-lab.
• Project-specific virtual environment venv stored inside quant-alpha-lab.

Keep everything running

No action needed. Choose this if you want to rerun the experiment or reference your research memo later.

• Keep quant-alpha-lab in its current location.
• Keep the existing PowerShell terminal active in Visual Studio Code for immediate reruns.
• Use research_report.md as a record of your verdict and red-team review.

Pause - I'll come back to this later

Pausing ends the active terminal session while preserving every local file. You can return to the same workspace later.

• End the active PowerShell terminal session through the terminal controls in Visual Studio Code.
• Close Visual Studio Code.
• Leave quant-alpha-lab in its current location.

Delete - I don't want to use this again

Deleting the folder is permanent, so this option fits only when you no longer need the lab. It removes your script, memo, red-team review, and project-specific environment.

• Copy research_report.md outside quant-alpha-lab if you want to preserve your memo.
• End the PowerShell terminal session if it is still active.
• Close Visual Studio Code if it is still open.
• Locate quant-alpha-lab in the Windows file browser.
• Use the permanent-delete option on quant-alpha-lab.
• Approve the confirmation prompt.
• Return to the location that previously contained quant-alpha-lab.

You should no longer see the quant-alpha-lab folder in its original location.

Nice Work!

Nice Work!

You made it! Your deterministic quant research lab now exposes false confidence in a momentum backtest. It tests the signal on unseen future data before writing an evidence-based verdict.

What you learned:

• Built a deterministic market simulation with a known regime change. The fixed seed makes every research run reproducible.
• Exposed full-sample parameter selection as a source of look-ahead bias. Replaced it with chronological selection plus unseen testing.
• Compared the unseen result with a chance-based null using a permutation test. Defended the final verdict in research_report.md. The memo records assumptions, costs, results, plus limitations.
• Secret Mission: red-teamed the research design by identifying three threats to validity. Precommitted the next falsification experiment plus its promotion hurdle.

Ready to quiz yourself?