Build an Invoice Investigation Agent
Build an auditable AI agent to investigate synthetic invoices with local tools.
Introduction
30 Second Summary
Invoice reviews become difficult when one payment record points to several places for answers. A reviewer needs to compare the evidence before deciding what happens next.
In this project, you will build LedgerGuard, a command-line AI agent that uses OpenRouter to investigate synthetic payment records. It will gather evidence with four local tools before preparing a recommendation for a human reviewer.
What You'll Build
You will run LedgerGuard against synthetic payment records to produce a human-ready report grounded in four retrieved evidence sources.
By the end of this project, you'll have:
- A model-only baseline for PAY-1048 that clearly names the company records it cannot access.
- Evidence-backed investigations that flag the mismatch in PAY-1048 for investigation while identifying PAY-1050 as ready for human approval.
- A JSON Lines audit trail in audit.jsonl where you can inspect every tool call plus the final report.
- Secret Mission: Add a missing purchase order case that proves LedgerGuard escalates incomplete evidence without guessing.
What do I need before starting?
You need basic experience with Python functions and dictionaries. You also need Python 3.11 plus Visual Studio Code on Windows.
Step 1 covers your OpenRouter account setup and API key.
Before We Start
This is your moment to lock in LedgerGuard's purpose before the hands-on work begins. It investigates synthetic invoice evidence and prepares recommendations while every payment decision stays with a human reviewer.
Set Up LedgerGuard and OpenRouter
LedgerGuard starts with a local workspace for its synthetic finance records. Its first request to OpenRouter also needs a credential that stays outside the project code.
In this step, you'll create the ledgerguard workspace in Visual Studio Code. You'll use the integrated Windows PowerShell terminal to verify Python 3.11 plus your session credential.
In this step, get ready to:
- Create the local ledgerguard workspace.
- Generate an OpenRouter API key for LedgerGuard.
- Verify the runtime setup from Windows PowerShell.
Create the project workspace
The workspace gives every future LedgerGuard file one predictable home. Opening the folder in Visual Studio Code also roots its integrated terminal in the correct location.
- Press the Windows key to open the search bar.
- Type Visual Studio Code into the search bar.
- Press Enter to open Visual Studio Code.
- Click File in the top menu bar.
- Click Open Folder.
- Select your Desktop in the folder picker.
- Click Select Folder.
The Explorer sidebar now shows your Desktop. This location keeps the LedgerGuard workspace easy to find.
- Click Terminal in the top menu bar.
- Click New Terminal.
- Confirm the terminal profile is Windows PowerShell.
- Create the project directory and enter it by running these commands:
New-Item -ItemType Directory -Path .\ledgerguard
Set-Location -Path .\ledgerguardWhat do these commands do?
- The New-Item command creates the ledgerguard folder inside your Desktop workspace.
- The Set-Location command moves the current PowerShell location into that folder.
- Click File in the top menu bar.
- Click Open Folder.
- Select the new ledgerguard folder on your Desktop.
- Click Select Folder.
- Click Terminal in the top menu bar.
- Click New Terminal.
The Explorer sidebar should show the open ledgerguard folder. The integrated terminal path should end with that folder name.
Your workspace is in place. Every script created later now has a clear local home.
Folder missing from the Explorer?
Confirm that your Desktop was open when you created the folder. Use File followed by Open Folder to select the ledgerguard folder directly.
If a different terminal profile opens, use the profile selector in the terminal panel to switch to Windows PowerShell.
Help me find and open my LedgerGuard project folder in Visual Studio Code.
Create the OpenRouter API key
An API key lets OpenRouter associate LedgerGuard's requests with your account. This is the sensitive part, so keep the key inside the current terminal session and out of screenshots.
- Open your browser to the OpenRouter API key page.
- Create an OpenRouter account if you do not have one.
- Sign in to your OpenRouter account.
- Create a new API key from the key page.
- Copy the generated API key.
- Return to the integrated Windows PowerShell terminal from earlier.
- Set the key for this terminal session by replacing your-api-key-here in the command below:
$env:OPENROUTER_API_KEY="your-api-key-here"How does session access work?
The OPENROUTER_API_KEY environment variable makes the credential available to Python processes started from this terminal. Windows PowerShell stores the value for the current session only.
Closing the terminal removes the session value. Keep this terminal open throughout the project.
Some providers used by the free router may log prompts or completions. LedgerGuard uses synthetic records to keep private financial data out of routed requests.
- Keep this integrated Windows PowerShell terminal open for the remaining steps.
Having trouble setting the key?
Check that you replaced the placeholder with the key copied from OpenRouter. Keep the quotation marks around the value.
If you opened another terminal after setting the key, return to the original terminal or run the environment-variable command again in the new session.
Help me set OPENROUTER_API_KEY safely in my Visual Studio Code Windows PowerShell terminal.
Verify the session setup
The final check confirms that the correct Python runtime is active. It also confirms that Python can detect the OpenRouter credential without printing the secret.
Before you run these checks, predict whether the runtime and session key are both ready.
- Verify Python plus the session credential by running these commands:
python --version
python -c "import os; print('Key loaded' if os.getenv('OPENROUTER_API_KEY') else 'Key missing')"What do these checks prove?
- The first check reports the active Python version in the integrated terminal.
- The second check uses Python to test whether the session environment contains the OpenRouter key.
- The second check prints a status message without exposing the credential value.
You should see Python 3.11 followed by Key loaded.
Your first checkpoint is complete. LedgerGuard now has the workspace plus session access needed for its first model request.
Seeing a missing key or different version?
If the key check reports a missing value, run the environment-variable command from the previous substep in this same terminal. Repeat both verification checks afterward.
If the Python version does not begin with 3.11, confirm that Visual Studio Code is using the Python installation already available on your Windows machine.
Help me diagnose my LedgerGuard Python or session-key verification result.
✔️ Awesome, I've got everything!
Your ledgerguard folder is open in Visual Studio Code. Its integrated Windows PowerShell terminal confirms Python 3.11.
The OPENROUTER_API_KEY value is available in the current session. Keep this terminal open while you continue.
ⓧ I'd like to double check the full code
This step creates setup artifacts without adding project code files. Your setup should match these exact states.
OPENROUTER_API_KEY: OpenRouter API key set in the current PowerShell session only.
ledgerguard/: Local project folder created and opened in VS Code.
LedgerGuard is ready to reach OpenRouter. Next, you'll make the evidence gap visible by asking the model to review a private record ID without access to any company records.
Expose the Model-Only Evidence Gap
Your LedgerGuard workspace is ready. Your OpenRouter credential can now carry the project's first model request.
A model given only PAY-1048 has no invoice or purchase order to inspect. It also lacks vendor history plus the fictional payment policy needed for a reliable review.
In this step, get ready to:
- Build a standard-library client that sends requests to OpenRouter.
- Expose a model-only baseline review through the command line.
- Run the baseline with PAY-1048 to reveal the missing evidence.
Create the OpenRouter request client
The request client serializes Python dictionaries as JSON. It sends that payload to an HTTP API using Python's built-in urllib.request module.
Why raw HTTP?
Python 3.11 already includes the modules needed for HTTP requests. This keeps LedgerGuard free from third-party packages.
You can see the request body plus the authentication header directly. That makes the agent's API traffic easier to trace.
- Select Explorer from the Activity Bar in Visual Studio Code.
- Select New File... in the Explorer view.
- Enter ledgerguard.py as the file name.
- Press Enter to create the file.
- Add the standard-library imports plus the OpenRouter settings by copying this into ledgerguard.py:
import json
import os
import sys
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
API_URL = "https://openrouter.ai/api/v1/chat/completions"
MODEL = "openrouter/free"What does this setup do?
- The imports cover JSON conversion plus environment access.
- The imports also cover command-line arguments plus HTTP error handling.
- The API_URL value points every request at OpenRouter's chat completions endpoint.
- The MODEL value uses the Free Models Router.
- Save ledgerguard.py.
- Return to the Explorer view.
You should see ledgerguard.py inside the open ledgerguard folder.
File missing from Explorer?
Check that you created ledgerguard.py inside the open ledgerguard folder. A similarly named file outside that folder does not belong to this project.
Ask for help checking the file location if it still does not appear. Help me confirm where ledgerguard.py was created in Visual Studio Code.
The helper builds one request around the supplied message history. Its optional tools parameter lets this baseline omit tools completely.
- Add the request helper below MODEL by copying this code:
def call_openrouter(
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
api_key = os.getenv("OPENROUTER_API_KEY")
if not api_key:
raise RuntimeError("OPENROUTER_API_KEY is not set in this PowerShell session.")
payload: dict[str, Any] = {"model": MODEL, "messages": messages}
if tools is not None:
payload["tools"] = tools
request = Request(
API_URL,
data=json.dumps(payload).encode("utf-8"),
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
},
method="POST",
)
try:
with urlopen(request, timeout=90) as response:
return json.loads(response.read().decode("utf-8"))
except HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"OpenRouter returned HTTP {error.code}: {detail}") from error
except URLError as error:
raise RuntimeError(f"Could not reach OpenRouter: {error.reason}") from errorWhat does this code do?
- The helper reads OPENROUTER_API_KEY from the current PowerShell session.
- The payload always includes the selected model plus the message history.
- The request includes tools only when the caller supplies them.
- The error handlers turn failed HTTP requests plus connection failures into readable Python errors.
- Save ledgerguard.py.
- Locate call_openrouter() in the saved file.
The function should begin below MODEL. Its final handler should remain aligned with the preceding HTTP handler.
Function indentation look uneven?
Make sure each statement inside call_openrouter() uses four spaces per indentation level. Keep both error handlers aligned with the try statement.
Use this guided check if the nesting remains unclear. Help me check the indentation of call_openrouter() without changing its behavior.
Add the baseline review path
The baseline prompt gives the model one internal record ID. It explicitly blocks invented company records.
- Add baseline_review() below call_openrouter() by copying this code:
def baseline_review(record_id: str) -> str:
messages = [
{
"role": "system",
"content": (
"You are testing the limits of a model without company tools. "
"Do not invent records or claim to have inspected private data."
),
},
{
"role": "user",
"content": (
f"Investigate payment record {record_id}. You have only this ID and no company records. "
"Explain what evidence is missing before a reliable review is possible."
),
},
]
response = call_openrouter(messages)
return response["choices"][0]["message"]["content"]How does the baseline stay grounded?
- The system message forbids claims about unseen private data.
- The user message supplies only the selected record_id.
- The call omits the optional tools argument.
- The function returns the model's text from the first response choice.
- Save ledgerguard.py.
- Locate the call to call_openrouter(messages) near the end of baseline_review().
That call has no tool list. The model receives the two messages plus no access to LedgerGuard records.
Baseline function nested inside the helper?
Keep def baseline_review aligned with def call_openrouter. Both functions must begin at the left edge of the file.
Use this comparison prompt if the function is still nested. Help me separate baseline_review() from call_openrouter() using the provided code.
The command-line entry point recognizes the --baseline flag. It prints the baseline response before returning.
- Add the command-line entry point below baseline_review() by copying this code:
def main() -> None:
if len(sys.argv) > 1 and sys.argv[1] == "--baseline":
record_id = sys.argv[2] if len(sys.argv) > 2 else "PAY-1048"
print(baseline_review(record_id))
return
record_id = sys.argv[1] if len(sys.argv) > 1 else "PAY-1048"
print(f"Investigating {record_id}...")
print(investigate(record_id))
if __name__ == "__main__":
main()How does the command path work?
- The first condition checks whether the first command-line argument is --baseline.
- The next argument becomes the payment record ID.
- The baseline branch prints the returned model response.
- The immediate return keeps this command inside the no-tools baseline path.
- Save ledgerguard.py.
- Confirm the final line calls main().
Run the evidence-gap test
This run asks for an investigation without supplying any operational records. Before you run it, which evidence can the model genuinely verify from PAY-1048 alone?
The routed model may take a moment to answer. A short pause means OpenRouter is processing the request.
- Run the model-only review in the PowerShell terminal with this command:
python ledgerguard.py --baseline PAY-1048What should I see?
The response should say that it cannot verify the invoice from the ID alone. It should name invoice evidence plus purchase order evidence as missing.
It should also ask for vendor history plus payment policy. Those missing records show why free-form reasoning cannot complete this investigation.
You have made the central safety gap visible. LedgerGuard asks for evidence before producing a reliable review.
Baseline request failed?
Keep using the PowerShell terminal from the setup step. A new terminal does not inherit the session-scoped OPENROUTER_API_KEY value.
Check the error details for an authentication problem or a network problem. Help me diagnose my LedgerGuard baseline request without exposing my API key.
✔️ Awesome, I've got everything!
Great. Double check that ledgerguard.py is saved. Your baseline response should identify the company records it cannot access.
ⓧ I'd like to double check the full code
Compare your saved ledgerguard.py with this complete baseline client:
import json
import os
import sys
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
API_URL = "https://openrouter.ai/api/v1/chat/completions"
MODEL = "openrouter/free"
def call_openrouter(
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
api_key = os.getenv("OPENROUTER_API_KEY")
if not api_key:
raise RuntimeError("OPENROUTER_API_KEY is not set in this PowerShell session.")
payload: dict[str, Any] = {"model": MODEL, "messages": messages}
if tools is not None:
payload["tools"] = tools
request = Request(
API_URL,
data=json.dumps(payload).encode("utf-8"),
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
},
method="POST",
)
try:
with urlopen(request, timeout=90) as response:
return json.loads(response.read().decode("utf-8"))
except HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"OpenRouter returned HTTP {error.code}: {detail}") from error
except URLError as error:
raise RuntimeError(f"Could not reach OpenRouter: {error.reason}") from error
def baseline_review(record_id: str) -> str:
messages = [
{
"role": "system",
"content": (
"You are testing the limits of a model without company tools. "
"Do not invent records or claim to have inspected private data."
),
},
{
"role": "user",
"content": (
f"Investigate payment record {record_id}. You have only this ID and no company records. "
"Explain what evidence is missing before a reliable review is possible."
),
},
]
response = call_openrouter(messages)
return response["choices"][0]["message"]["content"]
def main() -> None:
if len(sys.argv) > 1 and sys.argv[1] == "--baseline":
record_id = sys.argv[2] if len(sys.argv) > 2 else "PAY-1048"
print(baseline_review(record_id))
return
record_id = sys.argv[1] if len(sys.argv) > 1 else "PAY-1048"
print(f"Investigating {record_id}...")
print(investigate(record_id))
if __name__ == "__main__":
main()Your baseline now exposes why a private invoice cannot be investigated from an ID alone. Next, you will give LedgerGuard controlled access to synthetic financial records.
Build the Financial Record Tools
Your baseline review showed that OpenRouter cannot investigate a private payment record from its ID alone. LedgerGuard now needs controlled access to the missing business evidence.
Local Python tools provide that access through synthetic records. Each tool returns structured JSON data for the agent to use later.
In this step, get ready to:
- Define four typed financial record shapes.
- Add synthetic evidence for an exception case and a clean case.
- Test four local lookup tools with formatted JSON output.
Define the financial record shapes
A frozen dataclass gives each financial record a predictable structure. Freezing the record prevents its fields from changing after creation.
- Click the New File icon in the Explorer sidebar in Visual Studio Code.
- Enter records.py as the file name.
- Add the imports plus the first two record shapes by pasting this code:
from dataclasses import asdict, dataclass
import json
from typing import Any, Callable
@dataclass(frozen=True)
class LineItem:
description: str
quantity: int
unit_price: float
@dataclass(frozen=True)
class Invoice:
record_id: str
invoice_number: str
vendor_id: str
po_number: str
invoice_date: str
line_items: tuple[LineItem, ...]
total: floatWhat do these record shapes do?
- The LineItem class groups a description with its quantity and unit price.
- The Invoice class connects an internal payment record to its vendor and purchase order.
- The frozen=True setting prevents a created record from being modified.
- The asdict() helper prepares each record for dictionary conversion.
- Save records.py.
- Check the first two record shapes by running this command in the existing PowerShell terminal:
python records.pyWhat does this check prove?
PowerShell returns to its prompt without a traceback. That clean exit confirms that Python can parse the imports and both record shapes.
Seeing a traceback?
Check the indentation beneath each class declaration. Every field must sit inside its class.
Get targeted help with the syntax: Help me fix the syntax in my LineItem and Invoice dataclasses..
- Place your cursor below the Invoice class in records.py.
- Add the remaining record shapes by pasting this code:
@dataclass(frozen=True)
class PurchaseOrder:
po_number: str
vendor_id: str
line_items: tuple[LineItem, ...]
approved_total: float
@dataclass(frozen=True)
class PaidInvoice:
history_id: str
invoice_number: str
vendor_id: str
paid_date: str
total: floatHow do these records complete the model?
- The PurchaseOrder class stores approved line items and the approved total.
- The PaidInvoice class stores one historical payment for duplicate checking.
- Both invoice types use stable fields that the lookup tools can convert into dictionaries.
- Save records.py.
- Check all four record shapes by running this command:
python records.pyWhat does the second check prove?
PowerShell returns to its prompt without a traceback. Python can now parse all four frozen record shapes.
Still seeing a class error?
Check that each decorator sits directly above its class. Check that every field includes its type annotation.
Ask for help comparing the four definitions: Help me diagnose the dataclass error in records.py..
Add synthetic investigation evidence
These records are fictional because routed model providers may log prompts or completions. Synthetic evidence keeps confidential invoices outside this learning project.
The two payment records create a clear comparison. PAY-1048 contains exceptions while PAY-1050 provides a clean case.
- Place your cursor below the PaidInvoice class in records.py.
- Add both synthetic invoices by pasting this code:
INVOICES = {
"PAY-1048": Invoice(
record_id="PAY-1048",
invoice_number="INV-1048",
vendor_id="V-100",
po_number="PO-204",
invoice_date="2026-10-12",
line_items=(LineItem("Data migration support", 10, 120.0),),
total=1200.0,
),
"PAY-1050": Invoice(
record_id="PAY-1050",
invoice_number="INV-1050",
vendor_id="V-200",
po_number="PO-205",
invoice_date="2026-10-13",
line_items=(LineItem("Accessibility review", 5, 200.0),),
total=1000.0,
),
}What do the invoices establish?
- The PAY-1048 invoice charges 120.0 per unit for data migration support.
- The PAY-1050 invoice charges 200.0 per unit for an accessibility review.
- Each invoice contains the identifiers needed to retrieve related evidence.
- Save records.py.
- Check that both invoice objects match their record shape by running this command:
python records.pyWhat does this invoice check prove?
PowerShell returns to its prompt without a traceback. Both entries in INVOICES satisfy the fields defined by Invoice.
Cannot construct an invoice?
Compare every constructor keyword with the fields in Invoice. A missing field or changed field name prevents object creation.
Get help comparing the structures: Help me compare my INVOICES entries with the Invoice dataclass..
- Place your cursor below the closing brace of INVOICES.
- Add the purchase orders and vendor histories by pasting this code:
PURCHASE_ORDERS = {
"PO-204": PurchaseOrder(
po_number="PO-204",
vendor_id="V-100",
line_items=(LineItem("Data migration support", 10, 100.0),),
approved_total=1000.0,
),
"PO-205": PurchaseOrder(
po_number="PO-205",
vendor_id="V-200",
line_items=(LineItem("Accessibility review", 5, 200.0),),
approved_total=1000.0,
),
}
VENDOR_HISTORY = {
"V-100": (
PaidInvoice("HIST-778", "INV-1048", "V-100", "2026-09-18", 1200.0),
PaidInvoice("HIST-734", "INV-1031", "V-100", "2026-08-28", 800.0),
),
"V-200": (
PaidInvoice("HIST-790", "INV-1041", "V-200", "2026-09-30", 600.0),
),
}Where do the exceptions come from?
- Purchase order PO-204 approves a unit price of 100.0 for the item charged at 120.0 on its invoice.
- Vendor V-100 has a paid record with the repeated invoice number INV-1048.
- Purchase order PO-205 matches the amount on PAY-1050.
- Vendor V-200 has no history entry that repeats INV-1050.
- Save records.py.
- Check the related evidence objects by running this command:
python records.pyWhat does this evidence check prove?
PowerShell returns to its prompt without a traceback. The purchase orders and paid invoice histories satisfy their typed record shapes.
Seeing a constructor error?
Compare each purchase order with PurchaseOrder. Compare each history entry with the field order in PaidInvoice.
Ask for help matching values to record types: Help me diagnose a constructor error in PURCHASE_ORDERS or VENDOR_HISTORY..
- Place your cursor below the closing brace of VENDOR_HISTORY.
- Add the fictional payment policy by pasting this code:
PAYMENT_POLICY = {
"policy_name": "Fictional LedgerGuard Accounts Payable Policy",
"rules": [
{
"rule_id": "PO-01",
"rule": "Invoice total must not exceed the approved purchase order total.",
},
{
"rule_id": "PRICE-01",
"rule": "Invoice unit prices must match the purchase order unit prices.",
},
{
"rule_id": "DUP-01",
"rule": "A repeated invoice number for the same vendor requires investigation.",
},
{
"rule_id": "HUMAN-01",
"rule": "The agent may recommend a next step but cannot authorize or reject a payment.",
},
],
}What does the fictional policy cover?
- Rule PO-01 compares the invoice total with the approved purchase order total.
- Rule PRICE-01 compares line-item unit prices.
- Rule DUP-01 sends repeated invoice numbers for investigation.
- Rule HUMAN-01 keeps payment authority with a human reviewer.
- Save records.py.
- Check the complete evidence set by running this command:
python records.pyWhat does the policy check prove?
PowerShell returns to its prompt without a traceback. LedgerGuard now has structured evidence and fictional rules for interpreting it.
Seeing a policy syntax error?
Check that each policy rule has matching braces. Check that the rules list closes before the outer dictionary.
Get help checking the nested structure: Help me find the bracket or comma problem in PAYMENT_POLICY..
Build the lookup tools and smoke test
A lookup tool gives the future agent a narrow route to one evidence source. An explicit found value makes missing evidence visible.
- Place your cursor below the closing brace of PAYMENT_POLICY.
- Add the invoice and purchase order lookups by pasting this code:
def get_invoice(invoice_id: str) -> dict[str, Any]:
invoice = INVOICES.get(invoice_id)
if invoice is None:
return {"found": False, "invoice_id": invoice_id}
return {"found": True, "invoice": asdict(invoice)}
def get_purchase_order(po_number: str) -> dict[str, Any]:
purchase_order = PURCHASE_ORDERS.get(po_number)
if purchase_order is None:
return {"found": False, "po_number": po_number}
return {"found": True, "purchase_order": asdict(purchase_order)}How do these lookups report evidence?
- Each function searches its matching dictionary with the requested identifier.
- A missing record returns False through the found field.
- An existing record returns True through the found field.
- The asdict() conversion produces data that JSON can serialize.
- Save records.py.
- Check both identifier-based lookups by running this command:
python records.pyWhat does this lookup check prove?
PowerShell returns to its prompt without a traceback. Both identifier-based lookup functions are ready for direct testing.
Seeing an indentation error?
Check that each return statement remains inside its function. Check that each missing-record return remains inside its conditional block.
Get help inspecting the structure: Help me fix the indentation in get_invoice and get_purchase_order..
- Place your cursor below get_purchase_order() in records.py.
- Add the vendor-history and policy lookups by pasting this code:
def search_vendor_invoices(vendor_id: str) -> dict[str, Any]:
history = VENDOR_HISTORY.get(vendor_id, ())
return {
"found": bool(history),
"vendor_id": vendor_id,
"paid_invoices": [asdict(invoice) for invoice in history],
}
def get_payment_policy() -> dict[str, Any]:
return {"found": True, "policy": PAYMENT_POLICY}How do these tools handle their evidence?
- The search_vendor_invoices() function returns every paid invoice stored for one vendor.
- An empty history produces an empty paid_invoices list.
- The get_payment_policy() function exposes the same fictional rules to every investigation.
- Save records.py.
- Check all four lookup functions by running this command:
python records.pyWhat does this four-tool check prove?
PowerShell returns to its prompt without a traceback. Each evidence source now has a local lookup function.
Seeing an error in the history lookup?
Check that the list comprehension calls asdict(invoice) for each item in history. Check that the returned dictionary closes after the list comprehension.
Ask for help with the conversion logic: Help me troubleshoot search_vendor_invoices in records.py..
- Place your cursor below get_payment_policy() in records.py.
- Add the tool mapping and smoke test by pasting this code:
TOOL_MAPPING: dict[str, Callable[..., dict[str, Any]]] = {
"get_invoice": get_invoice,
"get_purchase_order": get_purchase_order,
"search_vendor_invoices": search_vendor_invoices,
"get_payment_policy": get_payment_policy,
}
def smoke_test() -> dict[str, Any]:
return {
"invoice": get_invoice("PAY-1048"),
"purchase_order": get_purchase_order("PO-204"),
"vendor_history": search_vendor_invoices("V-100"),
"payment_policy": get_payment_policy(),
}
if __name__ == "__main__":
print(json.dumps(smoke_test(), indent=2))How does the smoke test work?
- The TOOL_MAPPING dictionary connects tool names to the local functions that execute them.
- The smoke_test() function calls all four tools with identifiers from the exception case.
- The final condition runs the smoke test when records.py is executed as a script.
- The json.dumps() call proves that every returned value can become formatted JSON.
- Save records.py.
Before you run the completed smoke test, which four evidence sections do you expect to see?
- Run the completed smoke test in the existing PowerShell terminal with this command:
python records.pyWhat should you see?
You should see formatted JSON with invoice and purchase_order sections. You should also see vendor_history and payment_policy sections.
Each section contains a found value set to True. The output includes the PAY-1048 evidence and all four fictional policy rules.
That is the evidence layer working. LedgerGuard can now retrieve structured business records through four controlled functions.
Smoke test missing a section?
Check that smoke_test() returns all four dictionary entries. Check that the final print() call remains outside the function body.
Get help tracing the missing output: Help me diagnose why records.py does not print all four evidence sections..
✔️ Awesome, I've got everything!
Great work. Your saved records.py now returns typed invoice evidence through four tested lookup tools.
ⓧ I'd like to double check the full code
- Compare your saved records.py with this complete reference:
from dataclasses import asdict, dataclass
import json
from typing import Any, Callable
@dataclass(frozen=True)
class LineItem:
description: str
quantity: int
unit_price: float
@dataclass(frozen=True)
class Invoice:
record_id: str
invoice_number: str
vendor_id: str
po_number: str
invoice_date: str
line_items: tuple[LineItem, ...]
total: float
@dataclass(frozen=True)
class PurchaseOrder:
po_number: str
vendor_id: str
line_items: tuple[LineItem, ...]
approved_total: float
@dataclass(frozen=True)
class PaidInvoice:
history_id: str
invoice_number: str
vendor_id: str
paid_date: str
total: float
INVOICES = {
"PAY-1048": Invoice(
record_id="PAY-1048",
invoice_number="INV-1048",
vendor_id="V-100",
po_number="PO-204",
invoice_date="2026-10-12",
line_items=(LineItem("Data migration support", 10, 120.0),),
total=1200.0,
),
"PAY-1050": Invoice(
record_id="PAY-1050",
invoice_number="INV-1050",
vendor_id="V-200",
po_number="PO-205",
invoice_date="2026-10-13",
line_items=(LineItem("Accessibility review", 5, 200.0),),
total=1000.0,
),
}
PURCHASE_ORDERS = {
"PO-204": PurchaseOrder(
po_number="PO-204",
vendor_id="V-100",
line_items=(LineItem("Data migration support", 10, 100.0),),
approved_total=1000.0,
),
"PO-205": PurchaseOrder(
po_number="PO-205",
vendor_id="V-200",
line_items=(LineItem("Accessibility review", 5, 200.0),),
approved_total=1000.0,
),
}
VENDOR_HISTORY = {
"V-100": (
PaidInvoice("HIST-778", "INV-1048", "V-100", "2026-09-18", 1200.0),
PaidInvoice("HIST-734", "INV-1031", "V-100", "2026-08-28", 800.0),
),
"V-200": (
PaidInvoice("HIST-790", "INV-1041", "V-200", "2026-09-30", 600.0),
),
}
PAYMENT_POLICY = {
"policy_name": "Fictional LedgerGuard Accounts Payable Policy",
"rules": [
{
"rule_id": "PO-01",
"rule": "Invoice total must not exceed the approved purchase order total.",
},
{
"rule_id": "PRICE-01",
"rule": "Invoice unit prices must match the purchase order unit prices.",
},
{
"rule_id": "DUP-01",
"rule": "A repeated invoice number for the same vendor requires investigation.",
},
{
"rule_id": "HUMAN-01",
"rule": "The agent may recommend a next step but cannot authorize or reject a payment.",
},
],
}
def get_invoice(invoice_id: str) -> dict[str, Any]:
invoice = INVOICES.get(invoice_id)
if invoice is None:
return {"found": False, "invoice_id": invoice_id}
return {"found": True, "invoice": asdict(invoice)}
def get_purchase_order(po_number: str) -> dict[str, Any]:
purchase_order = PURCHASE_ORDERS.get(po_number)
if purchase_order is None:
return {"found": False, "po_number": po_number}
return {"found": True, "purchase_order": asdict(purchase_order)}
def search_vendor_invoices(vendor_id: str) -> dict[str, Any]:
history = VENDOR_HISTORY.get(vendor_id, ())
return {
"found": bool(history),
"vendor_id": vendor_id,
"paid_invoices": [asdict(invoice) for invoice in history],
}
def get_payment_policy() -> dict[str, Any]:
return {"found": True, "policy": PAYMENT_POLICY}
TOOL_MAPPING: dict[str, Callable[..., dict[str, Any]]] = {
"get_invoice": get_invoice,
"get_purchase_order": get_purchase_order,
"search_vendor_invoices": search_vendor_invoices,
"get_payment_policy": get_payment_policy,
}
def smoke_test() -> dict[str, Any]:
return {
"invoice": get_invoice("PAY-1048"),
"purchase_order": get_purchase_order("PO-204"),
"vendor_history": search_vendor_invoices("V-100"),
"payment_policy": get_payment_policy(),
}
if __name__ == "__main__":
print(json.dumps(smoke_test(), indent=2))What should match?
Check each class name and data identifier against your saved file. Check each lookup function and the final smoke_test() call too.
Your financial evidence layer is ready. Next, you will connect these four tools to a bounded agent loop that can choose which records to retrieve.
Add the Agentic Tool Loop
Your four local tools now return structured invoice evidence. That gives LedgerGuard a controlled view of the synthetic records it needs.
Manual lookups still leave you coordinating every check. In this step, an OpenRouter tool-calling loop lets the model choose a tool. Your Python code executes that tool before returning the evidence to the model.
In this step, get ready to:
- Describe the four local functions with JSON tool schemas.
- Build a bounded loop that dispatches approved tools.
- Investigate the exception invoice with retrieved evidence.
Define the tool interface
A tool schema describes a function in JSON. It tells the model which arguments the function accepts without giving the model direct access to your Python runtime.
- In the VS Code Explorer sidebar, select ledgerguard.py.
- Add the records import below the existing urllib.request import:
from records import TOOL_MAPPINGWhat does this import do?
The TOOL_MAPPING dictionary connects each permitted tool name to its local Python function. The dispatcher uses this mapping to prevent calls to arbitrary functions.
- Add the loop limit below the MODEL constant.
- Start the TOOLS list with the invoice schema by pasting this first section:
MAX_ITERATIONS = 8
TOOLS = [
{
"type": "function",
"function": {
"name": "get_invoice",
"description": "Retrieve one synthetic invoice by its internal payment record ID.",
"parameters": {
"type": "object",
"properties": {
"invoice_id": {
"type": "string",
"description": "Internal payment record ID, such as PAY-1048.",
}
},
"required": ["invoice_id"],
},
},
},How does the first schema work?
- The function type tells the model that this tool maps to executable code.
- The invoice_id property identifies the argument accepted by get_invoice.
- The required list prevents a tool call without a payment record ID.
- Continue the TOOLS list with the purchase-order schema:
{
"type": "function",
"function": {
"name": "get_purchase_order",
"description": "Retrieve the synthetic purchase order named on an invoice.",
"parameters": {
"type": "object",
"properties": {
"po_number": {
"type": "string",
"description": "Purchase order number returned by get_invoice.",
}
},
"required": ["po_number"],
},
},
},How does the purchase-order schema help?
The description tells the model to use the purchase order named by the invoice. The required po_number keeps the lookup tied to retrieved evidence.
- Continue the TOOLS list with the vendor-history schema:
{
"type": "function",
"function": {
"name": "search_vendor_invoices",
"description": "List paid invoices for the vendor named on the current invoice.",
"parameters": {
"type": "object",
"properties": {
"vendor_id": {
"type": "string",
"description": "Vendor ID returned by get_invoice.",
}
},
"required": ["vendor_id"],
},
},
},How does vendor history become evidence?
The vendor_id argument comes from the current invoice. This lets the model compare its invoice number with earlier paid invoices for the same vendor.
- Finish the TOOLS list with the policy schema:
{
"type": "function",
"function": {
"name": "get_payment_policy",
"description": "Retrieve the fictional policy rules used for this learning project.",
"parameters": {"type": "object", "properties": {}},
},
},
]Why does the policy tool have no arguments?
The learning project has one fictional payment policy. The tool can return it without an identifier.
- Save ledgerguard.py.
- Switch back to the PowerShell terminal from the previous step.
- Confirm the baseline still runs by executing:
python ledgerguard.py --baseline PAY-1048What should I see?
You should still see a response explaining that the model cannot verify PAY-1048 without company records. This proves the new schemas load without changing the model-only baseline.
Does the baseline stop with a syntax error?
Check that each schema ends with the same braces shown above. Confirm that the final policy schema is followed by the closing bracket for TOOLS.
Ask for help with the exact line number shown in your terminal: Help me find the syntax error in my LedgerGuard TOOLS list.
Build the bounded dispatcher loop
The schema is a description for the model. The dispatcher remains in control because it executes only functions found in TOOL_MAPPING.
- Add the required-tool set below the completed TOOLS list.
- Add the investigation prompt directly below that set by pasting:
REQUIRED_TOOLS = {
"get_invoice",
"get_purchase_order",
"search_vendor_invoices",
"get_payment_policy",
}
SYSTEM_PROMPT = """
You are LedgerGuard, an invoice investigation assistant for a human finance reviewer.
Use only facts returned by tools. Never invent an invoice, purchase order, vendor record, policy, or missing value.
For every investigation:
1. Call get_invoice with the internal payment record ID.
2. Use its po_number and vendor_id to call get_purchase_order and search_vendor_invoices.
3. Call get_payment_policy.
4. If a required record is missing, recommend investigation and name the missing evidence.
Your final report must use this structure:
Recommendation: READY FOR HUMAN APPROVAL or NEEDS INVESTIGATION
Evidence:
- Cite exact record fields returned by tools.
Policy checks:
- Cite the relevant fictional policy rule IDs.
Human action:
- State what a human reviewer should verify next.
""".strip()What does the prompt control?
- The REQUIRED_TOOLS set records which evidence sources must run before a report is accepted.
- The SYSTEM_PROMPT requires retrieved facts for each conclusion.
- The report structure keeps the recommendation separate from its evidence.
- Add the dispatcher below baseline_review() by pasting:
def dispatch_tool(tool_name: str, arguments: Any) -> dict[str, Any]:
if not isinstance(arguments, dict):
return {"error": "Tool arguments must be a JSON object."}
tool = TOOL_MAPPING.get(tool_name)
if tool is None:
return {"error": f"Unknown tool: {tool_name}"}
try:
return tool(**arguments)
except TypeError as error:
return {"error": f"Invalid arguments for {tool_name}: {error}"}What does the dispatcher protect?
- The type check rejects tool arguments that are not a JSON object.
- The mapping lookup blocks tool names that your project has not approved.
- The exception handler returns argument problems as evidence the loop can process.
- Save ledgerguard.py.
- Check the new prompt and dispatcher for syntax errors by rerunning:
python ledgerguard.py --baseline PAY-1048What should I see now?
You should see the same evidence-gap response from the baseline. This confirms the new definitions load successfully before you connect them to the agent path.
- Add investigate() below dispatch_tool() by starting with this section:
def investigate(record_id: str) -> str:
messages: list[dict[str, Any]] = [
{"role": "system", "content": SYSTEM_PROMPT},
{
"role": "user",
"content": f"Investigate internal payment record {record_id} for a human reviewer.",
},
]
used_tools: set[str] = set()
for iteration in range(1, MAX_ITERATIONS + 1):
response = call_openrouter(messages, TOOLS)
choices = response.get("choices")
if not choices:
raise RuntimeError("OpenRouter returned no choices.")
message = choices[0].get("message")
if not isinstance(message, dict):
raise RuntimeError("OpenRouter returned an invalid assistant message.")
messages.append(message)
tool_calls = message.get("tool_calls") or []How does the loop begin?
- The message history starts with the evidence rules and the requested record ID.
- The bounded range limits the investigation to MAX_ITERATIONS model turns.
- Every call passes TOOLS so tool calling remains available after earlier tool results.
- The assistant message is appended before its tool calls are processed.
- Continue investigate() with the final-response branch:
if not tool_calls:
missing_tools = sorted(REQUIRED_TOOLS - used_tools)
if missing_tools:
messages.append(
{
"role": "user",
"content": (
"Continue the investigation. Before the final report, call these remaining tools: "
+ ", ".join(missing_tools)
),
}
)
continue
report = message.get("content") or "No final report was returned."
return reportWhy can the loop reject an early report?
A normal assistant response may arrive before every evidence source has been checked. The set difference identifies missing tools before the loop asks the model to continue.
The report is returned only after REQUIRED_TOOLS has no missing entries. This turns evidence collection into an enforced condition.
- Finish investigate() with the tool-execution branch:
for tool_call in tool_calls:
function = tool_call.get("function", {})
tool_name = function.get("name", "")
raw_arguments = function.get("arguments", "{}")
try:
arguments = json.loads(raw_arguments)
except json.JSONDecodeError:
arguments = {"_invalid_json": raw_arguments}
result = dispatch_tool(tool_name, arguments)
if tool_name in REQUIRED_TOOLS:
used_tools.add(tool_name)
print(f"[tool] {tool_name}({json.dumps(arguments)})")
messages.append(
{
"role": "tool",
"tool_call_id": tool_call.get("id", ""),
"content": json.dumps(result),
}
)
raise RuntimeError("LedgerGuard reached its maximum tool-call iterations without a final report.")How does a tool result return to the model?
- The loop parses each function argument string with json.loads().
- The dispatcher executes the matching local function.
- The terminal prints a [tool] line so you can follow the investigation.
- The tool message returns the JSON result with the original tool_call_id.
- Save ledgerguard.py.
- Confirm the completed loop compiles by rerunning the baseline:
python ledgerguard.py --baseline PAY-1048What does this check prove?
You should see the baseline response again. Python has now loaded the complete investigate() function without reporting a syntax or import error.
- Replace the existing main() function with this command-line router:
def main() -> None:
if len(sys.argv) > 1 and sys.argv[1] == "--baseline":
record_id = sys.argv[2] if len(sys.argv) > 2 else "PAY-1048"
print(baseline_review(record_id))
return
record_id = sys.argv[1] if len(sys.argv) > 1 else "PAY-1048"
print(f"Investigating {record_id}...")
print(investigate(record_id))How does the command choose a path?
The --baseline argument preserves the model-only review. Any normal record ID now enters the agentic investigation path.
- Save ledgerguard.py.
Investigate the exception invoice
The invoice tool can now lead the model to PO-204 and vendor V-100. Those connected records give the final recommendation a traceable basis.
Before you run the investigation, which evidence do you expect LedgerGuard to compare before it reports on PAY-1048?
- Run the exception invoice through the agent by executing:
python ledgerguard.py PAY-1048What should I see?
You should see tool activity for get_invoice, get_purchase_order, search_vendor_invoices, and get_payment_policy.
The final report should show Recommendation: NEEDS INVESTIGATION. Its evidence should identify the price mismatch against PO-204 and the repeated INV-1048 invoice number.
That is the loop working: LedgerGuard has turned four controlled lookups into an evidence-backed investigation for a human reviewer.
Missing a tool or recommendation?
Confirm that TOOLS contains all four schemas. Check that REQUIRED_TOOLS uses the same four function names as TOOL_MAPPING in records.py.
Keep the current PowerShell session active because it holds your API key. If the routed model returns a temporary service failure, rerun the investigation after a short wait.
Share the tool lines and the final non-secret error details here: Help me debug my LedgerGuard agent loop.
✔️ Awesome, I've got everything!
Great work. Your bounded agent loop now gathers all four evidence sources before returning its PAY-1048 report.
ⓧ I'd like to double check the full code
Compare your cumulative ledgerguard.py file with this version:
import json
import os
import sys
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
from records import TOOL_MAPPING
API_URL = "https://openrouter.ai/api/v1/chat/completions"
MODEL = "openrouter/free"
MAX_ITERATIONS = 8
TOOLS = [
{
"type": "function",
"function": {
"name": "get_invoice",
"description": "Retrieve one synthetic invoice by its internal payment record ID.",
"parameters": {
"type": "object",
"properties": {
"invoice_id": {
"type": "string",
"description": "Internal payment record ID, such as PAY-1048.",
}
},
"required": ["invoice_id"],
},
},
},
{
"type": "function",
"function": {
"name": "get_purchase_order",
"description": "Retrieve the synthetic purchase order named on an invoice.",
"parameters": {
"type": "object",
"properties": {
"po_number": {
"type": "string",
"description": "Purchase order number returned by get_invoice.",
}
},
"required": ["po_number"],
},
},
},
{
"type": "function",
"function": {
"name": "search_vendor_invoices",
"description": "List paid invoices for the vendor named on the current invoice.",
"parameters": {
"type": "object",
"properties": {
"vendor_id": {
"type": "string",
"description": "Vendor ID returned by get_invoice.",
}
},
"required": ["vendor_id"],
},
},
},
{
"type": "function",
"function": {
"name": "get_payment_policy",
"description": "Retrieve the fictional policy rules used for this learning project.",
"parameters": {"type": "object", "properties": {}},
},
},
]
REQUIRED_TOOLS = {
"get_invoice",
"get_purchase_order",
"search_vendor_invoices",
"get_payment_policy",
}
SYSTEM_PROMPT = """
You are LedgerGuard, an invoice investigation assistant for a human finance reviewer.
Use only facts returned by tools. Never invent an invoice, purchase order, vendor record, policy, or missing value.
For every investigation:
1. Call get_invoice with the internal payment record ID.
2. Use its po_number and vendor_id to call get_purchase_order and search_vendor_invoices.
3. Call get_payment_policy.
4. If a required record is missing, recommend investigation and name the missing evidence.
Your final report must use this structure:
Recommendation: READY FOR HUMAN APPROVAL or NEEDS INVESTIGATION
Evidence:
- Cite exact record fields returned by tools.
Policy checks:
- Cite the relevant fictional policy rule IDs.
Human action:
- State what a human reviewer should verify next.
""".strip()
def call_openrouter(
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
api_key = os.getenv("OPENROUTER_API_KEY")
if not api_key:
raise RuntimeError("OPENROUTER_API_KEY is not set in this PowerShell session.")
payload: dict[str, Any] = {"model": MODEL, "messages": messages}
if tools is not None:
payload["tools"] = tools
request = Request(
API_URL,
data=json.dumps(payload).encode("utf-8"),
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
},
method="POST",
)
try:
with urlopen(request, timeout=90) as response:
return json.loads(response.read().decode("utf-8"))
except HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"OpenRouter returned HTTP {error.code}: {detail}") from error
except URLError as error:
raise RuntimeError(f"Could not reach OpenRouter: {error.reason}") from error
def baseline_review(record_id: str) -> str:
messages = [
{
"role": "system",
"content": (
"You are testing the limits of a model without company tools. "
"Do not invent records or claim to have inspected private data."
),
},
{
"role": "user",
"content": (
f"Investigate payment record {record_id}. You have only this ID and no company records. "
"Explain what evidence is missing before a reliable review is possible."
),
},
]
response = call_openrouter(messages)
return response["choices"][0]["message"]["content"]
def dispatch_tool(tool_name: str, arguments: Any) -> dict[str, Any]:
if not isinstance(arguments, dict):
return {"error": "Tool arguments must be a JSON object."}
tool = TOOL_MAPPING.get(tool_name)
if tool is None:
return {"error": f"Unknown tool: {tool_name}"}
try:
return tool(**arguments)
except TypeError as error:
return {"error": f"Invalid arguments for {tool_name}: {error}"}
def investigate(record_id: str) -> str:
messages: list[dict[str, Any]] = [
{"role": "system", "content": SYSTEM_PROMPT},
{
"role": "user",
"content": f"Investigate internal payment record {record_id} for a human reviewer.",
},
]
used_tools: set[str] = set()
for iteration in range(1, MAX_ITERATIONS + 1):
response = call_openrouter(messages, TOOLS)
choices = response.get("choices")
if not choices:
raise RuntimeError("OpenRouter returned no choices.")
message = choices[0].get("message")
if not isinstance(message, dict):
raise RuntimeError("OpenRouter returned an invalid assistant message.")
messages.append(message)
tool_calls = message.get("tool_calls") or []
if not tool_calls:
missing_tools = sorted(REQUIRED_TOOLS - used_tools)
if missing_tools:
messages.append(
{
"role": "user",
"content": (
"Continue the investigation. Before the final report, call these remaining tools: "
+ ", ".join(missing_tools)
),
}
)
continue
report = message.get("content") or "No final report was returned."
return report
for tool_call in tool_calls:
function = tool_call.get("function", {})
tool_name = function.get("name", "")
raw_arguments = function.get("arguments", "{}")
try:
arguments = json.loads(raw_arguments)
except json.JSONDecodeError:
arguments = {"_invalid_json": raw_arguments}
result = dispatch_tool(tool_name, arguments)
if tool_name in REQUIRED_TOOLS:
used_tools.add(tool_name)
print(f"[tool] {tool_name}({json.dumps(arguments)})")
messages.append(
{
"role": "tool",
"tool_call_id": tool_call.get("id", ""),
"content": json.dumps(result),
}
)
raise RuntimeError("LedgerGuard reached its maximum tool-call iterations without a final report.")
def main() -> None:
if len(sys.argv) > 1 and sys.argv[1] == "--baseline":
record_id = sys.argv[2] if len(sys.argv) > 2 else "PAY-1048"
print(baseline_review(record_id))
return
record_id = sys.argv[1] if len(sys.argv) > 1 else "PAY-1048"
print(f"Investigating {record_id}...")
print(investigate(record_id))
if __name__ == "__main__":
main()What should match?
Your file should retain the baseline path. It should also contain the four schemas, the approved dispatcher, the bounded investigation loop, and the updated command-line entry point.
Your agent now gathers evidence before it reports. Next, you will make its human review boundary explicit and record every investigation in an audit trail.
Add Human Boundaries and Audit Logs
Your bounded tool loop now investigates PAY-1048 with evidence from four local tools. Its report can identify the price mismatch plus the repeated invoice number.
An evidence-grounded report still needs clear authority boundaries. This step keeps every payment decision with a human reviewer while creating a JSON Lines trail of the agent's work.
In this step, get ready to:
- Restrict LedgerGuard to two human-review recommendation labels.
- Record final reports plus tool activity in audit.jsonl.
- Compare the exception invoice with the clean invoice.
Set human-review boundaries
Finance recommendations can influence consequential decisions. LedgerGuard needs a safety contract that limits its language while requiring evidence plus a specific human action.
- In the VS Code Explorer sidebar, select ledgerguard.py.
- Locate the SYSTEM_PROMPT assignment.
- Replace the complete SYSTEM_PROMPT assignment with this safety contract:
SYSTEM_PROMPT = """
You are LedgerGuard, an invoice investigation assistant for a human finance reviewer.
Use only facts returned by tools. Never invent an invoice, purchase order, vendor record, policy, or missing value.
For every investigation:
1. Call get_invoice with the internal payment record ID.
2. Use its po_number and vendor_id to call get_purchase_order and search_vendor_invoices.
3. Call get_payment_policy.
4. If a required record is missing, recommend investigation and name the missing evidence.
Your final report must use this structure:
Recommendation: READY FOR HUMAN APPROVAL or NEEDS INVESTIGATION
Evidence:
- Cite exact record fields returned by tools.
Policy checks:
- Cite the relevant fictional policy rule IDs.
Human action:
- State what a human reviewer should verify next.
Never write APPROVED, REJECTED, or FRAUD. You provide decision support only and cannot authorize, reject, or execute a payment.
""".strip()What does this safety contract do?
- The two recommendation labels keep the output within an advisory workflow.
- Exact record fields plus policy rule IDs make each recommendation traceable to retrieved evidence.
- The forbidden terms prevent LedgerGuard from claiming payment authority or making an unsupported accusation.
- The human action section identifies the review that still needs a person.
The audit trail also needs one stable destination. The AUDIT_PATH constant gives every event writer the same file name.
- Locate the constants where MODEL appears above MAX_ITERATIONS.
- Insert AUDIT_PATH beneath MODEL so the constants match this reference:
MODEL = "openrouter/free"
AUDIT_PATH = "audit.jsonl"
MAX_ITERATIONS = 8Why use one audit path?
The constant points every audit event to audit.jsonl. A single path keeps tool evidence plus final reports in one chronological trail.
- Save ledgerguard.py.
Before you run the exception invoice, do you think the final report can still claim that the payment is approved?
- Return to the PowerShell terminal from earlier.
- Test the new safety contract by running:
python ledgerguard.py PAY-1048What should you see?
You should see four [tool] lines followed by Recommendation: NEEDS INVESTIGATION. The report should cite the PO-204 price mismatch plus the repeated INV-1048 number.
The report should include policy rule IDs plus a human action. It should avoid language that authorizes or rejects the payment.
You have now turned the system prompt into an authority boundary. LedgerGuard provides decision support while the reviewer keeps control.
Report ignoring the boundaries?
Check that you replaced the complete SYSTEM_PROMPT assignment. Confirm the closing .strip() remains in place.
Confirm that investigate() still starts its message list with SYSTEM_PROMPT.
Help me check why LedgerGuard is not following its human-review-only system prompt.
Log each final report
An audit trail becomes useful when each event occupies one line of structured JSON. Appending each line preserves earlier investigations while keeping the file easy to inspect.
- In ledgerguard.py, locate dispatch_tool().
- Add log_event() directly above dispatch_tool() by pasting this function:
def log_event(event: dict[str, Any]) -> None:
with open(AUDIT_PATH, "a", encoding="utf-8") as audit_file:
audit_file.write(json.dumps(event) + "\n")What does this function do?
- Append mode preserves the events already stored in audit.jsonl.
- JSON serialization converts each event dictionary into structured text.
- The newline places every event on its own line.
A completed investigation should record the exact report returned to the reviewer. The event also needs the record ID plus the loop iteration that produced it.
- Inside investigate(), locate the line that assigns report from the assistant message.
- Insert this event block below the report assignment.
log_event(
{
"event": "final_report",
"record_id": record_id,
"iteration": iteration,
"report": report,
}
)What gets recorded?
The final_report event connects the returned report to its payment record plus loop iteration. That link shows which investigation produced the recommendation.
- Save ledgerguard.py.
Before you run the agent, what new event do you expect to find inside audit.jsonl?
- Return to the PowerShell terminal from earlier.
- Create the first audited report plus display the file by running:
python ledgerguard.py PAY-1048
Get-Content -Path .\audit.jsonlWhat should you see?
You should see the investigation report followed by one JSON object from audit.jsonl. That object should contain final_report, PAY-1048, the loop iteration, plus the complete report.
That first audit entry proves the append-only writer works. LedgerGuard can now preserve the report shown to a reviewer.
Audit file missing?
Check that log_event() uses AUDIT_PATH. Confirm the final-report event sits before return report.
Make sure the PowerShell prompt still points to the ledgerguard folder. The audit file is created in the current project folder.
Help me find why LedgerGuard is not creating audit.jsonl after returning a final report.
Complete the audit trail
A final report shows the conclusion. Tool-call events preserve the evidence path that led there.
- Return to ledgerguard.py in the VS Code editor.
- Inside the tool-call loop, locate the line that prints [tool] activity.
- Insert this event block directly beneath that print line:
log_event(
{
"event": "tool_call",
"record_id": record_id,
"iteration": iteration,
"tool": tool_name,
"arguments": arguments,
"result": result,
}
)What does a tool event capture?
- The record ID ties the lookup to one investigation.
- The loop iteration shows when the lookup happened.
- The tool name plus arguments show what LedgerGuard requested.
- The result preserves the returned synthetic evidence.
- Save ledgerguard.py.
Before you run both records, which one do you expect to need investigation?
- Return to the PowerShell terminal from earlier.
- Run both investigations plus display the completed audit trail by running:
python ledgerguard.py PAY-1048
python ledgerguard.py PAY-1050
Get-Content -Path .\audit.jsonlWhat should you see?
The PAY-1048 run should end with Recommendation: NEEDS INVESTIGATION. Its evidence should identify the purchase-order mismatch plus the repeated invoice number.
The PAY-1050 run should end with Recommendation: READY FOR HUMAN APPROVAL. Its evidence should show matching invoice plus purchase-order values.
The audit output should contain tool_call events plus final_report events for both record IDs. Each tool event should include its arguments plus returned evidence.
Strong work. LedgerGuard now distinguishes an exception from a clean invoice while preserving the evidence path behind both recommendations.
Missing tool events or a final report?
Confirm the tool event is logged inside the for tool_call in tool_calls loop. Its indentation should match the existing print statement.
Confirm the final-report event stays inside the branch that has no tool calls. It must run before the report is returned.
Help me compare my LedgerGuard audit logging with the expected event placement and indentation.
✔️ Awesome, I've got everything!
Your two investigations now produce different evidence-backed recommendations. Their tool calls plus final reports remain available in audit.jsonl.
ⓧ I'd like to double check the full code
- Compare your complete ledgerguard.py file with this reference:
import json
import os
import sys
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
from records import TOOL_MAPPING
API_URL = "https://openrouter.ai/api/v1/chat/completions"
MODEL = "openrouter/free"
AUDIT_PATH = "audit.jsonl"
MAX_ITERATIONS = 8
TOOLS = [
{
"type": "function",
"function": {
"name": "get_invoice",
"description": "Retrieve one synthetic invoice by its internal payment record ID.",
"parameters": {
"type": "object",
"properties": {
"invoice_id": {
"type": "string",
"description": "Internal payment record ID, such as PAY-1048.",
}
},
"required": ["invoice_id"],
},
},
},
{
"type": "function",
"function": {
"name": "get_purchase_order",
"description": "Retrieve the synthetic purchase order named on an invoice.",
"parameters": {
"type": "object",
"properties": {
"po_number": {
"type": "string",
"description": "Purchase order number returned by get_invoice.",
}
},
"required": ["po_number"],
},
},
},
{
"type": "function",
"function": {
"name": "search_vendor_invoices",
"description": "List paid invoices for the vendor named on the current invoice.",
"parameters": {
"type": "object",
"properties": {
"vendor_id": {
"type": "string",
"description": "Vendor ID returned by get_invoice.",
}
},
"required": ["vendor_id"],
},
},
},
{
"type": "function",
"function": {
"name": "get_payment_policy",
"description": "Retrieve the fictional policy rules used for this learning project.",
"parameters": {"type": "object", "properties": {}},
},
},
]
REQUIRED_TOOLS = {
"get_invoice",
"get_purchase_order",
"search_vendor_invoices",
"get_payment_policy",
}
SYSTEM_PROMPT = """
You are LedgerGuard, an invoice investigation assistant for a human finance reviewer.
Use only facts returned by tools. Never invent an invoice, purchase order, vendor record, policy, or missing value.
For every investigation:
1. Call get_invoice with the internal payment record ID.
2. Use its po_number and vendor_id to call get_purchase_order and search_vendor_invoices.
3. Call get_payment_policy.
4. If a required record is missing, recommend investigation and name the missing evidence.
Your final report must use this structure:
Recommendation: READY FOR HUMAN APPROVAL or NEEDS INVESTIGATION
Evidence:
- Cite exact record fields returned by tools.
Policy checks:
- Cite the relevant fictional policy rule IDs.
Human action:
- State what a human reviewer should verify next.
Never write APPROVED, REJECTED, or FRAUD. You provide decision support only and cannot authorize, reject, or execute a payment.
""".strip()
def call_openrouter(
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
api_key = os.getenv("OPENROUTER_API_KEY")
if not api_key:
raise RuntimeError("OPENROUTER_API_KEY is not set in this PowerShell session.")
payload: dict[str, Any] = {"model": MODEL, "messages": messages}
if tools is not None:
payload["tools"] = tools
request = Request(
API_URL,
data=json.dumps(payload).encode("utf-8"),
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
},
method="POST",
)
try:
with urlopen(request, timeout=90) as response:
return json.loads(response.read().decode("utf-8"))
except HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"OpenRouter returned HTTP {error.code}: {detail}") from error
except URLError as error:
raise RuntimeError(f"Could not reach OpenRouter: {error.reason}") from error
def baseline_review(record_id: str) -> str:
messages = [
{
"role": "system",
"content": (
"You are testing the limits of a model without company tools. "
"Do not invent records or claim to have inspected private data."
),
},
{
"role": "user",
"content": (
f"Investigate payment record {record_id}. You have only this ID and no company records. "
"Explain what evidence is missing before a reliable review is possible."
),
},
]
response = call_openrouter(messages)
return response["choices"][0]["message"]["content"]
def log_event(event: dict[str, Any]) -> None:
with open(AUDIT_PATH, "a", encoding="utf-8") as audit_file:
audit_file.write(json.dumps(event) + "\n")
def dispatch_tool(tool_name: str, arguments: Any) -> dict[str, Any]:
if not isinstance(arguments, dict):
return {"error": "Tool arguments must be a JSON object."}
tool = TOOL_MAPPING.get(tool_name)
if tool is None:
return {"error": f"Unknown tool: {tool_name}"}
try:
return tool(**arguments)
except TypeError as error:
return {"error": f"Invalid arguments for {tool_name}: {error}"}
def investigate(record_id: str) -> str:
messages: list[dict[str, Any]] = [
{"role": "system", "content": SYSTEM_PROMPT},
{
"role": "user",
"content": f"Investigate internal payment record {record_id} for a human reviewer.",
},
]
used_tools: set[str] = set()
for iteration in range(1, MAX_ITERATIONS + 1):
response = call_openrouter(messages, TOOLS)
choices = response.get("choices")
if not choices:
raise RuntimeError("OpenRouter returned no choices.")
message = choices[0].get("message")
if not isinstance(message, dict):
raise RuntimeError("OpenRouter returned an invalid assistant message.")
messages.append(message)
tool_calls = message.get("tool_calls") or []
if not tool_calls:
missing_tools = sorted(REQUIRED_TOOLS - used_tools)
if missing_tools:
messages.append(
{
"role": "user",
"content": (
"Continue the investigation. Before the final report, call these remaining tools: "
+ ", ".join(missing_tools)
),
}
)
continue
report = message.get("content") or "No final report was returned."
log_event(
{
"event": "final_report",
"record_id": record_id,
"iteration": iteration,
"report": report,
}
)
return report
for tool_call in tool_calls:
function = tool_call.get("function", {})
tool_name = function.get("name", "")
raw_arguments = function.get("arguments", "{}")
try:
arguments = json.loads(raw_arguments)
except json.JSONDecodeError:
arguments = {"_invalid_json": raw_arguments}
result = dispatch_tool(tool_name, arguments)
if tool_name in REQUIRED_TOOLS:
used_tools.add(tool_name)
print(f"[tool] {tool_name}({json.dumps(arguments)})")
log_event(
{
"event": "tool_call",
"record_id": record_id,
"iteration": iteration,
"tool": tool_name,
"arguments": arguments,
"result": result,
}
)
messages.append(
{
"role": "tool",
"tool_call_id": tool_call.get("id", ""),
"content": json.dumps(result),
}
)
raise RuntimeError("LedgerGuard reached its maximum tool-call iterations without a final report.")
def main() -> None:
if len(sys.argv) > 1 and sys.argv[1] == "--baseline":
record_id = sys.argv[2] if len(sys.argv) > 2 else "PAY-1048"
print(baseline_review(record_id))
return
record_id = sys.argv[1] if len(sys.argv) > 1 else "PAY-1048"
print(f"Investigating {record_id}...")
print(investigate(record_id))
if __name__ == "__main__":
main()How to compare the file
Check the positions of AUDIT_PATH, log_event(), plus both event blocks. Their indentation determines whether each event records at the correct moment.
Secret mission
Make Missing Evidence a Safe Failure
Real invoice queues contain incomplete records. Add a synthetic invoice that points to a missing purchase order. Then prove LedgerGuard escalates the gap without changing its existing loop or inventing evidence.
Clean Up Your Resources
Clean Up Your Resources
Choose whether to keep your LedgerGuard resources available, pause your work, or delete everything created for the project. This project has no ongoing cost because OpenRouter's openrouter/free route has zero prompt and completion token pricing.
Keep Financial Data Synthetic
Some providers available through the Free Models Router may log prompts or completions. The fictional records in LedgerGuard keep confidential finance data out of those requests.
Resources you used:
- Local ledgerguard folder containing ledgerguard.py, records.py, and audit.jsonl.
- Current PowerShell session containing OPENROUTER_API_KEY.
- OpenRouter API key created for LedgerGuard.
Keep everything running
No action is needed. Choose this option if you want to continue investigating synthetic records with LedgerGuard.
- Keep the current PowerShell session open while you continue working.
- Retain the ledgerguard folder for future investigations.
- Keep your OpenRouter API key active.
- Continue using synthetic financial records only.
Pause - I'll come back to this later
End the running session while keeping the code and audit history for later. Your OpenRouter API key remains active until you revoke it.
- Close the current PowerShell terminal in Visual Studio Code.
- Leave the ledgerguard folder unchanged.
- Keep your OpenRouter API key active for your next session.
Closing PowerShell removes the session-scoped OPENROUTER_API_KEY value. Your Python files and audit.jsonl remain on your computer.
Delete - I don't want to use this again
Removing these resources is permanent. Your OpenRouter account stays in place after you revoke the project key.
- Close the current PowerShell terminal in Visual Studio Code.
The session-scoped OPENROUTER_API_KEY value is now cleared from that PowerShell process.
Delete the local project.
- Use Windows search to open File Explorer.
- Navigate to the location where you created the ledgerguard folder in Step 1.
- Select the ledgerguard folder.
- Press Shift+Delete to remove the folder permanently.
- Confirm the deletion if Windows asks.
Revoke API access.
- Open the OpenRouter API key page in your browser.
- Find the API key you created for LedgerGuard.
- Use the delete control beside that key.
- Confirm the revocation.
Cleanup is complete. The revoked key can no longer authenticate LedgerGuard requests.
Nice Work!
Nice Work!
You did it! You built LedgerGuard, an explainable AI agent that investigates synthetic invoices for a human finance reviewer.
You've learned how to:
- Exposed the model-only evidence gap with a baseline OpenRouter request that identifies the company records needed for a reliable review.
- Modeled synthetic financial records with frozen dataclasses. Connected four local tools through a bounded agent loop.
- Compared PAY-1048 with PAY-1050 to produce evidence-backed recommendations. Kept payment authority with a human reviewer. Recorded every tool call plus each final report in a JSON Lines audit trail stored in audit.jsonl.
- Added a missing-purchase-order case in the Secret Mission. Proved PAY-1052 escalates for human review without inventing the contents of PO-999.
Ready to quiz yourself?