Build an AI Document Processor

Process text files through Claude and save structured JSON results.

Introduction

30 Second Summary

Every day at work, people spend hours reading documents and pulling out the important bits. Meeting notes need summaries. Emails need action items. Reports need classification. What if you could drop a file into a folder and have AI do all of that for you automatically?

In this project, you will build an AI Document Processor that reads text files, sends them to Claude for analysis, and saves structured results as JSON files. Along the way, you will level up your Python skills with proper error handling, logging, and file management.

What You'll Build

A production-quality Python document processing pipeline that reads files from an input folder, processes them through the Anthropic Messages API, and outputs structured JSON results with timestamped logging and graceful error handling.

By the end of this project, you'll have:

A multi-file Python project with JSON configuration, a document processor module, and a main entry point that handles single-file and folder-based processing.
Production-quality error handling and logging using try/except blocks and Python's logging module to record every processing step to a timestamped log file.
Hands-on understanding of how the Anthropic Messages API differs from the OpenAI-compatible pattern you used in Part 1, and why AI Automation Specialists need to know both.
Secret Mission: a batch processing mode that processes all files in a folder and generates a combined summary report with success/failure tracking.

Are there any prerequisites?

This is Part 2 of the AI Workflow Automator series. You should have completed Part 1 (AI Chatbot with Groq) and be comfortable with basic Python concepts like variables, functions, lists, dictionaries, and API calls.

You will also need a code editor (VS Code or Cursor) and Python installed on your machine.

Before We Start

In Part 1, you built a chatbot that could hold a conversation with an AI model. That was a great start, but production automation looks different.

In this step, you'll set up the project workspace, install the Anthropic Python SDK, grab an API key, and create a configuration file that controls how your processor behaves.

In this step, get ready to:

Create the project folder structure with input/, output/, and logs/ subfolders.
Install the Anthropic Python SDK and configure your API key.
Create a JSON configuration file for your document processor.

Create your project folder structure

Your document processor needs a clean workspace with designated folders for incoming files, processed results, and logs.

Open your terminal by pressing Cmd+Space (macOS) or the Windows key (Windows), typing Terminal, and pressing Enter.
Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder and move into it by running these commands:

mkdir ai-document-processor
cd ai-document-processor

Create the three subfolders your processor will use by running:

mkdir input output logs

Confirm the folders exist by running:

ls

You should see input, output, and logs listed in the output.

What are these folders for?

The input/ folder holds text files you want processed. The output/ folder receives structured JSON results from Claude. The logs/ folder stores a timestamped record of every processing step.

Install the Anthropic Python SDK by running:

pip install anthropic

What does this install?

The anthropic package is the official Python SDK for calling Claude's API. It handles authentication, request formatting, and response parsing for you.

Seeing a permission error or command not found?

If pip is not found, try pip3 instead. If you see a permission error on macOS, try pip install --user anthropic.

help me fix my pip install error for the anthropic package

Get your Anthropic API key

To call Claude from your code, you need an API key from the Anthropic developer console. The SDK reads this key automatically from an environment variable.

Open your browser and go to platform.claude.com.
Sign up for a new account (or sign in if you already have one).
Click Settings in the left sidebar, then click Billing.
Add a payment method. Anthropic uses pay-as-you-go billing, so you are only charged for the tokens you actually use.
Click API Keys in the left sidebar.
Click Create Key.
Give your key a name like document-processor.
Copy the key immediately and save it somewhere safe. Anthropic only shows the full key once.

How much will this cost?

This project uses Claude Haiku 4.5, which costs $1 per million input tokens and $5 per million output tokens. Each document you process uses roughly 500 to 1500 tokens. The entire project should cost well under $0.50.

Now set the API key as an environment variable so the SDK can find it.

macOS

In your terminal, set the environment variable by running:

export ANTHROPIC_API_KEY="your-api-key-here"

Replace your-api-key-here with the key you just copied.

Windows

In PowerShell, set the environment variable by running:

$env:ANTHROPIC_API_KEY = "your-api-key-here"

Replace your-api-key-here with the key you just copied.

Keep your key for the session

This environment variable only lasts for the current terminal session. If you close your terminal, you will need to set it again before running your processor.

Not sure if your key is set correctly?

You will verify the key works in the next step when you make your first API call. If it fails with an authentication error, return here and double-check you pasted the full key without extra spaces.

help me troubleshoot my Anthropic API key setup

Create the configuration file

Production tools use configuration files instead of hardcoded values. Your processor will read its settings from a JSON file, making it easy to change behavior without editing code.

Open your ai-document-processor folder in VS Code.
Create a new file called config.json in the root of your project folder.
Add the following configuration to config.json:

{
    "model": "claude-haiku-4-5",
    "max_tokens": 1024,
    "task": "summarize",
    "input_folder": "input",
    "output_folder": "output",
    "log_folder": "logs"
}

What does each setting control?

model sets which Claude model to use. Haiku 4.5 is fast and cost-effective for document processing.
max_tokens limits how long Claude's response can be.
task tells the processor what to do with your documents. You will add more task types later.
The folder settings point the processor to your input/, output/, and logs/ directories.

Save config.json.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file.

ⓧ I'd like to double check the full code

{
    "model": "claude-haiku-4-5",
    "max_tokens": 1024,
    "task": "summarize",
    "input_folder": "input",
    "output_folder": "output",
    "log_folder": "logs"
}

Your workspace is ready to go. Next up, you will learn how Claude's API differs from the Groq pattern you used in Part 1 and make your first API call.

Learn the Anthropic API Pattern

Your project structure is ready and the Anthropic SDK is installed. Now it's time to understand the tool you'll be working with.

In Part 1 of this series, you used Groq's API to talk to an AI model. Groq follows the OpenAI-compatible pattern. Claude uses a different pattern called the Anthropic Messages API. In this step, you'll build a side-by-side comparison script that shows both patterns and makes a live call to Claude.

In this step, get ready to:

Build a comparison script showing Groq's pattern vs Anthropic's pattern.
Understand the key differences between both API formats.
Set up proper error handling with try/except around a live Claude call.

Build a script comparing both API patterns

AI Automation Specialists work with multiple AI providers. Each provider has its own SDK and response format. The fastest way to learn a new pattern is to put it next to one you already know.

Why learn both API patterns?

Most AI providers (Groq, OpenAI, Mistral) share an identical "OpenAI-compatible" format. Anthropic chose a different structure for its Messages API. Knowing both patterns means you can switch providers without confusion.

Create a new file called compare_apis.py in your project folder by running:

touch compare_apis.py

Open compare_apis.py in your editor and paste the following code that shows the Groq/OpenAI pattern from Part 1:

"""
Side-by-side comparison: Groq (OpenAI-compatible) vs Anthropic (Claude) API patterns.
This shows why AI Automation Specialists need to understand both patterns.
"""

# ==================================================
# PATTERN 1: OpenAI-Compatible (Groq, OpenAI, etc.)
# Used in Part 1 of this series
# ==================================================

print("=" * 50)
print("PATTERN 1: OpenAI-Compatible (Groq)")
print("=" * 50)

print("""
from groq import Groq

client = Groq()  # Uses GROQ_API_KEY env var

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Summarize this text..."}
    ],
    max_tokens=1024,
    temperature=0.5
)

# Access the response
result = response.choices[0].message.content
print(result)
""")

What does this code do?

This is the pattern you learned in Part 1. The key bits are: client.chat.completions.create() as the method call, and the system prompt living INSIDE the messages array alongside the user message.

You access the response text via response.choices[0].message.content. Keep that in mind as you see how Anthropic does it differently.

Add the Anthropic pattern below your Pattern 1 code by pasting this next:

# ==================================================
# PATTERN 2: Anthropic (Claude)
# Used in Part 2 (this project!)
# ==================================================

print("=" * 50)
print("PATTERN 2: Anthropic (Claude)")
print("=" * 50)

print("""
import anthropic

client = anthropic.Anthropic()  # Uses ANTHROPIC_API_KEY env var

response = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=1024,
    system="You are a helpful assistant.",
    messages=[
        {"role": "user", "content": "Summarize this text..."}
    ]
)

# Access the response
result = response.content[0].text
print(result)
""")

What's different in the Anthropic pattern?

Three differences jump out immediately. The method is messages.create() instead of chat.completions.create(). The system prompt is a separate system parameter, not part of the messages array. And you access the response text via response.content[0].text instead of response.choices[0].message.content.

Add the key differences reference table below Pattern 2 by pasting this code:

# ==================================================
# KEY DIFFERENCES
# ==================================================

print("=" * 50)
print("KEY DIFFERENCES")
print("=" * 50)

print("""
| Feature            | Groq/OpenAI Pattern          | Anthropic Pattern            |
|--------------------|------------------------------|------------------------------|
| Client             | Groq() or OpenAI()           | anthropic.Anthropic()        |
| Method             | chat.completions.create()    | messages.create()            |
| System prompt      | In messages array            | Separate 'system' parameter  |
| Response text      | .choices[0].message.content  | .content[0].text             |
| Token usage        | .usage.total_tokens          | .usage.input_tokens +        |
|                    |                              |  .usage.output_tokens        |
| Env var            | GROQ_API_KEY                 | ANTHROPIC_API_KEY            |
| Stop reason        | .choices[0].finish_reason    | .stop_reason                 |
""")

What should I remember from this table?

The two most important rows are System prompt and Response text. These are where bugs happen most often when switching between providers. If you mix them up, your code will run but give you empty or wrong results.

Save compare_apis.py.
Run the script so far to confirm the reference output prints correctly:

python compare_apis.py

You should see three sections of output in your terminal: the Groq pattern, the Anthropic pattern, and the key differences table.

Seeing a syntax error?

Triple-quoted strings (the """ blocks) are sensitive to indentation. Make sure each print(""" has a matching """) on its own line. Also check that no extra quotes snuck in when pasting.

Help me fix a syntax error in my compare_apis.py script

Add a live Claude call with error handling

Printing reference code is useful for studying, but you also want to prove the pattern works. The next section makes a real API call to Claude and wraps it in try/except blocks so errors are handled gracefully.

Add the live demo section at the bottom of compare_apis.py by pasting this code:

# ==================================================
# LIVE DEMO: Actually call Claude
# ==================================================

print("=" * 50)
print("LIVE DEMO: Calling Claude")
print("=" * 50)

import anthropic

try:
    client = anthropic.Anthropic()
    response = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=256,
        system="You are a helpful assistant. Be concise.",
        messages=[
            {"role": "user", "content": "In one sentence, what is the main difference between the OpenAI API format and the Anthropic API format?"}
        ]
    )

    print(f"\nClaude says: {response.content[0].text}")
    print(f"\nTokens used: {response.usage.input_tokens} input + {response.usage.output_tokens} output")
    print(f"Stop reason: {response.stop_reason}")

except anthropic.AuthenticationError:
    print("\nError: ANTHROPIC_API_KEY not set or invalid.")
    print("Set it with: set ANTHROPIC_API_KEY=your-key-here")
except Exception as e:
    print(f"\nError: {e}")

What does this code do?

anthropic.Anthropic() creates a client that automatically reads your ANTHROPIC_API_KEY environment variable.
client.messages.create() sends your message to Claude. Notice that system is its own parameter, separate from messages.
response.content[0].text extracts the actual text Claude returned.
The try/except blocks catch authentication failures (wrong or missing key) and any other unexpected errors, printing a helpful message instead of crashing.

Save compare_apis.py.
Run the full script by running:

python compare_apis.py

At the bottom of the output, after the reference patterns and key differences table, you should see a "LIVE DEMO" section with Claude's one-sentence answer, the token count, and a stop reason of end_turn.

Seeing an authentication error?

If you see "ANTHROPIC_API_KEY not set or invalid", your environment variable isn't being picked up. Make sure you set it in the SAME terminal session you're running the script from. If you opened a new terminal window since Step 1, you'll need to set the variable again.

Help me fix my ANTHROPIC_API_KEY environment variable so my script can connect to Claude

✔️ Awesome, I've got everything!

Great. Double check you've saved your file, then move on to the next step.

ⓧ I'd like to double check the full code

Here's the complete compare_apis.py for comparison:

"""
Side-by-side comparison: Groq (OpenAI-compatible) vs Anthropic (Claude) API patterns.
This shows why AI Automation Specialists need to understand both patterns.
"""

# ==================================================
# PATTERN 1: OpenAI-Compatible (Groq, OpenAI, etc.)
# Used in Part 1 of this series
# ==================================================

print("=" * 50)
print("PATTERN 1: OpenAI-Compatible (Groq)")
print("=" * 50)

print("""
from groq import Groq

client = Groq()  # Uses GROQ_API_KEY env var

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Summarize this text..."}
    ],
    max_tokens=1024,
    temperature=0.5
)

# Access the response
result = response.choices[0].message.content
print(result)
""")

# ==================================================
# PATTERN 2: Anthropic (Claude)
# Used in Part 2 (this project!)
# ==================================================

print("=" * 50)
print("PATTERN 2: Anthropic (Claude)")
print("=" * 50)

print("""
import anthropic

client = anthropic.Anthropic()  # Uses ANTHROPIC_API_KEY env var

response = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=1024,
    system="You are a helpful assistant.",
    messages=[
        {"role": "user", "content": "Summarize this text..."}
    ]
)

# Access the response
result = response.content[0].text
print(result)
""")

# ==================================================
# KEY DIFFERENCES
# ==================================================

print("=" * 50)
print("KEY DIFFERENCES")
print("=" * 50)

print("""
| Feature            | Groq/OpenAI Pattern          | Anthropic Pattern            |
|--------------------|------------------------------|------------------------------|
| Client             | Groq() or OpenAI()           | anthropic.Anthropic()        |
| Method             | chat.completions.create()    | messages.create()            |
| System prompt      | In messages array            | Separate 'system' parameter  |
| Response text      | .choices[0].message.content  | .content[0].text             |
| Token usage        | .usage.total_tokens          | .usage.input_tokens +        |
|                    |                              |  .usage.output_tokens        |
| Env var            | GROQ_API_KEY                 | ANTHROPIC_API_KEY            |
| Stop reason        | .choices[0].finish_reason    | .stop_reason                 |
""")

# ==================================================
# LIVE DEMO: Actually call Claude
# ==================================================

print("=" * 50)
print("LIVE DEMO: Calling Claude")
print("=" * 50)

import anthropic

try:
    client = anthropic.Anthropic()
    response = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=256,
        system="You are a helpful assistant. Be concise.",
        messages=[
            {"role": "user", "content": "In one sentence, what is the main difference between the OpenAI API format and the Anthropic API format?"}
        ]
    )

    print(f"\nClaude says: {response.content[0].text}")
    print(f"\nTokens used: {response.usage.input_tokens} input + {response.usage.output_tokens} output")
    print(f"Stop reason: {response.stop_reason}")

except anthropic.AuthenticationError:
    print("\nError: ANTHROPIC_API_KEY not set or invalid.")
    print("Set it with: set ANTHROPIC_API_KEY=your-key-here")
except Exception as e:
    print(f"\nError: {e}")

You now have a working reference showing both API patterns side by side. Next up, you'll use the Anthropic pattern to build a real document processor that reads files, sends them to Claude, and writes structured JSON results.

Build the Document Processor

You have a working connection to Claude and understand how the Anthropic API differs from Groq. Now it's time to build the actual document processor.

A production document processor needs to handle the full lifecycle: read files, send them to an AI model, save structured results, and log everything that happens along the way. If something fails (a missing file, a bad API key, a malformed response), the processor needs to handle it gracefully instead of crashing.

In this step, get ready to:

Implement the core processor that reads files from input/, sends them to Claude, and writes JSON results to output/.
Add Python's logging module to record every step to a log file.
Add comprehensive try/except error handling for file operations and API failures.

Set up processor.py with logging

Production automation tools log everything they do. When something fails at 3am, logs are how you figure out what happened.

Create a new file called processor.py in your ai-document-processor project folder (the same folder where config.json lives).
Add the imports and logging setup function by pasting this code:

import anthropic
import json
import os
import logging
from datetime import datetime


def setup_logging(log_folder):
    # Create the log folder if it doesn't exist
    os.makedirs(log_folder, exist_ok=True)
    # Generate a unique log filename with the current timestamp
    log_filename = os.path.join(log_folder, f"processing_{datetime.now().strftime('%Y%m%d_%H%M%S')}.log")

    # Configure logging to write to both a file and the terminal
    logging.basicConfig(
        level=logging.INFO,
        format="%(asctime)s - %(levelname)s - %(message)s",
        handlers=[
            logging.FileHandler(log_filename, encoding="utf-8"),
            logging.StreamHandler()
        ]
    )

    logging.info(f"Logging initialized. Log file: {log_filename}")
    return log_filename

What does this code do?

The imports bring in everything the processor needs: the anthropic SDK for Claude, json for reading/writing structured data, os for file paths, and logging for recording events.

The setup_logging() function creates a timestamped log file in the logs/ folder. It uses two handlers: a FileHandler that writes to the log file, and a StreamHandler that prints to your terminal so you can see progress in real time.

What is Python's logging module?

Python's built-in logging module is the standard way to record events in production code. Every log entry gets a timestamp, severity level (INFO, WARNING, ERROR), and message.

Using logging instead of print() gives you control over where messages go, what severity they are, and whether they get saved to a file for later review.

Save processor.py.

Add config loading and file reading

The processor needs to load settings from your JSON config file and read input documents safely. Both operations can fail, so each gets its own try/except error handling.

In processor.py, add the load_config() function below the setup_logging() function:

def load_config(config_path="config.json"):
    try:
        with open(config_path, "r", encoding="utf-8") as f:
            config = json.load(f)
        logging.info(f"Config loaded from {config_path}: model={config['model']}, task={config['task']}")
        return config
    except FileNotFoundError:
        logging.error(f"Config file not found: {config_path}")
        raise
    except json.JSONDecodeError as e:
        logging.error(f"Invalid JSON in config file: {e}")
        raise

What does this code do?

This function opens config.json and parses it into a Python dictionary. It logs which model and task are configured so you can verify at a glance.

The try/except blocks catch two specific failures: the config file doesn't exist (FileNotFoundError), or it contains invalid JSON (json.JSONDecodeError). Both are logged as errors and re-raised so the caller knows something went wrong.

Below load_config(), add the read_input_file() function:

def read_input_file(file_path):
    try:
        with open(file_path, "r", encoding="utf-8") as f:
            content = f.read()
        logging.info(f"Read input file: {file_path} ({len(content)} characters)")
        return content
    except FileNotFoundError:
        logging.error(f"Input file not found: {file_path}")
        return None
    except Exception as e:
        logging.error(f"Error reading file {file_path}: {e}")
        return None

What does this code do?

This function reads a text file and returns its contents as a string. It logs the file path and character count for traceability.

If the file doesn't exist or can't be read for any reason, it returns None instead of crashing. The calling code checks for None and skips that file gracefully.

Save processor.py.

Seeing a syntax error?

Make sure each function starts at the same indentation level (no extra indentation) and that the try, except, and return lines use consistent 4-space indentation.

help me fix indentation errors in my processor.py file

Build the processing pipeline with error handling

Now for the heart of the processor: the functions that select a system prompt, call Claude, and save the structured results.

Below read_input_file(), add the get_system_prompt() function that maps task names to prompt templates:

def get_system_prompt(task):
    # Map each task type to a specific system prompt
    prompts = {
        "summarize": "You are a document summarizer. Provide a clear, concise summary of the document. Return your response as JSON with keys: 'summary' (string), 'key_points' (list of strings), 'word_count' (integer of original document).",
        "extract_actions": "You are an action item extractor. Identify all action items, tasks, and next steps from the document. Return your response as JSON with keys: 'action_items' (list of objects with 'task', 'priority', 'assignee' fields), 'total_actions' (integer).",
        "classify": "You are a document classifier. Classify the document into a category and assess its sentiment and urgency. Return your response as JSON with keys: 'category' (string), 'sentiment' (positive/neutral/negative), 'urgency' (low/medium/high), 'confidence' (float 0-1)."
    }
    return prompts.get(task, prompts["summarize"])

What does this code do?

This function acts as a lookup table. Based on the task value from your config file ("summarize", "extract_actions", or "classify"), it returns the matching system prompt.

Each prompt instructs Claude to return structured JSON, which makes parsing the output predictable. If an unknown task is provided, it defaults to the summarize prompt.

Next, add the core process_document() function that calls the Claude API:

def process_document(content, config):
    # Create the Anthropic client (reads ANTHROPIC_API_KEY from env)
    client = anthropic.Anthropic()

    system_prompt = get_system_prompt(config["task"])

    logging.info(f"Sending document to Claude ({config['model']}, task={config['task']})")

    try:
        response = client.messages.create(
            model=config["model"],
            max_tokens=config["max_tokens"],
            system=system_prompt,
            messages=[
                {"role": "user", "content": f"Process this document:\n\n{content}"}
            ]
        )

        # Extract the response text and token usage
        result_text = response.content[0].text
        tokens_used = response.usage.input_tokens + response.usage.output_tokens
        logging.info(f"Claude response received. Tokens used: {tokens_used}")

        # Try to parse Claude's response as JSON
        try:
            result = json.loads(result_text)
        except json.JSONDecodeError:
            logging.warning("Response was not valid JSON, wrapping as raw text")
            result = {"raw_response": result_text}

        # Add metadata about the processing run
        result["_metadata"] = {
            "model": config["model"],
            "task": config["task"],
            "tokens_used": tokens_used,
            "processed_at": datetime.now().isoformat()
        }

        return result

    except anthropic.AuthenticationError:
        logging.error("Authentication failed. Check your ANTHROPIC_API_KEY.")
        return None
    except anthropic.RateLimitError:
        logging.error("Rate limit exceeded. Wait and try again.")
        return None
    except anthropic.APIError as e:
        logging.error(f"Anthropic API error: {e}")
        return None
    except Exception as e:
        logging.error(f"Unexpected error during processing: {e}")
        return None

What does this code do?

This is the core function that sends a document to Claude and handles the response. It uses the same client.messages.create() pattern you tested in compare_apis.py, but now with proper error handling around it.

The nested try/except attempts to parse Claude's response as JSON. If Claude returns plain text instead of JSON, it wraps the response gracefully rather than crashing.

The outer try/except catches specific API errors: authentication failures, rate limits, and general API errors. Each error gets a clear log message so you know exactly what went wrong.

Add the save_output() function below process_document():

def save_output(result, input_filename, output_folder):
    # Create the output folder if it doesn't exist
    os.makedirs(output_folder, exist_ok=True)

    # Generate output filename based on the input filename
    base_name = os.path.splitext(os.path.basename(input_filename))[0]
    output_filename = f"{base_name}_result.json"
    output_path = os.path.join(output_folder, output_filename)

    try:
        with open(output_path, "w", encoding="utf-8") as f:
            json.dump(result, f, indent=2, ensure_ascii=False)
        logging.info(f"Result saved to: {output_path}")
        return output_path
    except Exception as e:
        logging.error(f"Error saving output: {e}")
        return None

What does this code do?

This function takes the processed result and saves it as a formatted JSON file in the output folder. It names the output file based on the input filename (meeting_notes.txt becomes meeting_notes_result.json).

The indent=2 parameter makes the JSON human-readable. ensure_ascii=False preserves any non-English characters in the document.

Finally, add the process_single_file() function that ties everything together:

def process_single_file(file_path, config):
    logging.info(f"--- Processing: {file_path} ---")

    # Step 1: Read the input file
    content = read_input_file(file_path)
    if content is None:
        return None

    # Step 2: Skip empty files
    if len(content.strip()) == 0:
        logging.warning(f"Skipping empty file: {file_path}")
        return None

    # Step 3: Send to Claude for processing
    result = process_document(content, config)
    if result is None:
        return None

    # Step 4: Save the result to the output folder
    output_path = save_output(result, file_path, config["output_folder"])
    return output_path

What does this code do?

This is the orchestrator function. It calls each step of the pipeline in order: read the file, check it's not empty, process it through Claude, and save the output.

At each step, if something returns None (indicating a failure), the function stops and returns None itself. The earlier functions already logged the specific error, so the caller just needs to know whether processing succeeded or not.

Save processor.py.

Seeing an error when you save?

Check that every function is separated by two blank lines (one blank line between the last line of a function and the def of the next function). Also verify that you pasted the functions in order, one after another, without accidentally nesting one inside another.

help me check the structure of my processor.py file

✔️ Awesome, I've got everything!

Great. Double check you've saved processor.py with all seven functions.

ⓧ I'd like to double check the full code

Here's the complete processor.py file for comparison:

import anthropic
import json
import os
import logging
from datetime import datetime


def setup_logging(log_folder):
    os.makedirs(log_folder, exist_ok=True)
    log_filename = os.path.join(log_folder, f"processing_{datetime.now().strftime('%Y%m%d_%H%M%S')}.log")

    logging.basicConfig(
        level=logging.INFO,
        format="%(asctime)s - %(levelname)s - %(message)s",
        handlers=[
            logging.FileHandler(log_filename, encoding="utf-8"),
            logging.StreamHandler()
        ]
    )

    logging.info(f"Logging initialized. Log file: {log_filename}")
    return log_filename


def load_config(config_path="config.json"):
    try:
        with open(config_path, "r", encoding="utf-8") as f:
            config = json.load(f)
        logging.info(f"Config loaded from {config_path}: model={config['model']}, task={config['task']}")
        return config
    except FileNotFoundError:
        logging.error(f"Config file not found: {config_path}")
        raise
    except json.JSONDecodeError as e:
        logging.error(f"Invalid JSON in config file: {e}")
        raise


def read_input_file(file_path):
    try:
        with open(file_path, "r", encoding="utf-8") as f:
            content = f.read()
        logging.info(f"Read input file: {file_path} ({len(content)} characters)")
        return content
    except FileNotFoundError:
        logging.error(f"Input file not found: {file_path}")
        return None
    except Exception as e:
        logging.error(f"Error reading file {file_path}: {e}")
        return None


def get_system_prompt(task):
    prompts = {
        "summarize": "You are a document summarizer. Provide a clear, concise summary of the document. Return your response as JSON with keys: 'summary' (string), 'key_points' (list of strings), 'word_count' (integer of original document).",
        "extract_actions": "You are an action item extractor. Identify all action items, tasks, and next steps from the document. Return your response as JSON with keys: 'action_items' (list of objects with 'task', 'priority', 'assignee' fields), 'total_actions' (integer).",
        "classify": "You are a document classifier. Classify the document into a category and assess its sentiment and urgency. Return your response as JSON with keys: 'category' (string), 'sentiment' (positive/neutral/negative), 'urgency' (low/medium/high), 'confidence' (float 0-1)."
    }
    return prompts.get(task, prompts["summarize"])


def process_document(content, config):
    client = anthropic.Anthropic()

    system_prompt = get_system_prompt(config["task"])

    logging.info(f"Sending document to Claude ({config['model']}, task={config['task']})")

    try:
        response = client.messages.create(
            model=config["model"],
            max_tokens=config["max_tokens"],
            system=system_prompt,
            messages=[
                {"role": "user", "content": f"Process this document:\n\n{content}"}
            ]
        )

        result_text = response.content[0].text
        tokens_used = response.usage.input_tokens + response.usage.output_tokens
        logging.info(f"Claude response received. Tokens used: {tokens_used}")

        try:
            result = json.loads(result_text)
        except json.JSONDecodeError:
            logging.warning("Response was not valid JSON, wrapping as raw text")
            result = {"raw_response": result_text}

        result["_metadata"] = {
            "model": config["model"],
            "task": config["task"],
            "tokens_used": tokens_used,
            "processed_at": datetime.now().isoformat()
        }

        return result

    except anthropic.AuthenticationError:
        logging.error("Authentication failed. Check your ANTHROPIC_API_KEY.")
        return None
    except anthropic.RateLimitError:
        logging.error("Rate limit exceeded. Wait and try again.")
        return None
    except anthropic.APIError as e:
        logging.error(f"Anthropic API error: {e}")
        return None
    except Exception as e:
        logging.error(f"Unexpected error during processing: {e}")
        return None


def save_output(result, input_filename, output_folder):
    os.makedirs(output_folder, exist_ok=True)

    base_name = os.path.splitext(os.path.basename(input_filename))[0]
    output_filename = f"{base_name}_result.json"
    output_path = os.path.join(output_folder, output_filename)

    try:
        with open(output_path, "w", encoding="utf-8") as f:
            json.dump(result, f, indent=2, ensure_ascii=False)
        logging.info(f"Result saved to: {output_path}")
        return output_path
    except Exception as e:
        logging.error(f"Error saving output: {e}")
        return None


def process_single_file(file_path, config):
    logging.info(f"--- Processing: {file_path} ---")

    content = read_input_file(file_path)
    if content is None:
        return None

    if len(content.strip()) == 0:
        logging.warning(f"Skipping empty file: {file_path}")
        return None

    result = process_document(content, config)
    if result is None:
        return None

    output_path = save_output(result, file_path, config["output_folder"])
    return output_path

Your document processor is built. Next up, you'll wire it into a main script and add multiple processing modes so you can summarize, extract actions, or classify any document by changing a single config value.

Add JSON Configuration and Processing Modes

Your processor functions are ready to go. Now you need a way to actually run them.

Production automation tools never hardcode their settings. They read from a configuration file so you can change behavior without touching code. In this step, you will create the main entry point script that loads settings from config.json, wires up the processing pipeline, and handles both single-file and batch-folder workflows.

In this step, get ready to:

Understand how config.json drives the three processing modes.
Create main.py to orchestrate the full processing pipeline.
Test the pipeline end-to-end with a sample document.

Understand config.json and the three processing modes

Your config.json file already exists. It controls everything about how your processor behaves without requiring any code changes.

Switch back to your editor and open config.json.
Review the six settings that drive your processor:

{
    "model": "claude-haiku-4-5",
    "max_tokens": 1024,
    "task": "summarize",
    "input_folder": "input",
    "output_folder": "output",
    "log_folder": "logs"
}

What does each setting do?

model picks which Claude model to use. claude-haiku-4-5 is fast and cheap at $1 per million input tokens.
max_tokens limits how long Claude's response can be.
task controls which processing mode runs. Your get_system_prompt() function in processor.py reads this value and picks the matching system prompt.
input_folder, output_folder, and log_folder set where files are read from, written to, and where logs are stored.

What are the three processing modes?

The task field accepts three values, each triggering a different AI behavior:

summarize returns a summary, key points, and word count as JSON.
extract_actions identifies action items with priority and assignee fields.
classify categorizes the document and assesses sentiment, urgency, and confidence.

To switch modes, you just change the task value in config.json and re-run. No code edits needed.

Create main.py to orchestrate the pipeline

Your processor functions live in processor.py, but they need an entry point that ties everything together. That is main.py.

Create a new file called main.py in your project folder (the same folder where processor.py and config.json live).
Paste the following code into main.py:

import os
import sys
from processor import setup_logging, load_config, process_single_file


def main():
    print("\n=== AI Document Processor ===")
    print("Part 2: AI Workflow Automator with Claude\n")

    # Load settings from config.json
    config = load_config()
    setup_logging(config["log_folder"])

    # Ensure input and output folders exist
    os.makedirs(config["input_folder"], exist_ok=True)
    os.makedirs(config["output_folder"], exist_ok=True)

    # Mode 1: Process a single file passed as a command-line argument
    if len(sys.argv) > 1:
        file_path = sys.argv[1]
        result = process_single_file(file_path, config)
        if result:
            print(f"\nSuccess! Output saved to: {result}")
        else:
            print("\nProcessing failed. Check the log file for details.")
    else:
        # Mode 2: Process all .txt files in the input folder
        input_files = [
            f for f in os.listdir(config["input_folder"])
            if f.endswith(".txt")
        ]

        if not input_files:
            print(f"No .txt files found in '{config['input_folder']}/' folder.")
            print(f"Add text files to '{config['input_folder']}/' and run again.")
            print(f"\nExample: Create '{config['input_folder']}/meeting_notes.txt' with some text content.")
            return

        print(f"Found {len(input_files)} file(s) to process.\n")

        for filename in input_files:
            file_path = os.path.join(config["input_folder"], filename)
            result = process_single_file(file_path, config)
            if result:
                print(f"  Done: {filename} -> {result}")
            else:
                print(f"  FAILED: {filename} (check logs)")

    print("\nProcessing complete. Check output/ and logs/ folders.")


if __name__ == "__main__":
    main()

What does this code do?

This script is the orchestrator. It does not process documents itself. It delegates to the functions you already built in processor.py.

load_config() reads config.json and returns a dictionary of settings.
setup_logging() creates a timestamped log file in the logs/ folder.
sys.argv checks if you passed a specific file path when running the script. If so, it processes just that one file.
If no file path is given, the script scans the input/ folder for all .txt files and processes each one.

Why use sys.argv for command-line arguments?

sys.argv is a list that Python populates with whatever you type after python main.py in the terminal. If you run python main.py input/notes.txt, then sys.argv[1] contains input/notes.txt.

This gives your processor two modes of operation. You can target a specific file or let it auto-discover everything in the input folder.

Save main.py.

Seeing a ModuleNotFoundError?

If you see an error mentioning processor not being found, make sure main.py is in the same folder as processor.py. Python imports from the same directory by default.

help me fix a ModuleNotFoundError when importing processor in main.py

✔️ Awesome, I've got everything!

Great. Double check you have saved main.py in the same folder as processor.py and config.json.

ⓧ I'd like to double check the full code

Here is the complete main.py file:

import os
import sys
from processor import setup_logging, load_config, process_single_file


def main():
    print("\n=== AI Document Processor ===")
    print("Part 2: AI Workflow Automator with Claude\n")

    config = load_config()
    setup_logging(config["log_folder"])

    os.makedirs(config["input_folder"], exist_ok=True)
    os.makedirs(config["output_folder"], exist_ok=True)

    if len(sys.argv) > 1:
        file_path = sys.argv[1]
        result = process_single_file(file_path, config)
        if result:
            print(f"\nSuccess! Output saved to: {result}")
        else:
            print("\nProcessing failed. Check the log file for details.")
    else:
        input_files = [
            f for f in os.listdir(config["input_folder"])
            if f.endswith(".txt")
        ]

        if not input_files:
            print(f"No .txt files found in '{config['input_folder']}/' folder.")
            print(f"Add text files to '{config['input_folder']}/' and run again.")
            print(f"\nExample: Create '{config['input_folder']}/meeting_notes.txt' with some text content.")
            return

        print(f"Found {len(input_files)} file(s) to process.\n")

        for filename in input_files:
            file_path = os.path.join(config["input_folder"], filename)
            result = process_single_file(file_path, config)
            if result:
                print(f"  Done: {filename} -> {result}")
            else:
                print(f"  FAILED: {filename} (check logs)")

    print("\nProcessing complete. Check output/ and logs/ folders.")


if __name__ == "__main__":
    main()

Test the full pipeline with a sample document

Time to see everything working together. You will create a sample text file, run the processor, and verify the JSON output.

Create a sample document in the input/ folder by running this command:

echo "Team meeting notes from Monday. We decided to launch the new feature by Friday. Sarah will handle the frontend. Mike will update the API documentation. Everyone should review the pull requests by Wednesday." > input/meeting_notes.txt

Confirm the file was created by running:

ls input/

You should see meeting_notes.txt listed in the output.

Run the processor by running this command:

python main.py

You should see output showing the processor found 1 file, processed it, and saved the result. It looks something like this:

=== AI Document Processor ===
Part 2: AI Workflow Automator with Claude

Found 1 file(s) to process.

  Done: meeting_notes.txt -> output/meeting_notes_result.json

Processing complete. Check output/ and logs/ folders.

Check the JSON output by viewing the result file:

cat output/meeting_notes_result.json

You should see structured JSON with a summary, key_points, word_count, and a _metadata section showing the model used, tokens consumed, and timestamp.

Check the log file by running:

ls logs/

You should see a log file with a timestamped name like processing_20260523_141500.log. This file records every step the processor took.

Seeing an authentication error?

If the log shows "Authentication failed", your ANTHROPIC_API_KEY environment variable is not set in this terminal session. Re-export it before running again.

help me fix an Anthropic authentication error when running main.py

Your pipeline is wired together and processing documents. Next up, you will stress-test it by creating more sample files, switching processing modes, and deliberately triggering errors to see how the error handling responds.

Run and Verify the Complete Pipeline

Your document processor is fully wired together. You have a config file controlling behavior, a processor with error handling and logging, and a main entry point that can handle both single files and folder scans.

Now it is time to put the whole system through its paces. You will feed it real documents, inspect the structured JSON output, read the log file, and deliberately break things to confirm your error handling catches every failure gracefully.

In this step, get ready to:

Create sample input documents and process them end-to-end.
Verify the JSON output structure and log file contents.
Intentionally trigger errors to confirm graceful handling.

Create sample input documents and run the processor

Your processor reads .txt files from the input/ folder. You need at least one real document to test the full pipeline.

In your code editor, create a new file called meeting_notes.txt inside the input/ folder.
Paste the following sample content into the file:

Team Sync - May 20, 2026

Attendees: Sarah, Michael, Priya

Discussion:
- The new onboarding flow is behind schedule by two days.
- Michael will reach out to the design team for updated mockups by Thursday.
- Priya raised concerns about the API rate limits on the analytics endpoint.
- Sarah suggested caching responses to reduce API calls by 40 percent.

Action items:
- Michael: Get updated mockups from design team by Thursday.
- Priya: Write a proposal for API caching strategy by Friday.
- Sarah: Schedule follow-up meeting for next Monday.

Next meeting: Monday May 26, 2026 at 10am.

Save meeting_notes.txt.
Return to your terminal and run the processor by running this command:

python main.py

You should see output similar to this:

=== AI Document Processor ===
Part 2: AI Workflow Automator with Claude

Found 1 file(s) to process.

  Done: meeting_notes.txt -> output/meeting_notes_result.json

Processing complete. Check output/ and logs/ folders.

What just happened?

Your processor loaded config.json, read meeting_notes.txt from input/, sent it to Claude with the "summarize" system prompt, parsed the JSON response, and saved it to output/meeting_notes_result.json.

Every step was logged to a timestamped file in logs/.

Seeing an authentication error?

Make sure your ANTHROPIC_API_KEY environment variable is still set in this terminal session. If you opened a new terminal, you need to set it again.

help me debug my ANTHROPIC_API_KEY environment variable not being recognized

Verify the JSON output and log file

A production pipeline is only useful if you can trust the output format. You will now inspect both the result file and the log to confirm everything is structured correctly.

Open output/meeting_notes_result.json in your code editor.
Confirm it contains a summary key, a key_points list, a word_count integer, and a _metadata object.

What should the JSON look like?

The exact wording of the summary will vary because Claude generates it fresh each time. The structure should look like this:

{
  "summary": "A team sync meeting discussed...",
  "key_points": [
    "Onboarding flow is behind schedule",
    "API rate limit concerns raised"
  ],
  "word_count": 130,
  "_metadata": {
    "model": "claude-haiku-4-5",
    "task": "summarize",
    "tokens_used": 542,
    "processed_at": "2026-05-23T14:30:00.000000"
  }
}

The _metadata block is added by your processor. It records which model was used, what task ran, how many tokens were consumed, and the exact timestamp.

Open the logs/ folder and open the most recent .log file.
Confirm you see timestamped entries for config loading, file reading, the API call, and the output save.

What should the log file look like?

Each line starts with a timestamp, a log level, and a message. You should see entries like:

INFO - Config loaded from config.json (config was read successfully).
INFO - Read input file (your file was opened and read).
INFO - Sending document to Claude (the API call was made).
INFO - Result saved to (JSON was written to output/).

Now test single-file mode by running the processor with a specific file path:

python main.py input/meeting_notes.txt

You should see the same success message. This confirms both modes work: folder scan (no argument) and direct file path (with an argument).

Intentionally trigger errors to confirm graceful handling

Production code needs to fail gracefully. Your processor has try/except blocks around every operation that can go wrong. You will now deliberately trigger three different failure modes to confirm they are caught.

Test a missing file by running this command:

python main.py input/nonexistent_file.txt

You should see "Processing failed. Check the log file for details." in the terminal. The processor does not crash. It logs the error and exits cleanly.

Now test with an empty file. Create a new file called empty.txt inside the input/ folder with no content at all.
Save empty.txt and run:

python main.py input/empty.txt

You should see the same "Processing failed" message. Your code detects empty files and skips them with a warning instead of wasting an API call.

Open your latest log file in logs/ to confirm you see the logged error messages for both failures.

Why does this matter?

In a real automation pipeline, a single bad file should never crash the entire system. Your processor logs the problem, returns None, and moves on.

This is exactly how production AI tools handle failures at work. The log file becomes your audit trail for debugging issues after the fact.

Try a different processing mode

Open config.json and change the task value from "summarize" to "extract_actions" or "classify". Run the processor again and compare the JSON output structure. Remember to set it back to "summarize" when you are done.

Log file is empty or missing?

Make sure you are looking at the most recent .log file. Each run creates a new timestamped log file. Sort the logs/ folder by date to find the latest one.

help me figure out why my log file is empty or not being created

Secret mission

Batch Processing Mode

Your document processor handles one file at a time. But what happens when you have a folder full of reports to process? In this extension, you'll add a batch mode that processes every file in the input folder at once and generates a combined summary report with stats on tokens used, processing time, and success rates.

Clean Up Your Resources

This project runs entirely on your local machine with pay-per-use API credits. There are no running services or recurring costs. You only spend API credits when you actively run the processor.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

Project folder containing processor.py, main.py, compare_apis.py, config.json, and the input/, output/, logs/ subfolders.
ANTHROPIC_API_KEY environment variable set in your terminal session.
anthropic Python package installed via pip.
Anthropic developer account on platform.claude.com with API credits.

Keep everything running

No action needed. Choose this if you plan to keep building on this project or want to reference the code later.

Your project folder stays on your Desktop for easy access.
Your ANTHROPIC_API_KEY environment variable will disappear when you close your terminal (unless you added it to a profile file).
Your Anthropic account retains any remaining API credits for future use.
The anthropic package remains installed in your Python environment.

Pause - I'll come back to this later

Free up terminal resources while keeping your files intact.

Close any terminal sessions running the processor.
Your project files remain on disk. No changes needed.
When you return, open a new terminal and set your ANTHROPIC_API_KEY again before running the processor.

Delete - I don't want to use this again

Remove all project files and undo your setup completely.

Delete your project folder by running this command (macOS/Linux):

cd ~/Desktop
rm -rf ai-document-processor

On Windows, run this in Command Prompt instead:

cd %USERPROFILE%\Desktop
rmdir /s /q ai-document-processor

Unset your API key environment variable. On macOS/Linux:

unset ANTHROPIC_API_KEY

On Windows (Command Prompt), the variable is session-only and clears automatically when you close the terminal. If you added it to your system environment variables, remove it via Settings > System > Advanced system settings > Environment Variables.
Uninstall the Anthropic Python package:

pip uninstall anthropic

To delete your Anthropic Console account, go to platform.claude.com. Navigate to Settings > Organization > Delete organization. Follow the confirmation prompts. You must be a Console Admin and settle any outstanding balance first.

Nice Work!

You just built a production-quality AI Document Processor using Python and the Anthropic Messages API.

You've learned how to:

Built a complete file processing pipeline that reads documents from an input folder, sends them to Claude, and writes structured JSON results to an output folder.
Implemented production-quality error handling with try/except for every failure point, plus logging with Python's logging module to record timestamped entries to a log file.
Learned how the Anthropic Messages API pattern (client.messages.create()) differs from the OpenAI-compatible pattern you used in Part 1 (client.chat.completions.create()), and configured your processor with a JSON config file.
Built batch processing that handles multiple files with failure isolation and generates summary reports.

Ready to quiz yourself?