Build a Text File Analytics Engine

Build a CLI tool that analyzes any text file and produces a formatted report.

Introduction

30 Second Summary

Every day, you encounter text files packed with information. Meeting notes, log files, exported data. Extracting useful insights from them by hand is slow and error-prone.

In this project, you will build a command-line text analytics engine in Python that reads any text file and produces a complete report with word counts, frequency analysis, duplicate detection, and search functionality.

What You'll Build

A command-line tool that analyzes .txt files and prints a formatted statistics report to your terminal.

By the end of this project, you'll have:

  • A working command-line tool that analyzes any text file and produces a formatted report including word counts, top frequent words, longest and shortest words, duplicate lines, and search results.
  • Practice decomposing problems into functions by deciding what each function should do, what it takes in, and what it returns.
  • A professional project structure using virtual environments, modules, and proper error handling.
  • Secret Mission: Add a comparison mode that analyzes two files side-by-side and highlights the differences in their statistics.

Are there any prerequisites?

You should have basic familiarity with Python syntax. If you can read Python code and understand what variables, loops, and functions do, you are ready.

Set Up Your Project Environment

To build your text analytics tool, you need an isolated workspace where your project files live separately from everything else on your machine.

In this step, you'll create a project folder, set up a virtual environment, and prepare the files your tool will use.

In this step, get ready to:
  • Create a project folder on your Desktop and set up a virtual environment.
  • Verify Python is running inside the activated environment.
  • Create project files with sample data for your analyzer.
Create a project folder and initialize a virtual environment

A virtual environment is a self-contained copy of Python for your project. It keeps your project's setup isolated so nothing interferes with other work on your machine.

  • Press the Windows key to open your search bar.
  • Type Terminal and press Enter to open Windows Terminal.
  • Move to your Desktop by running this command:
cd ~/Desktop

Using Command Prompt instead of PowerShell?

If your terminal opened Command Prompt instead of PowerShell, use cd %USERPROFILE%\Desktop instead. The rest of the commands in this project work in both shells.

  • Create your project folder and move into it by running these commands:
mkdir text-analytics
cd text-analytics
  • Create a virtual environment by running:
python -m venv .venv

What does this command do?

python -m venv .venv tells Python to create a virtual environment inside a folder called .venv. The dot prefix keeps the folder hidden from casual browsing, since you never need to edit its contents directly.

  • Activate the virtual environment by running:
.venv\Scripts\activate

You should see (.venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Not seeing (.venv) in your prompt?

If you are using PowerShell and see an error about running scripts being disabled, run the following command first, then try the activation again:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

This allows PowerShell to run local scripts like the virtual environment activator. Why can't I activate my virtual environment in PowerShell?

Verify your environment and create project files
  • Check your Python version by running:
python --version

✔️ I see Python 3.14

You're all set. Your virtual environment is running the correct version of Python.

ⓧ I see a version lower than 3.14

Your system has an older version of Python. You need to install Python 3.14 to continue.

  • Go to python.org/downloads and download the Windows installer (64-bit) for Python 3.14.
  • Run the installer. Make sure to check the box that says Add Python to PATH before clicking Install Now.
  • Close and reopen Windows Terminal after the installation finishes.
  • Navigate back to your project folder and recreate the virtual environment:
cd ~/Desktop/text-analytics
rmdir /s /q .venv
python -m venv .venv
.venv\Scripts\activate
python --version

You should now see Python 3.14 or higher.

ⓧ 'python' is not recognized

Python is not on your system PATH. You need to install it.

  • Go to python.org/downloads and download the Windows installer (64-bit) for Python 3.14.
  • Run the installer. Make sure to check the box that says Add Python to PATH before clicking Install Now.
  • Close and reopen Windows Terminal after the installation finishes.
  • Navigate back to your project folder and create the virtual environment:
cd ~/Desktop/text-analytics
python -m venv .venv
.venv\Scripts\activate
python --version

You should now see Python 3.14 or higher.

  • Open your project folder in VS Code by running:
code .

VS Code opens with your text-analytics folder loaded in the Explorer sidebar.

'code' is not recognized?

The VS Code installer usually adds code to your PATH automatically. If it did not, open VS Code manually (press the Windows key, type VS Code), then use File > Open Folder and select the text-analytics folder on your Desktop. How do I add VS Code to my PATH on Windows?

  • Create the three project files. In VS Code, click the New File icon at the top of the Explorer sidebar (the page icon with a plus sign).
  • Type main.py and press Enter.
  • Click the New File icon again and create analyzer.py.
  • Click the New File icon once more and create sample.txt.
  • Confirm all three files exist by running this in your terminal:
dir

You should see main.py, analyzer.py, and sample.txt listed in the output.

Don't see all three files?

Make sure you created each file directly inside the text-analytics folder, not inside a subfolder. If a file ended up in the wrong place, drag it in the Explorer sidebar to the root of the project. Help me find my missing project files.

Populate sample.txt with test data

Your analyzer needs text to work with. The sample below includes repeated words, duplicate lines, and varied sentence lengths to give your tool interesting patterns to detect.

  • In VS Code, click on sample.txt in the Explorer sidebar to open it.
  • Paste the following text into sample.txt:
The quick brown fox jumps over the lazy dog.
Python is a powerful programming language used worldwide.
Data science and machine learning rely heavily on Python.
The quick brown fox jumps over the lazy dog.
Programming requires practice and patience.
Every expert was once a beginner in their field.
Python is a powerful programming language used worldwide.
Lists, dictionaries, and tuples are fundamental data structures.
Error handling makes your code robust and reliable.
The best way to learn is by building real projects.
Virtual environments keep your dependencies organized.
Functions break complex problems into manageable pieces.
String manipulation is a common task in text processing.
File handling allows your programs to work with external data.
The quick brown fox jumps over the lazy dog.
Modules help you organize code into reusable components.
Problem solving requires breaking things into smaller steps.
Sorting and searching are fundamental algorithms.
List comprehensions make Python code concise and readable.
Every expert was once a beginner in their field.

Why this specific text?

This sample is carefully designed to test every feature you'll build. It has three copies of the "fox" line (to test duplicate detection), repeated words like "Python" (to test frequency counting), and sentences of varying length (to test the extremes finder).

  • Save sample.txt by pressing Ctrl+S.

Your Explorer sidebar should now show all three files. The main.py and analyzer.py files are empty for now. You'll fill them in over the next steps.

✔️ Awesome, I've got everything!

Great. Double check you have saved sample.txt with the full 20 lines of text.

ⓧ I'd like to double check the full code

Here is what each file should look like at this point:

main.py and analyzer.py should both be empty (no content at all).

sample.txt should contain exactly this:

The quick brown fox jumps over the lazy dog.
Python is a powerful programming language used worldwide.
Data science and machine learning rely heavily on Python.
The quick brown fox jumps over the lazy dog.
Programming requires practice and patience.
Every expert was once a beginner in their field.
Python is a powerful programming language used worldwide.
Lists, dictionaries, and tuples are fundamental data structures.
Error handling makes your code robust and reliable.
The best way to learn is by building real projects.
Virtual environments keep your dependencies organized.
Functions break complex problems into manageable pieces.
String manipulation is a common task in text processing.
File handling allows your programs to work with external data.
The quick brown fox jumps over the lazy dog.
Modules help you organize code into reusable components.
Problem solving requires breaking things into smaller steps.
Sorting and searching are fundamental algorithms.
List comprehensions make Python code concise and readable.
Every expert was once a beginner in their field.

Your environment is ready. Next up, you'll write the core analysis functions that turn raw text into meaningful statistics.

Build the Core Analysis Functions

Your project structure is in place with an activated virtual environment and three files waiting for code. The first real challenge is figuring out how to break "analyze a text file" into specific, solvable pieces.

In this step, you'll write three foundational functions in analyzer.py. Each function solves exactly one problem, which is the core pattern you'll use throughout this project.

In this step, get ready to:
  • Write a file-loading function with error handling.
  • Build a function to count words, lines, and characters.
  • Create a word frequency counter using Python's Counter class.
Write the file-loading function

Before you can analyze any text, you need a safe way to open and read a file. Files might not exist, or something else could go wrong during reading.

Why start with file loading?

Every other function in this project depends on having text to work with. If file loading fails silently, you'll get confusing errors later. Building error handling here means every downstream function can trust its input.

  • Open analyzer.py in VS Code and add the following code:
"""Text File Analytics Engine - Analysis Module"""

from collections import Counter
import string


def load_file(filepath):
    """Load and return the contents of a text file."""
    try:
        with open(filepath, "r", encoding="utf-8") as f:
            content = f.read()
        return content
    except FileNotFoundError:
        print(f"Error: File '{filepath}' not found.")
        return None
    except Exception as e:
        print(f"Error reading file: {e}")
        return None

What does this code do?

  • The module docstring and two imports at the top prepare the file for all the functions you'll write in this step.
  • with open(filepath, "r", encoding="utf-8") as f: is a context manager that opens the file for reading and automatically closes it when done.
  • except FileNotFoundError catches the specific case where the file path is wrong.
  • except Exception as e catches anything else that could go wrong (permissions, encoding issues).
  • The function returns None on failure instead of crashing, so your program can handle the error gracefully later.
  • Save analyzer.py.
  • Test the function by running this command in your terminal:
python -c "import analyzer; content = analyzer.load_file('sample.txt'); print('Loaded!' if content else 'Failed')"

You should see Loaded! printed in the terminal.

Seeing an error instead of Loaded?

  • Make sure you saved analyzer.py before running the command.
  • Confirm your terminal is in the project folder (the same folder that contains analyzer.py and sample.txt).
  • Check that your virtual environment is still activated. You should see (.venv) at the start of your terminal prompt.

Why is my load_file function not working?

Build the basic statistics counter

Now you need a function that answers the most fundamental questions about a text: how many words, lines, and characters does it contain? This forces you to decide how to split text into those pieces.

  • In analyzer.py, add the following function below load_file:
def count_basics(text):
    """Count basic text statistics."""
    words = text.split()
    lines = text.splitlines()
    characters = len(text)

    return {
        "word_count": len(words),
        "line_count": len(lines),
        "character_count": characters,
    }

What does this code do?

  • .split() with no arguments splits on any whitespace (spaces, tabs, newlines) and discards empty strings. This gives you a clean list of words.
  • .splitlines() splits on line boundaries specifically, preserving empty lines as separate entries. This is more reliable than splitting on \n alone.
  • len(text) counts every character including spaces and newlines.
  • The function returns a dictionary so callers can access any stat by name (e.g. result["word_count"]).
  • Save analyzer.py.
  • Test the function by running this command:
python -c "import analyzer; c = analyzer.load_file('sample.txt'); print(analyzer.count_basics(c))"

You should see a dictionary with three keys: word_count, line_count, and character_count, each with a number. The line_count should be 20 since your sample.txt has 20 lines.

Seeing a TypeError or None error?

  • If you see TypeError: argument of type 'NoneType', it means load_file returned None. Check that sample.txt exists in the same folder as analyzer.py.
  • Make sure count_basics is at the same indentation level as load_file (not nested inside it).

Why is my count_basics function returning an error?

Create the word frequency counter

Knowing which words appear most often reveals the themes of a text. Python's Counter class from the collections module handles the heavy lifting of counting occurrences.

What is Counter?

Counter is a dictionary subclass designed for counting things. You pass it a list and it returns a dictionary where keys are items and values are how many times each item appeared. Its .most_common(n) method returns the top N items as a list of tuples sorted by count.

  • In analyzer.py, add the following function below count_basics:
def find_word_frequencies(text, top_n=10):
    """Find the most common words in the text."""
    words = text.lower().split()
    cleaned_words = [word.strip(string.punctuation) for word in words]
    cleaned_words = [word for word in cleaned_words if word]

    word_counts = Counter(cleaned_words)
    return word_counts.most_common(top_n)

What does this code do?

  • text.lower().split() converts everything to lowercase first so "Python" and "python" count as the same word.
  • word.strip(string.punctuation) removes punctuation from the start and end of each word. Without this, "dog." and "dog" would count separately.
  • The second list comprehension filters out any empty strings that might result from stripping punctuation.
  • Counter(cleaned_words).most_common(top_n) returns the top N most frequent words as a list of tuples like [("the", 5), ("is", 3)].
  • Save analyzer.py.
  • Test the function by running this command:
python -c "import analyzer; c = analyzer.load_file('sample.txt'); print(analyzer.find_word_frequencies(c, 5))"

You should see a list of five tuples showing the most common words and their counts. Each tuple looks like ('word', count).

Seeing a NameError for Counter or string?

  • Check that from collections import Counter and import string are at the top of your file.
  • If you see ModuleNotFoundError, make sure your import line says from collections import Counter (lowercase collections).
  • If Python reports a cached version, restart your terminal and reactivate the virtual environment with .venv\Scripts\activate.

Why is my find_word_frequencies function raising a NameError?

✔️ Awesome, I've got everything!

Great. Double check you've saved analyzer.py and all three test commands produced output without errors.

ⓧ I'd like to double check the full code

Here is the complete analyzer.py at this point:

"""Text File Analytics Engine - Analysis Module"""

from collections import Counter
import string


def load_file(filepath):
    """Load and return the contents of a text file."""
    try:
        with open(filepath, "r", encoding="utf-8") as f:
            content = f.read()
        return content
    except FileNotFoundError:
        print(f"Error: File '{filepath}' not found.")
        return None
    except Exception as e:
        print(f"Error reading file: {e}")
        return None


def count_basics(text):
    """Count basic text statistics."""
    words = text.split()
    lines = text.splitlines()
    characters = len(text)

    return {
        "word_count": len(words),
        "line_count": len(lines),
        "character_count": characters,
    }


def find_word_frequencies(text, top_n=10):
    """Find the most common words in the text."""
    words = text.lower().split()
    cleaned_words = [word.strip(string.punctuation) for word in words]
    cleaned_words = [word for word in cleaned_words if word]

    word_counts = Counter(cleaned_words)
    return word_counts.most_common(top_n)

You've built the foundation: your module can load a file safely, count its basic statistics, and identify the most frequent words. Next up, you'll add functions that solve more interesting problems like finding the longest word, detecting duplicate lines, and searching through text.

Build the Problem-Solving Functions

Your core analysis functions handle the basics: loading files, counting words, and finding frequencies. Now you need functions that solve harder problems.

In this step, you will write three functions that each tackle a distinct problem-solving pattern. Finding extremes uses max() and min() with custom keys. Detecting duplicates uses Counter on lines instead of words. Searching uses list comprehensions with conditional filtering.

In this step, get ready to:
  • Write a function to find the longest and shortest words plus average word length.
  • Write a function to detect duplicate lines in a file.
  • Write a function to search for specific words or phrases.
Write the find_extremes function

Finding the longest and shortest words requires comparing every word by length. Python's max() and min() functions accept a key parameter that tells them how to rank items.

  • In analyzer.py, scroll to the bottom of the file (below your find_word_frequencies function).
  • Add the following function:
def find_extremes(text):
    """Find longest, shortest words and average word length."""
    words = text.lower().split()
    cleaned_words = [word.strip(string.punctuation) for word in words]
    # Filter out single characters (punctuation leftovers)
    cleaned_words = [word for word in cleaned_words if len(word) > 1]

    # Handle edge case where text has no valid words
    if not cleaned_words:
        return {"longest": "", "shortest": "", "average_length": 0}

    # Use key=lambda to compare words by their length
    longest = max(cleaned_words, key=lambda w: len(w))
    shortest = min(cleaned_words, key=lambda w: len(w))
    # Calculate average using sum of all lengths divided by word count
    average_length = sum(len(word) for word in cleaned_words) / len(cleaned_words)

    return {
        "longest": longest,
        "shortest": shortest,
        "average_length": round(average_length, 2),
    }

What does this code do?

This function solves the "find the extreme" problem pattern. It cleans words the same way find_word_frequencies does, then applies three techniques:

  • max(cleaned_words, key=lambda w: len(w)) tells Python to compare words by their length, not alphabetically.
  • sum(len(word) for word in cleaned_words) adds up every word's length in one expression, then divides by the count to get the average.
  • The if not cleaned_words guard prevents a crash when max() receives an empty list.
  • Save analyzer.py.
  • You won't see a change yet. The next function builds on a different pattern and the full pipeline runs in Step 4.

Seeing a syntax error on save?

  • Check that the lambda expressions use a colon after the parameter: lambda w: len(w).
  • Make sure the return dictionary uses curly braces {} and each key-value pair ends with a comma.

Help me fix a syntax error in my find_extremes function

Write the find_duplicates function

Detecting duplicates is a counting problem. You already used Counter to count words. The same tool works on lines. Then a list comprehension filters for lines that appear more than once.

  • Below your find_extremes function in analyzer.py, add the following function:
def find_duplicates(text):
    """Find duplicate lines in the text."""
    lines = text.splitlines()
    # Reuse Counter to count how many times each line appears
    line_counts = Counter(lines)

    # Keep only lines that appear more than once
    duplicates = [
        (line, count) for line, count in line_counts.items() if count > 1
    ]
    # Sort by count descending so the most repeated lines come first
    duplicates = sorted(duplicates, key=lambda x: x[1], reverse=True)

    return duplicates

What does this code do?

This function solves the "find duplicates" pattern by reusing Counter on whole lines instead of individual words.

  • Counter(lines) creates a dictionary where each key is a line and each value is how many times it appeared.
  • The list comprehension filters for entries where count > 1, keeping only actual duplicates.
  • sorted(..., key=lambda x: x[1], reverse=True) orders the results so the most repeated line appears first. The x[1] grabs the count from each tuple.
  • Save analyzer.py.
  • You won't see a change yet. One more function to go before you can test everything.

Red underline on the list comprehension?

  • Make sure the for line, count in line_counts.items() part uses a comma between line and count to unpack both values from each tuple.
  • Check that the square brackets [] wrap the entire comprehension, not just the tuple.

Help me fix my find_duplicates list comprehension

Write the search_text function

Searching means filtering a collection down to items that match a condition. A list comprehension with an if clause does this in a single readable line.

  • Below your find_duplicates function in analyzer.py, add this final function:
def search_text(text, query):
    """Search for lines containing the query (case-insensitive)."""
    lines = text.splitlines()
    # Use .lower() on both sides so "Python" matches "python"
    matches = [line for line in lines if query.lower() in line.lower()]
    return matches

What does this code do?

This function solves the "search" pattern. The list comprehension checks every line for a match and keeps only the lines that contain the query.

  • query.lower() in line.lower() converts both strings to lowercase before comparing. This means searching for "python" will also find lines containing "Python".
  • The in keyword checks whether one string exists inside another. It works for single words and multi-word phrases.
  • Save analyzer.py.
  • Verify your module loads without errors by running this command in your terminal:
python -c "import analyzer; print('Module loaded successfully')"

You should see Module loaded successfully printed in the terminal. This confirms all six functions have valid syntax.

Seeing an import error or syntax error?

  • Make sure your terminal is in the same folder as analyzer.py. If not, cd into your project folder.
  • Check that the virtual environment is still active. You should see (.venv) at the start of your terminal prompt.
  • If the error mentions a specific line number, check that line for missing colons, unmatched brackets, or indentation issues.

Help me debug my analyzer.py module import error

✔️ Awesome, I've got everything!

Great. Double check you have saved analyzer.py and your module imports cleanly.

ⓧ I'd like to double check the full code

Here is the complete analyzer.py file with all six functions. Compare it against yours line by line.

"""Text File Analytics Engine - Analysis Module"""

from collections import Counter
import string


def load_file(filepath):
    """Load and return the contents of a text file."""
    try:
        with open(filepath, "r", encoding="utf-8") as f:
            content = f.read()
        return content
    except FileNotFoundError:
        print(f"Error: File '{filepath}' not found.")
        return None
    except Exception as e:
        print(f"Error reading file: {e}")
        return None


def count_basics(text):
    """Count basic text statistics."""
    words = text.split()
    lines = text.splitlines()
    characters = len(text)

    return {
        "word_count": len(words),
        "line_count": len(lines),
        "character_count": characters,
    }


def find_word_frequencies(text, top_n=10):
    """Find the most common words in the text."""
    words = text.lower().split()
    cleaned_words = [word.strip(string.punctuation) for word in words]
    cleaned_words = [word for word in cleaned_words if word]

    word_counts = Counter(cleaned_words)
    return word_counts.most_common(top_n)


def find_extremes(text):
    """Find longest, shortest words and average word length."""
    words = text.lower().split()
    cleaned_words = [word.strip(string.punctuation) for word in words]
    cleaned_words = [word for word in cleaned_words if len(word) > 1]

    if not cleaned_words:
        return {"longest": "", "shortest": "", "average_length": 0}

    longest = max(cleaned_words, key=lambda w: len(w))
    shortest = min(cleaned_words, key=lambda w: len(w))
    average_length = sum(len(word) for word in cleaned_words) / len(cleaned_words)

    return {
        "longest": longest,
        "shortest": shortest,
        "average_length": round(average_length, 2),
    }


def find_duplicates(text):
    """Find duplicate lines in the text."""
    lines = text.splitlines()
    line_counts = Counter(lines)

    duplicates = [
        (line, count) for line, count in line_counts.items() if count > 1
    ]
    duplicates = sorted(duplicates, key=lambda x: x[1], reverse=True)

    return duplicates


def search_text(text, query):
    """Search for lines containing the query (case-insensitive)."""
    lines = text.splitlines()
    matches = [line for line in lines if query.lower() in line.lower()]
    return matches

Your analyzer module now has six functions covering counting, frequency analysis, finding extremes, detecting duplicates, and searching. Next up, you will wire all of these together into a formatted report with proper error handling.

Wire It All Together with Error Handling

You now have six analysis functions that each solve one specific problem. On their own, these functions sit idle in analyzer.py doing nothing visible. They need a caller that orchestrates them.

In this step, you'll build the entry point that ties all six functions together into a formatted terminal report. You'll also add command-line argument handling so your tool gives helpful messages when something goes wrong.

In this step, get ready to:
  • Import the analyzer module and build the report formatter.
  • Add command-line argument handling with helpful usage messages.
  • Test the full pipeline with your sample file.
Import your analyzer module and build the report function

Your main.py file is the command-line entry point. It needs to import the analyzer module, call each function in sequence, and format the results into a readable report.

  • In VS Code, open main.py (it should be empty) and paste the following code:
"""Text File Analytics Engine - Main Entry Point"""

import sys
import analyzer


def generate_report(filepath):
    """Generate and display a full analysis report."""
    content = analyzer.load_file(filepath)
    if content is None:
        return

    print("=" * 60)
    print(f"  TEXT ANALYTICS REPORT: {filepath}")
    print("=" * 60)

    # Basic counts
    basics = analyzer.count_basics(content)
    print("\n--- Basic Statistics ---")
    print(f"  Words:      {basics['word_count']}")
    print(f"  Lines:      {basics['line_count']}")
    print(f"  Characters: {basics['character_count']}")

    # Word frequencies
    frequencies = analyzer.find_word_frequencies(content)
    print("\n--- Top 10 Most Frequent Words ---")
    for rank, (word, count) in enumerate(frequencies, 1):
        print(f"  {rank:>2}. {word:<15} ({count} occurrences)")

    # Extremes
    extremes = analyzer.find_extremes(content)
    print("\n--- Word Length Analysis ---")
    print(f"  Longest word:  {extremes['longest']}")
    print(f"  Shortest word: {extremes['shortest']}")
    print(f"  Average length: {extremes['average_length']} characters")

    # Duplicates
    duplicates = analyzer.find_duplicates(content)
    print("\n--- Duplicate Lines ---")
    if duplicates:
        for line, count in duplicates:
            print(f"  [{count}x] {line}")
    else:
        print("  No duplicate lines found.")

    # Search demo
    search_query = "python"
    matches = analyzer.search_text(content, search_query)
    print(f"\n--- Search Results for '{search_query}' ---")
    if matches:
        for match in matches:
            print(f"  > {match}")
    else:
        print(f"  No matches found for '{search_query}'.")

    print("\n" + "=" * 60)
    print("  Analysis complete.")
    print("=" * 60)

What does this code do?

The generate_report function is the orchestrator. It calls each of your six analyzer functions in sequence and formats their output into labelled sections.

  • import analyzer makes every function in analyzer.py available as analyzer.load_file(), analyzer.count_basics(), and so on.
  • import sys gives access to command-line arguments (you'll use this in the next substep).
  • The if content is None: return guard exits early if the file failed to load, so the rest of the function never runs on bad data.
  • f-strings with alignment specifiers like {rank:>2} and {word:<15} keep columns neatly aligned in the terminal output.
  • Save main.py.

You won't see any output yet. The next substep adds the entry point that calls this function from the command line.

Seeing a red underline on import analyzer?

Make sure analyzer.py is in the same folder as main.py. Python looks for modules in the same directory first.

Also confirm your virtual environment is still activated. If your terminal prompt no longer shows (.venv), reactivate it with .venv\Scripts\activate.

Still stuck? Help me fix my analyzer import in main.py

Add the main function with argument handling

A command-line tool needs to know which file to analyze. The sys.argv list holds whatever the user types after python main.py in the terminal. Your main() function reads that list and passes the filepath to your report generator.

  • In main.py, add the following function below generate_report:
def main():
    """Main entry point with command-line argument handling."""
    if len(sys.argv) < 2:
        print("Usage: python main.py <filepath>")
        print("Example: python main.py sample.txt")
        sys.exit(1)

    filepath = sys.argv[1]
    generate_report(filepath)

What does this code do?

  • sys.argv is a list where the first element is the script name (main.py) and any following elements are arguments the user typed.
  • if len(sys.argv) < 2 checks whether the user forgot to provide a filename. If so, it prints usage instructions and exits cleanly with sys.exit(1).
  • sys.argv[1] grabs the first argument after the script name. That becomes the filepath passed to generate_report.
  • Save main.py.

You won't see a change yet. The next substep adds the guard that triggers main() when you run the script.

Not sure where to place this function?

Add it directly below the closing line of generate_report. Leave one blank line between the end of generate_report and the def main(): line.

Need help? Where do I put my main function in main.py?

Add the entry point guard and test the pipeline

Python files can be run directly or imported as modules. The if __name__ == "__main__" guard ensures main() only runs when you execute the script directly, not when another file imports it.

  • At the very bottom of main.py, below the main() function, add:
if __name__ == "__main__":
    main()

Why use a __name__ guard?

When Python runs a file directly, it sets the special variable __name__ to "__main__". When a file is imported as a module, __name__ is set to the module's name instead.

This guard means your script runs its main() function when executed from the terminal, but stays quiet if another script imports it for reuse.

  • Save main.py.
  • Run the full pipeline by entering this command in your terminal:
python main.py sample.txt

You should see a formatted report filling your terminal with five sections: Basic Statistics, Top 10 Most Frequent Words, Word Length Analysis, Duplicate Lines, and Search Results for 'python'.

What should I see?

The report starts and ends with a row of = characters. Between them you'll see labelled sections separated by --- lines. The word frequencies show numbered rankings, and any duplicate lines from sample.txt appear with their repeat count.

  • Test the error handling by running the command without any filename:
python main.py

You should see the usage message: Usage: python main.py <filepath> followed by an example.

Seeing a ModuleNotFoundError?

This means Python cannot find analyzer.py. Confirm you are running the command from inside your project folder (the same folder that contains both main.py and analyzer.py).

Also check that your virtual environment is active. Your terminal prompt should show (.venv) at the beginning.

Still stuck? Help me fix my ModuleNotFoundError when running main.py

✔️ Awesome, I've got everything!

Great. Double check you've saved main.py and confirm your report printed all five sections to the terminal.

ⓧ I'd like to double check the full code

Here is the complete main.py file at this point:

"""Text File Analytics Engine - Main Entry Point"""

import sys
import analyzer


def generate_report(filepath):
    """Generate and display a full analysis report."""
    content = analyzer.load_file(filepath)
    if content is None:
        return

    print("=" * 60)
    print(f"  TEXT ANALYTICS REPORT: {filepath}")
    print("=" * 60)

    # Basic counts
    basics = analyzer.count_basics(content)
    print("\n--- Basic Statistics ---")
    print(f"  Words:      {basics['word_count']}")
    print(f"  Lines:      {basics['line_count']}")
    print(f"  Characters: {basics['character_count']}")

    # Word frequencies
    frequencies = analyzer.find_word_frequencies(content)
    print("\n--- Top 10 Most Frequent Words ---")
    for rank, (word, count) in enumerate(frequencies, 1):
        print(f"  {rank:>2}. {word:<15} ({count} occurrences)")

    # Extremes
    extremes = analyzer.find_extremes(content)
    print("\n--- Word Length Analysis ---")
    print(f"  Longest word:  {extremes['longest']}")
    print(f"  Shortest word: {extremes['shortest']}")
    print(f"  Average length: {extremes['average_length']} characters")

    # Duplicates
    duplicates = analyzer.find_duplicates(content)
    print("\n--- Duplicate Lines ---")
    if duplicates:
        for line, count in duplicates:
            print(f"  [{count}x] {line}")
    else:
        print("  No duplicate lines found.")

    # Search demo
    search_query = "python"
    matches = analyzer.search_text(content, search_query)
    print(f"\n--- Search Results for '{search_query}' ---")
    if matches:
        for match in matches:
            print(f"  > {match}")
    else:
        print(f"  No matches found for '{search_query}'.")

    print("\n" + "=" * 60)
    print("  Analysis complete.")
    print("=" * 60)


def main():
    """Main entry point with command-line argument handling."""
    if len(sys.argv) < 2:
        print("Usage: python main.py <filepath>")
        print("Example: python main.py sample.txt")
        sys.exit(1)

    filepath = sys.argv[1]
    generate_report(filepath)


if __name__ == "__main__":
    main()

Your text analytics engine is complete. You've built a real command-line tool that reads any text file, runs six different analyses, and presents a clean formatted report. Up next, a secret mission challenges you to push this tool even further.

Secret mission

Add File Comparison Mode

Your analytics engine works great for a single file. But what if you could compare two files side-by-side and instantly see how they differ in word count, complexity, and vocabulary? In this secret mission, you'll build a comparison mode that runs the full analysis on two files and displays the results in a formatted table with difference percentages.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your project files for future reference, or delete them to free up disk space.

Resources you used:

  • Project folder on your Desktop (contains main.py, analyzer.py, sample.txt, sample2.txt).
  • Virtual environment (.venv folder inside the project).

Keep everything running

No action needed. Choose this if you want to keep your text analytics tool for future use or to experiment with new features.

  • Your project folder stays on your Desktop.
  • Your virtual environment stays ready. Reactivate it anytime by opening a terminal in your project folder and running .venv\Scripts\activate.
  • Run python main.py sample.txt to use the tool again whenever you like.

Pause - I'll come back to this later

Deactivate the virtual environment to free up your terminal session. Your files stay on disk for when you return.

  • If your virtual environment is still active, deactivate it by running:
deactivate
  • Close VS Code if you are done for now.
  • When you come back, reactivate with .venv\Scripts\activate from inside your project folder.

Delete - I don't want to use this again

Remove all project files and the virtual environment entirely. This cannot be undone.

  • If your virtual environment is still active, deactivate it first by running:
deactivate
  • Navigate to your Desktop and delete the project folder.

macOS

  • Delete the project folder by running:
cd ~/Desktop
rm -rf text-analytics

Windows

  • Delete the project folder by running:
cd %USERPROFILE%\Desktop
rmdir /s /q text-analytics
  • Confirm the folder is gone by checking your Desktop. You should no longer see the project folder listed.

Nice Work!

Nice Work!

Nice work! You just built a complete command-line text analytics engine from scratch using nothing but Python's standard library.

You've learned how to:

  • Decompose a real problem into specific, solvable functions. You broke "analyze a file" into discrete responsibilities like counting, finding extremes, detecting duplicates, and searching.
  • Apply core problem-solving patterns using Counter for frequency counting, max()/min() with key=lambda for extremes, sets for duplicates, and list comprehensions for filtering.
  • Structure a project professionally with virtual environments, modules, error handling, and the if __name__ == "__main__" guard.
  • Secret Mission: Added a file comparison mode that analyzes two files side-by-side and highlights differences with percentage calculations.

Ready to quiz yourself?