Build an AI Content Idea Pipeline

Crawl RSS feeds and summarize tech articles in Vietnamese with Gemini AI.

Introduction

30 Second Summary

If you create tech content, you know the drill. You spend hours browsing blogs, skimming articles, and trying to decide which topics are worth turning into videos or posts.

In this project, you will build a Python script that automatically crawls tech blogs, uses Google Gemini AI to summarize each article in Vietnamese, and saves everything to an Excel file where you approve ideas with a single keystroke.

What You'll Build

You open Excel tomorrow morning and see a curated list of fresh tech articles, already summarized in Vietnamese with categories. You mark two ideas with "Y", close the file, and the next run promotes them to your Masterfile with unique Content IDs.

By the end of this project, you'll have:

  • A daily auto-crawling script that pulls fresh articles from Viblo, Dev.to, and Medium into your Excel file without you lifting a finger.
  • AI-generated Vietnamese summaries and categories for every article so you can review 20+ ideas in seconds instead of minutes.
  • A Content ID system that automatically promotes approved ideas to your Masterfile sheet, ready for your production workflow.
  • Secret Mission: AI relevance scoring that rates each idea 1-10 and auto-highlights the best ones so you focus on winners instantly.

Are there any prerequisites?

None. This guide walks you through installing Python, setting up VS Code, and getting your free Gemini API key from scratch. You just need a Windows computer and a Google account.

Before We Start

Before We Start...

Before we dive in, let's lock in what you're about to build and why it matters.

This is your moment to commit to the problem you're solving: manually scrolling through tech blogs every day to find content ideas is slow and easy to forget.

Crawl RSS Feeds

Your environment is set up and ready to go. Now it is time to prove that your script can reach real tech blogs and pull article data into your terminal.

Before building the full pipeline with AI summaries and Excel, you need a working foundation. In this step, you will connect to three RSS feeds and see live article titles flowing through your script.

In this step, get ready to:
  • Define RSS feed URLs for Viblo, Dev.to, and Medium.
  • Build a crawler function that parses each feed with feedparser.
  • Run the script and see 20+ article titles printed in the terminal.
Set up feed configuration

RSS feeds are structured data streams that blogs publish so programs can read their latest articles automatically. You will point your script at three feeds covering Vietnamese tech, AI, and general artificial intelligence content.

  • In VS Code, create a new file called content_pipeline.py inside your content-pipeline folder.
  • Add the imports and feed URL configuration by pasting this code:
import feedparser
from datetime import datetime
import time

# --- Configuration ---
RSS_FEEDS = [
    "https://viblo.asia/rss",
    "https://dev.to/feed/tag/ai",
    "https://medium.com/feed/tag/artificial-intelligence",
]

What does this code do?

RSS_FEEDS is a list of three URLs, each pointing to a different tech blog's RSS feed.

  • viblo.asia/rss pulls the newest articles from Viblo, a Vietnamese developer community.
  • dev.to/feed/tag/ai pulls articles tagged with AI from Dev.to.
  • medium.com/feed/tag/artificial-intelligence pulls Medium articles in the artificial intelligence topic.

The datetime and time imports will be used in later steps when you add timestamps and rate limiting.

  • Save content_pipeline.py.

You will not run the script yet. The next chunk adds the function that actually fetches articles from these URLs.

Build the feed crawler and run it

feedparser handles all the complexity of reading RSS data. You call it with a URL and it returns a structured object containing every article in the feed.

  • In content_pipeline.py, add the following code below the RSS_FEEDS list:
def crawl_feeds():
    articles = []
    for feed_url in RSS_FEEDS:
        feed = feedparser.parse(feed_url)
        for entry in feed.entries:
            articles.append({
                "title": entry.get("title", ""),
                "link": entry.get("link", ""),
                "published": entry.get("published", ""),
                "source": feed.feed.get("title", feed_url),
                "description": entry.get("description", "")[:500],
            })
    return articles


if __name__ == "__main__":
    articles = crawl_feeds()
    for a in articles:
        print(a["title"], "|", a["link"], "|", a["source"])

What does this code do?

crawl_feeds() loops through each URL in RSS_FEEDS and parses the feed using feedparser.parse(). For each article entry, it extracts the title, link, publish date, source name, and a trimmed description.

The if __name__ == "__main__" block at the bottom runs when you execute the script directly. It calls the crawler and prints each article's title, link, and source separated by | characters.

  • Save content_pipeline.py.

Before you run this, how many articles do you think the script will find across three feeds?

  • Run your script by entering this command in the terminal:
python content_pipeline.py

You should see 20+ lines printed in your terminal, each showing an article title, its link, and which blog it came from. The output looks something like:

Huong dan su dung Docker | https://viblo.asia/p/... | Viblo
Building AI Agents | https://dev.to/... | DEV Community
Machine Learning in 2026 | https://medium.com/... | Artificial Intelligence

Not seeing any output?

  • Make sure you saved content_pipeline.py before running the command.
  • Check that the RSS_FEEDS URLs are copied exactly as shown, including the full paths like /feed/tag/ai.
  • If you see ModuleNotFoundError: No module named 'feedparser', run pip install feedparser in your terminal and try again.

help me debug why my content_pipeline.py script is not printing any articles

✔️ Awesome, I've got everything!

Great. Double check you have saved your file and your terminal shows article titles from all three sources.

ⓧ I'd like to double check the full code

Here is the complete content_pipeline.py file at this point:

import feedparser
from datetime import datetime
import time

# --- Configuration ---
RSS_FEEDS = [
    "https://viblo.asia/rss",
    "https://dev.to/feed/tag/ai",
    "https://medium.com/feed/tag/artificial-intelligence",
]


def crawl_feeds():
    articles = []
    for feed_url in RSS_FEEDS:
        feed = feedparser.parse(feed_url)
        for entry in feed.entries:
            articles.append({
                "title": entry.get("title", ""),
                "link": entry.get("link", ""),
                "published": entry.get("published", ""),
                "source": feed.feed.get("title", feed_url),
                "description": entry.get("description", "")[:500],
            })
    return articles


if __name__ == "__main__":
    articles = crawl_feeds()
    for a in articles:
        print(a["title"], "|", a["link"], "|", a["source"])

Your script is now pulling live data from the internet. Next up, you will save these articles to an Excel file so you can review, filter, and approve ideas without losing them when the terminal closes.

Save to Excel Raw Sheet

Your script pulls live articles from three RSS feeds. That is great progress. But terminal output vanishes the moment you close the window.

You need a persistent home for your articles. In this step, you will save crawled data into an Excel file with two sheets: Raw for incoming ideas and Masterfile for approved content. By the end, you will see exactly why AI summarization is the next piece of the puzzle.

In this step, get ready to:
  • Create an Excel workbook with Raw and Masterfile sheets (complete with bold headers).
  • Add duplicate detection so re-running the script never adds the same article twice.
  • Run the pipeline end-to-end and open the Excel file to see your data.
Set up Excel workbook creation

Before writing any articles to disk, the script needs new imports and a function that creates (or loads) the Excel file with the correct sheet structure.

  • In content_pipeline.py, find the imports section at the very top of your file.
  • Find these lines:
import feedparser
from datetime import datetime
import time
  • Replace those three lines and the configuration block below them with the following updated imports and configuration:
import feedparser
from openpyxl import Workbook, load_workbook
from openpyxl.styles import Font
from datetime import datetime
import os
import time

# --- Configuration ---
EXCEL_FILE = "content_pipeline.xlsx"

RSS_FEEDS = [
    "https://viblo.asia/rss",
    "https://dev.to/feed/tag/ai",
    "https://medium.com/feed/tag/artificial-intelligence",
]

What changed here?

  • openpyxl gives you the ability to create and edit Excel files directly from Python.
  • Font lets you bold the header row so columns are easy to identify.
  • os checks whether the Excel file already exists before creating a new one.
  • EXCEL_FILE stores the filename in one place so every function references the same file.
  • Below the crawl_feeds() function (after its return articles line), add this new function that creates or loads the workbook:
def get_or_create_workbook():
    if os.path.exists(EXCEL_FILE):
        wb = load_workbook(EXCEL_FILE)
    else:
        wb = Workbook()
        ws_raw = wb.active
        ws_raw.title = "Raw"
        raw_headers = [
            "Title", "Link", "Source", "Published",
            "Summary", "Category", "Approved", "Format",
        ]
        ws_raw.append(raw_headers)
        for cell in ws_raw[1]:
            cell.font = Font(bold=True)

        ws_master = wb.create_sheet("Masterfile")
        master_headers = [
            "Content ID", "Title", "Format", "Source", "Link",
            "Summary", "Category", "Date Approved",
            "Script", "Schema", "Assets", "Status",
        ]
        ws_master.append(master_headers)
        for cell in ws_master[1]:
            cell.font = Font(bold=True)

        wb.save(EXCEL_FILE)
    return wb

What does this function do?

If content_pipeline.xlsx already exists, it loads the file so you can add new rows without losing old data. If the file does not exist yet, it creates a brand new workbook with two sheets.

  • The Raw sheet holds every crawled article. The Summary and Category columns are empty for now. AI fills them in the next step.
  • The Masterfile sheet is your approved-ideas registry. It stays empty until you mark articles with "Y" later.
  • Save content_pipeline.py.
  • Verify the workbook creation works by running this command in your terminal:
python -c "from content_pipeline import get_or_create_workbook; get_or_create_workbook()"
  • Confirm the file was created by running:
ls

You should see content_pipeline.xlsx listed alongside content_pipeline.py.

Don't see the .xlsx file?

  • Check for typos in the import lines. A missing comma between Workbook and load_workbook will cause an ImportError.
  • Make sure you saved the file before running the command.
  • Confirm your terminal is inside the content-pipeline folder. Run cd ~/Desktop/content-pipeline if needed.

Help me debug my workbook creation

Add duplicate detection and article saving

Every time the script runs, it should add only new articles. Without duplicate detection, re-running would fill Excel with repeated entries.

  • Below get_or_create_workbook(), add these two functions that handle duplicate checking and writing articles to the Raw sheet:
def get_existing_links(ws):
    links = set()
    for row in ws.iter_rows(min_row=2, values_only=True):
        if row[1]:
            links.add(row[1])
    return links


def add_new_articles_to_raw(wb, articles):
    ws_raw = wb["Raw"]
    existing_links = get_existing_links(ws_raw)
    new_count = 0

    for article in articles:
        if article["link"] in existing_links:
            continue

        ws_raw.append([
            article["title"],
            article["link"],
            article["source"],
            article["published"],
            "",
            "",
            "",
            "",
        ])
        new_count += 1

    return new_count

What do these functions do?

  • get_existing_links() reads column B (the Link column) of every existing row and stores them in a set. Checking membership in a set is instant, even with hundreds of links.
  • add_new_articles_to_raw() loops through crawled articles. If the link already exists, it skips that article (continue). Otherwise it appends a new row. The four empty strings are placeholders for Summary, Category, Approved, and Format.
  • Save content_pipeline.py.

Seeing a NameError for get_existing_links?

Make sure get_existing_links() is defined above add_new_articles_to_raw() in your file. Python reads top to bottom, so a function must be defined before another function calls it.

Help me fix a NameError in my pipeline code

Wire up main() and run the full pipeline

The individual pieces are ready. Now you need a main() function that orchestrates the full flow: crawl, save to Excel, and report progress.

  • In content_pipeline.py, scroll to the bottom and find the existing if __name__ block:
if __name__ == "__main__":
    articles = crawl_feeds()
    for a in articles:
        print(a["title"], "|", a["link"], "|", a["source"])
  • Delete that entire block and replace it with this main() function and call:
def main():
    print("=" * 50)
    print("CONTENT PIPELINE - Auto Run")
    print(f"Time: {datetime.now().strftime('%Y-%m-%d %H:%M')}")
    print("=" * 50)

    print("\n[1/2] Crawling RSS feeds...")
    articles = crawl_feeds()
    print(f"      Found {len(articles)} articles total")

    print("\n[2/2] Adding new articles to Raw sheet...")
    wb = get_or_create_workbook()
    new_count = add_new_articles_to_raw(wb, articles)
    print(f"      Added {new_count} new articles")

    wb.save(EXCEL_FILE)
    print(f"\nDone! File saved: {EXCEL_FILE}")
    print("=" * 50)


if __name__ == "__main__":
    main()

What does main() do?

  • Step [1/2] crawls all three RSS feeds and collects article data into a list.
  • Step [2/2] creates (or loads) the workbook, then writes only new articles to the Raw sheet.
  • Finally it saves the workbook and prints a summary showing how many articles were added.
  • Save content_pipeline.py.
  • Before you run the script, predict: how many articles do you think will be added? Will it be the same count you saw in Step 2, or different?
  • Run the full pipeline:
python content_pipeline.py

You should see output like this:

==================================================
CONTENT PIPELINE - Auto Run
Time: 2026-07-25 14:30
==================================================

[1/2] Crawling RSS feeds...
      Found 45 articles total

[2/2] Adding new articles to Raw sheet...
      Added 45 new articles

Done! File saved: content_pipeline.xlsx
==================================================
  • Now open content_pipeline.xlsx in Excel (or double-click it in File Explorer).
  • Click the Raw sheet tab at the bottom.

You should see rows of articles with titles, links, sources, and published dates filled in. The Summary and Category columns are completely empty.

Try to judge which articles are worth turning into content. With only a title and a link, you would need to open every single article in your browser to understand what it covers. That manual review is exactly the bottleneck AI will solve next.

Why are Summary and Category empty?

Those columns are reserved for Gemini AI output. In the next step, you will send each article's raw description to Gemini and get back a two-sentence Vietnamese summary plus a category label. After that, scanning 40+ articles takes seconds instead of hours.

  • Run the script a second time to confirm duplicate detection works:
python content_pipeline.py

This time you should see Added 0 new articles because every link already exists in the Raw sheet.

Still seeing duplicate articles added?

  • Make sure get_existing_links() reads from column index row[1] (the Link column, which is the second column starting from index 0).
  • Verify you saved the Excel file before the second run. If the first run crashed before wb.save(), no data was persisted.
  • Check that EXCEL_FILE at the top of your script matches the actual filename on disk.

Help me fix duplicate detection in my pipeline

✔️ Awesome, I've got everything!

Great. Double check you have saved content_pipeline.py and that running the script a second time shows Added 0 new articles.

ⓧ I'd like to double check the full code

Here is the complete content_pipeline.py at this point. Compare your file against it:

  • Scroll up and confirm you have: the imports and config block, the crawl_feeds() function, the get_or_create_workbook() function, the get_existing_links() function, the add_new_articles_to_raw() function, and main() with if __name__ == "__main__": main() at the bottom.

Imports and configuration:

import feedparser
from openpyxl import Workbook, load_workbook
from openpyxl.styles import Font
from datetime import datetime
import os
import time

# --- Configuration ---
EXCEL_FILE = "content_pipeline.xlsx"

RSS_FEEDS = [
    "https://viblo.asia/rss",
    "https://dev.to/feed/tag/ai",
    "https://medium.com/feed/tag/artificial-intelligence",
]

The crawl_feeds() function (unchanged from Step 2):

def crawl_feeds():
    articles = []
    for feed_url in RSS_FEEDS:
        feed = feedparser.parse(feed_url)
        for entry in feed.entries:
            articles.append({
                "title": entry.get("title", ""),
                "link": entry.get("link", ""),
                "published": entry.get("published", ""),
                "source": feed.feed.get("title", feed_url),
                "description": entry.get("description", "")[:500],
            })
    return articles

The get_or_create_workbook() function:

def get_or_create_workbook():
    if os.path.exists(EXCEL_FILE):
        wb = load_workbook(EXCEL_FILE)
    else:
        wb = Workbook()
        ws_raw = wb.active
        ws_raw.title = "Raw"
        raw_headers = [
            "Title", "Link", "Source", "Published",
            "Summary", "Category", "Approved", "Format",
        ]
        ws_raw.append(raw_headers)
        for cell in ws_raw[1]:
            cell.font = Font(bold=True)

        ws_master = wb.create_sheet("Masterfile")
        master_headers = [
            "Content ID", "Title", "Format", "Source", "Link",
            "Summary", "Category", "Date Approved",
            "Script", "Schema", "Assets", "Status",
        ]
        ws_master.append(master_headers)
        for cell in ws_master[1]:
            cell.font = Font(bold=True)

        wb.save(EXCEL_FILE)
    return wb

The duplicate detection and article saving functions:

def get_existing_links(ws):
    links = set()
    for row in ws.iter_rows(min_row=2, values_only=True):
        if row[1]:
            links.add(row[1])
    return links


def add_new_articles_to_raw(wb, articles):
    ws_raw = wb["Raw"]
    existing_links = get_existing_links(ws_raw)
    new_count = 0

    for article in articles:
        if article["link"] in existing_links:
            continue

        ws_raw.append([
            article["title"],
            article["link"],
            article["source"],
            article["published"],
            "",
            "",
            "",
            "",
        ])
        new_count += 1

    return new_count

The main() function and entry point:

def main():
    print("=" * 50)
    print("CONTENT PIPELINE - Auto Run")
    print(f"Time: {datetime.now().strftime('%Y-%m-%d %H:%M')}")
    print("=" * 50)

    print("\n[1/2] Crawling RSS feeds...")
    articles = crawl_feeds()
    print(f"      Found {len(articles)} articles total")

    print("\n[2/2] Adding new articles to Raw sheet...")
    wb = get_or_create_workbook()
    new_count = add_new_articles_to_raw(wb, articles)
    print(f"      Added {new_count} new articles")

    wb.save(EXCEL_FILE)
    print(f"\nDone! File saved: {EXCEL_FILE}")
    print("=" * 50)


if __name__ == "__main__":
    main()

Your articles now live in a permanent Excel file. But reviewing them still means clicking through each link one by one. Next up, you will connect Gemini AI to summarize every article in Vietnamese and assign a category, so you can scan 40+ ideas in seconds.

Add Gemini AI Summarization

In the last step, you crawled RSS feeds and saved articles to Excel. But when you opened the file, you noticed a problem: the descriptions are in English, contain HTML fragments, and are too long to scan quickly.

In this step, you'll connect Google Gemini AI to your pipeline. For each new article, Gemini generates a 2-sentence Vietnamese summary and assigns a category so you can decide in seconds whether an idea is worth pursuing.

In this step, get ready to:
  • Connect to the Gemini API with your API key.
  • Write a function that sends articles to Gemini for Vietnamese summarization.
  • Integrate AI into the crawl loop so every new article gets a summary and category.
Connect to the Gemini API

Before Gemini can summarize anything, your script needs to authenticate with the API using the key you created in Step 1.

  • In VS Code, open content_pipeline.py.
  • Find the import lines at the top of the file. Add the Gemini import on a new line after from openpyxl.styles import Font:
from google import genai
  • Next, find the # --- Configuration --- section. Add your API key on a new line directly after EXCEL_FILE = "content_pipeline.xlsx":
GEMINI_API_KEY = "your-api-key-here"
  • Replace your-api-key-here with the actual API key you copied from Google AI Studio in Step 1.
  • Now scroll down past the RSS_FEEDS list. Add the Gemini client initialization on a new line after the closing ] of the list:
# --- Initialize Gemini ---
client = genai.Client(api_key=GEMINI_API_KEY)

What does this code do?

These three additions wire your script to the Gemini API. The genai.Client object handles authentication and all communication with Google's servers. You pass your API key directly so the script works even when run by Task Scheduler later (no environment variable setup needed).

  • Save content_pipeline.py.
  • Confirm there are no red squiggly lines under the new import or the client line in VS Code.

Seeing a red underline on the import?

Make sure you installed the package as google-genai (not google-generativeai). The older package is deprecated. Run this in your terminal to fix it:

pip install google-genai

help me fix a red underline on my google genai import

Write the summarization function

The core of this step is a function that takes an article's title and description, sends them to Gemini with a Vietnamese summarization prompt, and returns the AI's response.

  • In content_pipeline.py, scroll down to the crawl_feeds() function. Add the following new function on a blank line after crawl_feeds() ends (after its return articles line):
def summarize_article(title, description):
    prompt = (
        "Tom tat bai viet sau bang tieng Viet trong 2 cau ngan gon. "
        "Sau do phan loai vao MOT trong cac nhom: AI, Data, Web Dev, DevOps, Career.\n\n"
        f"Tieu de: {title}\n"
        f"Mo ta: {description}\n\n"
        "Tra loi theo format:\n"
        "Tom tat: [tom tat 2 cau]\n"
        "Nhom: [nhom]"
    )
    response = client.models.generate_content(
        model="gemini-2.0-flash",
        contents=prompt,
    )
    return response.text

What does this function do?

The prompt instructs Gemini to summarize the article in 2 short Vietnamese sentences, then classify it into exactly one category: AI, Data, Web Dev, DevOps, or Career.

The strict response format (Tom tat: ... and Nhom: ...) makes it easy to parse the AI's answer with simple string splitting later. response.text contains the raw text Gemini returns.

  • Save content_pipeline.py.
Integrate AI into the crawl loop

The function exists, but nothing calls it yet. You need to update add_new_articles_to_raw() so it calls Gemini for every new article before writing to Excel.

  • In content_pipeline.py, find the add_new_articles_to_raw function. Locate the for article in articles: loop.
  • Find the existing code inside the loop that writes empty strings to Excel:
        if article["link"] in existing_links:
            continue

        ws_raw.append([
            article["title"],
            article["link"],
            article["source"],
            article["published"],
            "",
            "",
            "",
            "",
        ])
        new_count += 1
  • Replace that entire block (from if article["link"] down to new_count += 1) with this code:
        if article["link"] in existing_links:
            continue

        # Call Gemini to summarize and categorize
        summary = ""
        category = ""
        try:
            ai_response = summarize_article(
                article["title"], article["description"]
            )
            for line in ai_response.split("\n"):
                if line.lower().startswith("tom tat:"):
                    summary = line.split(":", 1)[1].strip()
                elif line.lower().startswith("nhom:"):
                    category = line.split(":", 1)[1].strip()
        except Exception as e:
            summary = f"Error: {e}"

        ws_raw.append([
            article["title"],
            article["link"],
            article["source"],
            article["published"],
            summary,
            category,
            "",
            "",
        ])
        new_count += 1
        time.sleep(4)

What does this code do?

  • For each new article, it calls summarize_article() to get Gemini's response.
  • It parses the response by splitting on newlines and looking for lines starting with tom tat: and nhom: to extract the summary and category.
  • The try/except block handles API errors gracefully. If a call fails, the script writes the error message and moves on instead of crashing.
  • time.sleep(4) pauses 4 seconds between API calls to stay within Gemini's free tier rate limit of 15 requests per minute.
  • Save content_pipeline.py.

Not sure you got the indentation right?

The entire replaced block should be inside the for article in articles: loop, indented with 8 spaces (two levels). If Python complains about IndentationError, check that the if article["link"] line starts at the same column as the rest of the loop body.

help me fix an IndentationError in my add_new_articles_to_raw function

Before you run the script, what do you think will happen to the articles that are already in your Excel file from the last step?

  • Run the updated script in your terminal:
python content_pipeline.py

The script will take longer this time because it pauses 4 seconds between each Gemini API call. You should see the familiar output showing articles found and added.

  • Open content_pipeline.xlsx in Excel or any spreadsheet app.
  • Look at the Summary and Category columns. Old rows (from the previous run) still have empty summaries. New rows have clean Vietnamese summaries and a category like AI, Data, Web Dev, DevOps, or Career.

Why are old rows still empty?

Your duplicate detection skips any article whose link already exists in the Raw sheet. Only truly new articles go through the Gemini summarization. This is by design. It prevents re-processing the same articles and burning through your free API quota.

Seeing 'Added 0 new articles'?

If no new articles were published since your last run, the duplicate check skips everything. Delete content_pipeline.xlsx and run the script again so all articles are processed fresh with AI summaries.

help me troubleshoot why my script says 0 new articles added

✔️ Awesome, I've got everything!

Great. Double check you have saved content_pipeline.py and that your Excel file shows Vietnamese summaries for new articles.

ⓧ I'd like to double check the full code

Your content_pipeline.py should contain the following sections in order. Scroll through your file and confirm each is present:

  • Imports: feedparser, openpyxl, Font, genai, datetime, os, time.
  • Configuration: EXCEL_FILE, GEMINI_API_KEY, RSS_FEEDS list.
  • Gemini client: client = genai.Client(api_key=GEMINI_API_KEY).
  • crawl_feeds() function (unchanged from Step 3).
  • summarize_article(title, description) function with the Vietnamese prompt.
  • get_or_create_workbook() function (unchanged).
  • get_existing_links(ws) function (unchanged).
  • add_new_articles_to_raw(wb, articles) function with the Gemini call, response parsing, error handling, and time.sleep(4).
  • main() function and if __name__ == "__main__": block.

Your pipeline now transforms raw, unreadable article data into clean, categorized Vietnamese summaries. Next up, you'll build the approval workflow: mark ideas with "Y" in Excel and have the script automatically generate Content IDs and push them to the Masterfile.

Sync to Masterfile and Automate

Your pipeline crawls feeds and summarizes them beautifully. But approved ideas still sit in the Raw sheet forever, and you have to remember to run the script manually every day.

In this step, you will build the sync logic that moves approved items into a Masterfile with unique Content IDs. Then you will automate the entire pipeline with Windows Task Scheduler so it runs daily without you lifting a finger.

In this step, get ready to:
  • Build a sync function that generates Content IDs and pushes approved articles to the Masterfile sheet.
  • Create a batch file and schedule it with Task Scheduler for daily automation.
  • Test the full end-to-end pipeline from approval to Masterfile.
Build the sync function (part 1: collect existing data)

The sync function needs to know which articles are already in Masterfile so it does not duplicate them. It also needs to count how many Content IDs were generated today so the next ID gets the right sequence number.

  • In content_pipeline.py, find the end of the add_new_articles_to_raw function (after return new_count).
  • Add the first half of the sync function by pasting this code below it:
def sync_approved_to_masterfile(wb):
    ws_raw = wb["Raw"]
    ws_master = wb["Masterfile"]

    # Collect links already in Masterfile to avoid duplicates
    existing_links_master = set()
    existing_ids = set()
    for row in ws_master.iter_rows(min_row=2, values_only=True):
        if row[0]:
            existing_ids.add(row[0])
        if row[4]:
            existing_links_master.add(row[4])

    # Count how many IDs already exist for today
    today_str = datetime.now().strftime("%y%m%d")
    today_count = sum(
        1 for cid in existing_ids if cid.startswith(f"PID-{today_str}")
    )

What does this code do?

This sets up duplicate detection for the Masterfile. It scans existing rows to build a set of links already synced and a set of Content IDs already generated.

The today_count variable counts how many PID-YYMMDD IDs exist for today. This ensures the next ID gets the correct sequence number (e.g. -03 if two already exist).

Build the sync function (part 2: process approved rows)

Now the function needs to loop through Raw, find rows marked as approved, and append them to Masterfile with a generated Content ID.

  • Directly below the code you just pasted (still inside the sync_approved_to_masterfile function), add this continuation:
    synced_count = 0
    for row in ws_raw.iter_rows(min_row=2, values_only=True):
        title = row[0]
        link = row[1]
        source = row[2]
        summary = row[4]
        category = row[5]
        approved = str(row[6]).strip().upper() if row[6] else ""
        format_type = str(row[7]).strip() if row[7] else ""

        # Only sync rows marked Y with a format chosen
        if approved == "Y" and format_type and link not in existing_links_master:
            today_count += 1
            content_id = f"PID-{today_str}-{today_count:02d}"

            ws_master.append([
                content_id,
                title,
                format_type,
                source,
                link,
                summary,
                category,
                datetime.now().strftime("%Y-%m-%d"),
                "",
                "",
                "",
                "Idea Approved",
            ])
            existing_links_master.add(link)
            synced_count += 1

    return synced_count

What does this code do?

This loop checks every row in Raw. If the Approved column is Y and the Format column is filled, and the link is not already in Masterfile, it generates a Content ID using the pattern PID-YYMMDD-XX.

The 12 columns appended match the Masterfile headers: Content ID, Title, Format, Source, Link, Summary, Category, Date Approved, Script, Schema, Assets, and Status. The last four start empty except Status which is set to Idea Approved.

  • Save content_pipeline.py.
Update main() to call the sync function

The sync function exists but main() does not call it yet. You need to add it as step 3 of the pipeline and update the progress labels.

  • In content_pipeline.py, find the main() function.
  • Replace the entire main() function with this updated version:
def main():
    print("=" * 50)
    print("CONTENT PIPELINE - Auto Run")
    print(f"Time: {datetime.now().strftime('%Y-%m-%d %H:%M')}")
    print("=" * 50)

    print("\n[1/3] Crawling RSS feeds...")
    articles = crawl_feeds()
    print(f"      Found {len(articles)} articles total")

    print("\n[2/3] Adding new articles to Raw sheet...")
    wb = get_or_create_workbook()
    new_count = add_new_articles_to_raw(wb, articles)
    print(f"      Added {new_count} new articles")

    print("\n[3/3] Syncing approved items to Masterfile...")
    synced_count = sync_approved_to_masterfile(wb)
    print(f"      Synced {synced_count} items to Masterfile")

    wb.save(EXCEL_FILE)
    print(f"\nDone! File saved: {EXCEL_FILE}")
    print("=" * 50)

What changed in main()?

The step labels changed from [1/2] and [2/2] to [1/3], [2/3], and [3/3]. The new step 3 calls sync_approved_to_masterfile(wb) and prints how many items were synced.

  • Save content_pipeline.py.

Now test the sync by marking some articles as approved. Before you run the script, think about what you expect to see. Will items appear in Masterfile with Content IDs?

  • Open content_pipeline.xlsx in Excel (or any spreadsheet app).
  • In the Raw sheet, find 2 articles you like.
  • Type Y in the Approved column (column G) for each.
  • Type video in the Format column (column H) for each.
  • Save and close the Excel file.
  • Back in your VS Code terminal, run the script:
python content_pipeline.py

You should see the pipeline run all 3 steps. The output ends with a line showing how many items were synced to Masterfile.

  • Open content_pipeline.xlsx again and switch to the Masterfile sheet.

You should see your 2 approved articles with Content IDs like PID-260725-01 and PID-260725-02 (the date portion reflects today).

Masterfile sheet is still empty?

  • Make sure you saved and closed Excel before running the script. Python cannot write to a file that Excel has locked.
  • Check that you typed Y (uppercase) in the Approved column. The code checks for .upper() but the cell must not be empty.
  • Confirm the Format column has a value (e.g. video). Both Approved and Format must be filled for sync to trigger.

Still stuck? Help me debug why my approved articles are not syncing to Masterfile.

Create the batch file for automation

Windows Task Scheduler cannot run Python scripts directly. It needs a batch file that sets the correct working directory and then calls Python.

  • In VS Code, create a new file called run_pipeline.bat inside your content-pipeline folder.
  • Paste this code into the file:
@echo off
cd /d "%~dp0"
python content_pipeline.py

What does this batch file do?

The @echo off line hides the commands from printing in the console window. The cd /d "%~dp0" changes the working directory to wherever the batch file lives. This ensures Python can find content_pipeline.xlsx in the same folder regardless of where Task Scheduler launches from.

  • Save run_pipeline.bat.
  • Test the batch file by double-clicking it in File Explorer. A terminal window should flash open, run the pipeline, and close.
  • Open content_pipeline.xlsx to confirm the script ran (check the Raw sheet for any new articles or confirm no errors occurred).

Seeing 'python is not recognized'?

This means Python is not in your system PATH. Reinstall Python from python.org and make sure to check the Add to PATH checkbox during installation.

Need more help? Why is my batch file saying python is not recognized?

Set up Task Scheduler for daily automation

The batch file works when you run it manually. Now configure Windows to run it automatically every day.

  • Press the Windows key to open your search bar.
  • Type taskschd.msc and press Enter to open Task Scheduler.
  • In the right-hand Actions pane, click Create Basic Task.
  • Enter Content Pipeline Daily as the task name, then click Next.
  • Select Daily as the trigger, then click Next.
  • Set the start time to a time when your computer is usually on (e.g. 8:00 AM), leave recurrence at 1 day, then click Next.
  • Select Start a program as the action, then click Next.
  • Click Browse, navigate to your content-pipeline folder on the Desktop, select run_pipeline.bat, and click Open.
  • Click Next, review the summary, then click Finish.

What is Task Scheduler?

Task Scheduler is a built-in Windows tool that runs programs or scripts on a schedule. Your task is now configured to launch run_pipeline.bat once per day at the time you chose. Every morning, fresh articles will appear in your Raw sheet automatically.

Test the scheduled task

You do not need to wait until tomorrow to confirm the task works. Task Scheduler lets you trigger it manually right now.

  • In Task Scheduler, click Task Scheduler Library in the left pane.
  • Find Content Pipeline Daily in the task list.
  • Right-click the task and select Run.

A brief terminal window flashes while the script runs. After it closes, do you think new articles appeared in Raw and your previously-approved items are still in Masterfile?

  • Open content_pipeline.xlsx.
  • Check the Raw sheet. If any new articles were published on the feeds since your last run, you will see new rows with Vietnamese summaries.
  • Check the Masterfile sheet. Your previously-approved items are still there with their Content IDs intact.

Task ran but nothing changed in Excel?

  • Make sure Excel is closed before triggering the task. If Excel has the file open, Python cannot write to it.
  • Check that the path in Task Scheduler points to run_pipeline.bat inside your content-pipeline folder on the Desktop.
  • Try running run_pipeline.bat by double-clicking it directly. If it works manually but not from Task Scheduler, the path in the task is incorrect.

Still having issues? Help me troubleshoot why my Task Scheduler task is not running my Python pipeline correctly.

✔️ Awesome, I've got everything!

Great. Double check that you have saved both content_pipeline.py and run_pipeline.bat.

ⓧ I'd like to double check the full code

Your content_pipeline.py is now longer than a quick glance can cover. Scroll through and confirm these sections are present in order:

  • Imports (feedparser, openpyxl, google.genai, datetime, os, time).
  • Configuration section (EXCEL_FILE, GEMINI_API_KEY, RSS_FEEDS, Gemini client).
  • The crawl_feeds() function.
  • The summarize_article() function.
  • The get_or_create_workbook() function.
  • The get_existing_links() function.
  • The add_new_articles_to_raw() function.
  • The sync_approved_to_masterfile() function (with duplicate detection and Content ID generation).
  • The updated main() function showing [1/3], [2/3], [3/3] with the sync call.
  • The if __name__ == "__main__": block at the bottom.

Your run_pipeline.bat should contain exactly 3 lines: @echo off, cd /d "%~dp0", and python content_pipeline.py.

Your content pipeline is now fully automated. Every day, Task Scheduler runs the script. Fresh articles appear in Raw with AI summaries. Items you mark with Y get promoted to Masterfile with unique Content IDs. Next up: a secret mission to make Gemini score each article so the best ideas jump out instantly.

Secret mission

Add AI Relevance Scoring

With 20+ articles flooding in every day, even Vietnamese summaries still require scanning. What if Gemini could rate each idea 1-10, and the best ones jumped out in bright yellow the moment you opened Excel?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on your local machine with a free-tier API key. There are no ongoing cloud costs or billing risks. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • content-pipeline folder (contains content_pipeline.py, run_pipeline.bat, and content_pipeline.xlsx).
  • Google AI Studio API key (free tier, no billing attached).
  • Windows Task Scheduler task (daily trigger running your pipeline).
  • Python packages installed via pip (feedparser, openpyxl, google-genai).

Keep everything running

No action needed. Choose this if you plan to keep using the pipeline daily.

  • Task Scheduler continues running the script every day, filling your Raw sheet with fresh ideas.
  • Open content_pipeline.xlsx whenever you want to review and approve ideas.
  • Python 3.14 and VS Code remain installed on your machine for future projects.

Pause - I'll come back to this later

Disable the scheduled task so the script stops running automatically. Your files and API key stay intact for when you return.

  • Press the Windows key, type Task Scheduler, and press Enter.
  • Click Task Scheduler Library in the left pane.
  • Find your content pipeline task in the middle pane, right-click it, and select Disable.
  • When you want to resume, right-click the same task and select Enable.

Delete - I don't want to use this again

Remove all project resources completely. Follow each item below.

  • Delete the scheduled task: Press the Windows key, type Task Scheduler, and press Enter. Click Task Scheduler Library in the left pane. Right-click your content pipeline task and select Delete. Confirm when prompted.
  • Delete your API key: Open the Google Cloud API Credentials page at console.cloud.google.com/apis/credentials. Find the API key you used for this project, click the Actions icon (three dots), and select Delete API key. Confirm deletion in the modal.
  • Uninstall the Python packages: Open a terminal and run the command below.
pip uninstall feedparser openpyxl google-genai -y
  • Delete the project folder: Open a terminal and run the command below (adjust the path if you created the folder somewhere other than your Desktop).
rmdir /s /q "%USERPROFILE%\Desktop\content-pipeline"
  • Confirm the folder is gone by checking your Desktop. The content-pipeline folder should no longer appear.

Nice Work!

Nice Work!

You just built a fully automated content idea pipeline from scratch. Every day, fresh tech articles land in your Excel file, already summarized in Vietnamese and categorized, ready for you to approve with a single keystroke.

You've learned how to:

  • Build a Python script that crawls RSS feeds from Viblo, Dev.to, and Medium, then uses Google Gemini AI to auto-summarize each article in Vietnamese with categories.
  • Create a self-maintaining Excel pipeline with Raw and Masterfile sheets, including automatic Content ID generation (PID-YYMMDD-XX) for approved ideas.
  • Automate the entire pipeline with Windows Task Scheduler so it runs daily without any manual intervention.
  • Secret Mission: Added AI relevance scoring so Gemini rates each idea 1-10 and auto-highlights the best ones in yellow.

Ready to quiz yourself?