Build an AI Summarizer API with Go

Deploy a Go API to Cloud Run that summarizes text using Gemini AI.

Introduction

30 Second Summary

Every AI feature you ship needs a way to answer the question: "Is this actually good?"

In this project, you will build a Go API that summarizes text using Google's Gemini AI, deploy it to Cloud Run, and then build an evaluation pipeline with Vertex AI to scientifically measure and improve the quality of your summaries.

What You'll Build

A Go REST API that accepts text and returns AI-generated summaries using Gemini 2.5 Flash, deployed to Cloud Run with a Python evaluation pipeline that scores summary quality.

By the end of this project, you'll have:

  • A Go HTTP server with a /summarize endpoint powered by Gemini 2.5 Flash through the Google Gen AI SDK.
  • A Cloud Run deployment accessible via a public URL.
  • A Python evaluation script that scores summary quality using Vertex AI evaluation metrics like text quality and fluency.
  • Improved prompts based on evaluation feedback, with measurable before-and-after scores.
  • A bullet-point summary endpoint that you evaluate against your paragraph-style summaries to see which format scores higher.

Are there any prerequisites?

No prior experience with Go, Google Cloud, or AI APIs is required. This guide walks you through setting up every tool from scratch, including the gcloud CLI, a GCP account with free trial credits, and Python for the evaluation pipeline.

Before We Start

Before We Start...

Set Up Your GCP Account and Tools

Before you can build an AI-powered API, you need a place to run it. Google Cloud Platform provides the AI models, the serverless hosting, and the evaluation tools you'll use throughout this project. In this step, you'll create a GCP account, install the command-line tools, and enable the APIs your project depends on.

In This Step, Get Ready To:
  • Create a GCP account with $300 in free trial credits and set up a new project.
  • Install the gcloud CLI and verify your Python installation.
  • Enable the Vertex AI and Cloud Run APIs and configure authentication.
Create a GCP Account and Project

Google Cloud offers a free trial with $300 in credits that lasts 90 days. These credits cover Vertex AI API usage, Cloud Run deployments, and everything else you'll use in this project. You'll need a credit card for identity verification, but you won't be charged unless you manually upgrade your account.

  • Open the GCP free trial signup page in your browser.
  • Sign in with your Google account and follow the prompts to activate the free trial.
  • Once you're in the Google Cloud Console, click the project dropdown at the top of the page.
  • Click **New Project**.
  • Enter a project name like ai-summarizer and click **Create**.
  • Select your new project from the project dropdown to make it active.

Take note of your Project ID (visible on the project dashboard or in the project dropdown). This is different from the project name. You'll need it throughout this project.

What can the free trial credits cover?

The $300 free trial credits work with the Vertex AI Gemini API, which is what this project uses. They do not cover the Gemini Developer API (AI Studio). Since you're going through Vertex AI, your credits apply to every API call in this project.

Install the Gcloud CLI

The gcloud CLI is the command-line tool for managing Google Cloud resources. You'll use it to deploy your API, enable APIs, and authenticate your local development environment.

macOS

First, figure out which chip your Mac has. Click the Apple menu and select **About This Mac**. If you see "Apple" in the chip or processor field, you have Apple Silicon. If you see "Intel", you have an Intel chip.

  • Download the appropriate archive for your chip:
  • Open your terminal and navigate to where the file was downloaded (usually your Downloads folder).
  • Extract the archive.
cd ~/Downloads
tar -xf google-cloud-cli-darwin-arm.tar.gz

Pro tip: adjust for Intel

If you have an Intel Mac, replace darwin-arm with darwin-x86_64 in the tar command above.

  • Run the install script.
./google-cloud-sdk/install.sh

The installer will ask if you want to install Python. Say yes if you don't already have Python 3.10+ installed. The installer bundles Python 3.13, which covers both the gcloud CLI requirement and the Python requirement for your evaluation script later.

  • Open a new terminal window (so the PATH changes take effect) and initialize gcloud.
gcloud init

What does this command do?

This walks you through logging in with your Google account and selecting the project you just created. Follow the prompts in your browser and terminal to complete the setup.

Windows

  • Download the Google Cloud CLI installer for Windows.
  • Run the installer and follow the prompts. Make sure the option to install bundled Python is checked if you don't already have Python 3.10+ on your system.
  • Once the installer finishes, it will open a terminal and run gcloud init automatically.
  • Follow the prompts to log in with your Google account and select the project you created.

After initialization, verify that gcloud is working and pointing to the right project.

  • Run the following command to confirm your setup.
gcloud config list

What should I see?

You should see your Google account email under [core] and your project ID under project. If the project is wrong, run gcloud config set project YOUR_PROJECT_ID to fix it.

You also need Python for the evaluation pipeline you'll build later. Let's verify it's available.

  • Check your Python version.
python3 --version

I see a version number

If you see Python 3.10 or higher, you're all set. The gcloud installer may have installed Python 3.13 for you.

Command not found

If python3 is not found, try python --version instead (some systems use python without the 3). If neither works, re-run the gcloud installer and accept the option to install bundled Python.

If you're still stuck, help me troubleshoot my Python installation.

Enable the Required APIs

Google Cloud keeps its services disabled by default. You need to explicitly turn on each API your project will use. Your summarizer needs three APIs: Vertex AI for the Gemini model, Cloud Run for hosting your API, and Cloud Build for building your container image during deployment.

  • Run the following command to enable all three APIs at once.
gcloud services enable run.googleapis.com aiplatform.googleapis.com cloudbuild.googleapis.com

What does this command do?

This enables three Google Cloud APIs on your project. run.googleapis.com is Cloud Run (serverless hosting). aiplatform.googleapis.com is Vertex AI (Gemini models and evaluation). cloudbuild.googleapis.com is Cloud Build (builds your container image when you deploy).

This may take a minute or two. You'll see a success message for each API once it's enabled.

Next, you need to set up Application Default Credentials (ADC). This is how your local Go code authenticates with Google Cloud services without hardcoding any keys or secrets.

  • Run the following command to log in with ADC.
gcloud auth application-default login

What does this command do?

This opens your browser and asks you to sign in with your Google account. Once you approve, it saves a credential file on your machine. The Go Gen AI SDK automatically picks up these credentials when you run your code locally, so you never need to paste API keys into your code.

Follow the browser prompts to authorize access. Once complete, you'll see a confirmation message in your terminal.

Having trouble enabling APIs?

If you see a "billing not enabled" error, make sure your free trial is fully activated. Go to the Billing section in the Cloud Console and confirm your project is linked to your billing account.

If something else is going wrong, help me troubleshoot my GCP API setup.

Your GCP environment is fully configured. Next up, you'll build the Go API that talks to Gemini and generates text summaries.

Build the Go Summarizer API

Your environment is ready. Now it is time to build the core of this project: a Go API that takes text and returns an AI-generated summary.

Your API will accept text through a /summarize endpoint, send it to Gemini 2.5 Flash via the Google Gen AI SDK for Go, and return the summary as JSON. By the end of this step, you will have a working summarizer running on your local machine.

In This Step, Get Ready To:
  • Initialize a Go project with the Gen AI SDK dependency.
  • Write an HTTP server with a /summarize endpoint powered by Gemini 2.5 Flash.
  • Test the API locally with curl.
Initialize the Go Project

Every Go project starts with a module. This gives Go a way to manage your code and its dependencies.

  • Create a new directory for your project and navigate into it.
mkdir go-summarizer && cd go-summarizer
  • Initialize the Go module.
go mod init go-summarizer
  • Add the Google Gen AI SDK as a dependency.
go get google.golang.org/genai

What is the Gen AI SDK?

The google.golang.org/genai package is Google's official, unified Go SDK for both the Gemini Developer API and Vertex AI. It provides type-safe access to Gemini models and handles authentication automatically through Application Default Credentials.

Go will download the SDK and update your go.mod and go.sum files with the dependency information.

  • Open this project folder in your code editor.
Write the HTTP Server with a /summarize Endpoint

This is the heart of your project. You will build a Go HTTP server that accepts a POST request containing text, sends that text to Gemini 2.5 Flash with a system instruction for summarization, and returns the summary as JSON.

The server needs to do three things: connect to Vertex AI using the Gen AI SDK, define a /summarize handler that calls Gemini, and listen on the correct port for Cloud Run compatibility.

  • Create a new file called main.go in your project directory and add the following code.
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"os"

	"google.golang.org/genai"
)

// SummarizeRequest defines the expected JSON body from the client.
type SummarizeRequest struct {
	Text string `json:"text"`
}

// SummarizeResponse defines the JSON response sent back to the client.
type SummarizeResponse struct {
	Summary string `json:"summary"`
}

func main() {
	ctx := context.Background()

	// Create a Gen AI client configured for Vertex AI.
	// This reads GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION,
	// and GOOGLE_GENAI_USE_VERTEXAI from environment variables.
	client, err := genai.NewClient(ctx, &genai.ClientConfig{
		HTTPOptions: genai.HTTPOptions{APIVersion: "v1"},
	})
	if err != nil {
		log.Fatalf("Failed to create Gen AI client: %v", err)
	}

	// Register the /summarize endpoint.
	http.HandleFunc("/summarize", func(w http.ResponseWriter, r *http.Request) {
		if r.Method != http.MethodPost {
			http.Error(w, "Only POST is allowed", http.StatusMethodNotAllowed)
			return
		}

		// Decode the incoming JSON body.
		var req SummarizeRequest
		if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
			http.Error(w, "Invalid JSON body", http.StatusBadRequest)
			return
		}
		if req.Text == "" {
			http.Error(w, "Text field is required", http.StatusBadRequest)
			return
		}

		// Call Gemini 2.5 Flash with a system instruction for summarization.
		resp, err := client.Models.GenerateContent(ctx, "gemini-2.5-flash", genai.Text(req.Text), &genai.GenerateContentConfig{
			SystemInstruction: genai.NewContentFromText("You are a summarizer. Summarize the given text in a concise paragraph.", "user"),
		})
		if err != nil {
			log.Printf("Gemini API error: %v", err)
			http.Error(w, "Failed to generate summary", http.StatusInternalServerError)
			return
		}

		// Extract the text from the response.
		summary := resp.Text()

		// Send the summary back as JSON.
		w.Header().Set("Content-Type", "application/json")
		json.NewEncoder(w).Encode(SummarizeResponse{Summary: summary})
	})

	// Cloud Run sets the PORT environment variable.
	// Default to 8080 for local development.
	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
	}

	log.Printf("Server starting on port %s", port)
	log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
}

What does this code do?

This file sets up a complete HTTP server with one endpoint. Here is what each section handles:

  • genai.NewClient creates a connection to Vertex AI. It reads your project, location, and backend setting from environment variables you set in Step 1.
  • The /summarize handler decodes the JSON body, validates it, sends the text to Gemini with a system instruction, and returns the summary.
  • The SystemInstruction tells Gemini how to behave. It acts as a persistent prompt that shapes every response the model generates.
  • The server reads the PORT environment variable because Cloud Run injects this at runtime. Locally, it defaults to 8080.

What is a System Instruction?

A system instruction is a special prompt that defines the model's role and behavior before it sees any user input. Think of it as giving Gemini its job description. In Step 5, you will iterate on this instruction to improve summary quality based on evaluation results.

Test the API Locally

Before deploying to the cloud, you should verify that your API works on your local machine. The Gen AI SDK will use the Application Default Credentials you set up in Step 1 to authenticate with Vertex AI.

  • Make sure your environment variables are set for the Vertex AI backend. Run these in your terminal.
export GOOGLE_GENAI_USE_VERTEXAI=true
export GOOGLE_CLOUD_PROJECT=[[GCP_PROJECT_ID="your GCP project ID"]]
export GOOGLE_CLOUD_LOCATION=us-central1

Why us-central1?

The Vertex AI evaluation service (which you will use in Step 4) only supports specific regions. us-central1 is the most commonly used supported region. Setting it now keeps your project and evaluation pipeline in the same location.

  • Start the server.
go run .

You should see a log message: Server starting on port 8080.

  • Open a second terminal window and send a test request with curl.
curl -X POST http://localhost:8080/summarize \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Go is a statically typed, compiled programming language designed at Google. It is syntactically similar to C, but with memory safety, garbage collection, structural typing, and CSP-style concurrency. It was designed by Robert Griesemer, Rob Pike, and Ken Thompson. Go was publicly announced in November 2009, and version 1.0 was released in March 2012. Go is widely used in production at Google and in many other organizations and open-source projects."
  }'

What does this command do?

This curl command sends a POST request to your local server with a JSON body containing a paragraph about Go. Your server will forward this text to Gemini 2.5 Flash, which will generate a summary and return it in the response.

I see a JSON summary

You should see a JSON response with a summary field containing a concise paragraph summarizing the input text. The exact wording will vary each time since Gemini generates responses dynamically.

Your Go summarizer API is working. The connection to Vertex AI is live, and Gemini is generating summaries from your system instruction.

I see an error

Here are the most common issues:

  • "Failed to create Gen AI client" means your environment variables are not set correctly. Verify that GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_CLOUD_PROJECT, and GOOGLE_CLOUD_LOCATION are all exported in the terminal where you ran go run ..
  • "Gemini API error" or a 403 error means your Application Default Credentials are not set up. Run gcloud auth application-default login again and restart the server.
  • "Connection refused" on the curl command means the server is not running. Check the first terminal for error messages.

If you are still stuck, help me debug my Go summarizer API.

Your summarizer API is working locally. Next up, you will deploy it to Cloud Run so it is accessible from anywhere with a public URL.

Deploy to Cloud Run

Your summarizer API works locally. Now it is time to ship it to the cloud so anyone can access it. Cloud Run lets you deploy your Go app from source with a single command. It handles containerization, scaling, and HTTPS automatically.

In This Step, Get Ready To:
  • Deploy your Go API to Cloud Run from source.
  • Test your live, public summarization endpoint with a real request.
Deploy From Source

Cloud Run can build and deploy your app directly from source code. You do not need to write a Dockerfile. The gcloud run deploy --source . command uses Cloud Build to automatically detect your Go project, build a container image, push it to Artifact Registry, and deploy it as a Cloud Run service.

  • Make sure you are in your project directory (go-summarizer).
  • Run the following deploy command:
gcloud run deploy go-summarizer --source . --region us-central1

During the deployment, gcloud will prompt you with a few questions:

  • If asked to enable the Artifact Registry API, type y and press Enter.
  • If asked to create an Artifact Registry repository, type y and press Enter.
  • When asked whether to allow unauthenticated invocations, type y and press Enter. This makes your API publicly accessible.

The build and deploy process takes a few minutes. Cloud Build compiles your Go code, packages it into a container, and pushes it to Cloud Run.

What does this command do?

The --source . flag tells Cloud Run to use buildpacks to auto-detect your language and create a container image. You never need to write a Dockerfile.

The --region us-central1 flag sets the deployment region. This region is important because the Vertex AI evaluation service you will use later only supports specific regions, and us-central1 is one of them.

Why Cloud Run?

Cloud Run is the simplest serverless container option on GCP. It scales to zero when idle, so you are not paying for a server sitting around waiting. The free tier covers 2 million requests per month.

Compared to alternatives like Compute Engine (a full VM) or GKE (Kubernetes), Cloud Run requires zero infrastructure management. You give it source code and it handles everything else.

When the deployment finishes, you will see output that includes a Service URL. It looks something like https://go-summarizer-xxxxx-uc.a.run.app.

  • Copy this URL. You will need it for the next substep and for the evaluation pipeline in Step 4.

Build Failed or Service Won't Start?

If the build fails, check that your go.mod and go.sum files are present in the directory. Cloud Build needs them to resolve dependencies.

If the service deploys but crashes, make sure your Go server reads the PORT environment variable. Cloud Run injects this variable and expects your app to listen on it. Hardcoding a port will cause the health check to fail.

Still stuck? Help me debug my Cloud Run deployment.

Test Your Deployed API

Your API is now live on the internet. Time to send it a real request and confirm that Gemini is generating summaries through your Cloud Run service.

  • Run the following curl command, replacing the URL with your Cloud Run service URL:
curl -X POST [[CLOUD_RUN_URL="your Cloud Run service URL"]]/summarize \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Artificial intelligence has transformed industries ranging from healthcare to finance. Machine learning models can now diagnose diseases from medical images with accuracy rivaling human doctors. In finance, AI algorithms process millions of transactions per second to detect fraud. Natural language processing enables chatbots to handle customer service inquiries, reducing wait times and operational costs. However, these advances raise important questions about job displacement, algorithmic bias, and data privacy that society must address."
  }'

You should see a JSON response containing a summary of the input text, just like when you tested locally.

What should I see?

The response should be a JSON object with a summary field containing a concise summary of the AI paragraph you sent. The exact wording will vary each time because Gemini generates responses dynamically.

Getting a 403 Forbidden Error?

This means your service is not allowing unauthenticated access. You can fix this by running:

gcloud run services add-iam-policy-binding go-summarizer \
  --region us-central1 \
  --member="allUsers" \
  --role="roles/run.invoker"

This grants public access to your Cloud Run service. Try your curl command again after running it.

Your AI summarizer is live and accessible to anyone with the URL. Next up, you will build an evaluation pipeline to scientifically measure how good your summaries actually are.

Build the Evaluation Pipeline

Your summarizer API is live on Cloud Run and generating summaries with Gemini 2.5 Flash. But how good are those summaries? Without measurement, you are guessing.

In this step, you will build an evaluation pipeline using the Vertex AI Gen AI evaluation service. This service uses standardized metrics to score your summaries on qualities like text quality and fluency. The evaluation service is Python-only, which mirrors real production architecture where serving and evaluation are separate concerns.

In This Step, Get Ready To:
  • Set up a Python virtual environment with the Vertex AI evaluation SDK.
  • Write an evaluation script that scores your API's summaries on text quality and fluency.
  • Run the evaluation and review your summarizer's performance metrics.
Set Up the Python Evaluation Environment

The Vertex AI evaluation service has a Python SDK but no Go SDK. You will create a separate Python environment for your evaluation scripts. This keeps your evaluation tooling isolated from your Go API code.

  • Open your terminal and navigate to your project directory.
  • Create a new directory for your evaluation scripts.
mkdir eval
cd eval
  • Create and activate a Python virtual environment.

macOS

python3 -m venv venv
source venv/bin/activate

Windows

python -m venv venv
.\venv\Scripts\Activate
  • Install the Vertex AI SDK with evaluation support.
pip install google-cloud-aiplatform[evaluation]

What does this command do?

The [evaluation] suffix is a Python extras syntax. It installs the base google-cloud-aiplatform package plus additional dependencies needed for the evaluation service, like pandas for data handling.

This installation may take a minute or two since it pulls in several dependencies.

Why Python Instead of Go for Evaluation?

The Vertex AI evaluation service only has a Python SDK. This is common in production AI systems. The serving layer (your Go API) prioritizes speed and type safety. The evaluation layer (Python scripts) prioritizes access to data science tooling and ML frameworks.

Keeping these as separate tools is a feature, not a limitation. Your Go API stays lean while your evaluation scripts can leverage the full Python ML ecosystem.

Write the Evaluation Script

Your evaluation script needs to do three things: define a dataset of test texts with reference summaries, call your deployed API to generate summaries for each, and then use the Vertex AI evaluation service to score the results.

  • Create a new file called evaluate.py inside your eval directory.
  • Add the following code to evaluate.py.
import requests
import pandas as pd
from vertexai import Client
from vertexai import types

# === Configuration ===
PROJECT_ID = "[[PROJECT_ID=\"your-gcp-project-id\"]]"
LOCATION = "[[LOCATION=\"us-central1\"]]"
API_URL = "[[CLOUD_RUN_URL=\"https://your-service-url.run.app\"]]/summarize"

# === Evaluation Dataset ===
# Each sample has an input text and a reference summary to compare against
eval_data = [
    {
        "input_text": "The Great Wall of China is a series of fortifications built along the historical northern borders of China to protect against various nomadic groups. Several walls were built from as early as the 7th century BC, with selective stretches later joined by Qin Shi Huang in 220-206 BC. The best-known sections were built by the Ming dynasty from 1368 to 1644. The wall spans approximately 13,171 miles.",
        "reference_summary": "The Great Wall of China is a series of ancient fortifications spanning about 13,171 miles along China's northern borders, built over centuries to defend against nomadic groups, with the most famous sections dating to the Ming dynasty."
    },
    {
        "input_text": "Photosynthesis is the process by which green plants and some other organisms use sunlight to synthesize foods from carbon dioxide and water. The process involves the green pigment chlorophyll and generates oxygen as a byproduct. It is the primary source of energy for nearly all life on Earth and is responsible for producing the oxygen in our atmosphere.",
        "reference_summary": "Photosynthesis is how plants convert sunlight, carbon dioxide, and water into food using chlorophyll, producing oxygen as a byproduct and serving as the primary energy source for life on Earth."
    },
    {
        "input_text": "Machine learning is a subset of artificial intelligence that focuses on building systems that learn from data. Unlike traditional programming where rules are explicitly coded, machine learning algorithms identify patterns in data and make decisions with minimal human intervention. Common applications include email filtering, speech recognition, and computer vision.",
        "reference_summary": "Machine learning is an AI subset where systems learn patterns from data instead of following explicit rules, commonly used in email filtering, speech recognition, and computer vision."
    },
    {
        "input_text": "The Amazon rainforest produces approximately 20 percent of the world's oxygen and is home to 10 percent of all species on Earth. Covering 5.5 million square kilometers across nine countries, it plays a critical role in regulating the global climate by absorbing carbon dioxide. Deforestation threatens this ecosystem, with an estimated 17 percent of the forest lost in the last 50 years.",
        "reference_summary": "The Amazon rainforest spans 5.5 million square kilometers, produces 20 percent of the world's oxygen, houses 10 percent of all species, and is critical for climate regulation, though 17 percent has been lost to deforestation."
    },
    {
        "input_text": "The theory of general relativity, published by Albert Einstein in 1915, describes gravity not as a force but as a curvature of spacetime caused by mass and energy. This theory predicted phenomena such as gravitational lensing and time dilation, both of which have been confirmed through experiments. It remains one of the two pillars of modern physics alongside quantum mechanics.",
        "reference_summary": "Einstein's 1915 theory of general relativity describes gravity as spacetime curvature caused by mass and energy, predicting confirmed phenomena like gravitational lensing and time dilation."
    }
]

# === Step 1: Call your Cloud Run API to generate summaries ===
print("Generating summaries from your API...")
for sample in eval_data:
    response = requests.post(API_URL, json={"text": sample["input_text"]})
    response.raise_for_status()
    sample["response"] = response.json()["summary"]
    print(f"  Generated summary ({len(sample['response'])} chars)")

# === Step 2: Build the evaluation dataset ===
# The eval service expects a DataFrame with 'response' and 'reference' columns
eval_dataset = pd.DataFrame({
    "response": [s["response"] for s in eval_data],
    "reference": [s["reference_summary"] for s in eval_data]
})

print(f"\nEvaluation dataset: {len(eval_dataset)} samples")

# === Step 3: Run the Vertex AI evaluation ===
print("\nRunning Vertex AI evaluation...")
client = Client(project=PROJECT_ID, location=LOCATION)

# Score summaries on text quality and fluency using rubric-based metrics
eval_result = client.evals.evaluate(
    dataset=eval_dataset,
    metrics=[types.RubricMetric.TEXT_QUALITY, types.RubricMetric.FLUENCY]
)

# === Step 4: Display results ===
print("\n" + "=" * 50)
print("EVALUATION RESULTS")
print("=" * 50)

# Print summary metrics
summary = eval_result.summary_metrics
print(f"\nSummary Metrics:")
for metric_name, value in summary.items():
    print(f"  {metric_name}: {value}")

# Print per-sample scores
print(f"\nPer-Sample Scores:")
metrics_df = eval_result.metrics_table
print(metrics_df.to_string(index=False))

What Does This Script Do?

The script works in four stages:

  • It sends five different texts to your deployed Cloud Run API and collects the AI-generated summaries.
  • It builds a pandas DataFrame pairing each generated summary with a human-written reference summary.
  • It sends this dataset to the Vertex AI evaluation service, which scores each summary on text quality and fluency using rubric-based metrics.
  • It prints both the aggregate scores (mean, standard deviation) and the per-sample breakdown so you can see which summaries performed well and which ones need improvement.

What Are Text Quality and Fluency Metrics?

Text quality measures how well the generated summary captures the key information from the source text. Fluency measures how natural and readable the summary sounds. Both are scored by a rubric-based evaluation model, not by simple text matching.

  • Replace the three placeholder values at the top of the script with your actual GCP project ID, location, and Cloud Run URL.

Pro tip: finding your Cloud Run URL

If you need to look up your Cloud Run service URL again, run gcloud run services list in your terminal. The URL will be in the output.

Which Regions Support Evaluation?

The Vertex AI evaluation service is only available in specific regions: us-central1, us-east4, us-west1, us-west4, europe-west1, europe-west4, and europe-west9. Make sure your LOCATION variable matches one of these.

Run the Evaluation and Review Results

Now you will run the evaluation script and see how your summarizer actually performs. The evaluation service calls a rubric-based model to score each summary, so this step takes about 30-60 seconds to complete.

  • Make sure you are in the eval directory with your virtual environment activated.
  • Install the requests library, which the script uses to call your API.
pip install requests
  • Run the evaluation script.
python3 evaluate.py

What Should I See?

You should see output showing the script generating summaries from your API, followed by the evaluation results. The summary metrics section shows mean scores and standard deviation for each metric. The per-sample scores table shows how each individual summary scored.

Scores typically range from 1 to 5. A score of 4 or above is strong. Scores below 3 indicate room for improvement in your prompt.

Take a moment to review the per-sample scores. Look for patterns. Are certain types of text (scientific, historical, technical) getting lower scores? Are the fluency scores consistently higher or lower than text quality? These observations will guide your prompt improvements in the next step.

Getting a Connection Error?

If you see a connection error when the script tries to call your API, make sure your Cloud Run URL is correct and includes /summarize at the end. Also verify your Cloud Run service is still deployed by running gcloud run services list. Help me debug my evaluation script connection error.

Getting a Permission Denied Error From Vertex AI?

Make sure your Application Default Credentials are set up by running gcloud auth application-default login. Also verify that the Vertex AI API is enabled in your project and that your LOCATION is set to a supported evaluation region. Help me fix my Vertex AI permissions.

You now have real data on how your summarizer performs. In the next step, you will use these results to improve your system instruction and measure the difference.

Iterate on Your Prompts and Measure Improvement

Your evaluation pipeline just gave you hard data on how your summarizer performs. Now comes the part that separates hobby projects from production AI: using that data to improve your prompts and proving the improvement with numbers. This is the core loop that production AI teams run every day.

In This Step, Get Ready To:
  • Improve your system instruction based on evaluation feedback.
  • Redeploy to Cloud Run and re-evaluate to measure the improvement.
Improve Your System Instruction

Look at your evaluation results from the previous step. The scores for text quality and fluency tell you where your summaries are strong and where they fall short. Common weaknesses with a basic prompt include summaries that are too long, miss key facts, or feel generic.

Your original system instruction was broad. A more specific instruction gives Gemini clearer constraints, which typically produces higher-quality output. This is the essence of prompt engineering.

  • Open main.go in your editor.
  • Find the system instruction string in your summarize handler.
  • Replace it with a more specific instruction that addresses weaknesses you observed.

Here is an example of an improved system instruction. Adapt it based on your own evaluation results:

systemInstruction := "You are a precise text summarizer. Follow these rules strictly:\n" +
	"1. Keep summaries to 2-3 sentences maximum.\n" +
	"2. Preserve all key facts, names, and numbers from the original text.\n" +
	"3. Use clear, direct language. Avoid filler phrases.\n" +
	"4. Start with the most important point.\n" +
	"5. Never add information that is not in the original text."

What Makes This Instruction Better?

The original instruction was something like "You are a helpful assistant that summarizes text." That gives the model almost no constraints. The improved version adds specific rules: a length limit, a fact-preservation requirement, and a style guide. Each rule targets a specific weakness that evaluation metrics can detect.

Pro tip: tailor to your scores

If your text quality score was low, focus on adding constraints about accuracy and completeness. If fluency was the weak point, add rules about sentence structure and clarity. Let the data guide your changes.

Redeploy and Re-Evaluate

With your improved prompt in place, you need to deploy the updated API and run the same evaluation again. Using the same dataset and metrics lets you make a direct comparison between the two versions.

  • Redeploy your updated API to Cloud Run.
gcloud run deploy --source .

What does this command do?

This is the same deploy command you used in Step 3. Cloud Run rebuilds your container image with the updated code and replaces the running service. Your URL stays the same, so your evaluation script does not need any changes.

  • Accept the same settings as before when prompted (region, Artifact Registry).
  • Wait for the deployment to complete.

Now run your evaluation script again to score the improved summaries.

  • Navigate to your evaluation directory.
  • Activate your Python virtual environment.
  • Run the evaluation script.
source venv/bin/activate
python evaluate.py

Compare the new scores against your original results. You should see improvement in at least one metric. Write down both sets of scores so you can reference them.

What Should I See?

Your mean text quality and fluency scores should be higher than the first run. The improvement might be small (e.g., from 3.2 to 3.8 on a 5-point scale) or significant, depending on how much your original prompt was lacking. Even a small, consistent improvement across all samples proves that your prompt change had a measurable effect.

Scores Went Down?

If your scores decreased, your new constraints may be too restrictive or conflicting. Try relaxing one rule at a time and redeploying. For example, a very tight sentence limit might force the model to drop important facts, hurting text quality. Help me debug why my evaluation scores decreased after changing my prompt.

You just completed the build-deploy-evaluate-improve loop. This is the same cycle that production AI teams use to ship and refine AI features. Next up, you will wrap up the project and review what you have accomplished.

Secret mission

Add a Bullet-Point Summary Endpoint

Your summarizer works, but every summary comes back as a paragraph. What if users want quick, scannable bullet points instead? Add a second endpoint, deploy it, and use your evaluation pipeline to find out which format actually scores higher.

Before you go

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. Cloud Run scales to zero when idle, so costs are minimal if you leave it running. The Gemini API usage for this project is negligible and covered by your free trial credits.

Cost warning

Your GCP free trial provides $300 in credits for 90 days. Cloud Run has a free tier of 2 million requests per month, and Gemini 2.5 Flash costs only ~$0.30 per 1M input tokens. Costs for this project are negligible, but Artifact Registry stores your container images and may accumulate small storage charges over time. If your free trial expires or you upgrade to a paid account, consider deleting unused resources.

Resources you used:

  • GCP project with billing enabled.
  • Cloud Run service (your deployed Go summarizer API).
  • Artifact Registry container images (created by Cloud Build during deployment).
  • Vertex AI API usage (Gemini 2.5 Flash calls and evaluation service).
  • Local project files (Go module, Python virtual environment, evaluation script).
  • Application Default Credentials stored locally by gcloud auth application-default login.

Keep everything running

No action needed. Choose this if you're still actively building or want to keep experimenting with your summarizer API.

  • Your Cloud Run service scales to zero when idle, so you won't be charged for compute when no requests are coming in.
  • Artifact Registry will store your container images. Storage costs are minimal but will accumulate over time.
  • Your free trial credits cover all of these costs for up to 90 days.

Pause - I'll come back to this later

Delete the Cloud Run service to stop any potential charges, but keep your local project files and GCP project intact so you can redeploy later.

  • Delete your Cloud Run service by running gcloud run services delete SERVICE_NAME --region REGION. Replace SERVICE_NAME and REGION with the values you used during deployment.
  • Keep your local Go project and Python evaluation files. You can redeploy anytime with gcloud run deploy --source ..
  • Optionally, deactivate your Python virtual environment by running deactivate in your terminal.

Delete - I don't want to use this again

Remove all project resources and start fresh. This deletes everything you created during this project.

  • Delete the entire GCP project by running gcloud projects delete PROJECT_ID. This removes the Cloud Run service, Artifact Registry images, API enablements, and all other project resources at once.
  • Revoke your Application Default Credentials by running gcloud auth application-default revoke.
  • Delete your local project directory containing the Go module and Python evaluation files.
  • Delete the Python virtual environment folder if it is outside your project directory.

Mission accomplished

That's a Wrap!

Nice work! You just shipped an AI-powered summarization API and built a scientific evaluation workflow to measure its quality. That is exactly how production AI teams operate: build, deploy, measure, and iterate.

Here's what you accomplished:

  • Built a Go REST API that generates text summaries using Gemini 2.5 Flash via the Google Gen AI SDK, and deployed it to Cloud Run with a single command.
  • Created an evaluation pipeline using the Vertex AI Gen AI evaluation service to score your summaries on text quality and fluency metrics.
  • Iterated on your system instructions based on evaluation results, redeployed, and measured the improvement with real data.
  • Added a /summarize/bullet-points endpoint and compared bullet-point vs. paragraph summaries using evaluation data to see which format scored higher.

Ready to quiz yourself?