Book Catalog Enrichment Engine

Build a parallel book enrichment engine with read/write database splitting.

Introduction

30 Second Summary

Imagine your app needs to fetch metadata for five books from an external service. If each call takes a second and you process them one-by-one, your user stares at a spinner for five seconds while a smarter approach would finish in one.

In this project, you will take an existing Spring Boot CRUD app and supercharge it with parallel processing, smart database routing, and real integration tests. You will use CompletableFuture for parallel execution, route reads and writes to separate PostgreSQL instances, and validate everything against real database containers with Testcontainers.

What You'll Build

Fire a single API call and watch five books enrich in parallel in under a second, with routing logs revealing exactly which database handles each query.

By the end of this project, you'll have:

  • A parallel enrichment pipeline where you send five ISBNs to one endpoint and watch timing logs prove all five process in about one second instead of five.
  • Read/write database splitting where read queries silently route to a replica PostgreSQL while writes hit the primary, all visible through routing logs after every request.
  • An integration test suite backed by Testcontainers where real PostgreSQL containers spin up, validate your enrichment logic and routing decisions, and tear down on their own.
  • Secret Mission: Build a concurrent stress test that fires 50 parallel enrichment requests and verifies thread pool behavior, connection pool limits, and routing correctness under load.

Are there any prerequisites?

This project builds on an existing bookcatalog Spring Boot 4.0.7 CRUD app with a Book entity, BookRepository, BookController, and DTO records already set up. You also need Docker Desktop installed and running, plus Maven installed on your machine.

Before We Start

Before we dive in, let's lock in what you're about to build and why each piece matters. You have a working bookcatalog CRUD app, and this project will transform it into an interview-ready showcase of Java's most in-demand patterns.

Migrate from H2 to PostgreSQL with Docker Compose

In Step 0, you mapped out what you're building and why. Now it's time to lay the infrastructure foundation.

Your current app uses an H2 in-memory database. That works for quick demos, but you can't run two separate database instances from H2. To implement read/write splitting later, you need two real PostgreSQL servers: one acting as the primary and one acting as the replica.

In this step, get ready to:
  • Spin up two PostgreSQL 16 containers using Docker Compose.
  • Add the PostgreSQL driver and Testcontainers dependencies to your Maven project.
  • Create a Spring profile that connects your app to the real PostgreSQL database.
Create a Docker Compose file for your databases

Docker Compose lets you define and run multiple Docker containers from a single YAML file. You need two PostgreSQL containers: a primary on port 5433 and a replica on port 5434.

  • Create a new file called docker-compose-psql.yml in the root of your bookcatalog project (the same folder that contains pom.xml).
  • Paste the following configuration into the file:
services:
  postgres-primary:
    image: postgres:16-alpine
    container_name: postgres-primary
    environment:
      POSTGRES_DB: bookcatalog
      POSTGRES_USER: bookcatalog
      POSTGRES_PASSWORD: bookcatalog
    ports:
      - "5433:5432"
    volumes:
      - primary-data:/var/lib/postgresql/data

  postgres-replica:
    image: postgres:16-alpine
    container_name: postgres-replica
    environment:
      POSTGRES_DB: bookcatalog
      POSTGRES_USER: bookcatalog
      POSTGRES_PASSWORD: bookcatalog
    ports:
      - "5434:5432"
    volumes:
      - replica-data:/var/lib/postgresql/data

volumes:
  primary-data:
  replica-data:

What does this configuration do?

Each service block defines a PostgreSQL 16 container using the lightweight postgres:16-alpine image. The ports mapping exposes each container's internal port 5432 on a different host port so they don't collide.

The volumes section gives each container its own named volume for persistent data. This means your book data survives container restarts.

  • Save docker-compose-psql.yml.
  • Start both containers by running this command from your project root:
docker compose -f docker-compose-psql.yml up -d
  • Verify both containers are running by running:
docker ps

You should see two containers listed: postgres-primary and postgres-replica, both with a status of Up.

Don't see both containers running?

Make sure Docker Desktop is running before you retry the startup command.

docker compose -f docker-compose-psql.yml up -d
  • Check that ports 5433 and 5434 are not already in use by another process.
  • Help me troubleshoot my Docker Compose PostgreSQL containers not starting.
Add the PostgreSQL and Testcontainers dependencies to Maven

Your app currently uses the H2 driver. You need the PostgreSQL JDBC driver to connect to your new containers. You also need Testcontainers dependencies for the integration tests you will write in a later step.

  • Open pom.xml in your IDE.
  • Find the <dependencies> section. Add the PostgreSQL runtime dependency anywhere inside it:
<!-- PostgreSQL JDBC driver -->
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
</dependency>

Why no version number?

Spring Boot 4.0.7 manages the PostgreSQL driver version for you through its dependency BOM. You only need to declare the group and artifact. Spring Boot resolves it to version 42.7.11 automatically.

  • Now add a <dependencyManagement> section (if your pom.xml doesn't already have one) directly before or after the <dependencies> section. This imports the Testcontainers BOM so you don't need version numbers on individual Testcontainers modules:
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>testcontainers-bom</artifactId>
            <version>2.0.5</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>
  • Finally, add the two Testcontainers test dependencies inside your <dependencies> section:
<!-- Testcontainers PostgreSQL module -->
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers-postgresql</artifactId>
    <scope>test</scope>
</dependency>

<!-- Spring Boot Testcontainers integration -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-testcontainers</artifactId>
    <scope>test</scope>
</dependency>

What are these test dependencies for?

You won't use them in this step, but they prepare your project for Step 5 where you write integration tests. testcontainers-postgresql provides the PostgreSQLContainer class that spins up a real PostgreSQL instance inside your test suite.

spring-boot-testcontainers adds Spring Boot integration so your tests can auto-configure datasource properties from the container.

  • Save pom.xml.
  • Verify Maven can resolve all the new dependencies by running:
./mvnw dependency:resolve

You should see Maven downloading the PostgreSQL driver and Testcontainers modules, ending with BUILD SUCCESS.

Seeing dependency resolution errors?

  • Double check that the <dependencyManagement> section is a sibling of <dependencies>, not nested inside it.
  • Make sure the Testcontainers BOM version is exactly 2.0.5.
  • help me fix Maven dependency resolution errors in my Spring Boot project with Testcontainers
Create the PostgreSQL Spring profile and start the app

Spring Boot profiles let you swap configuration without changing code. You will create a postgres profile that points the app at your Docker PostgreSQL containers instead of H2.

  • Create a new file called application-postgres.properties inside src/main/resources/ (the same folder as your existing application.properties).
  • Paste the following configuration:
spring.application.name=bookcatalog

# Disable H2
spring.h2.console.enabled=false

# Primary PostgreSQL
spring.datasource.primary.url=jdbc:postgresql://localhost:5433/bookcatalog
spring.datasource.primary.username=bookcatalog
spring.datasource.primary.password=bookcatalog
spring.datasource.primary.driver-class-name=org.postgresql.Driver

# Replica PostgreSQL
spring.datasource.replica.url=jdbc:postgresql://localhost:5434/bookcatalog
spring.datasource.replica.username=bookcatalog
spring.datasource.replica.password=bookcatalog
spring.datasource.replica.driver-class-name=org.postgresql.Driver

# JPA/Hibernate settings
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect

What does this profile configure?

  • spring.h2.console.enabled=false disables the H2 web console since you're no longer using H2.
  • The primary and replica property prefixes define two separate datasource configurations. You will wire these into a routing datasource in Step 4.
  • spring.jpa.hibernate.ddl-auto=update tells Hibernate to create or update tables automatically based on your Book entity.
  • org.hibernate.dialect.PostgreSQLDialect tells Hibernate to generate PostgreSQL-compatible SQL instead of H2 SQL.
  • Save application-postgres.properties.
  • Before you start the app, think about this: do you expect the app to connect successfully? The properties file defines primary and replica prefixed datasources, but Spring Boot auto-configuration looks for spring.datasource.url by default.
  • Start the app with the postgres profile by running:
./mvnw spring-boot:run -Dspring-boot.run.profiles=postgres

The app starts successfully. Spring Boot auto-configuration picks up the spring.datasource.primary.* properties as a datasource candidate. Look for HikariPool-1 - Starting... in the console logs, confirming the connection pool connected to PostgreSQL.

  • Open a new terminal (keep the app running) and test the endpoint by running:
curl http://localhost:8080/api/books

You should see an empty JSON array [] returned. That empty array is coming from PostgreSQL, not H2.

App failing to start or not returning []?

  • Confirm both Docker containers are still running by running:
docker ps
  • Restart both containers if they stopped by running:
docker compose -f docker-compose-psql.yml up -d
  • Check that the JDBC URL in application-postgres.properties uses port 5433 for primary, not the default 5432.
  • Make sure you passed the profile flag correctly. The flag is -Dspring-boot.run.profiles=postgres, not --spring.profiles.active.
  • Help me troubleshoot my Spring Boot app not connecting to PostgreSQL via Docker.

✔️ Awesome, I've got everything!

Great. Double check you have saved both docker-compose-psql.yml and application-postgres.properties.

ⓧ I'd like to double check the full code

Here is the complete docker-compose-psql.yml:

services:
  postgres-primary:
    image: postgres:16-alpine
    container_name: postgres-primary
    environment:
      POSTGRES_DB: bookcatalog
      POSTGRES_USER: bookcatalog
      POSTGRES_PASSWORD: bookcatalog
    ports:
      - "5433:5432"
    volumes:
      - primary-data:/var/lib/postgresql/data

  postgres-replica:
    image: postgres:16-alpine
    container_name: postgres-replica
    environment:
      POSTGRES_DB: bookcatalog
      POSTGRES_USER: bookcatalog
      POSTGRES_PASSWORD: bookcatalog
    ports:
      - "5434:5432"
    volumes:
      - replica-data:/var/lib/postgresql/data

volumes:
  primary-data:
  replica-data:

Here is the complete application-postgres.properties:

spring.application.name=bookcatalog

# Disable H2
spring.h2.console.enabled=false

# Primary PostgreSQL
spring.datasource.primary.url=jdbc:postgresql://localhost:5433/bookcatalog
spring.datasource.primary.username=bookcatalog
spring.datasource.primary.password=bookcatalog
spring.datasource.primary.driver-class-name=org.postgresql.Driver

# Replica PostgreSQL
spring.datasource.replica.url=jdbc:postgresql://localhost:5434/bookcatalog
spring.datasource.replica.username=bookcatalog
spring.datasource.replica.password=bookcatalog
spring.datasource.replica.driver-class-name=org.postgresql.Driver

# JPA/Hibernate settings
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect

Your bookcatalog app is now running on real PostgreSQL. Next up, you will build the Streams-powered enrichment service that transforms book data and simulates external metadata fetches.

Build the Streams-Powered Enrichment Service

Your PostgreSQL databases are connected and returning data. Now it's time to build the service layer that transforms raw book entities into rich, queryable results.

Your current service layer passes entities straight through with no transformation. In this step, you'll build a Streams API-powered enrichment service that filters, maps, and collects book data into enriched DTOs. You'll also simulate external metadata fetching and discover a performance bottleneck along the way.

In this step, get ready to:
  • Create record DTOs for enriched book data and author statistics.
  • Build a service with Streams-based enrichment, statistics, and search methods.
  • Wire up controller endpoints and discover the sequential enrichment bottleneck.
Create the enrichment data models

Your enrichment pipeline needs new data shapes. Java records give you immutable, compact data carriers with auto-generated constructors, getters, and toString. You'll create two: one for enriched book data and one for per-author statistics.

  • Create a new file at src/main/java/com/example/bookcatalog/dto/EnrichedBookDTO.java and paste the following code:
package com.example.bookcatalog.dto;

import java.time.LocalDateTime;

// Holds enriched book data including external metadata
public record EnrichedBookDTO(
        Long id,
        String title,
        String author,
        String isbn,
        int pageCount,
        String genre,
        LocalDateTime enrichedAt
) {
}

What does this record define?

EnrichedBookDTO carries the original book fields (id, title, author, isbn) plus enrichment data that a real external API would provide: pageCount, genre, and a timestamp recording when the enrichment happened.

  • Create another new file at src/main/java/com/example/bookcatalog/dto/BookStatisticsDTO.java and paste this code:
package com.example.bookcatalog.dto;

// Per-author aggregation of book statistics
public record BookStatisticsDTO(
        String authorName,
        long bookCount,
        double averagePageCount
) {
}

What does BookStatisticsDTO hold?

This record captures per-author aggregation: the author's name, how many books they have, and the average page count across their titles. You'll populate these fields using Collectors.groupingBy later in the service.

  • Save both files.
  • Confirm the project compiles with the new records by running:
./mvnw compile

You should see BUILD SUCCESS in the output.

Seeing a compile error?

  • Check that both files are inside the dto package at src/main/java/com/example/bookcatalog/dto/.
  • Verify the package declaration at the top of each file reads com.example.bookcatalog.dto.
  • help me debug a compile error in my DTO records
Build the sequential enrichment service

The service layer is where the Streams API shines. You'll build a service that fetches books from the repository, filters them by ISBN, and maps each through a simulated external metadata fetch. Each fetch takes 1 second to simulate a real API call.

  • Create a new file at src/main/java/com/example/bookcatalog/service/BookEnrichmentService.java and paste the class shell:
package com.example.bookcatalog.service;

import java.time.LocalDateTime;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ThreadLocalRandom;
import java.util.stream.Collectors;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import com.example.bookcatalog.dto.BookStatisticsDTO;
import com.example.bookcatalog.dto.EnrichedBookDTO;
import com.example.bookcatalog.model.Book;
import com.example.bookcatalog.repository.BookRepository;

@Service
@Transactional(readOnly = true)
public class BookEnrichmentService {
    private static final Logger log = LoggerFactory.getLogger(BookEnrichmentService.class);
    // Simulated genres that a real enrichment API would return
    private static final String[] GENRES = {"Fiction", "Science", "History", "Technology", "Philosophy"};
    private final BookRepository bookRepository;

    public BookEnrichmentService(BookRepository bookRepository) {
        this.bookRepository = bookRepository;
    }
}

What does this class set up?

  • @Service marks this as a Spring-managed bean so it can be injected into controllers.
  • @Transactional(readOnly = true) marks all methods as read-only database transactions by default. You'll see why this annotation matters when you add read/write splitting in Step 4.
  • The GENRES array and ThreadLocalRandom simulate external metadata that a real enrichment API would return.
  • Inside the BookEnrichmentService class, add these two methods before the closing brace }:
    @Transactional
    public List<EnrichedBookDTO> enrichBooks(List<String> isbns) {
        long start = System.nanoTime();

        List<EnrichedBookDTO> results = bookRepository.findAll().stream()
                .filter(book -> isbns.contains(book.getIsbn()))
                .map(this::enrichSingleBook)
                .collect(Collectors.toList());

        long elapsed = (System.nanoTime() - start) / 1_000_000;
        log.info("Enriched {} books in {} ms", results.size(), elapsed);
        return results;
    }

    private EnrichedBookDTO enrichSingleBook(Book book) {
        log.info("Enriching book '{}' on thread: {}", book.getTitle(), Thread.currentThread().getName());

        // Simulate an external API call that takes 1 second
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new RuntimeException("Enrichment interrupted", e);
        }
        int pageCount = ThreadLocalRandom.current().nextInt(100, 800);
        String genre = GENRES[ThreadLocalRandom.current().nextInt(GENRES.length)];
        return new EnrichedBookDTO(
                book.getId(), book.getTitle(), book.getAuthor(),
                book.getIsbn(), pageCount, genre, LocalDateTime.now());
    }

What does this code do?

  • enrichBooks() uses a Streams pipeline: findAll() gets every book, .filter() keeps only books whose ISBNs match the input list, .map(this::enrichSingleBook) transforms each one, and .collect() gathers the results.
  • Thread.sleep(1000) in enrichSingleBook simulates calling an external API that takes 1 second per book. This runs sequentially. Five books means 5 seconds of waiting.
  • System.nanoTime() measures the total elapsed time and logs it. You'll use this timing log to compare sequential vs parallel performance.
  • @Transactional on enrichBooks overrides the class-level readOnly = true because enrichment is a write operation.
  • Save BookEnrichmentService.java.
  • Confirm the service compiles by running:
./mvnw compile

You should see BUILD SUCCESS again.

Compile error in the service?

  • Make sure both methods are inside the class braces, not after the closing } of the class.
  • Verify the file is in the service package at src/main/java/com/example/bookcatalog/service/.
  • Check that the Book and BookRepository imports match the package names from your existing CRUD app.
  • help me fix a compile error in BookEnrichmentService
Wire up the controller and hit the bottleneck

The service methods are ready but there is no way to call them over HTTP. You need a controller to expose the enrichment endpoint so you can observe the sequential bottleneck in action.

  • Create a new file at src/main/java/com/example/bookcatalog/controller/BookEnrichmentController.java and paste this code:
package com.example.bookcatalog.controller;

import java.util.List;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import com.example.bookcatalog.dto.BookStatisticsDTO;
import com.example.bookcatalog.dto.EnrichedBookDTO;
import com.example.bookcatalog.service.BookEnrichmentService;

@RestController
@RequestMapping("/api/books")
public class BookEnrichmentController {

    private final BookEnrichmentService enrichmentService;

    public BookEnrichmentController(BookEnrichmentService enrichmentService) {
        this.enrichmentService = enrichmentService;
    }

    @PostMapping("/enrich")
    public List<EnrichedBookDTO> enrichBooks(@RequestBody List<String> isbns) {
        return enrichmentService.enrichBooks(isbns);
    }
}

What does this controller do?

@RequestMapping("/api/books") shares the same base path as your existing BookController. Spring allows multiple controllers on the same base path as long as each endpoint path is unique.

@PostMapping("/enrich") accepts a JSON array of ISBN strings and passes them to the enrichment service. The extra imports (GetMapping, RequestParam, BookStatisticsDTO) are for endpoints you'll add shortly.

  • Save BookEnrichmentController.java.
  • Start the app with the postgres profile by running:
./mvnw spring-boot:run -Dspring-boot.run.profiles=postgres
  • Wait for the Started BookcatalogApplication message in the console.
  • Open a new terminal window for running curl commands while the app stays running.
  • Seed the database with 5 books by running these commands:
curl -X POST http://localhost:8080/api/books -H "Content-Type: application/json" -d '{"title":"Clean Code","author":"Robert Martin","isbn":"978-0132350884"}'
curl -X POST http://localhost:8080/api/books -H "Content-Type: application/json" -d '{"title":"Effective Java","author":"Joshua Bloch","isbn":"978-0134685991"}'
curl -X POST http://localhost:8080/api/books -H "Content-Type: application/json" -d '{"title":"The Pragmatic Programmer","author":"David Thomas","isbn":"978-0135957059"}'
curl -X POST http://localhost:8080/api/books -H "Content-Type: application/json" -d '{"title":"Refactoring","author":"Martin Fowler","isbn":"978-0134757599"}'
curl -X POST http://localhost:8080/api/books -H "Content-Type: application/json" -d '{"title":"Clean Architecture","author":"Robert Martin","isbn":"978-0134494166"}'

Each command returns the created book as JSON. Confirm all five succeed before moving on.

Now think about this: each book's enrichment simulates an external API call that takes 1 second. You have 5 books. How long do you expect the enrichment to take?

  • Call the enrichment endpoint by running:
curl -X POST http://localhost:8080/api/books/enrich -H "Content-Type: application/json" -d '["978-0132350884","978-0134685991","978-0135957059","978-0134757599","978-0134494166"]'

Switch back to the terminal where your app is running and look for a log line starting with Enriched.

You should see something like Enriched 5 books in 5023 ms. Five books, one at a time, 5 seconds total. Each enrichment ran sequentially because the .map(this::enrichSingleBook) call blocks on each book before moving to the next.

Why is it so slow?

The enrichBooks method processes books sequentially: it calls enrichSingleBook on each book one at a time. Since each call includes a 1-second Thread.sleep simulating a real external API, 5 books take 5 seconds.

In production, where you might enrich hundreds of books, this linear scaling makes the endpoint unusable. The thread name in each log line (all showing the same thread) confirms that no parallel execution happened. You'll fix this with CompletableFuture in the next step.

🙋‍♀️ Not seeing the timing log?

  • Make sure you are checking the terminal where the Spring Boot app is running, not the terminal where you ran curl.
  • If the curl command returned an error, confirm all 5 books were created successfully first by running curl http://localhost:8080/api/books and checking you see 5 entries.
  • help me debug why my enrichment endpoint isn't logging timing information

The sequential bottleneck is real. Before fixing it in the next step, finish the service with two more Streams-powered methods.

  • Stop the running application with Ctrl+C in the app terminal.
  • In BookEnrichmentService.java, add these two methods inside the class, between enrichBooks and enrichSingleBook:
    public List<BookStatisticsDTO> getBookStatistics() {
        // Group all books by their author
        Map<String, List<Book>> booksByAuthor = bookRepository.findAll().stream()
                .collect(Collectors.groupingBy(Book::getAuthor));

        return booksByAuthor.entrySet().stream()
                .map(entry -> new BookStatisticsDTO(
                        entry.getKey(),
                        entry.getValue().size(),
                        entry.getValue().stream()
                                .mapToInt(book -> book.getTitle().length() * 10)
                                .average()
                                .orElse(0.0)))
                .sorted((a, b) -> Long.compare(b.bookCount(), a.bookCount()))
                .collect(Collectors.toList());
    }

    public List<EnrichedBookDTO> searchBooks(String query) {
        String lowerQuery = query.toLowerCase();

        // Filter by title or author, sort alphabetically, map to DTOs
        return bookRepository.findAll().stream()
                .filter(book -> book.getTitle().toLowerCase().contains(lowerQuery)
                        || book.getAuthor().toLowerCase().contains(lowerQuery))
                .sorted((a, b) -> a.getTitle().compareToIgnoreCase(b.getTitle()))
                .map(book -> new EnrichedBookDTO(
                        book.getId(), book.getTitle(), book.getAuthor(),
                        book.getIsbn(), 0, "Unknown", LocalDateTime.now()))
                .collect(Collectors.toList());
    }

What do these methods do?

  • getBookStatistics() uses Collectors.groupingBy(Book::getAuthor) to group all books by author. It then maps each group into a BookStatisticsDTO with the author name, book count, and average page count. Results are sorted by book count (most prolific authors first).
  • searchBooks() filters the full book list using a case-insensitive match on title or author. Matching books are sorted alphabetically by title, then mapped to EnrichedBookDTO with placeholder values for pageCount and genre (since no enrichment API is called during a search).
  • Save BookEnrichmentService.java.
  • Confirm the new methods compile by running:
./mvnw compile

You should see BUILD SUCCESS.

✔️ Awesome, I've got everything!

Your service now has all three methods. Make sure BookEnrichmentService.java is saved.

ⓧ I'd like to double check the full code

Your BookEnrichmentService.java should contain these sections, in order:

  • Package declaration and imports (including Map, Collectors, ThreadLocalRandom, Transactional, both DTO imports, and Book/BookRepository).
  • Class declaration with @Service and @Transactional(readOnly = true), the log and GENRES static fields, and the constructor.
  • The enrichBooks() method with @Transactional, the filter/map/collect pipeline, and timing log.
  • The getBookStatistics() method with groupingBy.
  • The searchBooks() method with case-insensitive filter and sorted.
  • The private enrichSingleBook() helper with Thread.sleep(1000).

Scroll up and compare each section against the code blocks you pasted.

  • Now add the remaining endpoints to BookEnrichmentController.java. Find the closing brace } of the enrichBooks method and add these methods below it, before the class closing brace:
    // Returns per-author book statistics
    @GetMapping("/statistics")
    public List<BookStatisticsDTO> getStatistics() {
        return enrichmentService.getBookStatistics();
    }

    // Searches books by title or author
    @GetMapping("/search")
    public List<EnrichedBookDTO> searchBooks(@RequestParam("q") String query) {
        return enrichmentService.searchBooks(query);
    }

What do these endpoints do?

  • GET /api/books/statistics returns per-author book counts and average page counts, powered by the groupingBy stream you just built.
  • GET /api/books/search?q=... filters books by a query string matching title or author, using the case-insensitive stream filter.
  • Save BookEnrichmentController.java.
  • Restart the app by running:
./mvnw spring-boot:run -Dspring-boot.run.profiles=postgres
  • Wait for the Started BookcatalogApplication message.
  • In the second terminal, test the statistics endpoint by running:
curl http://localhost:8080/api/books/statistics

You should see a JSON array with author statistics. Robert Martin should show a bookCount of 2 (for Clean Code and Clean Architecture).

  • Test the search endpoint by running:
curl http://localhost:8080/api/books/search?q=clean

You should see two books returned: Clean Architecture and Clean Code, sorted alphabetically by title.

✔️ Awesome, I've got everything!

Your controller has all three endpoints. Make sure BookEnrichmentController.java is saved.

ⓧ I'd like to double check the full code

Your BookEnrichmentController.java should contain these sections, in order:

  • Package declaration and imports (including GetMapping, PostMapping, RequestBody, RequestParam, both DTO imports, and BookEnrichmentService).
  • Class declaration with @RestController and @RequestMapping("/api/books"), the service field, and constructor.
  • POST /api/books/enrich endpoint.
  • GET /api/books/statistics endpoint.
  • GET /api/books/search endpoint.

Scroll up and compare each section against the code blocks you pasted.

Statistics or search returning an error?

  • If you see a 404, make sure the new methods in the controller are inside the class braces and that you restarted the app after saving.
  • If statistics returns an empty array, confirm your 5 books are still in the database by running curl http://localhost:8080/api/books. If the database was cleared by a restart, re-run the 5 curl POST commands from earlier.
  • If search returns nothing for clean, check that your book titles match exactly (Clean Code and Clean Architecture).
  • help me debug my statistics or search endpoint

Your Streams-powered service is complete. You can enrich books, group statistics by author, and search across your catalog. There's just one problem: enrichment takes 5+ seconds for 5 books. In the next step, you'll fix that bottleneck with parallel processing using CompletableFuture.

Fix the Slowness with CompletableFuture

You saw five one-second enrichment calls take more than five seconds because each call waited for the previous one. The database returned quickly, but the request thread spent most of its time waiting on simulated network work.

In this step, you will use CompletableFuture with a bounded ThreadPoolTaskExecutor. You will cut the happy-path response to about one second while adding backpressure, timeouts, and visible per-ISBN failures.

In this step, get ready to:
  • Configure a bounded executor that slows producers when the pool is saturated.
  • Return explicit success or failure details for every requested ISBN.
  • Verify parallel execution, timing, and missing-book handling.
Bound the enrichment executor

A fixed pool limits active threads, but its default queue can still grow without a practical bound. A bounded queue caps pending work, while CallerRunsPolicy makes the submitting request thread perform work when the pool and queue are full.

Why this executor?

Spring's ThreadPoolTaskExecutor wraps Java's thread-pool machinery and participates in the application lifecycle. Its core size, maximum size, and queue capacity create an explicit concurrency budget.

The caller-runs policy applies backpressure by slowing task submission instead of silently growing the queue or dropping work. The service stays bounded when traffic spikes.

  • Replace the contents of src/main/java/com/example/bookcatalog/config/AsyncConfig.java with this bounded executor configuration:
package com.example.bookcatalog.config;

import java.util.concurrent.ThreadPoolExecutor;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

// Gives enrichment work a bounded concurrency budget
@Configuration
public class AsyncConfig {

    @Bean(name = "enrichmentExecutor")
    public ThreadPoolTaskExecutor enrichmentExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(5);
        executor.setMaxPoolSize(10);
        executor.setQueueCapacity(25);
        executor.setThreadNamePrefix("enrichment-");
        // Slow callers instead of accumulating unlimited queued work
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        executor.setWaitForTasksToCompleteOnShutdown(true);
        executor.setAwaitTerminationSeconds(10);
        executor.initialize();
        return executor;
    }
}

What does this configuration control?

  • The core size of 5 keeps five worker threads ready for the common five-book request.
  • The maximum size of 10 allows short bursts after the queue reaches capacity.
  • The queue capacity of 25 limits pending enrichment tasks before backpressure begins.
  • The shutdown settings give accepted tasks up to ten seconds to finish when Spring closes the application context.
  • Save AsyncConfig.java.
  • Confirm the executor configuration compiles by running:
./mvnw compile

You should see BUILD SUCCESS. Spring can now manage and shut down the bounded executor with the rest of the application.

Executor configuration not compiling?

  • Check that ThreadPoolTaskExecutor comes from org.springframework.scheduling.concurrent.
  • Check that CallerRunsPolicy is created through ThreadPoolExecutor.
  • Help me debug my bounded Spring executor configuration.
Report failures without hiding them

Parallel work introduces partial failure. A request can contain a missing ISBN, a slow dependency, or one failed enrichment while the other items still succeed.

You will return one result per requested ISBN. Each result carries either an enriched book or a clear error, so the client never has to guess why an item disappeared.

  • Create src/main/java/com/example/bookcatalog/dto/EnrichmentBatchDTO.java and add the response contract below:
package com.example.bookcatalog.dto;

import java.util.List;

// Reports one outcome for every requested ISBN
public record EnrichmentBatchDTO(
        List<EnrichmentItem> items,
        long elapsedMs
) {
    public record EnrichmentItem(
            String isbn,
            EnrichedBookDTO book,
            String error
    ) {
        public static EnrichmentItem success(String isbn, EnrichedBookDTO book) {
            return new EnrichmentItem(isbn, book, null);
        }

        public static EnrichmentItem failure(String isbn, String error) {
            return new EnrichmentItem(isbn, null, error);
        }
    }
}

Why return an outcome for every ISBN?

The nested EnrichmentItem record preserves the input ISBN beside its result. A missing or failed item remains visible in the response instead of being filtered out as null.

The batch also reports elapsed milliseconds. That gives clients an observable performance signal without making them inspect server logs.

  • Save EnrichmentBatchDTO.java.
  • In BookEnrichmentService.java, add imports for CompletableFuture, Executor, TimeUnit, Function, Qualifier, and the new DTO.
  • Replace the repository field and constructor with this version:
    private final BookRepository bookRepository;
    // Runs enrichment without using the shared common pool
    private final Executor enrichmentExecutor;

    public BookEnrichmentService(
            BookRepository bookRepository,
            @Qualifier("enrichmentExecutor") Executor enrichmentExecutor) {
        this.bookRepository = bookRepository;
        this.enrichmentExecutor = enrichmentExecutor;
    }
  • Replace the existing enrichBooks() method with this batch implementation:
    public EnrichmentBatchDTO enrichBooks(List<String> isbns) {
        long start = System.nanoTime();
        // Keep the current full-table read so Step 4 can expose its cost
        Map<String, Book> booksByIsbn = bookRepository.findAll().stream()
                .collect(Collectors.toMap(Book::getIsbn, Function.identity()));

        List<CompletableFuture<EnrichmentItem>> futures = isbns.stream()
                .map(isbn -> enrichOne(isbn, booksByIsbn))
                .toList();
        List<EnrichmentItem> items = futures.stream()
                .map(CompletableFuture::join)
                .toList();

        long elapsed = (System.nanoTime() - start) / 1_000_000;
        log.info("Processed {} enrichment outcomes in {} ms", items.size(), elapsed);
        return new EnrichmentBatchDTO(items, elapsed);
    }

What changed in the batch method?

  • The books are indexed by ISBN once before asynchronous work begins. Worker threads read from that in-memory map instead of sharing a database transaction.
  • Each requested ISBN creates one CompletableFuture. Calling join() after every task is submitted waits for the group instead of serializing submission.
  • The batch keeps every outcome and records total elapsed time.
  • Add this helper directly below enrichBooks():
    private CompletableFuture<EnrichmentItem> enrichOne(
            String isbn, Map<String, Book> booksByIsbn) {
        Book book = booksByIsbn.get(isbn);
        if (book == null) {
            return CompletableFuture.completedFuture(
                    EnrichmentItem.failure(isbn, "Book not found"));
        }

        return CompletableFuture
                .supplyAsync(() -> enrichSingleBook(book), enrichmentExecutor)
                // Stop waiting indefinitely for a slow dependency
                .orTimeout(2, TimeUnit.SECONDS)
                .handle((result, error) -> error == null
                        ? EnrichmentItem.success(isbn, result)
                        : EnrichmentItem.failure(
                                isbn, "Enrichment timed out or failed"));
    }

How do the guardrails work?

Passing enrichmentExecutor to supplyAsync() keeps this work off Java's shared common pool. The bounded executor controls how many calls run or wait.

The orTimeout(2, TimeUnit.SECONDS) stage completes exceptionally if an item exceeds two seconds. The final handler converts that exception into an explicit failure item while preserving successful results.

  • In BookEnrichmentController.java, import EnrichmentBatchDTO.
  • Replace the current enrichment endpoint method with this return type:
    @PostMapping("/enrich")
    public EnrichmentBatchDTO enrichBooks(@RequestBody List<String> isbns) {
        return enrichmentService.enrichBooks(isbns);
    }
  • Save BookEnrichmentService.java and BookEnrichmentController.java.
  • Confirm the DTO, service, and controller compile together by running:
./mvnw compile

You should see BUILD SUCCESS. The endpoint now exposes partial failures as data instead of silently dropping them.

Seeing unresolved enrichment types?

  • Import com.example.bookcatalog.dto.EnrichmentBatchDTO in both the service and controller.
  • Import the nested type with com.example.bookcatalog.dto.EnrichmentBatchDTO.EnrichmentItem in the service.
  • Help me fix the imports for EnrichmentBatchDTO and its nested EnrichmentItem record.
Verify speed and failure reporting

The implementation now has three observable promises: valid books run in parallel, missing ISBNs stay visible, and slow calls cannot hold the batch open forever.

  • Restart the application with the PostgreSQL profile by running:
./mvnw spring-boot:run -Dspring-boot.run.profiles=postgres
  • Wait for Started BookcatalogApplication.
  • Open a second terminal for the verification request.

Before you run the request, do you expect the missing ISBN to disappear or return beside the five successful items?

  • Send five valid ISBNs and one missing ISBN by running:
curl -X POST http://localhost:8080/api/books/enrich -H "Content-Type: application/json" -d '["978-0132350884","978-0134685991","978-0135957059","978-0134757599","978-0134494166","missing-isbn"]'

You should see six items. Five contain an enriched book, while missing-isbn contains "error":"Book not found".

Check elapsedMs in the response and the enrichment- thread names in the app log. The five one-second calls should finish in roughly one second on five worker threads.

Still taking about five seconds?

  • Confirm supplyAsync() receives enrichmentExecutor as its second argument.
  • Confirm the app log shows several thread names beginning with enrichment-.
  • Help me diagnose why my CompletableFuture enrichment still runs sequentially.

That is the production lesson behind the speedup: concurrency needs limits, deadlines, and visible failures. Next, you will move search work into PostgreSQL and measure exactly how indexes change its execution plan.

Tune PostgreSQL Search Performance

Your enrichment work now runs in parallel, so the external-call bottleneck no longer dominates each request. The catalog search still loads every book into Java before filtering it.

That approach becomes expensive as the table grows. In this step, you will move the work into PostgreSQL, bound every response with pagination, and inspect the real execution plan before adding an index.

In this step, get ready to:
  • Move case-insensitive search from a Java stream into a paginated repository query.
  • Use EXPLAIN (ANALYZE, BUFFERS) to measure how PostgreSQL executes the search.
  • Apply trigram indexes through a versioned Flyway migration.
Move search into the repository

The existing searchBooks() method calls findAll() and filters the complete table in memory. A repository query lets the database filter rows before sending a bounded page back to Java.

  • Open BookRepository.java inside src/main/java/com/example/bookcatalog/repository/.
  • Add the paging imports and derived query method shown below:
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;

public interface BookRepository extends JpaRepository<Book, Long> {

    // Filters in PostgreSQL and returns only the requested page
    Page<Book> findByTitleContainingIgnoreCaseOrAuthorContainingIgnoreCase(
            String title, String author, Pageable pageable);
}

Why move the filter?

Spring Data derives a SQL LIKE query from the method name. Pageable adds a limit and offset, so PostgreSQL returns one page instead of the entire catalog.

A Page also includes total rows and total pages. That metadata makes the API predictable for clients.

  • Open BookEnrichmentService.java.
  • Replace the existing searchBooks(String query) method with this paginated version:
public Page<EnrichedBookDTO> searchBooks(String query, Pageable pageable) {
    // Keep filtering and pagination inside PostgreSQL
    return bookRepository
            .findByTitleContainingIgnoreCaseOrAuthorContainingIgnoreCase(
                    query, query, pageable)
            .map(book -> new EnrichedBookDTO(
                    book.getId(), book.getTitle(), book.getAuthor(),
                    book.getIsbn(), 0, "Unknown", LocalDateTime.now()));
}
  • Add imports for Page and Pageable from org.springframework.data.domain.
  • Open BookEnrichmentController.java.
  • Replace the existing search endpoint with this paginated endpoint:
// Returns a bounded page of database-filtered results
@GetMapping("/search")
public Page<EnrichedBookDTO> searchBooks(
        @RequestParam("q") String query,
        @PageableDefault(size = 20, sort = "title") Pageable pageable) {
    return enrichmentService.searchBooks(query, pageable);
}
  • Add imports for Page, Pageable, and PageableDefault.
  • Save all three Java files.
  • Confirm the repository, service, and controller compile by running:
./mvnw compile

You should see BUILD SUCCESS. The search path now asks PostgreSQL for one page instead of loading every row.

Seeing a paging compile error?

  • Check that Page and Pageable come from org.springframework.data.domain.
  • Check that the repository method receives the query twice because it searches both title and author.
  • Help me debug the paginated Spring Data repository query.
Measure the query plan

Five rows are too small for a useful planner comparison. You will add 10,000 synthetic books to the local primary database, then inspect the search before creating an index.

  • Seed 10,000 synthetic rows in the primary container by running:
docker compose -f docker-compose-psql.yml exec -T postgres-primary psql -U bookcatalog -d bookcatalog -c "INSERT INTO books (title, author, isbn) SELECT 'Performance Book ' || n, 'Load Author ' || (n % 100), 'PERF-' || n FROM generate_series(1, 10000) AS n;"

The command should end with INSERT 0 10000. Your table is now large enough for the planner to make a meaningful choice.

Before you inspect the plan, do you think PostgreSQL can use an ordinary B-tree index for a search pattern that starts with %?

  • Capture the baseline plan by running:
docker compose -f docker-compose-psql.yml exec -T postgres-primary psql -U bookcatalog -d bookcatalog -c "EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM books WHERE lower(title) LIKE '%performance book 9999%' OR lower(author) LIKE '%performance book 9999%' ORDER BY title LIMIT 20;"

You should see a plan containing Seq Scan and actual timing information. PostgreSQL examines the table because a leading-wildcard substring search cannot use a normal B-tree index effectively.

How do I read this plan?

  • The estimated cost and rows values show what the planner expected before execution.
  • The actual time and actual rows values show what happened when the query ran.
  • The Buffers line reports how many data pages PostgreSQL read from cache or storage.
Add versioned trigram indexes

PostgreSQL's pg_trgm extension supports indexed substring matching. You will add it through a versioned migration so the optimization can be reproduced in other environments.

  • Open pom.xml and add these dependencies inside <dependencies>:
<!-- Runs versioned database migrations at startup -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-flyway</artifactId>
</dependency>

<!-- Adds PostgreSQL support to Flyway -->
<dependency>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-database-postgresql</artifactId>
</dependency>
  • Create src/main/resources/db/migration/V2__add_search_indexes.sql.
  • Paste the migration below into the new file:
-- Enable indexed substring matching
CREATE EXTENSION IF NOT EXISTS pg_trgm;

-- Accelerate case-insensitive title and author searches
CREATE INDEX IF NOT EXISTS idx_books_title_trgm
    ON books USING gin (lower(title) gin_trgm_ops);
CREATE INDEX IF NOT EXISTS idx_books_author_trgm
    ON books USING gin (lower(author) gin_trgm_ops);

-- Refresh planner statistics after the index change
ANALYZE books;

Why use this index?

A GIN trigram index breaks text into short character groups. PostgreSQL can use those groups to narrow a LIKE '%value%' search before checking matching rows.

The tradeoff is extra storage and more work during inserts and updates. Indexes pay for faster reads by making writes slightly heavier.

  • Open application-postgres.properties.
  • Add the following Flyway connection and baseline settings:
# Run migrations against the existing primary schema
spring.flyway.url=jdbc:postgresql://localhost:5433/bookcatalog
spring.flyway.user=bookcatalog
spring.flyway.password=bookcatalog
spring.flyway.baseline-on-migrate=true
spring.flyway.baseline-version=1

Check the database target

Baselining accepts the current non-empty schema as version 1 before Flyway applies version 2. Confirm the URL uses local port 5433 so you do not migrate the wrong database.

  • Save pom.xml, the migration file, and application-postgres.properties.
  • Stop the running app with Ctrl+C.
  • Restart the app and apply the migration by running:
./mvnw spring-boot:run -Dspring-boot.run.profiles=postgres

The first startup can take a minute while Maven downloads Flyway. You should see Flyway create its schema history table and apply V2__add_search_indexes.sql.

  • Run the execution plan again from your second terminal:
docker compose -f docker-compose-psql.yml exec -T postgres-primary psql -U bookcatalog -d bookcatalog -c "EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM books WHERE lower(title) LIKE '%performance book 9999%' OR lower(author) LIKE '%performance book 9999%' ORDER BY title LIMIT 20;"

You should now see bitmap index activity involving idx_books_title_trgm or idx_books_author_trgm. Compare the actual time and buffer counts with your baseline.

  • Verify the paginated endpoint by running:
curl "http://localhost:8080/api/books/search?q=performance&page=0&size=5&sort=title,asc"

You should see five results inside the content array plus page metadata. That response proves filtering, indexing, and pagination now work together.

Still seeing a sequential scan?

  • Confirm the migration succeeded by checking startup logs for Flyway version 2.
  • Run ANALYZE books again if the table changed after the migration.
  • Remember that PostgreSQL may still choose a sequential scan when a query matches a large percentage of the table. That can be the cheaper plan.
  • Help me interpret my PostgreSQL query plan.

Your search now scales at the database boundary instead of pulling the whole catalog into memory. Next, you will route those bounded read queries separately from writes.

Implement Read/Write Splitting with AbstractRoutingDataSource

Your search now filters and paginates inside PostgreSQL, and the planner can use trigram indexes for substring matching. The next scaling pressure is read traffic competing with writes on one connection pool.

You will route read-only transactions through a second datasource with AbstractRoutingDataSource. The second container is an independent simulated read target, so this step also exposes the stale-read problem that real replication must solve.

In this step, get ready to:
  • Create a schema-and-data snapshot in the simulated read target.
  • Route transactions through primary and read-only datasource pools.
  • Observe write routing, read routing, and a stale read.
Prepare the simulated read target

The two Docker containers do not replicate data. You will initialize the second database from the same migrations and copy a point-in-time snapshot from the primary before testing the routing layer.

What is simulated here?

A real PostgreSQL read replica continuously replays changes from the primary. This project deliberately uses an independent database so you can focus on Java routing without adding replication administration.

The snapshot gives both databases the same starting state. Any later write appears only on the primary, which lets you see why applications must account for replica lag.

  • Copy the baseline migration into the read-target container by running:
docker cp src/main/resources/db/migration/V1__create_books.sql postgres-replica:/tmp/V1__create_books.sql
docker exec postgres-replica psql -U bookcatalog -d bookcatalog -f /tmp/V1__create_books.sql
  • Copy the search-index migration into the same container by running:
docker cp src/main/resources/db/migration/V2__add_search_indexes.sql postgres-replica:/tmp/V2__add_search_indexes.sql
docker exec postgres-replica psql -U bookcatalog -d bookcatalog -f /tmp/V2__add_search_indexes.sql
  • Clear old read-target rows before loading the snapshot by running:
docker exec postgres-replica psql -U bookcatalog -d bookcatalog -c "TRUNCATE TABLE books RESTART IDENTITY;"
  • Copy the primary's books rows and sequence state into the read target by running:
docker exec postgres-primary pg_dump -U bookcatalog -d bookcatalog --data-only --table=books | docker exec -i postgres-replica psql -X -U bookcatalog -d bookcatalog
  • Compare row counts in both databases by running:
docker exec postgres-primary psql -U bookcatalog -d bookcatalog -c "SELECT count(*) FROM books;"
docker exec postgres-replica psql -U bookcatalog -d bookcatalog -c "SELECT count(*) FROM books;"

You should see the same count twice. The read target now holds a snapshot, but future primary writes still do not flow across automatically.

Snapshot copy failing?

  • Confirm both containers are running with docker ps.
  • If the restore reports duplicate keys, rerun the TRUNCATE TABLE command before repeating the dump.
  • Help me debug the primary-to-read-target snapshot.
Build the transaction-aware router

Spring binds transaction metadata to the request thread. The router reads that metadata when JPA asks for a connection, then selects the matching pool.

  • Create src/main/java/com/example/bookcatalog/config/DataSourceType.java with the two routing keys:
package com.example.bookcatalog.config;

// Names the writable and read-only connection targets
public enum DataSourceType {
    READ_WRITE,
    READ_ONLY
}
  • Create src/main/java/com/example/bookcatalog/config/TransactionRoutingDataSource.java with the routing decision:
package com.example.bookcatalog.config;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;
import org.springframework.transaction.support.TransactionSynchronizationManager;

// Chooses a pool from the current transaction's read-only flag
public class TransactionRoutingDataSource extends AbstractRoutingDataSource {
    private static final Logger log = LoggerFactory.getLogger(
            TransactionRoutingDataSource.class);

    @Override
    protected Object determineCurrentLookupKey() {
        boolean readOnly = TransactionSynchronizationManager
                .isCurrentTransactionReadOnly();
        DataSourceType target = readOnly
                ? DataSourceType.READ_ONLY
                : DataSourceType.READ_WRITE;
        log.info("Routing to: {}", target);
        return target;
    }
}

How does the lookup key work?

The router returns READ_ONLY when the active transaction is marked read-only. Every other connection request falls back to READ_WRITE.

The log line makes each decision observable. That signal becomes the proof for both manual checks and integration tests.

  • Create src/main/java/com/example/bookcatalog/config/DataSourceConfig.java with these imports and property beans:
package com.example.bookcatalog.config;

import java.util.Map;
import javax.sql.DataSource;

import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.jdbc.autoconfigure.DataSourceProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;
import org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy;

@Configuration
@Profile("postgres")
public class DataSourceConfig {

    @Bean("primaryProperties")
    @ConfigurationProperties("spring.datasource.primary")
    DataSourceProperties primaryProperties() {
        return new DataSourceProperties();
    }

    @Bean("replicaProperties")
    @ConfigurationProperties("spring.datasource.replica")
    DataSourceProperties replicaProperties() {
        return new DataSourceProperties();
    }
}
  • Add the two physical datasource beans before the closing brace:
    @Bean("primaryDataSource")
    DataSource primaryDataSource(
            @Qualifier("primaryProperties") DataSourceProperties properties) {
        // Translates url into the pool-specific JDBC URL
        return properties.initializeDataSourceBuilder().build();
    }

    @Bean("replicaDataSource")
    DataSource replicaDataSource(
            @Qualifier("replicaProperties") DataSourceProperties properties) {
        return properties.initializeDataSourceBuilder().build();
    }
  • Add the primary routing datasource below the physical datasource beans:
    @Bean
    @Primary
    DataSource dataSource(
            @Qualifier("primaryDataSource") DataSource primary,
            @Qualifier("replicaDataSource") DataSource replica) {
        TransactionRoutingDataSource router =
                new TransactionRoutingDataSource();
        router.setTargetDataSources(Map.of(
                DataSourceType.READ_WRITE, primary,
                DataSourceType.READ_ONLY, replica));
        router.setDefaultTargetDataSource(primary);
        router.afterPropertiesSet();

        // Delay connection selection until transaction metadata is available
        return new LazyConnectionDataSourceProxy(router);
    }

Why use properties and a lazy proxy?

DataSourceProperties handles the translation from each configured url to the pool-specific JDBC URL. This avoids brittle direct property binding against HikariCP.

LazyConnectionDataSourceProxy delays physical connection acquisition until the transaction's read-only flag is available. Without that delay, routing can happen too early.

  • Save all three routing files.
  • Confirm the routing configuration compiles by running:
./mvnw compile

You should see BUILD SUCCESS.

Routing configuration not compiling?

  • Use the Spring Boot 4 package org.springframework.boot.jdbc.autoconfigure.DataSourceProperties.
  • Confirm dataSource() carries @Primary.
  • Help me debug my Spring Boot 4 routing datasource configuration.
Observe routing and stale reads

The routing decision depends on transaction semantics. Repository save operations use writable transactions, while the enrichment service's class-level read-only transaction sends search traffic to the simulated read target.

  • Restart the application with the PostgreSQL profile by running:
./mvnw spring-boot:run -Dspring-boot.run.profiles=postgres

Before you create a book, where do you expect the write to route?

  • Create one primary-only book by running:
curl -X POST http://localhost:8080/api/books -H "Content-Type: application/json" -d '{"title":"Routing in Practice","author":"NextWork","isbn":"route-001"}'

You should see Routing to: READ_WRITE in the app log. The response confirms that the primary accepted the new row.

Before you search, do you expect the snapshot-based read target to contain that new book?

  • Search for the new title through the read-only service by running:
curl "http://localhost:8080/api/books/search?q=Routing&page=0&size=20"

You should see Routing to: READ_ONLY in the log and an empty page in the response. That stale read is expected because the second container has no continuous replication.

How would production differ?

In production, PostgreSQL streaming replication or a managed database service keeps replicas updated. Applications still account for replication lag when a user writes data and reads it immediately.

Common strategies include reading from the primary for a short period after a write, attaching a consistency token, or accepting eventual consistency for non-critical views. The correct choice depends on the product's consistency requirement.

🙋‍♀️ Not seeing both routing decisions?

  • Confirm the app log contains both READ_WRITE and READ_ONLY decisions.
  • If every request routes to READ_WRITE, verify that the search service runs inside @Transactional(readOnly = true).
  • Help me debug a router that never selects READ_ONLY.

You now have application-level read/write routing and a concrete example of eventual consistency. Next, you will test the asynchronous pipeline, optimized search, Flyway migrations, and routing against disposable PostgreSQL infrastructure.

Write Integration Tests with Testcontainers

Your enrichment pipeline processes books in parallel, and your routing logic directs reads to the replica while writes hit the primary. You have verified both patterns against the Docker Compose PostgreSQL containers.

The question is: how do you prove these patterns hold when you ship changes? Unit tests with mocks skip the real database entirely, so they never catch SQL dialect mismatches, connection pool issues, or routing bugs. Testcontainers solves this by spinning up real PostgreSQL containers for each test run and tearing them down automatically.

In this step, get ready to:
  • Create a reusable AbstractIntegrationTest base class with a Testcontainers PostgreSQL container.
  • Write integration tests verifying parallel enrichment, stream transformations, and author statistics.
  • Run the test suite and confirm all tests pass against real PostgreSQL.
Create the AbstractIntegrationTest base class

Every integration test in this project needs a running PostgreSQL container. Instead of repeating that setup in every test class, you will create a shared base class that handles the container lifecycle once.

  • Create a new file AbstractIntegrationTest.java in src/test/java/com/example/bookcatalog/ and paste the following code:
package com.example.bookcatalog;

import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.containers.PostgreSQLContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;

@SpringBootTest
@Testcontainers
public abstract class AbstractIntegrationTest {

    @Container
    static final PostgreSQLContainer<?> POSTGRES =
            new PostgreSQLContainer<>("postgres:16-alpine")
                    .withDatabaseName("bookcatalog_test")
                    .withUsername("test")
                    .withPassword("test");
}

What does this code do?

@SpringBootTest loads the full Spring application context, giving your tests access to all beans (services, repositories, config).

@Testcontainers activates the JUnit 5 extension that manages container lifecycles. The @Container annotation on the static field tells the extension to start this PostgreSQL container once before all tests and stop it after the last test finishes.

  • Next, add the @DynamicPropertySource method inside the class, just below the POSTGRES field and above the closing }:
    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.primary.url", POSTGRES::getJdbcUrl);
        registry.add("spring.datasource.primary.username", POSTGRES::getUsername);
        registry.add("spring.datasource.primary.password", POSTGRES::getPassword);
        registry.add("spring.datasource.primary.driver-class-name", () -> "org.postgresql.Driver");
        registry.add("spring.datasource.replica.url", POSTGRES::getJdbcUrl);
        registry.add("spring.datasource.replica.username", POSTGRES::getUsername);
        registry.add("spring.datasource.replica.password", POSTGRES::getPassword);
        registry.add("spring.datasource.replica.driver-class-name", () -> "org.postgresql.Driver");
        registry.add("spring.jpa.hibernate.ddl-auto", () -> "create-drop");
        registry.add("spring.h2.console.enabled", () -> "false");
    }

Why @DynamicPropertySource instead of @ServiceConnection?

Spring Boot's @ServiceConnection auto-wires a single spring.datasource.* set of properties. Your routing config needs two separate prefixes: spring.datasource.primary.* and spring.datasource.replica.*. @DynamicPropertySource gives you the flexibility to map both prefixes to the same Testcontainers PostgreSQL URL.

Both primary and replica point to the same container because you are testing routing logic, not actual PostgreSQL replication. The create-drop DDL mode ensures a fresh schema for every test run.

  • Save AbstractIntegrationTest.java.
  • Confirm the file exists by running:
ls src/test/java/com/example/bookcatalog/

You should see AbstractIntegrationTest.java listed alongside any existing test files.

Don't see the file?

Make sure you created it in src/test/java (the test source root), not src/main/java. The package path must be com/example/bookcatalog.

Help me set up my AbstractIntegrationTest base class for Testcontainers

Create the BookEnrichmentIntegrationTest class

Now you will write the actual test class that exercises your enrichment service, statistics grouping, and search functionality against the real PostgreSQL container.

  • Create a new file BookEnrichmentIntegrationTest.java in src/test/java/com/example/bookcatalog/ and paste the class skeleton:
package com.example.bookcatalog;

import java.util.List;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;

import com.example.bookcatalog.dto.BookStatisticsDTO;
import com.example.bookcatalog.dto.EnrichedBookDTO;
import com.example.bookcatalog.model.Book;
import com.example.bookcatalog.repository.BookRepository;
import com.example.bookcatalog.service.BookEnrichmentService;

import static org.junit.jupiter.api.Assertions.*;

class BookEnrichmentIntegrationTest extends AbstractIntegrationTest {

    @Autowired
    private BookEnrichmentService enrichmentService;

    @Autowired
    private BookRepository bookRepository;
}

What does this class skeleton do?

By extending AbstractIntegrationTest, this class inherits the @SpringBootTest, @Testcontainers, and the shared PostgreSQL container. No extra setup needed.

The @Autowired fields inject the real BookEnrichmentService and BookRepository beans from the Spring context. Tests will call these services against the real database, not mocks.

  • Next, add the test data seeding method and helper inside the class, above the closing }:
    @BeforeEach
    void setUp() {
        bookRepository.deleteAll();

        bookRepository.save(createBook("Clean Code", "Robert Martin", "978-0132350884"));
        bookRepository.save(createBook("Effective Java", "Joshua Bloch", "978-0134685991"));
        bookRepository.save(createBook("The Pragmatic Programmer", "David Thomas", "978-0135957059"));
        bookRepository.save(createBook("Refactoring", "Martin Fowler", "978-0134757599"));
        bookRepository.save(createBook("Clean Architecture", "Robert Martin", "978-0134494166"));
    }

    private Book createBook(String title, String author, String isbn) {
        Book book = new Book();
        book.setTitle(title);
        book.setAuthor(author);
        book.setIsbn(isbn);
        return book;
    }

What does @BeforeEach do here?

Before every test method, deleteAll() wipes the database clean. Then five books are seeded with known titles, authors, and ISBNs. This guarantees each test starts from an identical, predictable state.

Notice that Robert Martin has two books ("Clean Code" and "Clean Architecture"). The statistics test will verify that grouping works correctly by checking his book count.

  • Save BookEnrichmentIntegrationTest.java.
  • Confirm both test files exist by running:
ls src/test/java/com/example/bookcatalog/

You should see both AbstractIntegrationTest.java and BookEnrichmentIntegrationTest.java listed.

Add the enrichment test and run the suite

Time to write the test that proves your parallel enrichment works against real PostgreSQL. This test calls enrichBooks() with 3 ISBNs and asserts both correctness and speed.

  • In BookEnrichmentIntegrationTest.java, add the following test method above the createBook method:
    @Test
    void enrichBooks_shouldReturnEnrichedDTOs_inParallel() {
        List<String> isbns = List.of("978-0132350884", "978-0134685991", "978-0135957059");

        long start = System.nanoTime();
        List<EnrichedBookDTO> results = enrichmentService.enrichBooks(isbns);
        long elapsed = (System.nanoTime() - start) / 1_000_000;

        assertEquals(3, results.size());
        results.forEach(dto -> {
            assertNotNull(dto.genre());
            assertTrue(dto.pageCount() > 0);
            assertNotNull(dto.enrichedAt());
        });
        assertTrue(elapsed < 3000, "Parallel enrichment should complete in under 3 seconds, took " + elapsed + "ms");
    }

What does this test prove?

The test passes 3 ISBNs to enrichBooks() and asserts that all 3 come back with a non-null genre, a positive page count, and an enrichment timestamp.

The timing assertion (elapsed < 3000) is the key. If enrichment ran sequentially, 3 books would take 3+ seconds. With CompletableFuture parallelism, all 3 run simultaneously and finish in about 1 second.

  • Save BookEnrichmentIntegrationTest.java.

Before you run the tests, what do you think will happen? Will Testcontainers start a PostgreSQL container automatically, or will the tests try to connect to your Docker Compose containers?

  • Run the test suite by executing:
./mvnw test

Testcontainers starts its own PostgreSQL container on a random port. You should see log output showing the container starting, the Spring context loading, and 1 test passing.

What should I see?

Look for these key lines in the output:

  • Container postgres:16-alpine is starting followed by Container postgres:16-alpine started confirms Testcontainers spun up a fresh PostgreSQL instance.
  • Routing to: READ_WRITE in the logs confirms your datasource routing is active during the enrichment call.
  • Tests run: 1, Failures: 0, Errors: 0 confirms the enrichment test passed.

🙋‍♀️ Tests failing to start?

Make sure Docker Desktop is running. Testcontainers requires Docker to pull and start the postgres:16-alpine image.

If you see a connection refused error, check that your @DynamicPropertySource method sets both spring.datasource.primary.* and spring.datasource.replica.* properties. Missing either prefix means DataSourceConfig cannot build one of the datasources.

Help me debug my Testcontainers integration test setup

Add statistics and search tests

Your enrichment test proves the parallel pipeline works. Now add two more tests that verify your Streams API transformations: author grouping and case-insensitive search.

  • In BookEnrichmentIntegrationTest.java, add the following two test methods below enrichBooks_shouldReturnEnrichedDTOs_inParallel:
    @Test
    void getBookStatistics_shouldGroupByAuthor() {
        List<BookStatisticsDTO> stats = enrichmentService.getBookStatistics();

        assertFalse(stats.isEmpty());
        BookStatisticsDTO robertMartin = stats.stream()
                .filter(s -> s.authorName().equals("Robert Martin"))
                .findFirst()
                .orElseThrow();
        assertEquals(2, robertMartin.bookCount());
    }

    @Test
    void searchBooks_shouldFilterByTitleOrAuthor() {
        List<EnrichedBookDTO> results = enrichmentService.searchBooks("clean");

        assertEquals(2, results.size());
        assertTrue(results.get(0).title().compareToIgnoreCase(results.get(1).title()) <= 0);
    }

What do these tests verify?

getBookStatistics_shouldGroupByAuthor calls getBookStatistics() and drills into Robert Martin's entry. Since the @BeforeEach seeds two Robert Martin books, bookCount() must equal 2.

searchBooks_shouldFilterByTitleOrAuthor searches for "clean" and expects 2 results ("Clean Architecture" and "Clean Code"). The assertion also verifies that results are sorted alphabetically by title.

  • Save BookEnrichmentIntegrationTest.java.

Before running the full suite, how many tests do you expect to pass now?

  • Run the full test suite again by executing:
./mvnw test

You should see 3 tests pass this time. The Testcontainers container starts once, all 3 tests run against it, and then the container shuts down automatically.

What about the routing logs?

Scroll through the test output and you will see Routing to: READ_WRITE for the enrichBooks call (it uses @Transactional) and Routing to: READ_ONLY for getBookStatistics and searchBooks (they use @Transactional(readOnly = true)). This confirms your routing logic works correctly even under a Testcontainers environment.

✔️ Awesome, I've got everything!

Great. Double check you have saved both files.

ⓧ I'd like to double check the full code

Your AbstractIntegrationTest.java should contain the class declaration with the POSTGRES container field and the configureProperties method, in that order.

Your BookEnrichmentIntegrationTest.java should contain these sections in order:

  • Imports and class declaration extending AbstractIntegrationTest.
  • @Autowired fields for enrichmentService and bookRepository.
  • The @BeforeEach setUp() method with 5 book saves.
  • The enrichBooks_shouldReturnEnrichedDTOs_inParallel test with timing assertion.
  • The getBookStatistics_shouldGroupByAuthor test.
  • The searchBooks_shouldFilterByTitleOrAuthor test.
  • The createBook helper method.

Scroll up to each chunk to confirm your code matches.

Seeing test failures?

If getBookStatistics_shouldGroupByAuthor fails, check that your @BeforeEach seeds exactly two books with the author "Robert Martin" ("Clean Code" and "Clean Architecture").

If searchBooks_shouldFilterByTitleOrAuthor fails, confirm the search query is "clean" (lowercase). The searchBooks method does a case-insensitive match, so it should find both "Clean Code" and "Clean Architecture".

Help me debug my BookEnrichmentIntegrationTest failures

Your enrichment pipeline, stream transformations, and datasource routing are all verified against real PostgreSQL. No mocks, no in-memory substitutes. You now have a production-patterned application backed by real integration tests.

Secret mission

Concurrent Stress Test Under Load

Your enrichment pipeline handles 5 books in parallel. But can it survive 50 simultaneous requests hammering the thread pool and connection pool at once? Build a stress test using ExecutorService and CountDownLatch that proves your infrastructure holds up under real concurrency pressure.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally using Docker containers, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Two PostgreSQL 16 Docker containers (postgres-primary on port 5433 and postgres-replica on port 5434) running via Docker Compose.
  • Two Docker volumes (primary-data and replica-data) storing PostgreSQL data on your machine.

Keep everything running

No action needed. Choose this if you are still actively developing the bookcatalog app. The containers use minimal resources when idle.

  • Your postgres-primary and postgres-replica containers continue running in the background.
  • Restart both containers from the bookcatalog directory that contains docker-compose-psql.yml by running:
docker compose -f docker-compose-psql.yml up -d

Pause - I'll come back to this later

Stop the containers to free up memory, but keep your data volumes intact so you can resume later.

  • In a terminal, move to your bookcatalog directory, which contains docker-compose-psql.yml.
  • Stop both containers by running:
docker compose -f docker-compose-psql.yml stop
  • When you are ready to come back, resume the containers by running:
docker compose -f docker-compose-psql.yml start

Your data volumes are preserved, so any books you created will still be there.

Delete - I don't want to use this again

Remove both containers and their data volumes entirely. Your application source code is unaffected.

  • In a terminal, move to your bookcatalog directory, which contains docker-compose-psql.yml.
  • Remove both PostgreSQL containers and their data volumes by running:
docker compose -f docker-compose-psql.yml down -v
  • Confirm the containers are gone by running:
docker ps -a

You should no longer see postgres-primary or postgres-replica in the list. Your Spring Boot application code remains on your machine.

  • Create fresh containers from docker-compose-psql.yml by running:
docker compose -f docker-compose-psql.yml up -d

Nice Work!

Nice Work!

You just built a production-grade book catalog enrichment engine on top of your existing Spring Boot CRUD app. Your enrichment pipeline processes books in parallel, your database queries route intelligently between primary and replica, and your test suite runs against real infrastructure that spins up and tears down automatically.

You've learned how to:

  • Build a Streams API enrichment pipeline using filter, map, collect, and groupingBy, then parallelize it with CompletableFuture and a custom ExecutorService to cut enrichment time from 5+ seconds to roughly 1 second.
  • Implement read/write splitting using AbstractRoutingDataSource and LazyConnectionDataSourceProxy, routing @Transactional(readOnly = true) queries to a replica PostgreSQL and writes to the primary.
  • Write integration tests with Testcontainers and @DynamicPropertySource that verify enrichment correctness, stream transformations, and datasource routing against real PostgreSQL containers that spin up and tear down automatically.
  • Secret Mission: Built a concurrent stress test that fired 50 simultaneous enrichment requests using CountDownLatch and proved your thread pool, connection pool, and routing logic hold up under load.

Ready to quiz yourself?