Design GitHub's Repository Storage

Design a distributed storage system handling 630M+ Git repositories.

Introduction

30 Second Summary

Every time you push code to GitHub, your changes land safely on servers that collectively store over 630 million repositories. What happens when one of those servers crashes mid-push?

In this project, you will design a distributed storage system for GitHub's repository layer. You will follow the structured system design interview approach: define requirements, estimate capacity, build a naive architecture, find its flaws, and iteratively improve it.

What You'll Build

Imagine pulling up your GitHub repo in a system design interview and walking through rendered architecture diagrams that show exactly how 630 million repositories are stored, replicated, and recovered from failures.

By the end of this project, you'll have:

  • A polished design document on GitHub with rendered Mermaid architecture diagrams. You can show someone the full journey from a naive single-server design to a partitioned, replicated storage fleet handling 630M+ repositories.
  • A clear distinction between verified GitHub facts and your own design assumptions for every number in the document. You can tell an interviewer exactly where each value came from and why you chose it.
  • A concrete failure recovery walkthrough you can narrate to an interviewer. You can trace each phase from crash detection through zero-downtime rerouting to automatic re-replication.
  • Secret Mission: Design fork network deduplication using Git alternates to avoid storing thousands of redundant copies of popular repositories like linux/kernel.

Are there any prerequisites?

This project assumes familiarity with basic system design concepts like load balancing, caching, and databases. You should also know the basics of Git: clone, push, and pull.

Before We Start

Before we dive in, let's lock in what you're about to design and why the scale makes it such a fascinating challenge.

Set Up Your Design Workspace

Your system design document needs a home where you can write Markdown, preview architecture diagrams live, and share the finished result as a rendered page.

In this step, you'll create a local Git repository, write a skeleton document with placeholder sections, and push it to GitHub. Your architecture diagrams use Mermaid, a text-based diagramming language that GitHub renders as real visuals right inside your README.

In this step, get ready to:
  • Verify Git and create a local project repository.
  • Push a skeleton design document to GitHub and confirm Mermaid rendering.
  • Set up side-by-side Markdown and Mermaid preview in VS Code.
Verify Git and create the project

Every system design document you write in this project lives in a single README.md file tracked by Git. You need a project directory, a Git repository, and a skeleton document before you can start designing.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Confirm Git is ready by running this command:
git --version

You should see a version number in the output. The exact version does not matter for this project.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project directory and initialize a Git repository by running these commands:
mkdir github-storage-design
cd github-storage-design
git init

You should see Initialized empty Git repository in the output.

Don't see the initialized message?

Make sure you ran cd ~/Desktop first. If the mkdir command says the directory already exists, delete it with rm -rf github-storage-design and try again.

Help me troubleshoot my git init

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code and press Enter to open it.

VS Code opens to a Welcome tab or an empty editor.

  • Click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the github-storage-design folder, and click Open.

VS Code reloads with your project as the workspace root. The Explorer sidebar on the left shows the empty github-storage-design folder.

  • In the Explorer sidebar, hover over the project name to reveal the file action icons.
  • Click the New File icon (the page with a plus sign).
  • Type README.md and press Enter to create the file.
  • Copy the following document header and Table of Contents into README.md:
# Designing GitHub's Repository Storage System

A system design document for a distributed storage system that handles 630M+ Git repositories.

## Table of Contents

- [1. Requirements](#1-requirements)
- [2. Capacity Estimation](#2-capacity-estimation)
- [3. Storage Layer (Naive Design)](#3-storage-layer-naive-design)
- [4. Partitioning and Replication](#4-partitioning-and-replication)
- [5. Routing and Metadata Layer](#5-routing-and-metadata-layer)
- [6. Summary](#6-summary)

What does this set up?

  • The title declares this as a system design document for GitHub's repository storage.
  • The Table of Contents links to six sections you'll fill in throughout the project: Requirements, Capacity Estimation, Storage Layer, Partitioning, Routing, and Summary.
  • Below the Table of Contents in README.md, add the section placeholders and a test Mermaid diagram by pasting this content:

## 1. Requirements

<!-- TODO -->

## 2. Capacity Estimation

<!-- TODO -->

## 3. Storage Layer (Naive Design)

<!-- TODO -->

## 4. Partitioning and Replication

<!-- TODO -->

## 5. Routing and Metadata Layer

<!-- TODO -->

## 6. Summary

<!-- TODO -->

```mermaid
flowchart TD
    A[Test Node] --> B[Mermaid Works]
```

What does the test diagram do?

The six <!-- TODO --> placeholders mark sections you'll replace with real content in later steps.

The Mermaid code fence at the bottom is a quick test. If GitHub and VS Code can render this simple flowchart, they'll render the full architecture diagrams you build later.

  • Save README.md by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Open VS Code's integrated terminal by clicking Terminal in the top menu bar, then clicking New Terminal.
  • Stage and commit your skeleton document by running these commands:
git add README.md
git commit -m "Add skeleton design document"

You should see a commit summary showing 1 file changed with several insertions.

Seeing a Git identity error?

If Git asks you to set your name and email, run these two commands (replacing the placeholders with your own details):

git config --global user.name "Your Name"
git config --global user.email "your@email.com"

Then run the git commit command again.

Help me fix my Git commit error

✔️ Awesome, I've got everything!

Great. Double check that you saved README.md and that your commit succeeded.

ⓧ I'd like to double check the full code

Scroll up and confirm your README.md contains these sections in order:

  • The document title (# Designing GitHub's Repository Storage System) and a one-line description.
  • A Table of Contents with links to six sections.
  • Six section headings (Requirements through Summary), each with a <!-- TODO --> placeholder.
  • A Mermaid test diagram at the very bottom with A[Test Node] --> B[Mermaid Works].
Create a GitHub repo and verify Mermaid rendering

Your skeleton document is committed locally. Now push it to GitHub so the Mermaid test diagram renders as a real visual.

  • Open your browser and go to github.com/new.
  • Enter github-storage-design in the Repository name field.
  • Leave the repository as Public (or choose Private if you prefer).
  • Do not check Add a README file since you already have one locally.
  • Click Create repository.

GitHub shows a quick setup page with the remote URL for your new repository.

  • Copy the HTTPS URL shown on the quick setup page.
  • Switch back to VS Code's integrated terminal.
  • Connect your local repository to GitHub and push your document by running these commands:
git remote add origin [[YOUR_REPO_URL="https://github.com/your-username/github-storage-design.git"]]
git branch -M main
git push -u origin main

What do these commands do?

  • git remote add origin connects your local repo to the GitHub repo you just created.
  • git branch -M main renames the default branch to main.
  • git push -u origin main uploads your commits to GitHub and sets main as the default remote branch.

Push failed or asked for credentials?

GitHub requires authentication for pushing over HTTPS. If prompted, enter your GitHub username and a personal access token (not your password).

You can create a personal access token at github.com/settings/tokens. Select the repo scope when creating it.

Help me push to GitHub

Before you check GitHub, what do you think the Mermaid code block will look like on the page? Will it show as raw text, or will it render as a diagram?

  • Go to your GitHub repository page in your browser (the URL is github.com/your-username/github-storage-design).
  • Scroll down to the bottom of the README.

The test flowchart renders as a visual diagram with two boxes connected by an arrow. GitHub natively renders Mermaid diagrams inside fenced code blocks.

Still seeing raw code instead of a diagram?

Make sure the code fence in your README.md starts with exactly ```mermaid (three backticks followed by the word mermaid). A typo or extra space will prevent GitHub from recognizing it.

Try a hard refresh in your browser (Cmd+Shift+R on macOS, Ctrl+Shift+R on Windows).

Help me fix Mermaid rendering on GitHub

Set up side-by-side editing

For the rest of this project, you'll edit your design document on the left and see your Mermaid diagrams render in real time on the right. VS Code 1.121+ has built-in Mermaid rendering in its Markdown preview, so no extensions are needed.

  • Make sure README.md is open in VS Code's editor.
  • Press Cmd+K, release both keys, then press V (macOS) or Ctrl+K, release both keys, then press V (Windows) to open a side-by-side Markdown preview.

A preview pane opens to the right of your editor. Scroll down to the bottom of the preview.

You should see the test flowchart rendered as two boxes (Test Node and Mermaid Works) connected by an arrow.

Why side-by-side editing?

You'll be writing Markdown in the left pane and watching your Mermaid architecture diagrams render instantly in the right pane. Every time you save, the preview updates. This tight feedback loop is especially useful when you're tweaking diagram layouts in later steps.

Don't see the diagram in the preview?

Make sure you saved README.md before opening the preview.

If you pressed Cmd+Shift+V instead, that opens the preview in the same pane (replacing the editor). Close it and use Cmd+K then V for side-by-side.

If your VS Code version is below 1.121, update VS Code. Older versions require a separate extension for Mermaid rendering.

Help me set up Mermaid preview in VS Code

Your workspace is ready. You have a local Git repo, a skeleton document on GitHub with a working Mermaid diagram, and a side-by-side editor for live previewing. Next up, you'll fill in the requirements and run the capacity math that drives every design decision.

Define Requirements and Estimate Capacity

Your design workspace is ready and Mermaid diagrams are rendering on GitHub. Now it is time to start filling in the actual design.

Most candidates in system design interviews jump straight to drawing boxes and arrows. Starting with requirements and capacity math first shows the interviewer you think about scale constraints before designing. It also gives you concrete numbers to reference when you make architecture decisions later.

In this step, get ready to:
  • Define functional and non-functional requirements with source labels distinguishing GitHub facts from your own design assumptions.
  • Estimate storage capacity using back-of-envelope math from your own assumptions.
  • Estimate traffic with a read/write breakdown derived from GitHub's published ratio.
Update the document header with a fact-vs-assumption disclaimer

The key pedagogical thread of this entire document is separating what GitHub actually does from what you are choosing for your design. An interviewer who sees you make that distinction knows you can defend your numbers.

  • In README.md, find the subtitle line near the top of the file:
A system design document for a distributed storage system that handles 630M+ Git repositories.
  • Replace that entire line with the updated subtitle, disclaimer blockquote, and a horizontal rule, so it looks like this:
A system design document for a distributed storage system that handles 630M+ Git repositories with high durability, availability, and performance.

> Throughout this document, each value is labeled as either a **GitHub fact** (from an official GitHub source) or a **Design assumption** (a value I calculated or chose, with rationale). This separation lets me defend every number in an interview.

---
  • Scroll down past the Table of Contents list. Add a --- horizontal rule on a new line directly after the last TOC entry (- [6. Summary](...)).
  • Save README.md and check your VS Code preview pane.

You should see the blockquote rendered with a left border, and clean horizontal lines separating the header from the TOC and the TOC from the document body.

Define functional and non-functional requirements

In a real interview, the requirements section answers: what must this system do, and what quality bars must it hit? The functional requirements are straightforward. The non-functional requirements are where the fact/assumption distinction starts to matter.

  • In README.md, find the ## 1. Requirements section. Delete the <!-- TODO --> placeholder below it.
  • Replace it with the following requirements tables:
### Functional Requirements

| ID | Requirement |
|----|-------------|
| FR-1 | Push new commits to a repository (git push) |
| FR-2 | Clone or fetch a repository (git clone, git pull, git fetch) |
| FR-3 | Create a new repository |
| FR-4 | Fork an existing repository |

### Non-Functional Requirements

| ID | Requirement | Target | Source |
|----|-------------|--------|--------|
| NFR-1 | Durability | No data loss on single-server failure | Design assumption (standard for storage systems) |
| NFR-2 | Availability | 99.95%+ uptime | Design assumption (industry standard for tier-1 services) |
| NFR-3 | Consistency | Strong consistency for ref updates | GitHub fact: Spokes enforces identical state across quorum before committing |
| NFR-4 | Clone latency (p50) | Under 3s for a 50 MB repo | Design assumption (reasonable UX target) |
| NFR-5 | Push ACK latency | Under 2s | Design assumption, informed by GitHub fact: Spokes targets "a few hundred milliseconds per update" |
| NFR-6 | Read:Write ratio | ~100:1 | GitHub fact: Stretching Spokes blog states reads outnumber writes by about 100:1 |

What does the Source column do?

The Source column is what makes this an interview-grade document. Each requirement is labeled as either a GitHub fact (something GitHub's engineering blog confirms) or a Design assumption (a value you chose and can defend).

For example, the ~100:1 read:write ratio is a verified fact from GitHub's Stretching Spokes blog post. The latency targets are your own assumptions based on reasonable user experience goals. An interviewer can probe any number and you will know whether to cite a source or explain your reasoning.

  • Add a --- horizontal rule on a new line after the last row of the Non-Functional Requirements table, before the ## 2. Capacity Estimation heading.
  • Save README.md and check your VS Code preview.

You should see two cleanly rendered tables. The Non-Functional Requirements table has four columns, with the Source column on the right showing either "GitHub fact" or "Design assumption" for each row.

Tables not rendering correctly?

Markdown tables are sensitive to formatting. Make sure each row starts and ends with a | character, and that the separator row (the |----|-----| line) has the same number of columns as the header row.

Help me fix my Markdown table formatting in README.md

Estimate storage capacity with explicit assumptions

The capacity section is where you do the actual math. You start with a source table that clearly separates verified inputs (numbers from GitHub's own reports) from assumptions you are choosing for your design.

  • Find the ## 2. Capacity Estimation section. Delete the <!-- TODO --> placeholder below it.
  • Replace it with the source table and storage calculations:
### Source Table

| Input | Value | Category | Source / Rationale |
|-------|-------|----------|-------------------|
| Total repositories | 630M | GitHub fact | Octoverse 2025 |
| Total Git data stored | 18.6+ PB (across replicas) | GitHub fact | GitHub engineering blog, Sept 2022 |
| Commits per year | 986M | GitHub fact | Octoverse 2025 |
| Read:write ratio | ~100:1 | GitHub fact | Stretching Spokes blog |
| Replication factor | 3x | GitHub fact | Spokes: "at least three copies of every repository" |
| Rack-distributed replicas | Yes | GitHub fact | Spokes: "copies are in different racks" |
| Average repo size (compressed) | ~5 MB | Design assumption | Median repos are small; heavy-tail monorepos skew the average up |
| Annual repo growth rate | ~20% | Design assumption | Extrapolated from 121M new repos added in 2025 |
| Per-node SSD capacity | ~100 TB | Design assumption | Reasonable enterprise SSD server capacity |
| Peak-to-average traffic ratio | ~3x | Design assumption | Standard estimate for bursty traffic |

### Storage Calculations

- Raw storage: 630M x 5 MB = **~3.15 PB**
- Replicated storage: 3.15 PB x 3 = **~9.45 PB**
- 3-year projected storage: 9.45 PB x 1.2^3 = **~16.3 PB**

> Sanity check: GitHub reported 18.6+ PB of total Git data in Sept 2022, when they had fewer repositories. Our 9.45 PB replicated estimate for 630M repos is in a plausible range, suggesting the ~5 MB average is reasonable.

What do these calculations tell us?

The source table makes every input traceable. The top six rows are GitHub facts you can cite (630M repos from Octoverse 2025, 18.6+ PB from GitHub's engineering blog, 986M commits/year, 100:1 read:write ratio, 3x replication, rack-distributed replicas). The bottom four rows are your design assumptions that you chose and can justify.

The sanity check is important. Your estimate of ~9.45 PB replicated is compared against GitHub's own reported figure of 18.6+ PB. The numbers are in the same ballpark, which suggests your ~5 MB average repo size assumption is reasonable.

  • Save README.md and check your VS Code preview.

You should see the source table with a clear Category column distinguishing "GitHub fact" rows from "Design assumption" rows. Below it, the three storage calculations should render with the bold petabyte totals, followed by the sanity check blockquote.

Blockquote not rendering?

Make sure the sanity check line starts with > (a greater-than sign followed by a space). There should be a blank line before it, separating it from the bullet list above.

Help me fix my Markdown blockquote formatting

Estimate traffic with read/write breakdown

Storage tells you how much disk space you need. Traffic tells you how many requests per second your servers must handle. The read:write ratio shapes the entire architecture because it determines whether you optimize for reads or writes.

  • In README.md, add the traffic calculations directly below the sanity check blockquote you just wrote:
### Traffic Calculations

| Metric | Value | Derivation |
|--------|-------|------------|
| Pushes per day | ~2.7M | 986M commits/year / 365 (GitHub fact as input) |
| Reads per day | ~270M | 2.7M x 100 (GitHub fact: 100:1 ratio) |
| Peak pushes/sec | ~94 | 2.7M / 86400 x 3 (design assumption: 3x peak multiplier) |
| Peak reads/sec | ~9,375 | 270M / 86400 x 3 |

**Key insight:** Reads dominate by 100:1 (GitHub-confirmed ratio). Every replica we add is another server that can serve reads.

Why does the read:write ratio matter so much?

At ~9,375 peak reads/sec versus ~94 peak writes/sec, the system is overwhelmingly read-heavy. This means every replica you add for durability also doubles as a read server. Replication is not just about safety. It is also about performance.

Notice how each derivation in the table traces back to either a GitHub fact or a design assumption. The 986M commits/year and 100:1 ratio are GitHub facts. The 3x peak multiplier is your assumption. An interviewer can follow the math chain.

  • Add a --- horizontal rule on a new line after the "Key insight" paragraph, before the ## 3. Storage Layer (Naive Design) heading.
  • Scroll to the very bottom of the file. Find the test Mermaid diagram you added in Step 1:
```mermaid
flowchart TD
    A[Test Node] --> B[Mermaid Works]
```
  • Delete that entire test diagram block. It served its purpose and the real diagrams start in Step 3.
  • Save README.md.

Your VS Code preview should now show the traffic calculations table with all four rows, the bold key insight, and a clean horizontal rule separating the capacity section from the remaining TODO sections below.

Before you push, what do you expect to see when you view this document on GitHub? Will the tables, blockquotes, and horizontal rules all render the same way they do in VS Code?

  • Commit and push your changes to GitHub by running these commands:
git add README.md
git commit -m "Add requirements and capacity estimation"
git push
  • Open your GitHub repository in a browser and view the README.md.

You should see the complete Requirements section with two tables (Functional and Non-Functional) and the Capacity Estimation section with the source table, storage calculations, sanity check blockquote, traffic table, and key insight. Every table should render with clean rows and columns. The horizontal rules should appear as thin lines separating major sections.

Tables or formatting look broken on GitHub?

GitHub's Markdown renderer is slightly stricter than VS Code's. Make sure each table has a blank line before it (GitHub requires this to start a table). Check that the separator row (e.g. |----|----|) has the same column count as the header.

Help me debug why my Markdown tables render in VS Code but break on GitHub

✔️ Awesome, I've got everything!

Great. Double check you have saved README.md and pushed to GitHub.

ⓧ I'd like to double check the full code

Scroll through your README.md from top to bottom and confirm these sections are present in order:

  • Updated subtitle ending with "with high durability, availability, and performance."
  • A blockquote disclaimer about GitHub fact vs Design assumption labeling.
  • A --- horizontal rule before and after the Table of Contents.
  • Functional Requirements table with FR-1 through FR-4.
  • Non-Functional Requirements table with NFR-1 through NFR-6 and a Source column.
  • A --- horizontal rule between the Requirements and Capacity Estimation sections.
  • Source Table with 10 rows (6 GitHub facts, 4 design assumptions).
  • Storage Calculations with three bullet points: ~3.15 PB raw, ~9.45 PB replicated, ~16.3 PB projected.
  • A sanity check blockquote referencing 18.6+ PB.
  • Traffic Calculations table with 4 rows: pushes/day, reads/day, peak pushes/sec, peak reads/sec.
  • A bold "Key insight" paragraph about reads dominating by 100:1.
  • A --- horizontal rule before section 3.
  • The test Mermaid diagram from Step 1 has been deleted.

You now have the foundation every architecture decision will reference. Your requirements define what the system must do. Your capacity numbers tell you how big the problem is. Next up, you will sketch the simplest architecture that could serve repositories and expose exactly why it breaks at GitHub's scale.

Design the Storage Layer (Naive Approach)

Your design document now has solid foundations. You've defined what the system needs to do, and you've calculated the scale it needs to handle: 3.15 PB of raw data, 270M reads per day, and 99.95% uptime.

Most candidates in system design interviews jump straight to drawing distributed architectures. In this step, you'll take the opposite approach: start with the simplest design that could possibly work, then systematically expose why it fails. This is what separates strong candidates from the rest.

In this step, get ready to:
  • Map Git's internal storage model to understand what the system actually stores.
  • Diagram a naive single-server architecture using a Mermaid flowchart.
  • Identify three fundamental flaws that drive the next step's distributed design.
Map the Git object model and diagram a single-server architecture

Before you can design a storage system for Git repositories, you need to understand what Git actually stores on disk. This shapes every decision about how to partition, replicate, and serve the data.

  • In README.md, find the heading ## 3. Storage Layer (Naive Design).
  • Delete the <!-- TODO --> comment beneath it.
  • Add the Git Object Model section by copying this Markdown in its place:
### Git Object Model

Git stores data as content-addressed objects in a DAG:

- **Blob**: file contents, identified by SHA hash
- **Tree**: directory listing mapping filenames to blob/tree hashes
- **Commit**: snapshot pointing to a tree, parent commit(s), author, and timestamp
- **Tag**: named pointer to a commit

Objects are compressed into **packfiles** using delta compression. A repo with 100K commits and 10 GB of source might compress to ~500 MB in packfiles.

**Refs** (branches, tags) are lightweight pointers to commit hashes. A `git push` atomically updates a ref.

What does this section cover?

This section explains the four types of objects Git stores (blobs, trees, commits, tags) and how they're identified by SHA hashes. It also covers packfiles, which compress objects using delta encoding, and refs, which are the lightweight pointers (branches, tags) that a git push updates.

In an interview, showing you understand Git's internals proves you know what the storage system actually handles. It's not just "files." It's a directed acyclic graph of content-addressed objects.

  • Save README.md and check the Markdown preview pane on the right side of VS Code.
  • You should see the "Git Object Model" heading with a bulleted list of the four object types and two paragraphs about packfiles and refs.

Now that the object model is documented, you need the simplest possible architecture that could serve these objects. This is your "naive design" starting point.

  • Below the Git Object Model section you just added, copy this Mermaid diagram to create the naive architecture:
### Naive Architecture

```mermaid
flowchart TD
    Client[Developer] -->|git push / clone| LB[Load Balancer]
    LB --> GS[Git Service]
    GS --> SS[Storage Server\nLocal Disk]
    GS --> MDB[(Metadata DB\nMySQL)]
```

What does this diagram show?

This is the simplest architecture that could serve Git repositories. A developer pushes or clones through a load balancer, which routes to a Git service. That service reads and writes repository data on a single storage server with local disk, and tracks repository metadata in a MySQL database.

It works for a small number of repos. The point is to start here and then show your interviewer exactly why it breaks.

  • Save README.md and check the preview pane.
  • You should see a rendered flowchart showing the Developer, Load Balancer, Git Service, Storage Server, and Metadata DB connected by arrows.

Seeing raw Mermaid code instead of a diagram?

  • Make sure the code fence uses the mermaid language identifier (three backticks followed by the word mermaid).
  • Check that you're using the side-by-side Markdown preview (Cmd+K V on macOS, Ctrl+K V on Windows). VS Code 1.121+ has built-in Mermaid rendering in Markdown preview.
  • help me debug why my Mermaid diagram isn't rendering in VS Code's Markdown preview
Analyze three fundamental flaws in the naive design

Your naive architecture works for a small deployment. The real value in an interview comes from showing why it fails at GitHub's scale, using the exact numbers you calculated in Step 2.

  • Below the Mermaid diagram, add this analysis table by copying the following Markdown:
### Why This Breaks at Scale

| Problem | Description | Impact (from my capacity estimates) |
|---------|-------------|--------------------------------------|
| Durability | Single disk/server failure loses all repos on that server. No replicas. | Permanent data loss |
| Capacity | A single server cannot hold 3.15 PB (my estimate) of repository data. Vertical scaling has a hard ceiling. | Cannot grow past one machine |
| Uneven Placement | Without distribution, popular repos (linux/kernel, vscode) create hotspots. One server gets hammered while others idle. | Degraded performance for popular repos |

Why reference your own capacity numbers here?

Each flaw connects back to a concrete number from your Section 2 estimates. The capacity problem references your 3.15 PB raw storage calculation. The durability problem is grounded in NFR-1 (no data loss on single-server failure).

This cross-referencing is what makes the analysis interview-grade. You're not just listing abstract problems. You're proving each one is real at this specific scale.

  • Save README.md and check the preview pane.
  • You should see a three-row table with columns for Problem, Description, and Impact. The Capacity row should reference your 3.15 PB estimate from Section 2.
Summarize the design requirements these flaws create

Each flaw you identified points to a specific design requirement. This table translates problems into the solution directions you'll implement in Step 4.

  • Below the "Why This Breaks at Scale" table, add this requirements mapping and section separator by copying the following Markdown:
### Design Requirements from These Flaws

| Flaw | Requirement | Solution Direction |
|------|-------------|-------------------|
| No durability | Replicate data across independent servers | 3-replica replication with quorum writes (GitHub-confirmed approach) |
| Capacity ceiling | Distribute data horizontally | Partition repos across a storage fleet (partitioning strategy is my design choice) |
| Hotspot placement | Distribute load intelligently | Partitioning + latency-aware read routing |

---

What does this table do?

The Solution Direction column previews the architecture you'll build in Step 4. Notice the labels: "GitHub-confirmed approach" for the replication strategy (Spokes uses 3 replicas with quorum writes) and "my design choice" for the partitioning strategy. Keeping these labels consistent throughout your document is what makes it interview-ready.

  • Save README.md and check the preview pane.
  • You should see a three-row table mapping each flaw to a requirement and a solution direction, followed by a horizontal rule separator before Section 4.

✔️ Awesome, I've got everything!

Great. Double check you've saved README.md before moving on to the push.

ⓧ I'd like to double check the full code

Scroll through your Section 3 content under the ## 3. Storage Layer (Naive Design) heading and confirm these four subsections appear in order:

  • ### Git Object Model with four bullet points (Blob, Tree, Commit, Tag) and paragraphs about packfiles and refs.
  • ### Naive Architecture with a Mermaid flowchart showing Developer, Load Balancer, Git Service, Storage Server, and Metadata DB.
  • ### Why This Breaks at Scale with a three-row table (Durability, Capacity, Uneven Placement).
  • ### Design Requirements from These Flaws with a three-row table mapping flaws to solution directions, followed by a --- separator.

Make sure the <!-- TODO --> comment under Section 3 is gone and has been replaced by this content.

Now push your changes to GitHub so the Mermaid diagram renders online. Before you run the push, read through your "Why This Breaks at Scale" table. Can you trace each problem back to a specific number from your Capacity Estimation section?

  • Commit and push your changes by running these commands in your terminal:
git add README.md
git commit -m "Add naive storage layer design with flaw analysis"
git push
  • Open your repository on GitHub and scroll down to Section 3.
  • You should see the Mermaid flowchart rendered as a visual diagram (not raw code) showing the naive architecture. The "Why This Breaks at Scale" and "Design Requirements" tables should render as formatted tables beneath it.

Diagram not rendering on GitHub?

  • Confirm the code fence starts with exactly three backticks followed by mermaid (no extra spaces or capital letters).
  • Check that the closing fence is also exactly three backticks on its own line.
  • Try a hard refresh on the GitHub page (Cmd+Shift+R on macOS, Ctrl+Shift+R on Windows). GitHub sometimes caches the old version briefly.
  • help me figure out why my Mermaid diagram renders in VS Code but not on GitHub

You've built the naive design and proven it can't survive at GitHub's scale. Your three flaws (no durability, capacity ceiling, hotspot placement) each point to a concrete design requirement. Next up, you'll solve all three by partitioning repositories across a fleet of storage nodes and replicating each one with quorum-based writes, modeled on GitHub's Spokes system.

Design Partitioning and Replication

Your naive design from the last step exposed three fundamental problems: no durability, a hard capacity ceiling, and hotspot placement. Each flaw maps to a concrete requirement you already identified in your design requirements table.

This step solves all three by partitioning repositories across a fleet of storage nodes and replicating each repository to three independent servers with quorum-based writes. You will choose and justify your own partitioning strategy, calculate the fleet size from your Step 2 capacity estimates, and model the replication protocol on GitHub's Spokes system.

In this step, get ready to:
  • Design a partitioned storage fleet with a partitioning strategy you choose and justify, plus a fleet size you calculate from your own assumptions.
  • Design a 3-replica quorum-based write protocol modeled on GitHub's Spokes system, with a sequence diagram showing the 3-phase commit.
  • Document the design trade-offs, clearly labeling what comes from GitHub's architecture and what is your own design choice.
Design the partitioned storage fleet

The first part of your Section 4 explains why repositories need local SSDs, introduces your partitioning strategy, and justifies it against alternatives.

  • In README.md, find the <!-- TODO --> comment under ## 4. Partitioning and Replication.
  • Replace that comment with the following content:
### Partitioned Storage Fleet

Repositories are distributed across a fleet of storage nodes. Each node stores bare Git repos on **local SSDs**.

**Why local SSDs, not a distributed filesystem? (GitHub fact)**
Git operations like `git log` and `git blame` touch thousands of objects sequentially. Any per-object latency penalty from a network filesystem is multiplicative. GitHub's Spokes system uses plain Git on local filesystems for this reason.

**Partitioning strategy (my design choice):**
I use consistent hashing on the repository ID to distribute repos across nodes. GitHub does not publicly document their exact partitioning method. I chose consistent hashing because it minimizes redistribution when nodes are added/removed and distributes load evenly. Alternatives considered: range-based sharding (simpler but prone to hotspots), directory-based sharding (more flexible but adds a lookup layer).

What does this section do?

This section establishes two key design decisions. The local SSD choice is a GitHub-confirmed fact from Spokes. The consistent hashing partitioning strategy is YOUR design choice.

Labeling each decision this way is the whole point. In an interview, you can say "GitHub uses local SSDs because of sequential object access" (fact) and "I chose consistent hashing because it minimizes redistribution" (your reasoning).

  • Save README.md and check the Markdown preview pane on the right.
  • You should see the "Partitioned Storage Fleet" subsection with the local SSDs rationale and partitioning strategy rendered as formatted text.

Preview not showing the new content?

Make sure you replaced the <!-- TODO --> comment entirely. If both the comment and your new content appear, delete the leftover comment line.

If the preview pane is not visible, reopen it with Cmd+K V (macOS) or Ctrl+K V (Windows).

Help me troubleshoot my Markdown preview in VS Code.

Now add the fleet sizing calculation and a Mermaid diagram that visualizes the consistent hash ring.

  • Add the following content directly below the partitioning strategy paragraph you just wrote:

**Fleet sizing (calculated from my assumptions):**
- Raw storage: ~3.15 PB (from Section 2)
- Per-node capacity: ~100 TB SSD (my assumption)
- Nodes before replication: 3.15 PB / 100 TB = ~32
- Nodes with 3x replication: 32 x 3 = **~96 storage nodes**

```mermaid
flowchart LR
    subgraph Hash Ring
        H[Consistent Hash\nby Repo ID]
    end
    H --> N1[Node 1]
    H --> N2[Node 2]
    H --> N3[Node 3]
    H --> N4[Node 4]
    H --> Ndot[... N nodes]
```

> The node count (~96 in my calculation) is a function of my assumed average repo size and per-node capacity. Different assumptions yield different fleet sizes.

What does this calculation show?

The fleet sizing takes your raw storage estimate from Section 2 (3.15 PB), divides by your per-node capacity assumption (100 TB), then multiplies by the replication factor (3x). Every number in the calculation traces back to an assumption you already declared.

The Mermaid flowchart shows how the consistent hash function maps each repository ID to a specific node in the fleet. When you add or remove nodes, only a small fraction of repos need to move.

  • Save README.md and check the preview pane.
  • You should see the fleet sizing bullet list and a rendered Mermaid flowchart showing boxes for Node 1 through Node N branching from a "Consistent Hash by Repo ID" box.

Mermaid diagram not rendering?

Make sure the code fence starts with exactly ```mermaid (three backticks followed by the word mermaid, no space). Also confirm the closing ``` is on its own line.

Help me get my Mermaid flowchart rendering in VS Code's Markdown preview.

Design 3-replica quorum-based writes

With the storage fleet in place, you now need a protocol for replicating writes across multiple nodes. This is where your design closely follows GitHub's verified architecture.

  • Add the following content directly below the blockquote note about node count:

### 3-Replica Quorum Writes

The following replication design is modeled directly on GitHub's Spokes system, with sources noted:

- Every repository is stored on **3 storage nodes from different racks** _(GitHub fact: Spokes blog)_
- Writes use a **majority-quorum protocol**: committed only when at least 2 of 3 replicas confirm _(GitHub fact: Spokes blog)_
- If replicas **disagree on the resulting state**, the write is **rolled back** on all _(GitHub fact: Spokes blog)_
- The write protocol uses a **three-phase commit** with distributed locking _(GitHub fact: Stretching Spokes blog)_
- Reads route to the **closest healthy replica** _(GitHub fact: Stretching Spokes blog)_

Why label every bullet as a GitHub fact?

Every single bullet in this list comes from GitHub's own Spokes blog posts. That is the point. When an interviewer asks "how do you know GitHub uses quorum writes?" you can cite the source directly instead of guessing.

Compare this to the partitioning strategy above, which is labeled as YOUR design choice. The contrast shows you can distinguish between what the system actually does and what you are proposing.

  • Save README.md and check the preview pane.
  • You should see the "3-Replica Quorum Writes" subsection with five labeled bullet points, each tagged with its source.

Now add the sequence diagram that visualizes the 3-phase commit protocol. This is the diagram that makes the quorum logic concrete.

  • Add the following content directly below the quorum writes bullet list:

**Write protocol (3-phase commit):**

```mermaid
sequenceDiagram
    participant Client
    participant Proxy as Git Proxy
    participant A as Replica A
    participant B as Replica B
    participant C as Replica C

    Client->>Proxy: git push (packfile)
    Proxy->>A: prepare (send objects)
    Proxy->>B: prepare (send objects)
    Proxy->>C: prepare (send objects)
    A-->>Proxy: prepared OK
    B-->>Proxy: prepared OK
    C-->>Proxy: prepared (or timeout)
    Note over Proxy: Quorum reached (2 of 3)
    Proxy->>A: commit (update refs)
    Proxy->>B: commit (update refs)
    A-->>Proxy: committed
    B-->>Proxy: committed
    Proxy-->>Client: push OK
    Note over C: Repaired async later
```

What does this diagram show?

The sequence diagram walks through the three phases of a write. First, the Git Proxy sends objects to all three replicas (prepare). Second, it waits for a majority quorum (2 of 3 replicas confirm). Third, it commits the ref updates on the replicas that responded.

Notice Replica C times out but the write still succeeds. That is the power of quorum-based writes. Replica C gets repaired asynchronously later.

  • Save README.md and check the preview pane.
  • You should see a rendered sequence diagram with arrows flowing between Client, Git Proxy, and Replicas A, B, and C. The "Quorum reached (2 of 3)" note should appear over the Proxy.

Sequence diagram not rendering correctly?

Mermaid sequence diagrams are sensitive to arrow syntax. Make sure solid arrows use ->> and dashed response arrows use -->>. A missing > will break the whole diagram.

Help me debug my Mermaid sequence diagram in VS Code.

Document the design trade-offs

A strong system design document does not just present the chosen solution. It shows what alternatives existed and why each decision was made. This trade-offs section ties together everything from Section 4.

  • Add the following content directly below the 3-phase commit sequence diagram:

### Design Trade-offs

| Decision | Choice | Alternative | Reason | Source |
|----------|--------|-------------|--------|--------|
| Replica count | 3 | 2 or 5 | Minimum for majority quorum with single-failure tolerance | GitHub fact (Spokes uses 3) |
| Storage medium | Local SSDs | Distributed FS (Ceph, GFS) | Git's sequential object access makes per-object latency critical | GitHub fact (Spokes rationale) |
| Replication layer | Application-level (Git protocol) | Disk-level (DRBD) | Flexible placement, independent healing, read routing | GitHub fact (DGit/Spokes migration rationale) |
| Write quorum | 2-of-3 majority | All-3 or 1-of-3 | Balances durability with availability during failures | GitHub fact (Spokes quorum) |
| Partitioning | Consistent hashing | Range-based, directory-based | Minimizes redistribution on node add/remove | My design choice |
| Per-node capacity | ~100 TB SSD | Larger/smaller nodes | Balances cost with fleet manageability | My design assumption |

| Property | Naive Design | Partitioned + Replicated |
|----------|-------------|-------------------------|
| Durability | None (single copy) | 3 replicas across racks |
| Capacity | ~100 TB (one server) | Scales with fleet size (~96 nodes in my calculation) |
| Read throughput | 1 server | Up to 3 servers per repo, fleet-wide for all repos |
| Failure impact | Total loss of repos on that server | Automatic failover to healthy replicas |

What do these tables show?

The first table walks through each design decision, naming the choice, the alternatives you considered, the reasoning, and whether the decision is grounded in GitHub's architecture or your own analysis. The Source column is what sets this apart in an interview.

The second table is a before-and-after comparison that connects directly back to the three flaws from Step 3. Every flaw is now resolved. The numbers reference your own calculations, not pre-given values.

  • Save README.md and check the preview pane.
  • You should see two rendered tables. The first lists six design decisions with their sources. The second compares the naive design against the partitioned + replicated design across four properties.

Tables not rendering properly?

Markdown tables need a header row, a separator row of dashes and pipes, and data rows with matching column counts. If any row has a different number of pipe characters, the entire table breaks.

Help me fix my Markdown table formatting.

Section 4 is complete. Commit your changes and push to GitHub to verify the diagrams render there too.

  • Commit and push your changes by running these commands:
git add README.md
git commit -m "Add partitioning and replication design (Section 4)"
git push

Before you check GitHub, think about this: if Replica C is completely down during a write, does the write succeed or fail? And if Replicas B and C disagree with Replica A on the resulting state, what happens?

  • Open your repository on GitHub and scroll to Section 4.

The write succeeds when Replica C is down, because Replicas A and B form a quorum (2 of 3). If B and C disagree with A, the write is rolled back on all replicas. Both of these behaviors are visible in your sequence diagram and labeled as GitHub facts in your bullet list.

Confirm that the hash ring flowchart and the 3-phase commit sequence diagram both render as visual diagrams on GitHub, not as raw code blocks. Also verify that your trade-off tables render with clear columns.

Diagrams show as raw code on GitHub?

GitHub requires the code fence to start with exactly ```mermaid on its own line, with no extra spaces before the backticks. Check that your fences match this format exactly.

Help me get my Mermaid diagrams rendering on GitHub.

✔️ Awesome, I've got everything!

Great. Double check you have saved README.md and pushed to GitHub.

ⓧ I'd like to double check the full code

Your Section 4 content is long, so scroll through your file and confirm these subsections appear in order under the Section 4 heading:

  • Partitioned Storage Fleet subsection with the local SSDs rationale, partitioning strategy paragraph, fleet sizing bullet list, and the hash ring Mermaid flowchart followed by the blockquote note.
  • 3-Replica Quorum Writes subsection with five labeled bullet points and the 3-phase commit Mermaid sequence diagram.
  • Design Trade-offs subsection with two tables: a six-row decision table (each with a Source column) and a four-row comparison table (Naive Design vs Partitioned + Replicated).

Sections 5 and 6 should still show their <!-- TODO --> placeholders. If anything is missing, scroll up to the relevant chunk above and re-check.

Your storage layer now has partitioning, replication, and quorum writes. But nothing in the system knows WHICH repositories live on which nodes, whether those nodes are healthy, or how to recover when one fails. Next up, you will add the brain of the system: a metadata layer that tracks replica locations, routes requests, and orchestrates failure recovery.

Design the Routing and Metadata Layer

In the previous step, you built a partitioned storage fleet with 3-replica quorum writes modeled on GitHub's Spokes system. Your ~96 storage nodes can hold the data and survive single-server failures.

But right now, nothing in your design knows which repositories live on which nodes. Nothing checks whether those nodes are healthy, and nothing orchestrates recovery when a node goes down. This step adds the brain of the system: a metadata layer that tracks replica locations, routes every request to the right node, and automatically recovers from failures.

In this step, get ready to:
  • Design a metadata schema that tracks replica locations and health.
  • Walk through a concrete failure scenario from detection to recovery.
  • Create an end-to-end push flow sequence diagram that ties the entire system together.
Design the metadata schema and routing flow

Your storage fleet has ~96 nodes, but a git clone request needs to know exactly which 3 nodes hold that repository's replicas. That mapping lives in the metadata layer.

  • In README.md, find the <!-- TODO --> placeholder under the ## 5. Routing and Metadata Layer heading.
  • Delete the placeholder and paste this content in its place:
### Metadata Schema

The metadata database uses MySQL, sharded via Vitess _(GitHub fact: GitHub uses a MySQL-based metadata tier sharded with Vitess-style topology)_. The specific schema fields below are my design, informed by what the replication protocol needs to track:

| Field | Description |
|-------|-------------|
| repo_id | Unique repository identifier |
| replica_nodes | List of 3 storage nodes holding this repo |
| replica_health | Status per replica: healthy, degraded, rebuilding |
| generation_number | Increments on each write; verifies replicas are in sync |

What do these fields track?

Each field serves a specific purpose in the replication protocol you designed in Step 4.

  • replica_nodes tells the Git Proxy where to route reads and which 3 nodes to coordinate for quorum writes.
  • replica_health lets the proxy skip degraded nodes during reads and trigger re-replication when a node is down.
  • generation_number is how the system catches replicas that fell out of sync. If two replicas report different generation numbers, one needs repair.
  • Save README.md and check the Markdown preview pane.
  • You should see a "Metadata Schema" heading followed by a neatly rendered table with four rows.
  • Below the metadata schema table, add the full request path diagram by pasting this content:
### Full Request Path

```mermaid
flowchart TD
    Client[Developer] -->|git push / clone| DNS[DNS / Anycast]
    DNS --> LB[Load Balancer\nSSH 22 / HTTPS 443]
    LB --> Auth[Auth Service]
    Auth --> API[API / Rails\nPermissions Check]
    API --> Proxy[Git Proxy]
    Proxy -->|lookup repo_id| MDB[(Metadata DB\nMySQL / Vitess)]
    MDB -->|replica_nodes + health| Proxy
    Proxy -->|read: closest healthy| FS1[Storage Node A]
    Proxy -->|write: 3-phase commit| FS1
    Proxy -->|write: 3-phase commit| FS2[Storage Node B]
    Proxy -->|write: 3-phase commit| FS3[Storage Node C]
```

What does this diagram show?

This is the full path every Git request takes through your system. The Git Proxy is the critical decision-maker: it consults the Metadata DB to find which 3 nodes hold a repo, then either routes to the closest healthy replica (for reads) or coordinates the 3-phase quorum commit (for writes).

Notice the Proxy separates read and write paths. Reads hit a single node. Writes fan out to all 3. That is your 100:1 ratio from Step 2 in action.

  • Save README.md and check the preview pane.
  • You should see a flowchart showing the full path from Developer through DNS, Load Balancer, Auth, API, Git Proxy, and Metadata DB, fanning out to three storage nodes.

Diagram showing as a code block instead of a visual?

  • Make sure the opening fence says exactly ```mermaid with no extra spaces or characters.
  • Check that the closing fence is exactly three backticks on its own line.
  • Confirm you are using the built-in Markdown preview (Cmd+K V on macOS or Ctrl+K V on Windows). VS Code 1.121+ renders Mermaid natively.
  • Help me debug my Mermaid diagram in VS Code
Walk through a concrete failure scenario

A routing layer is only as good as its failure handling. In an interview, walking through a specific failure scenario proves you understand how the system behaves under stress, not just in the happy path.

  • Below the Full Request Path section, paste this failure scenario walkthrough with its 4-phase recovery diagram:
### Failure Scenario: A Storage Node Crashes

Each phase below notes whether the behavior is from GitHub's documented architecture or my design logic.

```mermaid
flowchart TD
    subgraph Phase 1: Detection
        F1[Git Proxy detects ~3 consecutive\nfailed operations to the node]
        F2[Marks node unavailable\nin Metadata DB]
        F1 --> F2
    end
    subgraph Phase 2: Rerouting
        F3[Reads for affected repos\nroute to 2 remaining replicas]
        F4[Writes still succeed:\n2 replicas form a quorum]
        F2 --> F3
        F2 --> F4
    end
    subgraph Phase 3: Re-replication
        F5[System identifies repos\nwith only 2 replicas]
        F6[Selects new target nodes\nfrom different racks]
        F7[N-to-N parallel copy:\nall surviving nodes as sources]
        F3 --> F5
        F5 --> F6
        F6 --> F7
    end
    subgraph Phase 4: Recovery
        F8[Metadata DB updated\nwith new replica locations]
        F9[System returns to full\n3-replica durability]
        F7 --> F8
        F8 --> F9
    end
```

What are the four phases?

  • Detection: The Git Proxy notices ~3 consecutive failed operations. This threshold comes from GitHub's Spokes documentation.
  • Rerouting: Reads immediately route to the 2 surviving replicas. Writes still succeed because 2-of-3 is a valid quorum.
  • Re-replication: N-to-N parallel repair copies data from all surviving nodes to new targets. The larger the cluster, the faster this completes.
  • Recovery: The Metadata DB updates with new replica locations, restoring full 3-replica durability. This final phase is your own design logic for updating the metadata.
  • Save README.md and check the preview pane.
  • You should see a flowchart with four grouped subgraphs. Each subgraph represents one recovery phase, connected by arrows showing the progression from detection through to full recovery.
  • Directly below the failure diagram, paste this source attribution table and key properties summary:
| Phase | Behavior | Source |
|-------|----------|--------|
| Detection | Node detected as down within ~3 failed operations | GitHub fact (Spokes blog) |
| Rerouting | Routing returns a list of replicas; if first fails, next is tried | GitHub fact (Spokes blog) |
| Rerouting | 2-of-3 quorum still accepts writes | GitHub fact (Spokes quorum design) |
| Re-replication | N-to-N parallel repair; larger cluster = faster recovery | GitHub fact (Spokes blog) |
| Recovery | Metadata DB updated, full durability restored | My design logic (specific metadata update flow) |

**Key properties:**
- **Zero read downtime**: reads route to surviving replicas immediately
- **Zero write downtime**: 2-of-3 quorum still holds
- **Fast re-replication**: N-to-N parallel repair (GitHub-confirmed) means the larger the cluster, the faster recovery
- **Automatic**: no human intervention needed (GitHub-confirmed)

Why does the source column matter for interviews?

Notice the Recovery row is labeled "My design logic" while the others are "GitHub fact." In an interview, this distinction is powerful. You can say: "GitHub confirms detection happens within about 3 failures, and they use N-to-N parallel repair. The specific metadata update flow is my design choice for how I would implement the final recovery step."

That kind of precision separates candidates who memorize architectures from those who understand them.

  • Save README.md and confirm the source table renders below the diagram in the preview.
Add the end-to-end push flow and summary

The end-to-end push flow is the diagram that ties your entire system together. In an interview, this is your "money slide": a single sequence diagram that traces a git push from the developer's terminal all the way to committed replicas.

  • Below the failure scenario section (after the key properties list), paste this end-to-end push flow diagram:
### End-to-End Push Flow

```mermaid
sequenceDiagram
    participant Dev as Developer
    participant DNS as DNS / Anycast
    participant LB as Load Balancer
    participant Auth as Auth Service
    participant API as API / Rails
    participant Proxy as Git Proxy
    participant MDB as Metadata DB
    participant A as Storage Node A
    participant B as Storage Node B
    participant C as Storage Node C

    Dev->>DNS: git push
    DNS->>LB: Route to nearest edge
    LB->>Auth: Authenticate (SSH key / PAT)
    Auth->>API: Check push permissions
    API->>Proxy: Forward push request
    Proxy->>MDB: Lookup repo_id replicas
    MDB-->>Proxy: Nodes A, B, C (all healthy)
    Proxy->>A: receive-pack (prepare)
    Proxy->>B: receive-pack (prepare)
    Proxy->>C: receive-pack (prepare)
    A-->>Proxy: prepared OK
    B-->>Proxy: prepared OK
    C-->>Proxy: prepared OK
    Proxy->>A: commit refs
    Proxy->>B: commit refs
    Proxy->>C: commit refs
    A-->>Proxy: committed
    B-->>Proxy: committed
    C-->>Proxy: committed
    Proxy->>MDB: Update generation number
    Proxy-->>Dev: Push OK
```

What does this diagram prove?

This single diagram connects every layer of your design. The routing path (DNS through Git Proxy) resolves to the correct 3 storage nodes via a metadata lookup. The write path follows the 3-phase quorum protocol from Step 4. The generation number update in the Metadata DB closes the loop.

If an interviewer asks "walk me through a push," this diagram is your complete answer.

  • Save README.md and check the preview pane.
  • You should see a sequence diagram with 10 participants across the top and arrows tracing the full push flow from Developer to Push OK.

The last piece of your document is the Summary section, which ties all six design decisions together. You also need a horizontal rule separator before it.

  • Directly above the existing ## 6. Summary heading, add a --- horizontal rule on its own line.
  • Find the <!-- TODO --> placeholder below ## 6. Summary and replace it with this content:
This design handles 630M+ Git repositories by:

1. **Partitioning** repos across a storage fleet via consistent hashing _(my design choice)_
2. **Replicating** each repo to 3 nodes across different racks _(GitHub-confirmed)_
3. **Writing** via a 3-phase quorum protocol: committed when 2-of-3 replicas confirm _(GitHub-confirmed)_
4. **Reading** from the closest healthy replica _(GitHub-confirmed)_
5. **Routing** via a Vitess-sharded metadata DB that tracks replica locations and health _(GitHub-confirmed technology choice)_
6. **Recovering** automatically: detection within ~3 failures, rerouting with zero downtime, N-to-N parallel re-replication _(GitHub-confirmed)_

Fleet size and storage totals depend on the assumptions in Section 2. With my assumptions (~5 MB avg repo, ~100 TB/node), the result is ~9.45 PB replicated across ~96 nodes, projected to grow to ~16.3 PB over 3 years.

What makes this summary interview-ready?

Each of the six numbered points labels its source. An interviewer can see immediately which decisions are grounded in GitHub's real architecture and which are your own defensible choices. The closing paragraph anchors the fleet size to your specific assumptions, so you can adjust the numbers live if an interviewer challenges them.

  • Save README.md.
  • Commit and push your completed document to GitHub by running these commands in the terminal:
git add README.md
git commit -m "Add routing, metadata, failure recovery, and summary sections"
git push

Before you check GitHub, take a moment to predict: will the end-to-end push flow sequence diagram render as a visual, or will it appear as a raw code block?

  • Open your repository on GitHub and view the README.md.
  • Scroll through the document and confirm every Mermaid diagram renders as a visual. You should see the routing flowchart, the 4-phase failure recovery flowchart, and the end-to-end push sequence diagram all rendered as diagrams.

Diagrams not rendering on GitHub?

  • Verify the opening fence is exactly ```mermaid (three backticks, then the word mermaid, no spaces between them).
  • Make sure there are no extra blank lines between the opening fence and the first line of the diagram.
  • Check that the closing fence is exactly three backticks on its own line with nothing after them.
  • If a specific diagram fails, look for typos in the Mermaid syntax. A missing arrow or unclosed subgraph block will cause the entire diagram to fall back to a code block.
  • Help me debug why my Mermaid diagrams render in VS Code but not on GitHub

Now for the real verification. Scroll to the failure scenario section in your document and trace through it as if an interviewer asked: "What happens when a storage node crashes?"

  • Starting from the node crash, trace each phase and answer these three questions from what you wrote: Does the system maintain read availability? Does it maintain write availability? How long until full durability is restored?
  • For each answer, confirm you can point to whether the behavior is a GitHub fact or your own design logic.

What should my answers be?

  • Read availability: yes, immediately. Reads route to surviving replicas (GitHub fact).
  • Write availability: yes. 2-of-3 quorum still holds (GitHub fact).
  • Full durability: restored after N-to-N re-replication completes and the Metadata DB is updated. Re-replication speed scales with cluster size (GitHub fact). The metadata update flow is your design logic.

✔️ Awesome, I've got everything!

Great. Double check you have saved README.md and pushed to GitHub.

ⓧ I'd like to double check the full code

Your README.md is a long document at this point. Scroll through it from top to bottom and confirm these sections are present in order:

  • Title, intro paragraph, and Table of Contents (from Step 1).
  • Section 1: Requirements with Functional and Non-Functional tables (from Step 2).
  • Section 2: Capacity Estimation with Source Table, Storage Calculations, and Traffic Calculations (from Step 2).
  • Section 3: Storage Layer (Naive Design) with the Git Object Model, naive architecture Mermaid diagram, "Why This Breaks at Scale" table, and Design Requirements table (from Step 3).
  • Section 4: Partitioning and Replication with the hash ring diagram, 3-phase commit sequence diagram, and trade-offs tables (from Step 4).
  • Section 5: Routing and Metadata Layer with the Metadata Schema table, Full Request Path flowchart, Failure Scenario flowchart with source attribution table and key properties, and End-to-End Push Flow sequence diagram (from this step).
  • Section 6: Summary with the six-point numbered list and closing paragraph (from this step).

Make sure there is a --- horizontal rule between each major section.

Secret mission

Design Fork Network Deduplication

Your current design stores every repository independently. But the linux/kernel repo has been forked over 60,000 times. That is 60,000 copies of essentially the same ~4 GB of data. In this mission, you will design a deduplication layer using Git's alternates mechanism, calculate the storage savings from your own assumptions, and handle the tricky edge cases around deletion and permissions.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses only free tools and services, so there are no ongoing costs.

Resources you used:

  • GitHub repository containing your system design document.
  • Local project directory with your Git repo and README.md.

Keep everything running

No action needed. Choose this if you want your system design document to stay on GitHub as a portfolio piece or interview reference.

  • Your GitHub repository is free to keep. There is no cost for public or private repos on GitHub's free tier.
  • Your local project directory stays on your Desktop for easy access.

Pause - I'll come back to this later

If you do not want the document publicly visible but want to keep it for later, change the repository visibility to private.

  • Go to your repository on GitHub.
  • Click Settings in the top navigation bar.
  • Scroll down to the Danger Zone section.
  • Click Change visibility and select Private.
  • Your local project directory can stay as is. No changes needed there.

Delete - I don't want to use this again

Remove all project resources and start fresh. This deletes your design document permanently.

  • Delete the GitHub repository by going to your repository page, clicking Settings, scrolling to the Danger Zone section, and clicking Delete this repository.
  • Delete the local project directory. On macOS/Linux, run the following command from your Desktop:
cd ~/Desktop
rm -rf github-storage-design
  • On Windows, run this command from your Desktop instead:
cd ~\Desktop
rmdir /s /q github-storage-design

Nice Work!

Nice Work!

Nice work! You just designed a distributed storage system for GitHub's 630M+ repositories. Your design document is live on GitHub with Mermaid architecture diagrams that render as real visuals, ready to walk through in an interview.

You've learned how to:

  • Build a complete system design document with a partitioned storage fleet, 3-replica quorum-based writes, a metadata routing layer with automatic failure recovery, and end-to-end sequence diagrams.
  • Practice the structured interview approach for system design: start with requirements, build a naive architecture, identify its fundamental flaws, then iteratively improve it with partitioning, replication, and intelligent routing.
  • Clearly separate verified architecture facts from your own design assumptions throughout a document, so you can tell an interviewer exactly which decisions are grounded in GitHub's engineering blog posts and which are your own defensible reasoning.
  • Secret Mission: Completed an optional challenge to push your skills further.

Ready to quiz yourself?