Build a SQL Database Engine in Go
Build a SQL engine from scratch with a lexer, parser, and REPL in Go.
Introduction
30 Second Summary
Every app you have ever used relies on a database to store and retrieve information. But what actually happens between typing a SQL query and getting results back?
In this project, you will build a mini SQL database engine from scratch in Go. You will create the same core components that power production databases: a tokenizer, a parser, a storage engine, and an interactive command-line interface.
What You'll Build
A fully functional SQL database engine called MiniDB that parses raw SQL text, executes queries against in-memory storage, formats results as aligned tables, and persists everything to disk as JSON.
By the end of this project, you'll have:
- A lexer that breaks raw SQL strings into typed tokens, and a recursive-descent parser that converts those tokens into an abstract syntax tree.
- A storage engine backed by Go maps with automatic JSON disk persistence, so your database survives restarts.
- An interactive REPL where you type raw SQL and see formatted table output for CREATE TABLE, INSERT, and SELECT (with WHERE filtering).
- Secret Mission: Add DELETE and UPDATE support to your engine.
Are there any prerequisites?
You should be comfortable with Go basics (functions, structs, maps, slices, for-loops). You should also be familiar with basic SQL syntax (SELECT, INSERT, CREATE TABLE).
Set Up the Project
Before you write any database logic, you need a clean project structure. Your engine will have four distinct components: a lexer, a parser, a storage engine, and a REPL. Each lives in its own file.
In this step, you will create a Go module, set up the four source files, and confirm everything compiles cleanly.
In this step, get ready to:
- Create the project directory and initialize a Go module.
- Create four source files for your engine's components.
- Verify the project compiles with no errors.
Create the project directory and initialize the Go module
Go uses modules to manage project dependencies and define the project boundary. Initializing a module creates a go.mod file that tells Go where your project starts and what version of Go it targets.
- In Cursor, open the integrated terminal by pressing Ctrl+` (backtick key, top-left of your keyboard).
- Move to your Desktop by running this command:
cd ~/Desktop- Create your project folder and move into it by running these commands:
mkdir minidb
cd minidb- Confirm the folder exists by running:
lsYou should see an empty output since the folder is brand new.
- Initialize the Go module by running:
go mod init minidb- Verify the module was created by running:
cat go.modYou should see output showing module minidb and go 1.25.0.
Why does it say go 1.25.0?
Even though you have Go 1.26 installed, the go mod init command sets the version in go.mod to go 1.25.0. This is normal behavior for the Go 1.26 toolchain.
Seeing 'command not found'?
If go mod init is not recognized, your Go installation may not be on your system PATH. Close and reopen your terminal, then try go version to confirm Go is available.
Help me troubleshoot why go is not found in my terminal
Create the four source files
Your database engine has four components that each get their own file: the lexer (tokenizer), the parser (AST builder), the engine (storage and execution), and the main entry point (REPL).
- Create all four files by running this command:
touch main.go lexer.go parser.go engine.go- Confirm the files exist by running:
lsYou should see engine.go, go.mod, lexer.go, main.go, and parser.go in the output.
- Open the minidb folder in Cursor by clicking File in the top menu bar.
- Click Open Folder.
- Navigate to your Desktop, select the minidb folder, then click Open.
You should now see all four .go files listed in the Explorer sidebar on the left.
- Click on lexer.go in the Explorer sidebar to open it.
- Type package main on the first line and save the file.
- Repeat for parser.go and engine.go. Click each file, type package main on line 1, and save.
Why does every file say package main?
In Go, all files in the same directory that share the same package name are compiled together as one unit. By declaring package main in every file, Go knows they all belong to the same executable program. Functions in lexer.go can call functions in engine.go without any import statements.
Verify the project compiles
Go requires every package main to have a func main() entry point. Without it, the compiler will refuse to build. You will add a placeholder for now and replace it with the full REPL loop in a later step.
- Click on main.go in the Explorer sidebar.
- Replace any existing content with the following code:
package main
func main() {}What does this code do?
This is the minimum valid Go program. func main() {} is an empty entry point that does nothing yet. It satisfies the compiler so you can verify your project structure is correct before writing real logic.
- Save main.go.
- Open the integrated terminal in Cursor by pressing Ctrl+` (backtick).
- Compile the project by running:
go build .You should see no output at all. In Go, silence means success.
Seeing a compile error?
- If you see expected 'package', make sure every .go file has package main as its very first line. Empty files will cause this error.
- If you see runtime.main_main·f: function main is undeclared, check that main.go contains func main() {}.
- Make sure your terminal is inside the minidb folder. Run pwd to confirm you are in the right directory.
Help me fix my Go build error in minidb
✔️ Awesome, I've got everything!
Great. Make sure all your files are saved before moving on.
ⓧ I'd like to double check the full code
Here is what each file should contain at this point:
package main
func main() {}package mainpackage mainpackage mainYour go.mod should look like this:
module minidb
go 1.25.0Your project skeleton is ready to go. Next up, you will build the lexer that breaks raw SQL text into structured tokens.
Build the Lexer
Before your database can understand SQL, it needs to break raw text into meaningful pieces. This is exactly what a lexer (also called a tokenizer) does.
A lexer reads a SQL string character by character and groups those characters into typed tokens. For example, the word SELECT becomes a keyword token, the number 42 becomes a number token, and users becomes an identifier token.
In this step, get ready to:
- Define token types for SQL keywords, literals, and symbols.
- Build a keyword lookup map for case-insensitive matching.
- Implement the Tokenize function that walks through input character by character.
Define the token types and struct
Every token your lexer produces needs a type (what kind of token it is) and a value (the original text). You'll define a set of constants to represent all possible token types.
- Open lexer.go in Cursor.
- Replace the contents of the file with the following code that defines your token types and Token struct:
package main
import (
"strings"
"unicode"
)
type TokenType int
const (
TokenSelect TokenType = iota
TokenFrom
TokenWhere
TokenInsert
TokenInto
TokenValues
TokenCreate
TokenTable
TokenAnd
TokenOr
TokenInt
TokenText
TokenIdentifier
TokenNumber
TokenString
TokenLeftParen
TokenRightParen
TokenComma
TokenSemicolon
TokenAsterisk
TokenEquals
TokenNotEquals
TokenLessThan
TokenGreaterThan
TokenEOF
)
type Token struct {
Type TokenType
Value string
}What does this code do?
TokenType int creates a custom integer type to represent token categories. The iota keyword auto-assigns incrementing numbers starting from 0, so TokenSelect = 0, TokenFrom = 1, and so on.
The Token struct holds the type (which category) and the original text value of each token. The imports bring in strings (for uppercasing keywords) and unicode (for checking character types).
- Save lexer.go.
- You won't see a change yet. The next chunk adds the keywords map that makes these types useful.
Create the keywords map
SQL is case-insensitive, so select, SELECT, and Select should all produce the same token. A lookup map solves this by mapping uppercase keywords to their token types.
- In lexer.go, add the following keywords map directly below the Token struct:
var keywords = map[string]TokenType{
"SELECT": TokenSelect,
"FROM": TokenFrom,
"WHERE": TokenWhere,
"INSERT": TokenInsert,
"INTO": TokenInto,
"VALUES": TokenValues,
"CREATE": TokenCreate,
"TABLE": TokenTable,
"AND": TokenAnd,
"OR": TokenOr,
"INT": TokenInt,
"TEXT": TokenText,
}What does this code do?
When the lexer encounters a word like select, it uppercases it and checks this map. If the uppercased word exists as a key, it's a keyword token. If not, it's treated as an identifier (a table or column name).
- Save lexer.go.
- You won't see a change yet. The next chunk uses this map inside the Tokenize function.
Implement the Tokenize function
This is the core of your lexer. The Tokenize function walks through the input string character by character, deciding what kind of token each sequence forms.
- In lexer.go, add the Tokenize function below the keywords map:
func Tokenize(input string) []Token {
var tokens []Token
i := 0
for i < len(input) {
// Skip whitespace
if unicode.IsSpace(rune(input[i])) {
i++
continue
}
// Handle single-character symbols
switch input[i] {
case '(':
tokens = append(tokens, Token{TokenLeftParen, "("})
i++
continue
case ')':
tokens = append(tokens, Token{TokenRightParen, ")"})
i++
continue
case ',':
tokens = append(tokens, Token{TokenComma, ","})
i++
continue
case ';':
tokens = append(tokens, Token{TokenSemicolon, ";"})
i++
continue
case '*':
tokens = append(tokens, Token{TokenAsterisk, "*"})
i++
continue
case '=':
tokens = append(tokens, Token{TokenEquals, "="})
i++
continue
case '<':
tokens = append(tokens, Token{TokenLessThan, "<"})
i++
continue
case '>':
tokens = append(tokens, Token{TokenGreaterThan, ">"})
i++
continue
case '!':
// Handle two-character != operator
if i+1 < len(input) && input[i+1] == '=' {
tokens = append(tokens, Token{TokenNotEquals, "!="})
i += 2
continue
}
}
// Handle single-quoted strings
if input[i] == '\'' {
i++
start := i
for i < len(input) && input[i] != '\'' {
i++
}
tokens = append(tokens, Token{TokenString, input[start:i]})
if i < len(input) {
i++
}
continue
}
// Handle numbers (sequences of digits)
if unicode.IsDigit(rune(input[i])) {
start := i
for i < len(input) && unicode.IsDigit(rune(input[i])) {
i++
}
tokens = append(tokens, Token{TokenNumber, input[start:i]})
continue
}
// Handle words (keywords or identifiers)
if unicode.IsLetter(rune(input[i])) || input[i] == '_' {
start := i
for i < len(input) && (unicode.IsLetter(rune(input[i])) || unicode.IsDigit(rune(input[i])) || input[i] == '_') {
i++
}
word := input[start:i]
upper := strings.ToUpper(word)
// Check if the word is a SQL keyword
if tokenType, ok := keywords[upper]; ok {
tokens = append(tokens, Token{tokenType, upper})
} else {
tokens = append(tokens, Token{TokenIdentifier, word})
}
continue
}
// Skip unknown characters
i++
}
// Append EOF sentinel token
tokens = append(tokens, Token{TokenEOF, ""})
return tokens
}What does this code do?
The function uses an index variable i to walk through the input one character at a time. At each position, it decides what kind of token starts there.
- Whitespace is skipped entirely (it separates tokens but never produces one).
- Single characters like (, *, = are matched via a switch statement.
- The != operator is checked before consuming a bare ! to avoid splitting a two-character operator.
- Strings are everything between single quotes. Numbers are sequences of digits.
- Words are uppercased and checked against the keywords map to distinguish keywords from identifiers.
- A final TokenEOF sentinel marks the end of input so the parser always knows when to stop.
- Save lexer.go.
Seeing a compile error?
- Make sure the import block at the top of lexer.go includes both "strings" and "unicode".
- Check that every opening brace { has a matching closing brace }. Missing braces are the most common cause of Go compile errors.
- Verify that the escaped single quotes '\'' in the string-handling section are typed correctly.
Help me fix a compile error in my lexer.go file
✔️ Awesome, I've got everything!
Great. Double check you have saved lexer.go before moving on.
ⓧ I'd like to double check the full code
package main
import (
"strings"
"unicode"
)
type TokenType int
const (
TokenSelect TokenType = iota
TokenFrom
TokenWhere
TokenInsert
TokenInto
TokenValues
TokenCreate
TokenTable
TokenAnd
TokenOr
TokenInt
TokenText
TokenIdentifier
TokenNumber
TokenString
TokenLeftParen
TokenRightParen
TokenComma
TokenSemicolon
TokenAsterisk
TokenEquals
TokenNotEquals
TokenLessThan
TokenGreaterThan
TokenEOF
)
type Token struct {
Type TokenType
Value string
}
var keywords = map[string]TokenType{
"SELECT": TokenSelect,
"FROM": TokenFrom,
"WHERE": TokenWhere,
"INSERT": TokenInsert,
"INTO": TokenInto,
"VALUES": TokenValues,
"CREATE": TokenCreate,
"TABLE": TokenTable,
"AND": TokenAnd,
"OR": TokenOr,
"INT": TokenInt,
"TEXT": TokenText,
}
func Tokenize(input string) []Token {
var tokens []Token
i := 0
for i < len(input) {
if unicode.IsSpace(rune(input[i])) {
i++
continue
}
switch input[i] {
case '(':
tokens = append(tokens, Token{TokenLeftParen, "("})
i++
continue
case ')':
tokens = append(tokens, Token{TokenRightParen, ")"})
i++
continue
case ',':
tokens = append(tokens, Token{TokenComma, ","})
i++
continue
case ';':
tokens = append(tokens, Token{TokenSemicolon, ";"})
i++
continue
case '*':
tokens = append(tokens, Token{TokenAsterisk, "*"})
i++
continue
case '=':
tokens = append(tokens, Token{TokenEquals, "="})
i++
continue
case '<':
tokens = append(tokens, Token{TokenLessThan, "<"})
i++
continue
case '>':
tokens = append(tokens, Token{TokenGreaterThan, ">"})
i++
continue
case '!':
if i+1 < len(input) && input[i+1] == '=' {
tokens = append(tokens, Token{TokenNotEquals, "!="})
i += 2
continue
}
}
if input[i] == '\'' {
i++
start := i
for i < len(input) && input[i] != '\'' {
i++
}
tokens = append(tokens, Token{TokenString, input[start:i]})
if i < len(input) {
i++
}
continue
}
if unicode.IsDigit(rune(input[i])) {
start := i
for i < len(input) && unicode.IsDigit(rune(input[i])) {
i++
}
tokens = append(tokens, Token{TokenNumber, input[start:i]})
continue
}
if unicode.IsLetter(rune(input[i])) || input[i] == '_' {
start := i
for i < len(input) && (unicode.IsLetter(rune(input[i])) || unicode.IsDigit(rune(input[i])) || input[i] == '_') {
i++
}
word := input[start:i]
upper := strings.ToUpper(word)
if tokenType, ok := keywords[upper]; ok {
tokens = append(tokens, Token{tokenType, upper})
} else {
tokens = append(tokens, Token{TokenIdentifier, word})
}
continue
}
i++
}
tokens = append(tokens, Token{TokenEOF, ""})
return tokens
}Test your lexer
Your lexer is complete but you have no way to see it work yet. You'll add temporary test code to main.go to tokenize a sample query and print the results.
- Open main.go and temporarily replace its contents with the following test code:
package main
import "fmt"
func main() {
// Test the lexer with a sample SQL query
tokens := Tokenize("SELECT name, age FROM users WHERE age > 21 AND name = 'Alice'")
for _, t := range tokens {
fmt.Printf("Type: %2d | Value: %q\n", t.Type, t.Value)
}
}What does this code do?
This passes a SQL query string into your Tokenize function, then prints each token's numeric type and string value. You'll use the output to verify that keywords, identifiers, numbers, strings, and symbols are all categorized correctly.
- Save main.go.
- Run your program by running:
go run .You should see a list of tokens printed to your terminal. Each line shows a numeric type and the token's value. Verify that SELECT has type 0 (the first constant), name has type 12 (TokenIdentifier), and Alice has type 14 (TokenString).
Not seeing the expected output?
- If you see an error about Tokenize being undefined, make sure lexer.go is saved and starts with package main. Go compiles all files in the same package together.
- If keyword tokens show up as identifiers (type 12 instead of 0-11), check that your keywords map keys are uppercase and you are calling strings.ToUpper on the word before looking it up.
- If you get no output at all, confirm that main.go has the test code (not the empty placeholder from earlier).
Help me debug my lexer output
- Once you've confirmed the output looks correct, revert main.go back to its placeholder state:
package main
func main() {}- Save main.go.
Why revert main.go?
You'll replace this placeholder with the full REPL in a later step. Keeping the test code around would conflict with that.
Your lexer can now break any SQL string into structured tokens. Next up, you'll build a parser that takes these tokens and assembles them into an abstract syntax tree representing the full SQL statement.
Build the Parser
Your lexer can break SQL strings into tokens, but tokens alone are just a flat list. The parser's job is to give that list structure by recognizing patterns (like CREATE TABLE name (columns)) and building a tree that represents the query's meaning.
This tree is called an Abstract Syntax Tree (AST). In this step, you will build a recursive-descent parser that consumes tokens left-to-right and produces an AST for CREATE TABLE, INSERT, and SELECT statements.
In this step, get ready to:
- Define AST struct types for each SQL statement.
- Build the parser's token-walking helpers and dispatcher.
- Implement parse methods for CREATE, INSERT, and SELECT (with WHERE filtering).
Define the AST types and parser core
Before writing any parsing logic, you need Go structs that represent what a parsed SQL statement looks like. Each statement type gets its own struct, and a wrapper Statement struct holds a pointer to whichever specific type was parsed.
- In parser.go, replace the single package main line with this import and the AST struct definitions:
package main
import (
"fmt"
)
type ColumnDef struct {
Name string `json:"name"`
Type string `json:"type"`
}
type CreateTableStatement struct {
TableName string
Columns []ColumnDef
}
type InsertStatement struct {
TableName string
Columns []string
Values []string
}
type Condition struct {
Column string
Operator string
Value string
}
type WhereClause struct {
Conditions []Condition
Operators []string
}
type SelectStatement struct {
Columns []string
TableName string
Where *WhereClause
}
type Statement struct {
Type string
Create *CreateTableStatement
Insert *InsertStatement
Select *SelectStatement
}What do these structs represent?
Each struct maps to one SQL grammar rule:
- ColumnDef holds a column name and its type (INT or TEXT). The json tags let the storage engine serialize columns to disk later.
- CreateTableStatement, InsertStatement, and SelectStatement capture all the information needed to execute each query type.
- Statement is a wrapper. Only one of its pointers (Create, Insert, or Select) will be non-nil at a time.
- WhereClause stores conditions and the logical operators (AND/OR) between them.
- Below the struct definitions, add the Parser struct and its helper methods:
type Parser struct {
tokens []Token
pos int
}
func NewParser(tokens []Token) *Parser {
return &Parser{tokens: tokens, pos: 0}
}
// peek returns the current token without advancing
func (p *Parser) peek() Token {
if p.pos >= len(p.tokens) {
return Token{TokenEOF, ""}
}
return p.tokens[p.pos]
}
// next consumes the current token and advances
func (p *Parser) next() Token {
token := p.peek()
p.pos++
return token
}
// expect consumes a token and returns an error if it is not the expected type
func (p *Parser) expect(tokenType TokenType) (Token, error) {
token := p.next()
if token.Type != tokenType {
return token, fmt.Errorf("expected token type %d, got %d (%q)", tokenType, token.Type, token.Value)
}
return token, nil
}
// Parse dispatches to the correct parse method based on the first token
func (p *Parser) Parse() (*Statement, error) {
token := p.peek()
switch token.Type {
case TokenCreate:
return p.parseCreate()
case TokenInsert:
return p.parseInsert()
case TokenSelect:
return p.parseSelect()
default:
return nil, fmt.Errorf("unexpected token: %q", token.Value)
}
}What does this code do?
The parser walks through tokens using three primitives: peek() to look at the current token without consuming it, next() to consume and advance, and expect() to consume while asserting the type.
The Parse() method peeks at the first token to decide which statement type is being parsed, then hands off to the appropriate method. This pattern is called recursive-descent parsing.
- Save parser.go.
Seeing compile errors?
If you see fmt imported and not used, that is expected for now. The import is needed by the methods you are about to add. If you see other errors, make sure the import block is at the top of the file and all struct definitions use correct Go syntax (capital field names, backtick-quoted struct tags).
help me debug a compile error in parser.go
Implement CREATE and INSERT parsers
Each parse method follows the same pattern: consume tokens left-to-right according to the grammar, branching where optionality exists. You will now add the methods that Parse() dispatches to.
- Below the Parse() method, add the parseCreate() function:
func (p *Parser) parseCreate() (*Statement, error) {
// Consume the CREATE token
p.next()
if _, err := p.expect(TokenTable); err != nil {
return nil, fmt.Errorf("expected TABLE after CREATE")
}
// Get the table name
nameToken, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected table name")
}
// Expect opening parenthesis for column definitions
if _, err := p.expect(TokenLeftParen); err != nil {
return nil, fmt.Errorf("expected '(' after table name")
}
// Parse column definitions: name type, name type, ...
var columns []ColumnDef
for {
colName, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected column name")
}
colType := p.next()
if colType.Type != TokenInt && colType.Type != TokenText {
return nil, fmt.Errorf("expected column type (INT or TEXT), got %q", colType.Value)
}
columns = append(columns, ColumnDef{Name: colName.Value, Type: colType.Value})
// If next token is a comma, there are more columns
if p.peek().Type == TokenComma {
p.next()
} else {
break
}
}
// Expect closing parenthesis
if _, err := p.expect(TokenRightParen); err != nil {
return nil, fmt.Errorf("expected ')' after column definitions")
}
return &Statement{
Type: "CREATE",
Create: &CreateTableStatement{
TableName: nameToken.Value,
Columns: columns,
},
}, nil
}What does this code do?
This method consumes tokens that match the grammar: CREATE TABLE <name> (<col1> <type1>, <col2> <type2>, ...). The for loop handles any number of column definitions separated by commas. It breaks out of the loop when there is no comma following a column definition.
Every unexpected token produces a descriptive error message rather than a panic. This keeps the parser safe and debuggable.
- Below parseCreate(), add the parseInsert() function:
func (p *Parser) parseInsert() (*Statement, error) {
// Consume the INSERT token
p.next()
if _, err := p.expect(TokenInto); err != nil {
return nil, fmt.Errorf("expected INTO after INSERT")
}
// Get the table name
nameToken, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected table name")
}
// Optional column list: (col1, col2, ...)
var columns []string
if p.peek().Type == TokenLeftParen {
p.next()
for {
col, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected column name")
}
columns = append(columns, col.Value)
if p.peek().Type == TokenComma {
p.next()
} else {
break
}
}
if _, err := p.expect(TokenRightParen); err != nil {
return nil, fmt.Errorf("expected ')' after column list")
}
}
// Expect VALUES keyword
if _, err := p.expect(TokenValues); err != nil {
return nil, fmt.Errorf("expected VALUES")
}
// Parse the values list: (val1, val2, ...)
if _, err := p.expect(TokenLeftParen); err != nil {
return nil, fmt.Errorf("expected '(' after VALUES")
}
var values []string
for {
val := p.next()
if val.Type == TokenNumber || val.Type == TokenString {
values = append(values, val.Value)
} else {
return nil, fmt.Errorf("expected value, got %q", val.Value)
}
if p.peek().Type == TokenComma {
p.next()
} else {
break
}
}
if _, err := p.expect(TokenRightParen); err != nil {
return nil, fmt.Errorf("expected ')' after values")
}
return &Statement{
Type: "INSERT",
Insert: &InsertStatement{
TableName: nameToken.Value,
Columns: columns,
Values: values,
},
}, nil
}What does this code do?
This method handles the grammar: INSERT INTO <table> [(<cols>)] VALUES (<vals>). The column list in square brackets is optional. The parser checks whether the next token is a ( to decide if columns were specified.
If no column list is given, the columns slice stays empty. The executor will later map values to columns in table-definition order.
- Save parser.go.
- Confirm the project still compiles by running:
go build .You should see no output, which means success.
Build failing?
Check that every opening brace { has a matching closing brace }. Also confirm that the method receivers say (p *Parser) and not just (p Parser). The pointer receiver is required because next() modifies pos.
help me fix my Go build error in parser.go
Implement the SELECT parser with WHERE filtering
SELECT is the most complex statement because it has an optional WHERE clause. The WHERE clause itself can contain multiple conditions chained with AND or OR.
- Below parseInsert(), add the parseSelect() function:
func (p *Parser) parseSelect() (*Statement, error) {
// Consume the SELECT token
p.next()
// Parse column list: * or col1, col2, ...
var columns []string
if p.peek().Type == TokenAsterisk {
columns = append(columns, "*")
p.next()
} else {
for {
col, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected column name")
}
columns = append(columns, col.Value)
if p.peek().Type == TokenComma {
p.next()
} else {
break
}
}
}
// Expect FROM keyword and table name
if _, err := p.expect(TokenFrom); err != nil {
return nil, fmt.Errorf("expected FROM")
}
nameToken, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected table name")
}
// Optional WHERE clause
var where *WhereClause
if p.peek().Type == TokenWhere {
p.next()
where, err = p.parseWhere()
if err != nil {
return nil, err
}
}
return &Statement{
Type: "SELECT",
Select: &SelectStatement{
Columns: columns,
TableName: nameToken.Value,
Where: where,
},
}, nil
}What does this code do?
This method handles: SELECT <cols> FROM <table> [WHERE <conditions>]. It first checks whether the column list is a * (select all) or a comma-separated list of identifiers.
The WHERE clause is optional. The parser peeks at the next token and only calls parseWhere() if it sees a WHERE keyword. If there is no WHERE, the where pointer stays nil.
- Below parseSelect(), add the parseWhere() helper function:
func (p *Parser) parseWhere() (*WhereClause, error) {
where := &WhereClause{}
for {
// Each condition is: column operator value
col, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected column name in WHERE clause")
}
op := p.next()
if op.Type != TokenEquals && op.Type != TokenNotEquals && op.Type != TokenLessThan && op.Type != TokenGreaterThan {
return nil, fmt.Errorf("expected operator, got %q", op.Value)
}
val := p.next()
if val.Type != TokenNumber && val.Type != TokenString {
return nil, fmt.Errorf("expected value in WHERE clause, got %q", val.Value)
}
where.Conditions = append(where.Conditions, Condition{
Column: col.Value,
Operator: op.Value,
Value: val.Value,
})
// Check for AND/OR to chain more conditions
if p.peek().Type == TokenAnd {
where.Operators = append(where.Operators, "AND")
p.next()
} else if p.peek().Type == TokenOr {
where.Operators = append(where.Operators, "OR")
p.next()
} else {
break
}
}
return where, nil
}What does this code do?
Each WHERE condition follows the pattern column operator value. The loop continues consuming conditions as long as it sees AND or OR tokens between them. The operators are stored separately so the executor can evaluate them in order.
Supported operators are =, !=, <, and >. Any other token in the operator position produces an error.
- Save parser.go.
- Verify the project compiles cleanly by running:
go build .No output means all four files compiled successfully together.
Seeing undeclared name errors?
If Go says a type like TokenIdentifier is undeclared, make sure your lexer.go file still has package main at the top. All files in the same directory with package main share types automatically. Also make sure you did not accidentally delete any token constants from lexer.go.
help me fix undeclared name errors when building my Go project
✔️ Awesome, I've got everything!
Great. Double check you have saved parser.go and that go build . shows no output.
ⓧ I'd like to double check the full code
Here is the complete parser.go file. Compare it against yours:
package main
import (
"fmt"
)
type ColumnDef struct {
Name string `json:"name"`
Type string `json:"type"`
}
type CreateTableStatement struct {
TableName string
Columns []ColumnDef
}
type InsertStatement struct {
TableName string
Columns []string
Values []string
}
type Condition struct {
Column string
Operator string
Value string
}
type WhereClause struct {
Conditions []Condition
Operators []string
}
type SelectStatement struct {
Columns []string
TableName string
Where *WhereClause
}
type Statement struct {
Type string
Create *CreateTableStatement
Insert *InsertStatement
Select *SelectStatement
}
type Parser struct {
tokens []Token
pos int
}
func NewParser(tokens []Token) *Parser {
return &Parser{tokens: tokens, pos: 0}
}
func (p *Parser) peek() Token {
if p.pos >= len(p.tokens) {
return Token{TokenEOF, ""}
}
return p.tokens[p.pos]
}
func (p *Parser) next() Token {
token := p.peek()
p.pos++
return token
}
func (p *Parser) expect(tokenType TokenType) (Token, error) {
token := p.next()
if token.Type != tokenType {
return token, fmt.Errorf("expected token type %d, got %d (%q)", tokenType, token.Type, token.Value)
}
return token, nil
}
func (p *Parser) Parse() (*Statement, error) {
token := p.peek()
switch token.Type {
case TokenCreate:
return p.parseCreate()
case TokenInsert:
return p.parseInsert()
case TokenSelect:
return p.parseSelect()
default:
return nil, fmt.Errorf("unexpected token: %q", token.Value)
}
}
func (p *Parser) parseCreate() (*Statement, error) {
p.next()
if _, err := p.expect(TokenTable); err != nil {
return nil, fmt.Errorf("expected TABLE after CREATE")
}
nameToken, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected table name")
}
if _, err := p.expect(TokenLeftParen); err != nil {
return nil, fmt.Errorf("expected '(' after table name")
}
var columns []ColumnDef
for {
colName, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected column name")
}
colType := p.next()
if colType.Type != TokenInt && colType.Type != TokenText {
return nil, fmt.Errorf("expected column type (INT or TEXT), got %q", colType.Value)
}
columns = append(columns, ColumnDef{Name: colName.Value, Type: colType.Value})
if p.peek().Type == TokenComma {
p.next()
} else {
break
}
}
if _, err := p.expect(TokenRightParen); err != nil {
return nil, fmt.Errorf("expected ')' after column definitions")
}
return &Statement{
Type: "CREATE",
Create: &CreateTableStatement{
TableName: nameToken.Value,
Columns: columns,
},
}, nil
}
func (p *Parser) parseInsert() (*Statement, error) {
p.next()
if _, err := p.expect(TokenInto); err != nil {
return nil, fmt.Errorf("expected INTO after INSERT")
}
nameToken, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected table name")
}
var columns []string
if p.peek().Type == TokenLeftParen {
p.next()
for {
col, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected column name")
}
columns = append(columns, col.Value)
if p.peek().Type == TokenComma {
p.next()
} else {
break
}
}
if _, err := p.expect(TokenRightParen); err != nil {
return nil, fmt.Errorf("expected ')' after column list")
}
}
if _, err := p.expect(TokenValues); err != nil {
return nil, fmt.Errorf("expected VALUES")
}
if _, err := p.expect(TokenLeftParen); err != nil {
return nil, fmt.Errorf("expected '(' after VALUES")
}
var values []string
for {
val := p.next()
if val.Type == TokenNumber || val.Type == TokenString {
values = append(values, val.Value)
} else {
return nil, fmt.Errorf("expected value, got %q", val.Value)
}
if p.peek().Type == TokenComma {
p.next()
} else {
break
}
}
if _, err := p.expect(TokenRightParen); err != nil {
return nil, fmt.Errorf("expected ')' after values")
}
return &Statement{
Type: "INSERT",
Insert: &InsertStatement{
TableName: nameToken.Value,
Columns: columns,
Values: values,
},
}, nil
}
func (p *Parser) parseSelect() (*Statement, error) {
p.next()
var columns []string
if p.peek().Type == TokenAsterisk {
columns = append(columns, "*")
p.next()
} else {
for {
col, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected column name")
}
columns = append(columns, col.Value)
if p.peek().Type == TokenComma {
p.next()
} else {
break
}
}
}
if _, err := p.expect(TokenFrom); err != nil {
return nil, fmt.Errorf("expected FROM")
}
nameToken, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected table name")
}
var where *WhereClause
if p.peek().Type == TokenWhere {
p.next()
where, err = p.parseWhere()
if err != nil {
return nil, err
}
}
return &Statement{
Type: "SELECT",
Select: &SelectStatement{
Columns: columns,
TableName: nameToken.Value,
Where: where,
},
}, nil
}
func (p *Parser) parseWhere() (*WhereClause, error) {
where := &WhereClause{}
for {
col, err := p.expect(TokenIdentifier)
if err != nil {
return nil, fmt.Errorf("expected column name in WHERE clause")
}
op := p.next()
if op.Type != TokenEquals && op.Type != TokenNotEquals && op.Type != TokenLessThan && op.Type != TokenGreaterThan {
return nil, fmt.Errorf("expected operator, got %q", op.Value)
}
val := p.next()
if val.Type != TokenNumber && val.Type != TokenString {
return nil, fmt.Errorf("expected value in WHERE clause, got %q", val.Value)
}
where.Conditions = append(where.Conditions, Condition{
Column: col.Value,
Operator: op.Value,
Value: val.Value,
})
if p.peek().Type == TokenAnd {
where.Operators = append(where.Operators, "AND")
p.next()
} else if p.peek().Type == TokenOr {
where.Operators = append(where.Operators, "OR")
p.next()
} else {
break
}
}
return where, nil
}Your parser is complete. It can turn a flat token list into a structured AST for any CREATE TABLE, INSERT, or SELECT query. Next up, you will build the storage engine and executor that bring these parsed statements to life.
Build the Storage Engine, Executor, and REPL
Your lexer breaks SQL into tokens. Your parser organizes those tokens into an AST. Now you need the part that actually does something with those structures.
In this step, you'll build the storage engine (where rows live in memory and on disk), the executor (which reads the AST and modifies or queries data), and a REPL (so you can type SQL interactively and see results).
In this step, get ready to:
- Define the data structures and persistence layer that store your tables on disk as JSON.
- Implement query executors for CREATE TABLE, INSERT, and SELECT with WHERE filtering.
- Wire up an interactive REPL that pipes user input through the full tokenize-parse-execute pipeline.
Define the storage engine
The storage engine is the foundation everything else sits on. It defines how rows, tables, and the database itself are represented in memory, and how they get saved to (and loaded from) a JSON file on disk.
- In engine.go, replace the placeholder content with the following storage engine code:
package main
import (
"encoding/json"
"fmt"
"os"
"strconv"
)
// Row holds a single database row as a map of column name to value
type Row struct {
Values map[string]string `json:"values"`
}
// Table holds the schema (columns) and all rows for one table
type Table struct {
Name string `json:"name"`
Columns []ColumnDef `json:"columns"`
Rows []Row `json:"rows"`
}
// Database holds all tables and the file path for persistence
type Database struct {
Tables map[string]*Table `json:"tables"`
path string
}
// NewDatabase creates a database and loads existing data from disk
func NewDatabase(path string) *Database {
db := &Database{
Tables: make(map[string]*Table),
path: path,
}
db.load()
return db
}
// save writes all tables to disk as indented JSON
func (db *Database) save() {
data, err := json.MarshalIndent(db.Tables, "", " ")
if err != nil {
return
}
os.WriteFile(db.path, data, 0644)
}
// load reads tables from disk if the file exists
func (db *Database) load() {
data, err := os.ReadFile(db.path)
if err != nil {
return
}
json.Unmarshal(data, &db.Tables)
}What does this code do?
Row stores column values as a string map. Every value is a string internally, even numbers. The executor handles type conversion when comparing.
Table pairs a schema (the Columns slice of ColumnDef from your parser) with the actual data (a slice of Row). The json struct tags tell Go how to serialize each field.
NewDatabase immediately calls load(), so if a minidb.json file already exists from a previous session, your data reappears automatically.
Why JSON for persistence?
Production databases use binary formats like B-trees or write-ahead logs. JSON is slower and larger, but you can open minidb.json in any text editor and see exactly what your database stored. That debuggability is worth the tradeoff for learning.
Implement the query executor
The executor takes a parsed AST and turns it into actions: creating a table, inserting a row, or querying data. Each statement type gets its own execution function.
- Add the Execute dispatcher, executeCreate, and executeInsert functions below the load() method in engine.go:
// Execute dispatches a parsed statement to the correct executor
func (db *Database) Execute(stmt *Statement) (string, error) {
switch stmt.Type {
case "CREATE":
return db.executeCreate(stmt.Create)
case "INSERT":
return db.executeInsert(stmt.Insert)
case "SELECT":
return db.executeSelect(stmt.Select)
default:
return "", fmt.Errorf("unknown statement type: %s", stmt.Type)
}
}
// executeCreate adds a new table to the database
func (db *Database) executeCreate(stmt *CreateTableStatement) (string, error) {
if _, exists := db.Tables[stmt.TableName]; exists {
return "", fmt.Errorf("table %q already exists", stmt.TableName)
}
db.Tables[stmt.TableName] = &Table{
Name: stmt.TableName,
Columns: stmt.Columns,
Rows: []Row{},
}
db.save()
return fmt.Sprintf("Table %q created.", stmt.TableName), nil
}
// executeInsert adds a row to an existing table
func (db *Database) executeInsert(stmt *InsertStatement) (string, error) {
table, exists := db.Tables[stmt.TableName]
if !exists {
return "", fmt.Errorf("table %q does not exist", stmt.TableName)
}
columns := stmt.Columns
if len(columns) == 0 {
for _, col := range table.Columns {
columns = append(columns, col.Name)
}
}
if len(columns) != len(stmt.Values) {
return "", fmt.Errorf("column count (%d) does not match value count (%d)", len(columns), len(stmt.Values))
}
row := Row{Values: make(map[string]string)}
for i, col := range columns {
row.Values[col] = stmt.Values[i]
}
table.Rows = append(table.Rows, row)
db.save()
return "1 row inserted.", nil
}What does this code do?
Execute is a simple router. It reads the stmt.Type field your parser set and calls the matching executor.
executeCreate checks for duplicates, builds a Table struct with the column definitions from the AST, and persists immediately. executeInsert maps values to columns (using the table's schema order if no columns were specified in the SQL), then appends the row and persists.
- Next, add the executeSelect, evaluateWhere, and evaluateCondition functions below executeInsert:
// executeSelect queries rows from a table, filtering with WHERE if present
func (db *Database) executeSelect(stmt *SelectStatement) (string, error) {
table, exists := db.Tables[stmt.TableName]
if !exists {
return "", fmt.Errorf("table %q does not exist", stmt.TableName)
}
columns := stmt.Columns
if len(columns) == 1 && columns[0] == "*" {
columns = make([]string, len(table.Columns))
for i, col := range table.Columns {
columns[i] = col.Name
}
}
var rows []Row
for _, row := range table.Rows {
if stmt.Where == nil || evaluateWhere(row, stmt.Where) {
rows = append(rows, row)
}
}
return formatTable(columns, rows), nil
}
// evaluateWhere checks if a row matches all WHERE conditions
func evaluateWhere(row Row, where *WhereClause) bool {
if len(where.Conditions) == 0 {
return true
}
results := make([]bool, len(where.Conditions))
for i, cond := range where.Conditions {
results[i] = evaluateCondition(row, cond)
}
result := results[0]
for i, op := range where.Operators {
switch op {
case "AND":
result = result && results[i+1]
case "OR":
result = result || results[i+1]
}
}
return result
}
// evaluateCondition compares a single row value against a condition
func evaluateCondition(row Row, cond Condition) bool {
rowVal, exists := row.Values[cond.Column]
if !exists {
return false
}
// Try numeric comparison first
rowNum, err1 := strconv.Atoi(rowVal)
condNum, err2 := strconv.Atoi(cond.Value)
if err1 == nil && err2 == nil {
switch cond.Operator {
case "=":
return rowNum == condNum
case "!=":
return rowNum != condNum
case "<":
return rowNum < condNum
case ">":
return rowNum > condNum
}
}
// Fall back to string comparison
switch cond.Operator {
case "=":
return rowVal == cond.Value
case "!=":
return rowVal != cond.Value
case "<":
return rowVal < cond.Value
case ">":
return rowVal > cond.Value
}
return false
}How does WHERE evaluation work?
evaluateWhere evaluates each condition independently, then chains them left-to-right with AND/OR. This is a simplification. Real SQL gives AND higher precedence than OR, but left-to-right is fine for this project.
evaluateCondition tries to parse both sides as integers first. If both parse successfully, it compares numerically (so 9 < 10 is true). Otherwise it falls back to string comparison.
- Finally, add the formatTable function below evaluateCondition:
// formatTable produces aligned column output for SELECT results
func formatTable(columns []string, rows []Row) string {
if len(rows) == 0 {
return "(0 rows)"
}
// Calculate the max width needed for each column
widths := make([]int, len(columns))
for i, col := range columns {
widths[i] = len(col)
}
for _, row := range rows {
for i, col := range columns {
val := row.Values[col]
if len(val) > widths[i] {
widths[i] = len(val)
}
}
}
output := ""
// Header row
for i, col := range columns {
output += fmt.Sprintf("| %-*s ", widths[i], col)
}
output += "|\n"
// Separator line
for _, w := range widths {
output += "+"
for j := 0; j < w+2; j++ {
output += "-"
}
}
output += "+\n"
// Data rows
for _, row := range rows {
for i, col := range columns {
val := row.Values[col]
output += fmt.Sprintf("| %-*s ", widths[i], val)
}
output += "|\n"
}
output += fmt.Sprintf("(%d rows)", len(rows))
return output
}What does this code do?
formatTable calculates the widest value in each column, then pads every cell to that width. The result looks like a neat grid with headers, a separator line of dashes, data rows, and a count at the bottom.
- Save engine.go.
Seeing a compile error?
If you see an error about strconv or encoding/json being undefined, make sure your import block at the top of engine.go includes all four packages: encoding/json, fmt, os, and strconv.
help me fix a compile error in engine.go
✔️ Awesome, I've got everything!
Great. Make sure you've saved engine.go before moving on.
ⓧ I'd like to double check the full code
Here is the complete engine.go file:
package main
import (
"encoding/json"
"fmt"
"os"
"strconv"
)
type Row struct {
Values map[string]string `json:"values"`
}
type Table struct {
Name string `json:"name"`
Columns []ColumnDef `json:"columns"`
Rows []Row `json:"rows"`
}
type Database struct {
Tables map[string]*Table `json:"tables"`
path string
}
func NewDatabase(path string) *Database {
db := &Database{
Tables: make(map[string]*Table),
path: path,
}
db.load()
return db
}
func (db *Database) Execute(stmt *Statement) (string, error) {
switch stmt.Type {
case "CREATE":
return db.executeCreate(stmt.Create)
case "INSERT":
return db.executeInsert(stmt.Insert)
case "SELECT":
return db.executeSelect(stmt.Select)
default:
return "", fmt.Errorf("unknown statement type: %s", stmt.Type)
}
}
func (db *Database) executeCreate(stmt *CreateTableStatement) (string, error) {
if _, exists := db.Tables[stmt.TableName]; exists {
return "", fmt.Errorf("table %q already exists", stmt.TableName)
}
db.Tables[stmt.TableName] = &Table{
Name: stmt.TableName,
Columns: stmt.Columns,
Rows: []Row{},
}
db.save()
return fmt.Sprintf("Table %q created.", stmt.TableName), nil
}
func (db *Database) executeInsert(stmt *InsertStatement) (string, error) {
table, exists := db.Tables[stmt.TableName]
if !exists {
return "", fmt.Errorf("table %q does not exist", stmt.TableName)
}
columns := stmt.Columns
if len(columns) == 0 {
for _, col := range table.Columns {
columns = append(columns, col.Name)
}
}
if len(columns) != len(stmt.Values) {
return "", fmt.Errorf("column count (%d) does not match value count (%d)", len(columns), len(stmt.Values))
}
row := Row{Values: make(map[string]string)}
for i, col := range columns {
row.Values[col] = stmt.Values[i]
}
table.Rows = append(table.Rows, row)
db.save()
return "1 row inserted.", nil
}
func (db *Database) executeSelect(stmt *SelectStatement) (string, error) {
table, exists := db.Tables[stmt.TableName]
if !exists {
return "", fmt.Errorf("table %q does not exist", stmt.TableName)
}
columns := stmt.Columns
if len(columns) == 1 && columns[0] == "*" {
columns = make([]string, len(table.Columns))
for i, col := range table.Columns {
columns[i] = col.Name
}
}
var rows []Row
for _, row := range table.Rows {
if stmt.Where == nil || evaluateWhere(row, stmt.Where) {
rows = append(rows, row)
}
}
return formatTable(columns, rows), nil
}
func evaluateWhere(row Row, where *WhereClause) bool {
if len(where.Conditions) == 0 {
return true
}
results := make([]bool, len(where.Conditions))
for i, cond := range where.Conditions {
results[i] = evaluateCondition(row, cond)
}
result := results[0]
for i, op := range where.Operators {
switch op {
case "AND":
result = result && results[i+1]
case "OR":
result = result || results[i+1]
}
}
return result
}
func evaluateCondition(row Row, cond Condition) bool {
rowVal, exists := row.Values[cond.Column]
if !exists {
return false
}
rowNum, err1 := strconv.Atoi(rowVal)
condNum, err2 := strconv.Atoi(cond.Value)
if err1 == nil && err2 == nil {
switch cond.Operator {
case "=":
return rowNum == condNum
case "!=":
return rowNum != condNum
case "<":
return rowNum < condNum
case ">":
return rowNum > condNum
}
}
switch cond.Operator {
case "=":
return rowVal == cond.Value
case "!=":
return rowVal != cond.Value
case "<":
return rowVal < cond.Value
case ">":
return rowVal > cond.Value
}
return false
}
func formatTable(columns []string, rows []Row) string {
if len(rows) == 0 {
return "(0 rows)"
}
widths := make([]int, len(columns))
for i, col := range columns {
widths[i] = len(col)
}
for _, row := range rows {
for i, col := range columns {
val := row.Values[col]
if len(val) > widths[i] {
widths[i] = len(val)
}
}
}
output := ""
for i, col := range columns {
output += fmt.Sprintf("| %-*s ", widths[i], col)
}
output += "|\n"
for _, w := range widths {
output += "+"
for j := 0; j < w+2; j++ {
output += "-"
}
}
output += "+\n"
for _, row := range rows {
for i, col := range columns {
val := row.Values[col]
output += fmt.Sprintf("| %-*s ", widths[i], val)
}
output += "|\n"
}
output += fmt.Sprintf("(%d rows)", len(rows))
return output
}
func (db *Database) save() {
data, err := json.MarshalIndent(db.Tables, "", " ")
if err != nil {
return
}
os.WriteFile(db.path, data, 0644)
}
func (db *Database) load() {
data, err := os.ReadFile(db.path)
if err != nil {
return
}
json.Unmarshal(data, &db.Tables)
}Wire up the REPL
The REPL (Read-Eval-Print Loop) is the user interface for your database. It reads a line of SQL, pipes it through the full tokenize-parse-execute pipeline, and prints the result.
- Replace the entire contents of main.go with the following REPL code:
package main
import (
"bufio"
"fmt"
"os"
"strings"
)
func main() {
// Initialize the database, loading any existing data from disk
db := NewDatabase("minidb.json")
fmt.Println("MiniDB - A SQL database built from scratch")
fmt.Println("Type SQL commands or 'exit' to quit.")
fmt.Println()
// Read input line by line
scanner := bufio.NewScanner(os.Stdin)
for {
fmt.Print("minidb> ")
if !scanner.Scan() {
break
}
input := strings.TrimSpace(scanner.Text())
if input == "" {
continue
}
if strings.ToLower(input) == "exit" {
fmt.Println("Goodbye!")
break
}
// Strip trailing semicolons
input = strings.TrimSuffix(input, ";")
// Pipeline: Tokenize -> Parse -> Execute
tokens := Tokenize(input)
parser := NewParser(tokens)
stmt, err := parser.Parse()
if err != nil {
fmt.Printf("Parse error: %s\n", err)
continue
}
result, err := db.Execute(stmt)
if err != nil {
fmt.Printf("Error: %s\n", err)
continue
}
fmt.Println(result)
}
}What does this code do?
NewDatabase("minidb.json") creates your database instance and loads any previously saved data. The scanner loop reads one line at a time, trims whitespace and trailing semicolons, then pipes the input through your full pipeline: Tokenize to NewParser to Parse to Execute.
Errors at any stage (parse or execute) print a message and loop back to the prompt instead of crashing. The exit command breaks the loop cleanly.
- Save main.go.
- Run your database by running:
go run .You should see the welcome message and the minidb> prompt waiting for input.
- Test the full pipeline by entering these commands one at a time:
CREATE TABLE users (id INT, name TEXT, age INT)
INSERT INTO users VALUES (1, 'Alice', 30)
INSERT INTO users VALUES (2, 'Bob', 25)
SELECT * FROM users
SELECT name FROM users WHERE age > 27The SELECT * FROM users command should display a formatted table with both rows. The SELECT name FROM users WHERE age > 27 command should show only Alice.
- Type exit to quit the REPL.
- Verify persistence by running go run . again and entering:
SELECT * FROM usersYour two rows (Alice and Bob) should still be there. The data survived because NewDatabase called load() on startup and read from minidb.json.
Seeing a parse error or empty output?
If you see Parse error: unexpected token, check that you typed the SQL exactly as shown. Keywords like CREATE, INSERT, and SELECT are case-insensitive, but string values must be wrapped in single quotes.
If your data does not persist after restarting, confirm that minidb.json was created in your minidb directory. Run ls to check.
help me debug my MiniDB REPL
✔️ Awesome, I've got everything!
Great. Double check you've saved both files and that persistence works across restarts.
ⓧ I'd like to double check the full code
Here is the complete main.go file:
package main
import (
"bufio"
"fmt"
"os"
"strings"
)
func main() {
db := NewDatabase("minidb.json")
fmt.Println("MiniDB - A SQL database built from scratch")
fmt.Println("Type SQL commands or 'exit' to quit.")
fmt.Println()
scanner := bufio.NewScanner(os.Stdin)
for {
fmt.Print("minidb> ")
if !scanner.Scan() {
break
}
input := strings.TrimSpace(scanner.Text())
if input == "" {
continue
}
if strings.ToLower(input) == "exit" {
fmt.Println("Goodbye!")
break
}
input = strings.TrimSuffix(input, ";")
tokens := Tokenize(input)
parser := NewParser(tokens)
stmt, err := parser.Parse()
if err != nil {
fmt.Printf("Parse error: %s\n", err)
continue
}
result, err := db.Execute(stmt)
if err != nil {
fmt.Printf("Error: %s\n", err)
continue
}
fmt.Println(result)
}
}Your database engine is fully functional. You can create tables, insert rows, and query with WHERE filters. The data persists to disk between sessions. Next up, you'll extend this engine with DELETE and UPDATE support in the Secret Mission.
Secret mission
Add DELETE and UPDATE Support
Your database can create tables, insert rows, and query data. But what about removing rows or changing existing values? Real databases need DELETE and UPDATE. Extend all three layers of your engine to support these two new statement types.
Clean Up Your Resources
Clean Up Your Resources
This project runs entirely locally, so there are no ongoing costs or cloud resources to worry about.
Decide whether to keep your project files to continue experimenting, or delete them entirely.
Resources you used:
- The minidb project folder on your Desktop (contains main.go, lexer.go, parser.go, engine.go, go.mod, and minidb.json).
Keep everything running
No action needed. Choose this if you want to keep experimenting with your database engine, adding new features, or revisiting the code later.
- Your minidb folder stays on your Desktop.
- Your minidb.json file retains all the data you inserted during the project.
- You can run go run . anytime from the minidb directory to start your database again.
Pause - I'll come back to this later
Since this is a local project with no running processes, there is nothing to pause. Your files sit idle on disk until you return.
- If you want a fresh database next time, delete just the minidb.json file. Your engine will create a new empty one on the next run.
Delete - I don't want to use this again
Remove the entire project folder to start fresh. This deletes all source code and your persisted database file.
- On macOS/Linux, remove the project folder by running:
cd ~/Desktop
rm -rf minidb- On Windows, remove the project folder by running:
cd ~\Desktop
rmdir /s /q minidb- Confirm the folder is gone by running ls on your Desktop. You should no longer see minidb in the listing.
Nice Work!
Nice Work!
You just built a working SQL database engine from scratch in Go. You now understand the fundamental architecture that powers every production database: lexing, parsing, execution, and persistence.
You learned how to:
- Build a SQL lexer and recursive-descent parser that turns raw SQL text into a structured abstract syntax tree.
- Build an in-memory storage engine backed by Go maps with a query executor supporting CREATE TABLE, INSERT, and SELECT with WHERE filtering.
- Add JSON-based disk persistence so your database survives restarts, and an interactive REPL with formatted table output.
- Secret Mission: Extended the engine with DELETE and UPDATE support across all three layers (lexer, parser, executor).
Ready to quiz yourself?