dbtpl (part of the xo ecosystem) connects to a database, introspects the schema, and generates type-safe structs and query methods through Go templates. Hand-written DB structs easily drift from the schema; dbtpl makes the schema the single source of truth.

Explanation

How it works

Database (PostgreSQL / MySQL / SQLite...)
    ↓ introspect schema (tables, columns, types, indexes, FK...)
dbtpl
    ↓ apply Go templates
Generated Go code (structs, query methods)

No need to maintain DB structs by hand – when the schema changes, re-run dbtpl to sync.

Two modes

Schema mode: generate from the entire DB schema

1dbtpl schema postgres://user:pass@host/dbname -o ./models

Generates a Go struct for each table, including methods related to primary keys, foreign keys, and indexes.

Query mode: generate type-safe result structs from custom SQL

1dbtpl query postgres://user:pass@host/dbname << ENDSQL
2SELECT a.name::varchar AS name, b.type::integer AS my_type
3FROM authors a
4JOIN authortypes b ON a.id = b.author_id
5WHERE a.id = %%authorID int%%
6ENDSQL

%%param type%% is dbtpl’s query parameter syntax, which generates a corresponding function signature.

What gets generated

• Go structs matching each table (field types aligned with DB schema)
• CRUD query methods (Insert, Update, Delete, Get by PK)
• Enum types
• Struct tags (db:"column_name" etc.)

Template customization

dbtpl’s generation logic uses Go text/template. You can dump the built-in templates and modify them:

1dbtpl dump --src base -o ./custom-templates
2# edit the .tpl files in custom-templates/
3dbtpl schema postgres://... --src ./custom-templates

Knowledge Sugar

Why not just hand-write structs?

dbtpl vs sqlc

Both generate Go code from a database, but they differ in approach:

• sqlc: SQL-query-centric – write SQL first, then generate the corresponding functions
• dbtpl: schema-centric – introspects the entire DB, with more template flexibility

Foreign keys and JOINs solve two different problems: FK ensures data integrity at the write layer, while JOIN associates data at the read layer. Not having a FK does not prevent JOINs, but you lose the database-level safety net.

Explanation

Two core functions of a Foreign Key

Using shipping_fees (FK) -> shipping_classes (PK) as an example:

1. Referential integrity on insert: shipping_fees cannot contain a shipping_class_id that does not exist in shipping_classes
2. Cascading protection on delete: as long as any shipping_fees row references a shipping_class, that shipping_class row cannot be deleted

1-- With FK, the database blocks both of these:
2INSERT INTO shipping_fees (shipping_class_id, ...) VALUES (999, ...);
3-- ERROR: Cannot add or update a child row: foreign key constraint fails
4
5DELETE FROM shipping_classes WHERE id = 1;
6-- ERROR: Cannot delete or update a parent row: foreign key constraint fails

JOIN does something different

JOIN only associates data from two tables at query time. It does not care whether the data is valid:

1SELECT sf.*, sc.name
2FROM shipping_fees sf
3JOIN shipping_classes sc ON sf.shipping_class_id = sc.id;

You can JOIN without a FK, but if there are orphan records, the JOIN silently excludes them without raising an error.

Why do DBRE teams dislike FK?

FK causes trouble at the DB operations layer:

• Emergency data fixes via DML (direct INSERT / DELETE) can be blocked by FK constraints
• Large data migrations require temporarily disabling FK checks
• FK behavior gets more complex in replication setups

The compromise

Keep the JOIN design (no FK), but the DBRE team uses DML validation at the operations layer to ensure data correctness – essentially replacing FK’s two protections with a manual process.

Knowledge Sugar

FK vs Application-level validation vs DML validation

The nature of this tradeoff

FK means “let the database worry about data correctness.” No FK means “we worry about it ourselves.” The former is safer but sacrifices operational flexibility; the latter is more flexible but shifts responsibility to the application and operations layers.

For high-traffic systems or those with urgent on-call needs, DBRE teams tend to prefer the latter – they do not want to be stuck on a constraint during a critical moment.

When a MySQL lock is not properly released, other transactions trying to access the same data keep waiting indefinitely, causing batch jobs to get stuck (blocking).

Explanation

MySQL uses locks to protect data consistency – when a transaction is reading or writing a row, it acquires a lock on it first, forcing other transactions to queue up and wait.

Under normal circumstances, locks are automatically released after a transaction COMMITs or ROLLBACKs. But if a transaction gets stuck for some reason (e.g., a connection not properly closed, a process crash, or a long-running query), the lock persists and everything behind it piles up.

In this case, a transaction was holding a lock without finishing. The batch job kept waiting. The DBRE team manually terminated that session with a KILL command, which released the lock and allowed the batch job to resume.

Knowledge Sugar

Common lock types:

• Row lock: locks only the specific rows; fine-grained; InnoDB’s default
• Table lock: locks the entire table; coarse-grained; used by MyISAM
• Gap lock: locks the “gaps” between index ranges to prevent phantom reads

How to find out who is blocking whom:

1-- Check which locks are currently waiting
2SELECT * FROM information_schema.INNODB_LOCK_WAITS;
3
4-- View all active transactions
5SELECT * FROM information_schema.INNODB_TRX;
6
7-- Manually terminate a stuck session (what the DBRE team did)
8KILL <process_id>;

Why are batch jobs particularly prone to this? Batch jobs typically process large volumes of data with long-running transactions. If another transaction also needs the same set of rows, deadlocks or long blocking are more likely to occur.

Related concepts to explore further:

• Deadlock vs Blocking: deadlock is mutual waiting (both sides stuck); blocking is one-directional waiting
• InnoDB’s innodb_lock_wait_timeout setting (auto-abort on timeout)
• SELECT ... FOR UPDATE explicitly acquires a row lock