← Lab Runs in your browser
Two hundred transfers, none of them failed
Three accounts holding 1,000 each. Two hundred transfers between them, all running at once. Every transfer reports success, nothing raises an error, and at the end the three accounts hold 3,296 instead of 3,000. This page runs that in your browser, then runs the versions that get it right and shows what each one costs.
What goes wrong
Moving money from one account to another is four steps: read the sender's balance, check there is enough, take the amount off one account, add it to the other.
Run two transfers at the same time and those steps overlap. The first reads a balance of 1,000. Before it writes anything back, the second reads the same 1,000. Both then write a new balance they worked out from it. The second write lands on top of the first, and the money the first transfer took out is back where it started — having already been delivered to the other account.
Nothing here is an error. Each write is a valid statement setting a column to a number, and a database has no way to know you worked that number out from something already out of date. What is wrong is the relationship between two writes, and no single statement can see that.
So it does not arrive as an incident. It arrives weeks later as an account that is short, in a log full of successful transfers with no error anywhere to explain it.
Show the six lines that do it
const from = await db.read(t.from);
const to = await db.read(t.to);
if (from.balance < t.amount) throw new Error("insufficient funds");
await db.write(t.from, from.balance - t.amount, from.version);
await db.write(t.to, to.balance + t.amount, to.version);Every line is correct. The bug is in the gaps between them — four separate trips to the database, with other transfers running in between.
Run it
The accounts below are plain JavaScript objects, and every read and write waits a real turn of the event loop before it takes effect. That is all it takes: twenty transfers in flight at once genuinely interleave, so the lost updates here are lost the same way they are lost against a real server.
Both buttons run the same list of transfers. Only one of them ends with the money still adding up.
Loading the demo…
The third fix — serializable, and the retries it costs
Instead of locking anything, the database lets every transaction read
freely and then checks at commit time whether a row it read has changed
since. If one has, the transaction saw a version of the world that never
existed and is thrown out with 40001. Nothing is wrong with
your code; the contract is that you run the whole thing again.
Two transactions that are each correct and together wrong
Two doctors are on call. Either may go off call, provided somebody else is still on. Both check at the same moment, both see two doctors on call, both conclude it is fine, and each writes their own row. Nobody overwrote anybody. There is no doctor on call.
This is write skew, and it survives the fixes above: neither transaction touched a row the other touched, so watching for overwritten rows catches nothing. They only read what the other then changed. Serializable catches it because it tracks what a transaction read, not only what it wrote.
Deadlock, and the one line that fixes it
Locking only works if everyone agrees on an order. A transfer from account 1
to account 2 locks 1 then 2. One going the other way locks 2 then 1. Run
both at once and each holds what the other is waiting for, so the database
kills one with 40P01. The fix is to sort the accounts before
locking them — any order, as long as everything uses the same one.
On Postgres that sort() was worth more than it sounds: the same
40 transfers took 70,504 ms with 142 deadlocks and 18 of
them abandoned, against 96 ms and none. Postgres does not
look for a deadlock cycle until a lock has waited a full second, and 142
of those seconds is where the 70 went.
The fixes
Three of them, in the order I would reach for them. They are not interchangeable, and the first is both the cheapest and the most limited.
Let the database do the arithmetic
UPDATE accounts SET balance = balance - $1 never sends a balance through the application, so there is no number that can go stale on the way. It was the cheapest fix measured here and the fastest. It only works when the new value is a function of the old one.
Lock the rows before reading them
SELECT … FOR UPDATE takes the locks up front and holds them until the end, so there is no gap left for anyone to write into. This is the one to reach for when the decision genuinely needs a read first. The cost is waiting.
Serializable, and retry
The database checks at commit whether the transaction could have run in some serial order, and throws it out if not. Correctness moves into the database; the job of running the whole thing again moves into your code. Skip the retry and most of the work never happens.
What the fixes cost
The browser demo is a model. These are measurements: PostgreSQL 16.15 in Docker, three accounts opening at 1,000 each, 200 concurrent transfers over a pool of 20 connections, each row one run of the same file at a different setting.
| Version | Wall | Committed | Thrown out by the DB | Total after |
|---|---|---|---|---|
| Arithmetic in the app | 176 ms | 200 / 200 | 0 | 3,011 |
| balance = balance - $1 | 170 ms | 200 / 200 | 0 | 3,000 |
| SELECT … FOR UPDATE | 254 ms | 200 / 200 | 0 | 3,000 |
| Serializable, with retry | 398 ms | 137 / 200 | 659 | 3,000 |
| Serializable, no retry | 83 ms | 26 / 200 | 174 | 3,000 |
The first row is the bug: fastest, no errors, wrong answer. The second is the one worth knowing — letting the database do the arithmetic was correct and as fast as the broken version, 170 ms against 176. When it fits, it costs nothing.
Serializable is the row worth sitting with. It produced the right total by throwing out 659 transactions to get 200 transfers done, and 63 of those transfers never happened at all — they ran out of attempts. Without the retry wrapper it commits 26 of 200. The money is correct in every case; whether the work got done is a separate question, and the isolation level does not answer it for you.
This is a deliberately cruel workload — every transfer touches two of only three rows, so almost everything conflicts with almost everything. Spread the same transfers over ten thousand accounts and the retry rate falls close to nothing.
What this does not prove
The thing in your browser is not Postgres. It has no snapshots, no write-ahead log, no planner and no disk. It models the interleaving of reads and writes, and one commit-time rule, because that is the part the page is about.
It also overstates the damage. Postgres locks a row for the duration of an
UPDATE even with no transaction around it, which leaves a narrow
window; the model locks nothing, so its window is the whole gap between a read
and a write. That is the difference between the browser losing about 300 out
of 3,000 and Postgres losing 11.
And the browser is reproducible in a way the database is not. Its ordering comes out the same every run, because nothing in it waits on anything real. Five consecutive runs of the identical 200 transfers against Postgres finished at 3,005, 3,008, 3,009, 3,013, 3,019 — same code, same input, a different wrong answer each time. That is what makes this kind of bug hard to reproduce from a ticket.
Where this came from
This is interview preparation, published. I wrote the SQL scenarios and the Node lab to answer isolation-level questions properly rather than from memory, and turning them into something runnable was the part that made them stick.
The interest is not abstract: my last role was backend work on an institutional crypto-custody platform for banks, where concurrent access to balances is the kind of thing the product is judged on. I did not build the ledger there — I owned the address book, worked on authentication between services, and wrote tests — so nothing on this page is a description of that work. It is what I went and learned afterwards.