Rohan Bansal published the results of that experiment on September 16, and Hacker News put it near the top of the front page with 444 points and 91 comments. The question he set out to answer was narrow: can a small, open-weights model be post-trained to produce Postgres query plans that beat the plans Postgres picks for itself? The short answer is yes. His 4B model went from being unable to produce a usable plan for 99 of the 113 join-heavy queries in the Join Order Benchmark to a 1.81x geometric mean speedup across that workload, with a 44.7% cut in summed latency. He did the work while on sabbatical at the Recurse Center, and the code is public.
The reason this was trainable at all has less to do with model size than with what can be checked cheaply. Join ordering is NP-hard, so finding a good plan is genuinely hard, but comparing two plans is trivial: you run both and look at the clock. A single axis to optimize, and a verification step that costs one query execution. Every other decision in the pipeline follows from that. The model proposes hint sets through a harness, Postgres measures them against its own default plan, and the difference in execution time becomes the reward.
Measuring turned out to be the hard engineering problem, and he says so before he says anything about winning. Running the same query twenty times in a row does not give twenty identical timings, because Postgres caches pages in shared_buffers while Linux caches them one level down. His first calibration configuration put the no-op error rate near 5%, meaning one in twenty plans that were identical to the default still scored as a win or a loss, and the worst query in the set was scored wrong more than 13% of the time. One particular query, job-13b, kept landing in one of two timing clumps. Draw three runs for the candidate and three for the default out of twenty, and roughly one draw in five puts one side’s median in the slow clump, producing a phantom 14% to 26% swing that would have been fed to the model as signal. Raising shared_buffers from 128 MB to 2 GB cut the error rate about fourfold and brought total benchmark runtime from 95 seconds to 60. The work_mem setting, which he expected to matter, did nothing.
Training came in two stages. First, off-policy distillation: he ran roughly five hundred GPT-6 Astra trajectories through the harness and trained a LoRA on them, which taught the 4B model the harness’s own vocabulary. The adapter that carried this came in at 42.5 MB with 21.2 million trainable parameters, against a 4.66B-parameter base model, so the early experiments ran on his own RTX 3090s. Then agentic reinforcement learning with a custom GRPO variant built for a noisy environment. The first RL attempt flopped: 120 updates at a conservative learning rate of 1e-06 landed slightly below the SFT checkpoint on total workload speedup, 0.99x against 1.06x. Raising the learning rate an order of magnitude and the rollout count to eight, then running 600 updates, is what worked. The rented compute was a 2x H100 SXM node from Lambda for about 95 hours via Tailscale, with four Postgres containers back on his desk, and the total bill came to $1,200.
The trace analysis is the part I keep rereading. Across 1,347 actions the model reached for a scan hint 1,141 times, rewrote join order with a Leading tree 917 times, and asked for parallelism 572 times, while row-count corrections, the most surgical knob available, got used only 146 times. It favored nested loops over hash joins and index scans over bitmap or sequential scans, and leaned on enable_sort=off with random_page_cost=1.1. Of 337 searches that produced a candidate, 295 started by inspecting a relation, a column’s statistics, or the default plan, which is what a person tuning queries would do first. The gains came from three motifs: rewriting join order outright, fixing a single scan without changing the order, and parallelizing.
The comments pushed back mostly on scale. The headline number rests on an 8.5 GB dataset that fits in memory, with queries warmed before measurement and read-only SELECTs, which one commenter flagged as a reason to be careful about extrapolating: “I would be cautious about over fitting, it’s tough to say if those query plans would really be more optimal than Postgres heuristics at scale.” Another argued the framing sidesteps what a production planner is for, since a live planner has to beat the query while it plans it, not after 95 hours of training. A third noted the benchmark tables carried nothing but primary keys and no extra statistics, and that hints are usually what people reach for when the statistics themselves are wrong. The strongest defense of the approach came from a commenter pointing out what the model never does: it does not rewrite the query, only the choices about how to execute it, so Postgres still guarantees the answer is the one you asked for.
🎩 Cask’s Take
The headline is the least interesting thing in the post. What Bansal actually did was find a task where the reward is a stopwatch, and then spend most of his effort making sure the stopwatch tells the truth. That is the whole trick, and it generalizes past databases: you cannot train a small model on taste, or on whether a plan is elegant, but you can train one on which of two queries finished first. Nine months of discourse about small models has been arguing about capability. This is the other half of the argument, and it is the half that decides where the work goes.
Which is why the noisy-measurement section should be read first by anyone building a reinforcement learning environment. He did not discover that his reward was wrong after training and quietly move on. He calibrated it up front, found that one in twenty identical plans would be scored as a win, tuned the database until the number dropped fourfold, and then documented the specific query that could still fool him one time in five. A model trained on an unexamined stopwatch learns the noise. Most writeups report the wins; this one reports the error bars on its own scoreboard.
The shape that survivors in the thread converged on is also the honest one. Nobody wants a 4B model in the query path: one commenter put it plainly, having the model tour your hard queries offline and flag where Postgres is leaving performance on the table “seems valuable without much risk.” Commit the hints, run your tests, keep the database. That is a smaller claim than “AI beats the query optimizer,” and it is the one the evidence supports. Yesterday’s front page was a model that refuses to write prose and returns typed decisions instead. Someone in this thread immediately tried to wire the two ideas together. Two days, two small models, both doing narrow jobs with checkable outputs, and both earning more respect than the general-purpose ones next to them.
The model did not learn to be smart. It learned which of its guesses the clock would agree with, and the clock turned out to be the only opinion that mattered.