Temper

Temper trains a reinforcement-learning agent to work a parent order — to sell a large position over a trading day without moving the price against itself more than necessary — and grades it against the Almgren–Chriss closed forms that FrontierView, another project in this portfolio, already implements. It is the differential-oracle pattern from Anvil applied to RL: an independently derived reference engine says what the right answer is, so the agent's score is never a matter of opinion.

The discipline is the point. Under the exact assumptions Almgren–Chriss makes, the closed form is optimal, so "the agent beats Almgren–Chriss" is a red flag rather than a result — it means something in the harness is wrong. Phase 1 therefore claims only that the agent independently converges to the analytic schedule within a tolerance written down before training started. Every milestone is specified in a brief that fixes ε, the seed count and the confidence level in advance; every reported number regenerates from a committed config and seed; every chart carries TWAP and the analytic schedule as baselines and shows the spread across seeds rather than the best run.

That discipline has caught things. The vendored κ turned out to be a continuum-limit expression that drops a factor at realistic order sizes — the two rates differ by about 2.2×, both produce sinh-shaped schedules, and the difference is invisible by eye. Grading against it would have shown a correctly trained agent scoring up to 18 % better than "optimal", which reads exactly like the red flag above. The oracle now carries both conventions, and the honest reading of the finding is narrow: the risk-aversion dial is mislabelled by about 5× at usable settings, not that Almgren–Chriss is wrong. The vendored schedule is the exactly optimal schedule for a different risk aversion, and its point sits on the efficient frontier, never inside it.

The second finding was about noise rather than finance. At the reference case the quantity being optimised is ~2.4 basis points while a single episode's cost varies by ~95, and at that signal-to-noise ratio training is a lottery rather than a slow climb: five seeds scored 0.066, 0.009, 0.819, 0.098 and 0.147 of the available gap — one essentially rediscovering the analytic schedule, another barely learning at all. The same agent with the same hyperparameters, trained on a reward whose noise is cancelled by an exact identity the simulator publishes, clears the bar with a spread orders of magnitude tighter. Both runs are committed and differ in one config field, which is what makes "the estimator closed the gap" a measurement rather than a story. A related discovery, now a portfolio-wide rule: the same seed scored 0.165 and 0.066 on four threads versus eight, because multithreaded floating-point reductions sum in a thread-dependent order — so thread count is a committed input, not a property of the host.

Phase 2 is where the agent is allowed to win, and it wins against something specific. FrontierView's temporary impact is a 0.6-power law; Almgren–Chriss has no closed form for that, so the library linearises at a tangent and solves the linear problem instead. Making the power law the actual world means the closed form is now solving the wrong problem — by 1.54 % of the objective at the reference case, which is 16 878 shares of a 100 000-share order in trajectory space. That world's own optimum has no formula, so it is solved: a Newton iteration on the KKT system, certified by a positive-definite Hessian, a relative residual of 1.2e-15, 3 600 perturbations that all go uphill, and a completely independent bisection solver agreeing to 3.1e-15 of the parent order. Ten seeds captured a median 99.4 % of the available difference, worst seed 99.0 %, landing 727 shares from the certified optimum where the closed form sits 16 878 away.

The second of Phase 2's three parts is the first advantage in the project that no fixed schedule can capture at all. Liquidity becomes a second, independent noise source and the agent gets to see it, so what has to be beaten is no longer a mis-specified formula but the best possible fixed schedule that knows the liquidity law — 2.49661 basis points against the 2.43449 of an optimum over reacting policies, an adaptive advantage of 0.06212 bps and 2.55 % of the objective, under a per-bin liquidity multiplier that is Temper's own invention with one parameter, σ_L = 0.5. That gap is the denominator and not the distance from M4a's schedule, because 0.00234 bps of the latter is a level shift any static solver picks up for free. Ten seeds captured a median 99.0 % of it, worst seed 96.3 %, landing 0.00064 bps above a reference that is converged and bracketed rather than certified — a stochastic dynamic program has no certificate to offer, and that word stays off it. The control is the claim rather than an appendix: re-graded with the observed liquidity drawn independently of the liquidity actually charged, the same policies score −1.01 against a bar of 0.15 fixed before the run, about one advantage-unit worse than the best static schedule. Reacting to a decorrelated signal costs almost exactly what reacting to the true signal gains, and the gap between +0.99 and −1.01 is what says the agent is using the signal rather than having found a better fixed schedule.

The third part is the one whose headline is not a number at all. M5 puts a weak one-step-ahead price signal in the observation and asks the agent to tilt the schedule for it — a signal that is Temper's own invention with one parameter, rho = 0.01, which is to say it explains one part in ten thousand of the next bin's return variance. Predicting the price has to be paid for: tilting to monetise a signal costs execution quality, and the converged optimum itself gives back 45.5 % of the gross effect in order to do it, so a single capture fraction cannot separate a policy that traded the signal well from one that traded it badly and executed well. The result is therefore three numbers, reported together and never one of them alone, each against a bar fixed before the run — alpha capture 0.9767, of the 0.14808 basis points of gross alpha available; execution premium 1.0480x, of the 0.06732 basis points the optimum itself pays; and net capture 0.9341, which is 0.00532 basis points over that optimum on an advantage of 0.08076. Ten seeds, worst seed 0.8925. The denominator is the net advantage rather than the gross alpha, which would have read 1.83x larger in the flattering direction, and the control is the claim here as it was for liquidity: re-graded with the observed signal shuffled, the same policies score a median −0.87 against a bar of −0.50 stated in advance. The reference is a dynamic program again, converged rather than certified — but with a certified floor underneath the half of the objective the signal cannot touch, because impact and risk are convex and carry no signal, so M4a's certified optimum bounds them for any policy whatever. That makes the red-flag test a proof rather than a threshold.

The two ends of the project belong beside each other, because it is the same agent at the same budget both times: at a 1:70 per-episode signal-to-noise ratio on the realised reward, rediscovering a closed form was a lottery, and a signal seven hundred times weaker than that one was recovered almost completely once the reward was the conditional grade and its conditional variance was exactly zero. It fails on a large signal under noise and succeeds on a tiny one without it, so across both phases the binding constraint was the estimator's variance rather than the signal's strength.

M4a's methodological finding is worth more than its result, and both milestones after it repeated the lesson in a new form. The tolerance every earlier milestone used — 5 % of the distance TWAP covers — is 1.8× the entire effect being measured, so an agent capturing none of the mis-specification would have passed it comfortably. A tolerance is only portable within the question it was written for. M5 took the same idea one step further: when the advantage is a difference of two larger numbers that move together, no single fraction is portable either, and the headline has to be the whole decomposition.

What none of this establishes is real-market performance. Every number here is a statement about a simulator whose dynamics are analytic and whose parameters are a calibrated synthetic set, with two exceptions named above: the liquidity process and the alpha signal are both Temper's own inventions, calibrated against nothing, so the 2.55 % that seeing liquidity is worth and the 0.081 bps the signal is worth are statements about those models rather than about a market. There are no real fills and no historical order-book data anywhere in the project. The differential tests are what make it a good simulator, and the honest reading of the results is "the machinery is correct", not "the strategy would make money".

The per-commit suite runs from a clean clone in about two minutes ten, against a three-minute ceiling every brief has stated in advance; the deep Monte-Carlo tiers and the multi-hour acceptance runs sit behind markers and drivers so the commit loop never waits on them. CPU only, and every milestone is sized to train in an evening on one desktop, because a project that needs a GPU to make progress is a project that stops.

It also ran on a wire. The trained policy — M4a's median seed, exported as a committed .npz and run through a numpy forward pass with no training stack on the client's import path — worked a 1,000-share parent order over 13 bins on a live Anvil book, another project in this portfolio. It is that protocol's third independent client after the browser UI and DepthCharge, and it required zero changes to Anvil. It reports 11.21 basis points of arrival slippage.

The number is not the result. It was predicted first. A committed counterparty ladder plus a deterministic policy plus deterministic matching makes every fill price and quantity computable in closed form before anything is sent — and three ladder shapes came back matching level for level: 11.21 bps on the reference book, 22.85 on a deliberately thin one, 33.63 on a wide-spread one, each to the digit and each reproduced across two revisions. A dozen lines of Python on one side, a C++ matching engine over a wire on the other. Arrival slippage measured against a book the client built itself was never going to say anything about execution quality; predicting it first makes it say something about client correctness, which is the one thing it can certify.

The thin book is where the closed loop stops being decorative. The simulator always filled exactly what was asked — its step function clips the trade to the inventory available — so a partial fill is a state the policy had never seen in training. Bin one asked for 421 shares against 375 of depth, swept all eight price levels, was filled short, had its resting remainder cancelled, and carried the shortfall into the next observation. Anvil has no market orders, so an aggressive order is a limit priced to cross, and a limit that doesn't cross rests silently: accepted is not filled.

The same client then worked the same order against the public deployment over TLS, on a shared and unauthenticated book. It filled 1,000 of 1,000, attributed every share, and reported the measurement void — 236 fills by other participants landed while the order was being worked. That is the milestone's own pre-stated condition firing for exactly the reason it was written: a third-party fill makes the run a successful demonstration and a void measurement, and both halves get reported. It is the only void run of the five, and it is void for the healthiest reason available — the venue was real and other people were trading on it. A clean number there would have meant nobody else was.

This is plumbing evidence and not execution-quality evidence, and the distinction is the point: the flow is a synthetic non-adversarial feeder, the sample is one order, and there is no baseline it could fairly be compared against. What it establishes is that the policy speaks a versioned venue protocol end to end — session, order entry, verdict channel, event stream, and a book that does not give it what it asked for.