// Detailed design
Every frame takes the same five steps
DepthCharge's entire correctness argument rests on one claim: the code between the socket and the pixels does not know which of them it is talking to. This is that code — the classes that exist today, the calls they actually make, what each of them costs, and the places where the design is carrying load it has not yet been asked to bear.
docs/DESIGN.html, the maintained design reference in the DepthCharge repo. That document is authoritative for the facts; this page is authoritative for how they are told here, and its figures are rebuilt in this site's visual language rather than embedded. Its companion design & architecture page renders a different source — ARCHITECTURE.md, the constitution — and answers a different question: what must be true, and why. This page draws the code that makes it true. Where the two disagree, the constitution wins. Status reflects M3 stage A.The shape of the whole thing
DepthCharge is a pipeline with swappable ends. Bytes arrive from a replay file on a desk or a TLS WebSocket on an ESP32-S3; pixels leave to a terminal or to a HUB75 DMA driver. Between those two swaps sits engine/, and the claim worth being precise about is that it is the same translation units in both cases — not a port, not a shared interface, the same code.
That is the bet the whole project makes, and it is enforced rather than asserted. CMake generates one translation unit per engine header which includes only that header, so the moment an ESP-IDF include leaks into engine/ the host build fails — because the host has no such header to find. The guard costs nothing at runtime and cannot be forgotten, which is the property that matters when the next session to touch the file has never read this page.
ctest; dashed boxes are M3. Everything inside the cyan boundary is linked unmodified into both — not a port and not a shared interface, the same translation units. The two rounded shapes are values rather than objects: they are copied, never referenced across a task.What the build actually produces
The CMake targets encode the same separation, which is what makes the claim checkable rather than aspirational. dc_engine is an INTERFACE library — header-only, so nothing can be compiled into it with the wrong flags. dc_engine_anvil is deliberately its own static library because it is the single target that touches nlohmann/json, and therefore the single target that must not reach the microcontroller.
Since M3, one more target closes a hole that had been open since the beginning. dc_engine_target_check compiles those same generated translation units with the PlatformIO xtensa compiler, at ESP-IDF-shaped flags. Until it existed, the header check proved the engine builds on the host and that was quietly read as proving it builds — which does not follow, because the two toolchains genuinely disagree. All eight engine headers pass today, and the hot path costs 1,541 bytes of .text at -Os.
The vocabulary that crosses the boundary
FeedEvent is a plain aggregate with no methods and no ownership. Four kinds share one flat layout, and which fields mean anything is a function of kind. That is a tagged union written without a union — deliberately, because a static_assert on trivial copyability is what lets the render hand-off be a memcpy and lets the whole feed path avoid the heap. It is 72 bytes, and it lives on the stack; nothing ever stores one.
Prices are int64_t ticks and quantities are int64_t steps. There is no floating-point number anywhere in the type, and the conversion from the wire's decimal strings happens in parse_scaled, which refuses to round: a price carrying more precision than the declared scale is a reported error, never a quietly truncated level.
The lifetime rule the compiler does not enforce
A Snapshot carries 84 to 126 levels per side across the committed baseline trace — measured, not assumed. Putting them in FeedEvent by value would make the boundary type kilobytes wide; putting them in a container would make it allocate, which invariant #7 forbids. So the levels stay in the adapter's reusable AnvilFrame and travel as two raw pointers, valid for exactly the duration of the sink call that delivered them.
FeedEventstays flat, trivially copyable and allocation-free by conveying a snapshot's levels as two raw pointers into the adapter's own staging buffer. The price is a lifetime rule the compiler does not enforce: the spans are valid for exactly the duration of the sink call that delivered them. Today the single consumer copies immediately, inside the window. Nothing in the type system says it must — which is why this is strain point 1 rather than a solved problem.The adapter, and the seam inside it
AnvilAdapter owns every Anvil-specific decision so that Book owns none. Three of those decisions are load-bearing, and each is a place a naive implementation would have gone wrong:
- The wire seq is decoded, and then ignored. Anvil's sequence number is one global counter shared by every ticker and frame type, so a single socket's subsequence runs backwards — 42 times in five minutes, measured at M0 on entirely healthy data. The adapter synthesises its own dense
Seqfrom receive order and keeps the wire value only as a diagnostic counter. It can therefore never emitGap{SeqGap}, which is safe only because Anvil's book frames are idempotent full replaces. The same shortcut at a delta venue would be indefensible. - snapshot and book are the same event. Both become a
Snapshot. Abookframe is not a delta, and treating it as one would corrupt the ladder silently — which is the specific failure this project is organised to prevent. - A malformed frame is dropped, not gapped. Anvil republishes the whole book every ~80 ms, so one bad frame self-heals within a refresh; greying the panel for it would be dishonest in the opposite direction. A systematic failure still shows up, as a stalled ladder plus a non-zero
parse_errorscount.
One structural choice sits underneath all three. Sink is a template parameter rather than an interface, so there is no virtual dispatch anywhere on the feed path. The cost of that is honest and worth stating: what an adapter looks like is currently convention rather than a type, which is strain point 3 below.
One declaration, two definitions, chosen by the linker
The only genuinely environment-specific work in the adapter is turning JSON bytes into fields. That work sits behind a single free-function declaration, and the implementation is selected by which static library a binary links. It is the cheapest possible seam — no virtual call, no template, no runtime branch — and it is precisely what M3 stage B swaps: the host keeps a heavyweight parser, the target gets an allocation-free streaming one, and engine/ does not change at all.
engine/ never defines. Which implementation runs is decided by which static library a binary links — no virtual call, no template, no runtime branch. The acceptance test for the swap already exists: link the new parser instead, run the replay goldens unchanged, and if a golden moves then the swap changed behaviour and is out of scope.The book, and why it is barely a book
Anvil publishes a full top-N replace every ~80 ms. Against a venue like that, book maintenance is not a data-structure problem — adopting the latest snapshot is the entire engine, plus a ring of recent prints. The tick-indexed dense window the constitution describes is real work, but it is M4 work, and building it now would mean building it against no delta stream to test it with.
So Book today is two fixed arrays, a ring, and a status flag. What makes it interesting is not the storage; it is that there are exactly two states, four ways into the unhappy one, and a single way back.
The asymmetry is the design, and reject_delta is its sharpest expression. Anvil never sends a delta, so on the only venue that exists this branch is unreachable. It exists because the alternative — silently ignoring an event kind the book cannot apply — would produce exactly the failure mode the project is built to prevent. It is a tripwire for a future adapter bug, and it costs one enum comparison per event.
publish is non-const because the version stamp is producer state, and it is driven by the caller rather than by the book itself — the book never decides when it is read. That is what keeps invariant #8 true by construction: there is one writer, and it is the task that owns the object.When it goes wrong
Anvil emits no error frame and no gap frame. Absence of data is the only signal there is, which means the disconnect is synthesised by the transport — and the harness has to synthesise it too, or the host replay would be an analogy rather than a preview.
Two rules were available for a capture file with no disconnect marker. The obvious one — treat a mid-stream snapshot as evidence we reconnected — was rejected for a reason worth keeping: the gap would be raised in the same breath as the snapshot that clears it, so the book would never actually be stale, and the invariant-5 proof would be vacuous. It would pass, and it would prove nothing.
The rule chosen instead is a watchdog, and the number is measured rather than picked. Across two five-minute captures the worst healthy inter-frame gap is 640 ms against a ~70 ms median, so a 1000 ms threshold sits 1.6× above the worst healthy silence and 4.5× below the observed drop.
The detail that makes the stale window real is the timestamp. The gap is dated at prev_rx + 1000 ms — the moment the watchdog would have fired — not at the moment the next frame eventually arrived. In the committed reconnect trace that is the difference between an instantaneous blip between two frames and 3.47 seconds of grey, which is the thing the panel is actually being asked to do. A second hole arriving while the panel is already grey folds into the open episode rather than opening a new one: the book has not been re-baselined in between, so it is the same outage continuing, and measuring it as two would report the first as never cleared.
The other failure, which deliberately does nothing
A price that arrives with more precision than the declared scale — "10.01234" against four decimals — fails in parse_scaled, increments price_errors, and emits no event at all. The book is never told, and the ladder holds its last good frame. Declare and verify, never guess and round: a server that started quoting five decimals fails loudly on the first frame instead of drawing a subtly wrong ladder for as long as nobody checks. The frame is dropped rather than gapped because the next republish is ~80 ms away.
How any of this is proved
The harness is not test scaffolding around the engine. It is the host stand-in for the M3 firmware feed task, and it is deliberately the only place the engine is wired together — which is why the goldens and the console ladder can never disagree about what a trace means. There is one assembly, and both read it. The replay object holds the adapter, the book and the channel as members, in the same arrangement the firmware feed task will own them, so the object is invariant #8's single writer rather than a model of it.
One detail carries more weight than its size suggests. TraceReader hands the adapter a string view sliced out of the capture line by a string-aware scanner, not a re-serialised object. The adapter sees exactly the bytes the server sent, key order and spacing included — otherwise the harness would be validating a parser against its own output, which is the most comfortable way to be wrong.
Invariant #7 is the one that would be easiest to assert and never check, so it is not asserted anywhere that matters — it is measured. alloc_probe.cpp replaces global operator new in the test binary, which turns “allocation-free steady state” into an arithmetic question: take the counter, run the feed path, take it again, require no change. Two cases do exactly that, one over the book and one over the channel round trip. Only the engine path is measured — doctest, the JSON parser and the console renderer all allocate freely and are simply kept outside the window.
What it costs
Every figure here is a measured sizeof under GCC 15.2 x86-64 rather than an estimate. Layout on the Xtensa target differs slightly in padding but not in order of magnitude.
Publishing costs one 1,168-byte copy per event, at the baseline trace's 13.6 events per second — about 16 KB/s of memcpy. The event rate sits below the 15.6 frames/s wire rate because summary frames emit no event at all. Neither number is close to a constraint, which is exactly the point of having measured them: the argument for the dense window at M4 now has to be made on delta-venue correctness, because it cannot be made on memory.
Where the design is carrying load it has not been asked to bear
Everything above describes code that works and is green. This is the counterweight, and it is reproduced here in full rather than curated: nine places where the current shape is unenforced, speculative, or about to be tested by a milestone that does not exist yet. Each is stated with what would break it and what would resolve it. One is already closed.
This used to be implicit and unenforced. The generated per-header check proved the engine compiles on the host; nothing proved it compiles for the ESP32-S3, and the two genuinely disagree — the toolchain espressif32 6.5.0 pins is xtensa GCC 8.4, which does not accept C++20 at all and whose standard library lacks <span>, <ranges>, <concepts>, <bit>, floating-point from_chars and constexpr algorithms. A header can be valid on the desk and reject on the board, silently, until M3 tries to link. The target check now compiles the same generated units with the PlatformIO compiler and all eight headers pass. The residual strain is that the check is optional: a contributor without PlatformIO gets a host-only guarantee and a status line saying so.
Resolves · CI that has the toolchain installed, so the target half is never the skipped half
LevelSpan is a lifetime contract with no compiler behind itThe rule — copy before you defer — is enforced today only by the fact that a single consumer exists and it copies immediately. Nothing stops an M3 feed task from pushing a FeedEvent onto a queue, and if it does there is no diagnostic, no sanitiser hit on the common path, and a ladder that is subtly wrong rather than obviously broken. Invariant #7 forbids the obvious fix, since a guard type would stop FeedEvent being trivially copyable.
Resolves · A debug-build generation counter on the staging frame, asserted in the span — no release cost, no layout change
SnapshotChannel was a claim; M3 stage A made it a mechanismclosedUntil stage A this was a single slot behind two plain integers, with the header saying plainly that concurrent use was undefined and the M1 test exercising it single-threaded only. It is now three slots and one atomic word, and neither thread ever touches the frame the other is using. Three rather than the two the constitution's wording implies, and that is the part worth arguing with: a seqlock and a two-slot buffer both let the reader copy a slot the writer is writing and then discard the result — the tear happens; the version check only stops it being drawn. Measured both ways, a seqlock of exactly that shape delivered 4.8 M frames with zero tears reaching the consumer, and ThreadSanitizer flagged it on the first. The case against it is not that it misbehaves on today's compiler; it is that the copy is a data race, and the only way to have both a seqlock and a clean report is a suppression sitting on the one cross-core path, on a target compiler generation nobody here controls. The evidence is committed: test_snapshot_channel.cpp races two threads over 100,000 frames whose every field is stamped from the frame's own version, so a delivered frame is either whole or provably spliced and the checker names the field where it split. Both mutation-verified — collapsing the slots back to M1's one fails at version 12.
Residual · Done — the residual strain is that ThreadSanitizer needs Linux, so the report is refreshed by hand rather than by the host gate
The architecture diagram names one; the code has AnvilAdapter with a template Sink, no base class and no concept. That is the right call for invariant #7 — no virtual dispatch on the feed path — but it means what an adapter is remains convention. Kraken at M4 is the forcing function: either a concept pins the shape, or two adapters drift apart with nothing to catch it. This is the largest open design question in the repo.
Resolves · Decide at M4, before the second adapter has behaviour worth preserving
Book capacity is defined as the maximum snapshot depth — 256 per side — while the panel renders 27. The justification is the M7 zoom mode, which is four milestones away and may never take that form. The coupling itself is defensible, since a snapshot the adapter has already accepted can never be truncated a second time, but the value is speculative capacity.
Resolves · Cheap to leave; revisit only if S3 SRAM gets tight against the framebuffer
Book's face has never met a deltaM4 requires the tick-indexed dense window to land behind this same apply / publish face. That is an untested assumption. Re-anchoring by bounded copy, a cold tail in a map, and CRC verification are all things a snapshot-only book never had to express, and if the face does not survive them, this is the section of the design that changes.
Resolves · M4 — and a face change there is a constitutional amendment, not a refactor
publish bumps the version unconditionally, so a trade-only event produces a new version and the render side redraws even though no level moved. At Anvil's ~15 events per second that is comfortably under the 30 fps target and costs nothing worth reclaiming. It becomes a real question only at a venue with a much higher per-event rate.
Resolves · Nothing to do now; note it before M5's Binance diff stream
format_px returns a heap stringThe console ladder allocates freely, which is correct for the display edge on a desk. But the offending function lives in a harness header that a firmware renderer will be tempted to imitate, and the target must format into a stack buffer instead. Nothing currently enforces the distinction, and the allocation probe covers the feed path, not the render path.
Resolves · M3 stage D — extend the allocation probe across the render side too
A stale episode uses a gap sequence of zero to mean “not yet set”, which is only safe because the adapter starts counting at 1 and reserves 0 for “no event yet”. That is a real convention, documented in the adapter — but it is an engine convention that harness logic now depends on, across a boundary neither file names.
Resolves · A one-line comment, or an explicit optional; low stakes, worth knowing
Keeping this true
A design document that drifts is worse than none, because it is believed. The source this page renders is a single file inside the repo, so it diffs with the code it describes, and it is deliberately scoped to the things that are expensive to rediscover: which type crosses which boundary, what a call actually does in order, and where the design is under strain.
It does not restate ARCHITECTURE.md. That file is the constitution and stays the source of truth for the invariants and for any decision with architectural weight; the design doc draws the code that implements them. When the two disagree, the constitution wins and the design doc is the one that is wrong. This page inherits that rule, which is why the invariants and the venue curriculum live on its companion page and not here.
Five things trigger an update to the source, and therefore to this page:
- A class gains, loses or renames a member that appears in a diagram.
- A call order changes — anything a sequence diagram would now draw differently.
- A new type crosses an existing boundary, or a new boundary appears.
- A strain point is resolved, newly created, or turns out to be wrong.
- A milestone completes: the status changes and the dashed boxes go solid.
Three of those are already queued against M3. The two dashed boxes in the first diagram go solid and the firmware column stops being hypothetical; the streaming parser loses its dashes and dc_engine_anvil becomes the harness reference only; and strain point 7 should close at stage D. Strain points 1 and 3 will not close at M3 — the first needs a debug-build mechanism nobody has written, and the second is waiting for a second adapter to have an opinion.
writeup-sources.json records the DepthCharge commit it was last refreshed against, and npm run check:writeups reports the exact commit interval when the design doc moves on — which is what makes the claim that this page is current something a script says rather than something I remember. The source itself was drawn from a green host workflow run: 6 of 6 ctest entries, 70 of 70 doctest cases across 6,970 assertions, and a clean ThreadSanitizer report.