Performance

The matching engine is a message-passing machine: each market is an actor — a dedicated OS thread that exclusively owns its order book, position map, and trigger state — and the MatchingEngine is a router that maps a market id to that actor's bounded mpsc sender. Every operation the exchange performs is therefore decomposable into a routing step (which actors receive the message, and how the caller learns which one owns the answer), a queueing step (mailbox depth and reply via oneshot), and an on-actor computation step (book traversal, position mutation, cache refresh). Because these steps are structural, each benchmarked operation admits a falsifiable scaling law derived from the topology itself: direct dispatch is flat in market count, broadcast fan-out is linear in it, bulk cancellation is linear in resting orders, and concurrent submission into a single actor exhibits the serialization-plus-coherence shape of the Universal Scalability Law.

The models below carry only symbolic constants ($c_0, c_1, \dots$): the form of each law comes from reading the actor architecture and the benchmark's timed closure, while the constants are machine properties of the CI runner and are fitted separately against the measured benchmark series. Each operation corresponds exactly to one measured sweep, so the fit is a direct confrontation of theory with instrument.

The residuals are the signal. If direct routing drifts upward with market count, the router's market map is not $O(1)$; if the contention series needs a large quadratic coefficient, point-to-point serialization has acquired a coherence cost; if bulk cancel bends superlinearly, the book's removal path is not the linear scan the model claims. The model is the hypothesis; the CI bench is the experiment.

Provenance: the scaling laws are derived from source (the actor architecture and data structures of the hot paths); the constants are fitted to CI measurement (a benchmark run captured 2026-07-01); the residuals are the comparison signal — model error, or a caught regression.

The architecture, mathematically

Formally, the engine's matching-engine service is a set of $M$ single-writer actors $\{A_1, \dots, A_M\}$, one per market, each spawned with std::thread::spawn and owning its state exclusively. Tokio handler tasks never touch market state; they send a message carrying a oneshot reply channel into the actor's bounded mpsc mailbox and suspend until the reply arrives. The cost of any request is thus $T = t_{route} + t_{queue} + t_{work} + t_{reply}$, where $t_{route}$ depends on how many senders the router must touch and $t_{work}$ runs serially on the owning actor's thread.

Parallelism and contention follow directly. Requests targeting different markets never share a lock — throughput scales with cores up to $M$ — but requests targeting the same market are serialized by that market's single mailbox: the actor is an $M/M/1$-like server, and adding submitters to one market adds queueing, not parallelism. This is a deliberate trade: the previous mutex-sharded design allowed races between the book lock and the position lock and exhausted tokio workers with synchronous matching work; the actor design buys serialized market access and predictable latency at the price of a per-market serial ceiling.

Routing topology is the other axis. A call that names its market (MatchingEngine::cancel_order_in_market) resolves one sender — flat in $M$. A call that does not (MatchingEngine::cancel_order with only an order id) must fan out to every actor, paying $M-1$ lookup misses — linear in $M$. Cross-margin risk lives outside the actors in risk_margin::RiskModule, whose equity cache aggregates over an account's $n$ positions per refresh and whose liquidation path (liquidate_portfolio) closes each of $n$ positions through its owning market actor in turn. The six operations below each isolate one of these structural costs.

Single-order cancel — direct dispatch

Each sample cancels a fixed batch of 400 pre-rested orders on MARKET-00 via MatchingEngine::cancel_order_in_market(id, account, market_id) while the engine hosts $n$ market actors (bench_cancel_routing in tests/benchmarks/benches/cancel_order.rs, mode="direct"). Because the caller names the owning market, the router resolves exactly one sender from its market map and dispatches to that single actor; the other $n-1$ actors are never messaged. The per-batch cost is therefore one map lookup, one mailbox round-trip, and one book removal per cancel — independent of how many markets exist. The predicted law is a constant.

$$T(n) = c_0$$

Architecture: Measures the flatness of keyed routing: a named-market dispatch touches one mailbox regardless of topology size, so any measured slope in n falsifies the O(1) market-map claim.

Fitted to the CI series: $c0 = \text{6.21ms}$; $R^2$ n/a for a constant model (O(1)).

Single-order cancel — direct dispatch model vs measurement

Dots: CI measurements. Curve: the source-derived model with fitted constants. Labels: residuals.

Single-order cancel — broadcast fan-out

Identical setup to the direct series, but each cancel goes through MatchingEngine::cancel_order(id, account) with no market id (bench_cancel_routing, mode="broadcast"). The router cannot know which actor owns the order, so it fans the cancel out to all $n$ market actors; $n-1$ of them perform a lookup miss and reply not-found, and one performs the removal. Each cancel thus pays $n$ mailbox round-trips instead of one, so the per-batch cost grows linearly in market count on top of the fixed removal work. The structural contrast with the direct series is the point: same book operation, different routing topology.

$$T(n) = c_0 + c_1 \, n$$

Architecture: Measures the linear price of ownership-blind routing: without a market hint, the actor topology forces an n-way fan-out with n−1 wasted mailbox round-trips per cancel.

Fitted to the CI series: $c0 = \text{1.19ms}$, $c1 = \text{1.17ms}$; $R^2 = 1.0000$ (O(n)).

Single-order cancel — broadcast fan-out model vs measurement

Dots: CI measurements. Curve: the source-derived model with fitted constants. Labels: residuals.

Bulk cancel of all resting orders on a market

One book is seeded with $n$ non-crossing resting buys at distinct 0.5-tick prices, then a single cancel_all_orders_for_market(account, market) call is timed (bench_cancel_all_for_market in cancel_order.rs). The call routes once to the owning actor, which walks the account's resting orders and removes each from the book — one price-level removal and one order-state transition per order. The dominant cost is the per-order removal loop, so the law is affine in $n$; the intercept captures the single routing round-trip and call overhead. Throughput::Elements(n) in the bench exposes the same law as a per-order rate.

$$T(n) = c_0 + c_1 \, n$$

Architecture: Measures the on-actor serial work of a bulk book mutation: one routing hop amortized over n removals, isolating the per-order removal cost of the book data structure.

Fitted to the CI series: $c0 = \text{-654µs}$, $c1 = \text{1.21µs}$; $R^2 = 0.9996$ (O(n)).

Bulk cancel of all resting orders on a market model vs measurement

Dots: CI measurements. Curve: the source-derived model with fitted constants. Labels: residuals.

Concurrent submitters into one market — contention scaling

$n$ tokio tasks each submit 1000 crossing limit orders (distinct account pairs per task) into a single-market engine on a current-thread runtime, and the wall-clock for all tasks to drain is timed (bench_contention in tests/benchmarks/benches/engine_throughput.rs). Total work is $1000n$ orders, but the market is one actor: every order serializes through the same mailbox, so the linear term $c_1 n$ is the actor's per-1000-order matching cost — the serialization term of the Universal Scalability Law. The quadratic term $c_2 n^2$ captures the superlinear costs of adding submitters to a saturated single-writer pipeline: mailbox contention on the shared mpsc sender, task-switching and wakeup churn among $n$ suspended submitters per reply, and cache-line ping-pong on the shared channel state — the USL coherence term. $c_0$ absorbs spawn/join overhead.

$$T(n) = c_0 + c_1 \, n + c_2 \, n^2$$

Architecture: Measures the single-writer ceiling of the per-market actor: work scales linearly with submitters (serialization) while cross-task coherence on the shared mailbox adds a quadratic penalty — the USL shape of the architecture's one deliberate serialization point.

Fitted to the CI series: $c0 = \text{17ms}$, $c1 = \text{1.68ms}$, $c2 = \text{77.6µs}$; $R^2 = 0.9994$ (O(n^2)).

Concurrent submitters into one market — contention scaling model vs measurement

Dots: CI measurements. Curve: the source-derived model with fitted constants. Labels: residuals.

Equity cache refresh for a cross-margin account

A RiskModule is snapshot-built with one account holding $n$ cross-mode positions across $n$ markets, and the timed loop calls risk.refresh_equity_for(ACCOUNT, "fill") repeatedly on the hot cache (bench_b7_equity_cache_update in tests/benchmarks/benches/cross_margin.rs; setup via build_account_snapshot). A refresh recomputes account equity as collateral plus the sum of unrealized PnL over every open position, each mark-to-market against its market's oracle price — one position-map traversal and one oracle lookup per position. The work is a fold over $n$ positions with constant per-element cost, so the law is affine; the intercept is the fixed cost of the call, account lookup, and cache write.

$$T(n) = c_0 + c_1 \, n$$

Architecture: Measures the per-position cost of cross-margin aggregation in the risk module — the price every fill pays to keep account equity coherent grows linearly with portfolio breadth.

Fitted to the CI series: $c0 = \text{-12.3ns}$, $c1 = \text{278ns}$; $R^2 = 0.9997$ (O(n)).

Equity cache refresh for a cross-margin account model vs measurement

Dots: CI measurements. Curve: the source-derived model with fitted constants. Labels: residuals.

Cross-margin portfolio liquidation roundtrip

Each iteration builds a fresh account holding $n$ cross-mode positions across $n$ markets (liquidation mutates state, so setup is per-iteration and outside the timed window) and times a single risk.liquidate_portfolio(ACCOUNT).await (bench_b9_liquidation_roundtrip in cross_margin.rs). The liquidation walks the account's portfolio and, for each of the $n$ positions, executes a closure through the owning market actor — a routing hop, a mailbox round-trip, and a position-close on that actor — plus the per-position risk accounting (equity/margin updates, insurance-fund interaction). Since each position lives on a distinct market actor and closures are issued sequentially by the risk module, the cost is one fixed portfolio-scan overhead plus $n$ per-market roundtrips: affine in $n$.

$$T(n) = c_0 + c_1 \, n$$

Architecture: Measures the cross-actor roundtrip cost of risk-driven control flow: liquidation crosses the risk-module/actor boundary once per position, so c1 is the price of one full risk→actor→risk closure hop.

Fitted to the CI series: $c0 = \text{3.92µs}$, $c1 = \text{1.07µs}$; $R^2 = 0.9990$ (O(n)).

Cross-margin portfolio liquidation roundtrip model vs measurement

Dots: CI measurements. Curve: the source-derived model with fitted constants. Labels: residuals.

Notes

• All constants are machine properties of the CI runner (shared GitHub-hosted hardware): absolute values are not portable across machines, and run-to-run variance on shared runners inflates residuals — the scaling shape, not the constant magnitudes, is the falsifiable claim.
• The concurrent_submitters quadratic term is a USL-shaped phenomenological approximation of coherence costs (mailbox contention, scheduler churn), not a first-principles derivation; with only four swept points (n=1,2,4,8) the c2 fit is weakly identified.
• Batched benchmarks (routing series: 400 cancels/sample; throughput series: 1000 orders/task) report per-batch wall-clock, so fitted constants are per-batch, not per-operation; divide by the batch size for per-op costs.
• cancel_all_for_market may hide a log(n) factor from ordered price-level removal inside c1 over the swept range (1000–16000); a systematic upward bend in residuals at large n would indicate it.
• The direct-routing model T(n)=c0 is deliberately slope-free: any statistically significant slope in the direct series is a falsification signal (router map not O(1) or idle-actor overhead), not something the model should absorb.
• Series sweep points are sparse (3–4 values each), so fits distinguish model families (constant vs linear vs quadratic) rather than resolving fine functional structure such as log factors.
• Timing on the CI runner uses coarse sample counts (sample_size=10 for destructive benches), so per-point uncertainty should be propagated into the fit rather than assuming homoscedastic noise.

References

• Derived from and adversarially verified against the Exchange's Rust implementation and its test suite.