Project · Research concept

WDMA

Wireless Direct Memory Access. Sharded inference over the network you already have: a mixture-of-experts model bigger than any one machine, split by layer across the machines in the building, served from a web page, and built to survive those machines coming and going. The weights stay where they are. The activations move.

Concept stage Nothing built yet Three experiments declared Experiment zero done

The problem

Capacity is the wall, and the room already holds it

Mixture-of-experts models changed what "big" costs. A 235B to 670B-class open model must be fully resident somewhere, but only a few experts fire per token, so a few percent of the weights actually move. Capacity is the lever; bandwidth is almost beside the point.

No consumer machine holds that much. The clusters that do are built on fabrics designed for dense training, where every parameter moves every step. That fabric is the wrong tool for sparse inference, and it is the reason the largest open models live in datacenters even though the per-token work would fit in an office.

Meanwhile a home or an office already contains hundreds of gigabytes of memory, spread across a laptop, a workstation, a mini PC and a storage box, most of it idle most of the time, all of it on one network. WDMA is the proposal that this pool of machines is the machine, and that a web page is how you use it.

Plainly

It is sharding, over the network you already have

Strip the name away and WDMA is a model sharded across the machines in a building, served over the network they already share, with wireless as the hardest version of that network. Three refinements keep the word honest.

Shard by layer, not by expert

Each machine holds a block of layers with every expert in it, so a token crosses the network once per machine. Sharding by expert sounded cleverer. Experiment zero, below, shows it buys little, because the router spreads work almost uniformly on purpose.

Static shards are prior art. Shards that move are not.

Petals, exo and the llama.cpp RPC backend split a model across hosts and leave it split. Here the shards are a cache over a storage box: a machine leaves, its block is re-placed onto what remains, and a late hop gets a resident stand-in instead of a stall. That part is ours.

Wireless adds exactly three problems

Round-trips per token, which is why the split is by layer. Tail latency, which is why every hop carries a deadline and dedicated channels matter. Churn, which is why membership is normal operation rather than a failure. The design is those three problems, answered.

In one line. Sharded inference over the network you already have, built to survive the machines coming and going. The wire is where it runs fastest. The air is what proves the "survive" part.

The idea

Three rules

Weights never move per token. Activations do.

Each expert lives on one machine and runs where its weights are. What crosses the network is a hidden state: a few kilobytes. An expert's weights are tens of megabytes; those cross a link only to fill a cache, never on the token path.

Pipeline first. Experts second.

Split the model by layer across machines, so a token crosses the network once per machine, not once per layer. Each machine holds a block of layers and every expert in them. Expert-level placement is the second tier: for experts that do not fit a machine's block, and for the hottest ones, which the orchestrator keeps at home.

Direct-memory semantics, in software, on the network you have.

Each node registers an input buffer, its experts and an output buffer; the orchestrator writes and reads them with as little of the host stack in the path as the link allows. Consumer network hardware has no RDMA, so on day one this is pre-registered buffers and polling, not hardware. The name is the destination. A wired LAN is where it starts; wireless is the mode it has to survive, not the premise.

The client is a web page. The orchestrator serves a site on the local network. A phone, a tablet or any browser is only the display: nothing resident, nothing to install, and the pool is reachable from wherever the site is. Nodes are machines that stay put and stay on: laptops, desktops, a mini PC, a storage box.

The physics

The budget is round-trips, and the baseline is a disk

Back-of-envelope for a 235B-class model with about 22B active parameters at 4-bit, around 100 layers and 8 active experts per layer, split across four machines. Measured numbers replace these the moment they exist.

Per token	Bytes read from memory	Bytes over the network	Round-trips
Dense 70B, one machine	~35 GB, all of it	none	0
MoE, one machine, fits in RAM	~11 GB, the active experts	none	0
MoE, one machine + NVMe offload	~11 GB; each miss reads ~10 MB from the drive	none	0
WDMA, pipeline over 4 machines	~11 GB, each machine its own block	4 hidden states, ~50 KB	~4
WDMA, expert-parallel over 4 machines	~11 GB, each expert on its host	~750 calls × ~16 KB ≈ 12 MB	~100

Neither network figure is a bandwidth problem. The round-trip count is the whole story, which is why the pipeline comes first.

Link, round-trip at p99	Topology	Network time per token	Bound on token rate
Wired LAN, ~0.3 ms	pipeline, 4 hops	~1 ms	not the limit
Wi-Fi, quiet room, ~3 ms	pipeline, 4 hops	~12 ms	~80 tok/s
Wi-Fi, under load, ~20 ms	pipeline, 4 hops	~80 ms	~12 tok/s
Wi-Fi, under load, ~20 ms	expert-parallel, ~100 round-trips	~2 s	~0.5 tok/s

Network time only; compute and memory bandwidth add on top. Wi-Fi at p99 under load is commonly 10 to 50 ms, not the 1 to 3 ms of a quiet benchmark, and power-save on battery adds beacon-interval stalls. A layer waits for its slowest call; a token waits for its slowest hop.

The baseline to beat is NVMe expert offload on one machine. A 4 TB drive holds any open model, and paging cold experts from it already gives single-digit tokens per second. WDMA's advantage over that is not capacity. It is aggregate memory bandwidth and compute: every node reads its own experts from its own RAM, in parallel. That makes WDMA a throughput machine for many users before it is a latency machine for one. If it cannot beat the disk, it is not worth the network.

120B-class MoE at 4-bit, ~3 GB active per token	One user	Many users	Added hardware
One machine, 16 GB GPU, experts paged from NVMe	~3 tok/s	does not scale	$0
Four machines, one 16 GB GPU each, sharded by layer over a wired LAN	~30 to 50 tok/s	every stage serves a different request	$0

What sharding buys, back-of-envelope: disk bandwidth against memory bandwidth, a 20 to 50× gap. One user is no faster on the pool than on a single machine that could hold the model. The pool's advantage is that no such machine exists in the building, and that its stages serve several people at once.

What the air is for. On round-trip, tail and bandwidth the wire wins or ties, and this page does not pretend otherwise. Wireless is where the machines already are: laptops without a free port, rooms without a cable run, robots and cameras that cannot have one. A pool built to survive the air runs on the wire without changing a line, so the realistic deployment is both: wire what has a port, and let everything else join from wherever it is.

Two ways to use remote memory. Fetch an expert's weights across the network into local memory (a literal direct memory read, tens of milliseconds per expert, affordable only for rare misses) or run the expert on the machine that holds it and move the hidden state (the steady-state path). The first fills caches; the second serves tokens.

Design sketch

Everything is about round-trips

Memory map and placementEvery node registers a region: size, bandwidth, power state, link quality. The full model sits cold on a storage box as the source of truth; everything resident anywhere is a cache over it, and placement is the cache policy.
Pipeline by layer, balanced by memory and linkBlocks of layers are assigned to machines so that each block fits its machine's RAM with every expert resident, and the slowest link carries the fewest hops. The orchestrator takes the first block, the KV cache for it, and the web page.
Co-activation locality for the expert tierWhere a block's experts do not fit, the router's own statistics decide which stay resident and which go to a neighbor. Experiment zero (below) says this buys little on a load-balanced model: routing is nearly uniform by training design. Kept as a lever only for models whose routing is concentrated.
A deadline on every hop, and a stand-in for the call that is lateEach remote call carries a deadline derived from the link's measured p99. When a node misses it, the layer substitutes a local stand-in: the next-best resident expert, a shared expert, or a low-rank sketch of the missing one. Output quality softens for that token; the model never stalls. Degradation is a measured, governed quantity, not a crash.
Amortize the round-tripsSpeculative decoding verifies several draft tokens in one pass through the pipeline, so one set of hops can pay for more than one token. Batching does the same for throughput when several people share the pool, which is the case WDMA is best at.
MembershipMachines come and go. A laptop leaves the office at 6 pm and its block is re-placed from the source of truth onto whatever remains. The model is a population, not a file, and it keeps answering while it rebalances.

What this protects, and what it does not. Nothing leaves the building: no cloud, no egress, no subscription. But a hidden state is visible to whichever machine runs the expert, and hidden states can be inverted back toward text. So "nothing leaves your machine" is false the moment a second node joins. Links are encrypted, and the pool is made of machines you own or trust. In an office that is a policy, not a technicality.

What it would unlock

Frontier-class open weights on the machines you already own

An office

Thirty desks of idle memory become one pool, used from a browser by everyone on the floor. Many users at once is where the aggregate bandwidth shows, so this is the strongest case. No rack, no subscription, no data egress.

A household

A 600B-class open model across a laptop, a workstation and a storage box, used from every phone in the house through a web page. Single-digit tokens per second for one person, shown honestly.

A fleet

Robots and cameras on one site sharing a pool over the air, each carrying its own attention and the experts it uses most. The sparse tail lives on the site, not in the cloud. This is the one case where wireless is the premise, and it goes last.

Experiments

Three experiments, in order, and what kills it

Declared before anything is built, so the result cannot be bent to fit. The threshold for "usable" is set in writing before the first run.

Experiment zero: no network, one machine. Done, 2026-10-15, results below.Run an open mixture-of-experts model over text from several domains and log its router. Measure how concentrated expert use is per domain, how large a resident hot set gets a 90, 95 and 99 percent hit rate, and how much quality is lost when a given fraction of remote expert calls is replaced by a local stand-in. Kill: if hot sets are not concentrated and a stand-in at a few percent miss rate costs measurable quality, the expert tier is dead and WDMA is a pipeline only.
Experiment one: two machines, one link, existing toolsThe same model pipelined across two consumer machines with tools that already exist, over a wired link and then a wireless one, against two controls: the model on one machine in RAM where it fits, and the model on one machine paging experts from NVMe where it does not. Tokens per second at p99, not at the median. Kill: if the pipeline over the LAN cannot beat NVMe offload on one machine, WDMA is dead and this page says so.
Experiment two: the WDMA layer itselfOnly then build placement, deadlines, stand-ins and membership on top of the pipeline that already works, and measure what each one adds. Anything that does not move the p99 or the quality curve is removed.

Built from published parts, and closer to shipped ones than most pitches admit. RDMA over Ethernet and GPU-direct remote memory give the one-sided memory semantics, on wired fabrics. Expert-parallel serving in datacenter inference systems gives the partitioning by expert. Petals runs a split model across the internet; exo discovers machines on a LAN and partitions a model by memory; the llama.cpp RPC backend pipelines layers across hosts. Expert offloading to NVMe is the same idea with a disk instead of a network. What is left for WDMA is co-activation placement for mixture-of-experts, deadline-plus-stand-in degradation, membership from a storage source of truth, and the measurement discipline: a runtime layer on top of what exists, the same shape as Janus.

Experiment zero · result

Placement buys little. Degradation is cheap.

Run 2026-10-15 on one consumer GPU in 397 seconds. Model: OLMoE-1B-7B (Apache-2.0), 64 experts, top-8, 16 mixture-of-experts layers. Five held-out domains, about 60,000 tokens each: encyclopedia, Python code, grade-school math, chat, news. The first half of each domain fits the hot sets, the second half is scored.

Q1 · Concentration

Routing is nearly uniform

Normalised entropy of expert use runs from 0.87 on code to 0.99 on chat. The busiest quarter of the experts takes 36 to 64 percent of the calls. Per-domain hot sets barely overlap each other. This is the load-balancing loss doing exactly what it was trained to do.

Q2 · Hot set

90 percent needs 56 of 64 resident

When the orchestrator does not know the domain, catching 90 percent of calls locally needs 56 of 64 experts resident per layer; 95 percent needs 56 to 64. Half the experts resident still sends 30 to 46 percent of calls remote.

Q3 · Stand-in

A few percent of misses costs a few percent

With half the experts resident, replacing 5 percent of remote calls with the best resident expert costs 1 to 3 percent perplexity; 10 percent costs 1 to 6; 25 percent costs 1 to 17. Code is nearly immune. The curve is gentle up to about a tenth of calls.

Hit rate, global fit	H=8	H=16	H=32	H=48	H=56
Encyclopedia	0.16	0.33	0.60	0.83	0.93
Code	0.33	0.47	0.69	0.86	0.94
Math	0.29	0.46	0.70	0.88	0.96
Chat	0.12	0.26	0.54	0.80	0.92

H = experts resident per layer, out of 64. Hit rate = share of the router's top-8 selections on held-out text that land in the resident set, when the set was fit on all domains together.

Perplexity ÷ baseline	1 %	5 %	10 %	25 %	50 %
H=8 (84 % remote)	1.00	1.02	1.05	1.22	2.38
H=16 (67 % remote)	1.00	1.02	1.04	1.16	1.79
H=32 (40 % remote)	1.00	1.01	1.03	1.10	1.30
H=48 (17 % remote)	1.00	1.01	1.01	1.03	1.09

Columns = share of remote calls that miss their deadline and get a resident stand-in. Mean over the five domains; the worst single domain at H=32 and 25 % is news at 1.16, the best is code at 1.04.

Two plots. Left: hit rate on held-out text rises with the number of resident experts per layer for five domains, reaching 0.9 only near 56 of 64. Right: perplexity relative to baseline versus miss rate on remote calls, flat near 1.0 up to about 10 percent and rising steeply beyond 25 percent for small resident sets. — Left: Q2, hit rate against resident experts per layer, with the hot set fit on all domains (solid) or on the domain itself (dashed). Right: Q3, perplexity cost of the stand-in against the miss rate on remote calls.

Against the rule declared above. The first half of the kill condition is met: hot sets are not concentrated, so keeping "the hottest experts at home" buys little on a model trained with a load-balancing loss. The second half is not: a stand-in at a few percent miss rate costs a few percent, not a cliff. So the expert tier as a placement idea is dead on this class of model, while the degradation mechanism, a deadline on every hop and a resident stand-in for the call that is late, is viable. Pipeline-by-layer stays the design; stand-ins become the robustness layer on top of it.

Caveats, stated before anyone asks. One model, one size. A first version of the stand-in test masked experts before the softmax and inflated the kept experts' weights, which made the cost look catastrophic; the numbers above come from the corrected version, which takes every weight from the original router probabilities and is bit-identical to baseline on calls that did not miss. Math shows ratios below 1.0 at some settings, meaning the resident stand-in helped; 29 scored chunks, so that is noise until a larger run says otherwise. Next: a 128-expert model, and a model with shared experts, where part of every token is resident by construction.

Work with us on this

MoE serving, local networks, or a building full of idle memory.

info@embercompute.ai

Also at Ember Compute

Janus

The runtime that learns while it serves on one machine. WDMA is the other half: serving a model bigger than any one machine.