All Resources
R-37
Electronics
Edge Compute: Deciding What Never Leaves the Building
Where to put the computation when the network is the bottleneck — the split rule, the tiers, the fleet update path, and the failure modes that only appear on a factory floor.
PAR2 Labs
August 21, 2026
15 min
Edge compute is usually sold on latency and usually bought for bandwidth, and the real reason is often neither: it is that the site cannot depend on a link it does not control. Deciding what runs at the edge is an architecture decision with a clean rule behind it, and this is the rule, the tiers it produces, and what it costs you to operate.
01
The split rule
Run at the edge anything where the round trip changes the outcome, the data volume makes transport uneconomic, or the site must keep functioning with the link down. Run in the cloud anything that benefits from aggregation across sites, needs elasticity, or changes faster than you can safely update a fleet.
Applied honestly, that rule usually produces a smaller edge footprint than people expect — and a much clearer one. Inference at the edge, training in the cloud. Control loops at the edge, analytics in the cloud. Buffering at the edge, retention in the cloud.
Fig 1 — the disconnection test is the fastest way to settle an argument about placement. Everything that must survive it belongs at the edge; everything else is a preference.
02
Three tiers, not one
'Edge' collapses three very different classes of hardware into one word, and specifying the wrong tier is the most expensive mistake in the project. A microcontroller and an on-site GPU server share almost nothing operationally.
| Tier | Typical hardware | Runs | Update path |
|---|---|---|---|
| Device | MCU, 64KB–1MB RAM, battery or line power | Sensing, signal conditioning, tiny models, a radio stack | Signed OTA image, A/B partitions, rollback on failed boot |
| Gateway | SBC or industrial PC, 2–8GB RAM, sometimes an NPU | Protocol translation, buffering, filtering, small-model inference | Containers with pinned digests, staged rollout by ring |
| On-site server | Rack or ruggedised box with a discrete GPU | Video analytics, multi-camera inference, local historian | Same as cloud CI/CD, but with a maintenance window |
Cost of a field visit dominates everything. Design so that no tier ever requires one to recover.
If the link goes down for a day, what must still work? That is your edge scope.
03
Standing it up
The order matters. Teams that start with the model and work outward end up rebuilding the transport and update layers under time pressure, which is when field-unrecoverable devices get shipped.
01
Write the disconnection spec first
TOOL
A one-page document
USE
What must work at 0 bandwidth, for how long
GET
Edge scope, decided
State plainly: with the uplink down, which functions continue, how much data is buffered locally, and for how long before loss. Get the site operator to agree it before hardware is chosen.
Why: This single document settles almost every later architecture argument and sizes local storage, which is otherwise guessed and always guessed low.
02
Pick the tier against the workload, not the brochure
TOOL
Bench measurement
USE
Measured inference time and power on candidate hardware
GET
A justified bill of materials
Benchmark the actual model on the actual candidate boards, at the real input resolution and duty cycle. Measure sustained throughput with the enclosure closed, not peak throughput on a bench in open air.
Why: Thermal throttling in a sealed enclosure routinely halves sustained performance. A device chosen on peak numbers will miss its deadline in the field and there is no software fix.
03
Normalise the model for the target
TOOL
ONNX Runtime, or the vendor runtime
USE
Export, quantise, and validate accuracy after quantisation
GET
A deployable artefact with known accuracy
Export to ONNX, quantise to INT8 with a representative calibration set, and re-measure accuracy against your held-out set. Where a vendor accelerator is present, compile through its toolchain and re-measure again.
Why: Quantisation is not free and the loss is not uniform — it usually lands on exactly the rare classes you care about. Measuring after every conversion is the only way to know what you shipped.
04
Choose transport for the worst link, not the demo link
TOOL
MQTT with persistent sessions
USE
QoS 1, clean session off, local queue with backpressure
GET
Delivery that survives an outage
Publish over MQTT with QoS 1 and a persistent session so the broker retains messages across reconnects. Buffer locally with an explicit ring size and a documented drop policy — oldest first, or lowest priority first.
Why: Networks at industrial sites drop for minutes at a time, routinely. A transport without a local queue and a defined drop policy silently loses the data that mattered most during the incident you needed it for.
05
Make updates safe before you make them frequent
TOOL
A/B partitions and signed images
USE
Signature verification, health check, automatic rollback
GET
A fleet you can update without a van
Ship dual partitions with signed images. Boot the new partition, run a health check with a watchdog, and roll back automatically if it does not report healthy within the window. Stage releases in rings, starting with a handful of devices.
Why: The failure that ends a project is a bad update that bricks a fleet in a location that needs an engineer to visit. Rollback is what makes every other release decision reversible.
06
Instrument the device, not just the data
TOOL
Your telemetry channel
USE
Uptime, temperature, queue depth, model latency, version
GET
Fleet visibility
Report device health on the same channel as the payload: firmware and model version, queue depth, sustained temperature, inference latency, and reboot count. Alert on queue depth trending up — it is the earliest signal of a link or throughput problem.
Why: Without version and health telemetry you cannot tell whether a fleet-wide anomaly is a bad model, a bad release, or a bad month for the network — and you will guess wrong.
04
What goes wrong on site
Every item here comes from the gap between a lab and a plant. None of them are visible during development.
Field failure modes
01
Thermal throttling in a sealed enclosure halves sustained throughput and the deadline is missed only in summer.
02
The local buffer has no defined drop policy, so during the outage it discarded exactly the readings the incident review needed.
03
An update shipped without rollback and a device that will not boot is a two-hour drive away.
04
Clocks drift without NTP and correlating events across devices becomes impossible after the fact.
05
Quantisation was validated on the aggregate accuracy number and quietly destroyed the rare class that justified the project.
06
A safety function was implemented in the cloud path. It must not be — safety interlocks are local, always.
05
Reference
The runtime and messaging documentation matters most here; hardware choices date quickly, but the transport and update patterns do not.
Primary documentation
ONNX Runtime
Portable inference across CPU, GPU and vendor accelerators, plus quantisation tooling.
NVIDIA embedded computing
The Jetson class of on-site inference hardware and its power modes.
MQTT
QoS levels, persistent sessions and last-will — the transport assumptions in step 04.
balena
Container delivery, staged rollout and rollback for gateway-tier fleets.
OPC UA — overview
The industrial protocol most gateway work has to translate to and from.
OpenTelemetry
Consistent telemetry from edge through cloud, so traces span the boundary.
Key Takeaways
01
Write the disconnection spec first — it decides edge scope and sizes local storage.
02
Benchmark the real model on the real board with the enclosure closed.
03
Re-measure accuracy after every quantisation and compilation step.
04
Ship A/B partitions with automatic rollback before you ship features.
PAR2 Labs · Electronics
Work With Us