How Much Battery an On-Device Model Actually Uses

“Does the model fit in the power budget” is a question with an arithmetic answer, and the arithmetic is short. What makes it surprising is that the model is usually not the expensive part of running the model.

The unit is joules, not FLOPs

Operation counts do not convert to energy at a fixed rate. The same multiply costs different amounts depending on where its operands come from: a value already in a register is nearly free, one from on-chip SRAM costs more, and one fetched from external flash or DRAM costs far more than the arithmetic that consumes it. This is why a model with fewer operations but a larger working set can burn more energy than a denser one that fits in cache, and why quantisation helps energy more than the operation count suggests — it shrinks what has to be moved.

The quantity that composes cleanly is energy per inference in joules, because it adds up over a day and divides into a battery. Get it from three numbers: the current drawn above idle while inferring, the supply voltage, and the duration.

E = I * V * t

I = current above idle during inference (amps)
V = supply voltage (volts)
t = wall-clock inference duration (seconds)

The derivation

The inputs below are assumptions for a plausible wearable-class device. They are not a measurement of any product; substitute your own numbers from the procedure in the last section.

ASSUMPTIONS
  inference duration    18 ms
  current above idle    45 mA during inference
  supply voltage        3.7 V
  battery               300 mAh at 3.7 V
  idle current          120 uA (MCU sleeping, sensor FIFO filling)

ENERGY PER INFERENCE
  power  = 0.045 A * 3.7 V           = 0.1665 W
  energy = 0.1665 W * 0.018 s        = 0.003 J = 3.0 mJ
  charge = 45 mA * 0.018 s           = 0.81 mA*s = 0.000225 mAh

BATTERY TOTAL ENERGY
  300 mAh * 3.7 V = 1,110 mWh = 3,996 J   (call it 4.0 kJ)

AT THREE INFERENCE RATES
  one every 30 s   2,880/day
    charge/day   2,880 * 0.000225 mAh  = 0.648 mAh/day
    added current 0.648 / 24           = 27 uA
    total current 120 + 27             = 147 uA
    lifetime      300,000 / 147        = 2,041 h = 85 days

  one every 5 s    17,280/day
    charge/day   17,280 * 0.000225     = 3.89 mAh/day
    added current 3.89 / 24            = 162 uA
    total current 120 + 162            = 282 uA
    lifetime      300,000 / 282        = 1,064 h = 44 days

  4 per second     345,600/day
    charge/day   345,600 * 0.000225    = 77.8 mAh/day
    added current 77.8 / 24            = 3,240 uA
    total current 120 + 3,240          = 3,360 uA
    lifetime      300,000 / 3,360      = 89 h = 3.7 days

Read the middle column rather than the endpoints. At one inference every thirty seconds the model contributes 27 µA against an idle draw of 120 — it is 18 percent of the budget and eliminating it entirely would extend life from 85 to 104 days. Optimising the model here is close to pointless; the sleep current is the target. At four inferences a second the model is 96 percent of the budget and every millisecond shaved off the duration converts directly into runtime.

That crossover, not any absolute figure, is the thing to carry away. Work out where your inference rate sits relative to the idle draw before deciding whether model optimisation is worth anyone’s time.

The wake-up cost nobody budgets for

The 18 ms above is the inference. Around it sit costs that are easy to forget and frequently larger.

• Waking from deep sleep is not instantaneous. Restoring clocks and regulators takes time during which the part is drawing active current and doing nothing useful. If waking costs 5 ms at 8 mA, that is 0.148 mJ — five percent of a 3 mJ inference, and a much larger share for a shorter one.
• Loading weights from external flash is a transfer. A 200 kB model read over a QSPI bus at each invocation costs energy proportional to the bytes moved, every time, unless the part can execute in place from flash or keep the model resident in RAM. Keeping it resident often costs less energy overall than the repeated fetch, even though resident RAM has a standing cost.
• Preprocessing counts. A 256-point FFT per window is arithmetic you pay for whether or not you call it part of the model. Measure the whole path from sample to decision.
• The floor is the sensor. Acquisition has its own duty cycle, derived separately in choosing a sampling rate, and on slow-inference designs it commonly exceeds the model.

Compared against sending the data instead

The comparison that justifies on-device inference is against transmitting the raw window and letting a server decide. Continuing the same assumptions and adding a radio:

ASSUMPTIONS
  BLE transmit current  10 mA at 3.7 V
  effective throughput  40 kB/s including connection overhead
  window payload        200 samples * 6 axes * 2 B = 2,400 B

TRANSMIT ONE WINDOW
  time   = 2,400 / 40,000                 = 0.060 s
  energy = 0.010 A * 3.7 V * 0.060 s      = 2.22 mJ

TRANSMIT ONE LABEL (8 bytes)
  time   = 8 / 40,000                     = 0.0002 s
  energy = 0.010 * 3.7 * 0.0002           = 0.0074 mJ

PER WINDOW
  infer locally, send label  3.0 + 0.0074 = 3.01 mJ
  send raw window            2.22 mJ

On these numbers, sending the raw window is cheaper than running the model — which is the opposite of the usual claim, and it is a real result for a short-range radio with a good connection. The on-device argument wins when the link is expensive rather than cheap: a cellular or LoRaWAN uplink costs orders of magnitude more energy per byte than BLE, the connection may not exist when the event happens, and per-message costs on the network side, worked through in ingest cost, scale with message count rather than with bytes.

The honest summary is that on-device inference is a bandwidth, latency, availability and privacy decision far more often than an energy one. Run the comparison for your link before assuming local is cheaper.

One further asymmetry is worth stating, because it inverts the usual software instinct. Making a model faster does not always make it cheaper in energy. On a part with dynamic voltage and frequency scaling, running the same work at a higher clock finishes sooner but draws disproportionately more power while it does, since dynamic power rises faster than linearly with frequency. The energy-optimal operating point is often the slowest clock that still meets the deadline, which is the opposite of the latency-optimal one. If the device has a governor, the inference duration and the current draw in the derivation above are not independent inputs, and measuring one without pinning the other produces numbers that do not reproduce.

How to measure it on your own device

1. Put the device on a supply that can log current at high rate — a source measure unit or a dedicated power profiler. A multimeter averages over a window far longer than an inference and will report something meaningless.
2. Record a baseline with the device idle for at least a minute, and take the mean. That is the idle term, and it is the number the whole budget is relative to.
3. Toggle a GPIO high immediately before the inference call and low immediately after, and capture it on the same time base as the current trace. This marks the window unambiguously and avoids trying to identify it by eye.
4. Integrate current over the marked window and subtract the idle current times the same duration. Multiply by supply voltage for joules. Repeat over at least a hundred inferences and report the median and the spread, since cache state and clock scaling make individual runs vary.
5. Repeat with the sensor path and preprocessing included, so you have both the model’s own cost and the cost of the full path, and know which one you are optimising.