Estimating Missing Reads with Load-Profile Interpolation

An interval meter that drops a read leaves a hole the billing engine cannot ignore: the period total still has to be allocated across intervals for time-of-use pricing, demand windows, and settlement. Filling that hole with a flat average is the naive move, and it is wrong in exactly the way that matters — it smears midday consumption into the evening peak and mis-prices the very intervals a time-of-use rate cares most about. A better estimate reshapes the gap to look like the customer actually behaves: take the known total for the period, subtract what was genuinely read, and distribute the remainder across the missing intervals in the proportions of a normalized load profile for that customer class, season, and day type. Every filled value is flagged ESTIMATED so it is auditable and cleanly superseded when the true read finally arrives. This page sits under fallback routing for missing rate data, part of the broader municipal utility billing architecture and rate taxonomy, and shows one focused, decimal-exact gap-filler.

# Minimal Prerequisites

• Python 3.11+ — for zoneinfo in the standard library and modern type-union syntax on the interval record.
• decimal.Decimal for every kWh quantity — estimation redistributes a real, billed total, so the parts must sum back to that total exactly. float cannot guarantee that, and a settlement that is off by a rounding crumb fails reconciliation.
• A normalized load profile — a sequence of non-negative weights, one per interval, describing the shape (not the magnitude) of typical consumption for the customer’s class, season, and day type. Only the ratios matter; the code normalizes by their sum.
• zoneinfo for interval boundaries — intervals are civil-time slots, so a DST day genuinely has 23 or 25 hourly intervals and the profile must align to the actual interval count, not a hardcoded 24.

Data assumptions: you have the interval reads for a period as an ordered list where a missing slot is None, and you know the period total independently — typically from a cumulative register read or a validated bulk total that brackets the gap. Estimation only ever fills the difference between that total and the sum of genuine reads.

# Annotated Implementation

The gap-filler is one function. It computes the residual (period total minus what was actually read), spreads that residual across only the missing intervals in load-profile proportions, and uses a largest-remainder pass so the estimated parts sum back to the residual to the cent — no drift, no silently dropped fractions.

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime
from decimal import Decimal, ROUND_HALF_UP

KWH = Decimal("0.001")  # interval reads carried to three decimal places


@dataclass(frozen=True)
class Interval:
    start: datetime          # timezone-aware interval start (civil time)
    kwh: Decimal | None      # None marks a missing read to be estimated


@dataclass(frozen=True)
class FilledInterval:
    start: datetime
    kwh: Decimal
    estimated: bool          # True only for values this routine invented
    profile_id: str | None   # which load profile shaped the estimate (audit)


def interpolate_missing(
    intervals: list[Interval],
    period_total: Decimal,
    profile_shape: list[Decimal],
    profile_id: str,
) -> list[FilledInterval]:
    """Fill None intervals by distributing the residual per a load-profile shape.

    profile_shape holds one non-negative weight per interval, aligned 1:1 with
    `intervals`. Only the ratios between missing-interval weights are used.
    """
    if len(profile_shape) != len(intervals):
        raise ValueError("profile_shape must align 1:1 with intervals")

    known = sum((iv.kwh for iv in intervals if iv.kwh is not None), Decimal("0"))
    residual = period_total - known
    if residual < 0:
        # Genuine reads already exceed the period total: this is a data-quality
        # fault (rollover, overlap), not a gap to fill. Refuse to invent values.
        raise ValueError("known reads exceed period total; nothing to estimate")

    missing_idx = [i for i, iv in enumerate(intervals) if iv.kwh is None]
    if not missing_idx:
        return [
            FilledInterval(iv.start, iv.kwh, False, None)  # type: ignore[arg-type]
            for iv in intervals
        ]

    weight_sum = sum((profile_shape[i] for i in missing_idx), Decimal("0"))
    n = len(missing_idx)

    # Proportional split, or an even split if the profile is flat/zero here.
    provisional: dict[int, Decimal] = {}
    for i in missing_idx:
        if weight_sum > 0:
            raw = residual * profile_shape[i] / weight_sum
        else:
            raw = residual / n
        provisional[i] = raw.quantize(KWH, rounding=ROUND_HALF_UP)

    # Largest-remainder reconciliation: push the rounding difference onto the
    # intervals whose truncation lost the most, so estimates sum to residual.
    drift = residual - sum(provisional.values(), Decimal("0"))
    if drift != 0:
        step = KWH if drift > 0 else -KWH
        # Order missing intervals by descending fractional remainder.
        ordered = sorted(
            missing_idx,
            key=lambda i: (residual * profile_shape[i] / weight_sum
                           if weight_sum > 0 else residual / n) - provisional[i],
            reverse=drift > 0,
        )
        units = int((abs(drift) / KWH).to_integral_value())
        for k in range(units):
            provisional[ordered[k % n]] += step

    out: list[FilledInterval] = []
    for i, iv in enumerate(intervals):
        if iv.kwh is not None:
            out.append(FilledInterval(iv.start, iv.kwh, False, None))
        else:
            out.append(FilledInterval(iv.start, provisional[i], True, profile_id))
    return out

Two properties make this billable rather than a guess. First, it only ever redistributes the residual — the reads that genuinely arrived are passed through untouched and never re-estimated, so an estimate can never overwrite a real measurement. Second, the largest-remainder pass guarantees the sum of every interval (real plus estimated) equals period_total exactly, which is the invariant settlement and reading anomaly detection both check. The estimated flag and profile_id travel with each value so an auditor can see precisely which slots were invented and which shape produced them — and so the record can be reversed. When a true read for a gap interval later arrives, you re-run the fill for the period with that slot no longer None; the residual shrinks and the surrounding estimates re-derive, exactly as the broader graceful fallback configuration governs.

# Edge Cases and Billing Gotchas

Interval estimation fails in ways specific to metering data and the billing calendar.

• Whole-period missing. When every interval is None, there are no genuine reads to subtract, so the residual equals the full period_total and the profile shape distributes the entire period. This is legitimate — the register or bulk total still bounds the estimate — but flag the whole period ESTIMATED and expect a heavier reconciliation review, since nothing anchors the shape to this customer’s actual behavior in the window.
• DST-length day. A civil day under a fall-back transition has 25 hourly intervals and a spring-forward day has 23. The profile_shape must be built for the actual interval count from zoneinfo, not a fixed 24, or the alignment check rejects the batch. A profile authored for 24 hours must be resampled to the day’s real length before it reaches this function.
• A later true read supersedes the estimate. An estimate is provisional. When the real read for a gap slot arrives, do not patch that one interval in place — re-run interpolate_missing for the whole period with the slot populated. The residual shrinks by the true value and every remaining estimate re-derives, keeping the period total exact and the audit trail coherent. Estimates carry estimated=True precisely so they are safe to discard.
• Negative or rollover values are excluded from estimation. A negative delta or a register rollover is a data-quality fault, not a shape to interpolate. If genuine reads already exceed the period total the function refuses rather than inventing negative fills; such intervals belong in reading anomaly detection, not in the gap-filler.

# Verification Snippet

Confirm the estimate reproduces the profile shape, that estimates plus known reads sum to the period total exactly, and that real reads are never overwritten.

from datetime import datetime, timedelta
from zoneinfo import ZoneInfo

def _slots(n: int) -> list[datetime]:
    base = datetime(2026, 7, 1, 0, 0, tzinfo=ZoneInfo("America/Chicago"))
    return [base + timedelta(hours=h) for h in range(n)]

def test_interpolation_preserves_total_and_shape() -> None:
    starts = _slots(6)
    # intervals 2 and 3 are missing; the rest are genuine reads
    intervals = [
        Interval(starts[0], Decimal("1.000")),
        Interval(starts[1], Decimal("2.000")),
        Interval(starts[2], None),
        Interval(starts[3], None),
        Interval(starts[4], Decimal("2.000")),
        Interval(starts[5], Decimal("1.000")),
    ]
    # evening-weighted shape: the second missing slot should get the larger share
    shape = [Decimal(w) for w in ("1", "1", "1", "3", "1", "1")]
    period_total = Decimal("12.000")

    filled = interpolate_missing(intervals, period_total, shape, "RES_SUMMER_WD")

    # residual = 12 - (1+2+2+1) = 6, split 1:3 across the two gaps -> 1.5 and 4.5
    assert filled[2].kwh == Decimal("1.500") and filled[2].estimated
    assert filled[3].kwh == Decimal("4.500") and filled[3].estimated
    assert filled[3].profile_id == "RES_SUMMER_WD"
    # real reads pass through untouched and unflagged
    assert filled[0].kwh == Decimal("1.000") and not filled[0].estimated
    # the invariant that makes it billable: parts sum to the period total exactly
    assert sum((f.kwh for f in filled), Decimal("0")) == period_total

Running this against fixtures drawn from real gaps — a whole missing day, a DST day, and a period where a late true read shrinks the residual — is what proves the reconciliation invariant holds before any estimated interval prices out.

# FAQ

# Related Topics

• Fallback Routing for Missing Rate Data — the parent guide to handling incomplete meter and rate data deterministically.
• Configuring Graceful Fallbacks for Incomplete Meter Data — the validated config object that governs window length, thresholds, and audit retention for this estimation.
• Reading Anomaly Detection Algorithms — the checks that decide whether a value is a genuine read, a fault, or a gap to estimate.
• Meter Data Ingestion & Validation Pipelines — the subsystem that delivers the interval reads this routine fills.
• Municipal Utility Billing Architecture & Rate Taxonomy — the architecture and rate-taxonomy overview this fallback workflow belongs to.

Up: Fallback Routing for Missing Rate Data · Municipal Utility Billing Architecture & Rate Taxonomy