Configuring Graceful Fallbacks for Incomplete Meter Data

A fallback engine is only as safe as the numbers that drive it. The decision layer described in Fallback Routing for Missing Rate Data — part of Municipal Utility Billing Architecture & Rate Taxonomy — resolves which estimation strategy a null or anomalous read receives; this page resolves how it is tuned. The failure mode it prevents is the buried constant: a historical-average window hard-coded to twelve months, a spike threshold pinned at 3.0, or a seasonal multiplier baked into a function body where no auditor can see it and no analyst can change it without a code deploy. When those numbers live in source, a commission-mandated change to estimation methodology becomes an engineering ticket, and a mis-tuned threshold silently over- or under-bills an entire customer class for a cycle. The fix is to lift every tunable into one validated, governed configuration object — versioned, range-checked, and decimal-safe — that the estimator merely reads.

Minimal prerequisites

  • Python 3.11+ — for zoneinfo in the standard library and Decimal | None union syntax.
  • Pydantic v2 (pydantic>=2.6) — the config contract uses v2 field_validator and model_validator semantics; v1 will not enforce these constraints.
  • decimal.Decimal for every rate, threshold, and multiplier — never float. Estimates quantize once, at the charge boundary, with ROUND_HALF_UP.
  • zoneinfo — the read timestamp is resolved in the utility’s civil time so a late-night read lands on the correct billing month for seasonal lookup.

Data assumptions: per-account monthly history is available as an ordered list of Decimal kWh values (oldest first), and the fallback configuration is loaded from a governed store — not a code constant — whose write access is controlled per Security Boundaries & Role-Based Access.

Annotated implementation

The whole point is that the estimator holds no magic numbers. A single FallbackConfig model validates every tunable at load time, so an out-of-range window or a negative threshold fails fast at configuration boundaries rather than producing a plausible-looking wrong bill deep inside a batch run.

from __future__ import annotations

from datetime import datetime
from decimal import Decimal, ROUND_HALF_UP
from enum import Enum
from zoneinfo import ZoneInfo

from pydantic import BaseModel, Field, field_validator, model_validator

CENTS = Decimal("0.01")


class CustomerClass(str, Enum):
    RESIDENTIAL = "residential"
    COMMERCIAL = "commercial"
    INDUSTRIAL = "industrial"


class EstimationMethod(str, Enum):
    ROLLING_AVERAGE = "rolling_average"      # smooth month-to-month noise
    SEASONAL_INDEX = "seasonal_index"        # weight the class's seasonal curve


class FallbackConfig(BaseModel):
    """Every tunable that governs estimation for an incomplete read.

    Loaded from a governed store and validated once, so no threshold or
    window ever lives as a hard-coded constant inside the estimator.
    """
    config_version: str = Field(..., pattern=r"^v\d+\.\d+$")   # for point-in-time audit
    timezone: str = "America/New_York"                          # civil time for month lookup

    # How many trailing monthly reads feed the baseline (1..24).
    history_window_months: int = Field(12, ge=1, le=24)
    # Minimum reads required before an automated estimate is allowed.
    min_history_reads: int = Field(3, ge=1)
    # A read above baseline * (1 + spike_sigma) is corruption, not a gap.
    spike_sigma: Decimal = Field(Decimal("3.0"), gt=Decimal("0"))
    # Per-class estimation method.
    method_by_class: dict[CustomerClass, EstimationMethod] = Field(default_factory=dict)
    # Month index (1..12) -> multiplier for SEASONAL_INDEX accounts.
    seasonal_multipliers: dict[int, Decimal] = Field(default_factory=dict)
    # Retention floor (days) for the fallback audit record; regulator-set.
    audit_retention_days: int = Field(2555, ge=365)             # ~7 years default

    @field_validator("timezone")
    @classmethod
    def known_zone(cls, v: str) -> str:
        ZoneInfo(v)  # raises ZoneInfoNotFoundError on an invalid IANA name
        return v

    @field_validator("seasonal_multipliers")
    @classmethod
    def valid_month_keys(cls, v: dict[int, Decimal]) -> dict[int, Decimal]:
        for month, factor in v.items():
            if not 1 <= month <= 12:
                raise ValueError(f"seasonal month {month} out of range 1..12")
            if factor <= 0:
                raise ValueError("seasonal multiplier must be positive")
        return v

    @model_validator(mode="after")
    def seasonal_classes_have_curve(self) -> "FallbackConfig":
        uses_seasonal = any(
            m is EstimationMethod.SEASONAL_INDEX for m in self.method_by_class.values()
        )
        if uses_seasonal and len(self.seasonal_multipliers) != 12:
            raise ValueError("SEASONAL_INDEX requires all 12 monthly multipliers")
        return self


def estimate_usage(
    history_kwh: list[Decimal],
    customer_class: CustomerClass,
    read_dt: datetime,
    config: FallbackConfig,
) -> Decimal:
    """Produce a decimal-safe estimated usage governed entirely by config."""
    if len(history_kwh) < config.min_history_reads:
        # Too little history to defend an automated figure — caller escalates.
        raise ValueError("insufficient history for a configured estimate")

    window = history_kwh[-config.history_window_months:]
    baseline = sum(window, Decimal("0")) / Decimal(len(window))

    method = config.method_by_class.get(customer_class, EstimationMethod.ROLLING_AVERAGE)
    if method is EstimationMethod.SEASONAL_INDEX:
        # Resolve the read's civil month, not UTC, so a DST-boundary
        # late-night read maps to the correct seasonal factor.
        local_month = read_dt.astimezone(ZoneInfo(config.timezone)).month
        factor = config.seasonal_multipliers[local_month]
        estimate = baseline * factor
    else:
        estimate = baseline

    # Quantize once, at the boundary, to avoid accumulated rounding drift.
    return estimate.quantize(CENTS, rounding=ROUND_HALF_UP)


def is_spike(read_kwh: Decimal, history_kwh: list[Decimal], config: FallbackConfig) -> bool:
    """A read far above the configured baseline is corruption, not a gap."""
    if not history_kwh:
        return False
    baseline = sum(history_kwh, Decimal("0")) / Decimal(len(history_kwh))
    return read_kwh > baseline * (Decimal("1") + config.spike_sigma)

Because spike_sigma, history_window_months, and the seasonal curve are all fields on one model, a mid-year methodology change is a data edit against the governed store, stamped with a new config_version. That version travels with every audit record, so a reviewer can reconstruct exactly which parameters priced any historical estimate — the point-in-time discipline required whenever a Public Utilities Commission (PUC) order changes how estimates are computed.

Edge cases and billing gotchas

  • Window longer than the account’s history. A twelve-month history_window_months on a six-month-old account must not divide by twelve. Slicing with history_kwh[-window:] and averaging over len(window) uses whatever exists; if that is still below min_history_reads, the estimator raises and the account escalates to manual review rather than fabricating a baseline. This is the configuration counterpart to the new-account gap handled in the parent Customer Class & Service Tier Mapping profiles.
  • Spike threshold that quietly launders corruption. Set spike_sigma too high and a wrapped-around (rollover) register reading passes as a real spike and poisons the very history a future estimate draws on. Keep the spike gate ahead of the average so anomalies are diverted to Reading Anomaly Detection Algorithms, never averaged in.
  • Seasonal multiplier at a DST boundary. A read timestamped during the autumn “fall-back” hour is ambiguous in local time. Resolving read_dt through zoneinfo before extracting .month keeps a late-night December read from being priced with a November multiplier. The model_validator also refuses a partial seasonal curve, so a class configured for SEASONAL_INDEX can never hit a missing-month KeyError mid-cycle.
  • Retention floor below the regulator’s minimum. audit_retention_days is range-checked at load, but the floor itself is policy: a value that satisfies code yet violates the record-keeping window in Data Governance & Privacy Compliance is a compliance defect the config review must catch, not the type system.

Verification snippet

Validate the configuration and the estimate it drives against a known fixture before any batch consumes it. The assertions confirm that range checks fire, that a short window degrades gracefully, and that a seasonal read lands on its civil month.

from decimal import Decimal

def test_config_and_estimate() -> None:
    cfg = FallbackConfig(
        config_version="v2.1",
        timezone="America/New_York",
        history_window_months=12,
        min_history_reads=3,
        spike_sigma=Decimal("3.0"),
        method_by_class={CustomerClass.RESIDENTIAL: EstimationMethod.ROLLING_AVERAGE},
        audit_retention_days=2555,
    )

    history = [Decimal("100"), Decimal("120"), Decimal("110"), Decimal("130")]
    read_dt = datetime(2026, 1, 15, 2, 30, tzinfo=ZoneInfo("America/New_York"))

    est = estimate_usage(history, CustomerClass.RESIDENTIAL, read_dt, cfg)
    assert est == Decimal("115.00")            # mean of the window, quantized once

    # A spike above baseline * (1 + sigma) is rejected, not averaged in.
    assert is_spike(Decimal("500"), history, cfg) is True

    # Too little history escalates rather than fabricating a baseline.
    try:
        estimate_usage(history[:2], CustomerClass.RESIDENTIAL, read_dt, cfg)
        assert False, "expected escalation on insufficient history"
    except ValueError:
        pass

    # An out-of-range window fails at load, not deep in a batch run.
    try:
        FallbackConfig(config_version="v2.1", history_window_months=99)
        assert False, "expected validation error on window"
    except Exception:
        pass

Run the same suite against a corpus of hand-verified historical bills: blank the read, price it under the candidate config, and confirm the estimate lands within tolerance of the known true figure. A config change that pushes a class’s estimates consistently off is caught here, before a cycle ships thousands of mis-tuned bills.

A governed FallbackConfig object driving the estimator One governed configuration object holds every tunable — config_version, history_window_months (1 to 24), min_history_reads, spike_sigma, seasonal_multipliers for the twelve months, and audit_retention_days. The object is range- and cross-field-validated once at load, then drives the estimate_usage function, which reads an account's monthly history of decimal kWh values. The history first passes a spike gate that diverts any read above baseline times one plus sigma to anomaly detection, so corruption never poisons the baseline; clean history feeds the estimator. The estimator slices the trailing window, averages a baseline, selects the method by customer class, applies the seasonal multiplier resolved on the read's civil month, and quantizes once with ROUND_HALF_UP. It emits a decimal-safe estimate, which is logged to an audit record stamped with the config_version for point-in-time review. FallbackConfig governed store · loaded, not hard-coded config_versionv2.1 history_window_months1–24 min_history_reads≥1 spike_sigma>0 seasonal_multipliers{1–12} audit_retention_days≥365 Validate once at load range + cross-field checks estimate_usage() 1window = history[-N:] 2baseline = mean(window) 3method by customer class 4seasonal × month (zoneinfo) 5quantize once · HALF_UP Account monthly history Decimal kWh · oldest first Spike gate > baseline·(1+σ)? Anomaly detection diverted · never averaged in Decimal-safe estimate quantized once Audit record stamped config_version point-in-time drives every tunable log clean history spike

Frequently Asked Questions

Why lift these thresholds into a config object instead of leaving sensible defaults in code?
Because estimation methodology is a governed, sometimes commission-regulated policy, not an implementation detail. A hard-coded window or spike threshold means every methodology change is a code deploy, and no auditor can see the number that priced a given estimate. A validated config object makes each tunable a range-checked, versioned data value; the same estimator code is provably driven by the parameters recorded in the audit trail.
How long should the historical window be?
Long enough to smooth month-to-month noise, short enough to track real consumption trends — twelve months is a common default because it spans a full seasonal year for rolling-average accounts. Accounts on a seasonal index can use a shorter window since the seasonal multiplier already carries the annual shape. The window is range-checked to 1–24 months; validate any candidate value against a fixture of known bills before shipping it.
What happens to an estimate if the config changes mid-billing-cycle?
Each estimate is priced under the config_version in force on the read's service date and stamps that version into its audit record, so a mid-cycle change never retroactively reprices reads already estimated. This is the same point-in-time resolution used for a mid-cycle PUC rate change: resolve against the version effective when the read occurred, never the current version applied backward over the whole period.
Should the spike threshold reject a read or estimate over it?
Reject it. A read above baseline * (1 + spike_sigma) is treated as corruption — a rollover, a swapped meter, or a transmission fault — not a legitimate gap. It is diverted to anomaly detection and excluded from the history that future estimates average, rather than substituted with an estimate. Estimation is only for missing reads; a present-but-implausible read is a different defect.
Why validate audit_retention_days in the config at all?
Because a retention floor set too low silently violates records-management policy, and the failure surfaces only during an audit years later. The field is range-checked so an obviously wrong value (under a year) fails at load, but the true minimum is a governance decision — the config review must confirm the value meets the record-keeping window the jurisdiction mandates for estimated charges.

Up one level: Fallback Routing for Missing Rate Data · Parent guide: Municipal Utility Billing Architecture & Rate Taxonomy · Return to the utilitybilling.org home.