Fallback Routing for Missing Rate Data in Municipal Utility Billing

A rate lookup that returns nothing forces the billing engine into one of three paths: fail loudly, guess silently, or fall back deliberately. Only the third is safe. When rate schedules, customer classifications, or consumption metrics fail to resolve during a billing cycle, unstructured error handling produces revenue leakage, compliance violations, and customer disputes in roughly that order. This guide sits under Municipal Utility Billing Architecture & Rate Taxonomy and treats a missing rate reference as a controlled state — a deterministic decision layer that intercepts null or malformed inputs before they propagate to invoice generation, produces a provisional charge on schedule, and tags the account for automated true-up once real data arrives.

The specific workflow failure this addresses is the silent estimate: a batch job that, faced with a null consumption_kwh or an unresolved rate_schedule_id, defaults to zero, to an arbitrary multiplier, or to the last value it happened to hold in memory. Each of those is a wrong bill posted to a real ratepayer. The remedy is to make the fallback path as rigorous as the happy path — strict schema validation at ingestion, an explicit gap classification, a tier-aware provisional estimate computed with decimal-safe arithmetic, preservation of assistance and jurisdiction flags, and an immutable record that lets an auditor reconstruct exactly why each estimated charge was produced.

# Prerequisites

This workflow targets a modern, reproducible Python stack. Pin the following before implementing anything below:

• Python 3.11+ — required for zoneinfo in the standard library and the Decimal | None union syntax used throughout.
• Pydantic v2 (pydantic>=2.6) — all payload contracts use v2 field_validator semantics; v1 will not validate these models. The same validation-first posture governs upstream schema validation & data quality checks.
• decimal.Decimal for all money and usage math — never float. Estimates are quantized once, at the charge boundary, with an explicit ROUND_HALF_UP.
• zoneinfo (not pytz) — the AMI read timestamp is resolved in the utility’s local civil time so that a late-night read lands on the correct billing day.

Data assumptions: reads arrive from the ingestion tier as candidate lookups carrying an account_id, a timezone-aware read_dt, a validated customer_class, a jurisdiction_code derived from the service address, and optionally consumption_kwh and rate_schedule_id — either of which may be null. Historical monthly consumption is available per account for baseline estimation. Rate schedules and block boundaries come from the authoritative rate repository whose write access is governed by Security Boundaries & Role-Based Access; fallback thresholds themselves are a governed configuration, not a code constant.

# Architecture Overview

Fallback routing is a staged decision layer, not a single except clause. A candidate lookup enters from the ingestion tier; a schema check diverts malformed records to quarantine; a gap classifier decides what is missing; a strategy selector maps the gap and the customer class to a deterministic estimation method; the estimate is computed with decimal-safe math; assistance and jurisdiction flags are preserved; and every application emits an audit record before the provisional account is queued for reconciliation.

Figure: Fallback routing as a deterministic decision layer — missing data follows a predefined, auditable path instead of failing the cycle.

The sections below implement each stage in order.

# Step-by-Step Implementation

# Step 1 — Validate the inbound lookup at ingestion

The foundation of reliable fallback routing is refusing to guess about malformed data — a null field is a controlled gap, but a garbled record is a defect that belongs in quarantine, not in an estimator. A Pydantic v2 contract enforces type constraints, mandatory fields, and cross-field integrity before any routing decision is made. Critically, consumption_kwh and rate_schedule_id are declared optional (they are the fields fallback exists to handle), while everything the fallback logic depends on — the account, the class, a timezone-aware timestamp, the jurisdiction — is mandatory.

from __future__ import annotations
from datetime import datetime
from decimal import Decimal
from enum import Enum
from pydantic import BaseModel, Field, field_validator

class CustomerClass(str, Enum):
    RESIDENTIAL = "residential"
    COMMERCIAL = "commercial"
    INDUSTRIAL = "industrial"

class RateLookup(BaseModel):
    account_id: str = Field(..., pattern=r"^ACCT-\d{6,}$")
    customer_class: CustomerClass
    read_dt: datetime                       # timezone-aware AMI read timestamp
    jurisdiction_code: str = Field(..., max_length=4)
    consumption_kwh: Decimal | None = None  # nullable: fallback handles this
    rate_schedule_id: str | None = None     # nullable: fallback handles this

    @field_validator("read_dt")
    @classmethod
    def require_tzaware(cls, v: datetime) -> datetime:
        if v.tzinfo is None:
            raise ValueError("read_dt must be timezone-aware")
        return v

    @field_validator("consumption_kwh")
    @classmethod
    def reject_negative(cls, v: Decimal | None) -> Decimal | None:
        # A negative register is corruption, not a gap — send it to quarantine.
        if v is not None and v < 0:
            raise ValueError("consumption_kwh cannot be negative")
        return v

Decoupling validation from calculation is what lets a high-throughput batch keep moving: conforming records with gaps proceed to routing, while genuinely broken payloads raise ValidationError and are logged to a quarantine queue for the same Reading Anomaly Detection Algorithms that flag negative-consumption and rollover defects. A negative cumulative register, for instance, is not a missing read — it is a corruption signal, and treating it as a fallback candidate would launder bad data into a plausible-looking bill.

# Step 2 — Classify the gap and select a strategy by customer class

Not all gaps are equal. A missing consumption value can be estimated from the account’s own history; a missing rate reference means the schedule itself failed to resolve and requires a different substitution; missing both is the most degraded state. The routing logic must also preserve the original Customer Class & Service Tier Mapping, because residential, commercial, and industrial accounts carry distinct fallback profiles — a residential account defaults to a rolling twelve-month average, while an industrial facility needs load-profile interpolation keyed to historical peak demand. Encoding the decision as an explicit table (rather than a nest of if statements) keeps it auditable and testable.

class GapType(str, Enum):
    NONE = "none"
    MISSING_CONSUMPTION = "missing_consumption"
    MISSING_RATE = "missing_rate"
    MISSING_BOTH = "missing_both"

class FallbackStrategy(str, Enum):
    PROCEED = "proceed_to_rate_engine"
    ROLLING_AVERAGE = "rolling_12mo_average"
    PEER_CLASS = "peer_class_substitution"
    LOAD_PROFILE = "load_profile_interpolation"
    REGULATORY_MINIMUM = "regulatory_minimum_charge"
    MANUAL_REVIEW = "flag_for_manual_review"

def classify_gap(lookup: RateLookup) -> GapType:
    missing_c = lookup.consumption_kwh is None
    missing_r = lookup.rate_schedule_id is None
    if missing_c and missing_r:
        return GapType.MISSING_BOTH
    if missing_c:
        return GapType.MISSING_CONSUMPTION
    if missing_r:
        return GapType.MISSING_RATE
    return GapType.NONE

# Deterministic strategy table keyed by (customer_class, gap_type).
STRATEGY_TABLE: dict[tuple[CustomerClass, GapType], FallbackStrategy] = {
    (CustomerClass.RESIDENTIAL, GapType.MISSING_CONSUMPTION): FallbackStrategy.ROLLING_AVERAGE,
    (CustomerClass.RESIDENTIAL, GapType.MISSING_RATE): FallbackStrategy.PEER_CLASS,
    (CustomerClass.COMMERCIAL, GapType.MISSING_CONSUMPTION): FallbackStrategy.PEER_CLASS,
    (CustomerClass.COMMERCIAL, GapType.MISSING_RATE): FallbackStrategy.PEER_CLASS,
    (CustomerClass.INDUSTRIAL, GapType.MISSING_CONSUMPTION): FallbackStrategy.LOAD_PROFILE,
    (CustomerClass.INDUSTRIAL, GapType.MISSING_RATE): FallbackStrategy.REGULATORY_MINIMUM,
}

def select_strategy(lookup: RateLookup, gap: GapType) -> FallbackStrategy:
    if gap is GapType.NONE:
        return FallbackStrategy.PROCEED
    # MISSING_BOTH and any unmapped pair fall through to manual review by design.
    return STRATEGY_TABLE.get((lookup.customer_class, gap), FallbackStrategy.MANUAL_REVIEW)

The table deliberately leaves MISSING_BOTH unmapped so it defaults to MANUAL_REVIEW: when both the meter reading and the rate schedule are absent there is no defensible basis for an automated estimate, and forcing one would be the silent-estimate failure this whole layer exists to prevent.

# Step 3 — Compute the provisional estimate with decimal-safe math

Once a strategy is chosen, the estimate must respect the underlying pricing architecture. For accounts on a stepped or block schedule, a missing read cannot be dumped into the highest or lowest band — that distorts conservation incentives and can violate Public Utilities Commission (PUC) guidance. The estimated usage is allocated marginally across block boundaries, exactly as real usage would be. The same slice-by-band model is documented for base pricing in Step-Rate vs Block-Rate Structure Design.

from decimal import Decimal, ROUND_HALF_UP

CENTS = Decimal("0.01")

def rolling_12mo_average(history_kwh: list[Decimal]) -> Decimal:
    """Baseline usage from up to twelve prior monthly reads."""
    if not history_kwh:
        raise ValueError("no history available for rolling average")
    window = history_kwh[-12:]
    return sum(window, Decimal("0")) / Decimal(len(window))

def estimate_block_charge(
    usage_kwh: Decimal,
    blocks: list[tuple[Decimal | None, Decimal]],
) -> Decimal:
    """Allocate estimated usage across block boundaries, marginally.

    blocks: ordered (upper_bound_kwh_or_None, rate_per_kwh); the final
    band uses None to mean 'no ceiling'.
    """
    total = Decimal("0")
    lower = Decimal("0")
    for upper, rate in blocks:
        ceiling = upper if upper is not None else usage_kwh
        slice_kwh = max(Decimal("0"), min(usage_kwh, ceiling) - lower)
        total += slice_kwh * rate
        lower = ceiling
        if upper is not None and usage_kwh <= upper:
            break
    # Quantize once, at the charge boundary, to avoid accumulated rounding drift.
    return total.quantize(CENTS, rounding=ROUND_HALF_UP)

Quantizing a single time at the end — rather than per band — is the decisive defense against the fractional-cent variance that reconciliation cannot later close. For residential accounts a rolling average smooths ordinary month-to-month noise; industrial accounts substitute a load-profile interpolation against historical peak demand, and commercial accounts on a PEER_CLASS strategy borrow a normalized profile from the same rate class and jurisdiction.

# Step 4 — Tag the provisional invoice and preserve compliance flags

A provisional charge is only safe if it is unambiguously marked provisional and if it cannot quietly strip a household’s protections. Estimated invoices carry an is_estimated flag and a requires_trueup flag so that downstream reconciliation knows to revisit them. Just as important, the fallback path must not disturb Assistance Program Eligibility Taxonomy enrollment: a provisional bill that later trues-up must never trigger a benefit clawback, so enrolled accounts receive a provisional_assistance_hold that freezes hardship-rate protections until the real read posts.

def build_provisional_charge(
    lookup: RateLookup,
    strategy: FallbackStrategy,
    estimated_usage: Decimal,
    estimated_charge: Decimal,
    assistance_enrolled: bool,
) -> dict:
    record = {
        "account_id": lookup.account_id,
        "jurisdiction_code": lookup.jurisdiction_code,
        "estimated_usage_kwh": str(estimated_usage),
        "provisional_charge": str(estimated_charge),
        "fallback_strategy": strategy.value,
        "is_estimated": True,
        "requires_trueup": True,
    }
    # A provisional bill must never jeopardize hardship protections.
    if assistance_enrolled:
        record["provisional_assistance_hold"] = True
    return record

Note what this record does not contain: no name, no address, no meter serial. Jurisdictional surcharges, franchise fees, and state utility taxes still apply to the provisional charge, but they are resolved from the jurisdiction_code (derived from service-address geocoding) rather than from estimated consumption — keeping the fee layer that decorates this bill, described in Surcharge & Fee Application Logic, correct even on an estimate.

# Step 5 — Emit an immutable audit record and queue for reconciliation

Every fallback application must leave verifiable proof of how the estimate was derived. Each record chains the previous record’s hash into its own SHA-256 digest, forming an append-only log: altering any historical estimate invalidates every hash after it. This is the fallback-side counterpart to the tamper-evident trails required across the platform, and it satisfies the metadata-only discipline mandated by Data Governance & Privacy Compliance — the log captures account IDs, strategy codes, amounts, and timestamps, never personally identifiable information.

import hashlib
from datetime import datetime, timezone

def audit_fallback(record: dict, prev_hash: str) -> dict:
    ts = datetime.now(timezone.utc).isoformat()
    payload = (
        f"{prev_hash}|{record['account_id']}|{record['fallback_strategy']}"
        f"|{record['provisional_charge']}|{ts}"
    )
    digest = hashlib.sha256(payload.encode()).hexdigest()
    return {
        "account_id": record["account_id"],
        "fallback_strategy": record["fallback_strategy"],
        "provisional_charge": record["provisional_charge"],
        "processed_at": ts,
        "prev_hash": prev_hash,
        "integrity_hash": digest,
    }

The audited provisional account then enters a pending-reconciliation queue rather than posting as final. When the actual meter read arrives in a subsequent cycle, the system auto-generates a true-up journal entry, posts the differential to the general ledger, and closes the exception ticket — a closed loop that prevents balance drift and preserves GAAP-compliant revenue recognition.

# Edge-Case Handling

Fallback logic lives or dies on its boundary conditions. The scenarios below are the ones that most often turn a “graceful” fallback into a silent error:

• Null AMI read (the base case). A communication dropout leaves consumption_kwh null while everything else is intact. This is the designed path: classify as MISSING_CONSUMPTION, estimate from history, tag requires_trueup. Never substitute zero — zero is a real, wrong bill.
• Missing both read and rate. With no meter reading and no schedule, no automated estimate is defensible. The strategy table routes this to MANUAL_REVIEW; the cycle still completes for every other account.
• Negative or rollover register. A meter that wraps past its maximum can report a negative delta. This is corruption, not a gap — Step 1 rejects it so it is diverted to anomaly detection, not laundered into a rolling average.
• New account with no history. rolling_12mo_average raises on an empty history list rather than dividing by zero. Genuinely new accounts fall back to PEER_CLASS substitution or REGULATORY_MINIMUM, never to a fabricated baseline.
• DST boundary read. A read timestamped during the autumn “fall-back” hour is ambiguous in local time. Resolving read_dt through zoneinfo and comparing on the extracted local .date() keeps a late-night estimate on the correct billing day.
• Mid-cycle PUC rate change. When a commission order takes effect partway through a cycle, the provisional estimate must be priced under the schedule version in force on the read’s service date — point-in-time resolution, never the current version applied retroactively to the whole period.
• Assistance-enrolled account. An estimate for a household on a hardship rate must set provisional_assistance_hold so the later true-up cannot claw back a benefit the ratepayer was entitled to.

# Verification and Audit Trail

Correctness is confirmed two ways: deterministic replay of the estimate, and reconciliation of the true-up once the real read arrives. Replay proves the same lookup yields the same provisional charge; the true-up check proves the estimate stayed within an acceptable band of reality.

from decimal import Decimal

def verify_trueup(
    provisional: Decimal,
    actual: Decimal,
    tolerance: Decimal = Decimal("0.15"),
) -> dict:
    """Compare a provisional estimate to the real charge once it posts."""
    delta = (actual - provisional).quantize(Decimal("0.01"))
    rel = abs(delta) / actual if actual > 0 else Decimal("0")
    return {
        "adjustment": delta,             # signed GL true-up amount
        "relative_error": rel,
        "within_tolerance": rel <= tolerance,
    }

Run the same routing suite against a fixture of historical bills whose real reads and totals are already known: for each, blank the consumption, route it through fallback, and confirm the provisional estimate lands within tolerance of the true figure. Estimates that repeatedly breach tolerance for a given class or jurisdiction signal a stale baseline or a mis-tuned strategy table — caught in test, before a cycle. To confirm the audit chain, recompute each integrity_hash from its prev_hash and stored fields and compare to the persisted digest; any mismatch localizes the exact record that was altered.

# Troubleshooting

# Related Topics

• Municipal Utility Billing Architecture & Rate Taxonomy — the parent guide covering the full rate and account model this routing protects.
• Customer Class & Service Tier Mapping — the class profiles that select each fallback strategy.
• Step-Rate vs Block-Rate Structure Design — the marginal-slice model reused to price a provisional estimate.
• Surcharge & Fee Application Logic — the fee layer that decorates a provisional bill and diverts null-base accounts here.
• Error Handling & Retry Workflows — resilient posting of audit records and true-up journal entries.
• Configuring Graceful Fallbacks for Incomplete Meter Data — parameter tuning for historical windows, exception thresholds, and audit retention.

Up: Municipal Utility Billing Architecture & Rate Taxonomy