Watchdog: teaching pricing alerts context

Why a threshold was not enough

Pricing metrics have weekday shape, small-cohort noise, refunds and dependencies that finish at different times. A red flag based on yesterday versus a flat average can be mathematically correct and operationally misleading.

Watchdog began as an exploratory workbook and query set. The production task was to preserve the analytical reasoning while making every run repeatable, stateful and safe to consume.

This is a public-safe account of the work. Internal identifiers, proprietary thresholds, prompts, infrastructure details and operational commands are intentionally omitted.

Expectation and eligibility

The service builds expectations from comparable weekdays rather than an undifferentiated trailing mean. GBV and conversion are evaluated independently because their denominators, failure modes and business meanings differ. Sparse segments and insufficient history are made ineligible instead of being forced through the same thresholds.

Negative aggregate value is retained for reporting and financial impact. It is floored only inside calculations that mathematically require a positive input. Erasing refunds to make a transform convenient would produce a prettier statistic and a worse system.

Points, trends and transitions

A large one-day miss and a smaller persistent decline are different events. Watchdog evaluates both point magnitude and trend behaviour, then publishes the worse state. The current status is compared only with the latest earlier successful run, so a crashed or partial execution cannot rewrite alert history.

Each run is archived. The latest alert view is published only after the evaluation succeeds, keeping analytical history separate from the snapshot used by reporting tools.

Replay before trust

The implementation was replayed over a long historical window and compared with the analytical notebook. Regression tests covered negative values, state lookup and persistence behaviour. This was less glamorous than inventing a new anomaly model and far more important.

One limitation remains explicit: structural checks cannot prove every upstream source has completed when no authoritative readiness signal exists. The schedule and source contract still matter. Watchdog reports that boundary rather than converting uncertainty into false confidence.