跳轉到

Recap Architecture Notes

Supplementary architectural notes for the recap system (daily + weekly + monthly). See also docs/recap-frontend-integration.md for the frontend-facing API surface.

Periodic recap indicator set

The periodic recap (weekly / monthly) event classifier reads a minimal set of technical indicators rather than the full 30+ indicator surface of the daily pipeline. The contract is published as:

data_pipeline.indicators.IndicatorCalculator.RECAP_INDICATOR_COLUMNS

Columns in the contract:

  • ma_20, ma_60
  • rsi
  • volume_ma_20
  • donchian_high_20, donchian_low_20
  • donchian_high_55, donchian_low_55
  • supertrend_direction

Callers must import from this tuple rather than hard-coding their own list of columns. Behavioural coverage lives in tests/test_periodic_technical_raw_ma60_fix.py::TestMa60BugFix, which asserts that ma_60 is populated, pct_above_sma_60 can be True, and trend_state correctly reflects uptrend / downtrend directions. A drift in the contract that drops a column the aggregator reads will surface as a failing test there (or as a KeyError at runtime if a column is removed without updating its consumer).

calculate_for_recap

The recap hot path uses IndicatorCalculator.calculate_for_recap(df), a @classmethod that builds an ephemeral calculator with a narrowed config and computes only the 9-column RECAP_INDICATOR_COLUMNS set. Micro-benchmarks on a 100-bar weekly series show it runs ~5× faster than calculate_all (2.8ms vs 14.9ms). At the periodic aggregator level the speedup translates to roughly a 2.8× wall-time reduction on the full 2300-ticker Taiwan universe (18.9s → ~6.7s).

Values on the overlapping columns are bit-identical to calculate_all: calculate_for_recap reuses the same _add_* methods and the same arithmetic, just with a narrowed config. This is enforced by test_calculate_for_recap.test_bit_identical_with_calculate_all_for_overlapping_columns.

Historical ma_60 bug (pre-PR before 2026-04)

Before the calculate_for_recap rewrite, periodic_technical_raw instantiated IndicatorCalculator() with the default config, whose MA periods list [5, 10, 20, 50, 100, 200] silently omitted 60. Reading latest["ma_60"] always returned None, so:

  • _classify_ma_alignment always returned "neutral"
  • pct_above_sma_60 was always False
  • trend_state never landed on uptrend / downtrend via ma_alignment

The fix uses MA periods [20, 60] inside calculate_for_recap's narrow config. The trend_state and pct_above_sma_60 columns in historical backfill data are corrupted and require a full re-run of the 2018-2026 periodic backfill (scripts/backfill_periodic_recap.py) to correct.

The scheduler's live weekly / monthly jobs remain safe to run during the backfill window: every periodic table write uses ON CONFLICT DO UPDATE UPSERT semantics, so overlap between a live run and the backfill just overwrites a row with another correct row from the same code path.

Shared periodic queries

services.periodic_shared_queries owns two DB helpers (previously duplicated byte-for-byte in both periodic_technical_raw and periodic_chip_raw):

  • load_active_tickers_with_groups(session, market)
  • count_trading_sessions(session, market, start, end)

The run_periodic_recap runner prefetches both once and threads the result into each aggregator via a shared_inputs dict. Each aggregator still accepts a shared_inputs=None fallback that queries itself — this keeps the public API callable by integration tests and scripts outside the orchestrator.