Skip to content
fitme·story
fitme·story
View as
Case studies
v7.8.5 · 3 min read · 2026-06-09

Analytics Observability — closing the measurement-debt loop (taxonomy cleanup + a live GA4 path)

The project had shipped extensive analytics instrumentation but had no clean taxonomy and no live way to observe events landing — instrumentation was write-and-hope. This feature closed the measurement-debt loop across 16 merged PRs: the analytics taxonomy CSV was brought back into sync (56 missing rows → 0), declared-but-unfired iOS events went to zero, iOS event test coverage rose to 100%, and a live GA4 observability path shipped (MCP poll + SSE mirror + debug sink + analytics-watch CLI). It's a retroactive closure — the substance shipped across the PR roster and the one remaining task (D-2, GA4 dashboard conversion config) is operator-only and deferred to App Store launch.

iOS event test coverage
81%
Instrumentation shipped, but declared events went untested — no proof they fired.
iOS event test coverage
100%
112/112 events covered via 19 new XCTest methods; declared-but-unfired events 4 → 0.
What do T1 / T2 / T3 mean?▾
T1Instrumented — from a ledger/commit
T2Declared — stated, not measured
T3Narrative — estimate from memory
56 → 0 taxonomy drift · 81% → 100% event coverage · live GA4 path
The measurement-debt loop, finally closed — CSV ↔ enum ↔ code in sync, plus a live way to watch events land
Analytics was write-and-hope: a CSV missing ~56 rows, declared events that never fired, a disconnected GA4 MCP, and iOS events silently dropped by a plist target-membership bug. This feature brought the taxonomy back into sync and built a live observability path.

Problem

Before this feature, the project had shipped extensive analytics instrumentation but had no clean taxonomy and no live feedback loop. The taxonomy CSV was missing ~56 rows, several declared events never fired, the GA4 MCP was disconnected, and iOS analytics events were silently not firing because the analytics plist lacked correct target membership. There was no way to confirm events actually reached GA4 — instrumentation was write-and-hope. (T2 — declared from the feature's PR roster + 2026-05-13 session notes.)

What shipped (16 PRs, #332 → #388)

Phased across three tracks:

  • Taxonomy + instrumentation correctness: CSV taxonomy backfill (PR #334), delete the unfired ai_recommendation event (PR #335), a forward-declared-events convention (PR #336), spec extensions (PR #337), iOS analytics tests (PR #338), external sync status (PR #339), and the iOS analytics plist target-membership fix that made events actually fire on device (PR #388).
  • Observability tooling: a local SSE mirror (PR #342), the analytics-watch CLI (PR #345), a debug-sink adapter (PR #349), and live GA4 polling via the GA4 MCP (PR #351).
  • Live-connection fixes: a GA4 MCP env-var fix (PR #362) + a GA4 access-binding guide (PR #376).
  • Phase scaffolding + reconcile: Phase 1 PRD (PR #332), Phase 3A spec scaffold (PR #354), Phase 3A reconcile-complete (PR #358).

Outcome

The taxonomy is in sync and the observability path is live — analytics is no longer write-and-hope. (T2 — declared.)

  • Taxonomy drift: 56 missing CSV rows → 0; CSV ↔ AnalyticsEvent enum ↔ code in sync (T2).
  • iOS event test coverage: 81% → 100% (112/112 events) via 19 new XCTest methods (T1 — instrumented from the XCTest suite).
  • Declared-but-unfired iOS events: 4 → 0 (T2).
  • GA4 MCP connectivity: disconnected → connected (env-var + access-binding fixes), with the analytics-watch CLI + SSE mirror + debug-sink adapter operational (T2).

Honest framing

This is a retroactive closure: the feature shipped its substance across 16 merged PRs and sat in testing because the only remaining task is operator-gated. No formal PRD-defined primary metric was recorded in state.json at the time, so the metrics here are honestly tiered T2 (declared) rather than presented as instrumented outcomes.

The one open item — D-2 — is operator-only: configure GA4 conversions (workout_complete + nutrition_meal_logged) in the GA4 dashboard. No code; deferred to App Store launch because pre-launch TestFlight traffic is 100% same-day new-users (vanity numbers, not measurement). The Phase 1.B enforcement gates were folded into the v8.0 docket (F19/F20) rather than shipped in-feature, so the kill criterion's >5%-false-positive condition has no active gate to fire against — it was not triggered, and the hygiene + observability outcome shipped as planned.

Source case study (FitTracker2): docs/case-studies/analytics-observability-case-study.md.

Honest disclosures
  • •This is a retroactive closure — no formal PRD-defined primary metric existed in state.json at the time, so most headline numbers (taxonomy drift, declared-but-unfired events, GA4 connectivity) are honestly tiered T2 (declared), not T1-instrumented; the source is largely the PR roster + session notes.
  • •The live GA4 loop is not yet validated end-to-end: the one remaining task (D-2 — configuring GA4 conversions for workout_complete + nutrition_meal_logged) is operator-only and deferred to App Store launch, because pre-launch TestFlight traffic is 100% same-day new users (vanity numbers, not measurement).
  • •The Phase 1.B enforcement gates (CSV_TAXONOMY_DRIFT + GA4_MCP_DISCONNECTED) that this feature was scoped around were folded into the v8.0 docket (F19/F20) rather than shipped in-feature — so the taxonomy is cleaned but not yet mechanically gated against re-drifting.
Kill criterion · not fired
  • If the Phase 1.B taxonomy/connectivity gates (CSV_TAXONOMY_DRIFT + GA4_MCP_DISCONNECTED) produce >5% false-positive rate in their calibration window, defer gate enforcement to v7.10 and ship the hygiene-only outcome (taxonomy cleanup + instrumentation fixes retained regardless).