Skip to main content

Analytics architecture

Scope: the eight read-only reporting endpoints under /portal/analytics, their window/bucketing math, and the three portal tabs' consumption of them. Not the write-side data layers that feed the tables, and not the Home dashboard's use of overview (→ Home dashboard architecture).

This page explains the analytics subsystem: how the eight reporting endpoints aggregate org-scoped data, the window and bucketing math, how the three portal tabs consume them, and the gotchas that have bitten tests and audits. The consumer pages are covered in Home dashboard architecture (pulse reuses overview) and the route/shell context in Architecture overview. A template-attribution probe is in Link templates — Runtime verification; the UX fix-pass record lives in docs/FLOW-005.md.

Source-checked against optolink-backend @ 29f8589519ac85445ed0598c1b9cfc125a9dd6f2 and optolink-portal @ 7d31d457ad4e02e6d9b6bd3cb84a47439f2903ea, 2026-10-07 (B-015 re-verification pass — src/analytics/ and the controller are byte-unchanged since the FLOW-005 harvest @ 583da0c; portal unchanged since 2026-09-15). Facts are code-read plus the 2026-09-07/08 live records; nothing was re-run live (report §11).

Module layout​

  • Backend: src/analytics/analytics.service.ts (all aggregation; exports CLICK_DIMENSIONS (11) and TREND_METRICS (clicks, matches)), src/analytics/analytics.module.ts, unit tests in analytics.service.spec.ts (29 tests), e2e in test/analytics.e2e-spec.ts (30 tests).
  • Backend routes: src/portal/portal-analytics.controller.ts: 8 GETs under /portal/analytics, inline DTOs, guard stack ClerkAuthGuard → RolesGuard → RateLimitGuard, @RateLimit(RateLimitPresets.portal) per route, no @Roles anywhere (controller comment: "Readable by every org role").
  • Frontend pages: src/app/(app)/analytics/page.tsx, breakdowns/page.tsx, events/page.tsx; /analytics/templates is a loader redirect in src/app/routes.tsx → /analytics/breakdowns?by=template (params preserved).
  • Frontend plumbing: src/hooks/use-analytics.ts, components in src/components/analytics/ (nav, range selector, granularity toggle, delta-chip, trend cards, export-csv-button), src/lib/csv.ts, src/lib/constants.ts, dimension list in src/types/api.ts.

No domain events, no crons, no webhooks participate; this is pure read aggregation over tables written elsewhere (click-tracking/ → ClickEvent, resolution/ → LinkMatch, SDK session → DeviceProfile, app-event/ → AppEvent).

The org-scoping invariant​

Every method is org-scoped by construction: orgId arrives from the Clerk session via @OrgFromUser(OrgResolutionPipe), never from request input, and every Prisma query filters on organizationId/orgId (analytics.service.ts class doc). There is no code path where a client-supplied org id reaches an aggregation query. Nothing here touches EntitlementsService: analytics is deliberately not plan- or quota-gated (the analytics_export plan feature exists in plan_config but gates no endpoint).

Endpoints​

GET pathQuery paramsNotes
/portal/analytics/overviewdays 1–365 (default 30), offsetDays 0–365 (default 0)Composes everything the Overview tab needs (below)
/portal/analytics/clicksby required (one of 11 dimensions), days, limit 1–100 (default 10; portal sends 20)Breakdown rows; by=template returns the template shape
/portal/analytics/funneldaysAlso embedded in overview
/portal/analytics/maudaysAlso embedded in overview
/portal/analytics/mau/trenddays, bucket=day|weekRaw-SQL bucketed, zero-filled
/portal/analytics/trendmetrics (subset of clicks,matches), days, bucketRaw-SQL bucketed, zero-filled
/portal/analytics/templatesdaysZero-filled per-template performance
/portal/analytics/eventsdays, limit 1–100 (default 10)topEvents

Validation is strict: every out-of-range param is a 400 (days/limit bounds, by not in the dimension set, bucket not day|week, unknown metrics via @IsIn(each: true)). There is no silent server-side fallback; the silent fallbacks live in the frontend hooks (below).

Frontend-orphaned endpoints, all live backend-side: /funnel, /mau, /templates (overview composes funnel+mau; template aggregates were folded into clicks?by=template), plus the legacy GET /portal/dashboard → AnalyticsService.topLinks (no frontend caller). Their disposition (public API vs delete) is an open flag in the FLOW-005 report §11; do not build new frontend paths on them.

Window math and the delta mechanism​

Every query takes an inclusive from (now − days·24h) and optional exclusive to via rangeFilter(from, to). The delta feature rides on one extra parameter: overview?offsetDays=N (0–365) ends the window N days ago, i.e. [now−(days+N)·24h, now−N·24h), while offsetDays=0/absent keeps the legacy open-ended window byte-identical (portal-analytics.controller.ts overview() comment + rangeFilter).

The client exploits this by issuing overview twice per Overview mount: current window, then a second query with offsetDays=days for the previous window (use-analytics.ts). The hook appends &offsetDays= only when > 0, keeping the main request identical to the pre-delta call. DeltaChip renders ▲/▼ N% vs prev N days with the edge cases: prev=0 & current>0 → "New"; current=0 & prev>0 → "▼ -100%"; both 0 → "—"; chips hidden while the previous-window query loads or errors (the page's own error state stays keyed to the main query). MAU deltas compare distinct counts: prev+current don't partition the whole; documented, user-approved. The red ▼ path is unreachable with the current seed (unit-verified only).

Overview composition​

overview() (analytics.service.ts:567-606) is one Promise.all, pure Prisma, no Redis:

  • clicks: total, bot count, topCountries/topLinks (fixed 5 — topLinks(orgId, from, 5))
  • installs.total: DeviceProfile.count(firstSeenAt in window), i.e. first-seen device = install (SDK mints a fresh deviceId per install; reinstalls count). Backed by @@index([orgId, firstSeenAt]), migration 20260908071324_add_device_profile_first_seen_index (FLOW-005 D1 rev).
  • matches: total, matched, unmatched, matchedRate (division-guarded to 0 when no attempts), byMethod, byConfidence
  • funnel: clicks, matches, devices, identifiedUsers
  • mau: total, named, anonymous

Funnel stages 3–4 are presence-based: devices = those with lastSeenAt in range (devices arrive on every SDK touchpoint, not only via matched links); identified = those with non-null activeEndUserId. MAU formula (locked P6-002): COUNT(DISTINCT COALESCE(activeEndUserId, deviceId)) over devices with lastSeenAt ≥ from.

The dead byMethod field​

byMethod (raw MatchMethod enum keys: CLIPBOARD, INSTALL_REFERRER, FINGERPRINT_EXACT, FINGERPRINT_SCORED, IP_FUZZY, DIRECT, NONE, prisma/schema.prisma:76-84) is still in the response but rendered nowhere. Portal commit 5e60cc1 (2026-09-09) replaced match-method display with confidence tiers, and constants.ts now carries only MATCH_CONFIDENCE_LABELS/_TIERS (a stale orphan comment for MATCH_METHOD_LABELS remains at constants.ts:7-9). The UI shows byConfidence, whose string keys ("exact"|"high"|"medium"|"low"|"none") are written by resolution/match.controller.ts. Caution: LinkMatch.confidence is a free String (schema.prisma:462), not a Prisma enum, so an off-tier value would be silently dropped by MatchesByConfidenceList (which filters to known tiers). Whether any writer emits off-tier values is unverified (report §11).

Breakdown mechanics​

clickBreakdown(by, …) dispatches per dimension:

  • 8 direct columns (country/os/device/browser/utmSource/utmMedium/utmCampaign/referrer): one Prisma groupBy on ClickEvent.
  • channel/link: groupBy on linkId, then regrouped through the org's Link rows to readable keys (channel name / shortCode).
  • template: wraps templatePerformance.

All breakdown responses carry total and uniqueCount (distinct values pre-slice) so clients can label truncation honestly ("Top N of M · total" vs "All rows (N total)"); pct is rounded to 3 decimals. Chart slices top 10 (title appends "· top 10"); the breakdown table fetches limit=20; events fetch 10; overview tops are fixed 5.

by=template zero-fills across all org templates; system templates appear only when ≥1 org link references them (no zero-fill of the 7 sys-* starters — after attaching sys-email-to-app, seeded clicks attribute to that one system-template row; the other 6 stay absent). Per-link template overrides don't affect aggregation; the binding is the Link FK. total/uniqueCount here cover template-attributed clicks only (clicks on untemplated links have no row). templatePerformance computes conversions = AppEvents in range from devices holding a matched LinkMatch on that template's links. ?focus=<templateId> is a pure client-side row highlight (◆ + caption); it is reachable only by manual URL since the templates-list Performance link was dropped with the old page (report §11 disposition open).

Trend bucketing SQL​

Both /mau/trend and /trend run one raw SQL statement each, because Prisma groupBy can't date_trunc and JS bucketing would pull every row into Node. Shape: date_trunc in UTC (day → UTC midnight, week → ISO Monday), a CTE unioning AppEvent ∪ LinkMatch (MAU) or ClickEvent ∪ matched LinkMatch (trend), results zero-filled into a continuous bucket series, and the window start widened to a bucket boundary (truncateBucket). P7-005 measured 366 daily buckets at 0.74s.

"Active in bucket" = the device recorded an AppEvent or produced a LinkMatch in that bucket. Session-only opens that write no row are invisible; this is a user-approved P7-005 decision, and the rejected alternative (bucketing on lastSeenAt) produced churn-shaped charts. This caveat is documented on the endpoint Swagger. Trend matches counts LinkMatch.matched=true, the same number as the Overview Matches KPI and funnel stage.

Events​

topEvents groupBys AppEvent.eventName, sorts desc, slices to limit. It ignores any upper bound: open-ended from only, no to, no offsetDays, and the DTO doesn't accept one. pct is computed against the total of all event names before slicing, so the table's share column can legitimately sum to under 100% under truncation. AppEvent ingestion itself (eventName 1–100 chars, properties ≤10 KiB → explicit 400, unknown deviceId → 404, sdkEvents 600/min per key) is P6-004 territory; cross-link, don't duplicate.

Frontend plumbing​

  • URL state: useRangeDays (?range=, default 30, validated against RANGE_OPTIONS), useBreakdownBy (?by=, fallback country), useMauBucket (?bucket=, fallback day). Invalid values fall back silently; client behavior diverges from the server's 400s (recorded non-blocker). analytics-nav.tsx forwards all search params between tabs, so range survives tab switches.
  • Query keys: ["analytics", <surface>, days, offsetDays|bucket|by|limit], all hooks staleTime: 60_000 vs the app-global client's staleTime: 30s, refetchOnWindowFocus: false (src/lib/query-client.ts). Consequence: a warm-cache refetch failure can be masked until a fresh mount (FLOW-005 non-blocker).
  • Errors: every tab renders the shared QueryErrorBanner + Retry (FB-1); Overview's previous-window failure hides chips only.
  • CSV: src/lib/csv.ts (buildCsv/csvFilename/downloadCsvFile): client-side, CRLF, quote-escaping, exports exactly the visible post-limit rows; filename optolink-<table>-<dimension>-<utc-window>.csv. Buttons render only in the loaded, non-empty table branch.
  • Mount-time calls: overview ×2 per Overview mount; ×2 on Home pulse with days=7 (see Home dashboard architecture).

Rate limits, env, roles​

  • Rate limit: RateLimitPresets.portal = 30 req / 1s window, Redis keyPrefix rl:portal, keyed per apiKeyId or request identity/IP (rate-limit.decorator.ts:23, rate-limit.guard.ts resolveKey); 429 body { error: 'Too Many Requests', message: 'Rate limit exceeded. Retry after N second(s).' }. E2e escape hatch: RATE_LIMIT_DISABLED=true; Redis-backed counters accumulate across suites, which is why e2e sets it (guard comment).
  • Env: no analytics-specific env vars or config keys exist. The only schema change in the flow's history is the D1-rev DeviceProfile index migration, which carries no env.

Gotchas​

  • Seed dependency: meaningful numbers need pnpm prisma db seed (demo org: 5 links, 200 clicks (~5% bots), matches = 40% of non-bot clicks (≈76; random-derived, varies per run), 12 AppEvents, org template "Summer Campaign" — which has no attached links; the two template-bearing links welcome1/shoesale attach to the sys-email-to-app system starter. 91/200 clicks landing in template analytics was the P6-005 seed run's count — per-run numbers drift with the randomness). Zero-data orgs (QA Shell) render the dedicated empty states on every tab.
  • E2e fixture style: test/analytics.e2e-spec.ts builds its own org/template/links/devices with explicit firstSeenAt 1/3/10 days ago and asserts the additive-partition property (prev + current = whole). Reuse this fixture style for new aggregations.
  • Dev cold load to first data is 5–8s (Clerk handshake → onboarding/state 401→retry→200 → guard → page queries); one console 401 per hard reload self-heals. Dev-only.
  • Network capture in dev: raise the Vite resource-timing buffer first: performance.setResourceTimingBufferSize(2000) (FLOW-005 re-run caveat).
  • CSV reconciliation: CSV exports reflect visible truncated rows only; don't compare CSV sums against /overview totals when a dimension has >20 values.
  • UTC everywhere: day buckets are UTC midnights, weeks ISO Mondays, and the on-screen range label is computed client-side in UTC (src/lib/utils.ts utcRangeLabel); a user-local reading of "today" will not match the buckets.
  • History caveat: docs/FLOW-005.md Rounds 1–8 (2026-09-08) predate the confidence-tier swap; its R2 "match-method labels" description is superseded by 5e60cc1 (2026-09-09). Trust code over that record; this page is built from code at the SHAs above.