Analytics & Audit - Version Mapping
Domain: Analytics & Audit Domain Number: 12 First Introduced: MVP.0 Status: Specification Complete Last Updated: 2025-10-25
Overview
This document maps Analytics & Audit domain features and entities to specific MVP and version releases. It provides a quick reference for understanding when telemetry, audit, and analytics capabilities were introduced and how they evolve from compliance-focused audit trails to comprehensive business intelligence.
Version Timeline
| Version | Status | Key Features |
|---|---|---|
| MVP.0 | Included | Audit events, compliance trail, basic logging |
| MVP.1 | Enhanced | Expanded event types, retention policies |
| V1.0 | Planned | Full event stream, analytics snapshots, reports |
| V1.1 | Future | Advanced analytics, BI integration, scheduled reports |
| V2.0 | Future | Machine learning, predictive analytics |
| V2.1 | Future | Data warehouse integration, federated analytics |
Entity Version Mapping
MVP.0 Entities
| Entity | Table Name | Purpose | Status |
|---|---|---|---|
| AuditEvent | audit_events | Compliance-grade audit trail | ✅ Included |
V1.0 Entities
| Entity | Table Name | Purpose | Status |
|---|---|---|---|
| Event | events | Generic system event log | 📋 Planned |
| AnalyticsSnapshot | analytics_snapshots | Pre-computed metrics | 📋 Planned |
| Report | reports | Saved report configurations | 📋 Planned |
V1.1 Entities
| Entity | Table Name | Purpose | Status |
|---|---|---|---|
| ReportSchedule | report_schedules | Scheduled report generation | 📋 Future |
| MetricDefinition | metric_definitions | User-defined metrics | 📋 Future |
V2.0 Entities
| Entity | Table Name | Purpose | Status |
|---|---|---|---|
| PredictiveModel | predictive_models | ML model configurations | 📋 Future |
| AnomalyDetection | anomaly_detections | Detected anomalies | 📋 Future |
Feature Version Mapping
MVP.0 Features (Included)
Audit Trail:
- ✅ AuditEvent entity with HMAC signatures
- ✅ Time-based monthly partitioning
- ✅ Immutable append-only logs
- ✅ Before/after state snapshots
- ✅ IP address and user agent tracking
- ✅ Session context tracking
- ✅ 7-year minimum retention for compliance
Event Coverage:
- ✅ Booking lifecycle events (created, confirmed, cancelled)
- ✅ Payment transaction events (succeeded, failed, refunded)
- ✅ Space/Unit CRUD operations
- ✅ Channel sync operations (outbound success/failure)
- ✅ iCal import events
- ✅ Authorization changes (role assignments, permission updates)
Tamper Detection:
- ✅ HMAC-SHA256 signature generation
- ✅ Signature verification on read
- ✅ Organization-specific signing keys
- ✅ Quarterly key rotation
Query Support:
- ✅ Manual SQL queries for audit reporting
- ✅ Indexes on org_id, user_id, resource_type, action
- ✅ Time-range queries optimized via partitioning
Compliance:
- ✅ SOC2 audit trail requirements
- ✅ GDPR right to erasure logging (redaction tracking)
- ✅ PCI-DSS payment audit logs
- ✅ Permanent retention (7+ years)
Retention:
- ✅ 12 months for operational events
- ✅ 7+ years for audit_events
- ✅ Partition archival to cold storage (2+ years)
V1.0 Features (Planned)
Generic Event Stream:
- 📋 Event entity for all domain actions
- 📋 Event-driven architecture with pub/sub
- 📋 Distributed tracing correlation (trace_id)
- 📋 Fan-out to multiple subscribers
- 📋 At-least-once delivery guarantee
- 📋 Dead-letter queue for failed writes
Pre-Computed Analytics:
- 📋 AnalyticsSnapshot entity
- 📋 Scheduled aggregation jobs (hourly, nightly)
- 📋 Pre-defined metrics:
- Bookings per day/week/month
- Gross revenue by channel/region
- Occupancy rate
- Average booking value
- Conversion funnel metrics
- 📋 Dimensional breakdowns (channel, region, space_type)
- 📋 Fast dashboard queries (no expensive aggregations)
Saved Reports:
- 📋 Report entity for user-defined configurations
- 📋 Metric selection and filter configuration
- 📋 Visibility controls (private, org, account)
- 📋 Report status management (active, paused, archived)
Cross-Domain Event Correlation:
- 📋 trace_id propagation across services
- 📋 Distributed operation reconstruction
- 📋 End-to-end request flow analysis
Observability:
- 📋 Real-time operations dashboards
- 📋 Booking funnel analysis
- 📋 Revenue trend reporting
- 📋 Performance monitoring (API latency, job duration)
V1.1 Features (Future)
Scheduled Reports:
- 📋 Cron-based report generation
- 📋 CSV/PDF export formats
- 📋 Email delivery
- 📋 S3 storage for large exports
BI Tool Integration:
- 📋 Looker connector
- 📋 Tableau integration
- 📋 Metabase support
- 📋 SQL query interface
Custom Metrics:
- 📋 User-defined aggregation rules
- 📋 Calculated metrics (ratios, percentages)
- 📋 Metric versioning and history
Anomaly Detection:
- 📋 Automated anomaly alerts
- 📋 Booking drop detection
- 📋 Payment failure spike alerts
- 📋 Unusual access pattern detection
V2.0 Features (Future)
Machine Learning:
- 📋 Predictive occupancy forecasting
- 📋 Revenue forecasting models
- 📋 Cancellation risk scoring
- 📋 Dynamic pricing recommendations
- 📋 Fraud detection models
Advanced Analytics:
- 📋 Recommendation engine ("users who viewed this...")
- 📋 Seasonal demand predictions
- 📋 Market trend analysis
- 📋 A/B test result tracking
V2.1 Features (Future)
Data Warehouse:
- 📋 Streaming ETL to BigQuery/Snowflake/Redshift
- 📋 Event replay for backfill
- 📋 Historical analytics on archived data
- 📋 Tiered storage (hot/warm/cold)
Federated Analytics:
- 📋 Cross-org analytics (white-label marketplace)
- 📋 Partner analytics dashboards
- 📋 Aggregated platform-wide metrics
- 📋 Privacy-preserving aggregation
Workflow Version Mapping
| Workflow | MVP.0 | V1.0 | V1.1 | V2.0 | V2.1 |
|---|---|---|---|---|---|
| Record System Event | ✅ Audit only | ✅ Full stream | ✅ | ✅ | ✅ |
| Create Audit Trail Entry | ✅ | ✅ | ✅ | ✅ | ✅ |
| Generate Analytics Snapshot | ❌ | ✅ | ✅ | ✅ ML-enhanced | ✅ |
| Query Audit Trail | ✅ SQL | ✅ API | ✅ Reports | ✅ | ✅ |
| Scheduled Report Generation | ❌ | ❌ | ✅ | ✅ | ✅ |
| Anomaly Detection | ❌ | ❌ | ✅ Basic | ✅ ML-based | ✅ |
Dependency Version Mapping
MVP.0 Dependencies
Required Domains:
- ✅ Identity & Tenancy (MVP.0) - User, Org, Account for actor tracking
Sourced From:
- ✅ ALL domains (MVP.0+) - Events from every domain action
Depended On By:
- ✅ Authorization & Access (MVP.0) - Audit logs for access control changes
- ✅ Channels & Distribution (MVP.0) - Sync event logging
- ✅ Payments & Financials (MVP.0) - Financial audit trail
Related:
- 🔄 System Architecture (V1.0) - Event bus infrastructure
- 🔄 Delegation & Collaboration (V1.1) - Cross-org audit trails
Technical Implementation Timeline
MVP.0 Implementation
Database:
- ✅ 1 table: audit_events (partitioned by month)
- ✅ 4+ indexes for performance
- ✅ Constraints for data integrity
- ✅ Partition maintenance automation
Partitioning:
- ✅ Monthly partitions: audit_events_YYYY_MM
- ✅ Automatic partition creation (3 months ahead)
- ✅ Retention policy: 7+ years minimum
- ✅ Archival to cold storage after 2 years
Signature Generation:
- ✅ HMAC-SHA256 over: org_id|user_id|action|resource_type|resource_id|timestamp|payload_hash
- ✅ Organization-specific signing keys
- ✅ Quarterly key rotation schedule
Event Coverage:
- ✅ Booking: created, confirmed, cancelled
- ✅ Payment: succeeded, failed, refunded
- ✅ Space/Unit: created, updated, deleted
- ✅ Channel sync: success, failure, retry
- ✅ Authorization: role_assigned, permission_granted
Query Support:
- ✅ Manual SQL for audit reports
- ✅ Indexes: (org_id, occurred_at), (resource_type, resource_id), (user_id, occurred_at), (action, occurred_at)
V1.0 Implementation
Database:
- 📋 3 new tables: events, analytics_snapshots, reports
- 📋 12+ additional indexes
- 📋 Constraint additions
Event Stream:
- 📋 Redis Streams or Apache Kafka for pub/sub
- 📋 Event producer in each domain service
- 📋 Event consumer workers for analytics
- 📋 Dead-letter queue for failed processing
Aggregation Jobs:
- 📋 Hourly aggregation for day metrics
- 📋 Nightly aggregation for week/month metrics
- 📋 Checkpoint management for incremental processing
- 📋 Idempotent aggregation logic
API Endpoints:
- 📋 GET /api/analytics/snapshots - Query pre-computed metrics
- 📋 POST /api/analytics/reports - Create saved report
- 📋 GET /api/analytics/reports/:id - Get report data
- 📋 GET /api/audit/events - Query audit trail (with filters)
Dashboard:
- 📋 Real-time operations dashboard
- 📋 Revenue trends charts
- 📋 Booking funnel visualization
- 📋 Performance monitoring graphs
Business Rules by Version
MVP.0 Rules
- ✅ Append-Only Immutability: Never update or delete audit events
- ✅ Time-Based Partitioning: Monthly partitions for compliance queries
- ✅ Signature Verification: All AuditEvent signatures verified on read
- ✅ Retention Policies: 7+ years minimum for audit_events
- ✅ Event Namespace: domain.action format (booking.created, etc.)
- ✅ Org Isolation: All queries filtered by org_id
- ✅ GDPR Redaction: User data masked with [REDACTED] marker
V1.0 Rules
- 📋 Trace ID Propagation: All related events share trace_id
- 📋 At-Least-Once Delivery: Events retried from dead-letter queue
- 📋 Compensating Events: Corrections via new events, not updates
- 📋 Snapshot Uniqueness: Unique (org_id, metric_key, time_bucket, dimensions_hash)
Performance Targets by Version
MVP.0 Targets
| Metric | Target | Measurement |
|---|---|---|
| Audit write latency | < 50ms | p95 insert time |
| Audit query (90 days) | < 2s | Typical audit trail query |
| Signature verification | < 10ms | Per record verification |
| Partition creation | Automated | 3 months ahead |
| Failed writes | 0% | Must not lose audit events |
V1.0 Targets
| Metric | Target | Status |
|---|---|---|
| Event write latency | < 100ms | 📋 Async pub/sub |
| Snapshot query latency | < 500ms | 📋 Pre-computed metrics |
| Event throughput | > 10,000/sec | 📋 Peak event rate |
| Aggregation lag | < 1 hour | 📋 Hourly jobs |
| Dashboard load time | < 2s | 📋 Cached snapshots |
Migration Notes
From Pre-MVP to MVP.0
New Capabilities:
- Compliance-grade audit trail
- Tamper-evident logging
- Long-term retention
Breaking Changes:
- None (greenfield feature)
Initial Setup:
- Deploy audit_events table with partitioning
- Create initial 3-month partition set
- Configure signing key per organization
- Enable audit logging in all domain services
- Verify signature generation and verification
- Set up partition maintenance cron job
- Configure cold storage archival process
Data Validation:
- Verify all sensitive operations create audit events
- Test signature verification on sample records
- Validate partition creation automation
- Confirm retention policies applied correctly
From MVP.0 to V1.0
New Capabilities:
- Generic event stream
- Pre-computed analytics
- Saved reports
- Real-time dashboards
Breaking Changes:
- None (additive only)
Migration Steps:
- Deploy new tables: events, analytics_snapshots, reports
- Set up event stream infrastructure (Redis Streams/Kafka)
- Enable event producers in domain services
- Deploy aggregation worker jobs
- Perform initial backfill of analytics_snapshots from audit_events
- Deploy analytics API endpoints
- Build and deploy dashboard UI
- Migrate existing manual queries to saved reports
Backfill Strategy:
- Historical snapshots generated from audit_events
- Start with last 12 months of data
- Run backfill during low-traffic period
- Validate snapshot accuracy against raw events
From V1.0 to V1.1
New Capabilities:
- Scheduled reports
- BI tool integration
- Custom metrics
- Basic anomaly detection
Breaking Changes:
- None (additive only)
Migration Steps:
- Deploy report_schedules and metric_definitions tables
- Enable scheduled report generation workers
- Configure email delivery service
- Set up BI tool connectors
- Create initial set of standard metrics
- Train users on custom metric creation
Related Documents
- Analytics & Audit Specification - Complete domain spec
- Identity & Tenancy Specification - User/Org context
- System Architecture Specification - Event bus infrastructure
- Compliance Guide - SOC2/GDPR requirements
- Platform Specification - Analytics & Audit section
- MVP.0 Overview
- V1 Vision
- Roadmap
Glossary
- AuditEvent: Compliance-grade, tamper-evident record of sensitive state changes
- Event: Generic system or user action log for telemetry
- AnalyticsSnapshot: Pre-computed metric for fast dashboard queries
- HMAC Signature: Hash-based message authentication code for tamper detection
- Partition: Time-based table subdivision (monthly for audit_events)
- Trace ID: Distributed operation correlation identifier
- Compensating Event: New event recording a correction (not an update)
- Dead-Letter Queue: Failed event storage for retry processing
- Cold Storage: Long-term archival (S3 Glacier, etc.) for old partitions
- Backfill: Populating historical analytics from past events
- Retention Policy: Rules for how long data is kept before archival/deletion
- Redaction: Masking PII while preserving audit trail (GDPR right to erasure)