TVL Platform - API Key & Token Rotation Runbook

Summary

This runbook provides comprehensive procedures for rotating API keys, tokens, and credentials for all external services integrated with the TVL platform. Includes rotation schedules, step-by-step procedures, emergency rotation, and validation steps.

Table of Contents

• Rotation Schedule
• Pre-Rotation Checklist
• Service Rotation Procedures
  • Stripe API Keys
  • Hostaway API Keys
  • Google OAuth Credentials
  • SendGrid API Keys
  • AWS S3 Access Keys
  • Supabase Service Keys
  • Redis Passwords
• Emergency Rotation
• Post-Rotation Verification
• Incident Response

---

Rotation Schedule

Regular Rotation Schedule

Service	Rotation Frequency	Last Rotation	Next Rotation	Owner
Stripe API Keys	Every 90 days	2025-07-24	2025-10-24	Platform Team
Hostaway API Keys	Every 90 days	2025-08-15	2025-11-15	Integrations Team
Google OAuth Client Secret	Every 180 days	2025-04-01	2025-10-01	Platform Team
SendGrid API Keys	Every 90 days	2025-08-01	2025-11-01	Platform Team
AWS S3 Access Keys	Every 90 days	2025-07-10	2025-10-10	Platform Team
Supabase Service Keys	Every 180 days	2025-05-01	2025-11-01	Platform Team
Redis Passwords	Every 180 days	2025-06-01	2025-12-01	Platform Team
JWT Signing Keys	Every 365 days	2025-01-01	2026-01-01	Security Team

Rotation Triggers

Scheduled Rotation:

• 30 days before expiration: Create rotation ticket
• 7 days before expiration: Send reminder notification
• Day of expiration: Execute rotation
• 1 day after rotation: Verify and monitor

Emergency Rotation (within 4 hours):

• Key exposure detected (leaked in logs, GitHub, etc.)
• Unauthorized access detected
• Service compromise suspected
• Compliance audit requirement
• Employee offboarding with key access

---

Pre-Rotation Checklist

Planning

• Rotation ticket created in Linear
• Rotation scheduled during low-traffic window (10am-2pm EST weekday)
• On-call engineer notified and available
• Deployment freeze confirmed (no other deployments scheduled)
• Backup of current secrets documented

Preparation

• New key/credential generated (where applicable)
• Access to secret management systems confirmed:
  • Railway/Fly dashboard
  • Vercel dashboard
  • AWS Secrets Manager (if used)
  • 1Password/HashiCorp Vault (if used)
• Rollback plan documented
• Communication plan ready (#engineering-ops channel)

Validation

• Current services operational (no ongoing incidents)
• Monitoring dashboards green
• Recent backups confirmed
• Test environment rotation completed successfully

---

Service Rotation Procedures

Stripe API Keys

Service: Payment processing Key Type: Secret API keys (sk_live_xxx) Rotation Frequency: Every 90 days Downtime Risk: High (payments will fail if misconfigured)

Procedure

Step 1: Generate New API Key

# 1. Login to Stripe Dashboard
# https://dashboard.stripe.com/test/apikeys (staging)
# https://dashboard.stripe.com/apikeys (production)

# 2. Click "Create secret key"
# Name: "TVL API Key - 2025-10-24"
# Description: "Backend API server key, rotated 2025-10-24"

# 3. Copy new key (shown only once)
NEW_STRIPE_KEY="sk_live_xxxxxxxxxxxxx"

# 4. Store in secure location temporarily
echo $NEW_STRIPE_KEY > /tmp/stripe_key_new.txt
chmod 600 /tmp/stripe_key_new.txt

Step 2: Update Staging Environment

# Update Railway/Fly staging environment
railway variables set STRIPE_SECRET_KEY=$NEW_STRIPE_KEY --environment staging

# Verify staging
curl -X POST https://api.staging.tvl.app/api/v1/payment-intents \
  -H "Authorization: Bearer <test-token>" \
  -H "Content-Type: application/json" \
  -d '{
    "amount": 100,
    "currency": "usd",
    "booking_id": "<test-booking-id>"
  }'

# Expected: Payment intent created successfully

Step 3: Update Production Environment

# Backup current key
OLD_STRIPE_KEY=$(railway variables get STRIPE_SECRET_KEY --environment production)
echo $OLD_STRIPE_KEY > /tmp/stripe_key_old.txt

# Update production
railway variables set STRIPE_SECRET_KEY=$NEW_STRIPE_KEY --environment production

# Restart services (automatic in most cases)
railway restart --environment production

Step 4: Verify Production

# Wait 30 seconds for services to restart
sleep 30

# Health check
curl https://api.tvl.com/health | jq '.checks.stripe'
# Expected: "healthy"

# Test payment intent creation (use test booking)
# (Run smoke test script)
./scripts/smoke-test-payments.sh

Step 5: Monitor and Cleanup

# Monitor for 30 minutes
# - Check Sentry for payment errors
# - Check Grafana for payment success rate
# - Check Stripe webhook delivery

# If all successful, revoke old key
# Stripe Dashboard → API Keys → Reveal old key → Delete

# Clean up temporary files
rm /tmp/stripe_key_new.txt /tmp/stripe_key_old.txt

# Update rotation tracking
# Linear: Mark ticket complete
# Update rotation schedule spreadsheet

Rollback Procedure:

# If issues detected within 30 minutes
railway variables set STRIPE_SECRET_KEY=$OLD_STRIPE_KEY --environment production
railway restart --environment production

# Verify rollback
curl https://api.tvl.com/health | jq '.checks.stripe'

---

Hostaway API Keys

Service: PMS connector (property sync) Key Type: API Key + Account ID Rotation Frequency: Every 90 days Downtime Risk: Medium (sync delays, no immediate customer impact)

Procedure

Step 1: Generate New API Key

# 1. Login to Hostaway Dashboard
# https://dashboard.hostaway.com/api-keys

# 2. Create new API key
# Name: "TVL Integration - 2025-10-24"
# Permissions: Read/Write for Properties, Bookings, Calendars

# 3. Copy new key
NEW_HOSTAWAY_KEY="ha_xxxxxxxxxxxxx"
HOSTAWAY_ACCOUNT_ID="12345"

# 4. Store securely
echo $NEW_HOSTAWAY_KEY > /tmp/hostaway_key_new.txt
chmod 600 /tmp/hostaway_key_new.txt

Step 2: Update Staging

# Update environment variables
railway variables set HOSTAWAY_API_KEY=$NEW_HOSTAWAY_KEY --environment staging
railway variables set HOSTAWAY_ACCOUNT_ID=$HOSTAWAY_ACCOUNT_ID --environment staging

# Test connection
curl -X GET "https://api.hostaway.com/v1/properties" \
  -H "Authorization: Bearer $NEW_HOSTAWAY_KEY" \
  -H "X-Account-Id: $HOSTAWAY_ACCOUNT_ID"

# Expected: 200 OK with properties list

Step 3: Update Production

# Backup old key
OLD_HOSTAWAY_KEY=$(railway variables get HOSTAWAY_API_KEY --environment production)

# Update production
railway variables set HOSTAWAY_API_KEY=$NEW_HOSTAWAY_KEY --environment production

# Restart workers (sync workers use this key)
railway restart sync-worker --environment production

Step 4: Verify Sync Jobs

# Check sync worker logs
railway logs sync-worker --follow --environment production

# Verify sync jobs processing
redis-cli -u $REDIS_URL
> LLEN bull:sync-availability-job:active
> LLEN bull:sync-availability-job:failed

# Expected: Active jobs > 0, failed = 0

# Test manual sync
curl -X POST https://api.tvl.com/api/v1/connectors/hostaway/sync \
  -H "Authorization: Bearer <admin-token>" \
  -d '{"property_id": "<test-property-id>"}'

Step 5: Revoke Old Key

# After 1 hour of successful operation
# Hostaway Dashboard → API Keys → Delete old key

# Clean up
rm /tmp/hostaway_key_new.txt

# Update documentation
# Update connector configuration documentation

---

Google OAuth Credentials

Service: User authentication (Google OIDC) Key Type: Client ID + Client Secret Rotation Frequency: Every 180 days Downtime Risk: Critical (all user logins will fail)

Procedure

Step 1: Generate New OAuth Client

# 1. Go to Google Cloud Console
# https://console.cloud.google.com/apis/credentials

# 2. Select TVL Production project

# 3. Create new OAuth 2.0 Client ID
# Application type: Web application
# Name: "TVL Platform - 2025-10-24"
# Authorized redirect URIs:
#   - https://app.tvl.com/auth/callback
#   - https://api.tvl.com/auth/google/callback

# 4. Copy credentials
NEW_GOOGLE_CLIENT_ID="xxxxx.apps.googleusercontent.com"
NEW_GOOGLE_CLIENT_SECRET="GOCSPX-xxxxxxxxxxxxx"

# 5. Store securely
echo "CLIENT_ID=$NEW_GOOGLE_CLIENT_ID" > /tmp/google_oauth_new.txt
echo "CLIENT_SECRET=$NEW_GOOGLE_CLIENT_SECRET" >> /tmp/google_oauth_new.txt
chmod 600 /tmp/google_oauth_new.txt

Step 2: Update Supabase Auth Configuration

# 1. Login to Supabase Dashboard
# https://app.supabase.com/project/<project-id>/auth/providers

# 2. Navigate to: Authentication → Providers → Google

# 3. Update credentials:
# - Client ID: <NEW_GOOGLE_CLIENT_ID>
# - Client Secret: <NEW_GOOGLE_CLIENT_SECRET>

# 4. Click "Save"

# IMPORTANT: Keep old credentials active temporarily
# Google allows multiple OAuth clients simultaneously

Step 3: Update Frontend Environment Variables

# Update Vercel environment variables
vercel env add VITE_GOOGLE_CLIENT_ID production
# Paste: <NEW_GOOGLE_CLIENT_ID>

# Trigger redeployment
vercel --prod

# Wait for deployment
vercel inspect <deployment-url>

Step 4: Test Authentication

# Test login flow (manual testing required)
# 1. Open incognito browser window
# 2. Navigate to https://app.tvl.com
# 3. Click "Sign in with Google"
# 4. Complete OAuth flow
# 5. Verify successful login

# Check Supabase logs
# Supabase Dashboard → Logs → Auth logs
# Verify successful authentication events

Step 5: Revoke Old OAuth Client

# After 24 hours of successful operation
# Google Cloud Console → Credentials → Delete old OAuth client

# Clean up
rm /tmp/google_oauth_new.txt

# Update documentation

Rollback Procedure:

# If authentication fails:
# 1. Revert Supabase provider configuration to old credentials
# 2. Revert Vercel environment variable
# 3. Trigger frontend redeployment
# 4. Verify login works with old credentials

---

SendGrid API Keys

Service: Transactional email delivery Key Type: API Key Rotation Frequency: Every 90 days Downtime Risk: Medium (emails will fail to send)

Procedure

Step 1: Generate New API Key

# 1. Login to SendGrid Dashboard
# https://app.sendgrid.com/settings/api_keys

# 2. Create API Key
# Name: "TVL Platform - 2025-10-24"
# Permissions: Full Access (Mail Send)

# 3. Copy key (shown only once)
NEW_SENDGRID_KEY="SG.xxxxxxxxxxxxx"

echo $NEW_SENDGRID_KEY > /tmp/sendgrid_key_new.txt
chmod 600 /tmp/sendgrid_key_new.txt

Step 2: Update Environment Variables

# Update staging
railway variables set SENDGRID_API_KEY=$NEW_SENDGRID_KEY --environment staging

# Test email sending
curl -X POST https://api.staging.tvl.app/api/v1/test/send-email \
  -H "Authorization: Bearer <admin-token>" \
  -d '{
    "to": "test@tvl.com",
    "subject": "Test email - rotation validation",
    "body": "If you receive this, SendGrid rotation successful"
  }'

# Check email received
# Check SendGrid activity logs

# Update production
OLD_SENDGRID_KEY=$(railway variables get SENDGRID_API_KEY --environment production)
railway variables set SENDGRID_API_KEY=$NEW_SENDGRID_KEY --environment production

# Restart notification worker
railway restart notification-worker --environment production

Step 3: Verify Email Delivery

# Monitor notification worker logs
railway logs notification-worker --follow --environment production

# Send test email via production
# (Use admin panel to trigger test notification)

# Verify in SendGrid Dashboard
# Activity → Email Activity
# Check for successful deliveries with new API key

Step 4: Revoke Old Key

# After 1 hour of successful operation
# SendGrid Dashboard → API Keys → Delete old key

rm /tmp/sendgrid_key_new.txt

---

AWS S3 Access Keys

Service: Media storage (property images, documents) Key Type: Access Key ID + Secret Access Key Rotation Frequency: Every 90 days Downtime Risk: Medium (media uploads/downloads will fail)

Procedure

Step 1: Generate New Access Key

# 1. Login to AWS Console
# https://console.aws.amazon.com/iam/home#/users/tvl-platform-user

# 2. Select user: tvl-platform-user

# 3. Security credentials tab → Access keys → Create access key

# 4. Copy credentials
NEW_AWS_ACCESS_KEY_ID="AKIA..."
NEW_AWS_SECRET_ACCESS_KEY="xxxxxxxxxxxxx"

echo "AWS_ACCESS_KEY_ID=$NEW_AWS_ACCESS_KEY_ID" > /tmp/aws_keys_new.txt
echo "AWS_SECRET_ACCESS_KEY=$NEW_AWS_SECRET_ACCESS_KEY" >> /tmp/aws_keys_new.txt
chmod 600 /tmp/aws_keys_new.txt

Step 2: Update Environment Variables

# Update staging
railway variables set AWS_ACCESS_KEY_ID=$NEW_AWS_ACCESS_KEY_ID --environment staging
railway variables set AWS_SECRET_ACCESS_KEY=$NEW_AWS_SECRET_ACCESS_KEY --environment staging

# Test S3 access
aws s3 ls s3://tvl-staging-media --region us-east-1

# Update production
railway variables set AWS_ACCESS_KEY_ID=$NEW_AWS_ACCESS_KEY_ID --environment production
railway variables set AWS_SECRET_ACCESS_KEY=$NEW_AWS_SECRET_ACCESS_KEY --environment production

railway restart --environment production

Step 3: Verify S3 Operations

# Test upload
curl -X POST https://api.tvl.com/api/v1/properties/<property-id>/media \
  -H "Authorization: Bearer <test-token>" \
  -F "file=@test-image.jpg"

# Expected: 200 OK with media URL

# Test download
curl -I <returned-media-url>
# Expected: 200 OK

Step 4: Deactivate Old Key

# After 1 hour of successful operation
# AWS Console → IAM → Users → tvl-platform-user → Security credentials
# → Deactivate old access key
# (Keep deactivated for 24 hours before deleting)

# After 24 hours: Delete old access key

rm /tmp/aws_keys_new.txt

---

Supabase Service Keys

Service: Database access, admin operations Key Type: Service Role Key (JWT) Rotation Frequency: Every 180 days Downtime Risk: Critical (database operations will fail)

Procedure

Step 1: Generate New Service Key

# 1. Login to Supabase Dashboard
# https://app.supabase.com/project/<project-id>/settings/api

# 2. Navigate to: Settings → API

# 3. Click "Reset service_role key"
# WARNING: This invalidates the old key immediately

# 4. Copy new service_role key
NEW_SUPABASE_SERVICE_KEY="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."

echo $NEW_SUPABASE_SERVICE_KEY > /tmp/supabase_key_new.txt
chmod 600 /tmp/supabase_key_new.txt

Step 2: Update Environment Variables Immediately

# CRITICAL: Must update immediately as old key is now invalid

# Update staging
railway variables set SUPABASE_SERVICE_KEY=$NEW_SUPABASE_SERVICE_KEY --environment staging
railway restart --environment staging

# Verify staging works
curl https://api.staging.tvl.app/health | jq '.checks.database'
# Expected: "healthy"

# Update production
OLD_SUPABASE_KEY=$(railway variables get SUPABASE_SERVICE_KEY --environment production)
railway variables set SUPABASE_SERVICE_KEY=$NEW_SUPABASE_SERVICE_KEY --environment production
railway restart --environment production

Step 3: Verify Database Operations

# Health check
curl https://api.tvl.com/health | jq '.checks.database'

# Test database query
curl -X GET https://api.tvl.com/api/v1/properties \
  -H "Authorization: Bearer <test-token>"

# Check Supabase logs
# Supabase Dashboard → Logs → Postgres logs
# Verify no authentication errors

Step 4: Update Backup Scripts

# Update any scripts using service key
# Example: backup scripts, migration scripts, admin tools

# Update .env.production file
sed -i "s/SUPABASE_SERVICE_KEY=.*/SUPABASE_SERVICE_KEY=$NEW_SUPABASE_SERVICE_KEY/" .env.production

# Commit changes (encrypted)
git add .env.production.encrypted
git commit -m "chore: Rotate Supabase service key"
git push

Note: Supabase service key rotation is immediate and cannot be rolled back. Plan accordingly.

---

Redis Passwords

Service: Cache, job queue, session storage Key Type: Redis password Rotation Frequency: Every 180 days Downtime Risk: Critical (all caching and jobs will fail)

Procedure

Step 1: Generate New Password in Upstash

# 1. Login to Upstash Console
# https://console.upstash.com/redis/<database-id>

# 2. Navigate to: Details → Security

# 3. Click "Reset Password"
# Upstash will provide new connection string

# 4. Copy new Redis URL
NEW_REDIS_URL="rediss://default:xxxxxxxxxxxxx@us1-example.upstash.io:6379"

echo $NEW_REDIS_URL > /tmp/redis_url_new.txt
chmod 600 /tmp/redis_url_new.txt

Step 2: Update Staging

# Update staging environment
railway variables set REDIS_URL=$NEW_REDIS_URL --environment staging
railway restart --environment staging

# Wait for services to reconnect
sleep 30

# Test Redis connection
redis-cli -u $NEW_REDIS_URL ping
# Expected: PONG

# Test cache operations
curl https://api.staging.tvl.app/health | jq '.checks.redis'
# Expected: "healthy"

Step 3: Update Production (Zero-Downtime)

# Upstash supports both old and new passwords for a grace period
# Update production environment
OLD_REDIS_URL=$(railway variables get REDIS_URL --environment production)
railway variables set REDIS_URL=$NEW_REDIS_URL --environment production

# Restart services one by one (rolling restart)
railway restart web --environment production
sleep 30
railway restart sync-worker --environment production
sleep 30
railway restart payment-worker --environment production
sleep 30
railway restart notification-worker --environment production

Step 4: Verify All Services

# Health check
curl https://api.tvl.com/health | jq '.checks.redis'

# Check job processing
redis-cli -u $NEW_REDIS_URL
> INFO stats
> LLEN bull:sync-worker:active
> LLEN bull:sync-worker:failed

# Monitor for 30 minutes
# - Check Grafana for Redis connection errors
# - Check Sentry for cache errors
# - Verify job processing rates normal

Step 5: Finalize Rotation

# After 1 hour, old password is automatically invalidated by Upstash

rm /tmp/redis_url_new.txt

# Update documentation

---

Emergency Rotation

Immediate Response (Key Compromise)

Timeline: Complete rotation within 4 hours

Step 1: Assess Impact (0-15 minutes)

# Identify compromised key
COMPROMISED_SERVICE="<service-name>"  # e.g., "stripe", "hostaway"

# Check for unauthorized usage
# - Review service logs (Stripe dashboard, AWS CloudTrail, etc.)
# - Check for unusual API calls
# - Verify data access patterns

# Create incident ticket
gh issue create \
  --title "[SECURITY] Emergency key rotation - $COMPROMISED_SERVICE" \
  --label "security,incident,p0" \
  --body "Key compromise detected. Initiating emergency rotation."

Step 2: Disable Compromised Key (15-30 minutes)

# Immediately disable/revoke old key in service provider dashboard
# (Service-specific, see procedures above)

# Examples:
# - Stripe: Dashboard → API Keys → Delete key
# - AWS: IAM → Users → Deactivate access key
# - Google OAuth: Cloud Console → Credentials → Delete OAuth client

# Verify key revoked
# Test old key returns 401 Unauthorized

Step 3: Generate and Deploy New Key (30-90 minutes)

# Follow standard rotation procedure for service (see above)
# Accelerate timeline:
# - Skip staging testing (deploy directly to production)
# - Monitor closely during and after deployment

# Immediate deployment
railway variables set ${SERVICE}_KEY=$NEW_KEY --environment production
railway restart --environment production

# Verify immediately
./scripts/verify-deployment.sh

Step 4: Investigate and Contain (90-240 minutes)

# Identify compromise source
# - Check git history for leaked keys
# - Review log files for exposed credentials
# - Scan codebase for hardcoded secrets
# - Review recent deployments

# Remediation actions
# - Remove leaked secrets from Git history (git filter-branch or BFG)
# - Rotate all related credentials
# - Update security policies
# - Add secret scanning to CI/CD

# Example: Remove secrets from Git
git filter-branch --force --index-filter \
  "git rm --cached --ignore-unmatch .env" \
  --prune-empty --tag-name-filter cat -- --all

git push --force --all

Step 5: Post-Incident (After 4 hours)

# Create post-mortem
# - Root cause analysis
# - Timeline of events
# - Remediation actions taken
# - Preventive measures

# Update runbooks and procedures
# Conduct security training
# Implement automated secret detection (e.g., GitGuardian, TruffleHog)

---

Post-Rotation Verification

Verification Checklist

Immediate (within 15 minutes):

• Health check endpoint returns healthy status
• Service-specific functionality tested
• No errors in Sentry
• No failed jobs in queue
• External service connectivity verified

Short-term (within 1 hour):

• Monitor error rates (should be < 1%)
• Check Grafana for anomalies
• Verify scheduled jobs running
• Test end-to-end user flows
• Check customer support channels

Long-term (within 24 hours):

• Review error trends
• Verify data consistency
• Update rotation tracking spreadsheet
• Document any issues encountered
• Update runbook with improvements

Automated Verification Script

#!/bin/bash
# scripts/verify-key-rotation.sh

SERVICE=$1  # e.g., "stripe", "hostaway", "sendgrid"
API_URL="https://api.tvl.com"

echo "=== Verifying $SERVICE Key Rotation ==="

case $SERVICE in
  stripe)
    echo "Testing Stripe connectivity..."
    RESPONSE=$(curl -s $API_URL/health | jq -r '.checks.stripe')
    if [ "$RESPONSE" = "healthy" ]; then
      echo "✓ Stripe: OK"
    else
      echo "✗ Stripe: FAILED"
      exit 1
    fi
    ;;

  hostaway)
    echo "Testing Hostaway connectivity..."
    RESPONSE=$(curl -s $API_URL/health | jq -r '.checks.hostaway')
    if [ "$RESPONSE" = "healthy" ]; then
      echo "✓ Hostaway: OK"
    else
      echo "✗ Hostaway: FAILED"
      exit 1
    fi
    ;;

  sendgrid)
    echo "Testing SendGrid connectivity..."
    # Send test email
    curl -X POST $API_URL/api/v1/test/send-email \
      -H "Authorization: Bearer $ADMIN_TOKEN" \
      -d '{
        "to": "ops@tvl.com",
        "subject": "SendGrid rotation test",
        "body": "Key rotation successful"
      }'
    echo "✓ SendGrid: Test email sent"
    ;;

  *)
    echo "Unknown service: $SERVICE"
    exit 1
    ;;
esac

echo ""
echo "✓ Rotation verification complete"

---

Incident Response

Key Exposure Scenarios

Scenario 1: Key Leaked in Git Repository

# 1. Revoke key immediately (see Emergency Rotation)

# 2. Remove from Git history
bfg --replace-text secrets.txt  # Create secrets.txt with exposed key
git reflog expire --expire=now --all
git gc --prune=now --aggressive

# 3. Force push (coordinate with team)
git push --force --all

# 4. Rotate key
# (Follow standard rotation procedure)

Scenario 2: Key Exposed in Logs

# 1. Identify exposure scope
# - Which log files?
# - Which time range?
# - Who has access to logs?

# 2. Revoke key immediately

# 3. Redact from logs
# Railway: Cannot delete logs, rotate immediately
# CloudWatch: Delete log streams (if applicable)

# 4. Update logging configuration to mask secrets
# Example: Add secret masking to logger

Scenario 3: Unauthorized API Usage Detected

# 1. Review service audit logs
# Stripe: Dashboard → Logs → API logs
# AWS: CloudTrail → Event history

# 2. Identify unauthorized calls
# - Source IP addresses
# - API endpoints called
# - Time range

# 3. Revoke key immediately

# 4. Implement IP allowlisting (if supported)
# AWS: Add IP restrictions to IAM policy
# Stripe: Use IP allowlists for restricted keys

# 5. Alert security team

Escalation

Severity	Response Time	Escalation Path
P0 - Key actively exploited	Immediate	Security Team → CTO → Legal
P1 - Key exposed but no exploitation	4 hours	Platform Team → Security Team
P2 - Scheduled rotation overdue	24 hours	Platform Team

Communication Templates

Key Exposure Notification:

Subject: [SECURITY] API Key Rotation Required - <Service>

Team,

We have detected potential exposure of our <Service> API key.

Impact: <brief description>
Timeline: Rotation in progress, completion ETA <time>
Action Required: None (ops team handling)

Status updates will be posted to #incidents.

On-call: <name>

---

Sources

• /mnt/c/GitHub/claude-test/docs/01-architecture/deployment-views.md
• /mnt/c/GitHub/claude-test/docs/00-overview/platform-overview.md
• Security best practices (OWASP, NIST)