slight update

.kiro/specs/email-tracking/tasks.md

@@ -0,0 +1,170 @@
+# Implementation Plan
+
+## Phase 1: Core Tracking Infrastructure
+
+- [x] 1. Set up database collections and indexes
+  - Create MongoDB collections: `newsletter_sends`, `link_clicks`, `subscriber_activity`
+  - Add indexes for performance: `tracking_id` (unique), `newsletter_id`, `subscriber_email`, `sent_at`
+  - Write database initialization script
+  - _Requirements: 1.3, 2.4_
+
+- [x] 2. Implement tracking service
+- [x] 2.1 Create tracking ID generator
+  - Write `generate_tracking_id()` function using UUID4
+  - Write `create_newsletter_tracking()` function to create tracking records
+  - Add configuration for tracking enable/disable
+  - _Requirements: 1.1, 2.1_
+
+- [x] 2.2 Implement tracking pixel endpoint
+  - Create Flask route `GET /api/track/pixel/<tracking_id>`
+  - Generate 1x1 transparent PNG (base64 encoded)
+  - Log email open event to `newsletter_sends` collection
+  - Handle multiple opens (update `last_opened_at` and `open_count`)
+  - Return PNG with proper headers (Content-Type: image/png)
+  - _Requirements: 1.2, 1.3, 1.4, 1.5_
+
+- [x] 2.3 Implement link redirect endpoint
+  - Create Flask route `GET /api/track/click/<tracking_id>`
+  - Look up original article URL from tracking record
+  - Log click event to `link_clicks` collection
+  - Redirect to original URL with 302 status
+  - Handle invalid tracking_id (redirect to homepage)
+  - Ensure redirect completes within 200ms
+  - _Requirements: 2.2, 2.3, 2.4, 2.5_
+
+- [x] 2.4 Write unit tests for tracking endpoints
+  - Test pixel endpoint returns PNG for valid tracking_id
+  - Test pixel endpoint returns PNG for invalid tracking_id (fail silently)
+  - Test link redirect works correctly
+  - Test link redirect handles invalid tracking_id
+  - Test database logging for opens and clicks
+  - _Requirements: 1.2, 1.4, 2.2, 2.3_
+
+## Phase 2: Newsletter Integration
+
+- [x] 3. Integrate tracking into sender service
+- [x] 3.1 Create tracking integration module
+  - Write `inject_tracking_pixel()` function to add pixel to HTML
+  - Write `replace_article_links()` function to replace links with tracking URLs
+  - Write `generate_tracking_urls()` function to create tracking records for all links
+  - Add tracking configuration to sender service
+  - _Requirements: 1.1, 2.1_
+
+- [x] 3.2 Modify newsletter sending flow
+  - Update `send_newsletter()` to generate tracking IDs for each subscriber
+  - Create tracking records in database before sending
+  - Inject tracking pixel into newsletter HTML
+  - Replace article links with tracking URLs
+  - Store newsletter_id and tracking metadata
+  - _Requirements: 1.1, 1.3, 2.1, 2.4_
+
+- [x] 3.3 Update newsletter template
+  - Ensure template supports dynamic tracking pixel injection
+  - Ensure article links are properly structured for replacement
+  - Add privacy notice to footer about tracking
+  - _Requirements: 4.1_
+
+- [x] 3.4 Test newsletter with tracking
+  - Send test newsletter with tracking enabled
+  - Verify tracking pixel is embedded correctly
+  - Verify article links are replaced with tracking URLs
+  - Test email open tracking works
+  - Test link click tracking works
+  - _Requirements: 1.1, 1.2, 2.1, 2.2_
+
+## Phase 3: Analytics and Reporting
+
+- [x] 4. Implement analytics service
+- [x] 4.1 Create analytics calculation functions
+  - Write `get_open_rate(newsletter_id)` function
+  - Write `get_click_rate(article_url)` function
+  - Write `get_newsletter_metrics(newsletter_id)` function for overall stats
+  - Write `get_article_performance(article_url)` function
+  - _Requirements: 3.1, 3.2, 3.3, 3.4_
+
+- [x] 4.2 Implement subscriber activity classification
+  - Write `get_subscriber_activity_status(email)` function
+  - Classify as 'active' (opened in last 30 days)
+  - Classify as 'inactive' (no opens in 30-60 days)
+  - Classify as 'dormant' (no opens in 60+ days)
+  - Write `update_subscriber_activity_statuses()` batch function
+  - _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5_
+
+- [x] 4.3 Create analytics API endpoints
+  - Create `GET /api/analytics/newsletter/<newsletter_id>` endpoint
+  - Create `GET /api/analytics/article/<article_id>` endpoint
+  - Create `GET /api/analytics/subscriber/<email>` endpoint
+  - Create `POST /api/analytics/update-activity` endpoint
+  - Return JSON with engagement metrics
+  - _Requirements: 3.1, 3.2, 3.4, 5.4_
+
+- [x] 4.4 Write unit tests for analytics
+  - Test open rate calculation
+  - Test click rate calculation
+  - Test activity status classification
+  - Test edge cases (no opens, no clicks)
+  - _Requirements: 3.3, 3.4, 5.1, 5.2, 5.3_
+
+## Phase 4: Privacy and Compliance
+
+- [x] 5. Implement privacy features
+- [x] 5.1 Create data anonymization function
+  - Write function to anonymize tracking data older than 90 days
+  - Remove email addresses from old records
+  - Keep aggregated metrics
+  - Create scheduled task to run daily
+  - _Requirements: 4.2_
+
+- [x] 5.2 Implement user data deletion
+  - Create `DELETE /api/tracking/subscriber/<email>` endpoint
+  - Delete all tracking records for subscriber
+  - Delete from `newsletter_sends`, `link_clicks`, `subscriber_activity`
+  - Return confirmation response
+  - _Requirements: 4.3, 4.5_
+
+- [x] 5.3 Add tracking opt-out support
+  - Add `tracking_enabled` field to subscribers collection
+  - Check opt-out status before creating tracking records
+  - Skip tracking for opted-out subscribers
+  - Update newsletter sending to respect opt-out
+  - _Requirements: 4.4_
+
+- [x] 5.4 Create anonymization endpoint
+  - Create `POST /api/tracking/anonymize` endpoint
+  - Trigger anonymization of old data
+  - Return count of anonymized records
+  - Add authentication/authorization
+  - _Requirements: 4.2_
+
+- [x] 5.5 Write privacy compliance tests
+  - Test data anonymization works correctly
+  - Test user data deletion removes all records
+  - Test opt-out prevents tracking
+  - Test anonymization preserves aggregated metrics
+  - _Requirements: 4.2, 4.3, 4.4, 4.5_
+
+## Phase 5: Configuration and Documentation
+
+- [x] 6. Add configuration and environment setup
+  - Add `TRACKING_ENABLED` to environment variables
+  - Add `TRACKING_API_URL` configuration
+  - Add `TRACKING_DATA_RETENTION_DAYS` configuration
+  - Update `.env.template` with tracking variables
+  - Update configuration documentation
+  - _Requirements: All_
+
+- [x] 7. Update database schema documentation
+  - Document `newsletter_sends` collection schema
+  - Document `link_clicks` collection schema
+  - Document `subscriber_activity` collection schema
+  - Add indexes documentation
+  - Update DATABASE_SCHEMA.md
+  - _Requirements: 1.3, 2.4, 5.4_
+
+- [x] 8. Create tracking usage documentation
+  - Document how to enable/disable tracking
+  - Document analytics API endpoints
+  - Document privacy features
+  - Add examples of querying tracking data
+  - Create README for tracking system
+  - _Requirements: All_