Munich-news/news_sender/README.md

News Sender Microservice

Standalone service for sending Munich News Daily newsletters to subscribers.

Features

• 📧 Sends beautiful HTML newsletters
• 🤖 Uses AI-generated article summaries
• 📊 Tracks sending statistics
• 🧪 Test mode for development
• 📝 Preview generation
• 🔄 Fetches data from shared MongoDB

Installation

cd news_sender
pip install -r requirements.txt

Configuration

The service uses the same .env file as the backend (../backend/.env):

# MongoDB
MONGODB_URI=mongodb://localhost:27017/

# Email (Gmail example)
SMTP_SERVER=smtp.gmail.com
SMTP_PORT=587
EMAIL_USER=your-email@gmail.com
EMAIL_PASSWORD=your-app-password

# Newsletter Settings (optional)
NEWSLETTER_MAX_ARTICLES=10
WEBSITE_URL=http://localhost:3000

Gmail Setup:

1. Enable 2-factor authentication
2. Generate an App Password: https://support.google.com/accounts/answer/185833
3. Use the App Password (not your regular password)

Usage

1. Preview Newsletter

Generate HTML preview without sending:

python sender_service.py preview

This creates newsletter_preview.html - open it in your browser to see how the newsletter looks.

2. Send Test Email

Send to a single email address for testing:

python sender_service.py test your-email@example.com

3. Send to All Subscribers

Send newsletter to all active subscribers:

# Send with default article count (10)
python sender_service.py send

# Send with custom article count
python sender_service.py send 15

4. Use as Python Module

from sender_service import send_newsletter, preview_newsletter

# Send newsletter
result = send_newsletter(max_articles=10)
print(f"Sent to {result['sent_count']} subscribers")

# Generate preview
html = preview_newsletter(max_articles=5)

How It Works

┌─────────────────────────────────────────────────────────┐
│  1. Fetch Articles from MongoDB                         │
│     - Get latest articles with AI summaries             │
│     - Sort by creation date (newest first)              │
└─────────────────────────────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────┐
│  2. Fetch Active Subscribers                            │
│     - Get all subscribers with status='active'          │
└─────────────────────────────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────┐
│  3. Render Newsletter HTML                              │
│     - Load newsletter_template.html                     │
│     - Populate with articles and metadata               │
│     - Generate beautiful HTML email                     │
└─────────────────────────────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────┐
│  4. Send Emails                                         │
│     - Connect to SMTP server                            │
│     - Send to each subscriber                           │
│     - Track success/failure                             │
└─────────────────────────────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────┐
│  5. Report Statistics                                   │
│     - Total sent                                        │
│     - Failed sends                                      │
│     - Error details                                     │
└─────────────────────────────────────────────────────────┘

Output Example

======================================================================
📧 Munich News Daily - Newsletter Sender
======================================================================

Fetching latest 10 articles with AI summaries...
✓ Found 10 articles

Fetching active subscribers...
✓ Found 150 active subscriber(s)

Rendering newsletter HTML...
✓ Newsletter rendered

Sending newsletter: 'Munich News Daily - November 10, 2024'
----------------------------------------------------------------------
[1/150] Sending to user1@example.com... ✓
[2/150] Sending to user2@example.com... ✓
[3/150] Sending to user3@example.com... ✓
...

======================================================================
📊 Sending Complete
======================================================================
✓ Successfully sent: 148
✗ Failed: 2
📰 Articles included: 10
======================================================================

Scheduling

Using Cron (Linux/Mac)

Send newsletter daily at 8 AM:

# Edit crontab
crontab -e

# Add this line
0 8 * * * cd /path/to/news_sender && /path/to/venv/bin/python sender_service.py send

Using systemd Timer (Linux)

Create /etc/systemd/system/news-sender.service:

[Unit]
Description=Munich News Sender

[Service]
Type=oneshot
WorkingDirectory=/path/to/news_sender
ExecStart=/path/to/venv/bin/python sender_service.py send
User=your-user

Create /etc/systemd/system/news-sender.timer:

[Unit]
Description=Send Munich News Daily at 8 AM

[Timer]
OnCalendar=daily
OnCalendar=*-*-* 08:00:00

[Install]
WantedBy=timers.target

Enable and start:

sudo systemctl enable news-sender.timer
sudo systemctl start news-sender.timer

Using Docker

Create Dockerfile:

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY sender_service.py newsletter_template.html ./

CMD ["python", "sender_service.py", "send"]

Build and run:

docker build -t news-sender .
docker run --env-file ../backend/.env news-sender

Troubleshooting

"Email credentials not configured"

• Check that EMAIL_USER and EMAIL_PASSWORD are set in .env
• For Gmail, use an App Password, not your regular password

"No articles with summaries found"

• Run the crawler first: cd ../news_crawler && python crawler_service.py 10
• Make sure Ollama is enabled and working
• Check MongoDB has articles with summary field

"No active subscribers found"

• Add subscribers via the backend API
• Check subscriber status is 'active' in MongoDB

SMTP Connection Errors

• Verify SMTP server and port are correct
• Check firewall isn't blocking SMTP port
• For Gmail, ensure "Less secure app access" is enabled or use App Password

Emails Going to Spam

• Set up SPF, DKIM, and DMARC records for your domain
• Use a verified email address
• Avoid spam trigger words in subject/content
• Include unsubscribe link (already included in template)

Architecture

This is a standalone microservice that:

• Runs independently of the backend
• Shares the same MongoDB database
• Can be deployed separately
• Can be scheduled independently
• Has no dependencies on backend code

Integration with Other Services

┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│   Backend    │     │   Crawler    │     │    Sender    │
│   (Flask)    │     │  (Scraper)   │     │   (Email)    │
└──────┬───────┘     └──────┬───────┘     └──────┬───────┘
       │                    │                     │
       │                    │                     │
       └────────────────────┴─────────────────────┘
                            │
                    ┌───────▼────────┐
                    │    MongoDB     │
                    │  (Shared DB)   │
                    └────────────────┘

Next Steps

1. Test the newsletter:

   python sender_service.py test your-email@example.com
   

2. Schedule daily sending:

   • Set up cron job or systemd timer
   • Choose appropriate time (e.g., 8 AM)

3. Monitor sending:

   • Check logs for errors
   • Track open rates (requires email tracking service)
   • Monitor spam complaints

4. Optimize:

   • Add email tracking pixels
   • A/B test subject lines
   • Personalize content per subscriber