304 lines
9.4 KiB
Markdown
304 lines
9.4 KiB
Markdown
# News Sender Microservice
|
|
|
|
Standalone service for sending Munich News Daily newsletters to subscribers.
|
|
|
|
## Features
|
|
|
|
- 📧 Sends beautiful HTML newsletters
|
|
- 🤖 Uses AI-generated article summaries
|
|
- 📊 Tracks sending statistics
|
|
- 🧪 Test mode for development
|
|
- 📝 Preview generation
|
|
- 🔄 Fetches data from shared MongoDB
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
cd news_sender
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## Configuration
|
|
|
|
The service uses the same `.env` file as the backend (`../backend/.env`):
|
|
|
|
```env
|
|
# MongoDB
|
|
MONGODB_URI=mongodb://localhost:27017/
|
|
|
|
# Email (Gmail example)
|
|
SMTP_SERVER=smtp.gmail.com
|
|
SMTP_PORT=587
|
|
EMAIL_USER=your-email@gmail.com
|
|
EMAIL_PASSWORD=your-app-password
|
|
|
|
# Newsletter Settings (optional)
|
|
NEWSLETTER_MAX_ARTICLES=10
|
|
WEBSITE_URL=http://localhost:3000
|
|
```
|
|
|
|
**Gmail Setup:**
|
|
1. Enable 2-factor authentication
|
|
2. Generate an App Password: https://support.google.com/accounts/answer/185833
|
|
3. Use the App Password (not your regular password)
|
|
|
|
## Usage
|
|
|
|
### 1. Preview Newsletter
|
|
|
|
Generate HTML preview without sending:
|
|
|
|
```bash
|
|
python sender_service.py preview
|
|
```
|
|
|
|
This creates `newsletter_preview.html` - open it in your browser to see how the newsletter looks.
|
|
|
|
### 2. Send Test Email
|
|
|
|
Send to a single email address for testing:
|
|
|
|
```bash
|
|
python sender_service.py test your-email@example.com
|
|
```
|
|
|
|
### 3. Send to All Subscribers
|
|
|
|
Send newsletter to all active subscribers:
|
|
|
|
```bash
|
|
# Send with default article count (10)
|
|
python sender_service.py send
|
|
|
|
# Send with custom article count
|
|
python sender_service.py send 15
|
|
```
|
|
|
|
### 4. Use as Python Module
|
|
|
|
```python
|
|
from sender_service import send_newsletter, preview_newsletter
|
|
|
|
# Send newsletter
|
|
result = send_newsletter(max_articles=10)
|
|
print(f"Sent to {result['sent_count']} subscribers")
|
|
|
|
# Generate preview
|
|
html = preview_newsletter(max_articles=5)
|
|
```
|
|
|
|
## How It Works
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ 1. Fetch Articles from MongoDB │
|
|
│ - Get latest articles with AI summaries │
|
|
│ - Sort by creation date (newest first) │
|
|
└─────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ 2. Fetch Active Subscribers │
|
|
│ - Get all subscribers with status='active' │
|
|
└─────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ 3. Render Newsletter HTML │
|
|
│ - Load newsletter_template.html │
|
|
│ - Populate with articles and metadata │
|
|
│ - Generate beautiful HTML email │
|
|
└─────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ 4. Send Emails │
|
|
│ - Connect to SMTP server │
|
|
│ - Send to each subscriber │
|
|
│ - Track success/failure │
|
|
└─────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ 5. Report Statistics │
|
|
│ - Total sent │
|
|
│ - Failed sends │
|
|
│ - Error details │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Output Example
|
|
|
|
```
|
|
======================================================================
|
|
📧 Munich News Daily - Newsletter Sender
|
|
======================================================================
|
|
|
|
Fetching latest 10 articles with AI summaries...
|
|
✓ Found 10 articles
|
|
|
|
Fetching active subscribers...
|
|
✓ Found 150 active subscriber(s)
|
|
|
|
Rendering newsletter HTML...
|
|
✓ Newsletter rendered
|
|
|
|
Sending newsletter: 'Munich News Daily - November 10, 2024'
|
|
----------------------------------------------------------------------
|
|
[1/150] Sending to user1@example.com... ✓
|
|
[2/150] Sending to user2@example.com... ✓
|
|
[3/150] Sending to user3@example.com... ✓
|
|
...
|
|
|
|
======================================================================
|
|
📊 Sending Complete
|
|
======================================================================
|
|
✓ Successfully sent: 148
|
|
✗ Failed: 2
|
|
📰 Articles included: 10
|
|
======================================================================
|
|
```
|
|
|
|
## Scheduling
|
|
|
|
### Using Cron (Linux/Mac)
|
|
|
|
Send newsletter daily at 8 AM:
|
|
|
|
```bash
|
|
# Edit crontab
|
|
crontab -e
|
|
|
|
# Add this line
|
|
0 8 * * * cd /path/to/news_sender && /path/to/venv/bin/python sender_service.py send
|
|
```
|
|
|
|
### Using systemd Timer (Linux)
|
|
|
|
Create `/etc/systemd/system/news-sender.service`:
|
|
|
|
```ini
|
|
[Unit]
|
|
Description=Munich News Sender
|
|
|
|
[Service]
|
|
Type=oneshot
|
|
WorkingDirectory=/path/to/news_sender
|
|
ExecStart=/path/to/venv/bin/python sender_service.py send
|
|
User=your-user
|
|
```
|
|
|
|
Create `/etc/systemd/system/news-sender.timer`:
|
|
|
|
```ini
|
|
[Unit]
|
|
Description=Send Munich News Daily at 8 AM
|
|
|
|
[Timer]
|
|
OnCalendar=daily
|
|
OnCalendar=*-*-* 08:00:00
|
|
|
|
[Install]
|
|
WantedBy=timers.target
|
|
```
|
|
|
|
Enable and start:
|
|
|
|
```bash
|
|
sudo systemctl enable news-sender.timer
|
|
sudo systemctl start news-sender.timer
|
|
```
|
|
|
|
### Using Docker
|
|
|
|
Create `Dockerfile`:
|
|
|
|
```dockerfile
|
|
FROM python:3.11-slim
|
|
|
|
WORKDIR /app
|
|
|
|
COPY requirements.txt .
|
|
RUN pip install --no-cache-dir -r requirements.txt
|
|
|
|
COPY sender_service.py newsletter_template.html ./
|
|
|
|
CMD ["python", "sender_service.py", "send"]
|
|
```
|
|
|
|
Build and run:
|
|
|
|
```bash
|
|
docker build -t news-sender .
|
|
docker run --env-file ../backend/.env news-sender
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### "Email credentials not configured"
|
|
- Check that `EMAIL_USER` and `EMAIL_PASSWORD` are set in `.env`
|
|
- For Gmail, use an App Password, not your regular password
|
|
|
|
### "No articles with summaries found"
|
|
- Run the crawler first: `cd ../news_crawler && python crawler_service.py 10`
|
|
- Make sure Ollama is enabled and working
|
|
- Check MongoDB has articles with `summary` field
|
|
|
|
### "No active subscribers found"
|
|
- Add subscribers via the backend API
|
|
- Check subscriber status is 'active' in MongoDB
|
|
|
|
### SMTP Connection Errors
|
|
- Verify SMTP server and port are correct
|
|
- Check firewall isn't blocking SMTP port
|
|
- For Gmail, ensure "Less secure app access" is enabled or use App Password
|
|
|
|
### Emails Going to Spam
|
|
- Set up SPF, DKIM, and DMARC records for your domain
|
|
- Use a verified email address
|
|
- Avoid spam trigger words in subject/content
|
|
- Include unsubscribe link (already included in template)
|
|
|
|
## Architecture
|
|
|
|
This is a standalone microservice that:
|
|
- Runs independently of the backend
|
|
- Shares the same MongoDB database
|
|
- Can be deployed separately
|
|
- Can be scheduled independently
|
|
- Has no dependencies on backend code
|
|
|
|
## Integration with Other Services
|
|
|
|
```
|
|
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
|
│ Backend │ │ Crawler │ │ Sender │
|
|
│ (Flask) │ │ (Scraper) │ │ (Email) │
|
|
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
|
|
│ │ │
|
|
│ │ │
|
|
└────────────────────┴─────────────────────┘
|
|
│
|
|
┌───────▼────────┐
|
|
│ MongoDB │
|
|
│ (Shared DB) │
|
|
└────────────────┘
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
1. **Test the newsletter:**
|
|
```bash
|
|
python sender_service.py test your-email@example.com
|
|
```
|
|
|
|
2. **Schedule daily sending:**
|
|
- Set up cron job or systemd timer
|
|
- Choose appropriate time (e.g., 8 AM)
|
|
|
|
3. **Monitor sending:**
|
|
- Check logs for errors
|
|
- Track open rates (requires email tracking service)
|
|
- Monitor spam complaints
|
|
|
|
4. **Optimize:**
|
|
- Add email tracking pixels
|
|
- A/B test subject lines
|
|
- Personalize content per subscriber
|