update
This commit is contained in:
303
news_sender/README.md
Normal file
303
news_sender/README.md
Normal file
@@ -0,0 +1,303 @@
|
||||
# News Sender Microservice
|
||||
|
||||
Standalone service for sending Munich News Daily newsletters to subscribers.
|
||||
|
||||
## Features
|
||||
|
||||
- 📧 Sends beautiful HTML newsletters
|
||||
- 🤖 Uses AI-generated article summaries
|
||||
- 📊 Tracks sending statistics
|
||||
- 🧪 Test mode for development
|
||||
- 📝 Preview generation
|
||||
- 🔄 Fetches data from shared MongoDB
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
cd news_sender
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The service uses the same `.env` file as the backend (`../backend/.env`):
|
||||
|
||||
```env
|
||||
# MongoDB
|
||||
MONGODB_URI=mongodb://localhost:27017/
|
||||
|
||||
# Email (Gmail example)
|
||||
SMTP_SERVER=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
EMAIL_USER=your-email@gmail.com
|
||||
EMAIL_PASSWORD=your-app-password
|
||||
|
||||
# Newsletter Settings (optional)
|
||||
NEWSLETTER_MAX_ARTICLES=10
|
||||
WEBSITE_URL=http://localhost:3000
|
||||
```
|
||||
|
||||
**Gmail Setup:**
|
||||
1. Enable 2-factor authentication
|
||||
2. Generate an App Password: https://support.google.com/accounts/answer/185833
|
||||
3. Use the App Password (not your regular password)
|
||||
|
||||
## Usage
|
||||
|
||||
### 1. Preview Newsletter
|
||||
|
||||
Generate HTML preview without sending:
|
||||
|
||||
```bash
|
||||
python sender_service.py preview
|
||||
```
|
||||
|
||||
This creates `newsletter_preview.html` - open it in your browser to see how the newsletter looks.
|
||||
|
||||
### 2. Send Test Email
|
||||
|
||||
Send to a single email address for testing:
|
||||
|
||||
```bash
|
||||
python sender_service.py test your-email@example.com
|
||||
```
|
||||
|
||||
### 3. Send to All Subscribers
|
||||
|
||||
Send newsletter to all active subscribers:
|
||||
|
||||
```bash
|
||||
# Send with default article count (10)
|
||||
python sender_service.py send
|
||||
|
||||
# Send with custom article count
|
||||
python sender_service.py send 15
|
||||
```
|
||||
|
||||
### 4. Use as Python Module
|
||||
|
||||
```python
|
||||
from sender_service import send_newsletter, preview_newsletter
|
||||
|
||||
# Send newsletter
|
||||
result = send_newsletter(max_articles=10)
|
||||
print(f"Sent to {result['sent_count']} subscribers")
|
||||
|
||||
# Generate preview
|
||||
html = preview_newsletter(max_articles=5)
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 1. Fetch Articles from MongoDB │
|
||||
│ - Get latest articles with AI summaries │
|
||||
│ - Sort by creation date (newest first) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 2. Fetch Active Subscribers │
|
||||
│ - Get all subscribers with status='active' │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 3. Render Newsletter HTML │
|
||||
│ - Load newsletter_template.html │
|
||||
│ - Populate with articles and metadata │
|
||||
│ - Generate beautiful HTML email │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 4. Send Emails │
|
||||
│ - Connect to SMTP server │
|
||||
│ - Send to each subscriber │
|
||||
│ - Track success/failure │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 5. Report Statistics │
|
||||
│ - Total sent │
|
||||
│ - Failed sends │
|
||||
│ - Error details │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Output Example
|
||||
|
||||
```
|
||||
======================================================================
|
||||
📧 Munich News Daily - Newsletter Sender
|
||||
======================================================================
|
||||
|
||||
Fetching latest 10 articles with AI summaries...
|
||||
✓ Found 10 articles
|
||||
|
||||
Fetching active subscribers...
|
||||
✓ Found 150 active subscriber(s)
|
||||
|
||||
Rendering newsletter HTML...
|
||||
✓ Newsletter rendered
|
||||
|
||||
Sending newsletter: 'Munich News Daily - November 10, 2024'
|
||||
----------------------------------------------------------------------
|
||||
[1/150] Sending to user1@example.com... ✓
|
||||
[2/150] Sending to user2@example.com... ✓
|
||||
[3/150] Sending to user3@example.com... ✓
|
||||
...
|
||||
|
||||
======================================================================
|
||||
📊 Sending Complete
|
||||
======================================================================
|
||||
✓ Successfully sent: 148
|
||||
✗ Failed: 2
|
||||
📰 Articles included: 10
|
||||
======================================================================
|
||||
```
|
||||
|
||||
## Scheduling
|
||||
|
||||
### Using Cron (Linux/Mac)
|
||||
|
||||
Send newsletter daily at 8 AM:
|
||||
|
||||
```bash
|
||||
# Edit crontab
|
||||
crontab -e
|
||||
|
||||
# Add this line
|
||||
0 8 * * * cd /path/to/news_sender && /path/to/venv/bin/python sender_service.py send
|
||||
```
|
||||
|
||||
### Using systemd Timer (Linux)
|
||||
|
||||
Create `/etc/systemd/system/news-sender.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Munich News Sender
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
WorkingDirectory=/path/to/news_sender
|
||||
ExecStart=/path/to/venv/bin/python sender_service.py send
|
||||
User=your-user
|
||||
```
|
||||
|
||||
Create `/etc/systemd/system/news-sender.timer`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Send Munich News Daily at 8 AM
|
||||
|
||||
[Timer]
|
||||
OnCalendar=daily
|
||||
OnCalendar=*-*-* 08:00:00
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
```
|
||||
|
||||
Enable and start:
|
||||
|
||||
```bash
|
||||
sudo systemctl enable news-sender.timer
|
||||
sudo systemctl start news-sender.timer
|
||||
```
|
||||
|
||||
### Using Docker
|
||||
|
||||
Create `Dockerfile`:
|
||||
|
||||
```dockerfile
|
||||
FROM python:3.11-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY sender_service.py newsletter_template.html ./
|
||||
|
||||
CMD ["python", "sender_service.py", "send"]
|
||||
```
|
||||
|
||||
Build and run:
|
||||
|
||||
```bash
|
||||
docker build -t news-sender .
|
||||
docker run --env-file ../backend/.env news-sender
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Email credentials not configured"
|
||||
- Check that `EMAIL_USER` and `EMAIL_PASSWORD` are set in `.env`
|
||||
- For Gmail, use an App Password, not your regular password
|
||||
|
||||
### "No articles with summaries found"
|
||||
- Run the crawler first: `cd ../news_crawler && python crawler_service.py 10`
|
||||
- Make sure Ollama is enabled and working
|
||||
- Check MongoDB has articles with `summary` field
|
||||
|
||||
### "No active subscribers found"
|
||||
- Add subscribers via the backend API
|
||||
- Check subscriber status is 'active' in MongoDB
|
||||
|
||||
### SMTP Connection Errors
|
||||
- Verify SMTP server and port are correct
|
||||
- Check firewall isn't blocking SMTP port
|
||||
- For Gmail, ensure "Less secure app access" is enabled or use App Password
|
||||
|
||||
### Emails Going to Spam
|
||||
- Set up SPF, DKIM, and DMARC records for your domain
|
||||
- Use a verified email address
|
||||
- Avoid spam trigger words in subject/content
|
||||
- Include unsubscribe link (already included in template)
|
||||
|
||||
## Architecture
|
||||
|
||||
This is a standalone microservice that:
|
||||
- Runs independently of the backend
|
||||
- Shares the same MongoDB database
|
||||
- Can be deployed separately
|
||||
- Can be scheduled independently
|
||||
- Has no dependencies on backend code
|
||||
|
||||
## Integration with Other Services
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ Backend │ │ Crawler │ │ Sender │
|
||||
│ (Flask) │ │ (Scraper) │ │ (Email) │
|
||||
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
|
||||
│ │ │
|
||||
│ │ │
|
||||
└────────────────────┴─────────────────────┘
|
||||
│
|
||||
┌───────▼────────┐
|
||||
│ MongoDB │
|
||||
│ (Shared DB) │
|
||||
└────────────────┘
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Test the newsletter:**
|
||||
```bash
|
||||
python sender_service.py test your-email@example.com
|
||||
```
|
||||
|
||||
2. **Schedule daily sending:**
|
||||
- Set up cron job or systemd timer
|
||||
- Choose appropriate time (e.g., 8 AM)
|
||||
|
||||
3. **Monitor sending:**
|
||||
- Check logs for errors
|
||||
- Track open rates (requires email tracking service)
|
||||
- Monitor spam complaints
|
||||
|
||||
4. **Optimize:**
|
||||
- Add email tracking pixels
|
||||
- A/B test subject lines
|
||||
- Personalize content per subscriber
|
||||
Reference in New Issue
Block a user